|
--- |
|
language: |
|
- en |
|
license: apache-2.0 |
|
library_name: sentence-transformers |
|
tags: |
|
- sentence-transformers |
|
- sentence-similarity |
|
- feature-extraction |
|
- generated_from_trainer |
|
- dataset_size:6300 |
|
- loss:MatryoshkaLoss |
|
- loss:MultipleNegativesRankingLoss |
|
base_model: BAAI/bge-base-en |
|
datasets: [] |
|
metrics: |
|
- cosine_accuracy@1 |
|
- cosine_accuracy@3 |
|
- cosine_accuracy@5 |
|
- cosine_accuracy@10 |
|
- cosine_precision@1 |
|
- cosine_precision@3 |
|
- cosine_precision@5 |
|
- cosine_precision@10 |
|
- cosine_recall@1 |
|
- cosine_recall@3 |
|
- cosine_recall@5 |
|
- cosine_recall@10 |
|
- cosine_ndcg@10 |
|
- cosine_mrr@10 |
|
- cosine_map@100 |
|
widget: |
|
- source_sentence: As of January 31, 2023, the Company's net operating loss and capital |
|
loss carryforwards totaled approximately $32.3 billion. |
|
sentences: |
|
- What was the percentage change in general and administrative expenses in 2023 |
|
compared to 2022? |
|
- What was the amount of the company's net operating loss and capital loss carryforwards |
|
as of January 31, 2023? |
|
- What are common challenges in pharmaceutical research and development? |
|
- source_sentence: A 0.50% increase in completion factors, which consider aspects |
|
like claim levels and processing cycles, raises medical costs payable by $585 |
|
million as of December 31, 2023. |
|
sentences: |
|
- What were the total assets of Hasbro, Inc. as of December 31, 2023? |
|
- How does a 0.50% increase in completion factors impact medical costs payable as |
|
of December 31, 2023? |
|
- By what percentage did Gaming revenue change in fiscal year 2023 compared to fiscal |
|
year 2022? |
|
- source_sentence: Alex G. Balazs was appointed as the Executive Vice President and |
|
Chief Technology Officer effective September 5, 2023. |
|
sentences: |
|
- When was Alex G. Balazs appointed as the Executive Vice President and Chief Technology |
|
Officer? |
|
- What was AMC's minimum liquidity requirement under the Credit Agreement? |
|
- What was the nature of the legal action initiated by Aqua-Chem against the company |
|
in Wisconsin on the same day the company filed its lawsuit? |
|
- source_sentence: Item 8. Financial Statements and Supplementary Data |
|
sentences: |
|
- How did the carrying amount of goodwill change from March 31, 2022 to March 31, |
|
2023? |
|
- What types of revenue does the payments company generate from its various products |
|
and services? |
|
- What is the content of Item 8 in a financial document? |
|
- source_sentence: The company offers Medicare eligible persons under HMO, PPO, Private |
|
Fee-For-Service, or PFFS, and Special Needs Plans, including Dual Eligible Special |
|
Needs, or D-SNP, plans in exchange for contractual payments received from CMS. |
|
With each of these products, the beneficiary receives benefits in excess of Medicare |
|
FFS, typically including reduced cost sharing, enhanced prescription drug benefits, |
|
care coordination, data analysis techniques to help identify member needs, complex |
|
case management, tools to guide members in their health care decisions, care management |
|
programs, wellness and prevention programs and, in some instances, a reduced monthly |
|
Part B premium. Most Medicare Advantage plans offer the prescription drug benefit |
|
under Part D as part of the basic plan, subject to cost sharing and other limitations. |
|
sentences: |
|
- What types of Medicare plans does the company offer and what are the key benefits |
|
provided? |
|
- What were the total cash discounts provided by AbbVie in 2023, 2022, and 2021? |
|
- How does a company account for potential liabilities from legal proceedings in |
|
its financial statements? |
|
pipeline_tag: sentence-similarity |
|
model-index: |
|
- name: BGE base Financial Matryoshka |
|
results: |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 768 |
|
type: dim_768 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.7028571428571428 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.8371428571428572 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.87 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9114285714285715 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.7028571428571428 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.27904761904761904 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.174 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09114285714285714 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.7028571428571428 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.8371428571428572 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.87 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.9114285714285715 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.8100174465587288 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.7773446712018138 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.7807079942767247 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 512 |
|
type: dim_512 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.6942857142857143 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.83 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.87 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9128571428571428 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.6942857142857143 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.27666666666666667 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.174 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09128571428571428 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.6942857142857143 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.83 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.87 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.9128571428571428 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.8078520466243649 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.7740147392290249 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.7772770435826438 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 256 |
|
type: dim_256 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.6914285714285714 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.8271428571428572 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.8685714285714285 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9114285714285715 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.6914285714285714 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.2757142857142857 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.1737142857142857 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09114285714285712 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.6914285714285714 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.8271428571428572 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8685714285714285 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.9114285714285715 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.8048419939996826 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.7705011337868479 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.7738179161222841 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 128 |
|
type: dim_128 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.6814285714285714 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.82 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.8628571428571429 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.91 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.6814285714285714 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.2733333333333333 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.17257142857142854 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09099999999999998 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.6814285714285714 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.82 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8628571428571429 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.91 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.7983213130859076 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.7624348072562357 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.7654098753888775 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 64 |
|
type: dim_64 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.6628571428571428 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.7985714285714286 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.8414285714285714 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.8971428571428571 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.6628571428571428 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.26619047619047614 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.16828571428571426 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.0897142857142857 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.6628571428571428 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.7985714285714286 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8414285714285714 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.8971428571428571 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.7801763622372425 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.7428265306122449 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.7467214067895231 |
|
name: Cosine Map@100 |
|
--- |
|
|
|
# BGE base Financial Matryoshka |
|
|
|
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en](https://huggingface.co./BAAI/bge-base-en). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
- **Model Type:** Sentence Transformer |
|
- **Base model:** [BAAI/bge-base-en](https://huggingface.co./BAAI/bge-base-en) <!-- at revision b737bf5dcc6ee8bdc530531266b4804a5d77b5d8 --> |
|
- **Maximum Sequence Length:** 512 tokens |
|
- **Output Dimensionality:** 768 tokens |
|
- **Similarity Function:** Cosine Similarity |
|
<!-- - **Training Dataset:** Unknown --> |
|
- **Language:** en |
|
- **License:** apache-2.0 |
|
|
|
### Model Sources |
|
|
|
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net) |
|
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) |
|
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers) |
|
|
|
### Full Model Architecture |
|
|
|
``` |
|
SentenceTransformer( |
|
(0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel |
|
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) |
|
(2): Normalize() |
|
) |
|
``` |
|
|
|
## Usage |
|
|
|
### Direct Usage (Sentence Transformers) |
|
|
|
First install the Sentence Transformers library: |
|
|
|
```bash |
|
pip install -U sentence-transformers |
|
``` |
|
|
|
Then you can load this model and run inference. |
|
```python |
|
from sentence_transformers import SentenceTransformer |
|
|
|
# Download from the 🤗 Hub |
|
model = SentenceTransformer("riphunter7001x/bge-base-financial") |
|
# Run inference |
|
sentences = [ |
|
'The company offers Medicare eligible persons under HMO, PPO, Private Fee-For-Service, or PFFS, and Special Needs Plans, including Dual Eligible Special Needs, or D-SNP, plans in exchange for contractual payments received from CMS. With each of these products, the beneficiary receives benefits in excess of Medicare FFS, typically including reduced cost sharing, enhanced prescription drug benefits, care coordination, data analysis techniques to help identify member needs, complex case management, tools to guide members in their health care decisions, care management programs, wellness and prevention programs and, in some instances, a reduced monthly Part B premium. Most Medicare Advantage plans offer the prescription drug benefit under Part D as part of the basic plan, subject to cost sharing and other limitations.', |
|
'What types of Medicare plans does the company offer and what are the key benefits provided?', |
|
'What were the total cash discounts provided by AbbVie in 2023, 2022, and 2021?', |
|
] |
|
embeddings = model.encode(sentences) |
|
print(embeddings.shape) |
|
# [3, 768] |
|
|
|
# Get the similarity scores for the embeddings |
|
similarities = model.similarity(embeddings, embeddings) |
|
print(similarities.shape) |
|
# [3, 3] |
|
``` |
|
|
|
<!-- |
|
### Direct Usage (Transformers) |
|
|
|
<details><summary>Click to see the direct usage in Transformers</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Downstream Usage (Sentence Transformers) |
|
|
|
You can finetune this model on your own dataset. |
|
|
|
<details><summary>Click to expand</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Out-of-Scope Use |
|
|
|
*List how the model may foreseeably be misused and address what users ought not to do with the model.* |
|
--> |
|
|
|
## Evaluation |
|
|
|
### Metrics |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_768` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.7029 | |
|
| cosine_accuracy@3 | 0.8371 | |
|
| cosine_accuracy@5 | 0.87 | |
|
| cosine_accuracy@10 | 0.9114 | |
|
| cosine_precision@1 | 0.7029 | |
|
| cosine_precision@3 | 0.279 | |
|
| cosine_precision@5 | 0.174 | |
|
| cosine_precision@10 | 0.0911 | |
|
| cosine_recall@1 | 0.7029 | |
|
| cosine_recall@3 | 0.8371 | |
|
| cosine_recall@5 | 0.87 | |
|
| cosine_recall@10 | 0.9114 | |
|
| cosine_ndcg@10 | 0.81 | |
|
| cosine_mrr@10 | 0.7773 | |
|
| **cosine_map@100** | **0.7807** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_512` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.6943 | |
|
| cosine_accuracy@3 | 0.83 | |
|
| cosine_accuracy@5 | 0.87 | |
|
| cosine_accuracy@10 | 0.9129 | |
|
| cosine_precision@1 | 0.6943 | |
|
| cosine_precision@3 | 0.2767 | |
|
| cosine_precision@5 | 0.174 | |
|
| cosine_precision@10 | 0.0913 | |
|
| cosine_recall@1 | 0.6943 | |
|
| cosine_recall@3 | 0.83 | |
|
| cosine_recall@5 | 0.87 | |
|
| cosine_recall@10 | 0.9129 | |
|
| cosine_ndcg@10 | 0.8079 | |
|
| cosine_mrr@10 | 0.774 | |
|
| **cosine_map@100** | **0.7773** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_256` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.6914 | |
|
| cosine_accuracy@3 | 0.8271 | |
|
| cosine_accuracy@5 | 0.8686 | |
|
| cosine_accuracy@10 | 0.9114 | |
|
| cosine_precision@1 | 0.6914 | |
|
| cosine_precision@3 | 0.2757 | |
|
| cosine_precision@5 | 0.1737 | |
|
| cosine_precision@10 | 0.0911 | |
|
| cosine_recall@1 | 0.6914 | |
|
| cosine_recall@3 | 0.8271 | |
|
| cosine_recall@5 | 0.8686 | |
|
| cosine_recall@10 | 0.9114 | |
|
| cosine_ndcg@10 | 0.8048 | |
|
| cosine_mrr@10 | 0.7705 | |
|
| **cosine_map@100** | **0.7738** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_128` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.6814 | |
|
| cosine_accuracy@3 | 0.82 | |
|
| cosine_accuracy@5 | 0.8629 | |
|
| cosine_accuracy@10 | 0.91 | |
|
| cosine_precision@1 | 0.6814 | |
|
| cosine_precision@3 | 0.2733 | |
|
| cosine_precision@5 | 0.1726 | |
|
| cosine_precision@10 | 0.091 | |
|
| cosine_recall@1 | 0.6814 | |
|
| cosine_recall@3 | 0.82 | |
|
| cosine_recall@5 | 0.8629 | |
|
| cosine_recall@10 | 0.91 | |
|
| cosine_ndcg@10 | 0.7983 | |
|
| cosine_mrr@10 | 0.7624 | |
|
| **cosine_map@100** | **0.7654** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_64` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.6629 | |
|
| cosine_accuracy@3 | 0.7986 | |
|
| cosine_accuracy@5 | 0.8414 | |
|
| cosine_accuracy@10 | 0.8971 | |
|
| cosine_precision@1 | 0.6629 | |
|
| cosine_precision@3 | 0.2662 | |
|
| cosine_precision@5 | 0.1683 | |
|
| cosine_precision@10 | 0.0897 | |
|
| cosine_recall@1 | 0.6629 | |
|
| cosine_recall@3 | 0.7986 | |
|
| cosine_recall@5 | 0.8414 | |
|
| cosine_recall@10 | 0.8971 | |
|
| cosine_ndcg@10 | 0.7802 | |
|
| cosine_mrr@10 | 0.7428 | |
|
| **cosine_map@100** | **0.7467** | |
|
|
|
<!-- |
|
## Bias, Risks and Limitations |
|
|
|
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
|
--> |
|
|
|
<!-- |
|
### Recommendations |
|
|
|
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
|
--> |
|
|
|
## Training Details |
|
|
|
### Training Dataset |
|
|
|
#### Unnamed Dataset |
|
|
|
|
|
* Size: 6,300 training samples |
|
* Columns: <code>positive</code> and <code>anchor</code> |
|
* Approximate statistics based on the first 1000 samples: |
|
| | positive | anchor | |
|
|:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| |
|
| type | string | string | |
|
| details | <ul><li>min: 2 tokens</li><li>mean: 45.98 tokens</li><li>max: 208 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 20.76 tokens</li><li>max: 43 tokens</li></ul> | |
|
* Samples: |
|
| positive | anchor | |
|
|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
|
| <code>Adjusted EBITDA does not reflect costs associated with product recall related matters including adjustments to the return reserves, inventory write-downs, logistics costs associated with Member requests, the cost to move the recalled product for those that elect the option, subscription waiver costs of service, and recall-related hardware development and repair costs.</code> | <code>What specific costs associated with product recalls are excluded from Adjusted EBITDA?</code> | |
|
| <code>The Company sold $17,704 million and $10,709 million of trade accounts receivables under this program during the years ended December 31, 2023 and 2022, respectively.</code> | <code>How much did the Company sell in trade accounts receivables in the year ended December 31, 2023?</code> | |
|
| <code>Free cash flow less equipment finance leases and principal repayments of all other finance leases and financing obligations was -$12,786 million in 2022 and improved to $35,549 million in 2023.</code> | <code>How did the free cash flow less equipment finance leases and principal repayments of all other finance leases and financing obligations change from 2022 to 2023?</code> | |
|
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: |
|
```json |
|
{ |
|
"loss": "MultipleNegativesRankingLoss", |
|
"matryoshka_dims": [ |
|
768, |
|
512, |
|
256, |
|
128, |
|
64 |
|
], |
|
"matryoshka_weights": [ |
|
1, |
|
1, |
|
1, |
|
1, |
|
1 |
|
], |
|
"n_dims_per_step": -1 |
|
} |
|
``` |
|
|
|
### Training Hyperparameters |
|
#### Non-Default Hyperparameters |
|
|
|
- `eval_strategy`: steps |
|
- `per_device_train_batch_size`: 16 |
|
- `per_device_eval_batch_size`: 16 |
|
- `num_train_epochs`: 10 |
|
- `warmup_ratio`: 0.1 |
|
- `fp16`: True |
|
- `batch_sampler`: no_duplicates |
|
|
|
#### All Hyperparameters |
|
<details><summary>Click to expand</summary> |
|
|
|
- `overwrite_output_dir`: False |
|
- `do_predict`: False |
|
- `eval_strategy`: steps |
|
- `prediction_loss_only`: True |
|
- `per_device_train_batch_size`: 16 |
|
- `per_device_eval_batch_size`: 16 |
|
- `per_gpu_train_batch_size`: None |
|
- `per_gpu_eval_batch_size`: None |
|
- `gradient_accumulation_steps`: 1 |
|
- `eval_accumulation_steps`: None |
|
- `learning_rate`: 5e-05 |
|
- `weight_decay`: 0.0 |
|
- `adam_beta1`: 0.9 |
|
- `adam_beta2`: 0.999 |
|
- `adam_epsilon`: 1e-08 |
|
- `max_grad_norm`: 1.0 |
|
- `num_train_epochs`: 10 |
|
- `max_steps`: -1 |
|
- `lr_scheduler_type`: linear |
|
- `lr_scheduler_kwargs`: {} |
|
- `warmup_ratio`: 0.1 |
|
- `warmup_steps`: 0 |
|
- `log_level`: passive |
|
- `log_level_replica`: warning |
|
- `log_on_each_node`: True |
|
- `logging_nan_inf_filter`: True |
|
- `save_safetensors`: True |
|
- `save_on_each_node`: False |
|
- `save_only_model`: False |
|
- `restore_callback_states_from_checkpoint`: False |
|
- `no_cuda`: False |
|
- `use_cpu`: False |
|
- `use_mps_device`: False |
|
- `seed`: 42 |
|
- `data_seed`: None |
|
- `jit_mode_eval`: False |
|
- `use_ipex`: False |
|
- `bf16`: False |
|
- `fp16`: True |
|
- `fp16_opt_level`: O1 |
|
- `half_precision_backend`: auto |
|
- `bf16_full_eval`: False |
|
- `fp16_full_eval`: False |
|
- `tf32`: None |
|
- `local_rank`: 0 |
|
- `ddp_backend`: None |
|
- `tpu_num_cores`: None |
|
- `tpu_metrics_debug`: False |
|
- `debug`: [] |
|
- `dataloader_drop_last`: False |
|
- `dataloader_num_workers`: 0 |
|
- `dataloader_prefetch_factor`: None |
|
- `past_index`: -1 |
|
- `disable_tqdm`: False |
|
- `remove_unused_columns`: True |
|
- `label_names`: None |
|
- `load_best_model_at_end`: False |
|
- `ignore_data_skip`: False |
|
- `fsdp`: [] |
|
- `fsdp_min_num_params`: 0 |
|
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} |
|
- `fsdp_transformer_layer_cls_to_wrap`: None |
|
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} |
|
- `deepspeed`: None |
|
- `label_smoothing_factor`: 0.0 |
|
- `optim`: adamw_torch |
|
- `optim_args`: None |
|
- `adafactor`: False |
|
- `group_by_length`: False |
|
- `length_column_name`: length |
|
- `ddp_find_unused_parameters`: None |
|
- `ddp_bucket_cap_mb`: None |
|
- `ddp_broadcast_buffers`: False |
|
- `dataloader_pin_memory`: True |
|
- `dataloader_persistent_workers`: False |
|
- `skip_memory_metrics`: True |
|
- `use_legacy_prediction_loop`: False |
|
- `push_to_hub`: False |
|
- `resume_from_checkpoint`: None |
|
- `hub_model_id`: None |
|
- `hub_strategy`: every_save |
|
- `hub_private_repo`: False |
|
- `hub_always_push`: False |
|
- `gradient_checkpointing`: False |
|
- `gradient_checkpointing_kwargs`: None |
|
- `include_inputs_for_metrics`: False |
|
- `eval_do_concat_batches`: True |
|
- `fp16_backend`: auto |
|
- `push_to_hub_model_id`: None |
|
- `push_to_hub_organization`: None |
|
- `mp_parameters`: |
|
- `auto_find_batch_size`: False |
|
- `full_determinism`: False |
|
- `torchdynamo`: None |
|
- `ray_scope`: last |
|
- `ddp_timeout`: 1800 |
|
- `torch_compile`: False |
|
- `torch_compile_backend`: None |
|
- `torch_compile_mode`: None |
|
- `dispatch_batches`: None |
|
- `split_batches`: None |
|
- `include_tokens_per_second`: False |
|
- `include_num_input_tokens_seen`: False |
|
- `neftune_noise_alpha`: None |
|
- `optim_target_modules`: None |
|
- `batch_eval_metrics`: False |
|
- `batch_sampler`: no_duplicates |
|
- `multi_dataset_batch_sampler`: proportional |
|
|
|
</details> |
|
|
|
### Training Logs |
|
| Epoch | Step | Training Loss | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_512_cosine_map@100 | dim_64_cosine_map@100 | dim_768_cosine_map@100 | |
|
|:------:|:----:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|:----------------------:| |
|
| 0.2538 | 100 | 2.4219 | 0.7320 | 0.7542 | 0.7582 | 0.6929 | 0.7561 | |
|
| 0.5076 | 200 | 0.468 | 0.7343 | 0.7543 | 0.7574 | 0.7044 | 0.7569 | |
|
| 0.7614 | 300 | 0.3159 | 0.7569 | 0.7691 | 0.7749 | 0.7288 | 0.7713 | |
|
| 1.0152 | 400 | 0.317 | 0.7455 | 0.7607 | 0.7646 | 0.7124 | 0.7643 | |
|
| 1.2690 | 500 | 0.2062 | 0.7465 | 0.7691 | 0.7741 | 0.7211 | 0.7748 | |
|
| 1.5228 | 600 | 0.1075 | 0.7495 | 0.7599 | 0.7696 | 0.7214 | 0.7697 | |
|
| 1.7766 | 700 | 0.1079 | 0.7572 | 0.7660 | 0.7752 | 0.7287 | 0.7764 | |
|
| 2.0305 | 800 | 0.0477 | 0.7447 | 0.7696 | 0.7760 | 0.7211 | 0.7786 | |
|
| 2.2843 | 900 | 0.0547 | 0.7569 | 0.7728 | 0.7757 | 0.7406 | 0.7746 | |
|
| 2.5381 | 1000 | 0.0283 | 0.7668 | 0.7756 | 0.7823 | 0.7414 | 0.7841 | |
|
| 2.7919 | 1100 | 0.0268 | 0.7540 | 0.7673 | 0.7766 | 0.7432 | 0.7748 | |
|
| 3.0457 | 1200 | 0.0201 | 0.7633 | 0.7739 | 0.7799 | 0.7411 | 0.7775 | |
|
| 3.2995 | 1300 | 0.0174 | 0.7635 | 0.7745 | 0.7856 | 0.7469 | 0.7851 | |
|
| 3.5533 | 1400 | 0.0161 | 0.7595 | 0.7765 | 0.7825 | 0.7412 | 0.7782 | |
|
| 3.8071 | 1500 | 0.0071 | 0.7552 | 0.7680 | 0.7754 | 0.7395 | 0.7739 | |
|
| 4.0609 | 1600 | 0.009 | 0.7633 | 0.7767 | 0.7834 | 0.7423 | 0.7843 | |
|
| 4.3147 | 1700 | 0.0079 | 0.7639 | 0.7714 | 0.7770 | 0.7414 | 0.7728 | |
|
| 4.5685 | 1800 | 0.0109 | 0.7662 | 0.7775 | 0.7845 | 0.7369 | 0.7843 | |
|
| 4.8223 | 1900 | 0.0024 | 0.7674 | 0.7732 | 0.7776 | 0.7425 | 0.7810 | |
|
| 5.0761 | 2000 | 0.0052 | 0.7729 | 0.7746 | 0.7820 | 0.7455 | 0.7849 | |
|
| 5.3299 | 2100 | 0.0022 | 0.7615 | 0.7754 | 0.7813 | 0.7446 | 0.7862 | |
|
| 5.5838 | 2200 | 0.0065 | 0.7691 | 0.7761 | 0.7809 | 0.7437 | 0.7777 | |
|
| 5.8376 | 2300 | 0.0011 | 0.7672 | 0.7728 | 0.7757 | 0.7446 | 0.7772 | |
|
| 6.0914 | 2400 | 0.0046 | 0.7671 | 0.7778 | 0.7805 | 0.7494 | 0.7838 | |
|
| 6.3452 | 2500 | 0.0013 | 0.7655 | 0.7732 | 0.7780 | 0.7478 | 0.7806 | |
|
| 6.5990 | 2600 | 0.0058 | 0.7673 | 0.7753 | 0.7779 | 0.7542 | 0.7797 | |
|
| 6.8528 | 2700 | 0.001 | 0.7654 | 0.7716 | 0.7738 | 0.7535 | 0.7776 | |
|
| 7.1066 | 2800 | 0.0071 | 0.7684 | 0.7754 | 0.7792 | 0.7518 | 0.7824 | |
|
| 7.3604 | 2900 | 0.001 | 0.7723 | 0.7765 | 0.7814 | 0.7502 | 0.7826 | |
|
| 7.6142 | 3000 | 0.0028 | 0.7720 | 0.7754 | 0.7807 | 0.7498 | 0.7806 | |
|
| 7.8680 | 3100 | 0.0007 | 0.7685 | 0.7728 | 0.7773 | 0.7475 | 0.7816 | |
|
| 8.1218 | 3200 | 0.004 | 0.7690 | 0.7741 | 0.7773 | 0.7496 | 0.7806 | |
|
| 8.3756 | 3300 | 0.0006 | 0.7683 | 0.7723 | 0.7755 | 0.7491 | 0.7791 | |
|
| 8.6294 | 3400 | 0.0011 | 0.7678 | 0.7724 | 0.7756 | 0.7508 | 0.7804 | |
|
| 8.8832 | 3500 | 0.0006 | 0.7655 | 0.7721 | 0.7769 | 0.7467 | 0.7825 | |
|
| 9.1371 | 3600 | 0.0013 | 0.7674 | 0.7751 | 0.7788 | 0.7463 | 0.7802 | |
|
| 9.3909 | 3700 | 0.0006 | 0.7664 | 0.7741 | 0.7793 | 0.7468 | 0.7821 | |
|
| 9.6447 | 3800 | 0.0011 | 0.7662 | 0.7753 | 0.7782 | 0.7481 | 0.7803 | |
|
| 9.8985 | 3900 | 0.0005 | 0.7654 | 0.7738 | 0.7773 | 0.7467 | 0.7807 | |
|
|
|
|
|
### Framework Versions |
|
- Python: 3.10.12 |
|
- Sentence Transformers: 3.0.1 |
|
- Transformers: 4.41.2 |
|
- PyTorch: 2.3.0+cu121 |
|
- Accelerate: 0.31.0 |
|
- Datasets: 2.19.2 |
|
- Tokenizers: 0.19.1 |
|
|
|
## Citation |
|
|
|
### BibTeX |
|
|
|
#### Sentence Transformers |
|
```bibtex |
|
@inproceedings{reimers-2019-sentence-bert, |
|
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", |
|
author = "Reimers, Nils and Gurevych, Iryna", |
|
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", |
|
month = "11", |
|
year = "2019", |
|
publisher = "Association for Computational Linguistics", |
|
url = "https://arxiv.org/abs/1908.10084", |
|
} |
|
``` |
|
|
|
#### MatryoshkaLoss |
|
```bibtex |
|
@misc{kusupati2024matryoshka, |
|
title={Matryoshka Representation Learning}, |
|
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, |
|
year={2024}, |
|
eprint={2205.13147}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.LG} |
|
} |
|
``` |
|
|
|
#### MultipleNegativesRankingLoss |
|
```bibtex |
|
@misc{henderson2017efficient, |
|
title={Efficient Natural Language Response Suggestion for Smart Reply}, |
|
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, |
|
year={2017}, |
|
eprint={1705.00652}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
``` |
|
|
|
<!-- |
|
## Glossary |
|
|
|
*Clearly define terms in order to be accessible across audiences.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Authors |
|
|
|
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Contact |
|
|
|
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
|
--> |
