--- language: - en license: apache-2.0 tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:181 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss base_model: mixedbread-ai/mxbai-embed-large-v1 widget: - source_sentence: '[TextBlock(text=''What type of radiotherapy should be considered for circumscribed, notably symptomatic lesions?'', type=''text'')]' sentences: - For diagnosing IDH-wild-type glioblastoma in diffuse gliomas without microvascular proliferation and necrosis, which are histological features of WHO grade 4, it is crucial to test for a combination of chromosome 7 gain and chromosome 10 loss (the +7/+10 signature), EGFR amplification, and TERT promoter mutation. This testing accurately identifies IDH-wild-type glioblastoma in such cases. - Focal radiotherapy should be considered for circumscribed, notably symptomatic lesions. - For patients with leptomeningeal metastasis CSF studies should be carried out every 6-12 weeks in patients undergoing intra-CSF pharmacotherapy. - source_sentence: '[TextBlock(text=''What is the recommended management strategy for patients with incompletely resected WHO grade 1 meningioma without neurological deficits?'', type=''text'')]' sentences: - Screening and prevention have no major role for patients with gliomas. The counselling and screening of asymptomatic relatives of patients with glioma who are found to be carriers of germline mutations associated with gliomagenesis should be conducted with caution and in cooperation with clinical geneticists. No known measures to prevent the development of gliomas exist. - The combination of intended subtotal surgery and radiosurgery or fractionated radiotherapy in WHO grade 1 meningioma should be considered for comprehensive tumor treatment with reduced risk of tumor progression. - Patients with incompletely resected WHO grade 1 meningioma without neurological deficits may be managed by a watch-and-scan strategy. - source_sentence: '[TextBlock(text=''What are the advantages of using next-generation-sequencing (NGS) for suspected glioma diagnosis?'', type=''text'')]' sentences: - Goal of surgery is gross total resection of the meningeoma including resection of the underlying bone and associated dura whenever safely feasible. - Low Molecular Weight Heparin (LMWH) should be considered as the first line of primary thromboprophylaxis of venous thromboembolism (VTE) for patients with brain tumours after brain tumour surgery. - For suspected glioma the advantages of next-generation-sequencing (NGS) in covering a variety of alterations, including those of low abundance, within a single assay and with small input amounts should be considered when selecting the testing methodology; this is particularly relevant when the diagnosis is challenging and thus the spectrum of potentially relevant genetic variants being broad. - source_sentence: '[TextBlock(text="What factors should be considered when assessing a person''s competency to drive?", type=''text'')]' sentences: - Refractory and relapsed Primary Vitreoretinal Lymphoma should be treated according to the patients characteristics and prior treatments. Potential treatment options include intravitreal injections of methotrexate (MTX), focal radiotherapy, Whole brain radiation therapy , systemic chemotherapy, targeted treatment and High-Dose Chemotherapy /Autologous Stem Cell Transplant (AST). In general, Patients with relapsed and refractory PCNSL should be enrolled into clinical trials. - Judgements on the competency to drive need to adhere to national guidelines and law and should consider not only epilepsy but also other aspects of neurological and neurocognitive function. - Supportive care with omission of whole brain radiation therapy should be considered in patients with multiple brain metastases not eligible for stereotactic radiosurgery and poor performance index. - source_sentence: '[TextBlock(text=''What is critical to avoid misinterpretation of immunohistochemical stainings of glioma tissue?'', type=''text'')]' sentences: - Cerebral MRI should include axial T1-weighted, axial FLAIR, axial diffusion-weighted, axial T2-weighted, post-gadolinium 3D T1-weighted and post-gadolinium 3D FLAIR sequences. Spinal MRI should include post-gadolinium sagittal T1-weighted sequences. Spine sagittal T1-weighted sequences without contrast and sagittal fat-suppression T2-weighted sequences, combined with axial T1-weighted images with contrast of regions of interest, may also be considered. - If glioma tissue volume is limited, the potential value and input requirements of high-throughput analyses should be considered early in the diagnostic decision-making process, taking into account the strengths and weaknesses of analytical methods as outlined below, especially in cases that do not qualify for targeted analyses. - To avoid misinterpretation of immunohistochemical stainings of glioma tissue it is critical to choose antibodies that work well on FFPE material, to optimize and validate tissue pretreatment and staining protocols stringently, and to perform tests alongside appropriate negative and positive controls. For example, ATRX and H3 p.K28me3 immunohistochemistry are sensitive to hypoxia and crush artifacts; in this setting nuclear staining in non-neoplastic cells can serve as a positive internal control. pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 model-index: - name: MXBD REMEDY Matryoshka_v2 results: - task: type: information-retrieval name: Information Retrieval dataset: name: dim 768 type: dim_768 metrics: - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.33333333333333326 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.20000000000000004 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.10000000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.33333333333333326 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.20000000000000004 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.10000000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 512 type: dim_512 metrics: - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.33333333333333326 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.20000000000000004 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.10000000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.33333333333333326 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.20000000000000004 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.10000000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 256 type: dim_256 metrics: - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.33333333333333326 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.20000000000000004 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.10000000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.33333333333333326 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.20000000000000004 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.10000000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 128 type: dim_128 metrics: - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.33333333333333326 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.20000000000000004 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.10000000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.33333333333333326 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.20000000000000004 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.10000000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 64 type: dim_64 metrics: - type: cosine_accuracy@1 value: 0.9523809523809523 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.9523809523809523 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.33333333333333326 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.20000000000000004 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.10000000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.9523809523809523 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.9824252263605456 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.9761904761904762 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.9761904761904762 name: Cosine Map@100 - type: cosine_accuracy@1 value: 0.9523809523809523 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.9523809523809523 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.33333333333333326 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.20000000000000004 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.10000000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.9523809523809523 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.9824252263605456 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.9761904761904762 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.9761904761904762 name: Cosine Map@100 --- # MXBD REMEDY Matryoshka_v2 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [mixedbread-ai/mxbai-embed-large-v1](https://huggingface.co./mixedbread-ai/mxbai-embed-large-v1) on the json dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [mixedbread-ai/mxbai-embed-large-v1](https://huggingface.co./mixedbread-ai/mxbai-embed-large-v1) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 1024 dimensions - **Similarity Function:** Cosine Similarity - **Training Dataset:** - json - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("NeurologyAI/mxbai-remedy-matryoshka_v3") # Run inference sentences = [ "[TextBlock(text='What is critical to avoid misinterpretation of immunohistochemical stainings of glioma tissue?', type='text')]", 'To avoid misinterpretation of immunohistochemical stainings of glioma tissue it is critical to choose antibodies that work well on FFPE material, to optimize and validate tissue pretreatment and staining protocols stringently, and to perform tests alongside appropriate negative and positive controls. For example, ATRX and H3 p.K28me3 immunohistochemistry are sensitive to hypoxia and crush artifacts; in this setting nuclear staining in non-neoplastic cells can serve as a positive internal control.', 'Cerebral MRI should include axial T1-weighted, axial FLAIR, axial diffusion-weighted, axial T2-weighted, post-gadolinium 3D T1-weighted and post-gadolinium 3D FLAIR sequences. Spinal MRI should include post-gadolinium sagittal T1-weighted sequences. Spine sagittal T1-weighted sequences without contrast and sagittal fat-suppression T2-weighted sequences, combined with axial T1-weighted images with contrast of regions of interest, may also be considered.', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 1024] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Datasets: `dim_768`, `dim_512`, `dim_256`, `dim_128`, `dim_64`, `dim_768`, `dim_512`, `dim_256`, `dim_128` and `dim_64` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | dim_768 | dim_512 | dim_256 | dim_128 | dim_64 | |:--------------------|:--------|:--------|:--------|:--------|:-----------| | cosine_accuracy@1 | 1.0 | 1.0 | 1.0 | 1.0 | 0.9524 | | cosine_accuracy@3 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | | cosine_accuracy@5 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | | cosine_accuracy@10 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | | cosine_precision@1 | 1.0 | 1.0 | 1.0 | 1.0 | 0.9524 | | cosine_precision@3 | 0.3333 | 0.3333 | 0.3333 | 0.3333 | 0.3333 | | cosine_precision@5 | 0.2 | 0.2 | 0.2 | 0.2 | 0.2 | | cosine_precision@10 | 0.1 | 0.1 | 0.1 | 0.1 | 0.1 | | cosine_recall@1 | 1.0 | 1.0 | 1.0 | 1.0 | 0.9524 | | cosine_recall@3 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | | cosine_recall@5 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | | cosine_recall@10 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | | **cosine_ndcg@10** | **1.0** | **1.0** | **1.0** | **1.0** | **0.9824** | | cosine_mrr@10 | 1.0 | 1.0 | 1.0 | 1.0 | 0.9762 | | cosine_map@100 | 1.0 | 1.0 | 1.0 | 1.0 | 0.9762 | ## Training Details ### Training Dataset #### json * Dataset: json * Size: 181 training samples * Columns: anchor and positive * Approximate statistics based on the first 181 samples: | | anchor | positive | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | anchor | positive | |:-------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [TextBlock(text='What is the recommended treatment for patients with recurrent or atypical meningiomas?', type='text')] | Patients with recurrent or atypical meningiomas hould receive fractionated radiotherapy. | | [TextBlock(text='What is the recommended treatment for headache in patients with gliomas?', type='text')] | Pain or headache. Corticosteroids (dexamethasone) should be the mainstay of treatment for headache in patients with gliomas. Analgesics and co-analgesics could also be considered in the treatment of headache in patients with gliomas in accordance with the WHO cancer pain ladder. During care in the end-of-life phase, consideration needs to be given to the management of headache with corticosteroids; advantages of corticosteroids (alleviation of symptoms) should be weighed against side-effects (such as delirium). | | [TextBlock(text='What is the priority in the current surgical approach to gliomas?', type='text')] | The extent of resection is a prognostic factor and thus, efforts at obtaining complete resections are justified across all glioma entities. In the current surgical approach to gliomas, the prevention of new permanent neurological deficits has higher priority than the extent of resection. | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 768, 512, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: epoch - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `gradient_accumulation_steps`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 4 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `bf16`: True - `tf32`: True - `load_best_model_at_end`: True - `optim`: adamw_torch_fused - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: epoch - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 16 - `eval_accumulation_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 4 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: True - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 | |:-------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:| | **1.0** | **1** | **0.9235** | **0.8645** | **0.8624** | **0.7593** | **0.6138** | | 2.0 | 2 | 1.0 | 0.9649 | 1.0 | 0.9824 | 0.9473 | | 3.0 | 4 | 1.0 | 1.0 | 1.0 | 1.0 | 0.9473 | | **1.0** | **1** | **1.0** | **1.0** | **1.0** | **1.0** | **0.9473** | | 2.0 | 2 | 1.0 | 1.0 | 1.0 | 1.0 | 0.9824 | | 3.0 | 4 | 1.0 | 1.0 | 1.0 | 1.0 | 0.9824 | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.10.12 - Sentence Transformers: 3.3.1 - Transformers: 4.41.2 - PyTorch: 2.1.2+cu121 - Accelerate: 1.1.1 - Datasets: 2.19.1 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```