--- base_model: Snowflake/snowflake-arctic-embed-m library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 - dot_accuracy@1 - dot_accuracy@3 - dot_accuracy@5 - dot_accuracy@10 - dot_precision@1 - dot_precision@3 - dot_precision@5 - dot_precision@10 - dot_recall@1 - dot_recall@3 - dot_recall@5 - dot_recall@10 - dot_ndcg@10 - dot_mrr@10 - dot_map@100 pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:600 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss widget: - source_sentence: What is the purpose of the Artificial Intelligence Ethics for the Intelligence Community as mentioned in the context? sentences: - "You should be able to opt out, where appropriate, and \nhave access to a person\ \ who can quickly consider and \nremedy problems you encounter. You should be\ \ able to opt \nout from automated systems in favor of a human alternative, where\ \ \nappropriate. Appropriateness should be determined based on rea­\nsonable expectations\ \ in a given context and with a focus on ensuring \nbroad accessibility and protecting\ \ the public from especially harm­\nful impacts. In some cases, a human or other\ \ alternative may be re­\nquired by law. You should have access to timely human\ \ consider­\nation and remedy by a fallback and escalation process if an automat­\n\ ed system fails, it produces an error, or you would like to appeal or \ncontest\ \ its impacts on you. Human consideration and fallback \nshould be accessible,\ \ equitable, effective, maintained, accompanied \nby appropriate operator training,\ \ and should not impose an unrea­\nsonable burden on the public. Automated systems\ \ with an intended" - "points to numerous examples of effective and proactive stakeholder engagement,\ \ including the Community-\nBased Participatory Research Program developed by\ \ the National Institutes of Health and the participatory \ntechnology assessments\ \ developed by the National Oceanic and Atmospheric Administration.18\nThe National\ \ Institute of Standards and Technology (NIST) is developing a risk \nmanagement\ \ framework to better manage risks posed to individuals, organizations, and \n\ society by AI.19 The NIST AI Risk Management Framework, as mandated by Congress,\ \ is intended for \nvoluntary use to help incorporate trustworthiness considerations\ \ into the design, development, use, and \nevaluation of AI products, services,\ \ and systems. The NIST framework is being developed through a consensus-\ndriven,\ \ open, transparent, and collaborative process that includes workshops and other\ \ opportunities to provide \ninput. The NIST framework aims to foster the development\ \ of innovative approaches to address" - "of Artificial Intelligence Ethics for the Intelligence Community to guide personnel\ \ on whether and how to \ndevelop and use AI in furtherance of the IC's mission,\ \ as well as an AI Ethics Framework to help implement \nthese principles.22\n\ The National Science Foundation (NSF) funds extensive research to help foster\ \ the \ndevelopment of automated systems that adhere to and advance their safety,\ \ security and \neffectiveness. Multiple NSF programs support research that directly\ \ addresses many of these principles: \nthe National AI Research Institutes23\ \ support research on all aspects of safe, trustworthy, fair, and explainable\ \ \nAI algorithms and systems; the Cyber Physical Systems24 program supports research\ \ on developing safe \nautonomous and cyber physical systems with AI components;\ \ the Secure and Trustworthy Cyberspace25 \nprogram supports research on cybersecurity\ \ and privacy enhancing technologies in automated systems; the" - source_sentence: How does the Department of Defense's approach to AI ethics differ from that of the Department of Energy? sentences: - "NOTICE & \nEXPLANATION \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations\ \ for automated systems are meant to serve as a blueprint for the development\ \ of additional \ntechnical standards and practices that are tailored for particular\ \ sectors and contexts. \nTailored to the level of risk. An assessment should\ \ be done to determine the level of risk of the auto­\nmated system. In settings\ \ where the consequences are high as determined by a risk assessment, or extensive\ \ \noversight is expected (e.g., in criminal justice or some public sector settings),\ \ explanatory mechanisms should \nbe built into the system design so that the\ \ system’s full behavior can be explained in advance (i.e., only fully \ntransparent\ \ models should be used), rather than as an after-the-decision interpretation.\ \ In other settings, the \nextent of explanation provided should be tailored to\ \ the risk level." - "SAFE AND EFFECTIVE \nSYSTEMS \nHOW THESE PRINCIPLES CAN MOVE INTO PRACTICE\n\ Real-life examples of how these principles can become reality, through laws, policies,\ \ and practical \ntechnical and sociotechnical approaches to protecting rights,\ \ opportunities, and access. ­\nSome U.S government agencies have developed specific\ \ frameworks for ethical use of AI \nsystems. The Department of Energy (DOE) has\ \ activated the AI Advancement Council that oversees coordina-\ntion and advises\ \ on implementation of the DOE AI Strategy and addresses issues and/or escalations\ \ on the \nethical use and development of AI systems.20 The Department of Defense\ \ has adopted Artificial Intelligence \nEthical Principles, and tenets for Responsible\ \ Artificial Intelligence specifically tailored to its national \nsecurity and\ \ defense activities.21 Similarly, the U.S. Intelligence Community (IC) has developed\ \ the Principles" - "Formal Methods in the Field26 program supports research on rigorous formal verification\ \ and analysis of \nautomated systems and machine learning, and the Designing\ \ Accountable Software Systems27 program supports \nresearch on rigorous and reproducible\ \ methodologies for developing software systems with legal and regulatory \ncompliance\ \ in mind. \nSome state legislatures have placed strong transparency and validity\ \ requirements on \nthe use of pretrial risk assessments. The use of algorithmic\ \ pretrial risk assessments has been a \ncause of concern for civil rights groups.28\ \ Idaho Code Section 19-1910, enacted in 2019,29 requires that any \npretrial\ \ risk assessment, before use in the state, first be \"shown to be free of bias\ \ against any class of \nindividuals protected from discrimination by state or\ \ federal law\", that any locality using a pretrial risk \nassessment must first\ \ formally validate the claim of its being free of bias, that \"all documents,\ \ records, and" - source_sentence: What are the expectations for automated systems intended to serve as a blueprint for? sentences: - "help to mitigate biases and potential harms. \nGuarding against proxies. Directly\ \ using demographic information in the design, development, or \ndeployment of\ \ an automated system (for purposes other than evaluating a system for discrimination\ \ or using \na system to counter discrimination) runs a high risk of leading to\ \ algorithmic discrimination and should be \navoided. In many cases, attributes\ \ that are highly correlated with demographic features, known as proxies, can\ \ \ncontribute to algorithmic discrimination. In cases where use of the demographic\ \ features themselves would \nlead to illegal algorithmic discrimination, reliance\ \ on such proxies in decision-making (such as that facilitated \nby an algorithm)\ \ may also be prohibited by law. Proactive testing should be performed to identify\ \ proxies by \ntesting for correlation between demographic information and attributes\ \ in any data used as part of system" - "describes three broad challenges for mitigating bias – datasets, testing and\ \ evaluation, and human factors – and \nintroduces preliminary guidance for addressing\ \ them. Throughout, the special publication takes a socio-\ntechnical perspective\ \ to identifying and managing AI bias. \n29\nAlgorithmic \nDiscrimination \nProtections" - "SAFE AND EFFECTIVE \nSYSTEMS \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\n\ The expectations for automated systems are meant to serve as a blueprint for the\ \ development of additional \ntechnical standards and practices that are tailored\ \ for particular sectors and contexts. \nDerived data sources tracked and reviewed\ \ carefully. Data that is derived from other data through \nthe use of algorithms,\ \ such as data derived or inferred from prior model outputs, should be identified\ \ and \ntracked, e.g., via a specialized type in a data schema. Derived data should\ \ be viewed as potentially high-risk \ninputs that may lead to feedback loops,\ \ compounded harm, or inaccurate results. Such sources should be care­\nfully\ \ validated against the risk of collateral consequences. \nData reuse limits in\ \ sensitive domains. Data reuse, and especially data reuse in a new context, can\ \ result \nin the spreading and scaling of harms. Data from some domains, including\ \ criminal justice data and data indi­" - source_sentence: What should individuals have access to regarding their data decisions and the impact of surveillance technologies? sentences: - '• Searches for “Black girls,” “Asian girls,” or “Latina girls” return predominantly39 sexualized content, rather than role models, toys, or activities.40 Some search engines have been working to reduce the prevalence of these results, but the problem remains.41 • Advertisement delivery systems that predict who is most likely to click on a job advertisement end up deliv- ering ads in ways that reinforce racial and gender stereotypes, such as overwhelmingly directing supermar- ket cashier ads to women and jobs with taxi companies to primarily Black people.42­ • Body scanners, used by TSA at airport checkpoints, require the operator to select a “male” or “female” scanning setting based on the passenger’s sex, but the setting is chosen based on the operator’s perception of the passenger’s gender identity. These scanners are more likely to flag transgender travelers as requiring extra screening done by a person. Transgender travelers have described degrading experiences associated' - "information used to build or validate the risk assessment shall be open to public\ \ inspection,\" and that assertions \nof trade secrets cannot be used \"to quash\ \ discovery in a criminal matter by a party to a criminal case.\" \n22" - "tect privacy and civil liberties. Continuous surveillance and monitoring \nshould\ \ not be used in education, work, housing, or in other contexts where the \nuse\ \ of such surveillance technologies is likely to limit rights, opportunities,\ \ or \naccess. Whenever possible, you should have access to reporting that confirms\ \ \nyour data decisions have been respected and provides an assessment of the\ \ \npotential impact of surveillance technologies on your rights, opportunities,\ \ or \naccess. \nDATA PRIVACY\n30" - source_sentence: What are the implications of the digital divide highlighted in Andrew Kenney's article regarding unemployment benefits? sentences: - "cating adverse outcomes in domains such as finance, employment, and housing,\ \ is especially sensitive, and in \nsome cases its reuse is limited by law. Accordingly,\ \ such data should be subject to extra oversight to ensure \nsafety and efficacy.\ \ Data reuse of sensitive domain data in other contexts (e.g., criminal data reuse\ \ for civil legal \nmatters or private sector use) should only occur where use\ \ of such data is legally authorized and, after examina­\ntion, has benefits for\ \ those impacted by the system that outweigh identified risks and, as appropriate,\ \ reason­\nable measures have been implemented to mitigate the identified risks.\ \ Such data should be clearly labeled to \nidentify contexts for limited reuse\ \ based on sensitivity. Where possible, aggregated datasets may be useful for\ \ \nreplacing individual-level sensitive data. \nDemonstrate the safety and effectiveness\ \ of the system \nIndependent evaluation. Automated systems should be designed\ \ to allow for independent evaluation (e.g.," - "5. Environmental Impacts: Impacts due to high compute resource utilization in\ \ training or \noperating GAI models, and related outcomes that may adversely\ \ impact ecosystems. \n6. Harmful Bias or Homogenization: Amplification and exacerbation\ \ of historical, societal, and \nsystemic biases; performance disparities8 between\ \ sub-groups or languages, possibly due to \nnon-representative training data,\ \ that result in discrimination, amplification of biases, or \nincorrect presumptions\ \ about performance; undesired homogeneity that skews system or model \noutputs,\ \ which may be erroneous, lead to ill-founded decision-making, or amplify harmful\ \ \nbiases. \n7. Human-AI Configuration: Arrangements of or interactions between\ \ a human and an AI system \nwhich can result in the human inappropriately anthropomorphizing\ \ GAI systems or experiencing \nalgorithmic aversion, automation bias, over-reliance,\ \ or emotional entanglement with GAI \nsystems." - 'https://bipartisanpolicy.org/blog/the-low-down-on-ballot-curing/ 101. Andrew Kenney. ''I''m shocked that they need to have a smartphone'': System for unemployment benefits exposes digital divide. USA Today. May 2, 2021. https://www.usatoday.com/story/tech/news/2021/05/02/unemployment-benefits-system-leaving­ people-behind/4915248001/ 102. Allie Gross. UIA lawsuit shows how the state criminalizes the unemployed. Detroit Metro-Times. Sep. 18, 2015. https://www.metrotimes.com/news/uia-lawsuit-shows-how-the-state-criminalizes-the­ unemployed-2369412 103. Maia Szalavitz. The Pain Was Unbearable. So Why Did Doctors Turn Her Away? Wired. Aug. 11, 2021. https://www.wired.com/story/opioid-drug-addiction-algorithm-chronic-pain/ 104. Spencer Soper. Fired by Bot at Amazon: "It''s You Against the Machine". Bloomberg, Jun. 28, 2021. https://www.bloomberg.com/news/features/2021-06-28/fired-by-bot-amazon-turns-to-machine­ managers-and-workers-are-losing-out' model-index: - name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-m results: - task: type: information-retrieval name: Information Retrieval dataset: name: Unknown type: unknown metrics: - type: cosine_accuracy@1 value: 0.73 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.9 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.935 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.96 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.73 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.187 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.096 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.73 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.9 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.935 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.96 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8511693160760204 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.8155396825396827 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.8172228277187864 name: Cosine Map@100 - type: dot_accuracy@1 value: 0.73 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.9 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.935 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.96 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.73 name: Dot Precision@1 - type: dot_precision@3 value: 0.3 name: Dot Precision@3 - type: dot_precision@5 value: 0.187 name: Dot Precision@5 - type: dot_precision@10 value: 0.096 name: Dot Precision@10 - type: dot_recall@1 value: 0.73 name: Dot Recall@1 - type: dot_recall@3 value: 0.9 name: Dot Recall@3 - type: dot_recall@5 value: 0.935 name: Dot Recall@5 - type: dot_recall@10 value: 0.96 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.8511693160760204 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.8155396825396827 name: Dot Mrr@10 - type: dot_map@100 value: 0.8172228277187864 name: Dot Map@100 --- # SentenceTransformer based on Snowflake/snowflake-arctic-embed-m This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-m](https://huggingface.co./Snowflake/snowflake-arctic-embed-m). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [Snowflake/snowflake-arctic-embed-m](https://huggingface.co./Snowflake/snowflake-arctic-embed-m) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 tokens - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("ldldld/snowflake-arctic-embed-m-finetuned") # Run inference sentences = [ "What are the implications of the digital divide highlighted in Andrew Kenney's article regarding unemployment benefits?", 'https://bipartisanpolicy.org/blog/the-low-down-on-ballot-curing/\n101. Andrew Kenney. \'I\'m shocked that they need to have a smartphone\': System for unemployment\nbenefits exposes digital divide. USA Today. May 2, 2021.\nhttps://www.usatoday.com/story/tech/news/2021/05/02/unemployment-benefits-system-leaving\xad\npeople-behind/4915248001/\n102. Allie Gross. UIA lawsuit shows how the state criminalizes the unemployed. Detroit Metro-Times.\nSep. 18, 2015.\nhttps://www.metrotimes.com/news/uia-lawsuit-shows-how-the-state-criminalizes-the\xad\nunemployed-2369412\n103. Maia Szalavitz. The Pain Was Unbearable. So Why Did Doctors Turn Her Away? Wired. Aug. 11,\n2021. https://www.wired.com/story/opioid-drug-addiction-algorithm-chronic-pain/\n104. Spencer Soper. Fired by Bot at Amazon: "It\'s You Against the Machine". Bloomberg, Jun. 28, 2021.\nhttps://www.bloomberg.com/news/features/2021-06-28/fired-by-bot-amazon-turns-to-machine\xad\nmanagers-and-workers-are-losing-out', '5. Environmental Impacts: Impacts due to high compute resource utilization in training or \noperating GAI models, and related outcomes that may adversely impact ecosystems. \n6. Harmful Bias or Homogenization: Amplification and exacerbation of historical, societal, and \nsystemic biases; performance disparities8 between sub-groups or languages, possibly due to \nnon-representative training data, that result in discrimination, amplification of biases, or \nincorrect presumptions about performance; undesired homogeneity that skews system or model \noutputs, which may be erroneous, lead to ill-founded decision-making, or amplify harmful \nbiases. \n7. Human-AI Configuration: Arrangements of or interactions between a human and an AI system \nwhich can result in the human inappropriately anthropomorphizing GAI systems or experiencing \nalgorithmic aversion, automation bias, over-reliance, or emotional entanglement with GAI \nsystems.', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.73 | | cosine_accuracy@3 | 0.9 | | cosine_accuracy@5 | 0.935 | | cosine_accuracy@10 | 0.96 | | cosine_precision@1 | 0.73 | | cosine_precision@3 | 0.3 | | cosine_precision@5 | 0.187 | | cosine_precision@10 | 0.096 | | cosine_recall@1 | 0.73 | | cosine_recall@3 | 0.9 | | cosine_recall@5 | 0.935 | | cosine_recall@10 | 0.96 | | cosine_ndcg@10 | 0.8512 | | cosine_mrr@10 | 0.8155 | | **cosine_map@100** | **0.8172** | | dot_accuracy@1 | 0.73 | | dot_accuracy@3 | 0.9 | | dot_accuracy@5 | 0.935 | | dot_accuracy@10 | 0.96 | | dot_precision@1 | 0.73 | | dot_precision@3 | 0.3 | | dot_precision@5 | 0.187 | | dot_precision@10 | 0.096 | | dot_recall@1 | 0.73 | | dot_recall@3 | 0.9 | | dot_recall@5 | 0.935 | | dot_recall@10 | 0.96 | | dot_ndcg@10 | 0.8512 | | dot_mrr@10 | 0.8155 | | dot_map@100 | 0.8172 | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 600 training samples * Columns: sentence_0 and sentence_1 * Approximate statistics based on the first 600 samples: | | sentence_0 | sentence_1 | |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | sentence_0 | sentence_1 | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | What is the main purpose of the "Blueprint for an AI Bill of Rights" as indicated in the context? | BLUEPRINT FOR AN
AI BILL OF
RIGHTS
MAKING AUTOMATED
SYSTEMS WORK FOR
THE AMERICAN PEOPLE
OCTOBER 2022 | | When was the "Blueprint for an AI Bill of Rights" created? | BLUEPRINT FOR AN
AI BILL OF
RIGHTS
MAKING AUTOMATED
SYSTEMS WORK FOR
THE AMERICAN PEOPLE
OCTOBER 2022 | | What was the purpose of the Blueprint for an AI Bill of Rights published by the White House Office of Science and Technology Policy in October 2022? | About this Document
The Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People was
published by the White House Office of Science and Technology Policy in October 2022. This framework was
released one year after OSTP announced the launch of a process to develop “a bill of rights for an AI-powered
world.” Its release follows a year of public engagement to inform this initiative. The framework is available
online at: https://www.whitehouse.gov/ostp/ai-bill-of-rights
About the Office of Science and Technology Policy
The Office of Science and Technology Policy (OSTP) was established by the National Science and Technology
Policy, Organization, and Priorities Act of 1976 to provide the President and others within the Executive Office
of the President with advice on the scientific, engineering, and technological aspects of the economy, national | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 768, 512, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 20 - `per_device_eval_batch_size`: 20 - `num_train_epochs`: 5 - `multi_dataset_batch_sampler`: round_robin #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 20 - `per_device_eval_batch_size`: 20 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 5e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1 - `num_train_epochs`: 5 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.0 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `eval_use_gather_object`: False - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: round_robin
### Training Logs | Epoch | Step | cosine_map@100 | |:------:|:----:|:--------------:| | 1.0 | 30 | 0.7953 | | 1.6667 | 50 | 0.8326 | | 2.0 | 60 | 0.8277 | | 3.0 | 90 | 0.8250 | | 3.3333 | 100 | 0.8284 | | 4.0 | 120 | 0.8200 | | 5.0 | 150 | 0.8172 | ### Framework Versions - Python: 3.10.12 - Sentence Transformers: 3.1.1 - Transformers: 4.44.2 - PyTorch: 2.4.1+cu121 - Accelerate: 0.34.2 - Datasets: 3.0.0 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```