metadata

base_model: Snowflake/snowflake-arctic-embed-m
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
  - dot_accuracy@1
  - dot_accuracy@3
  - dot_accuracy@5
  - dot_accuracy@10
  - dot_precision@1
  - dot_precision@3
  - dot_precision@5
  - dot_precision@10
  - dot_recall@1
  - dot_recall@3
  - dot_recall@5
  - dot_recall@10
  - dot_ndcg@10
  - dot_mrr@10
  - dot_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:714
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: What are some examples of data privacy issues mentioned in the context?
    sentences:
      - >-
        on a principle of local control, such that those individuals closest to
        the data subject have more access while 

        those who are less proximate do not (e.g., a teacher has access to their
        students’ daily progress data while a 

        superintendent does not). 

        Reporting. In addition to the reporting on data privacy (as listed above
        for non-sensitive data), entities devel-

        oping technologies related to a sensitive domain and those collecting,
        using, storing, or sharing sensitive data 

        should, whenever appropriate, regularly provide public reports
        describing: any data security lapses or breaches 

        that resulted in sensitive data leaks; the number, type, and outcomes of
        ethical pre-reviews undertaken; a 

        description of any data sold, shared, or made public, and how that data
        was assessed to determine it did not pres-

        ent a sensitive data risk; and ongoing risk identification and
        management procedures, and any mitigation added
      - >-
        DATA PRIVACY 

        HOW THESE PRINCIPLES CAN MOVE INTO PRACTICE

        Real-life examples of how these principles can become reality, through
        laws, policies, and practical 

        technical and sociotechnical approaches to protecting rights,
        opportunities, and access. 

        The Privacy Act of 1974 requires privacy protections for personal
        information in federal 

        records systems, including limits on data retention, and also provides
        individuals a general 

        right to access and correct their data. Among other things, the Privacy
        Act limits the storage of individual 

        information in federal systems of records, illustrating the principle of
        limiting the scope of data retention. Under 

        the Privacy Act, federal agencies may only retain data about an
        individual that is “relevant and necessary” to 

        accomplish an agency’s statutory purpose or to comply with an Executive
        Order of the President. The law allows
      - >-
        DATA PRIVACY 

        WHY THIS PRINCIPLE IS IMPORTANT

        This section provides a brief summary of the problems which the
        principle seeks to address and protect 

        against, including illustrative examples. 

        •

        An insurer might collect data from a person's social media presence as
        part of deciding what life

        insurance rates they should be offered.64

        •

        A data broker harvested large amounts of personal data and then suffered
        a breach, exposing hundreds of

        thousands of people to potential identity theft. 65

        •

        A local public housing authority installed a facial recognition system
        at the entrance to housing complexes to

        assist law enforcement with identifying individuals viewed via camera
        when police reports are filed, leading

        the community, both those living in the housing complex and not, to have
        videos of them sent to the local

        police department and made available for scanning by its facial
        recognition software.66

        •
  - source_sentence: >-
      What are the main topics covered in the National Institute of Standards
      and Technology's AI Risk Management Framework?
    sentences:
      - >-
        https://www.rand.org/pubs/research_reports/RRA2977-2.html. 

        Nicoletti, L. et al. (2023) Humans Are Biased. Generative Ai Is Even
        Worse. Bloomberg. 

        https://www.bloomberg.com/graphics/2023-generative-ai-bias/. 

        National Institute of Standards and Technology (2024) Adversarial
        Machine Learning: A Taxonomy and 

        Terminology of Attacks and Mitigations
        https://csrc.nist.gov/pubs/ai/100/2/e2023/ﬁnal 

        National Institute of Standards and Technology (2023) AI Risk Management
        Framework. 

        https://www.nist.gov/itl/ai-risk-management-framework 

        National Institute of Standards and Technology (2023) AI Risk Management
        Framework, Chapter 3: AI 

        Risks and Trustworthiness. 

        https://airc.nist.gov/AI_RMF_Knowledge_Base/AI_RMF/Foundational_Information/3-sec-characteristics 

        National Institute of Standards and Technology (2023) AI Risk Management
        Framework, Chapter 6: AI 

        RMF Proﬁles.
        https://airc.nist.gov/AI_RMF_Knowledge_Base/AI_RMF/Core_And_Proﬁles/6-sec-proﬁle
      - >-
        (e.g., via red-teaming, ﬁeld testing, participatory engagements,
        performance 

        assessments, user feedback mechanisms). 

        Human-AI Conﬁguration 

        AI Actor Tasks: AI Development, AI Deployment, AI Impact Assessment,
        Operation and Monitoring 
         
        MANAGE 2.2: Mechanisms are in place and applied to sustain the value of
        deployed AI systems. 

        Action ID 

        Suggested Action 

        GAI Risks 

        MG-2.2-001 

        Compare GAI system outputs against pre-deﬁned organization risk
        tolerance, 

        guidelines, and principles, and review and test AI-generated content
        against 

        these guidelines. 

        CBRN Information or Capabilities; 

        Obscene, Degrading, and/or 

        Abusive Content; Harmful Bias and 

        Homogenization; Dangerous, 

        Violent, or Hateful Content 

        MG-2.2-002 

        Document training data sources to trace the origin and provenance of AI-

        generated content. 

        Information Integrity 

        MG-2.2-003 

        Evaluate feedback loops between GAI system content provenance and human
      - >-
        domain or for functions that are required for administrative reasons
        (e.g., school attendance records), unless 

        consent is acquired, if appropriate, and the additional expectations in
        this section are met. Consent for non-

        necessary functions should be optional, i.e., should not be required,
        incentivized, or coerced in order to 

        receive opportunities or access to services. In cases where data is
        provided to an entity (e.g., health insurance 

        company) in order to facilitate payment for such a need, that data
        should only be used for that purpose. 

        Ethical review and use prohibitions. Any use of sensitive data or
        decision process based in part on sensi-

        tive data that might limit rights, opportunities, or access, whether the
        decision is automated or not, should go 

        through a thorough ethical review and monitoring, both in advance and by
        periodic review (e.g., via an indepen-

        dent ethics committee or similarly robust process). In some cases, this
        ethical review may determine that data
  - source_sentence: >-
      How can organizations leverage user feedback to enhance content provenance
      and risk management efforts?
    sentences:
      - >-
        tested, there will always be situations for which the system fails. The
        American public deserves protection via human 

        review against these outlying or unexpected scenarios. In the case of
        time-critical systems, the public should not have 

        to wait—immediate human consideration and fallback should be available.
        In many time-critical systems, such a 

        remedy is already immediately available, such as a building manager who
        can open a door in the case an automated 

        card access system fails. 

        In the criminal justice system, employment, education, healthcare, and
        other sensitive domains, automated systems 

        are used for many purposes, from pre-trial risk assessments and parole
        decisions to technologies that help doctors 

        diagnose disease. Absent appropriate safeguards, these technologies can
        lead to unfair, inaccurate, or dangerous 

        outcomes. These sensitive domains require extra protections. It is
        critically important that there is extensive human 

        oversight in such settings.
      - >-
        enable organizations to maximize the utility of provenance data and risk
        management eﬀorts. 

        A.1.7. Enhancing Content Provenance through Structured Public Feedback 

        While indirect feedback methods such as automated error collection
        systems are useful, they often lack 

        the context and depth that direct input from end users can provide.
        Organizations can leverage feedback 

        approaches described in the Pre-Deployment Testing section to capture
        input from external sources such 

        as through AI red-teaming.  

        Integrating pre- and post-deployment external feedback into the
        monitoring process for GAI models and 

        corresponding applications can help enhance awareness of performance
        changes and mitigate potential 

        risks and harms from outputs. There are many ways to capture and make
        use of user feedback – before 

        and after GAI systems and digital content transparency approaches are
        deployed – to gain insights about
      - >-
        A.1. Governance 

        A.1.1. Overview 

        Like any other technology system, governance principles and techniques
        can be used to manage risks 

        related to generative AI models, capabilities, and applications.
        Organizations may choose to apply their 

        existing risk tiering to GAI systems, or they may opt to revise or
        update AI system risk levels to address 

        these unique GAI risks. This section describes how organizational
        governance regimes may be re-

        evaluated and adjusted for GAI contexts. It also addresses third-party
        considerations for governing across 

        the AI value chain.  

        A.1.2. Organizational Governance 

        GAI opportunities, risks and long-term performance characteristics are
        typically less well-understood 

        than non-generative AI tools and may be perceived and acted upon by
        humans in ways that vary greatly. 

        Accordingly, GAI may call for diﬀerent levels of oversight from AI
        Actors or diﬀerent human-AI
  - source_sentence: >-
      What should be ensured for users who have trouble with the automated
      system?
    sentences:
      - >-
        32 

        MEASURE 2.6: The AI system is evaluated regularly for safety risks – as
        identiﬁed in the MAP function. The AI system to be 

        deployed is demonstrated to be safe, its residual negative risk does not
        exceed the risk tolerance, and it can fail safely, particularly if 

        made to operate beyond its knowledge limits. Safety metrics reﬂect
        system reliability and robustness, real-time monitoring, and 

        response times for AI system failures. 

        Action ID 

        Suggested Action 

        GAI Risks 

        MS-2.6-001 

        Assess adverse impacts, including health and wellbeing impacts for value
        chain 

        or other AI Actors that are exposed to sexually explicit, oﬀensive, or
        violent 

        information during GAI training and maintenance. 

        Human-AI Conﬁguration; Obscene, 

        Degrading, and/or Abusive 

        Content; Value Chain and 

        Component Integration; 

        Dangerous, Violent, or Hateful 

        Content 

        MS-2.6-002 

        Assess existence or levels of harmful bias, intellectual property
        infringement,
      - >-
        APPENDIX

        Systems that impact the safety of communities such as automated traffic
        control systems, elec 

        -ctrical grid controls, smart city technologies, and industrial
        emissions and environmental

        impact control algorithms; and

        Systems related to access to benefits or services or assignment of
        penalties such as systems that

        support decision-makers who adjudicate benefits such as collating or
        analyzing information or

        matching records, systems which similarly assist in the adjudication of
        administrative or criminal

        penalties, fraud detection algorithms, services or benefits access
        control algorithms, biometric

        systems used as access control, and systems which make benefits or
        services related decisions on a

        fully or partially autonomous basis (such as a determination to revoke
        benefits).

        54
      - >-
        meaningfully impact rights, opportunities, or access should have greater
        availability (e.g., staffing) and over

        sight of human consideration and fallback mechanisms. 

        Accessible. Mechanisms for human consideration and fallback, whether
        in-person, on paper, by phone, or 

        otherwise provided, should be easy to find and use. These mechanisms
        should be tested to ensure that users 

        who have trouble with the automated system are able to use human
        consideration and fallback, with the under

        standing that it may be these users who are most likely to need the
        human assistance. Similarly, it should be 

        tested to ensure that users with disabilities are able to find and use
        human consideration and fallback and also 

        request reasonable accommodations or modifications. 

        Convenient. Mechanisms for human consideration and fallback should not
        be unreasonably burdensome as 

        compared to the automated system’s equivalent. 

        49
  - source_sentence: >-
      What must lenders provide to consumers who are denied credit under the
      Fair Credit Reporting Act?
    sentences:
      - >-
        8 

        Trustworthy AI Characteristics: Accountable and Transparent, Privacy
        Enhanced, Safe, Secure and 

        Resilient 

        2.5. Environmental Impacts 

        Training, maintaining, and operating (running inference on) GAI systems
        are resource-intensive activities, 

        with potentially large energy and environmental footprints. Energy and
        carbon emissions vary based on 

        what is being done with the GAI model (i.e., pre-training, ﬁne-tuning,
        inference), the modality of the 

        content, hardware used, and type of task or application. 

        Current estimates suggest that training a single transformer LLM can
        emit as much carbon as 300 round-

        trip ﬂights between San Francisco and New York. In a study comparing
        energy consumption and carbon 

        emissions for LLM inference, generative tasks (e.g., text summarization)
        were found to be more energy- 

        and carbon-intensive than discriminative or non-generative tasks (e.g.,
        text classiﬁcation).
      - >-
        that consumers who are denied credit receive "adverse action" notices.
        Anyone who relies on the information in a 

        credit report to deny a consumer credit must, under the Fair Credit
        Reporting Act, provide an "adverse action" 

        notice to the consumer, which includes "notice of the reasons a creditor
        took adverse action on the application 

        or on an existing credit account."90 In addition, under the risk-based
        pricing rule,91 lenders must either inform 

        borrowers of their credit score, or else tell consumers when "they are
        getting worse terms because of 

        information in their credit report." The CFPB has also asserted that
        "[t]he law gives every applicant the right to 

        a specific explanation if their application for credit was denied, and
        that right is not diminished simply because 

        a company uses a complex algorithm that it doesn't understand."92 Such
        explanations illustrate a shared value 

        that certain decisions need to be explained.
      - >-
        measures to prevent, ﬂag, or take other action in response to outputs
        that 

        reproduce particular training data (e.g., plagiarized, trademarked,
        patented, 

        licensed content or trade secret material). 

        Intellectual Property; CBRN 

        Information or Capabilities
model-index:
  - name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: Unknown
          type: unknown
        metrics:
          - type: cosine_accuracy@1
            value: 0.881578947368421
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9671052631578947
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9868421052631579
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 1
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.881578947368421
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.3223684210526316
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.19736842105263155
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09999999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.881578947368421
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.9671052631578947
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9868421052631579
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 1
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.9460063349721777
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.9282346491228071
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.9282346491228068
            name: Cosine Map@100
          - type: dot_accuracy@1
            value: 0.881578947368421
            name: Dot Accuracy@1
          - type: dot_accuracy@3
            value: 0.9671052631578947
            name: Dot Accuracy@3
          - type: dot_accuracy@5
            value: 0.9868421052631579
            name: Dot Accuracy@5
          - type: dot_accuracy@10
            value: 1
            name: Dot Accuracy@10
          - type: dot_precision@1
            value: 0.881578947368421
            name: Dot Precision@1
          - type: dot_precision@3
            value: 0.3223684210526316
            name: Dot Precision@3
          - type: dot_precision@5
            value: 0.19736842105263155
            name: Dot Precision@5
          - type: dot_precision@10
            value: 0.09999999999999999
            name: Dot Precision@10
          - type: dot_recall@1
            value: 0.881578947368421
            name: Dot Recall@1
          - type: dot_recall@3
            value: 0.9671052631578947
            name: Dot Recall@3
          - type: dot_recall@5
            value: 0.9868421052631579
            name: Dot Recall@5
          - type: dot_recall@10
            value: 1
            name: Dot Recall@10
          - type: dot_ndcg@10
            value: 0.9460063349721777
            name: Dot Ndcg@10
          - type: dot_mrr@10
            value: 0.9282346491228071
            name: Dot Mrr@10
          - type: dot_map@100
            value: 0.9282346491228068
            name: Dot Map@100

SentenceTransformer based on Snowflake/snowflake-arctic-embed-m

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: Snowflake/snowflake-arctic-embed-m
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("jet-taekyo/snowflake_finetuned_recursive")
# Run inference
sentences = [
    'What must lenders provide to consumers who are denied credit under the Fair Credit Reporting Act?',
    'that consumers who are denied credit receive "adverse action" notices. Anyone who relies on the information in a \ncredit report to deny a consumer credit must, under the Fair Credit Reporting Act, provide an "adverse action" \nnotice to the consumer, which includes "notice of the reasons a creditor took adverse action on the application \nor on an existing credit account."90 In addition, under the risk-based pricing rule,91 lenders must either inform \nborrowers of their credit score, or else tell consumers when "they are getting worse terms because of \ninformation in their credit report." The CFPB has also asserted that "[t]he law gives every applicant the right to \na specific explanation if their application for credit was denied, and that right is not diminished simply because \na company uses a complex algorithm that it doesn\'t understand."92 Such explanations illustrate a shared value \nthat certain decisions need to be explained.',
    'measures to prevent, ﬂag, or take other action in response to outputs that \nreproduce particular training data (e.g., plagiarized, trademarked, patented, \nlicensed content or trade secret material). \nIntellectual Property; CBRN \nInformation or Capabilities',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.8816
cosine_accuracy@3	0.9671
cosine_accuracy@5	0.9868
cosine_accuracy@10	1.0
cosine_precision@1	0.8816
cosine_precision@3	0.3224
cosine_precision@5	0.1974
cosine_precision@10	0.1
cosine_recall@1	0.8816
cosine_recall@3	0.9671
cosine_recall@5	0.9868
cosine_recall@10	1.0
cosine_ndcg@10	0.946
cosine_mrr@10	0.9282
cosine_map@100	0.9282
dot_accuracy@1	0.8816
dot_accuracy@3	0.9671
dot_accuracy@5	0.9868
dot_accuracy@10	1.0
dot_precision@1	0.8816
dot_precision@3	0.3224
dot_precision@5	0.1974
dot_precision@10	0.1
dot_recall@1	0.8816
dot_recall@3	0.9671
dot_recall@5	0.9868
dot_recall@10	1.0
dot_ndcg@10	0.946
dot_mrr@10	0.9282
dot_map@100	0.9282

Training Details

Training Dataset

Unnamed Dataset

Size: 714 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 714 samples:
sentence_0 sentence_1
type string string
details
min: 11 tokens
mean: 18.46 tokens
max: 32 tokens

min: 21 tokens
mean: 175.32 tokens
max: 512 tokens

	sentence_0	sentence_1
type	string	string
details	min: 11 tokens mean: 18.46 tokens max: 32 tokens	min: 21 tokens mean: 175.32 tokens max: 512 tokens

Samples:

sentence_0	sentence_1
`What is the purpose of conducting adversarial testing in the context of GAI risks?`	Human-AI Conﬁguration; Information Integrity; Harmful Bias and Homogenization AI Actor Tasks: AI Deployment, Aﬀected Individuals and Communities, End-Users, Operation and Monitoring, TEVV MEASURE 4.2: Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are informed by input from domain experts and relevant AI Actors to validate whether the system is performing consistently as intended. Results are documented. Action ID Suggested Action GAI Risks MS-4.2-001 Conduct adversarial testing at a regular cadence to map and measure GAI risks, including tests to address attempts to deceive or manipulate the application of provenance techniques or other misuses. Identify vulnerabilities and understand potential misuse scenarios and unintended outputs. Information Integrity; Information Security MS-4.2-002 Evaluate GAI system performance in real-world scenarios to observe its
`How are measurement results regarding AI system trustworthiness documented and validated?`	Human-AI Conﬁguration; Information Integrity; Harmful Bias and Homogenization AI Actor Tasks: AI Deployment, Aﬀected Individuals and Communities, End-Users, Operation and Monitoring, TEVV MEASURE 4.2: Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are informed by input from domain experts and relevant AI Actors to validate whether the system is performing consistently as intended. Results are documented. Action ID Suggested Action GAI Risks MS-4.2-001 Conduct adversarial testing at a regular cadence to map and measure GAI risks, including tests to address attempts to deceive or manipulate the application of provenance techniques or other misuses. Identify vulnerabilities and understand potential misuse scenarios and unintended outputs. Information Integrity; Information Security MS-4.2-002 Evaluate GAI system performance in real-world scenarios to observe its
`What types of data provenance information are included in the GAI system inventory entries?`	following items in GAI system inventory entries: Data provenance information (e.g., source, signatures, versioning, watermarks); Known issues reported from internal bug tracking or external information sharing resources (e.g., AI incident database, AVID, CVE, NVD, or OECD AI incident monitor); Human oversight roles and responsibilities; Special rights and considerations for intellectual property, licensed works, or personal, privileged, proprietary or sensitive data; Underlying foundation models, versions of underlying models, and access modes. Data Privacy; Human-AI Conﬁguration; Information Integrity; Intellectual Property; Value Chain and Component Integration AI Actor Tasks: Governance and Oversight

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 20
per_device_eval_batch_size: 20
num_train_epochs: 5
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 20
per_device_eval_batch_size: 20
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 5
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
eval_use_gather_object: False
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_map@100
1.0	36	0.9145
1.3889	50	0.9256
2.0	72	0.9246
2.7778	100	0.9282

Framework Versions

Python: 3.11.9
Sentence Transformers: 3.1.0
Transformers: 4.44.2
PyTorch: 2.4.1+cu121
Accelerate: 0.34.2
Datasets: 3.0.0
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}