metadata

language:
  - en
license: apache-2.0
library_name: sentence-transformers
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:3853
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-base-en-v1.5
datasets: []
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
widget:
  - source_sentence: |-

      UDPv6TransportDescriptor
      ------------------------
    sentences:
      - >-
        What is the primary purpose of the "Status" objects in the context of
        entities?
      - >-
        What is the main concept that this piece of code demonstrates, and how
        do the provided topology and QoS policy settings relate to it?
      - >-
        What is the primary characteristic of UDP transport in terms of
        connection establishment?
  - source_sentence: |-
      With a fragment size of 64 kB, the Publisher has to send about 1100
      fragments to send the whole file. A possible configuration for this
      scenario could be:
    sentences:
      - >-
        What is the likely reason why the Publisher needs to use "RELIABLE_
        RELIABILITY_QOS" in this scenario?
      - >-
        What is the effect of defining a custom Metatraffic Unicast Locators on
        the behavior of a DomainParticipant?
      - >-
        What is the primary function of the transport layer in DDS, as described
        in the provided context?
  - source_sentence: >-
      +------------------------------------+---------------------------------------------------+------------+

      | QosPolicy class                    |
      Accessor/Mutator                                  | Mutable    |

      |====================================|===================================================|============|

      | RTPSEndpointQos                    |
      "endpoint()"                                      | No         |

      +------------------------------------+---------------------------------------------------+------------+
    sentences:
      - >-
        What is the effect of setting "ON" as the DataSharingKind in the context
        of data-sharing delivery?
      - >-
        What is the purpose of the RTPSEndpointQos class in the context of
        DataWriter QoS policies?
      - >-
        What is the primary purpose of the RTPSEndpointQos policy in a DDS (Data
        Distribution Service) system?
  - source_sentence: |-
      Note:  When "non_blocking_send" is set to "true", send operations will
        return immediately if the send buffer might get full, but no error
        will be returned to the upper layer. This means that the application
        will behave as if the packet is sent and lost.When set to "false",
        send operations will block until the network buffer has space for
        the packet.
    sentences:
      - What happens when "non_blocking_send" is set to "true" in TCP transport?
      - >-
        What is the purpose of the "<default_external_unicast_locators>" element
        in the RTPS configuration?
      - >-
        What is the purpose of the "<enabled>" value in the DisablePositiveAcks
        QoS policy?
  - source_sentence: |-
      After calling the "DataReader::read()" or "DataReader::take()"
      operations, accessing the data on the returned sequences is quite
      easy. The sequences API provides a **length()** operation returning
      the number of elements in the collections. The application code just
      needs to check this value and use the **[]** operator to access the
      corresponding elements. Elements on the DDS data sequence should only
      be accessed when the corresponding element on the SampleInfo sequence
      indicate that valid data is present. When using Data Sharing, it is
      also important to check that the sample is valid (i.e, not replaced,
      refer to DataReader and DataWriter history coupling for further
      information in this regard).
    sentences:
      - >-
        What is the primary method described in the text for accessing data on
        returned sequences after calling "DataReader::read()" or
        "DataReader::take()" operations?
      - >-
        What is the primary advantage of using Shared Memory Transport (SHM)
        compared to other network transports like UDP/TCP?
      - >-
        What are the steps to install Fast DDS library, Python bindings, and Gen
        generation tool from sources in a Linux environment?
pipeline_tag: sentence-similarity
model-index:
  - name: Fine tuning poc1-30e
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.34265734265734266
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5291375291375291
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5757575757575758
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6643356643356644
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.34265734265734266
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.17637917637917636
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.11515151515151513
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.06643356643356643
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.34265734265734266
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5291375291375291
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.5757575757575758
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.6643356643356644
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4999219586168879
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.44783734783734785
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.45732757969458965
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.3333333333333333
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5314685314685315
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5804195804195804
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.655011655011655
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3333333333333333
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.17715617715617715
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.11608391608391608
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.0655011655011655
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.3333333333333333
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5314685314685315
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.5804195804195804
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.655011655011655
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4931410715247713
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.44150664150664165
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4520914166409126
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.331002331002331
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5384615384615384
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5734265734265734
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.662004662004662
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.331002331002331
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1794871794871795
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.11468531468531468
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.0662004662004662
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.331002331002331
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5384615384615384
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.5734265734265734
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.662004662004662
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4946456648216315
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4414687164687165
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4517532849343265
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.32867132867132864
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5291375291375291
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.578088578088578
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6643356643356644
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.32867132867132864
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.17637917637917636
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.11561771561771561
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.06643356643356643
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.32867132867132864
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5291375291375291
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.578088578088578
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.6643356643356644
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.491729303526411
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4370564620564619
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4465064100234966
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.317016317016317
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5058275058275058
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5734265734265734
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.655011655011655
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.317016317016317
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1686091686091686
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.11468531468531468
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.06550116550116548
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.317016317016317
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5058275058275058
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.5734265734265734
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.655011655011655
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4805357725353263
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.42515355015355016
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.43416870212536746
            name: Cosine Map@100

Fine tuning poc1-30e

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-base-en-v1.5
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("cferreiragonz/bge-base-fastdds-questions-30-epochs")
# Run inference
sentences = [
    'After calling the "DataReader::read()" or "DataReader::take()"\noperations, accessing the data on the returned sequences is quite\neasy. The sequences API provides a **length()** operation returning\nthe number of elements in the collections. The application code just\nneeds to check this value and use the **[]** operator to access the\ncorresponding elements. Elements on the DDS data sequence should only\nbe accessed when the corresponding element on the SampleInfo sequence\nindicate that valid data is present. When using Data Sharing, it is\nalso important to check that the sample is valid (i.e, not replaced,\nrefer to DataReader and DataWriter history coupling for further\ninformation in this regard).',
    'What is the primary method described in the text for accessing data on returned sequences after calling "DataReader::read()" or "DataReader::take()" operations?',
    'What are the steps to install Fast DDS library, Python bindings, and Gen generation tool from sources in a Linux environment?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: dim_768
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.3427
cosine_accuracy@3	0.5291
cosine_accuracy@5	0.5758
cosine_accuracy@10	0.6643
cosine_precision@1	0.3427
cosine_precision@3	0.1764
cosine_precision@5	0.1152
cosine_precision@10	0.0664
cosine_recall@1	0.3427
cosine_recall@3	0.5291
cosine_recall@5	0.5758
cosine_recall@10	0.6643
cosine_ndcg@10	0.4999
cosine_mrr@10	0.4478
cosine_map@100	0.4573

Information Retrieval

Dataset: dim_512
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.3333
cosine_accuracy@3	0.5315
cosine_accuracy@5	0.5804
cosine_accuracy@10	0.655
cosine_precision@1	0.3333
cosine_precision@3	0.1772
cosine_precision@5	0.1161
cosine_precision@10	0.0655
cosine_recall@1	0.3333
cosine_recall@3	0.5315
cosine_recall@5	0.5804
cosine_recall@10	0.655
cosine_ndcg@10	0.4931
cosine_mrr@10	0.4415
cosine_map@100	0.4521

Information Retrieval

Dataset: dim_256
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.331
cosine_accuracy@3	0.5385
cosine_accuracy@5	0.5734
cosine_accuracy@10	0.662
cosine_precision@1	0.331
cosine_precision@3	0.1795
cosine_precision@5	0.1147
cosine_precision@10	0.0662
cosine_recall@1	0.331
cosine_recall@3	0.5385
cosine_recall@5	0.5734
cosine_recall@10	0.662
cosine_ndcg@10	0.4946
cosine_mrr@10	0.4415
cosine_map@100	0.4518

Information Retrieval

Dataset: dim_128
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.3287
cosine_accuracy@3	0.5291
cosine_accuracy@5	0.5781
cosine_accuracy@10	0.6643
cosine_precision@1	0.3287
cosine_precision@3	0.1764
cosine_precision@5	0.1156
cosine_precision@10	0.0664
cosine_recall@1	0.3287
cosine_recall@3	0.5291
cosine_recall@5	0.5781
cosine_recall@10	0.6643
cosine_ndcg@10	0.4917
cosine_mrr@10	0.4371
cosine_map@100	0.4465

Information Retrieval

Dataset: dim_64
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.317
cosine_accuracy@3	0.5058
cosine_accuracy@5	0.5734
cosine_accuracy@10	0.655
cosine_precision@1	0.317
cosine_precision@3	0.1686
cosine_precision@5	0.1147
cosine_precision@10	0.0655
cosine_recall@1	0.317
cosine_recall@3	0.5058
cosine_recall@5	0.5734
cosine_recall@10	0.655
cosine_ndcg@10	0.4805
cosine_mrr@10	0.4252
cosine_map@100	0.4342

Training Details

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
gradient_accumulation_steps: 16
learning_rate: 2e-05
num_train_epochs: 30
lr_scheduler_type: cosine
warmup_ratio: 0.1
fp16: True
tf32: False
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 16
eval_accumulation_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 30
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: False
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_128_cosine_map@100	dim_256_cosine_map@100	dim_512_cosine_map@100	dim_64_cosine_map@100	dim_768_cosine_map@100
0.6639	10	5.6138	-	-	-	-	-
0.9959	15	-	0.3594	0.3735	0.3723	0.3161	0.3807
1.3278	20	4.9173	-	-	-	-	-
1.9917	30	3.7581	0.3874	0.4014	0.4026	0.3729	0.4032
2.6556	40	3.0018	-	-	-	-	-
2.9876	45	-	0.4031	0.4200	0.4212	0.3858	0.4223
3.3195	50	2.5035	-	-	-	-	-
3.9834	60	1.9031	0.4187	0.4303	0.4178	0.3958	0.4291
4.6473	70	1.474	-	-	-	-	-
4.9793	75	-	0.4293	0.4332	0.4318	0.4172	0.4401
5.3112	80	1.2801	-	-	-	-	-
5.9751	90	0.9577	0.4397	0.4382	0.4444	0.4275	0.4518
6.6390	100	0.7539	-	-	-	-	-
6.9710	105	-	0.4434	0.4414	0.4496	0.4262	0.4466
7.3029	110	0.694	-	-	-	-	-
7.9668	120	0.5147	0.4423	0.4488	0.4507	0.4358	0.4495
8.6307	130	0.4589	-	-	-	-	-
8.9627	135	-	0.4488	0.4575	0.4544	0.4407	0.4493
9.2946	140	0.3843	-	-	-	-	-
9.9585	150	0.3506	0.4521	0.4465	0.4559	0.4420	0.4485
10.6224	160	0.2723	-	-	-	-	-
10.9544	165	-	0.4497	0.4435	0.4499	0.4304	0.4453
11.2863	170	0.2555	-	-	-	-	-
11.9502	180	0.2077	0.4448	0.4472	0.4468	0.4287	0.4453
12.6141	190	0.1894	-	-	-	-	-
12.9461	195	-	0.4516	0.4463	0.4566	0.4336	0.452
13.2780	200	0.1725	-	-	-	-	-
13.9419	210	0.1395	0.4528	0.4520	0.4561	0.4333	0.4534
14.6058	220	0.155	-	-	-	-	-
14.9378	225	-	0.4461	0.4491	0.4527	0.4369	0.4517
15.2697	230	0.132	-	-	-	-	-
15.9336	240	0.1148	-	-	-	-	-
16.0	241	-	0.4482	0.4537	0.4540	0.4303	0.4538
16.5975	250	0.1061	-	-	-	-	-
16.9959	256	-	0.4464	0.4538	0.4551	0.4294	0.4577
17.2614	260	0.0961	-	-	-	-	-
17.9253	270	0.087	-	-	-	-	-
17.9917	271	-	0.4485	0.4483	0.4495	0.4326	0.4568
18.5892	280	0.1009	-	-	-	-	-
18.9876	286	-	0.4483	0.4517	0.4545	0.4396	0.4565
19.2531	290	0.0854	-	-	-	-	-
19.9170	300	0.073	-	-	-	-	-
19.9834	301	-	0.4473	0.4502	0.4521	0.4349	0.4548
20.5809	310	0.0726	-	-	-	-	-
20.9793	316	-	0.4466	0.4525	0.4538	0.4341	0.4583
21.2448	320	0.0747	-	-	-	-	-
21.9087	330	0.0621	-	-	-	-	-
21.9751	331	-	0.4441	0.4537	0.4534	0.4388	0.4564
22.5726	340	0.0682	-	-	-	-	-
22.9710	346	-	0.4454	0.4529	0.4544	0.4385	0.4589
23.2365	350	0.0612	-	-	-	-	-
23.9004	360	0.0546	-	-	-	-	-
23.9668	361	-	0.4464	0.4494	0.4551	0.4381	0.4567
24.5643	370	0.0599	-	-	-	-	-
24.9627	376	-	0.4465	0.4506	0.4553	0.4363	0.4567
25.2282	380	0.0591	-	-	-	-	-
25.8921	390	0.0562	-	-	-	-	-
25.9585	391	-	0.4454	0.4515	0.4532	0.4343	0.4575
26.5560	400	0.0623	-	-	-	-	-
26.9544	406	-	0.4452	0.4531	0.4544	0.4342	0.4573
27.2199	410	0.061	-	-	-	-	-
27.8838	420	0.053	-	-	-	-	-
27.9502	421	-	0.4454	0.4514	0.4533	0.4330	0.4573
28.5477	430	0.0564	-	-	-	-	-
28.9461	436	-	0.4465	0.4516	0.4533	0.4338	0.4562
29.2116	440	0.056	-	-	-	-	-
29.8755	450	0.0586	0.4465	0.4518	0.4521	0.4342	0.4573

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.13
Sentence Transformers: 3.0.1
Transformers: 4.41.2
PyTorch: 2.1.2
Accelerate: 0.30.1
Datasets: 2.19.1
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}