README.md

metadata

base_model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
datasets: []
language: []
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:64000
  - loss:DenoisingAutoEncoderLoss
widget:
  - source_sentence: 𑀟चन𑀙𑀢𑀟 𑀞च𑀪च𑀠च 𑀫𑁣प𑁣 𑀞न𑀠च 𑀞𑁣𑀱च ब𑀢𑀪𑀠च𑀯
    sentences:
      - '  णच ब𑀢𑀪𑀠च पच𑀪𑁦 𑀣च 𑀠च𑀫च𑀢𑀲𑀢णच𑀪𑀳च 𑀣च झच𑀟𑁦𑀟𑀳च ञचणच𑀦 𑀞च𑀠च𑀪 णच𑀣𑀣च 𑀠च𑀫च𑀢𑀲𑀢𑀟𑀳च णच ढच𑀪 𑀢णचल𑀢𑀯'
      - ' 𑀣च𑀟बच𑀟𑁦 𑀣च 𑀟चन𑀙𑀢𑀟 𑀠𑁣पच𑀪𑀦 पच𑀟च 𑀢णच 𑀤च𑀠च ढचढढच 𑀞𑁣 𑀞च𑀪च𑀠च 𑀢𑀣च𑀟 च𑀞च 𑀞𑀱चपच𑀟पच 𑀣च 𑀠𑁣पच𑀪 𑀣चन𑀞च𑀪 𑀫𑁣प𑁣 𑀣च 𑀳नख𑀦 𑀞न𑀠च णच 𑀲𑀢 𑀟च 𑀞𑁣𑀱च ब𑀢𑀪𑀠च𑀯'
      - >-
        पच𑀪𑁦𑀠𑀢 णच ढनबच 𑀱च झन𑀟ब𑀢णच𑀪 झ𑀱चलल𑁣𑀟 झच𑀲च पच ञचल𑀢ढ𑀢𑀟
        झच𑀳च𑀪 𑀢𑀪च𑀟 च बच𑀳च𑀪 पन𑀪𑀞𑀢णणच 𑀞न𑀠च णच त𑀢 𑀱च झन𑀟ब𑀢णच𑀪
        𑀞𑀱चललचण𑁦 थ𑀯
  - source_sentence: णच𑀟च बचढच 𑀣च लन𑀪च 𑀣च 𑀣च पच 𑀲𑀢 𑀣च
    sentences:
      - >-
        𑀘𑁣𑀫𑀟 𑀠𑀢त𑀫च𑁦ल 𑁣ब𑀢𑀣𑀢 𑀝च𑀟 𑀫च𑀢𑀲𑁦𑀳𑀫𑀢 𑀪च𑀟च𑀪 𑀗 बच
        𑀱चपच𑀟 𑀣𑀢𑀳च𑀠ढच𑀦 𑀭थ𑀖थ𑀮𑀯
      - ' 𑀱च𑀟𑀟च𑀟 णच𑀟च पच𑀢𑀠च𑀞च 𑀱च झ𑀱च𑀪च𑀪𑀪न𑀟 𑀫𑀪 𑀳न त𑀢 बचढच 𑀣च लन𑀪च 𑀣च 𑀣न𑀞 ढनञचञञ𑁦𑀟 चणणन𑀞च𑀟𑀳न 𑀣च 𑀠च𑀳न 𑀟𑁦𑀠च पच 𑀫च𑀟णच𑀪 𑀣च पच 𑀲𑀢 𑀳चन𑀪𑀢 𑀣च 𑀳चनझ𑀢 𑀲𑀢ण𑁦 𑀣च 𑀣च𑀯'
      - ' च 𑀞च𑀪𑀞च𑀳𑀫𑀢𑀟 𑀣𑁣𑀞च𑀪𑀦 𑀠च𑀘चल𑀢𑀳च𑀪 लचनण𑁣ण𑀢𑀟 𑀢𑀟𑀣𑀢णच 𑀢पच त𑁦 ढचढढच𑀪 𑀫न𑀞न𑀠च𑀪 𑀞नलच 𑀣च 𑀫च𑀪𑀞𑁣𑀞𑀢𑀟 𑀳𑀫च𑀪𑀢𑀙च च 𑀢𑀟𑀣𑀢णच 𑀣च 𑀞न𑀠च पचढढचपच𑀪 𑀣च ढ𑀢𑀟 𑀣𑁣𑀞च 𑀣च 𑀞𑀢णचण𑁦 𑀞च𑀙𑀢𑀣𑁣𑀘𑀢𑀟 𑀞𑀱च𑀪च𑀪𑀪न पच 𑀫च𑀟णच𑀪 𑀞𑀱च𑀪च𑀪𑀪न𑀟 लचनणच च 𑀞च𑀳च𑀪𑀯'
  - source_sentence: 𑀣नढच ढढत𑀕 𑀠च𑀠च𑀪 चलचप𑁣न𑀠𑀢
    sentences:
      - >-
        𑀣नढच 𑀞न𑀠च  𑀣𑁦𑀟𑀞ष𑀣𑁦𑀟𑀞𑀠च𑀟च𑀤च𑀪पच  ढढत𑀕 𑀠च𑀠च𑀪 𑀞च𑀳𑀳𑁦ण
        चलचप𑁣न𑀠𑀢 𑀯
      - '  च𑀟 𑀲च𑀪च 𑀳च𑀠च𑀪𑀱च 𑀞न𑀠च 𑀣चबच ढचणच च𑀟 𑀲च𑀣च𑀣च चणणन𑀞च𑀟 बच 𑀳चन𑀪च𑀟 𑀢णचलच𑀢 𑀟च 𑀟च𑀘𑁦𑀪𑀢णच 𑀠च𑀳न णच𑀪च𑀯'
      - '    𑀫च𑁥च𑀞च च𑀤चढपच𑀪𑀱च णच𑀟च 𑀣च 𑀱च𑀫चलच 𑀠न𑀳च𑀠𑀠च𑀟 च त𑀢𑀞𑀢𑀟 चणणन𑀞च𑀟 णचझ𑀢 𑀣च पच𑀱चबच𑀪𑀯'
  - source_sentence: च𑀟
    sentences:
      - >-
        𑀠नपन𑀱च च 𑀪च𑀟च𑀪 र बच 𑀱चपच𑀟 𑀠चणन𑀟 ठ𑀧𑀧ठ𑀦 च𑀞न 𑀟च त𑀢𑀞𑀢𑀟
        𑀲च𑀳𑀢𑀟𑀘𑁣𑀘𑀢 𑀬𑀧 𑀣च 𑀞𑁦 त𑀢𑀞𑀢𑀟 𑀱च𑀟𑀢 𑀘𑀢𑀪ब𑀢𑀟 𑀣च णच ण𑀢
        𑀫चप𑀳च𑀪𑀢𑀟 𑀠𑀢𑀟पन𑀟च 𑀞चञच𑀟 ढचणच𑀟 पच𑀳𑀫𑀢𑀟𑀳च च 𑀞च𑀟𑁣𑀯
      - '  च𑀟 ण𑀢 𑀢𑀠च𑀟𑀢𑀟 𑀳𑀯'
      - ' 𑀲च𑀫च𑀣 णच 𑀞च𑀠𑀠चलच 𑀞च𑀞च𑀪 ठ𑀧𑀭ठट𑀭𑀰 𑀣च 𑀞𑀱चललचण𑁦 𑀭𑀧 𑀠च𑀳न ढच𑀟 𑀳𑀫च𑀙च𑀱च च 𑀱च𑀳च𑀟𑀟𑀢 ठ𑁢 च 𑀣न𑀞 बच𑀳च𑀯'
  - source_sentence: >-
      ब𑀫𑁣𑀳प 𑀢𑀢 𑀳𑀫𑀢𑀟𑁦 𑀠च𑀲𑀢 𑀠च𑀫𑀢𑀠𑀠च𑀟त𑀢𑀦 पच𑀢𑀠च𑀞𑁣𑀟 𑀣च
      𑀲च𑀳चलनललन𑀞च णच𑀟च ढच 𑀠च𑀤चन𑀟च त𑀢𑀞𑀢𑀟 𑀫च𑀟𑀞चल𑀢 णचण𑀢𑀟
    sentences:
      - >-
        च𑀠𑀢𑀟पचतत𑀢णच च त𑀢𑀞𑀢𑀟 ब𑀫𑁣𑀳प 𑀳𑁦𑀪𑀢𑁦𑀳 𑀢𑀢 𑀳𑀫𑀢𑀟𑁦
        𑀠च𑀲𑀢 𑀠च𑀫𑀢𑀠𑀠च𑀟त𑀢𑀦 पच𑀪𑁦 𑀣च ञ𑀢𑀠ढ𑀢𑀟 𑀢𑀟बच𑀟पचपपन𑀟
        प𑀳च𑀪𑀢𑀟 पच𑀢𑀠च𑀞𑁣𑀟 𑀣𑀢𑀪𑁦ढच 𑀣च 𑀲च𑀳चलनललन𑀞च 𑀟च च𑀠𑀢𑀟त𑀢𑀦
        णच𑀟च ढच 𑀠च𑀤चन𑀟च त𑀢𑀞𑀢𑀟 𑀞𑀱च𑀟त𑀢णच𑀪 𑀫च𑀟𑀞चल𑀢 णचण𑀢𑀟
        पच𑀲𑀢णच𑀪𑀳न𑀯
      - >-
        प𑁣ध𑀳ण ध𑀫𑀢𑀪𑀢 𑀝च𑀟 𑀫च𑀢𑀲𑁦 𑀳𑀫𑀢 च 𑀪च𑀟च𑀪 𑀭𑀭 बच 𑀱चपच𑀟
        चबन𑀳पच 𑀭थ𑀗𑀧𑀮 ञच𑀟 𑀱च𑀳च𑀟 ढच𑀣𑀠𑀢𑀟प𑁣𑀟 ञच𑀟 𑀤च𑀠ढ𑀢च 𑀟𑁦𑀯
      - >-
        पचबबच𑀲च𑀣𑀢 𑀠चप𑀳नबन𑀟𑀢𑀟 𑀠नपच𑀟𑁦 𑀟𑁦 च 𑀳च𑀳𑀫𑁦𑀟 च𑀪ल𑀢प
        𑀣च𑀞𑁦 णच𑀟𑀞𑀢𑀟 चबच𑀣𑁦𑀤 च च𑀪𑁦𑀱च पच प𑀳च𑀞𑀢णच𑀪 𑀟𑀢𑀘च𑀪𑀯

SentenceTransformer based on sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a sentence-transformers model finetuned from sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("T-Blue/tsdae_pro_MiniLM_L12_2")
# Run inference
sentences = [
    'ब𑀫𑁣𑀳प 𑀢𑀢 𑀳𑀫𑀢𑀟𑁦 𑀠च𑀲𑀢 𑀠च𑀫𑀢𑀠𑀠च𑀟त𑀢𑀦 पच𑀢𑀠च𑀞𑁣𑀟 𑀣च 𑀲च𑀳चलनललन𑀞च णच𑀟च ढच 𑀠च𑀤चन𑀟च त𑀢𑀞𑀢𑀟 𑀫च𑀟𑀞चल𑀢 णचण𑀢𑀟',
    'च𑀠𑀢𑀟पचतत𑀢णच च त𑀢𑀞𑀢𑀟 ब𑀫𑁣𑀳प 𑀳𑁦𑀪𑀢𑁦𑀳 𑀢𑀢 𑀳𑀫𑀢𑀟𑁦 𑀠च𑀲𑀢 𑀠च𑀫𑀢𑀠𑀠च𑀟त𑀢𑀦 पच𑀪𑁦 𑀣च ञ𑀢𑀠ढ𑀢𑀟 𑀢𑀟बच𑀟पचपपन𑀟 प𑀳च𑀪𑀢𑀟 पच𑀢𑀠च𑀞𑁣𑀟 𑀣𑀢𑀪𑁦ढच 𑀣च 𑀲च𑀳चलनललन𑀞च 𑀟च च𑀠𑀢𑀟त𑀢𑀦 णच𑀟च ढच 𑀠च𑀤चन𑀟च त𑀢𑀞𑀢𑀟 𑀞𑀱च𑀟त𑀢णच𑀪 𑀫च𑀟𑀞चल𑀢 णचण𑀢𑀟 पच𑀲𑀢णच𑀪𑀳न𑀯',
    'प𑁣ध𑀳ण ध𑀫𑀢𑀪𑀢 𑀝च𑀟 𑀫च𑀢𑀲𑁦 𑀳𑀫𑀢 च 𑀪च𑀟च𑀪 𑀭𑀭 बच 𑀱चपच𑀟 चबन𑀳पच 𑀭थ𑀗𑀧𑀮 ञच𑀟 𑀱च𑀳च𑀟 ढच𑀣𑀠𑀢𑀟प𑁣𑀟 ञच𑀟 𑀤च𑀠ढ𑀢च 𑀟𑁦𑀯',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 64,000 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 4 tokens mean: 37.72 tokens max: 292 tokens	min: 4 tokens mean: 90.07 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1
  𑀞न𑀣न ढ𑀢𑀪𑀟𑀢𑀟𑀦𑀞न𑀳च प𑁦𑀞न𑀟	प𑁦𑀞न𑀟 पचबच णच𑀟च 𑀞न𑀣न 𑀣च ढ𑀢𑀪𑀟𑀢𑀟𑀦𑀞न𑀳च 𑀣च प𑁦𑀞न𑀟 पचत𑀫𑁣बच𑀯
  च त𑀢ढ𑀢ण𑁣ण𑀢𑀟 𑀳च𑀣च𑀪𑀱च𑀪 𑀳न झच𑀪च 𑀠चप𑀳चण𑀢𑀟	चढ𑁣𑀞च𑀢𑀞च𑀠च𑀪 च णच𑀱च𑀟त𑀢𑀟 त𑀢ढ𑀢ण𑁣ण𑀢𑀟 𑀳च𑀣च𑀪𑀱च𑀪 𑀘च𑀠च𑀙च𑀦 𑀠च𑀳न च𑀠𑀲च𑀟𑀢 𑀤च 𑀳न 𑀢णच झच𑀪च 𑀠नपच𑀟𑁦 च 𑀠चप𑀳चण𑀢𑀟 चढ𑁣𑀞च𑀟𑀳न𑀯
  𑀣च बन𑀣न𑀠𑀠च𑀱च 𑀘च𑀪𑀢𑀣न𑀟 𑀠न𑀘चललन पच 𑀯	पच ढच 𑀣च बन𑀣न𑀠𑀠च𑀱च बच 𑀘च𑀪𑀢𑀣न𑀟 च𑀟च𑀪त𑀫𑀢𑀳प 𑀣चढच𑀟ष𑀣चढच𑀟 𑀣च 𑀠न𑀘चललन 𑀠च𑀳न चलचझच 𑀣च झन𑀟ब𑀢णच𑀪 𑀠च𑀙च𑀢𑀞चपच 𑀙णच𑀟त𑀢 पच 𑀘च𑀠न𑀳 𑀯

• Loss: DenoisingAutoEncoderLoss

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
0.125	500	2.5392
0.25	1000	1.4129
0.375	1500	1.3383
0.5	2000	1.288
0.625	2500	1.2627
0.75	3000	1.239
0.875	3500	1.2208
1.0	4000	1.2041
1.125	4500	1.1743
1.25	5000	1.1633
1.375	5500	1.1526
1.5	6000	1.1375
1.625	6500	1.1313
1.75	7000	1.1246
1.875	7500	1.1162
2.0	8000	1.1096
2.125	8500	1.0876
2.25	9000	1.0839
2.375	9500	1.0791
2.5	10000	1.0697
2.625	10500	1.0671
2.75	11000	1.0644
2.875	11500	1.0579
3.0	12000	1.0528

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.0.1
• Transformers: 4.42.4
• PyTorch: 2.3.1+cu121
• Accelerate: 0.33.0
• Datasets: 2.18.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

DenoisingAutoEncoderLoss

@inproceedings{wang-2021-TSDAE,
    title = "TSDAE: Using Transformer-based Sequential Denoising Auto-Encoderfor Unsupervised Sentence Embedding Learning",
    author = "Wang, Kexin and Reimers, Nils and Gurevych, Iryna", 
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    pages = "671--688",
    url = "https://arxiv.org/abs/2104.06979",
}