metadata

tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:21769
  - loss:MultipleNegativesRankingLoss
base_model: am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e
widget:
  - source_sentence: >-
      Day 1 - Job Losses Biden CANCELS Keystone Pipeline on - Day 1. XX 83k jobs
      lost. XX Get ready for $4 Gas prices by Summer.
    sentences:
      - >-
        Pedro Castillo manipulated a photograph of a rally Photo of a Castillo
        rally in Peru was altered to show alleged manipulation
      - >-
        Biden's Keystone XL pipeline reversal will cost tens of thousands of
        jobs Posts inflate job losses from Biden's Keystone pipeline reversal
      - >-
        Delete System32 folder to clear your privacy browsing history? This hoax
        has been around for years, don’t delete System32 folder
  - source_sentence: ' Thiago Brazil  The "Russian missile" turns cars and destroys the street but the windows of the buildings did not break down and neither did smeared with dust. 1 more fake advertising piece of the neo-Nazi Zelensky. need do more next time!!'
    sentences:
      - >-
        Intact windows in Ukraine prove Russian attack simulation Intact windows
        and overturned cars do not prove simulated attack in Bucha, Ukraine
      - >-
        Editorial of the newspaper Le Monde says that Mexico is on the way to
        ruin The newspaper Le Monde did not publish an editorial titled "Mexico
        on a direct path to ruin"
      - >-
        A photo of 189-year-old Jonathan the turtle This photo does not show the
        189-year-old Jonathan tortoise, but a giant tortoise in Australia
  - source_sentence: ' madri Greta Thunberg, urged the Chinese to do without traditional chopsticks to protect the trees The Chinese then asked Greta to return to the school she was in could find out that traditional Chopsticks are made from bamboo, and bamboo is a grass! The Chinese have Greta and her friends too asked to give up toilet paper to wipe her ass ''cause this one will made from trees.'
    sentences:
      - >-
        This is how they repress in Bolivia tear gas grenade to the head The man
        killed by the shot of a tear gas grenade to the head was the victim of
        the repression of a protest in Iraq, not in Bolivia
      - >-
        Greta Thunberg urged China to ban chopsticks There is no evidence for
        Greta Thunberg's demand for a ban on chopsticks
      - >-
        Accurate reporting on Pfizer-BioNTech Covid-19 vaccination drive and
        deaths in Germany Social media posts misrepresent Pfizer-BioNTech
        Covid-19 vaccinations in Germany
  - source_sentence: >-
      Do you want to know the truth? The "Vaccine" they are voting for is mRNA,
      it circulates through the blood until it finds a receptor for the enzyme
      ACE2, which exists mainly in the testicles, a little in the ovaries, and
      in the myelin of neurons. The m is for messenger, through the receptor,
      the RNA penetrates the cell and rewrites its genetic code. The cell no
      longer serves what nature created it for, it serves to create what the
      laboratories designed RNA for. Theoretically "defenses". The price is that
      97% of the inoculated males will remain sterile, but also, if they are
      young children, they will never develop secondary sexual characteristics.
      They will be androgynous, without sexual desire, or very moderate, and
      probably more manageable and obedient. 45% of girls will be sterile.
      Neuronal damage, on the other hand, will affect part of your frontal
      cortex. You will be able to work, even drive a car, but not think deeply.
      Perfect slaves of the New Normality. It is PHASE 3 of the plan, as it was
      finalized in EVENT 201. PHASE 1 was to scare you, isolate you and lock you
      up, due to a virus that, as a single cause, only killed a tiny handful of
      people. Much less than last year's flu. PHASE 2 makes you wear a grotesque
      and useless mask, which depersonalizes you and deprives you of oxygen. May
      you lose your job, partner and affections. PHASE 3, when you are already
      desperate, is the "Vaccine". They're going to tell you I'm lying, so ask
      what's in it. They will answer that by law not even doctors can know.
      Secret. You don't have the maturity to know it, trust the government, the
      media, the WHO and the employees of George Soros and Bill Gates, like Dr.
      Pedro Cahn and his sinister Fundación Huésped. They will tell you that the
      laboratories are responsible, but by law you will not be able to claim
      anyone. Let's see, when your balls dry, you'll only have to cry about
      what's left of them. When you know that you will never have a grandchild,
      that you will never see your son become a father, nor graduate from a
      career, because his brain will be lobotomized. The opposition"? I don't
      want to make you bitter, but 90% receive money from Soros' Open Society,
      from the Bill and Melinda Gates Foundation, from the Ford Foundation, from
      the Rockefeller Foundation, from the Chinese Communist Party through its
      figurehead, the investor Ming Wai Lau. Who will tell you the truth? Dr.
      Roxana Bruno, Dr. Chinda Brandolino, Dr. Heiko Schöning, Doctors for
      Truth, Lawyers for Truth, Teachers for Truth. Turn off the TV, burn your
      muzzle, breathe, go out to hug your parents, embrace freedom, don't let
      any doctor who isn't the truth touch you. In this way the Global Sanitary
      Dictatorship will collapse, and we will wake up from this nightmare. From
      the wall of Horacio Rivara CLIPARTO CLIPARTO CLIPARTO CLIPARTO CLIPARTO
      CLIPARTO C C
    sentences:
      - >-
        Today- Mayor of São Paulo thanking the Bolsonaro government on vaccines
        against covid-19 Video with thanks to the federal government was for
        funds for a hospital in May 2020
      - >-
        This photo shows Bolsonaro surrounded by global leaders at a G20 meeting
        No, the photo is a montage and who appears in the original is US
        President Donald Trump
      - >-
        Vaccines to prevent covid-19 cause infertility Vaccines to prevent
        covid-19 are not designed to affect fertility
  - source_sentence: >-
      The moment of the death of President Mohamed Morsi, may God have mercy on
      him, God willing 
    sentences:
      - >-
        Cuba has Interferon Alpha 2B, the cure, the vaccine against the
        coronavirus The Cuban antiviral Interferon Alfa 2B is used in China to
        treat patients with the new coronavirus, but it is neither a vaccine nor
        a cure
      - >-
        José Antonio Kast said: "Juvenile delinquent of 16 or 17 years will not
        go to sename, he will do compulsory military service" Chilean
        presidential candidate Kast does not propose that young people who
        commit crimes do military service
      - >-
        The moment of the death of President Mohamed Morsi This video belongs to
        the trial of those accused of the Port Said events and does not show the
        moment of the death of former Egyptian President Mohamed Morsi
pipeline_tag: sentence-similarity
library_name: sentence-transformers

SentenceTransformer based on am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e

This is a sentence-transformers model finetuned from am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e. It maps sentences & paragraphs to a 896-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e
Maximum Sequence Length: 512 tokens
Output Dimensionality: 896 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: Qwen2Model 
  (1): Pooling({'word_embedding_dimension': 896, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'The moment of the death of President Mohamed Morsi, may God have mercy on him, God willing ',
    'The moment of the death of President Mohamed Morsi This video belongs to the trial of those accused of the Port Said events and does not show the moment of the death of former Egyptian President Mohamed Morsi',
    'Cuba has Interferon Alpha 2B, the cure, the vaccine against the coronavirus The Cuban antiviral Interferon Alfa 2B is used in China to treat patients with the new coronavirus, but it is neither a vaccine nor a cure',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 896]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

Size: 21,769 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 6 tokens
mean: 114.75 tokens
max: 512 tokens

min: 11 tokens
mean: 34.58 tokens
max: 120 tokens

	sentence_0	sentence_1
type	string	string
details	min: 6 tokens mean: 114.75 tokens max: 512 tokens	min: 11 tokens mean: 34.58 tokens max: 120 tokens

Samples:

sentence_0	sentence_1
`Palestine false positive against Israel. Makeup for international newscasts.`	`Video of Palestinians wearing makeup pretending to be injured by Israeli bombing in Gaza A video showing Palestinians bleeding corresponds to a medical training in 2017`
`Regrowth After a Australia bushfire`	`Photos of regrowth after Australian bushfires Most of these photos were taken years before the recent Australian bushfires`
`LET'S GO, THANK GOD!! CNN IN SPANISH 16:48 21°1 GENERAL ELECTIONS IN PERU THE SURVEYS: CAN 3. 4% 22% 16% eleven% 6% 5% 4% GENERAL ELECTIONS LN PM RAFAEL LOPEZ ALIAGA LEADS THE SURVEYS IN PERU`	`CNN published a poll where the favorite is López Aliaga CNN did not publish a survey of the presidential elections in Peru that gives the advantage to López Aliaga`

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim"
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 2
per_device_eval_batch_size: 2
num_train_epochs: 1
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 2
per_device_eval_batch_size: 2
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
0.0459	500	0.0083
0.0919	1000	0.019
0.1378	1500	0.0255
0.1837	2000	0.0372
0.2297	2500	0.0315
0.2756	3000	0.0258
0.3215	3500	0.0211
0.3675	4000	0.0187
0.4134	4500	0.0264
0.4593	5000	0.0348
0.5053	5500	0.0197
0.5512	6000	0.0102
0.5972	6500	0.0092
0.6431	7000	0.0169
0.6890	7500	0.0109
0.7350	8000	0.0115
0.7809	8500	0.0173
0.8268	9000	0.0196
0.8728	9500	0.028
0.9187	10000	0.0218
0.9646	10500	0.0169

Framework Versions

Python: 3.11.11
Sentence Transformers: 3.4.1
Transformers: 4.48.3
PyTorch: 2.5.1+cu124
Accelerate: 1.3.0
Datasets: 3.3.2
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}