README.md

---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:21769
- loss:MultipleNegativesRankingLoss
base_model: am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e
widget:
- source_sentence: Day 1 - Job Losses Biden CANCELS Keystone Pipeline on - Day 1.
    XX 83k jobs lost. XX Get ready for $4 Gas prices by Summer.
  sentences:
  - Pedro Castillo manipulated a photograph of a rally Photo of a Castillo rally in
    Peru was altered to show alleged manipulation
  - Biden's Keystone XL pipeline reversal will cost tens of thousands of jobs Posts
    inflate job losses from Biden's Keystone pipeline reversal
  - Delete System32 folder to clear your privacy browsing history? This hoax has been
    around for years, don’t delete System32 folder
- source_sentence: ' Thiago Brazil  The "Russian missile" turns cars and destroys
    the street but the windows of the buildings did not break down and neither did
    smeared with dust. 1 more fake advertising piece of the neo-Nazi Zelensky. need
    do more next time!!'
  sentences:
  - Intact windows in Ukraine prove Russian attack simulation Intact windows and overturned
    cars do not prove simulated attack in Bucha, Ukraine
  - Editorial of the newspaper Le Monde says that Mexico is on the way to ruin The
    newspaper Le Monde did not publish an editorial titled "Mexico on a direct path
    to ruin"
  - A photo of 189-year-old Jonathan the turtle This photo does not show the 189-year-old
    Jonathan tortoise, but a giant tortoise in Australia
- source_sentence: ' madri Greta Thunberg, urged the Chinese to do without traditional
    chopsticks to protect the trees The Chinese then asked Greta to return to the
    school she was in could find out that traditional Chopsticks are made from bamboo,
    and bamboo is a grass! The Chinese have Greta and her friends too asked to give
    up toilet paper to wipe her ass ''cause this one will made from trees.'
  sentences:
  - This is how they repress in Bolivia tear gas grenade to the head The man killed
    by the shot of a tear gas grenade to the head was the victim of the repression
    of a protest in Iraq, not in Bolivia
  - Greta Thunberg urged China to ban chopsticks There is no evidence for Greta Thunberg's
    demand for a ban on chopsticks
  - Accurate reporting on Pfizer-BioNTech Covid-19 vaccination drive and deaths in
    Germany Social media posts misrepresent Pfizer-BioNTech Covid-19 vaccinations
    in Germany
- source_sentence: Do you want to know the truth? The "Vaccine" they are voting for
    is mRNA, it circulates through the blood until it finds a receptor for the enzyme
    ACE2, which exists mainly in the testicles, a little in the ovaries, and in the
    myelin of neurons. The m is for messenger, through the receptor, the RNA penetrates
    the cell and rewrites its genetic code. The cell no longer serves what nature
    created it for, it serves to create what the laboratories designed RNA for. Theoretically
    "defenses". The price is that 97% of the inoculated males will remain sterile,
    but also, if they are young children, they will never develop secondary sexual
    characteristics. They will be androgynous, without sexual desire, or very moderate,
    and probably more manageable and obedient. 45% of girls will be sterile. Neuronal
    damage, on the other hand, will affect part of your frontal cortex. You will be
    able to work, even drive a car, but not think deeply. Perfect slaves of the New
    Normality. It is PHASE 3 of the plan, as it was finalized in EVENT 201. PHASE
    1 was to scare you, isolate you and lock you up, due to a virus that, as a single
    cause, only killed a tiny handful of people. Much less than last year's flu. PHASE
    2 makes you wear a grotesque and useless mask, which depersonalizes you and deprives
    you of oxygen. May you lose your job, partner and affections. PHASE 3, when you
    are already desperate, is the "Vaccine". They're going to tell you I'm lying,
    so ask what's in it. They will answer that by law not even doctors can know. Secret.
    You don't have the maturity to know it, trust the government, the media, the WHO
    and the employees of George Soros and Bill Gates, like Dr. Pedro Cahn and his
    sinister Fundación Huésped. They will tell you that the laboratories are responsible,
    but by law you will not be able to claim anyone. Let's see, when your balls dry,
    you'll only have to cry about what's left of them. When you know that you will
    never have a grandchild, that you will never see your son become a father, nor
    graduate from a career, because his brain will be lobotomized. The opposition"?
    I don't want to make you bitter, but 90% receive money from Soros' Open Society,
    from the Bill and Melinda Gates Foundation, from the Ford Foundation, from the
    Rockefeller Foundation, from the Chinese Communist Party through its figurehead,
    the investor Ming Wai Lau. Who will tell you the truth? Dr. Roxana Bruno, Dr.
    Chinda Brandolino, Dr. Heiko Schöning, Doctors for Truth, Lawyers for Truth, Teachers
    for Truth. Turn off the TV, burn your muzzle, breathe, go out to hug your parents,
    embrace freedom, don't let any doctor who isn't the truth touch you. In this way
    the Global Sanitary Dictatorship will collapse, and we will wake up from this
    nightmare. From the wall of Horacio Rivara CLIPARTO CLIPARTO CLIPARTO CLIPARTO
    CLIPARTO CLIPARTO C C
  sentences:
  - Today- Mayor of São Paulo thanking the Bolsonaro government on vaccines against
    covid-19 Video with thanks to the federal government was for funds for a hospital
    in May 2020
  - This photo shows Bolsonaro surrounded by global leaders at a G20 meeting No, the
    photo is a montage and who appears in the original is US President Donald Trump
  - Vaccines to prevent covid-19 cause infertility Vaccines to prevent covid-19 are
    not designed to affect fertility
- source_sentence: 'The moment of the death of President Mohamed Morsi, may God have
    mercy on him, God willing '
  sentences:
  - Cuba has Interferon Alpha 2B, the cure, the vaccine against the coronavirus The
    Cuban antiviral Interferon Alfa 2B is used in China to treat patients with the
    new coronavirus, but it is neither a vaccine nor a cure
  - 'José Antonio Kast said: "Juvenile delinquent of 16 or 17 years will not go to
    sename, he will do compulsory military service" Chilean presidential candidate
    Kast does not propose that young people who commit crimes do military service'
  - The moment of the death of President Mohamed Morsi This video belongs to the trial
    of those accused of the Port Said events and does not show the moment of the death
    of former Egyptian President Mohamed Morsi
pipeline_tag: sentence-similarity
library_name: sentence-transformers
---

# SentenceTransformer based on am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e](https://huggingface.co./am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e). It maps sentences & paragraphs to a 896-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e](https://huggingface.co./am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e) <!-- at revision db90e52b2078421f04b71e31b5a90f5bf8d321d7 -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 896 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: Qwen2Model 
  (1): Pooling({'word_embedding_dimension': 896, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'The moment of the death of President Mohamed Morsi, may God have mercy on him, God willing ',
    'The moment of the death of President Mohamed Morsi This video belongs to the trial of those accused of the Port Said events and does not show the moment of the death of former Egyptian President Mohamed Morsi',
    'Cuba has Interferon Alpha 2B, the cure, the vaccine against the coronavirus The Cuban antiviral Interferon Alfa 2B is used in China to treat patients with the new coronavirus, but it is neither a vaccine nor a cure',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 896]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset

* Size: 21,769 training samples
* Columns: <code>sentence_0</code> and <code>sentence_1</code>
* Approximate statistics based on the first 1000 samples:
  |         | sentence_0                                                                          | sentence_1                                                                          |
  |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                              | string                                                                              |
  | details | <ul><li>min: 6 tokens</li><li>mean: 114.75 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 11 tokens</li><li>mean: 34.58 tokens</li><li>max: 120 tokens</li></ul> |
* Samples:
  | sentence_0                                                                                                                                                                                                    | sentence_1                                                                                                                                                                            |
  |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>Palestine false positive against Israel. Makeup for international newscasts. </code>                                                                                                                    | <code>Video of Palestinians wearing makeup pretending to be injured by Israeli bombing in Gaza A video showing Palestinians bleeding corresponds to a medical training in 2017</code> |
  | <code>Regrowth After a Australia bushfire </code>                                                                                                                                                             | <code>Photos of regrowth after Australian bushfires Most of these photos were taken years before the recent Australian bushfires</code>                                               |
  | <code>LET'S GO, THANK GOD!! CNN IN SPANISH 16:48 21°1 GENERAL ELECTIONS IN PERU THE SURVEYS: CAN 3. 4% 22% 16% eleven% 6% 5% 4%  GENERAL ELECTIONS LN PM RAFAEL LOPEZ ALIAGA LEADS THE SURVEYS IN PERU</code> | <code>CNN published a poll where the favorite is López Aliaga CNN did not publish a survey of the presidential elections in Peru that gives the advantage to López Aliaga</code>      |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `per_device_train_batch_size`: 2
- `per_device_eval_batch_size`: 2
- `num_train_epochs`: 1
- `multi_dataset_batch_sampler`: round_robin

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 2
- `per_device_eval_batch_size`: 2
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin

</details>

### Training Logs
| Epoch  | Step  | Training Loss |
|:------:|:-----:|:-------------:|
| 0.0459 | 500   | 0.0083        |
| 0.0919 | 1000  | 0.019         |
| 0.1378 | 1500  | 0.0255        |
| 0.1837 | 2000  | 0.0372        |
| 0.2297 | 2500  | 0.0315        |
| 0.2756 | 3000  | 0.0258        |
| 0.3215 | 3500  | 0.0211        |
| 0.3675 | 4000  | 0.0187        |
| 0.4134 | 4500  | 0.0264        |
| 0.4593 | 5000  | 0.0348        |
| 0.5053 | 5500  | 0.0197        |
| 0.5512 | 6000  | 0.0102        |
| 0.5972 | 6500  | 0.0092        |
| 0.6431 | 7000  | 0.0169        |
| 0.6890 | 7500  | 0.0109        |
| 0.7350 | 8000  | 0.0115        |
| 0.7809 | 8500  | 0.0173        |
| 0.8268 | 9000  | 0.0196        |
| 0.8728 | 9500  | 0.028         |
| 0.9187 | 10000 | 0.0218        |
| 0.9646 | 10500 | 0.0169        |


### Framework Versions
- Python: 3.11.11
- Sentence Transformers: 3.4.1
- Transformers: 4.48.3
- PyTorch: 2.5.1+cu124
- Accelerate: 1.3.0
- Datasets: 3.3.2
- Tokenizers: 0.21.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->