--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:21769 - loss:MultipleNegativesRankingLoss base_model: am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e widget: - source_sentence: Day 1 - Job Losses Biden CANCELS Keystone Pipeline on - Day 1. XX 83k jobs lost. XX Get ready for $4 Gas prices by Summer. sentences: - Pedro Castillo manipulated a photograph of a rally Photo of a Castillo rally in Peru was altered to show alleged manipulation - Biden's Keystone XL pipeline reversal will cost tens of thousands of jobs Posts inflate job losses from Biden's Keystone pipeline reversal - Delete System32 folder to clear your privacy browsing history? This hoax has been around for years, don’t delete System32 folder - source_sentence: ' Thiago Brazil The "Russian missile" turns cars and destroys the street but the windows of the buildings did not break down and neither did smeared with dust. 1 more fake advertising piece of the neo-Nazi Zelensky. need do more next time!!' sentences: - Intact windows in Ukraine prove Russian attack simulation Intact windows and overturned cars do not prove simulated attack in Bucha, Ukraine - Editorial of the newspaper Le Monde says that Mexico is on the way to ruin The newspaper Le Monde did not publish an editorial titled "Mexico on a direct path to ruin" - A photo of 189-year-old Jonathan the turtle This photo does not show the 189-year-old Jonathan tortoise, but a giant tortoise in Australia - source_sentence: ' madri Greta Thunberg, urged the Chinese to do without traditional chopsticks to protect the trees The Chinese then asked Greta to return to the school she was in could find out that traditional Chopsticks are made from bamboo, and bamboo is a grass! The Chinese have Greta and her friends too asked to give up toilet paper to wipe her ass ''cause this one will made from trees.' sentences: - This is how they repress in Bolivia tear gas grenade to the head The man killed by the shot of a tear gas grenade to the head was the victim of the repression of a protest in Iraq, not in Bolivia - Greta Thunberg urged China to ban chopsticks There is no evidence for Greta Thunberg's demand for a ban on chopsticks - Accurate reporting on Pfizer-BioNTech Covid-19 vaccination drive and deaths in Germany Social media posts misrepresent Pfizer-BioNTech Covid-19 vaccinations in Germany - source_sentence: Do you want to know the truth? The "Vaccine" they are voting for is mRNA, it circulates through the blood until it finds a receptor for the enzyme ACE2, which exists mainly in the testicles, a little in the ovaries, and in the myelin of neurons. The m is for messenger, through the receptor, the RNA penetrates the cell and rewrites its genetic code. The cell no longer serves what nature created it for, it serves to create what the laboratories designed RNA for. Theoretically "defenses". The price is that 97% of the inoculated males will remain sterile, but also, if they are young children, they will never develop secondary sexual characteristics. They will be androgynous, without sexual desire, or very moderate, and probably more manageable and obedient. 45% of girls will be sterile. Neuronal damage, on the other hand, will affect part of your frontal cortex. You will be able to work, even drive a car, but not think deeply. Perfect slaves of the New Normality. It is PHASE 3 of the plan, as it was finalized in EVENT 201. PHASE 1 was to scare you, isolate you and lock you up, due to a virus that, as a single cause, only killed a tiny handful of people. Much less than last year's flu. PHASE 2 makes you wear a grotesque and useless mask, which depersonalizes you and deprives you of oxygen. May you lose your job, partner and affections. PHASE 3, when you are already desperate, is the "Vaccine". They're going to tell you I'm lying, so ask what's in it. They will answer that by law not even doctors can know. Secret. You don't have the maturity to know it, trust the government, the media, the WHO and the employees of George Soros and Bill Gates, like Dr. Pedro Cahn and his sinister Fundación Huésped. They will tell you that the laboratories are responsible, but by law you will not be able to claim anyone. Let's see, when your balls dry, you'll only have to cry about what's left of them. When you know that you will never have a grandchild, that you will never see your son become a father, nor graduate from a career, because his brain will be lobotomized. The opposition"? I don't want to make you bitter, but 90% receive money from Soros' Open Society, from the Bill and Melinda Gates Foundation, from the Ford Foundation, from the Rockefeller Foundation, from the Chinese Communist Party through its figurehead, the investor Ming Wai Lau. Who will tell you the truth? Dr. Roxana Bruno, Dr. Chinda Brandolino, Dr. Heiko Schöning, Doctors for Truth, Lawyers for Truth, Teachers for Truth. Turn off the TV, burn your muzzle, breathe, go out to hug your parents, embrace freedom, don't let any doctor who isn't the truth touch you. In this way the Global Sanitary Dictatorship will collapse, and we will wake up from this nightmare. From the wall of Horacio Rivara CLIPARTO CLIPARTO CLIPARTO CLIPARTO CLIPARTO CLIPARTO C C sentences: - Today- Mayor of São Paulo thanking the Bolsonaro government on vaccines against covid-19 Video with thanks to the federal government was for funds for a hospital in May 2020 - This photo shows Bolsonaro surrounded by global leaders at a G20 meeting No, the photo is a montage and who appears in the original is US President Donald Trump - Vaccines to prevent covid-19 cause infertility Vaccines to prevent covid-19 are not designed to affect fertility - source_sentence: 'The moment of the death of President Mohamed Morsi, may God have mercy on him, God willing ' sentences: - Cuba has Interferon Alpha 2B, the cure, the vaccine against the coronavirus The Cuban antiviral Interferon Alfa 2B is used in China to treat patients with the new coronavirus, but it is neither a vaccine nor a cure - 'José Antonio Kast said: "Juvenile delinquent of 16 or 17 years will not go to sename, he will do compulsory military service" Chilean presidential candidate Kast does not propose that young people who commit crimes do military service' - The moment of the death of President Mohamed Morsi This video belongs to the trial of those accused of the Port Said events and does not show the moment of the death of former Egyptian President Mohamed Morsi pipeline_tag: sentence-similarity library_name: sentence-transformers --- # SentenceTransformer based on am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e](https://huggingface.co./am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e). It maps sentences & paragraphs to a 896-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e](https://huggingface.co./am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 896 dimensions - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: Qwen2Model (1): Pooling({'word_embedding_dimension': 896, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("sentence_transformers_model_id") # Run inference sentences = [ 'The moment of the death of President Mohamed Morsi, may God have mercy on him, God willing ', 'The moment of the death of President Mohamed Morsi This video belongs to the trial of those accused of the Port Said events and does not show the moment of the death of former Egyptian President Mohamed Morsi', 'Cuba has Interferon Alpha 2B, the cure, the vaccine against the coronavirus The Cuban antiviral Interferon Alfa 2B is used in China to treat patients with the new coronavirus, but it is neither a vaccine nor a cure', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 896] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 21,769 training samples * Columns: sentence_0 and sentence_1 * Approximate statistics based on the first 1000 samples: | | sentence_0 | sentence_1 | |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | sentence_0 | sentence_1 | |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Palestine false positive against Israel. Makeup for international newscasts. | Video of Palestinians wearing makeup pretending to be injured by Israeli bombing in Gaza A video showing Palestinians bleeding corresponds to a medical training in 2017 | | Regrowth After a Australia bushfire | Photos of regrowth after Australian bushfires Most of these photos were taken years before the recent Australian bushfires | | LET'S GO, THANK GOD!! CNN IN SPANISH 16:48 21°1 GENERAL ELECTIONS IN PERU THE SURVEYS: CAN 3. 4% 22% 16% eleven% 6% 5% 4% GENERAL ELECTIONS LN PM RAFAEL LOPEZ ALIAGA LEADS THE SURVEYS IN PERU | CNN published a poll where the favorite is López Aliaga CNN did not publish a survey of the presidential elections in Peru that gives the advantage to López Aliaga | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `per_device_train_batch_size`: 2 - `per_device_eval_batch_size`: 2 - `num_train_epochs`: 1 - `multi_dataset_batch_sampler`: round_robin #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: no - `prediction_loss_only`: True - `per_device_train_batch_size`: 2 - `per_device_eval_batch_size`: 2 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 5e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1 - `num_train_epochs`: 1 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.0 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: round_robin
### Training Logs | Epoch | Step | Training Loss | |:------:|:-----:|:-------------:| | 0.0459 | 500 | 0.0083 | | 0.0919 | 1000 | 0.019 | | 0.1378 | 1500 | 0.0255 | | 0.1837 | 2000 | 0.0372 | | 0.2297 | 2500 | 0.0315 | | 0.2756 | 3000 | 0.0258 | | 0.3215 | 3500 | 0.0211 | | 0.3675 | 4000 | 0.0187 | | 0.4134 | 4500 | 0.0264 | | 0.4593 | 5000 | 0.0348 | | 0.5053 | 5500 | 0.0197 | | 0.5512 | 6000 | 0.0102 | | 0.5972 | 6500 | 0.0092 | | 0.6431 | 7000 | 0.0169 | | 0.6890 | 7500 | 0.0109 | | 0.7350 | 8000 | 0.0115 | | 0.7809 | 8500 | 0.0173 | | 0.8268 | 9000 | 0.0196 | | 0.8728 | 9500 | 0.028 | | 0.9187 | 10000 | 0.0218 | | 0.9646 | 10500 | 0.0169 | ### Framework Versions - Python: 3.11.11 - Sentence Transformers: 3.4.1 - Transformers: 4.48.3 - PyTorch: 2.5.1+cu124 - Accelerate: 1.3.0 - Datasets: 3.3.2 - Tokenizers: 0.21.0 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```