--- base_model: BAAI/bge-small-en datasets: - sentence-transformers/hotpotqa language: - en library_name: sentence-transformers license: apache-2.0 metrics: - cosine_accuracy - dot_accuracy - manhattan_accuracy - euclidean_accuracy - max_accuracy pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:76064 - loss:MatryoshkaLoss - loss:TripletLoss widget: - source_sentence: When was the brewery founded by the family who resided in the Lemp Mansion founded? sentences: - Latrobe Brewing Company Latrobe Brewing Company in Latrobe, Pennsylvania, founded in 1839, was one of the largest breweries in the United States and the maker of "Rolling Rock" beer (famous for its small green bottles). It was purchased by Labatt Brewing Company in 1987, which in turn was purchased in 1995 by the Belgian brewing conglomerate corporation Interbrew, which merged later into InBev in 2004. - Lemp Mansion The Lemp Mansion (3322 DeMenil Place, St. Louis, Missouri) is a historical house in Benton Park, St. Louis, Missouri. It is also the site of four suicides by Lemp family members after the death of the son Frederick Lemp, whose William J. Lemp Brewing Co. dominated the St. Louis beer market before Prohibition with its Falstaff beer brand. The mansion is said to be haunted by members of the Lemp family. - 'Man of the House (song) "Man of the House" is a song co-written and recorded by American country music artist Chuck Wicks. It was released in January 2009 as the third single from the album "Starting Now". The song reached #27 on the "Billboard" Hot Country Songs chart. The song was written by Wicks and Michael Mobley.' - source_sentence: Who is the translator and researcher who served as a dean at Hartford Junior College and translated her Belgian-French partner's 1968 work into English? sentences: - William L. Shirer William Lawrence Shirer (February 23, 1904 – December 28, 1993) was an American journalist and war correspondent. He wrote "The Rise and Fall of the Third Reich", a history of Nazi Germany that has been read by many and cited in scholarly works for more than 50 years. Originally a foreign correspondent for the "Chicago Tribune" and the International News Service, Shirer was the first reporter hired by Edward R. Murrow for what would become a CBS radio team of journalists known as "Murrow's Boys". He became known for his broadcasts from Berlin, from the rise of the Nazi dictatorship through the first year of World War II (1940). With Murrow, he organized the first broadcast world news roundup, a format still followed by news broadcasts. - 'The Abyss (Yourcenar novel) The Abyss (French: L''Œuvre au noir ) is a 1968 novel by the Belgian-French writer Marguerite Yourcenar. Its narrative centers on the life and death of Zeno, a physician, philosopher, scientist and alchemist born in Bruges during the Renaissance era. The book was published in France in 1968 and was met with immediate popular interest as well as critical acclaim, obtaining the Prix Femina with unanimous votes the year of its publication. The English translation by Grace Frick has been published under the title "The Abyss" or alternatively Zeno of Bruges. Belgian filmmaker André Delvaux adapted it into a film in 1988.' - Pierre L. van den Berghe Pierre L. van den Berghe (born 1933) is professor emeritus of sociology and anthropology at the University of Washington, where he has worked since 1965. Born in the Belgian Congo to Belgian parents, and spending World War II in occupied Belgium, he was an early witness to ethnic conflict and racism, which eventually led him to become a leading authority on ethnic relations. He has conducted field work in South Africa, Mexico, Guatemala, Iran, Lebanon, Nigeria, Peru, and Israel. Early in his career, he lectured at the University of Natal alongside Leo Kuper and Fatima Meer. A student of Talcott Parsons at Harvard (receiving the Ph.D. in 1960), he nevertheless had little interest in structural functionalism and was one of the first proponents of sociobiological approaches to social phenomena. - source_sentence: Them Crooked Vultures and The Vines are both part of what music genres? sentences: - Friendly Center Friendly Center is a large, open-air shopping center located in northwestern Greensboro, North Carolina, near the intersection of Wendover Avenue and Friendly Avenue. The shopping center opened in August 1957, and with its inward orientation, Friendly Center could be classified as an outdoor lifestyle center. Its anchor tenants include Belk, Macy's, and Sears. Other tenants include Barnes & Noble, Old Navy, The Grande Theatre is a 16-screen multiplex cinema operated by Regal Cinemas. It also contains Harris Teeter's flagship supermarket location encompassing 72,000 square feet (6,700 m2) and Whole Foods Market. There are specialty "foodie" stores tucked away in the back corner by Harris Teeter such as the Savory Spice Shop and Midtown Olive Oil. It features a number of national retailers such as Banana Republic, Victoria's Secret, The Limited, Bath & Body Works, Express, The GAP, Eddie Bauer, Talbots, Birkenstock Feet First, Pier 1, and New York & Company. - Twin Wild Twin Wild is a British four-piece alternative rock band. Formed in 2012, the band is made up of the collective creative energies of Richard Hutchison (vocals, guitars), Imran Mair (drums), David Cuzner (guitars) and Edward Thomas (bass). In 2014, the band self-released their track "Fears", which garnered over half a million plays on Soundcloud and charted in Spotify’s Top 20 viral chart. Their style of music has been compared to the likes of Foals, The Neighbourhood, and Bastille. The band have been hailed by Edith Bowman as "The love child of Bastille and Biffy Clyro". - Them Crooked Vultures Them Crooked Vultures is a rock supergroup formed in Los Angeles in 2009 by John Paul Jones (former member of Led Zeppelin) on bass and keyboards, Dave Grohl (of Foo Fighters and formerly of Nirvana) on drums and backing vocals, and Josh Homme (of Queens of the Stone Age, Eagles of Death Metal and formerly of Kyuss) on guitar and vocals. The group also includes guitarist Alain Johannes during live performances. The band began recording in February 2009, and performed their first gig on August 9, 2009, in Chicago, followed by a European debut on August 19. On October 1 the group embarked on a worldwide tour titled "Deserve the Future" with dates going into 2010. The band's first single "New Fang" was released in October 2009, followed by the group's self-titled debut album the following month, debuting at number 12 on the "Billboard" 200. The group won the 2011 Grammy Award for Best Hard Rock Performance for "New Fang". - source_sentence: What type of collection does Nådens år and Agnetha Fältskog have in common? sentences: - Gardiner Island (Nunavut) Gardiner Island is one of the many uninhabited Canadian arctic islands in Qikiqtaaluk Region, Nunavut. It is a Baffin Island offshore island located in Frobisher Bay south of the capital city of Iqaluit. - Nådens år Nådens år (The Year of Grace) is the third studio album by the Swedish rock artist Ulf Lundell. It was released in April 1978 on EMI and Parlophone. It was recorded in EMI Studio, Stockholm, and produced by Kjell Andersson and Lundell. It includes "Snön faller och vi med den" ("The snow is falling and we are too"), one of Lundell's more famous songs. Agnetha Fältskog is involved in the song. The cover picture shows Lundell sitting on a rock next to a dog and was taken in Åre in 1977. "Nådens år" achieved Gold status in Sweden. - Åsa Elzén Åsa Elzén is an artist whose work is informed by feminist theory, intersectionality and post-colonialism. Elzén was born in Sweden in 1972 and currently lives and works in Berlin. - source_sentence: Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with the biography of a 19th century actor born in which year ? sentences: - Herbert Campbell Herbert Campbell (22 December 1844 – 19 July 1904) born Herbert Edward Story was an English comedian and actor who appeared in music hall, Victorian burlesques and musical comedies during the Victorian era. He was famous for starring, for forty years, in the Theatre Royal, Drury Lane's annual Christmas pantomimes, predominantly as a dame. - Leptinella Leptinella is a genus of alpine flowering plant in the Asteraceae family, comprising 33 species, distributed in New Guinea, Australia, New Zealand, South Africa, and South America. Many of the species are endemic to New Zealand. - Red Velvet (play) Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with the biography of the 19th century actor Ira Aldridge and his taking the role of "Othello". It premiered at the Tricycle Theatre (directed by its new artistic director Indhu Rubasingham) from 11 October to 24 November 2012, with Aldridge played by Adrian Lester. model-index: - name: BGE-base-en-v1.5-Hotpotqa results: - task: type: triplet name: Triplet dataset: name: dim 384 type: dim_384 metrics: - type: cosine_accuracy value: 0.8761239943208708 name: Cosine Accuracy - type: dot_accuracy value: 0.1238760056791292 name: Dot Accuracy - type: manhattan_accuracy value: 0.8770705158542357 name: Manhattan Accuracy - type: euclidean_accuracy value: 0.8761239943208708 name: Euclidean Accuracy - type: max_accuracy value: 0.8770705158542357 name: Max Accuracy - task: type: triplet name: Triplet dataset: name: dim 256 type: dim_256 metrics: - type: cosine_accuracy value: 0.8761239943208708 name: Cosine Accuracy - type: dot_accuracy value: 0.1250591575958353 name: Dot Accuracy - type: manhattan_accuracy value: 0.8797917652626597 name: Manhattan Accuracy - type: euclidean_accuracy value: 0.8761239943208708 name: Euclidean Accuracy - type: max_accuracy value: 0.8797917652626597 name: Max Accuracy - task: type: triplet name: Triplet dataset: name: dim 128 type: dim_128 metrics: - type: cosine_accuracy value: 0.8724562233790819 name: Cosine Accuracy - type: dot_accuracy value: 0.13073828679602462 name: Dot Accuracy - type: manhattan_accuracy value: 0.8783719829626124 name: Manhattan Accuracy - type: euclidean_accuracy value: 0.8728111689540937 name: Euclidean Accuracy - type: max_accuracy value: 0.8783719829626124 name: Max Accuracy - task: type: triplet name: Triplet dataset: name: dim 64 type: dim_64 metrics: - type: cosine_accuracy value: 0.8710364410790346 name: Cosine Accuracy - type: dot_accuracy value: 0.143989588263133 name: Dot Accuracy - type: manhattan_accuracy value: 0.8764789398958827 name: Manhattan Accuracy - type: euclidean_accuracy value: 0.867841930903928 name: Euclidean Accuracy - type: max_accuracy value: 0.8764789398958827 name: Max Accuracy --- # BGE-base-en-v1.5-Hotpotqa This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-small-en](https://huggingface.co./BAAI/bge-small-en) on the [sentence-transformers/hotpotqa](https://huggingface.co./datasets/sentence-transformers/hotpotqa) dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [BAAI/bge-small-en](https://huggingface.co./BAAI/bge-small-en) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 384 tokens - **Similarity Function:** Cosine Similarity - **Training Dataset:** - [sentence-transformers/hotpotqa](https://huggingface.co./datasets/sentence-transformers/hotpotqa) - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("sentence_transformers_model_id") # Run inference sentences = [ 'Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with the biography of a 19th century actor born in which year ?', 'Red Velvet (play) Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with the biography of the 19th century actor Ira Aldridge and his taking the role of "Othello". It premiered at the Tricycle Theatre (directed by its new artistic director Indhu Rubasingham) from 11 October to 24 November 2012, with Aldridge played by Adrian Lester.', "Herbert Campbell Herbert Campbell (22 December 1844 – 19 July 1904) born Herbert Edward Story was an English comedian and actor who appeared in music hall, Victorian burlesques and musical comedies during the Victorian era. He was famous for starring, for forty years, in the Theatre Royal, Drury Lane's annual Christmas pantomimes, predominantly as a dame.", ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 384] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Triplet * Dataset: `dim_384` * Evaluated with [TripletEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator) | Metric | Value | |:--------------------|:-----------| | **cosine_accuracy** | **0.8761** | | dot_accuracy | 0.1239 | | manhattan_accuracy | 0.8771 | | euclidean_accuracy | 0.8761 | | max_accuracy | 0.8771 | #### Triplet * Dataset: `dim_256` * Evaluated with [TripletEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator) | Metric | Value | |:--------------------|:-----------| | **cosine_accuracy** | **0.8761** | | dot_accuracy | 0.1251 | | manhattan_accuracy | 0.8798 | | euclidean_accuracy | 0.8761 | | max_accuracy | 0.8798 | #### Triplet * Dataset: `dim_128` * Evaluated with [TripletEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator) | Metric | Value | |:--------------------|:-----------| | **cosine_accuracy** | **0.8725** | | dot_accuracy | 0.1307 | | manhattan_accuracy | 0.8784 | | euclidean_accuracy | 0.8728 | | max_accuracy | 0.8784 | #### Triplet * Dataset: `dim_64` * Evaluated with [TripletEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator) | Metric | Value | |:--------------------|:----------| | **cosine_accuracy** | **0.871** | | dot_accuracy | 0.144 | | manhattan_accuracy | 0.8765 | | euclidean_accuracy | 0.8678 | | max_accuracy | 0.8765 | ## Training Details ### Training Dataset #### sentence-transformers/hotpotqa * Dataset: [sentence-transformers/hotpotqa](https://huggingface.co./datasets/sentence-transformers/hotpotqa) at [f07d3cd](https://huggingface.co./datasets/sentence-transformers/hotpotqa/tree/f07d3cd2d290ea2e83ed35e33d67d6a4658b8786) * Size: 76,064 training samples * Columns: anchor, positive, and negative * Approximate statistics based on the first 1000 samples: | | anchor | positive | negative | |:--------|:----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | anchor | positive | negative | |:-------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | What nationality was the player named MVP in 2017 World Baseball Classic – Pool C ? | 2017 World Baseball Classic – Pool C Pool C of the First Round of the 2017 World Baseball Classic was held at Marlins Park, Miami, Florida, United States, from March 9 to 12, 2017, between Canada, Colombia, the Dominican Republic, and the United States. Pool C was a round-robin tournament. Each team played the other three teams once, with the top two teams – the Dominican Republic and the United States – advancing to Pool F, one of two second-round pools. Manny Machado of the Dominican Republic was named MVP for the first-round Pool C bracket of the WBC, after batting .357. | 2017 World Baseball Classic – Qualifier 2 Qualifier 2 of the Qualifying Round of the 2017 World Baseball Classic was held at Estadio B'Air, Mexicali, Mexico from March 17 to 20, 2016. | | Karl Kraepelin specialized in the study of what predatory arachnids? | Karl Kraepelin Karl Matthias Friedrich Magnus Kraepelin (14 December 1848 Neustrelitz – 28 June 1915 Hamburg), was a German naturalist who specialised in the study of scorpions, centipedes, spiders and solfugids, and was noted for his monograph ""Scorpiones und Pedipalpi"" (Berlin) in 1899, which was an exhaustive survey of the taxonomy of the Order Scorpiones. From 1889–1914 he was Director of the "Naturhistorisches Museum Hamburg ", which was destroyed during World War II, and worked on myriapods from 1901–1916. | Teuthology Teuthology (from Greek "τεῦθος" , "cuttlefish, squid", and -λογία , "-logia") is the study of cephalopods. | | Who directed the 1990 American crime film in which Vito Pick me played a bit part? | Vito Picone Vito Picone (born March 20, 1941) is the lead singer of The Elegants, and along with Jimmy Mochella is a remaining original member. He has also played bit parts in "Goodfellas", "Analyze This", and "The Sopranos". | The Rookie (1990 film) The Rookie is a 1990 American buddy cop film directed by Clint Eastwood and produced by Howard G. Kazanjian, Steven Siebert and David Valdes. It was written from a screenplay conceived by Boaz Yakin and Scott Spiegel. The film stars Charlie Sheen, Clint Eastwood, Raúl Juliá, Sônia Braga, Lara Flynn Boyle, and Tom Skerritt. Eastwood plays a veteran police officer teamed up with a younger detective played by Sheen ("the rookie"), whose intent is to take down a German crime lord in downtown Los Angeles following months of investigation into an exotic car theft ring. | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "TripletLoss", "matryoshka_dims": [ 384, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Evaluation Dataset #### sentence-transformers/hotpotqa * Dataset: [sentence-transformers/hotpotqa](https://huggingface.co./datasets/sentence-transformers/hotpotqa) at [f07d3cd](https://huggingface.co./datasets/sentence-transformers/hotpotqa/tree/f07d3cd2d290ea2e83ed35e33d67d6a4658b8786) * Size: 8,452 evaluation samples * Columns: anchor, positive, and negative * Approximate statistics based on the first 1000 samples: | | anchor | positive | negative | |:--------|:------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | anchor | positive | negative | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Which actress, known for her role as Harper Munroe on the MTV comedy series "Happyland", starred alongside Laura Marano, Parker Mack, Michelle Clunie and Kathleen Wilhoite in the film A Sort of Homecoming? | A Sort of Homecoming (film) A Sort of Homecoming is an American drama directed by Maria Burton, her fifth feature film. The films stars Katherine McNamara, Laura Marano, Parker Mack, Michelle Clunie and Kathleen Wilhoite. The film premiered March 14, 2015 at the Omaha Film Festival. | Nellie Bellflower Nellie Bellflower (born May 1, 1946 in Phoenix, Arizona) is an American actress and voice artist who provided the voice of Princess Ariel in the Ruby-Spears animated television series "Thundarr the Barbarian". She has also been in "The Last Unicorn" (voice), Rankin/Bass "The Return of the King", "Americathon", the miniseries "East of Eden", and guest roles on various TV shows such as "Barnaby Jones", "Barney Miller", "Starsky and Hutch", and "Happy Days" as Fonzie's ex-fiancée Maureen Johnson, a.k.a. "The Lone Stripper", in the Season 2 episode of the series titled "Fonzie's Getting Married" (episode #13). Nellie has been involved in movie production with three projects: "The Girl in Melanie Klein" (2008), "Miss Pettigrew Lives for a Day" (2008) and "Finding Neverland" (2004), for which she was nominated for an Academy Award as Producer for Best Picture. She is married to Michael Mislove. | | Between Pizza Fusion and Pizzeria Venti, which restaurant emphasizes organic ingredients and green building methods? | Pizza Fusion Pizza Fusion is a Deerfield Beach, Florida-based pizza restaurant chain. Using mostly organic ingredients and emphasizing green building methods, the restaurants operate under the tagline Saving the Earth, One Pizza at a Time. | Pizza Schmizza Pizza Schmizza is an American pizza chain with 23 locations throughout the Portland, Oregon area, and two in southern Oregon. Pizza Schmizza, primarily selling thin crust pizza by-the-slice. | | What company distributed the stop motion spin-off special "The Year Without a Santa Claus," which aired on December 10, 1974? | A Miser Brothers' Christmas A Miser Brothers' Christmas is a stop motion spin-off special based on some of the characters from the 1974 Rankin-Bass special "The Year Without a Santa Claus". Distributed by Warner Bros. Animation under their Warner Premiere label (the rights holders of the post-1974 Rankin-Bass library) and Toronto-based Cuppa Coffee Studios, the one-hour special premiered on ABC Family on Saturday, December 13, 2008, during the network's annual The 25 Days of Christmas programming. Mickey Rooney and George S. Irving reprised their respective roles as Santa Claus and Heat Miser at ages 88 and 86. Snow Miser, originally portrayed by Dick Shawn who died in 1987, was voiced by Juan Chioran, while Mrs. Claus, voiced by Shirley Booth in the original, was portrayed by Catherine Disher (because Booth had died in 1992). The movie aimed to emulate the Rankin/Bass animation style. This is the last Christmas special to feature Mickey Rooney as Santa Claus, as he died in 2014, as well as the last time George Irving voiced Heat Miser, as he died in 2016. | Holidaze: The Christmas That Almost Didn't Happen Holidaze: The Christmas That Almost Didn't Happen is an American stop-motion animated Christmas television special directed by David H. Brooks, that originally aired in 2006 and produced by BixPix Entertainment, Once Upon a Frog and Madison Street Entertainment. The show's plot has Rusty Reindeer (Fred Savage) the brother of Rudolph the Red Nosed Reindeer joining a support group for depressed holiday icons, and he and the other characters search for the meaning of Christmas and help a young boy (Dylan and Cole Sprouse) to get on Santa's nice list. Rusty's cohorts include Candie, the Easter Bunny (Gladys Knight); Mr. C, the grouchy cherub (Paul Rodriguez); Albert, the Thanksgiving Turkey (Harland Williams); And Trick and Treat (Brenda Song and Emily Osment) the teenage Halloween Ghosts. | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "TripletLoss", "matryoshka_dims": [ 384, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 32 - `gradient_accumulation_steps`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 50 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `bf16`: True - `tf32`: True - `load_best_model_at_end`: True - `optim`: adamw_torch_fused - `resume_from_checkpoint`: bge-small-hotpotwa-matryoshka - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 32 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 16 - `eval_accumulation_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 50 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: True - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: bge-small-hotpotwa-matryoshka - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | Training Loss | loss | dim_128_cosine_accuracy | dim_256_cosine_accuracy | dim_384_cosine_accuracy | dim_64_cosine_accuracy | |:-------:|:----:|:-------------:|:-------:|:-----------------------:|:-----------------------:|:-----------------------:|:----------------------:| | 0.3366 | 50 | 19.5758 | 19.3933 | 0.9552 | 0.9663 | 0.9668 | 0.9359 | | 0.6731 | 100 | 19.4573 | 19.0971 | 0.9571 | 0.9646 | 0.9653 | 0.9450 | | 1.0097 | 150 | 19.1409 | 18.4070 | 0.9385 | 0.9434 | 0.9473 | 0.9307 | | 1.3462 | 200 | 18.6431 | 17.3292 | 0.9126 | 0.9164 | 0.9184 | 0.9094 | | 1.6828 | 250 | 18.2288 | 16.8751 | 0.9063 | 0.9071 | 0.9100 | 0.9023 | | 2.0194 | 300 | 18.0425 | 16.6981 | 0.9020 | 0.9032 | 0.9045 | 0.8990 | | 2.3559 | 350 | 17.9458 | 16.6155 | 0.9037 | 0.9013 | 0.9022 | 0.8984 | | 2.6925 | 400 | 17.8525 | 16.5536 | 0.8978 | 0.8971 | 0.8974 | 0.8948 | | 3.0290 | 450 | 17.7529 | 16.5136 | 0.8980 | 0.8956 | 0.8953 | 0.8951 | | 3.3656 | 500 | 17.6709 | 16.4824 | 0.8932 | 0.8914 | 0.8928 | 0.8907 | | 3.7021 | 550 | 17.5348 | 16.4632 | 0.8863 | 0.8858 | 0.8859 | 0.8849 | | 4.0387 | 600 | 17.4198 | 16.4601 | 0.8852 | 0.8862 | 0.8859 | 0.8839 | | 4.3753 | 650 | 17.3673 | 16.4405 | 0.8854 | 0.8864 | 0.8865 | 0.8842 | | 4.7118 | 700 | 17.2603 | 16.4356 | 0.8835 | 0.8838 | 0.8838 | 0.8807 | | 5.0484 | 750 | 17.1807 | 16.4443 | 0.8850 | 0.8864 | 0.8859 | 0.8838 | | 5.3849 | 800 | 17.1629 | 16.4202 | 0.8848 | 0.8862 | 0.8867 | 0.8842 | | 5.7215 | 850 | 17.0747 | 16.4162 | 0.8854 | 0.8875 | 0.8869 | 0.8837 | | 6.0581 | 900 | 17.0161 | 16.4192 | 0.8852 | 0.8863 | 0.8856 | 0.8856 | | 6.3946 | 950 | 17.0146 | 16.4033 | 0.8849 | 0.8854 | 0.8856 | 0.8844 | | 6.7312 | 1000 | 16.9393 | 16.4053 | 0.8829 | 0.8839 | 0.8848 | 0.8835 | | 7.0677 | 1050 | 16.899 | 16.4162 | 0.8826 | 0.8829 | 0.8833 | 0.8818 | | 7.4043 | 1100 | 16.9112 | 16.4051 | 0.8829 | 0.8835 | 0.8833 | 0.8820 | | 7.7408 | 1150 | 16.8508 | 16.4044 | 0.8822 | 0.8825 | 0.8830 | 0.8820 | | 8.0774 | 1200 | 16.8104 | 16.4063 | 0.8816 | 0.8816 | 0.8814 | 0.8817 | | 8.4140 | 1250 | 16.8212 | 16.4040 | 0.8835 | 0.8822 | 0.8822 | 0.8820 | | 8.7505 | 1300 | 16.7743 | 16.3934 | 0.8822 | 0.8824 | 0.8817 | 0.8810 | | 9.0871 | 1350 | 16.7383 | 16.3963 | 0.8810 | 0.8820 | 0.8807 | 0.8800 | | 9.4236 | 1400 | 16.743 | 16.4067 | 0.8819 | 0.8822 | 0.8819 | 0.8798 | | 9.7602 | 1450 | 16.7047 | 16.3959 | 0.8804 | 0.8810 | 0.8810 | 0.8797 | | 10.0968 | 1500 | 16.6782 | 16.3986 | 0.8788 | 0.8791 | 0.8796 | 0.8784 | | 10.4333 | 1550 | 16.6708 | 16.4016 | 0.8794 | 0.8792 | 0.8797 | 0.8791 | | 10.7699 | 1600 | 16.6485 | 16.3963 | 0.8790 | 0.8801 | 0.8791 | 0.8781 | | 11.1064 | 1650 | 16.6205 | 16.4012 | 0.8779 | 0.8787 | 0.8793 | 0.8771 | | 11.4430 | 1700 | 16.6095 | 16.4131 | 0.8786 | 0.8790 | 0.8794 | 0.8791 | | 11.7796 | 1750 | 16.5891 | 16.4070 | 0.8807 | 0.8805 | 0.8810 | 0.8801 | | 12.1161 | 1800 | 16.5619 | 16.3963 | 0.8794 | 0.8800 | 0.8797 | 0.8780 | | 12.4527 | 1850 | 16.5467 | 16.3991 | 0.8796 | 0.8806 | 0.8804 | 0.8790 | | 12.7892 | 1900 | 16.5398 | 16.3970 | 0.8792 | 0.8798 | 0.8801 | 0.8788 | | 13.1258 | 1950 | 16.5047 | 16.3964 | 0.8796 | 0.8804 | 0.8804 | 0.8788 | | 13.4623 | 2000 | 16.4985 | 16.4025 | 0.8793 | 0.8798 | 0.8807 | 0.8790 | | 13.7989 | 2050 | 16.4852 | 16.4107 | 0.8801 | 0.8810 | 0.8800 | 0.8793 | | 14.1355 | 2100 | 16.4526 | 16.3929 | 0.8797 | 0.8801 | 0.8809 | 0.8779 | | 14.4720 | 2150 | 16.4343 | 16.4075 | 0.8788 | 0.8791 | 0.8797 | 0.8774 | | 14.8086 | 2200 | 16.4244 | 16.4027 | 0.8804 | 0.8819 | 0.8820 | 0.8809 | | 15.1451 | 2250 | 16.3947 | 16.4102 | 0.8791 | 0.8792 | 0.8803 | 0.8773 | | 15.4817 | 2300 | 16.3827 | 16.4042 | 0.8804 | 0.8813 | 0.8813 | 0.8781 | | 15.8183 | 2350 | 16.3719 | 16.4003 | 0.8801 | 0.8818 | 0.8820 | 0.8791 | | 16.1548 | 2400 | 16.3403 | 16.4132 | 0.8781 | 0.8791 | 0.8799 | 0.8767 | | 16.4914 | 2450 | 16.3357 | 16.4149 | 0.8804 | 0.8809 | 0.8807 | 0.8792 | | 16.8279 | 2500 | 16.3203 | 16.4081 | 0.8804 | 0.8814 | 0.8816 | 0.8791 | | 17.1645 | 2550 | 16.2986 | 16.4139 | 0.8798 | 0.8800 | 0.8820 | 0.8791 | | 17.5011 | 2600 | 16.2923 | 16.4062 | 0.8786 | 0.8792 | 0.8799 | 0.8768 | | 17.8376 | 2650 | 16.2649 | 16.4106 | 0.8800 | 0.8807 | 0.8814 | 0.8787 | | 18.1742 | 2700 | 16.2505 | 16.4188 | 0.8786 | 0.8793 | 0.8803 | 0.8771 | | 18.5107 | 2750 | 16.226 | 16.4149 | 0.8771 | 0.8781 | 0.8780 | 0.8766 | | 18.8473 | 2800 | 16.2106 | 16.4230 | 0.8780 | 0.8799 | 0.8791 | 0.8767 | | 19.1838 | 2850 | 16.2052 | 16.4351 | 0.8770 | 0.8777 | 0.8785 | 0.8745 | | 19.5204 | 2900 | 16.186 | 16.4331 | 0.8777 | 0.8793 | 0.8792 | 0.8762 | | 19.8570 | 2950 | 16.1496 | 16.4377 | 0.8774 | 0.8781 | 0.8780 | 0.8771 | | 20.1935 | 3000 | 16.151 | 16.4407 | 0.8766 | 0.8780 | 0.8780 | 0.8751 | | 20.5301 | 3050 | 16.1081 | 16.4426 | 0.8759 | 0.8775 | 0.8774 | 0.8749 | | 20.8666 | 3100 | 16.0864 | 16.4412 | 0.8774 | 0.8781 | 0.8787 | 0.8746 | | 21.2032 | 3150 | 16.0934 | 16.4547 | 0.8768 | 0.8783 | 0.8794 | 0.8746 | | 21.5398 | 3200 | 16.0382 | 16.4589 | 0.8742 | 0.8752 | 0.8766 | 0.8723 | | 21.8763 | 3250 | 16.0279 | 16.4668 | 0.8752 | 0.8766 | 0.8773 | 0.8728 | | 22.2129 | 3300 | 16.0327 | 16.4737 | 0.8742 | 0.8768 | 0.8773 | 0.8727 | | 22.5494 | 3350 | 15.979 | 16.4686 | 0.8740 | 0.8771 | 0.8771 | 0.8722 | | 22.8860 | 3400 | 15.9622 | 16.4736 | 0.8743 | 0.8760 | 0.8765 | 0.8721 | | 23.2225 | 3450 | 15.9881 | 16.4802 | 0.8743 | 0.8757 | 0.8755 | 0.8723 | | 23.5591 | 3500 | 15.9482 | 16.4821 | 0.8725 | 0.8761 | 0.8761 | 0.8710 | ### Framework Versions - Python: 3.10.10 - Sentence Transformers: 3.0.1 - Transformers: 4.41.2 - PyTorch: 2.1.2+cu121 - Accelerate: 0.33.0 - Datasets: 2.19.1 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### TripletLoss ```bibtex @misc{hermans2017defense, title={In Defense of the Triplet Loss for Person Re-Identification}, author={Alexander Hermans and Lucas Beyer and Bastian Leibe}, year={2017}, eprint={1703.07737}, archivePrefix={arXiv}, primaryClass={cs.CV} } ```