bge-small-hotpotwa-matryoshka-fine-tuned-50/checkpoint-4000/README.md

---
base_model: BAAI/bge-small-en
datasets:
- sentence-transformers/hotpotqa
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy
- dot_accuracy
- manhattan_accuracy
- euclidean_accuracy
- max_accuracy
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:76064
- loss:MatryoshkaLoss
- loss:TripletLoss
widget:
- source_sentence: When was the brewery founded by the family who resided in the Lemp
    Mansion founded?
  sentences:
  - Latrobe Brewing Company Latrobe Brewing Company in Latrobe, Pennsylvania, founded
    in 1839, was one of the largest breweries in the United States and the maker of
    "Rolling Rock" beer (famous for its small green bottles). It was purchased by
    Labatt Brewing Company in 1987, which in turn was purchased in 1995 by the Belgian
    brewing conglomerate corporation Interbrew, which merged later into InBev in 2004.
  - Lemp Mansion The Lemp Mansion (3322 DeMenil Place, St. Louis, Missouri) is a historical
    house in Benton Park, St. Louis, Missouri. It is also the site of four suicides
    by Lemp family members after the death of the son Frederick Lemp, whose William
    J. Lemp Brewing Co. dominated the St. Louis beer market before Prohibition with
    its Falstaff beer brand. The mansion is said to be haunted by members of the Lemp
    family.
  - 'Man of the House (song) "Man of the House" is a song co-written and recorded
    by American country music artist Chuck Wicks. It was released in January 2009
    as the third single from the album "Starting Now". The song reached #27 on the
    "Billboard" Hot Country Songs chart. The song was written by Wicks and Michael
    Mobley.'
- source_sentence: Who is the translator and researcher who served as a dean at Hartford
    Junior College and translated her Belgian-French partner's 1968 work into English?
  sentences:
  - William L. Shirer William Lawrence Shirer (February 23, 1904 – December 28, 1993)
    was an American journalist and war correspondent. He wrote "The Rise and Fall
    of the Third Reich", a history of Nazi Germany that has been read by many and
    cited in scholarly works for more than 50 years. Originally a foreign correspondent
    for the "Chicago Tribune" and the International News Service, Shirer was the first
    reporter hired by Edward R. Murrow for what would become a CBS radio team of journalists
    known as "Murrow's Boys". He became known for his broadcasts from Berlin, from
    the rise of the Nazi dictatorship through the first year of World War II (1940).
    With Murrow, he organized the first broadcast world news roundup, a format still
    followed by news broadcasts.
  - 'The Abyss (Yourcenar novel) The Abyss (French: L''Œuvre au noir ) is a 1968 novel
    by the Belgian-French writer Marguerite Yourcenar. Its narrative centers on the
    life and death of Zeno, a physician, philosopher, scientist and alchemist born
    in Bruges during the Renaissance era. The book was published in France in 1968
    and was met with immediate popular interest as well as critical acclaim, obtaining
    the Prix Femina with unanimous votes the year of its publication. The English
    translation by Grace Frick has been published under the title "The Abyss" or alternatively
    Zeno of Bruges. Belgian filmmaker André Delvaux adapted it into a film in 1988.'
  - Pierre L. van den Berghe Pierre L. van den Berghe (born 1933) is professor emeritus
    of sociology and anthropology at the University of Washington, where he has worked
    since 1965. Born in the Belgian Congo to Belgian parents, and spending World War
    II in occupied Belgium, he was an early witness to ethnic conflict and racism,
    which eventually led him to become a leading authority on ethnic relations. He
    has conducted field work in South Africa, Mexico, Guatemala, Iran, Lebanon, Nigeria,
    Peru, and Israel. Early in his career, he lectured at the University of Natal
    alongside Leo Kuper and Fatima Meer. A student of Talcott Parsons at Harvard (receiving
    the Ph.D. in 1960), he nevertheless had little interest in structural functionalism
    and was one of the first proponents of sociobiological approaches to social phenomena.
- source_sentence: Them Crooked Vultures and The Vines are both part of what music
    genres?
  sentences:
  - Friendly Center Friendly Center is a large, open-air shopping center located in
    northwestern Greensboro, North Carolina, near the intersection of Wendover Avenue
    and Friendly Avenue. The shopping center opened in August 1957, and with its inward
    orientation, Friendly Center could be classified as an outdoor lifestyle center.
    Its anchor tenants include Belk, Macy's, and Sears. Other tenants include Barnes
    & Noble, Old Navy, The Grande Theatre is a 16-screen multiplex cinema operated
    by Regal Cinemas. It also contains Harris Teeter's flagship supermarket location
    encompassing 72,000 square feet (6,700 m2) and Whole Foods Market. There are specialty
    "foodie" stores tucked away in the back corner by Harris Teeter such as the Savory
    Spice Shop and Midtown Olive Oil. It features a number of national retailers such
    as Banana Republic, Victoria's Secret, The Limited, Bath & Body Works, Express,
    The GAP, Eddie Bauer, Talbots, Birkenstock Feet First, Pier 1, and New York &
    Company.
  - Twin Wild Twin Wild is a British four-piece alternative rock band. Formed in 2012,
    the band is made up of the collective creative energies of Richard Hutchison (vocals,
    guitars), Imran Mair (drums), David Cuzner (guitars) and Edward Thomas (bass).
    In 2014, the band self-released their track "Fears", which garnered over half
    a million plays on Soundcloud and charted in Spotify’s Top 20 viral chart. Their
    style of music has been compared to the likes of Foals, The Neighbourhood, and
    Bastille. The band have been hailed by Edith Bowman as "The love child of Bastille
    and Biffy Clyro".
  - Them Crooked Vultures Them Crooked Vultures is a rock supergroup formed in Los
    Angeles in 2009 by John Paul Jones (former member of Led Zeppelin) on bass and
    keyboards, Dave Grohl (of Foo Fighters and formerly of Nirvana) on drums and backing
    vocals, and Josh Homme (of Queens of the Stone Age, Eagles of Death Metal and
    formerly of Kyuss) on guitar and vocals. The group also includes guitarist Alain
    Johannes during live performances. The band began recording in February 2009,
    and performed their first gig on August 9, 2009, in Chicago, followed by a European
    debut on August 19. On October 1 the group embarked on a worldwide tour titled
    "Deserve the Future" with dates going into 2010. The band's first single "New
    Fang" was released in October 2009, followed by the group's self-titled debut
    album the following month, debuting at number 12 on the "Billboard" 200. The group
    won the 2011 Grammy Award for Best Hard Rock Performance for "New Fang".
- source_sentence: What type of collection does Nådens år and Agnetha Fältskog have
    in common?
  sentences:
  - Gardiner Island (Nunavut) Gardiner Island is one of the many uninhabited Canadian
    arctic islands in Qikiqtaaluk Region, Nunavut. It is a Baffin Island offshore
    island located in Frobisher Bay south of the capital city of Iqaluit.
  - Nådens år Nådens år (The Year of Grace) is the third studio album by the Swedish
    rock artist Ulf Lundell. It was released in April 1978 on EMI and Parlophone.
    It was recorded in EMI Studio, Stockholm, and produced by Kjell Andersson and
    Lundell. It includes "Snön faller och vi med den" ("The snow is falling and we
    are too"), one of Lundell's more famous songs. Agnetha Fältskog is involved in
    the song. The cover picture shows Lundell sitting on a rock next to a dog and
    was taken in Åre in 1977. "Nådens år" achieved Gold status in Sweden.
  - Åsa Elzén Åsa Elzén is an artist whose work is informed by feminist theory, intersectionality
    and post-colonialism. Elzén was born in Sweden in 1972 and currently lives and
    works in Berlin.
- source_sentence: Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with the
    biography of a 19th century actor born in which year ?
  sentences:
  - Herbert Campbell Herbert Campbell (22 December 1844 – 19 July 1904) born Herbert
    Edward Story was an English comedian and actor who appeared in music hall, Victorian
    burlesques and musical comedies during the Victorian era. He was famous for starring,
    for forty years, in the Theatre Royal, Drury Lane's annual Christmas pantomimes,
    predominantly as a dame.
  - Leptinella Leptinella is a genus of alpine flowering plant in the Asteraceae family,
    comprising 33 species, distributed in New Guinea, Australia, New Zealand, South
    Africa, and South America. Many of the species are endemic to New Zealand.
  - Red Velvet (play) Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with
    the biography of the 19th century actor Ira Aldridge and his taking the role of
    "Othello". It premiered at the Tricycle Theatre (directed by its new artistic
    director Indhu Rubasingham) from 11 October to 24 November 2012, with Aldridge
    played by Adrian Lester.
model-index:
- name: BGE-base-en-v1.5-Hotpotqa
  results:
  - task:
      type: triplet
      name: Triplet
    dataset:
      name: dim 384
      type: dim_384
    metrics:
    - type: cosine_accuracy
      value: 0.8698532891623284
      name: Cosine Accuracy
    - type: dot_accuracy
      value: 0.13014671083767157
      name: Dot Accuracy
    - type: manhattan_accuracy
      value: 0.8737576904874585
      name: Manhattan Accuracy
    - type: euclidean_accuracy
      value: 0.8698532891623284
      name: Euclidean Accuracy
    - type: max_accuracy
      value: 0.8737576904874585
      name: Max Accuracy
  - task:
      type: triplet
      name: Triplet
    dataset:
      name: dim 256
      type: dim_256
    metrics:
    - type: cosine_accuracy
      value: 0.86819687647894
      name: Cosine Accuracy
    - type: dot_accuracy
      value: 0.1319214387127307
      name: Dot Accuracy
    - type: manhattan_accuracy
      value: 0.8762423095125415
      name: Manhattan Accuracy
    - type: euclidean_accuracy
      value: 0.8680785612872692
      name: Euclidean Accuracy
    - type: max_accuracy
      value: 0.8762423095125415
      name: Max Accuracy
  - task:
      type: triplet
      name: Triplet
    dataset:
      name: dim 128
      type: dim_128
    metrics:
    - type: cosine_accuracy
      value: 0.8663038334122102
      name: Cosine Accuracy
    - type: dot_accuracy
      value: 0.1386654046379555
      name: Dot Accuracy
    - type: manhattan_accuracy
      value: 0.8742309512541411
      name: Manhattan Accuracy
    - type: euclidean_accuracy
      value: 0.865712257453857
      name: Euclidean Accuracy
    - type: max_accuracy
      value: 0.8742309512541411
      name: Max Accuracy
  - task:
      type: triplet
      name: Triplet
    dataset:
      name: dim 64
      type: dim_64
    metrics:
    - type: cosine_accuracy
      value: 0.8634642688121155
      name: Cosine Accuracy
    - type: dot_accuracy
      value: 0.1508518693800284
      name: Dot Accuracy
    - type: manhattan_accuracy
      value: 0.8728111689540937
      name: Manhattan Accuracy
    - type: euclidean_accuracy
      value: 0.8637008991954567
      name: Euclidean Accuracy
    - type: max_accuracy
      value: 0.8728111689540937
      name: Max Accuracy
---

# BGE-base-en-v1.5-Hotpotqa

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-small-en](https://huggingface.co./BAAI/bge-small-en) on the [sentence-transformers/hotpotqa](https://huggingface.co./datasets/sentence-transformers/hotpotqa) dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-small-en](https://huggingface.co./BAAI/bge-small-en) <!-- at revision 2275a7bdee235e9b4f01fa73aa60d3311983cfea -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 384 tokens
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
    - [sentence-transformers/hotpotqa](https://huggingface.co./datasets/sentence-transformers/hotpotqa)
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with the biography of a 19th century actor born in which year ?',
    'Red Velvet (play) Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with the biography of the 19th century actor Ira Aldridge and his taking the role of "Othello". It premiered at the Tricycle Theatre (directed by its new artistic director Indhu Rubasingham) from 11 October to 24 November 2012, with Aldridge played by Adrian Lester.',
    "Herbert Campbell Herbert Campbell (22 December 1844 – 19 July 1904) born Herbert Edward Story was an English comedian and actor who appeared in music hall, Victorian burlesques and musical comedies during the Victorian era. He was famous for starring, for forty years, in the Theatre Royal, Drury Lane's annual Christmas pantomimes, predominantly as a dame.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Triplet
* Dataset: `dim_384`
* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| **cosine_accuracy** | **0.8699** |
| dot_accuracy        | 0.1301     |
| manhattan_accuracy  | 0.8738     |
| euclidean_accuracy  | 0.8699     |
| max_accuracy        | 0.8738     |

#### Triplet
* Dataset: `dim_256`
* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| **cosine_accuracy** | **0.8682** |
| dot_accuracy        | 0.1319     |
| manhattan_accuracy  | 0.8762     |
| euclidean_accuracy  | 0.8681     |
| max_accuracy        | 0.8762     |

#### Triplet
* Dataset: `dim_128`
* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| **cosine_accuracy** | **0.8663** |
| dot_accuracy        | 0.1387     |
| manhattan_accuracy  | 0.8742     |
| euclidean_accuracy  | 0.8657     |
| max_accuracy        | 0.8742     |

#### Triplet
* Dataset: `dim_64`
* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| **cosine_accuracy** | **0.8635** |
| dot_accuracy        | 0.1509     |
| manhattan_accuracy  | 0.8728     |
| euclidean_accuracy  | 0.8637     |
| max_accuracy        | 0.8728     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### sentence-transformers/hotpotqa

* Dataset: [sentence-transformers/hotpotqa](https://huggingface.co./datasets/sentence-transformers/hotpotqa) at [f07d3cd](https://huggingface.co./datasets/sentence-transformers/hotpotqa/tree/f07d3cd2d290ea2e83ed35e33d67d6a4658b8786)
* Size: 76,064 training samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                            | positive                                                                             | negative                                                                            |
  |:--------|:----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                            | string                                                                               | string                                                                              |
  | details | <ul><li>min: 8 tokens</li><li>mean: 24.08 tokens</li><li>max: 95 tokens</li></ul> | <ul><li>min: 23 tokens</li><li>mean: 100.12 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 88.02 tokens</li><li>max: 512 tokens</li></ul> |
* Samples:
  | anchor                                                                                           | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | negative                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
  |:-------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>What nationality was the player named MVP in 2017 World Baseball Classic – Pool C ?</code> | <code>2017 World Baseball Classic – Pool C Pool C of the First Round of the 2017 World Baseball Classic was held at Marlins Park, Miami, Florida, United States, from March 9 to 12, 2017, between Canada, Colombia, the Dominican Republic, and the United States. Pool C was a round-robin tournament. Each team played the other three teams once, with the top two teams – the Dominican Republic and the United States – advancing to Pool F, one of two second-round pools. Manny Machado of the Dominican Republic was named MVP for the first-round Pool C bracket of the WBC, after batting .357.</code> | <code>2017 World Baseball Classic – Qualifier 2 Qualifier 2 of the Qualifying Round of the 2017 World Baseball Classic was held at Estadio B'Air, Mexicali, Mexico from March 17 to 20, 2016.</code>                                                                                                                                                                                                                                                                                                                                                                                                                          |
  | <code>Karl Kraepelin specialized in the study of what predatory arachnids?</code>                | <code>Karl Kraepelin Karl Matthias Friedrich Magnus Kraepelin (14 December 1848 Neustrelitz – 28 June 1915 Hamburg), was a German naturalist who specialised in the study of scorpions, centipedes, spiders and solfugids, and was noted for his monograph ""Scorpiones und Pedipalpi"" (Berlin) in 1899, which was an exhaustive survey of the taxonomy of the Order Scorpiones. From 1889–1914 he was Director of the "Naturhistorisches Museum Hamburg ", which was destroyed during World War II, and worked on myriapods from 1901–1916.</code>                                                              | <code>Teuthology Teuthology (from Greek "τεῦθος" , "cuttlefish, squid", and -λογία , "-logia") is the study of cephalopods.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
  | <code>Who directed the 1990 American crime film in which Vito Pick me played a bit part?</code>  | <code>Vito Picone Vito Picone (born March 20, 1941) is the lead singer of The Elegants, and along with Jimmy Mochella is a remaining original member. He has also played bit parts in "Goodfellas", "Analyze This", and "The Sopranos".</code>                                                                                                                                                                                                                                                                                                                                                                    | <code>The Rookie (1990 film) The Rookie is a 1990 American buddy cop film directed by Clint Eastwood and produced by Howard G. Kazanjian, Steven Siebert and David Valdes. It was written from a screenplay conceived by Boaz Yakin and Scott Spiegel. The film stars Charlie Sheen, Clint Eastwood, Raúl Juliá, Sônia Braga, Lara Flynn Boyle, and Tom Skerritt. Eastwood plays a veteran police officer teamed up with a younger detective played by Sheen ("the rookie"), whose intent is to take down a German crime lord in downtown Los Angeles following months of investigation into an exotic car theft ring.</code> |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "TripletLoss",
      "matryoshka_dims": [
          384,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Evaluation Dataset

#### sentence-transformers/hotpotqa

* Dataset: [sentence-transformers/hotpotqa](https://huggingface.co./datasets/sentence-transformers/hotpotqa) at [f07d3cd](https://huggingface.co./datasets/sentence-transformers/hotpotqa/tree/f07d3cd2d290ea2e83ed35e33d67d6a4658b8786)
* Size: 8,452 evaluation samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                              | positive                                                                             | negative                                                                            |
  |:--------|:------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                              | string                                                                               | string                                                                              |
  | details | <ul><li>min: 10 tokens</li><li>mean: 24.53 tokens</li><li>max: 114 tokens</li></ul> | <ul><li>min: 19 tokens</li><li>mean: 103.87 tokens</li><li>max: 407 tokens</li></ul> | <ul><li>min: 17 tokens</li><li>mean: 88.32 tokens</li><li>max: 356 tokens</li></ul> |
* Samples:
  | anchor                                                                                                                                                                                                                     | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | negative                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
  |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>Which actress, known for her role as Harper Munroe on the MTV comedy series "Happyland", starred alongside Laura Marano, Parker Mack, Michelle Clunie and Kathleen Wilhoite in the film A Sort of Homecoming?</code> | <code>A Sort of Homecoming (film) A Sort of Homecoming is an American drama directed by Maria Burton, her fifth feature film. The films stars Katherine McNamara, Laura Marano, Parker Mack, Michelle Clunie and Kathleen Wilhoite. The film premiered March 14, 2015 at the Omaha Film Festival.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | <code>Nellie Bellflower Nellie Bellflower (born May 1, 1946 in Phoenix, Arizona) is an American actress and voice artist who provided the voice of Princess Ariel in the Ruby-Spears animated television series "Thundarr the Barbarian". She has also been in "The Last Unicorn" (voice), Rankin/Bass "The Return of the King", "Americathon", the miniseries "East of Eden", and guest roles on various TV shows such as "Barnaby Jones", "Barney Miller", "Starsky and Hutch", and "Happy Days" as Fonzie's ex-fiancée Maureen Johnson, a.k.a. "The Lone Stripper", in the Season 2 episode of the series titled "Fonzie's Getting Married" (episode #13). Nellie has been involved in movie production with three projects: "The Girl in Melanie Klein" (2008), "Miss Pettigrew Lives for a Day" (2008) and "Finding Neverland" (2004), for which she was nominated for an Academy Award as Producer for Best Picture. She is married to Michael Mislove.</code> |
  | <code>Between Pizza Fusion and Pizzeria Venti, which restaurant emphasizes organic ingredients and green building methods?</code>                                                                                          | <code>Pizza Fusion Pizza Fusion is a Deerfield Beach, Florida-based pizza restaurant chain. Using mostly organic ingredients and emphasizing green building methods, the restaurants operate under the tagline Saving the Earth, One Pizza at a Time.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | <code>Pizza Schmizza Pizza Schmizza is an American pizza chain with 23 locations throughout the Portland, Oregon area, and two in southern Oregon. Pizza Schmizza, primarily selling thin crust pizza by-the-slice.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
  | <code>What company distributed the stop motion spin-off special "The Year Without a Santa Claus," which aired on December 10, 1974?</code>                                                                                 | <code>A Miser Brothers' Christmas A Miser Brothers' Christmas is a stop motion spin-off special based on some of the characters from the 1974 Rankin-Bass special "The Year Without a Santa Claus". Distributed by Warner Bros. Animation under their Warner Premiere label (the rights holders of the post-1974 Rankin-Bass library) and Toronto-based Cuppa Coffee Studios, the one-hour special premiered on ABC Family on Saturday, December 13, 2008, during the network's annual The 25 Days of Christmas programming. Mickey Rooney and George S. Irving reprised their respective roles as Santa Claus and Heat Miser at ages 88 and 86. Snow Miser, originally portrayed by Dick Shawn who died in 1987, was voiced by Juan Chioran, while Mrs. Claus, voiced by Shirley Booth in the original, was portrayed by Catherine Disher (because Booth had died in 1992). The movie aimed to emulate the Rankin/Bass animation style. This is the last Christmas special to feature Mickey Rooney as Santa Claus, as he died in 2014, as well as the last time George Irving voiced Heat Miser, as he died in 2016.</code> | <code>Holidaze: The Christmas That Almost Didn't Happen Holidaze: The Christmas That Almost Didn't Happen is an American stop-motion animated Christmas television special directed by David H. Brooks, that originally aired in 2006 and produced by BixPix Entertainment, Once Upon a Frog and Madison Street Entertainment. The show's plot has Rusty Reindeer (Fred Savage) the brother of Rudolph the Red Nosed Reindeer joining a support group for depressed holiday icons, and he and the other characters search for the meaning of Christmas and help a young boy (Dylan and Cole Sprouse) to get on Santa's nice list. Rusty's cohorts include Candie, the Easter Bunny (Gladys Knight); Mr. C, the grouchy cherub (Paul Rodriguez); Albert, the Thanksgiving Turkey (Harland Williams); And Trick and Treat (Brenda Song and Emily Osment) the teenage Halloween Ghosts.</code>                                                                          |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "TripletLoss",
      "matryoshka_dims": [
          384,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 32
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 50
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `resume_from_checkpoint`: bge-small-hotpotwa-matryoshka
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 32
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 50
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: True
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: bge-small-hotpotwa-matryoshka
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch   | Step | Training Loss | loss    | dim_128_cosine_accuracy | dim_256_cosine_accuracy | dim_384_cosine_accuracy | dim_64_cosine_accuracy |
|:-------:|:----:|:-------------:|:-------:|:-----------------------:|:-----------------------:|:-----------------------:|:----------------------:|
| 0.3366  | 50   | 19.5758       | 19.3933 | 0.9552                  | 0.9663                  | 0.9668                  | 0.9359                 |
| 0.6731  | 100  | 19.4573       | 19.0971 | 0.9571                  | 0.9646                  | 0.9653                  | 0.9450                 |
| 1.0097  | 150  | 19.1409       | 18.4070 | 0.9385                  | 0.9434                  | 0.9473                  | 0.9307                 |
| 1.3462  | 200  | 18.6431       | 17.3292 | 0.9126                  | 0.9164                  | 0.9184                  | 0.9094                 |
| 1.6828  | 250  | 18.2288       | 16.8751 | 0.9063                  | 0.9071                  | 0.9100                  | 0.9023                 |
| 2.0194  | 300  | 18.0425       | 16.6981 | 0.9020                  | 0.9032                  | 0.9045                  | 0.8990                 |
| 2.3559  | 350  | 17.9458       | 16.6155 | 0.9037                  | 0.9013                  | 0.9022                  | 0.8984                 |
| 2.6925  | 400  | 17.8525       | 16.5536 | 0.8978                  | 0.8971                  | 0.8974                  | 0.8948                 |
| 3.0290  | 450  | 17.7529       | 16.5136 | 0.8980                  | 0.8956                  | 0.8953                  | 0.8951                 |
| 3.3656  | 500  | 17.6709       | 16.4824 | 0.8932                  | 0.8914                  | 0.8928                  | 0.8907                 |
| 3.7021  | 550  | 17.5348       | 16.4632 | 0.8863                  | 0.8858                  | 0.8859                  | 0.8849                 |
| 4.0387  | 600  | 17.4198       | 16.4601 | 0.8852                  | 0.8862                  | 0.8859                  | 0.8839                 |
| 4.3753  | 650  | 17.3673       | 16.4405 | 0.8854                  | 0.8864                  | 0.8865                  | 0.8842                 |
| 4.7118  | 700  | 17.2603       | 16.4356 | 0.8835                  | 0.8838                  | 0.8838                  | 0.8807                 |
| 5.0484  | 750  | 17.1807       | 16.4443 | 0.8850                  | 0.8864                  | 0.8859                  | 0.8838                 |
| 5.3849  | 800  | 17.1629       | 16.4202 | 0.8848                  | 0.8862                  | 0.8867                  | 0.8842                 |
| 5.7215  | 850  | 17.0747       | 16.4162 | 0.8854                  | 0.8875                  | 0.8869                  | 0.8837                 |
| 6.0581  | 900  | 17.0161       | 16.4192 | 0.8852                  | 0.8863                  | 0.8856                  | 0.8856                 |
| 6.3946  | 950  | 17.0146       | 16.4033 | 0.8849                  | 0.8854                  | 0.8856                  | 0.8844                 |
| 6.7312  | 1000 | 16.9393       | 16.4053 | 0.8829                  | 0.8839                  | 0.8848                  | 0.8835                 |
| 7.0677  | 1050 | 16.899        | 16.4162 | 0.8826                  | 0.8829                  | 0.8833                  | 0.8818                 |
| 7.4043  | 1100 | 16.9112       | 16.4051 | 0.8829                  | 0.8835                  | 0.8833                  | 0.8820                 |
| 7.7408  | 1150 | 16.8508       | 16.4044 | 0.8822                  | 0.8825                  | 0.8830                  | 0.8820                 |
| 8.0774  | 1200 | 16.8104       | 16.4063 | 0.8816                  | 0.8816                  | 0.8814                  | 0.8817                 |
| 8.4140  | 1250 | 16.8212       | 16.4040 | 0.8835                  | 0.8822                  | 0.8822                  | 0.8820                 |
| 8.7505  | 1300 | 16.7743       | 16.3934 | 0.8822                  | 0.8824                  | 0.8817                  | 0.8810                 |
| 9.0871  | 1350 | 16.7383       | 16.3963 | 0.8810                  | 0.8820                  | 0.8807                  | 0.8800                 |
| 9.4236  | 1400 | 16.743        | 16.4067 | 0.8819                  | 0.8822                  | 0.8819                  | 0.8798                 |
| 9.7602  | 1450 | 16.7047       | 16.3959 | 0.8804                  | 0.8810                  | 0.8810                  | 0.8797                 |
| 10.0968 | 1500 | 16.6782       | 16.3986 | 0.8788                  | 0.8791                  | 0.8796                  | 0.8784                 |
| 10.4333 | 1550 | 16.6708       | 16.4016 | 0.8794                  | 0.8792                  | 0.8797                  | 0.8791                 |
| 10.7699 | 1600 | 16.6485       | 16.3963 | 0.8790                  | 0.8801                  | 0.8791                  | 0.8781                 |
| 11.1064 | 1650 | 16.6205       | 16.4012 | 0.8779                  | 0.8787                  | 0.8793                  | 0.8771                 |
| 11.4430 | 1700 | 16.6095       | 16.4131 | 0.8786                  | 0.8790                  | 0.8794                  | 0.8791                 |
| 11.7796 | 1750 | 16.5891       | 16.4070 | 0.8807                  | 0.8805                  | 0.8810                  | 0.8801                 |
| 12.1161 | 1800 | 16.5619       | 16.3963 | 0.8794                  | 0.8800                  | 0.8797                  | 0.8780                 |
| 12.4527 | 1850 | 16.5467       | 16.3991 | 0.8796                  | 0.8806                  | 0.8804                  | 0.8790                 |
| 12.7892 | 1900 | 16.5398       | 16.3970 | 0.8792                  | 0.8798                  | 0.8801                  | 0.8788                 |
| 13.1258 | 1950 | 16.5047       | 16.3964 | 0.8796                  | 0.8804                  | 0.8804                  | 0.8788                 |
| 13.4623 | 2000 | 16.4985       | 16.4025 | 0.8793                  | 0.8798                  | 0.8807                  | 0.8790                 |
| 13.7989 | 2050 | 16.4852       | 16.4107 | 0.8801                  | 0.8810                  | 0.8800                  | 0.8793                 |
| 14.1355 | 2100 | 16.4526       | 16.3929 | 0.8797                  | 0.8801                  | 0.8809                  | 0.8779                 |
| 14.4720 | 2150 | 16.4343       | 16.4075 | 0.8788                  | 0.8791                  | 0.8797                  | 0.8774                 |
| 14.8086 | 2200 | 16.4244       | 16.4027 | 0.8804                  | 0.8819                  | 0.8820                  | 0.8809                 |
| 15.1451 | 2250 | 16.3947       | 16.4102 | 0.8791                  | 0.8792                  | 0.8803                  | 0.8773                 |
| 15.4817 | 2300 | 16.3827       | 16.4042 | 0.8804                  | 0.8813                  | 0.8813                  | 0.8781                 |
| 15.8183 | 2350 | 16.3719       | 16.4003 | 0.8801                  | 0.8818                  | 0.8820                  | 0.8791                 |
| 16.1548 | 2400 | 16.3403       | 16.4132 | 0.8781                  | 0.8791                  | 0.8799                  | 0.8767                 |
| 16.4914 | 2450 | 16.3357       | 16.4149 | 0.8804                  | 0.8809                  | 0.8807                  | 0.8792                 |
| 16.8279 | 2500 | 16.3203       | 16.4081 | 0.8804                  | 0.8814                  | 0.8816                  | 0.8791                 |
| 17.1645 | 2550 | 16.2986       | 16.4139 | 0.8798                  | 0.8800                  | 0.8820                  | 0.8791                 |
| 17.5011 | 2600 | 16.2923       | 16.4062 | 0.8786                  | 0.8792                  | 0.8799                  | 0.8768                 |
| 17.8376 | 2650 | 16.2649       | 16.4106 | 0.8800                  | 0.8807                  | 0.8814                  | 0.8787                 |
| 18.1742 | 2700 | 16.2505       | 16.4188 | 0.8786                  | 0.8793                  | 0.8803                  | 0.8771                 |
| 18.5107 | 2750 | 16.226        | 16.4149 | 0.8771                  | 0.8781                  | 0.8780                  | 0.8766                 |
| 18.8473 | 2800 | 16.2106       | 16.4230 | 0.8780                  | 0.8799                  | 0.8791                  | 0.8767                 |
| 19.1838 | 2850 | 16.2052       | 16.4351 | 0.8770                  | 0.8777                  | 0.8785                  | 0.8745                 |
| 19.5204 | 2900 | 16.186        | 16.4331 | 0.8777                  | 0.8793                  | 0.8792                  | 0.8762                 |
| 19.8570 | 2950 | 16.1496       | 16.4377 | 0.8774                  | 0.8781                  | 0.8780                  | 0.8771                 |
| 20.1935 | 3000 | 16.151        | 16.4407 | 0.8766                  | 0.8780                  | 0.8780                  | 0.8751                 |
| 20.5301 | 3050 | 16.1081       | 16.4426 | 0.8759                  | 0.8775                  | 0.8774                  | 0.8749                 |
| 20.8666 | 3100 | 16.0864       | 16.4412 | 0.8774                  | 0.8781                  | 0.8787                  | 0.8746                 |
| 21.2032 | 3150 | 16.0934       | 16.4547 | 0.8768                  | 0.8783                  | 0.8794                  | 0.8746                 |
| 21.5398 | 3200 | 16.0382       | 16.4589 | 0.8742                  | 0.8752                  | 0.8766                  | 0.8723                 |
| 21.8763 | 3250 | 16.0279       | 16.4668 | 0.8752                  | 0.8766                  | 0.8773                  | 0.8728                 |
| 22.2129 | 3300 | 16.0327       | 16.4737 | 0.8742                  | 0.8768                  | 0.8773                  | 0.8727                 |
| 22.5494 | 3350 | 15.979        | 16.4686 | 0.8740                  | 0.8771                  | 0.8771                  | 0.8722                 |
| 22.8860 | 3400 | 15.9622       | 16.4736 | 0.8743                  | 0.8760                  | 0.8765                  | 0.8721                 |
| 23.2225 | 3450 | 15.9881       | 16.4802 | 0.8743                  | 0.8757                  | 0.8755                  | 0.8723                 |
| 23.5591 | 3500 | 15.9482       | 16.4821 | 0.8725                  | 0.8761                  | 0.8761                  | 0.8710                 |
| 23.8957 | 3550 | 15.9228       | 16.4996 | 0.8726                  | 0.8748                  | 0.8751                  | 0.8709                 |
| 24.2322 | 3600 | 15.9418       | 16.4973 | 0.8709                  | 0.8728                  | 0.8734                  | 0.8699                 |
| 24.5688 | 3650 | 15.896        | 16.4985 | 0.8696                  | 0.8716                  | 0.8727                  | 0.8686                 |
| 24.9053 | 3700 | 15.8788       | 16.5172 | 0.8691                  | 0.8715                  | 0.8717                  | 0.8662                 |
| 25.2419 | 3750 | 15.9147       | 16.5062 | 0.8677                  | 0.8706                  | 0.8712                  | 0.8662                 |
| 25.5785 | 3800 | 15.857        | 16.5058 | 0.8683                  | 0.8717                  | 0.8732                  | 0.8663                 |
| 25.9150 | 3850 | 15.8291       | 16.5207 | 0.8674                  | 0.8702                  | 0.8706                  | 0.8644                 |
| 26.2516 | 3900 | 15.8802       | 16.5233 | 0.8678                  | 0.8697                  | 0.8714                  | 0.8664                 |
| 26.5881 | 3950 | 15.846        | 16.5170 | 0.8686                  | 0.8713                  | 0.8717                  | 0.8655                 |
| 26.9247 | 4000 | 15.8012       | 16.5336 | 0.8663                  | 0.8682                  | 0.8699                  | 0.8635                 |


### Framework Versions
- Python: 3.10.10
- Sentence Transformers: 3.0.1
- Transformers: 4.41.2
- PyTorch: 2.1.2+cu121
- Accelerate: 0.33.0
- Datasets: 2.19.1
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### TripletLoss
```bibtex
@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification}, 
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->