metadata

base_model: BAAI/bge-small-en
datasets:
  - sentence-transformers/hotpotqa
language:
  - en
library_name: sentence-transformers
license: apache-2.0
metrics:
  - cosine_accuracy
  - dot_accuracy
  - manhattan_accuracy
  - euclidean_accuracy
  - max_accuracy
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:76064
  - loss:MatryoshkaLoss
  - loss:TripletLoss
widget:
  - source_sentence: >-
      When was the brewery founded by the family who resided in the Lemp Mansion
      founded?
    sentences:
      - >-
        Latrobe Brewing Company Latrobe Brewing Company in Latrobe,
        Pennsylvania, founded in 1839, was one of the largest breweries in the
        United States and the maker of "Rolling Rock" beer (famous for its small
        green bottles). It was purchased by Labatt Brewing Company in 1987,
        which in turn was purchased in 1995 by the Belgian brewing conglomerate
        corporation Interbrew, which merged later into InBev in 2004.
      - >-
        Lemp Mansion The Lemp Mansion (3322 DeMenil Place, St. Louis, Missouri)
        is a historical house in Benton Park, St. Louis, Missouri. It is also
        the site of four suicides by Lemp family members after the death of the
        son Frederick Lemp, whose William J. Lemp Brewing Co. dominated the St.
        Louis beer market before Prohibition with its Falstaff beer brand. The
        mansion is said to be haunted by members of the Lemp family.
      - >-
        Man of the House (song) "Man of the House" is a song co-written and
        recorded by American country music artist Chuck Wicks. It was released
        in January 2009 as the third single from the album "Starting Now". The
        song reached #27 on the "Billboard" Hot Country Songs chart. The song
        was written by Wicks and Michael Mobley.
  - source_sentence: >-
      Who is the translator and researcher who served as a dean at Hartford
      Junior College and translated her Belgian-French partner's 1968 work into
      English?
    sentences:
      - >-
        William L. Shirer William Lawrence Shirer (February 23, 1904 – December
        28, 1993) was an American journalist and war correspondent. He wrote
        "The Rise and Fall of the Third Reich", a history of Nazi Germany that
        has been read by many and cited in scholarly works for more than 50
        years. Originally a foreign correspondent for the "Chicago Tribune" and
        the International News Service, Shirer was the first reporter hired by
        Edward R. Murrow for what would become a CBS radio team of journalists
        known as "Murrow's Boys". He became known for his broadcasts from
        Berlin, from the rise of the Nazi dictatorship through the first year of
        World War II (1940). With Murrow, he organized the first broadcast world
        news roundup, a format still followed by news broadcasts.
      - >-
        The Abyss (Yourcenar novel) The Abyss (French: L'Œuvre au noir ) is a
        1968 novel by the Belgian-French writer Marguerite Yourcenar. Its
        narrative centers on the life and death of Zeno, a physician,
        philosopher, scientist and alchemist born in Bruges during the
        Renaissance era. The book was published in France in 1968 and was met
        with immediate popular interest as well as critical acclaim, obtaining
        the Prix Femina with unanimous votes the year of its publication. The
        English translation by Grace Frick has been published under the title
        "The Abyss" or alternatively Zeno of Bruges. Belgian filmmaker André
        Delvaux adapted it into a film in 1988.
      - >-
        Pierre L. van den Berghe Pierre L. van den Berghe (born 1933) is
        professor emeritus of sociology and anthropology at the University of
        Washington, where he has worked since 1965. Born in the Belgian Congo to
        Belgian parents, and spending World War II in occupied Belgium, he was
        an early witness to ethnic conflict and racism, which eventually led him
        to become a leading authority on ethnic relations. He has conducted
        field work in South Africa, Mexico, Guatemala, Iran, Lebanon, Nigeria,
        Peru, and Israel. Early in his career, he lectured at the University of
        Natal alongside Leo Kuper and Fatima Meer. A student of Talcott Parsons
        at Harvard (receiving the Ph.D. in 1960), he nevertheless had little
        interest in structural functionalism and was one of the first proponents
        of sociobiological approaches to social phenomena.
  - source_sentence: Them Crooked Vultures and The Vines are both part of what music genres?
    sentences:
      - >-
        Friendly Center Friendly Center is a large, open-air shopping center
        located in northwestern Greensboro, North Carolina, near the
        intersection of Wendover Avenue and Friendly Avenue. The shopping center
        opened in August 1957, and with its inward orientation, Friendly Center
        could be classified as an outdoor lifestyle center. Its anchor tenants
        include Belk, Macy's, and Sears. Other tenants include Barnes & Noble,
        Old Navy, The Grande Theatre is a 16-screen multiplex cinema operated by
        Regal Cinemas. It also contains Harris Teeter's flagship supermarket
        location encompassing 72,000 square feet (6,700 m2) and Whole Foods
        Market. There are specialty "foodie" stores tucked away in the back
        corner by Harris Teeter such as the Savory Spice Shop and Midtown Olive
        Oil. It features a number of national retailers such as Banana Republic,
        Victoria's Secret, The Limited, Bath & Body Works, Express, The GAP,
        Eddie Bauer, Talbots, Birkenstock Feet First, Pier 1, and New York &
        Company.
      - >-
        Twin Wild Twin Wild is a British four-piece alternative rock band.
        Formed in 2012, the band is made up of the collective creative energies
        of Richard Hutchison (vocals, guitars), Imran Mair (drums), David Cuzner
        (guitars) and Edward Thomas (bass). In 2014, the band self-released
        their track "Fears", which garnered over half a million plays on
        Soundcloud and charted in Spotify’s Top 20 viral chart. Their style of
        music has been compared to the likes of Foals, The Neighbourhood, and
        Bastille. The band have been hailed by Edith Bowman as "The love child
        of Bastille and Biffy Clyro".
      - >-
        Them Crooked Vultures Them Crooked Vultures is a rock supergroup formed
        in Los Angeles in 2009 by John Paul Jones (former member of Led
        Zeppelin) on bass and keyboards, Dave Grohl (of Foo Fighters and
        formerly of Nirvana) on drums and backing vocals, and Josh Homme (of
        Queens of the Stone Age, Eagles of Death Metal and formerly of Kyuss) on
        guitar and vocals. The group also includes guitarist Alain Johannes
        during live performances. The band began recording in February 2009, and
        performed their first gig on August 9, 2009, in Chicago, followed by a
        European debut on August 19. On October 1 the group embarked on a
        worldwide tour titled "Deserve the Future" with dates going into 2010.
        The band's first single "New Fang" was released in October 2009,
        followed by the group's self-titled debut album the following month,
        debuting at number 12 on the "Billboard" 200. The group won the 2011
        Grammy Award for Best Hard Rock Performance for "New Fang".
  - source_sentence: >-
      What type of collection does Nådens år and Agnetha Fältskog have in
      common?
    sentences:
      - >-
        Gardiner Island (Nunavut) Gardiner Island is one of the many uninhabited
        Canadian arctic islands in Qikiqtaaluk Region, Nunavut. It is a Baffin
        Island offshore island located in Frobisher Bay south of the capital
        city of Iqaluit.
      - >-
        Nådens år Nådens år (The Year of Grace) is the third studio album by the
        Swedish rock artist Ulf Lundell. It was released in April 1978 on EMI
        and Parlophone. It was recorded in EMI Studio, Stockholm, and produced
        by Kjell Andersson and Lundell. It includes "Snön faller och vi med den"
        ("The snow is falling and we are too"), one of Lundell's more famous
        songs. Agnetha Fältskog is involved in the song. The cover picture shows
        Lundell sitting on a rock next to a dog and was taken in Åre in 1977.
        "Nådens år" achieved Gold status in Sweden.
      - >-
        Åsa Elzén Åsa Elzén is an artist whose work is informed by feminist
        theory, intersectionality and post-colonialism. Elzén was born in Sweden
        in 1972 and currently lives and works in Berlin.
  - source_sentence: >-
      Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with the
      biography of a 19th century actor born in which year ?
    sentences:
      - >-
        Herbert Campbell Herbert Campbell (22 December 1844 – 19 July 1904) born
        Herbert Edward Story was an English comedian and actor who appeared in
        music hall, Victorian burlesques and musical comedies during the
        Victorian era. He was famous for starring, for forty years, in the
        Theatre Royal, Drury Lane's annual Christmas pantomimes, predominantly
        as a dame.
      - >-
        Leptinella Leptinella is a genus of alpine flowering plant in the
        Asteraceae family, comprising 33 species, distributed in New Guinea,
        Australia, New Zealand, South Africa, and South America. Many of the
        species are endemic to New Zealand.
      - >-
        Red Velvet (play) Red Velvet is a 2012 play by Lolita Chakrabarti,
        dealing with the biography of the 19th century actor Ira Aldridge and
        his taking the role of "Othello". It premiered at the Tricycle Theatre
        (directed by its new artistic director Indhu Rubasingham) from 11
        October to 24 November 2012, with Aldridge played by Adrian Lester.
model-index:
  - name: BGE-base-en-v1.5-Hotpotqa
    results:
      - task:
          type: triplet
          name: Triplet
        dataset:
          name: dim 384
          type: dim_384
        metrics:
          - type: cosine_accuracy
            value: 0.8698532891623284
            name: Cosine Accuracy
          - type: dot_accuracy
            value: 0.13014671083767157
            name: Dot Accuracy
          - type: manhattan_accuracy
            value: 0.8737576904874585
            name: Manhattan Accuracy
          - type: euclidean_accuracy
            value: 0.8698532891623284
            name: Euclidean Accuracy
          - type: max_accuracy
            value: 0.8737576904874585
            name: Max Accuracy
      - task:
          type: triplet
          name: Triplet
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy
            value: 0.86819687647894
            name: Cosine Accuracy
          - type: dot_accuracy
            value: 0.1319214387127307
            name: Dot Accuracy
          - type: manhattan_accuracy
            value: 0.8762423095125415
            name: Manhattan Accuracy
          - type: euclidean_accuracy
            value: 0.8680785612872692
            name: Euclidean Accuracy
          - type: max_accuracy
            value: 0.8762423095125415
            name: Max Accuracy
      - task:
          type: triplet
          name: Triplet
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy
            value: 0.8663038334122102
            name: Cosine Accuracy
          - type: dot_accuracy
            value: 0.1386654046379555
            name: Dot Accuracy
          - type: manhattan_accuracy
            value: 0.8742309512541411
            name: Manhattan Accuracy
          - type: euclidean_accuracy
            value: 0.865712257453857
            name: Euclidean Accuracy
          - type: max_accuracy
            value: 0.8742309512541411
            name: Max Accuracy
      - task:
          type: triplet
          name: Triplet
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy
            value: 0.8634642688121155
            name: Cosine Accuracy
          - type: dot_accuracy
            value: 0.1508518693800284
            name: Dot Accuracy
          - type: manhattan_accuracy
            value: 0.8728111689540937
            name: Manhattan Accuracy
          - type: euclidean_accuracy
            value: 0.8637008991954567
            name: Euclidean Accuracy
          - type: max_accuracy
            value: 0.8728111689540937
            name: Max Accuracy

BGE-base-en-v1.5-Hotpotqa

This is a sentence-transformers model finetuned from BAAI/bge-small-en on the sentence-transformers/hotpotqa dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-small-en
Maximum Sequence Length: 512 tokens
Output Dimensionality: 384 tokens
Similarity Function: Cosine Similarity
Training Dataset:
- sentence-transformers/hotpotqa
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with the biography of a 19th century actor born in which year ?',
    'Red Velvet (play) Red Velvet is a 2012 play by Lolita Chakrabarti, dealing with the biography of the 19th century actor Ira Aldridge and his taking the role of "Othello". It premiered at the Tricycle Theatre (directed by its new artistic director Indhu Rubasingham) from 11 October to 24 November 2012, with Aldridge played by Adrian Lester.',
    "Herbert Campbell Herbert Campbell (22 December 1844 – 19 July 1904) born Herbert Edward Story was an English comedian and actor who appeared in music hall, Victorian burlesques and musical comedies during the Victorian era. He was famous for starring, for forty years, in the Theatre Royal, Drury Lane's annual Christmas pantomimes, predominantly as a dame.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

Dataset: dim_384
Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.8699
dot_accuracy	0.1301
manhattan_accuracy	0.8738
euclidean_accuracy	0.8699
max_accuracy	0.8738

Triplet

Dataset: dim_256
Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.8682
dot_accuracy	0.1319
manhattan_accuracy	0.8762
euclidean_accuracy	0.8681
max_accuracy	0.8762

Triplet

Dataset: dim_128
Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.8663
dot_accuracy	0.1387
manhattan_accuracy	0.8742
euclidean_accuracy	0.8657
max_accuracy	0.8742

Triplet

Dataset: dim_64
Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.8635
dot_accuracy	0.1509
manhattan_accuracy	0.8728
euclidean_accuracy	0.8637
max_accuracy	0.8728

Training Details

Training Dataset

sentence-transformers/hotpotqa

Dataset: sentence-transformers/hotpotqa at f07d3cd
Size: 76,064 training samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 8 tokens mean: 24.08 tokens max: 95 tokens	min: 23 tokens mean: 100.12 tokens max: 512 tokens	min: 15 tokens mean: 88.02 tokens max: 512 tokens

Samples:

anchor	positive	negative
`What nationality was the player named MVP in 2017 World Baseball Classic – Pool C ?`	2017 World Baseball Classic – Pool C Pool C of the First Round of the 2017 World Baseball Classic was held at Marlins Park, Miami, Florida, United States, from March 9 to 12, 2017, between Canada, Colombia, the Dominican Republic, and the United States. Pool C was a round-robin tournament. Each team played the other three teams once, with the top two teams – the Dominican Republic and the United States – advancing to Pool F, one of two second-round pools. Manny Machado of the Dominican Republic was named MVP for the first-round Pool C bracket of the WBC, after batting .357.	`2017 World Baseball Classic – Qualifier 2 Qualifier 2 of the Qualifying Round of the 2017 World Baseball Classic was held at Estadio B'Air, Mexicali, Mexico from March 17 to 20, 2016.`
`Karl Kraepelin specialized in the study of what predatory arachnids?`	Karl Kraepelin Karl Matthias Friedrich Magnus Kraepelin (14 December 1848 Neustrelitz – 28 June 1915 Hamburg), was a German naturalist who specialised in the study of scorpions, centipedes, spiders and solfugids, and was noted for his monograph ""Scorpiones und Pedipalpi"" (Berlin) in 1899, which was an exhaustive survey of the taxonomy of the Order Scorpiones. From 1889–1914 he was Director of the "Naturhistorisches Museum Hamburg ", which was destroyed during World War II, and worked on myriapods from 1901–1916.	`Teuthology Teuthology (from Greek "τεῦθος" , "cuttlefish, squid", and -λογία , "-logia") is the study of cephalopods.`
`Who directed the 1990 American crime film in which Vito Pick me played a bit part?`	`Vito Picone Vito Picone (born March 20, 1941) is the lead singer of The Elegants, and along with Jimmy Mochella is a remaining original member. He has also played bit parts in "Goodfellas", "Analyze This", and "The Sopranos".`	The Rookie (1990 film) The Rookie is a 1990 American buddy cop film directed by Clint Eastwood and produced by Howard G. Kazanjian, Steven Siebert and David Valdes. It was written from a screenplay conceived by Boaz Yakin and Scott Spiegel. The film stars Charlie Sheen, Clint Eastwood, Raúl Juliá, Sônia Braga, Lara Flynn Boyle, and Tom Skerritt. Eastwood plays a veteran police officer teamed up with a younger detective played by Sheen ("the rookie"), whose intent is to take down a German crime lord in downtown Los Angeles following months of investigation into an exotic car theft ring.

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "TripletLoss",
    "matryoshka_dims": [
        384,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Evaluation Dataset

sentence-transformers/hotpotqa

Dataset: sentence-transformers/hotpotqa at f07d3cd
Size: 8,452 evaluation samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 10 tokens mean: 24.53 tokens max: 114 tokens	min: 19 tokens mean: 103.87 tokens max: 407 tokens	min: 17 tokens mean: 88.32 tokens max: 356 tokens

Samples:

anchor	positive	negative
`Which actress, known for her role as Harper Munroe on the MTV comedy series "Happyland", starred alongside Laura Marano, Parker Mack, Michelle Clunie and Kathleen Wilhoite in the film A Sort of Homecoming?`	`A Sort of Homecoming (film) A Sort of Homecoming is an American drama directed by Maria Burton, her fifth feature film. The films stars Katherine McNamara, Laura Marano, Parker Mack, Michelle Clunie and Kathleen Wilhoite. The film premiered March 14, 2015 at the Omaha Film Festival.`	Nellie Bellflower Nellie Bellflower (born May 1, 1946 in Phoenix, Arizona) is an American actress and voice artist who provided the voice of Princess Ariel in the Ruby-Spears animated television series "Thundarr the Barbarian". She has also been in "The Last Unicorn" (voice), Rankin/Bass "The Return of the King", "Americathon", the miniseries "East of Eden", and guest roles on various TV shows such as "Barnaby Jones", "Barney Miller", "Starsky and Hutch", and "Happy Days" as Fonzie's ex-fiancée Maureen Johnson, a.k.a. "The Lone Stripper", in the Season 2 episode of the series titled "Fonzie's Getting Married" (episode #13). Nellie has been involved in movie production with three projects: "The Girl in Melanie Klein" (2008), "Miss Pettigrew Lives for a Day" (2008) and "Finding Neverland" (2004), for which she was nominated for an Academy Award as Producer for Best Picture. She is married to Michael Mislove.
`Between Pizza Fusion and Pizzeria Venti, which restaurant emphasizes organic ingredients and green building methods?`	`Pizza Fusion Pizza Fusion is a Deerfield Beach, Florida-based pizza restaurant chain. Using mostly organic ingredients and emphasizing green building methods, the restaurants operate under the tagline Saving the Earth, One Pizza at a Time.`	`Pizza Schmizza Pizza Schmizza is an American pizza chain with 23 locations throughout the Portland, Oregon area, and two in southern Oregon. Pizza Schmizza, primarily selling thin crust pizza by-the-slice.`
`What company distributed the stop motion spin-off special "The Year Without a Santa Claus," which aired on December 10, 1974?`	A Miser Brothers' Christmas A Miser Brothers' Christmas is a stop motion spin-off special based on some of the characters from the 1974 Rankin-Bass special "The Year Without a Santa Claus". Distributed by Warner Bros. Animation under their Warner Premiere label (the rights holders of the post-1974 Rankin-Bass library) and Toronto-based Cuppa Coffee Studios, the one-hour special premiered on ABC Family on Saturday, December 13, 2008, during the network's annual The 25 Days of Christmas programming. Mickey Rooney and George S. Irving reprised their respective roles as Santa Claus and Heat Miser at ages 88 and 86. Snow Miser, originally portrayed by Dick Shawn who died in 1987, was voiced by Juan Chioran, while Mrs. Claus, voiced by Shirley Booth in the original, was portrayed by Catherine Disher (because Booth had died in 1992). The movie aimed to emulate the Rankin/Bass animation style. This is the last Christmas special to feature Mickey Rooney as Santa Claus, as he died in 2014, as well as the last time George Irving voiced Heat Miser, as he died in 2016.	Holidaze: The Christmas That Almost Didn't Happen Holidaze: The Christmas That Almost Didn't Happen is an American stop-motion animated Christmas television special directed by David H. Brooks, that originally aired in 2006 and produced by BixPix Entertainment, Once Upon a Frog and Madison Street Entertainment. The show's plot has Rusty Reindeer (Fred Savage) the brother of Rudolph the Red Nosed Reindeer joining a support group for depressed holiday icons, and he and the other characters search for the meaning of Christmas and help a young boy (Dylan and Cole Sprouse) to get on Santa's nice list. Rusty's cohorts include Candie, the Easter Bunny (Gladys Knight); Mr. C, the grouchy cherub (Paul Rodriguez); Albert, the Thanksgiving Turkey (Harland Williams); And Trick and Treat (Brenda Song and Emily Osment) the teenage Halloween Ghosts.

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "TripletLoss",
    "matryoshka_dims": [
        384,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
gradient_accumulation_steps: 16
learning_rate: 2e-05
num_train_epochs: 50
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
tf32: True
load_best_model_at_end: True
optim: adamw_torch_fused
resume_from_checkpoint: bge-small-hotpotwa-matryoshka
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 16
eval_accumulation_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 50
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: True
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: bge-small-hotpotwa-matryoshka
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	loss	dim_128_cosine_accuracy	dim_256_cosine_accuracy	dim_384_cosine_accuracy	dim_64_cosine_accuracy
0.3366	50	19.5758	19.3933	0.9552	0.9663	0.9668	0.9359
0.6731	100	19.4573	19.0971	0.9571	0.9646	0.9653	0.9450
1.0097	150	19.1409	18.4070	0.9385	0.9434	0.9473	0.9307
1.3462	200	18.6431	17.3292	0.9126	0.9164	0.9184	0.9094
1.6828	250	18.2288	16.8751	0.9063	0.9071	0.9100	0.9023
2.0194	300	18.0425	16.6981	0.9020	0.9032	0.9045	0.8990
2.3559	350	17.9458	16.6155	0.9037	0.9013	0.9022	0.8984
2.6925	400	17.8525	16.5536	0.8978	0.8971	0.8974	0.8948
3.0290	450	17.7529	16.5136	0.8980	0.8956	0.8953	0.8951
3.3656	500	17.6709	16.4824	0.8932	0.8914	0.8928	0.8907
3.7021	550	17.5348	16.4632	0.8863	0.8858	0.8859	0.8849
4.0387	600	17.4198	16.4601	0.8852	0.8862	0.8859	0.8839
4.3753	650	17.3673	16.4405	0.8854	0.8864	0.8865	0.8842
4.7118	700	17.2603	16.4356	0.8835	0.8838	0.8838	0.8807
5.0484	750	17.1807	16.4443	0.8850	0.8864	0.8859	0.8838
5.3849	800	17.1629	16.4202	0.8848	0.8862	0.8867	0.8842
5.7215	850	17.0747	16.4162	0.8854	0.8875	0.8869	0.8837
6.0581	900	17.0161	16.4192	0.8852	0.8863	0.8856	0.8856
6.3946	950	17.0146	16.4033	0.8849	0.8854	0.8856	0.8844
6.7312	1000	16.9393	16.4053	0.8829	0.8839	0.8848	0.8835
7.0677	1050	16.899	16.4162	0.8826	0.8829	0.8833	0.8818
7.4043	1100	16.9112	16.4051	0.8829	0.8835	0.8833	0.8820
7.7408	1150	16.8508	16.4044	0.8822	0.8825	0.8830	0.8820
8.0774	1200	16.8104	16.4063	0.8816	0.8816	0.8814	0.8817
8.4140	1250	16.8212	16.4040	0.8835	0.8822	0.8822	0.8820
8.7505	1300	16.7743	16.3934	0.8822	0.8824	0.8817	0.8810
9.0871	1350	16.7383	16.3963	0.8810	0.8820	0.8807	0.8800
9.4236	1400	16.743	16.4067	0.8819	0.8822	0.8819	0.8798
9.7602	1450	16.7047	16.3959	0.8804	0.8810	0.8810	0.8797
10.0968	1500	16.6782	16.3986	0.8788	0.8791	0.8796	0.8784
10.4333	1550	16.6708	16.4016	0.8794	0.8792	0.8797	0.8791
10.7699	1600	16.6485	16.3963	0.8790	0.8801	0.8791	0.8781
11.1064	1650	16.6205	16.4012	0.8779	0.8787	0.8793	0.8771
11.4430	1700	16.6095	16.4131	0.8786	0.8790	0.8794	0.8791
11.7796	1750	16.5891	16.4070	0.8807	0.8805	0.8810	0.8801
12.1161	1800	16.5619	16.3963	0.8794	0.8800	0.8797	0.8780
12.4527	1850	16.5467	16.3991	0.8796	0.8806	0.8804	0.8790
12.7892	1900	16.5398	16.3970	0.8792	0.8798	0.8801	0.8788
13.1258	1950	16.5047	16.3964	0.8796	0.8804	0.8804	0.8788
13.4623	2000	16.4985	16.4025	0.8793	0.8798	0.8807	0.8790
13.7989	2050	16.4852	16.4107	0.8801	0.8810	0.8800	0.8793
14.1355	2100	16.4526	16.3929	0.8797	0.8801	0.8809	0.8779
14.4720	2150	16.4343	16.4075	0.8788	0.8791	0.8797	0.8774
14.8086	2200	16.4244	16.4027	0.8804	0.8819	0.8820	0.8809
15.1451	2250	16.3947	16.4102	0.8791	0.8792	0.8803	0.8773
15.4817	2300	16.3827	16.4042	0.8804	0.8813	0.8813	0.8781
15.8183	2350	16.3719	16.4003	0.8801	0.8818	0.8820	0.8791
16.1548	2400	16.3403	16.4132	0.8781	0.8791	0.8799	0.8767
16.4914	2450	16.3357	16.4149	0.8804	0.8809	0.8807	0.8792
16.8279	2500	16.3203	16.4081	0.8804	0.8814	0.8816	0.8791
17.1645	2550	16.2986	16.4139	0.8798	0.8800	0.8820	0.8791
17.5011	2600	16.2923	16.4062	0.8786	0.8792	0.8799	0.8768
17.8376	2650	16.2649	16.4106	0.8800	0.8807	0.8814	0.8787
18.1742	2700	16.2505	16.4188	0.8786	0.8793	0.8803	0.8771
18.5107	2750	16.226	16.4149	0.8771	0.8781	0.8780	0.8766
18.8473	2800	16.2106	16.4230	0.8780	0.8799	0.8791	0.8767
19.1838	2850	16.2052	16.4351	0.8770	0.8777	0.8785	0.8745
19.5204	2900	16.186	16.4331	0.8777	0.8793	0.8792	0.8762
19.8570	2950	16.1496	16.4377	0.8774	0.8781	0.8780	0.8771
20.1935	3000	16.151	16.4407	0.8766	0.8780	0.8780	0.8751
20.5301	3050	16.1081	16.4426	0.8759	0.8775	0.8774	0.8749
20.8666	3100	16.0864	16.4412	0.8774	0.8781	0.8787	0.8746
21.2032	3150	16.0934	16.4547	0.8768	0.8783	0.8794	0.8746
21.5398	3200	16.0382	16.4589	0.8742	0.8752	0.8766	0.8723
21.8763	3250	16.0279	16.4668	0.8752	0.8766	0.8773	0.8728
22.2129	3300	16.0327	16.4737	0.8742	0.8768	0.8773	0.8727
22.5494	3350	15.979	16.4686	0.8740	0.8771	0.8771	0.8722
22.8860	3400	15.9622	16.4736	0.8743	0.8760	0.8765	0.8721
23.2225	3450	15.9881	16.4802	0.8743	0.8757	0.8755	0.8723
23.5591	3500	15.9482	16.4821	0.8725	0.8761	0.8761	0.8710
23.8957	3550	15.9228	16.4996	0.8726	0.8748	0.8751	0.8709
24.2322	3600	15.9418	16.4973	0.8709	0.8728	0.8734	0.8699
24.5688	3650	15.896	16.4985	0.8696	0.8716	0.8727	0.8686
24.9053	3700	15.8788	16.5172	0.8691	0.8715	0.8717	0.8662
25.2419	3750	15.9147	16.5062	0.8677	0.8706	0.8712	0.8662
25.5785	3800	15.857	16.5058	0.8683	0.8717	0.8732	0.8663
25.9150	3850	15.8291	16.5207	0.8674	0.8702	0.8706	0.8644
26.2516	3900	15.8802	16.5233	0.8678	0.8697	0.8714	0.8664
26.5881	3950	15.846	16.5170	0.8686	0.8713	0.8717	0.8655
26.9247	4000	15.8012	16.5336	0.8663	0.8682	0.8699	0.8635

Framework Versions

Python: 3.10.10
Sentence Transformers: 3.0.1
Transformers: 4.41.2
PyTorch: 2.1.2+cu121
Accelerate: 0.33.0
Datasets: 2.19.1
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification}, 
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}