Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,819 @@
+---
+language:
+- en
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:100231
+- loss:DebiasedMultipleNegativesRankingLoss
+base_model: answerdotai/ModernBERT-base
+widget:
+- source_sentence: who led the army that defeated the aztecs
+  sentences:
+  - Spanish conquest of the Aztec Empire The Spanish conquest of the Aztec Empire,
+    or the Spanish-Aztec War (1519-21)[3] was one of the most significant and complex
+    events in world history. There are multiple sixteenth-century narratives of the
+    events by Spanish conquerors, their indigenous allies, and the defeated Aztecs.
+    It was not solely a contest between a small contingent of Spaniards defeating
+    the Aztec Empire, but rather the creation of a coalition of Spanish invaders with
+    tributaries to the Aztecs, and most especially the Aztecs' indigenous enemies
+    and rivals. They combined forces to defeat the Mexica of Tenochtitlan over a two-year
+    period. For the Spanish, the expedition to Mexico was part of a project of Spanish
+    colonization of the New World after twenty-five years of permanent Spanish settlement
+    and further exploration in the Caribbean. The Spanish made landfall in Mexico
+    in 1517. A Spanish settler in Cuba, Hernán Cortés, led an expedition (entrada)
+    to Mexico, landing in February 1519, following an earlier expedition led by Juan
+    de Grijalva to Yucatán in 1517. Two years later Cortés and his retinue set sail,
+    thus beginning the expedition of exploration and conquest.[4] The Spanish campaign
+    against the Aztec Empire had its final victory on August 13, 1521, when a coalition
+    army of Spanish forces and native Tlaxcalan warriors led by Cortés and Xicotencatl
+    the Younger captured the emperor Cuauhtemoc and Tenochtitlan, the capital of the
+    Aztec Empire. The fall of Tenochtitlan marks the beginning of Spanish rule in
+    central Mexico, and they established their capital of Mexico City on the ruins
+    of Tenochtitlan.
+  - The Girl with All the Gifts Justineau awakens in the Rosalind Franklin. Melanie
+    leads her to a group of intelligent hungries, to whom Justineau, wearing an environmental
+    protection suit, starts teaching the alphabet.
+  - 'Wendy Makkena In 1992 she had a supporting role in the movie Sister Act as the
+    shy but talented singing nun Sister Mary Robert, a role she reprised in Sister
+    Act 2: Back in the Habit the following year. She appeared in various other television
+    roles until 1997, when she starred in Air Bud, followed by the independent film
+    Finding North. She continued appearing on television shows such as The Job, Oliver
+    Beene, and Listen Up![citation needed]'
+- source_sentence: who went to the most nba finals in a row
+  sentences:
+  - List of NBA franchise post-season streaks The San Antonio Spurs hold the longest
+    active consecutive playoff appearances with 21 appearances, starting in the 1998
+    NBA Playoffs (also the longest active playoff streak in any major North American
+    sports league as of 2017). The Spurs have won five NBA championships during the
+    streak. The Philadelphia 76ers (formerly known as Syracuse Nationals) hold the
+    all-time record for consecutive playoff appearances with 22 straight appearances
+    between 1950 and 1971. The 76ers won two NBA championships during their streak.
+    The Boston Celtics hold the longest consecutive NBA Finals appearance streak with
+    ten appearances between 1957 and 1966. During the streak, the Celtics won eight
+    consecutive NBA championships—also an NBA record.
+  - Dear Dumb Diary Dear Dumb Diary is a series of children's novels by Jim Benton.
+    Each book is written in the first person view of a middle school girl named Jamie
+    Kelly. The series is published by Scholastic in English and Random House in Korean.
+    Film rights to the series have been optioned by the Gotham Group.[2]
+  - Voting rights in the United States Eligibility to vote in the United States is
+    established both through the federal constitution and by state law. Several constitutional
+    amendments (the 15th, 19th, and 26th specifically) require that voting rights
+    cannot be abridged on account of race, color, previous condition of servitude,
+    sex, or age for those above 18; the constitution as originally written did not
+    establish any such rights during 1787–1870. In the absence of a specific federal
+    law or constitutional provision, each state is given considerable discretion to
+    establish qualifications for suffrage and candidacy within its own respective
+    jurisdiction; in addition, states and lower level jurisdictions establish election
+    systems, such as at-large or single member district elections for county councils
+    or school boards.
+- source_sentence: who did the vocals on mcdonald's jingle i'm loving it
+  sentences:
+  - I'm Lovin' It (song) "I'm Lovin' It" is a song recorded by American singer-songwriter
+    Justin Timberlake. It was written by Pusha T and produced by The Neptunes.
+  - Vallabhbhai Patel As the first Home Minister and Deputy Prime Minister of India,
+    Patel organised relief efforts for refugees fleeing from Punjab and Delhi and
+    worked to restore peace across the nation. He led the task of forging a united
+    India, successfully integrating into the newly independent nation those British
+    colonial provinces that had been "allocated" to India. Besides those provinces
+    that had been under direct British rule, approximately 565 self-governing princely
+    states had been released from British suzerainty by the Indian Independence Act
+    of 1947. Employing frank diplomacy with the expressed option to deploy military
+    force, Patel persuaded almost every princely state to accede to India. His commitment
+    to national integration in the newly independent country was total and uncompromising,
+    earning him the sobriquet "Iron Man of India".[3] He is also affectionately remembered
+    as the "Patron saint of India's civil servants" for having established the modern
+    all-India services system. He is also called the Unifier of India.[4]
+  - National debt of the United States As of July 31, 2018, debt held by the public
+    was $15.6 trillion and intragovernmental holdings were $5.7 trillion, for a total
+    or "National Debt" of $21.3 trillion.[5] Debt held by the public was approximately
+    77% of GDP in 2017, ranked 43rd highest out of 207 countries.[6] The Congressional
+    Budget Office forecast in April 2018 that the ratio will rise to nearly 100% by
+    2028, perhaps higher if current policies are extended beyond their scheduled expiration
+    date.[7] As of December 2017, $6.3 trillion or approximately 45% of the debt held
+    by the public was owned by foreign investors, the largest being China (about $1.18
+    trillion) then Japan (about $1.06 trillion).[8]
+- source_sentence: who is the actress of harley quinn in suicide squad
+  sentences:
+  - Tariffs in United States history Tariffs were the main source of revenue for the
+    federal government from 1789 to 1914. During this period, there was vigorous debate
+    between the various political parties over the setting of tariff rates. In general
+    Democrats favored a tariff that would pay the cost of government, but no higher.
+    Whigs and Republicans favored higher tariffs to protect and encourage American
+    industry and industrial workers. Since the early 20th century, however, U.S. tariffs
+    have been very low and have been much less a matter of partisan debate.
+  - The Rolling Stones The Rolling Stones are an English rock band formed in London,
+    England in 1962. The first stable line-up consisted of Brian Jones (guitar, harmonica),
+    Mick Jagger (lead vocals), Keith Richards (guitar, backing vocals), Bill Wyman
+    (bass), Charlie Watts (drums), and Ian Stewart (piano). Stewart was removed from
+    the official line-up in 1963 but continued as a touring member until his death
+    in 1985. Jones left the band less than a month prior to his death in 1969, having
+    already been replaced by Mick Taylor, who remained until 1974. After Taylor left
+    the band, Ronnie Wood took his place in 1975 and has been on guitar in tandem
+    with Richards ever since. Following Wyman's departure in 1993, Darryl Jones joined
+    as their touring bassist. Touring keyboardists for the band have been Nicky Hopkins
+    (1967–1982), Ian McLagan (1978–1981), Billy Preston (through the mid-1970s) and
+    Chuck Leavell (1982–present). The band was first led by Brian Jones, but after
+    developing into the band's songwriters, Jagger and Richards assumed leadership
+    while Jones dealt with legal and personal troubles.
+  - Margot Robbie After moving to the United States, Robbie starred in the short-lived
+    ABC drama series Pan Am (2011–2012). In 2013, she made her big screen debut in
+    Richard Curtis's romantic comedy-drama film About Time and co-starred in Martin
+    Scorsese's biographical black comedy The Wolf of Wall Street. In 2015, Robbie
+    co-starred in the romantic comedy-drama film Focus, appeared in the romantic World
+    War II drama film Suite Française and starred in the science fiction film Z for
+    Zachariah. That same year, she played herself in The Big Short. In 2016, she portrayed
+    Jane Porter in the action-adventure film The Legend of Tarzan and Harley Quinn
+    in the superhero film Suicide Squad. She appeared on Time magazine's "The Most
+    Influential People of 2017" list.[4]
+- source_sentence: what is meaning of am and pm in time
+  sentences:
+  - America's Got Talent America's Got Talent (often abbreviated as AGT) is a televised
+    American talent show competition, broadcast on the NBC television network. It
+    is part of the global Got Talent franchise created by Simon Cowell, and is produced
+    by Fremantle North America and SYCOtv, with distribution done by Fremantle. Since
+    its premiere in June 2006, each season is run during the network's summer schedule,
+    with the show having featured various hosts - it is currently hosted by Tyra Banks,
+    since 2017.[2] It is the first global edition of the franchise, after plans for
+    a British edition in 2005 were suspended, following a dispute between Paul O'Grady,
+    the planned host, and the British broadcaster ITV; production of this edition
+    later resumed in 2007.[3]
+  - Times Square Times Square is a major commercial intersection, tourist destination,
+    entertainment center and neighborhood in the Midtown Manhattan section of New
+    York City at the junction of Broadway and Seventh Avenue. It stretches from West
+    42nd to West 47th Streets.[1] Brightly adorned with billboards and advertisements,
+    Times Square is sometimes referred to as "The Crossroads of the World",[2] "The
+    Center of the Universe",[3] "the heart of The Great White Way",[4][5][6] and the
+    "heart of the world".[7] One of the world's busiest pedestrian areas,[8] it is
+    also the hub of the Broadway Theater District[9] and a major center of the world's
+    entertainment industry.[10] Times Square is one of the world's most visited tourist
+    attractions, drawing an estimated 50 million visitors annually.[11] Approximately
+    330,000 people pass through Times Square daily,[12] many of them tourists,[13]
+    while over 460,000 pedestrians walk through Times Square on its busiest days.[7]
+  - '12-hour clock The 12-hour clock is a time convention in which the 24 hours of
+    the day are divided into two periods:[1] a.m. (from the Latin, ante meridiem,
+    meaning before midday) and p.m. (post meridiem, meaning past midday).[2] Each
+    period consists of 12 hours numbered: 12 (acting as zero),[3] 1, 2, 3, 4, 5, 6,
+    7, 8, 9, 10, and 11. The 24 hour/day cycle starts at 12 midnight (often indicated
+    as 12 a.m.), runs through 12 noon (often indicated as 12 p.m.), and continues
+    to the midnight at the end of the day. The 12-hour clock was developed over time
+    from the mid-second millennium BC to the 16th century AD.'
+datasets:
+- sentence-transformers/natural-questions
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy@1
+- cosine_accuracy@3
+- cosine_accuracy@5
+- cosine_accuracy@10
+- cosine_precision@1
+- cosine_precision@3
+- cosine_precision@5
+- cosine_precision@10
+- cosine_recall@1
+- cosine_recall@3
+- cosine_recall@5
+- cosine_recall@10
+- cosine_ndcg@10
+- cosine_mrr@10
+- cosine_map@100
+model-index:
+- name: SentenceTransformer based on answerdotai/ModernBERT-base
+  results:
+  - task:
+      type: information-retrieval
+      name: Information Retrieval
+    dataset:
+      name: NanoMSMARCO
+      type: NanoMSMARCO
+    metrics:
+    - type: cosine_accuracy@1
+      value: 0.14
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 0.24
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 0.3
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 0.4
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 0.14
+      name: Cosine Precision@1
+    - type: cosine_precision@3
+      value: 0.07999999999999999
+      name: Cosine Precision@3
+    - type: cosine_precision@5
+      value: 0.06
+      name: Cosine Precision@5
+    - type: cosine_precision@10
+      value: 0.04000000000000001
+      name: Cosine Precision@10
+    - type: cosine_recall@1
+      value: 0.14
+      name: Cosine Recall@1
+    - type: cosine_recall@3
+      value: 0.24
+      name: Cosine Recall@3
+    - type: cosine_recall@5
+      value: 0.3
+      name: Cosine Recall@5
+    - type: cosine_recall@10
+      value: 0.4
+      name: Cosine Recall@10
+    - type: cosine_ndcg@10
+      value: 0.25076046577886124
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 0.20557936507936506
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 0.21939187046366332
+      name: Cosine Map@100
+  - task:
+      type: information-retrieval
+      name: Information Retrieval
+    dataset:
+      name: NanoHotpotQA
+      type: NanoHotpotQA
+    metrics:
+    - type: cosine_accuracy@1
+      value: 0.14
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 0.28
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 0.3
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 0.36
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 0.14
+      name: Cosine Precision@1
+    - type: cosine_precision@3
+      value: 0.09333333333333332
+      name: Cosine Precision@3
+    - type: cosine_precision@5
+      value: 0.064
+      name: Cosine Precision@5
+    - type: cosine_precision@10
+      value: 0.038
+      name: Cosine Precision@10
+    - type: cosine_recall@1
+      value: 0.07
+      name: Cosine Recall@1
+    - type: cosine_recall@3
+      value: 0.14
+      name: Cosine Recall@3
+    - type: cosine_recall@5
+      value: 0.16
+      name: Cosine Recall@5
+    - type: cosine_recall@10
+      value: 0.19
+      name: Cosine Recall@10
+    - type: cosine_ndcg@10
+      value: 0.15720914647954295
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 0.2121904761904762
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 0.12322210117624575
+      name: Cosine Map@100
+  - task:
+      type: nano-beir
+      name: Nano BEIR
+    dataset:
+      name: NanoBEIR mean
+      type: NanoBEIR_mean
+    metrics:
+    - type: cosine_accuracy@1
+      value: 0.14
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 0.26
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 0.3
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 0.38
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 0.14
+      name: Cosine Precision@1
+    - type: cosine_precision@3
+      value: 0.08666666666666666
+      name: Cosine Precision@3
+    - type: cosine_precision@5
+      value: 0.062
+      name: Cosine Precision@5
+    - type: cosine_precision@10
+      value: 0.03900000000000001
+      name: Cosine Precision@10
+    - type: cosine_recall@1
+      value: 0.10500000000000001
+      name: Cosine Recall@1
+    - type: cosine_recall@3
+      value: 0.19
+      name: Cosine Recall@3
+    - type: cosine_recall@5
+      value: 0.22999999999999998
+      name: Cosine Recall@5
+    - type: cosine_recall@10
+      value: 0.29500000000000004
+      name: Cosine Recall@10
+    - type: cosine_ndcg@10
+      value: 0.2039848061292021
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 0.20888492063492065
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 0.17130698581995454
+      name: Cosine Map@100
+---
+
+# SentenceTransformer based on answerdotai/ModernBERT-base
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on the [natural-questions](https://huggingface.co/datasets/sentence-transformers/natural-questions) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) <!-- at revision 6e461621ae9e2dffc138de99490e9baee354deb5 -->
+- **Maximum Sequence Length:** 8192 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - [natural-questions](https://huggingface.co/datasets/sentence-transformers/natural-questions)
+- **Language:** en
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("tomaarsen/ModernBERT-base-nq-debiased-mnrl")
+# Run inference
+sentences = [
+    'what is meaning of am and pm in time',
+    '12-hour clock The 12-hour clock is a time convention in which the 24 hours of the day are divided into two periods:[1] a.m. (from the Latin, ante meridiem, meaning before midday) and p.m. (post meridiem, meaning past midday).[2] Each period consists of 12 hours numbered: 12 (acting as zero),[3] 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, and 11. The 24 hour/day cycle starts at 12 midnight (often indicated as 12 a.m.), runs through 12 noon (often indicated as 12 p.m.), and continues to the midnight at the end of the day. The 12-hour clock was developed over time from the mid-second millennium BC to the 16th century AD.',
+    "America's Got Talent America's Got Talent (often abbreviated as AGT) is a televised American talent show competition, broadcast on the NBC television network. It is part of the global Got Talent franchise created by Simon Cowell, and is produced by Fremantle North America and SYCOtv, with distribution done by Fremantle. Since its premiere in June 2006, each season is run during the network's summer schedule, with the show having featured various hosts - it is currently hosted by Tyra Banks, since 2017.[2] It is the first global edition of the franchise, after plans for a British edition in 2005 were suspended, following a dispute between Paul O'Grady, the planned host, and the British broadcaster ITV; production of this edition later resumed in 2007.[3]",
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Information Retrieval
+
+* Datasets: `NanoMSMARCO` and `NanoHotpotQA`
+* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
+
+| Metric              | NanoMSMARCO | NanoHotpotQA |
+|:--------------------|:------------|:-------------|
+| cosine_accuracy@1   | 0.14        | 0.14         |
+| cosine_accuracy@3   | 0.24        | 0.28         |
+| cosine_accuracy@5   | 0.3         | 0.3          |
+| cosine_accuracy@10  | 0.4         | 0.36         |
+| cosine_precision@1  | 0.14        | 0.14         |
+| cosine_precision@3  | 0.08        | 0.0933       |
+| cosine_precision@5  | 0.06        | 0.064        |
+| cosine_precision@10 | 0.04        | 0.038        |
+| cosine_recall@1     | 0.14        | 0.07         |
+| cosine_recall@3     | 0.24        | 0.14         |
+| cosine_recall@5     | 0.3         | 0.16         |
+| cosine_recall@10    | 0.4         | 0.19         |
+| **cosine_ndcg@10**  | **0.2508**  | **0.1572**   |
+| cosine_mrr@10       | 0.2056      | 0.2122       |
+| cosine_map@100      | 0.2194      | 0.1232       |
+
+#### Nano BEIR
+
+* Dataset: `NanoBEIR_mean`
+* Evaluated with [<code>NanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.NanoBEIREvaluator)
+
+| Metric              | Value     |
+|:--------------------|:----------|
+| cosine_accuracy@1   | 0.14      |
+| cosine_accuracy@3   | 0.26      |
+| cosine_accuracy@5   | 0.3       |
+| cosine_accuracy@10  | 0.38      |
+| cosine_precision@1  | 0.14      |
+| cosine_precision@3  | 0.0867    |
+| cosine_precision@5  | 0.062     |
+| cosine_precision@10 | 0.039     |
+| cosine_recall@1     | 0.105     |
+| cosine_recall@3     | 0.19      |
+| cosine_recall@5     | 0.23      |
+| cosine_recall@10    | 0.295     |
+| **cosine_ndcg@10**  | **0.204** |
+| cosine_mrr@10       | 0.2089    |
+| cosine_map@100      | 0.1713    |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### natural-questions
+
+* Dataset: [natural-questions](https://huggingface.co/datasets/sentence-transformers/natural-questions) at [f9e894e](https://huggingface.co/datasets/sentence-transformers/natural-questions/tree/f9e894e1081e206e577b4eaa9ee6de2b06ae6f17)
+* Size: 100,231 training samples
+* Columns: <code>query</code> and <code>answer</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                              | answer                                                                               |
+  |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                               |
+  | details | <ul><li>min: 10 tokens</li><li>mean: 12.46 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>min: 16 tokens</li><li>mean: 139.02 tokens</li><li>max: 537 tokens</li></ul> |
+* Samples:
+  | query                                                                   | answer                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+  |:------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>who is required to report according to the hmda</code>            | <code>Home Mortgage Disclosure Act US financial institutions must report HMDA data to their regulator if they meet certain criteria, such as having assets above a specific threshold. The criteria is different for depository and non-depository institutions and are available on the FFIEC website.[4] In 2012, there were 7,400 institutions that reported a total of 18.7 million HMDA records.[5]</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
+  | <code>what is the definition of endoplasmic reticulum in biology</code> | <code>Endoplasmic reticulum The endoplasmic reticulum (ER) is a type of organelle in eukaryotic cells that forms an interconnected network of flattened, membrane-enclosed sacs or tube-like structures known as cisternae. The membranes of the ER are continuous with the outer nuclear membrane. The endoplasmic reticulum occurs in most types of eukaryotic cells, but is absent from red blood cells and spermatozoa. There are two types of endoplasmic reticulum: rough and smooth. The outer (cytosolic) face of the rough endoplasmic reticulum is studded with ribosomes that are the sites of protein synthesis. The rough endoplasmic reticulum is especially prominent in cells such as hepatocytes. The smooth endoplasmic reticulum lacks ribosomes and functions in lipid manufacture and metabolism, the production of steroid hormones, and detoxification.[1] The smooth ER is especially abundant in mammalian liver and gonad cells. The lacy membranes of the endoplasmic reticulum were first seen in 1945 using elect...</code> |
+  | <code>what does the ski mean in polish names</code>                     | <code>Polish name Since the High Middle Ages, Polish-sounding surnames ending with the masculine -ski suffix, including -cki and -dzki, and the corresponding feminine suffix -ska/-cka/-dzka were associated with the nobility (Polish szlachta), which alone, in the early years, had such suffix distinctions.[1] They are widely popular today.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+* Loss: [<code>DebiasedMultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#debiasedmultiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 1.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Evaluation Dataset
+
+#### natural-questions
+
+* Dataset: [natural-questions](https://huggingface.co/datasets/sentence-transformers/natural-questions) at [f9e894e](https://huggingface.co/datasets/sentence-transformers/natural-questions/tree/f9e894e1081e206e577b4eaa9ee6de2b06ae6f17)
+* Size: 100,231 evaluation samples
+* Columns: <code>query</code> and <code>answer</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                              | answer                                                                              |
+  |:--------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                              |
+  | details | <ul><li>min: 10 tokens</li><li>mean: 12.46 tokens</li><li>max: 22 tokens</li></ul> | <ul><li>min: 12 tokens</li><li>mean: 138.0 tokens</li><li>max: 649 tokens</li></ul> |
+* Samples:
+  | query                                                             | answer                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+  |:------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>difference between russian blue and british blue cat</code> | <code>Russian Blue The coat is known as a "double coat", with the undercoat being soft, downy and equal in length to the guard hairs, which are an even blue with silver tips. However, the tail may have a few very dull, almost unnoticeable stripes. The coat is described as thick, plush and soft to the touch. The feeling is softer than the softest silk. The silver tips give the coat a shimmering appearance. Its eyes are almost always a dark and vivid green. Any white patches of fur or yellow eyes in adulthood are seen as flaws in show cats.[3] Russian Blues should not be confused with British Blues (which are not a distinct breed, but rather a British Shorthair with a blue coat as the British Shorthair breed itself comes in a wide variety of colors and patterns), nor the Chartreux or Korat which are two other naturally occurring breeds of blue cats, although they have similar traits.</code> |
+  | <code>who played the little girl on mrs doubtfire</code>          | <code>Mara Wilson Mara Elizabeth Wilson[2] (born July 24, 1987) is an American writer and former child actress. She is known for playing Natalie Hillard in Mrs. Doubtfire (1993), Susan Walker in Miracle on 34th Street (1994), Matilda Wormwood in Matilda (1996) and Lily Stone in Thomas and the Magic Railroad (2000). Since retiring from film acting, Wilson has focused on writing.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+  | <code>what year did the movie the sound of music come out</code>  | <code>The Sound of Music (film) The film was released on March 2, 1965 in the United States, initially as a limited roadshow theatrical release. Although critical response to the film was widely mixed, the film was a major commercial success, becoming the number one box office movie after four weeks, and the highest-grossing film of 1965. By November 1966, The Sound of Music had become the highest-grossing film of all-time—surpassing Gone with the Wind—and held that distinction for five years. The film was just as popular throughout the world, breaking previous box-office records in twenty-nine countries. Following an initial theatrical release that lasted four and a half years, and two successful re-releases, the film sold 283 million admissions worldwide and earned a total worldwide gross of $286,000,000.</code>                                                                             |
+* Loss: [<code>DebiasedMultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#debiasedmultiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 1.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 128
+- `per_device_eval_batch_size`: 128
+- `learning_rate`: 8e-05
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.05
+- `seed`: 12
+- `bf16`: True
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 128
+- `per_device_eval_batch_size`: 128
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 8e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.05
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 12
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch  | Step | Training Loss | Validation Loss | NanoMSMARCO_cosine_ndcg@10 | NanoHotpotQA_cosine_ndcg@10 | NanoBEIR_mean_cosine_ndcg@10 |
+|:------:|:----:|:-------------:|:---------------:|:--------------------------:|:---------------------------:|:----------------------------:|
+| 0      | 0    | -             | -               | 0.0785                     | 0.1489                      | 0.1137                       |
+| 0.0129 | 10   | 4.8033        | -               | -                          | -                           | -                            |
+| 0.0258 | 20   | 4.5295        | -               | -                          | -                           | -                            |
+| 0.0387 | 30   | 4.2124        | -               | -                          | -                           | -                            |
+| 0.0515 | 40   | 4.0863        | -               | -                          | -                           | -                            |
+| 0.0644 | 50   | 4.0048        | 3.9563          | 0.1444                     | 0.1660                      | 0.1552                       |
+| 0.0773 | 60   | 3.9686        | -               | -                          | -                           | -                            |
+| 0.0902 | 70   | 3.9192        | -               | -                          | -                           | -                            |
+| 0.1031 | 80   | 3.9276        | -               | -                          | -                           | -                            |
+| 0.1160 | 90   | 3.9104        | -               | -                          | -                           | -                            |
+| 0.1289 | 100  | 3.8971        | 3.8877          | 0.2041                     | 0.1293                      | 0.1667                       |
+| 0.1418 | 110  | 3.8987        | -               | -                          | -                           | -                            |
+| 0.1546 | 120  | 3.8861        | -               | -                          | -                           | -                            |
+| 0.1675 | 130  | 3.8987        | -               | -                          | -                           | -                            |
+| 0.1804 | 140  | 3.8811        | -               | -                          | -                           | -                            |
+| 0.1933 | 150  | 3.8697        | 3.8478          | 0.1918                     | 0.1084                      | 0.1501                       |
+| 0.2062 | 160  | 3.8621        | -               | -                          | -                           | -                            |
+| 0.2191 | 170  | 3.8628        | -               | -                          | -                           | -                            |
+| 0.2320 | 180  | 3.8733        | -               | -                          | -                           | -                            |
+| 0.2448 | 190  | 3.8551        | -               | -                          | -                           | -                            |
+| 0.2577 | 200  | 3.862         | 3.8324          | 0.1940                     | 0.0977                      | 0.1458                       |
+| 0.2706 | 210  | 3.8545        | -               | -                          | -                           | -                            |
+| 0.2835 | 220  | 3.8495        | -               | -                          | -                           | -                            |
+| 0.2964 | 230  | 3.8459        | -               | -                          | -                           | -                            |
+| 0.3093 | 240  | 3.8438        | -               | -                          | -                           | -                            |
+| 0.3222 | 250  | 3.8425        | 3.8238          | 0.1933                     | 0.1498                      | 0.1716                       |
+| 0.3351 | 260  | 3.843         | -               | -                          | -                           | -                            |
+| 0.3479 | 270  | 3.8486        | -               | -                          | -                           | -                            |
+| 0.3608 | 280  | 3.8409        | -               | -                          | -                           | -                            |
+| 0.3737 | 290  | 3.8345        | -               | -                          | -                           | -                            |
+| 0.3866 | 300  | 3.8446        | 3.8154          | 0.1937                     | 0.1532                      | 0.1735                       |
+| 0.3995 | 310  | 3.8281        | -               | -                          | -                           | -                            |
+| 0.4124 | 320  | 3.8316        | -               | -                          | -                           | -                            |
+| 0.4253 | 330  | 3.8325        | -               | -                          | -                           | -                            |
+| 0.4381 | 340  | 3.8298        | -               | -                          | -                           | -                            |
+| 0.4510 | 350  | 3.8379        | 3.8104          | 0.1690                     | 0.1559                      | 0.1624                       |
+| 0.4639 | 360  | 3.821         | -               | -                          | -                           | -                            |
+| 0.4768 | 370  | 3.8297        | -               | -                          | -                           | -                            |
+| 0.4897 | 380  | 3.8206        | -               | -                          | -                           | -                            |
+| 0.5026 | 390  | 3.8222        | -               | -                          | -                           | -                            |
+| 0.5155 | 400  | 3.8243        | 3.8031          | 0.2141                     | 0.1544                      | 0.1843                       |
+| 0.5284 | 410  | 3.8328        | -               | -                          | -                           | -                            |
+| 0.5412 | 420  | 3.8211        | -               | -                          | -                           | -                            |
+| 0.5541 | 430  | 3.82          | -               | -                          | -                           | -                            |
+| 0.5670 | 440  | 3.8167        | -               | -                          | -                           | -                            |
+| 0.5799 | 450  | 3.8062        | 3.7988          | 0.2281                     | 0.1392                      | 0.1837                       |
+| 0.5928 | 460  | 3.8166        | -               | -                          | -                           | -                            |
+| 0.6057 | 470  | 3.8164        | -               | -                          | -                           | -                            |
+| 0.6186 | 480  | 3.8207        | -               | -                          | -                           | -                            |
+| 0.6314 | 490  | 3.815         | -               | -                          | -                           | -                            |
+| 0.6443 | 500  | 3.813         | 3.7943          | 0.2381                     | 0.1260                      | 0.1821                       |
+| 0.6572 | 510  | 3.8144        | -               | -                          | -                           | -                            |
+| 0.6701 | 520  | 3.8172        | -               | -                          | -                           | -                            |
+| 0.6830 | 530  | 3.8175        | -               | -                          | -                           | -                            |
+| 0.6959 | 540  | 3.8126        | -               | -                          | -                           | -                            |
+| 0.7088 | 550  | 3.8077        | 3.7913          | 0.2501                     | 0.1395                      | 0.1948                       |
+| 0.7216 | 560  | 3.8022        | -               | -                          | -                           | -                            |
+| 0.7345 | 570  | 3.8131        | -               | -                          | -                           | -                            |
+| 0.7474 | 580  | 3.8067        | -               | -                          | -                           | -                            |
+| 0.7603 | 590  | 3.8175        | -               | -                          | -                           | -                            |
+| 0.7732 | 600  | 3.8084        | 3.7870          | 0.2751                     | 0.1480                      | 0.2116                       |
+| 0.7861 | 610  | 3.8029        | -               | -                          | -                           | -                            |
+| 0.7990 | 620  | 3.8125        | -               | -                          | -                           | -                            |
+| 0.8119 | 630  | 3.817         | -               | -                          | -                           | -                            |
+| 0.8247 | 640  | 3.8038        | -               | -                          | -                           | -                            |
+| 0.8376 | 650  | 3.8054        | 3.7877          | 0.2274                     | 0.1449                      | 0.1861                       |
+| 0.8505 | 660  | 3.8041        | -               | -                          | -                           | -                            |
+| 0.8634 | 670  | 3.8012        | -               | -                          | -                           | -                            |
+| 0.8763 | 680  | 3.8117        | -               | -                          | -                           | -                            |
+| 0.8892 | 690  | 3.8098        | -               | -                          | -                           | -                            |
+| 0.9021 | 700  | 3.8008        | 3.7848          | 0.2466                     | 0.1551                      | 0.2008                       |
+| 0.9149 | 710  | 3.8038        | -               | -                          | -                           | -                            |
+| 0.9278 | 720  | 3.7949        | -               | -                          | -                           | -                            |
+| 0.9407 | 730  | 3.8044        | -               | -                          | -                           | -                            |
+| 0.9536 | 740  | 3.7982        | -               | -                          | -                           | -                            |
+| 0.9665 | 750  | 3.804         | 3.7832          | 0.2585                     | 0.1587                      | 0.2086                       |
+| 0.9794 | 760  | 3.8038        | -               | -                          | -                           | -                            |
+| 0.9923 | 770  | 3.8046        | -               | -                          | -                           | -                            |
+| 1.0    | 776  | -             | -               | 0.2508                     | 0.1572                      | 0.2040                       |
+
+
+### Framework Versions
+- Python: 3.11.10
+- Sentence Transformers: 3.4.0.dev0
+- Transformers: 4.48.0.dev0
+- PyTorch: 2.6.0.dev20241112+cu121
+- Accelerate: 1.2.0
+- Datasets: 3.2.0
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### DebiasedMultipleNegativesRankingLoss
+```bibtex
+@inproceedings{chuang2020debiased,
+    title={Debiased Contrastive Learning},
+    author={Ching-Yao Chuang and Joshua Robinson and Lin Yen-Chen and Antonio Torralba and Stefanie Jegelka},
+    booktitle={Advances in Neural Information Processing Systems},
+    year={2020},
+    url={https://arxiv.org/pdf/2007.00224}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,46 @@
+{
+  "_name_or_path": "answerdotai/ModernBERT-base",
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "reference_compile": true,
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.0.dev0",
+  "vocab_size": 50368
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.4.0.dev0",
+    "transformers": "4.48.0.dev0",
+    "pytorch": "2.6.0.dev20241112+cu121"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b79f66daa2ff58390d78fc0b6a8603804d39ffae94faa713bd876ec3b7db59f
+size 596070136

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 8192,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,945 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50285": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "[UNK]"
+}