Hugging Face's logo

tomaarsen
/
reranker-modernbert-base-msmarco-bce

Text Classification
sentence-transformers
Safetensors
modernbert
cross-encoder
Generated from Trainer
dataset_size:19990000
loss:BinaryCrossEntropyLoss
Model card Files Community
tomaarsen HF staff commited on
Commit
1d84d75
·
verified ·
1 Parent(s): 9ff6c38

Add new CrossEncoder model

Browse files
Files changed (6) hide show
  1. README.md +544 -0
  2. config.json +53 -0
  3. model.safetensors +3 -0
  4. special_tokens_map.json +37 -0
  5. tokenizer.json +0 -0
  6. tokenizer_config.json +945 -0
README.md ADDED
@@ -0,0 +1,544 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - cross-encoder
5
+ - text-classification
6
+ - generated_from_trainer
7
+ - dataset_size:19990000
8
+ - loss:BinaryCrossEntropyLoss
9
+ base_model: answerdotai/ModernBERT-base
10
+ pipeline_tag: text-classification
11
+ library_name: sentence-transformers
12
+ metrics:
13
+ - map
14
+ - mrr@10
15
+ - ndcg@10
16
+ model-index:
17
+ - name: CrossEncoder based on answerdotai/ModernBERT-base
18
+ results: []
19
+ ---
20
+
21
+ # CrossEncoder based on answerdotai/ModernBERT-base
22
+
23
+ This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
24
+
25
+ ## Model Details
26
+
27
+ ### Model Description
28
+ - **Model Type:** Cross Encoder
29
+ - **Base model:** [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) <!-- at revision 8949b909ec900327062f0ebf497f51aef5e6f0c8 -->
30
+ - **Maximum Sequence Length:** 8192 tokens
31
+ - **Number of Output Labels:** 1 label
32
+ <!-- - **Training Dataset:** Unknown -->
33
+ <!-- - **Language:** Unknown -->
34
+ <!-- - **License:** Unknown -->
35
+
36
+ ### Model Sources
37
+
38
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
39
+ - **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
40
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
41
+ - **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
42
+
43
+ ## Usage
44
+
45
+ ### Direct Usage (Sentence Transformers)
46
+
47
+ First install the Sentence Transformers library:
48
+
49
+ ```bash
50
+ pip install -U sentence-transformers
51
+ ```
52
+
53
+ Then you can load this model and run inference.
54
+ ```python
55
+ from sentence_transformers import CrossEncoder
56
+
57
+ # Download from the 🤗 Hub
58
+ model = CrossEncoder("tomaarsen/reranker-modernbert-base-msmarco-bce")
59
+ # Get scores for pairs of texts
60
+ pairs = [
61
+ ['what gb sp model is bright', 'From the list above it is easy to understand that Gegabyte is bigger than Megabyte. Or GB is bigger between MB and GB. Thanks.'],
62
+ ['does immunotherapy work', 'The US and Drug Administration (FDA) this week convened a panel of outside experts to weigh in on the readiness of a first-of-its-kind cancer therapy. The treatment, which works by tweaking a patientâ\x80\x99s own cells, is a type of immunotherapy called CAR T-cell therapy and has been in clinical trials for several years. One drug maker is now seeking FDA approval to use the treatment in pediatric and young adult patients ages 3 to 25 with B-cell acute lymphoblastic leukemia (ALL) that has not responded to standard care.'],
63
+ ['how long to wear oasis contacts', 'There is something wrong with my Xperia Z last week, and I did a resetting to make the phone become original, but I forgot backing up some important contacts, so I have to find the way for Sony Xperia Z contacts recovery, finanlly, I got this Android Data Recovery software to recover lost contacts from my phone.here is something wrong with my Xperia Z last week, and I did a resetting to make the phone become original, but I forgot backing up some important contacts, so I have to find the way for Sony Xperia Z contacts recovery, finanlly, I got this Android Data Recovery software to recover lost contacts from my phone.'],
64
+ ['water baby definition', "Someone very comfortable in the water, Good swimmers, and never scared while in bodies of water. There's Jadine, back in the lake. She's such a water baby. #water #aqua #babies #water babies #water kids."],
65
+ ['youngest suicide case', 'Samantha Kuberskki was found hanging by a belt at her home in Oregon after being sent to her room for arguing with her mother. A six-year-old girl who was sent to her room for punishment is feared to be one of the youngest people to have ever committed suicide in the U.S. Samantha Kuberskki was found hanging by a belt at her home in Oregon after being sent to her room for arguing with her mother. Her death was ruled as suicide by the coroner - sparking a bitter row with police who investigated her death and insist it was an accident.'],
66
+ ]
67
+ scores = model.predict(pairs)
68
+ print(scores.shape)
69
+ # (5,)
70
+
71
+ # Or rank different texts based on similarity to a single text
72
+ ranks = model.rank(
73
+ 'what gb sp model is bright',
74
+ [
75
+ 'From the list above it is easy to understand that Gegabyte is bigger than Megabyte. Or GB is bigger between MB and GB. Thanks.',
76
+ 'The US and Drug Administration (FDA) this week convened a panel of outside experts to weigh in on the readiness of a first-of-its-kind cancer therapy. The treatment, which works by tweaking a patientâ\x80\x99s own cells, is a type of immunotherapy called CAR T-cell therapy and has been in clinical trials for several years. One drug maker is now seeking FDA approval to use the treatment in pediatric and young adult patients ages 3 to 25 with B-cell acute lymphoblastic leukemia (ALL) that has not responded to standard care.',
77
+ 'There is something wrong with my Xperia Z last week, and I did a resetting to make the phone become original, but I forgot backing up some important contacts, so I have to find the way for Sony Xperia Z contacts recovery, finanlly, I got this Android Data Recovery software to recover lost contacts from my phone.here is something wrong with my Xperia Z last week, and I did a resetting to make the phone become original, but I forgot backing up some important contacts, so I have to find the way for Sony Xperia Z contacts recovery, finanlly, I got this Android Data Recovery software to recover lost contacts from my phone.',
78
+ "Someone very comfortable in the water, Good swimmers, and never scared while in bodies of water. There's Jadine, back in the lake. She's such a water baby. #water #aqua #babies #water babies #water kids.",
79
+ 'Samantha Kuberskki was found hanging by a belt at her home in Oregon after being sent to her room for arguing with her mother. A six-year-old girl who was sent to her room for punishment is feared to be one of the youngest people to have ever committed suicide in the U.S. Samantha Kuberskki was found hanging by a belt at her home in Oregon after being sent to her room for arguing with her mother. Her death was ruled as suicide by the coroner - sparking a bitter row with police who investigated her death and insist it was an accident.',
80
+ ]
81
+ )
82
+ # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
83
+ ```
84
+
85
+ <!--
86
+ ### Direct Usage (Transformers)
87
+
88
+ <details><summary>Click to see the direct usage in Transformers</summary>
89
+
90
+ </details>
91
+ -->
92
+
93
+ <!--
94
+ ### Downstream Usage (Sentence Transformers)
95
+
96
+ You can finetune this model on your own dataset.
97
+
98
+ <details><summary>Click to expand</summary>
99
+
100
+ </details>
101
+ -->
102
+
103
+ <!--
104
+ ### Out-of-Scope Use
105
+
106
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
107
+ -->
108
+
109
+ ## Evaluation
110
+
111
+ ### Metrics
112
+
113
+ #### Cross Encoder Reranking
114
+
115
+ * Datasets: `NanoMSMARCO`, `NanoNFCorpus` and `NanoNQ`
116
+ * Evaluated with [<code>CERerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CERerankingEvaluator)
117
+
118
+ | Metric | NanoMSMARCO | NanoNFCorpus | NanoNQ |
119
+ |:------------|:---------------------|:---------------------|:---------------------|
120
+ | map | 0.6519 (+0.1623) | 0.3432 (+0.0728) | 0.6951 (+0.2744) |
121
+ | mrr@10 | 0.6449 (+0.1674) | 0.5016 (+0.0017) | 0.7152 (+0.2885) |
122
+ | **ndcg@10** | **0.7069 (+0.1665)** | **0.3801 (+0.0550)** | **0.7469 (+0.2462)** |
123
+
124
+ #### Cross Encoder Nano BEIR
125
+
126
+ * Dataset: `NanoBEIR_mean`
127
+ * Evaluated with [<code>CENanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CENanoBEIREvaluator)
128
+
129
+ | Metric | Value |
130
+ |:------------|:---------------------|
131
+ | map | 0.5634 (+0.1698) |
132
+ | mrr@10 | 0.6206 (+0.1525) |
133
+ | **ndcg@10** | **0.6113 (+0.1559)** |
134
+
135
+ <!--
136
+ ## Bias, Risks and Limitations
137
+
138
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
139
+ -->
140
+
141
+ <!--
142
+ ### Recommendations
143
+
144
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
145
+ -->
146
+
147
+ ## Training Details
148
+
149
+ ### Training Dataset
150
+
151
+ #### Unnamed Dataset
152
+
153
+ * Size: 19,990,000 training samples
154
+ * Columns: <code>query</code>, <code>answer</code>, and <code>label</code>
155
+ * Approximate statistics based on the first 1000 samples:
156
+ | | query | answer | label |
157
+ |:--------|:------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|:------------------------------------------------|
158
+ | type | string | string | int |
159
+ | details | <ul><li>min: 10 characters</li><li>mean: 34.21 characters</li><li>max: 197 characters</li></ul> | <ul><li>min: 82 characters</li><li>mean: 350.38 characters</li><li>max: 860 characters</li></ul> | <ul><li>0: ~73.10%</li><li>1: ~26.90%</li></ul> |
160
+ * Samples:
161
+ | query | answer | label |
162
+ |:-------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
163
+ | <code>who plays the trickster on flash</code> | <code>The Flash (2014 TV series) The Flash is a TV show based on the fictional character Flash, a costumed superhero crime-fighter who appears in comic books published by DC Comics.</code> | <code>0</code> |
164
+ | <code>what type of business is plastics engineering company</code> | <code>Plastics Engineering Company is a leading North American manufacturer of phenolic resins and thermoset molding materials, selling products under its trademark Plenco. If you have a phenolic resin or thermoset molding material project, chances are, the Plenco team can make it work. We've been doing it for over 80 years. Come and benefit from the Plenco difference. Plastics Engineering Company, a family owned and managed business founded in 1934, established as its corporate mission a sincere desire to respond efficiently to the needs of our customers through development, manufacture, and servicing of useful, high-value products.</code> | <code>1</code> |
165
+ | <code>what is allianz global assistance</code> | <code>Please choose 'Allianz Direct Customers' for Car, Home, Pet, Boat and Horse & Rider Insurance. Allianz Direct Customers Allianz Direct Customers Car, Home, Pet, Boat and Horse & Rider Insurance. Phone. In the Republic of Ireland: 01 448 48 48. Outside Republic of Ireland: 00 353 1 448 48 48. Opening Hours: Monday to Friday 8am - 6pm and Saturday 9am - 1pm.</code> | <code>0</code> |
166
+ * Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
167
+ ```json
168
+ {
169
+ "activation_fct": "Identity",
170
+ "pos_weight": 4
171
+ }
172
+ ```
173
+
174
+ ### Evaluation Dataset
175
+
176
+ #### Unnamed Dataset
177
+
178
+ * Size: 10,000 evaluation samples
179
+ * Columns: <code>query</code>, <code>answer</code>, and <code>label</code>
180
+ * Approximate statistics based on the first 1000 samples:
181
+ | | query | answer | label |
182
+ |:--------|:------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|:------------------------------------------------|
183
+ | type | string | string | int |
184
+ | details | <ul><li>min: 11 characters</li><li>mean: 33.77 characters</li><li>max: 215 characters</li></ul> | <ul><li>min: 73 characters</li><li>mean: 351.17 characters</li><li>max: 935 characters</li></ul> | <ul><li>0: ~75.80%</li><li>1: ~24.20%</li></ul> |
185
+ * Samples:
186
+ | query | answer | label |
187
+ |:---------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
188
+ | <code>what gb sp model is bright</code> | <code>From the list above it is easy to understand that Gegabyte is bigger than Megabyte. Or GB is bigger between MB and GB. Thanks.</code> | <code>0</code> |
189
+ | <code>does immunotherapy work</code> | <code>The US and Drug Administration (FDA) this week convened a panel of outside experts to weigh in on the readiness of a first-of-its-kind cancer therapy. The treatment, which works by tweaking a patient’s own cells, is a type of immunotherapy called CAR T-cell therapy and has been in clinical trials for several years. One drug maker is now seeking FDA approval to use the treatment in pediatric and young adult patients ages 3 to 25 with B-cell acute lymphoblastic leukemia (ALL) that has not responded to standard care.</code> | <code>0</code> |
190
+ | <code>how long to wear oasis contacts</code> | <code>There is something wrong with my Xperia Z last week, and I did a resetting to make the phone become original, but I forgot backing up some important contacts, so I have to find the way for Sony Xperia Z contacts recovery, finanlly, I got this Android Data Recovery software to recover lost contacts from my phone.here is something wrong with my Xperia Z last week, and I did a resetting to make the phone become original, but I forgot backing up some important contacts, so I have to find the way for Sony Xperia Z contacts recovery, finanlly, I got this Android Data Recovery software to recover lost contacts from my phone.</code> | <code>0</code> |
191
+ * Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
192
+ ```json
193
+ {
194
+ "activation_fct": "Identity",
195
+ "pos_weight": 4
196
+ }
197
+ ```
198
+
199
+ ### Training Hyperparameters
200
+ #### Non-Default Hyperparameters
201
+
202
+ - `eval_strategy`: steps
203
+ - `per_device_train_batch_size`: 128
204
+ - `per_device_eval_batch_size`: 128
205
+ - `learning_rate`: 8e-05
206
+ - `num_train_epochs`: 1
207
+ - `warmup_ratio`: 0.1
208
+ - `seed`: 12
209
+ - `bf16`: True
210
+ - `dataloader_num_workers`: 4
211
+ - `load_best_model_at_end`: True
212
+
213
+ #### All Hyperparameters
214
+ <details><summary>Click to expand</summary>
215
+
216
+ - `overwrite_output_dir`: False
217
+ - `do_predict`: False
218
+ - `eval_strategy`: steps
219
+ - `prediction_loss_only`: True
220
+ - `per_device_train_batch_size`: 128
221
+ - `per_device_eval_batch_size`: 128
222
+ - `per_gpu_train_batch_size`: None
223
+ - `per_gpu_eval_batch_size`: None
224
+ - `gradient_accumulation_steps`: 1
225
+ - `eval_accumulation_steps`: None
226
+ - `torch_empty_cache_steps`: None
227
+ - `learning_rate`: 8e-05
228
+ - `weight_decay`: 0.0
229
+ - `adam_beta1`: 0.9
230
+ - `adam_beta2`: 0.999
231
+ - `adam_epsilon`: 1e-08
232
+ - `max_grad_norm`: 1.0
233
+ - `num_train_epochs`: 1
234
+ - `max_steps`: -1
235
+ - `lr_scheduler_type`: linear
236
+ - `lr_scheduler_kwargs`: {}
237
+ - `warmup_ratio`: 0.1
238
+ - `warmup_steps`: 0
239
+ - `log_level`: passive
240
+ - `log_level_replica`: warning
241
+ - `log_on_each_node`: True
242
+ - `logging_nan_inf_filter`: True
243
+ - `save_safetensors`: True
244
+ - `save_on_each_node`: False
245
+ - `save_only_model`: False
246
+ - `restore_callback_states_from_checkpoint`: False
247
+ - `no_cuda`: False
248
+ - `use_cpu`: False
249
+ - `use_mps_device`: False
250
+ - `seed`: 12
251
+ - `data_seed`: None
252
+ - `jit_mode_eval`: False
253
+ - `use_ipex`: False
254
+ - `bf16`: True
255
+ - `fp16`: False
256
+ - `fp16_opt_level`: O1
257
+ - `half_precision_backend`: auto
258
+ - `bf16_full_eval`: False
259
+ - `fp16_full_eval`: False
260
+ - `tf32`: None
261
+ - `local_rank`: 0
262
+ - `ddp_backend`: None
263
+ - `tpu_num_cores`: None
264
+ - `tpu_metrics_debug`: False
265
+ - `debug`: []
266
+ - `dataloader_drop_last`: False
267
+ - `dataloader_num_workers`: 4
268
+ - `dataloader_prefetch_factor`: None
269
+ - `past_index`: -1
270
+ - `disable_tqdm`: False
271
+ - `remove_unused_columns`: True
272
+ - `label_names`: None
273
+ - `load_best_model_at_end`: True
274
+ - `ignore_data_skip`: False
275
+ - `fsdp`: []
276
+ - `fsdp_min_num_params`: 0
277
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
278
+ - `fsdp_transformer_layer_cls_to_wrap`: None
279
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
280
+ - `deepspeed`: None
281
+ - `label_smoothing_factor`: 0.0
282
+ - `optim`: adamw_torch
283
+ - `optim_args`: None
284
+ - `adafactor`: False
285
+ - `group_by_length`: False
286
+ - `length_column_name`: length
287
+ - `ddp_find_unused_parameters`: None
288
+ - `ddp_bucket_cap_mb`: None
289
+ - `ddp_broadcast_buffers`: False
290
+ - `dataloader_pin_memory`: True
291
+ - `dataloader_persistent_workers`: False
292
+ - `skip_memory_metrics`: True
293
+ - `use_legacy_prediction_loop`: False
294
+ - `push_to_hub`: False
295
+ - `resume_from_checkpoint`: None
296
+ - `hub_model_id`: None
297
+ - `hub_strategy`: every_save
298
+ - `hub_private_repo`: None
299
+ - `hub_always_push`: False
300
+ - `gradient_checkpointing`: False
301
+ - `gradient_checkpointing_kwargs`: None
302
+ - `include_inputs_for_metrics`: False
303
+ - `include_for_metrics`: []
304
+ - `eval_do_concat_batches`: True
305
+ - `fp16_backend`: auto
306
+ - `push_to_hub_model_id`: None
307
+ - `push_to_hub_organization`: None
308
+ - `mp_parameters`:
309
+ - `auto_find_batch_size`: False
310
+ - `full_determinism`: False
311
+ - `torchdynamo`: None
312
+ - `ray_scope`: last
313
+ - `ddp_timeout`: 1800
314
+ - `torch_compile`: False
315
+ - `torch_compile_backend`: None
316
+ - `torch_compile_mode`: None
317
+ - `dispatch_batches`: None
318
+ - `split_batches`: None
319
+ - `include_tokens_per_second`: False
320
+ - `include_num_input_tokens_seen`: False
321
+ - `neftune_noise_alpha`: None
322
+ - `optim_target_modules`: None
323
+ - `batch_eval_metrics`: False
324
+ - `eval_on_start`: False
325
+ - `use_liger_kernel`: False
326
+ - `eval_use_gather_object`: False
327
+ - `average_tokens_across_devices`: False
328
+ - `prompts`: None
329
+ - `batch_sampler`: batch_sampler
330
+ - `multi_dataset_batch_sampler`: proportional
331
+
332
+ </details>
333
+
334
+ ### Training Logs
335
+ <details><summary>Click to expand</summary>
336
+
337
+ | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_ndcg@10 | NanoNFCorpus_ndcg@10 | NanoNQ_ndcg@10 | NanoBEIR_mean_ndcg@10 |
338
+ |:----------:|:---------:|:-------------:|:---------------:|:--------------------:|:--------------------:|:--------------------:|:---------------------:|
339
+ | -1 | -1 | - | - | 0.0245 (-0.5159) | 0.2709 (-0.0541) | 0.0858 (-0.4148) | 0.1271 (-0.3283) |
340
+ | 0.0000 | 1 | 1.1359 | - | - | - | - | - |
341
+ | 0.0064 | 1000 | 0.9962 | - | - | - | - | - |
342
+ | 0.0128 | 2000 | 0.3958 | - | - | - | - | - |
343
+ | 0.0192 | 3000 | 0.3292 | - | - | - | - | - |
344
+ | 0.0256 | 4000 | 0.3023 | - | - | - | - | - |
345
+ | 0.0320 | 5000 | 0.2839 | 0.2495 | 0.6231 (+0.0827) | 0.3748 (+0.0498) | 0.7295 (+0.2288) | 0.5758 (+0.1204) |
346
+ | 0.0384 | 6000 | 0.2812 | - | - | - | - | - |
347
+ | 0.0448 | 7000 | 0.2755 | - | - | - | - | - |
348
+ | 0.0512 | 8000 | 0.2672 | - | - | - | - | - |
349
+ | 0.0576 | 9000 | 0.2624 | - | - | - | - | - |
350
+ | 0.0640 | 10000 | 0.2627 | 0.2368 | 0.6976 (+0.1572) | 0.4094 (+0.0844) | 0.7139 (+0.2133) | 0.6070 (+0.1516) |
351
+ | 0.0704 | 11000 | 0.2586 | - | - | - | - | - |
352
+ | 0.0768 | 12000 | 0.253 | - | - | - | - | - |
353
+ | 0.0832 | 13000 | 0.25 | - | - | - | - | - |
354
+ | 0.0896 | 14000 | 0.2545 | - | - | - | - | - |
355
+ | 0.0960 | 15000 | 0.2516 | 0.2297 | 0.6671 (+0.1267) | 0.3685 (+0.0434) | 0.7301 (+0.2295) | 0.5886 (+0.1332) |
356
+ | 0.1025 | 16000 | 0.241 | - | - | - | - | - |
357
+ | 0.1089 | 17000 | 0.2459 | - | - | - | - | - |
358
+ | 0.1153 | 18000 | 0.2371 | - | - | - | - | - |
359
+ | 0.1217 | 19000 | 0.2374 | - | - | - | - | - |
360
+ | 0.1281 | 20000 | 0.234 | 0.2226 | 0.6377 (+0.0973) | 0.3988 (+0.0737) | 0.7002 (+0.1996) | 0.5789 (+0.1235) |
361
+ | 0.1345 | 21000 | 0.2293 | - | - | - | - | - |
362
+ | 0.1409 | 22000 | 0.2222 | - | - | - | - | - |
363
+ | 0.1473 | 23000 | 0.2231 | - | - | - | - | - |
364
+ | 0.1537 | 24000 | 0.2212 | - | - | - | - | - |
365
+ | 0.1601 | 25000 | 0.2165 | 0.2266 | 0.7114 (+0.1710) | 0.3775 (+0.0524) | 0.7314 (+0.2308) | 0.6068 (+0.1514) |
366
+ | 0.1665 | 26000 | 0.2119 | - | - | - | - | - |
367
+ | 0.1729 | 27000 | 0.2086 | - | - | - | - | - |
368
+ | 0.1793 | 28000 | 0.204 | - | - | - | - | - |
369
+ | 0.1857 | 29000 | 0.204 | - | - | - | - | - |
370
+ | 0.1921 | 30000 | 0.1959 | 0.1913 | 0.6630 (+0.1225) | 0.3962 (+0.0712) | 0.7027 (+0.2020) | 0.5873 (+0.1319) |
371
+ | 0.1985 | 31000 | 0.195 | - | - | - | - | - |
372
+ | 0.2049 | 32000 | 0.1899 | - | - | - | - | - |
373
+ | 0.2113 | 33000 | 0.1887 | - | - | - | - | - |
374
+ | 0.2177 | 34000 | 0.1865 | - | - | - | - | - |
375
+ | 0.2241 | 35000 | 0.1878 | 0.1765 | 0.6709 (+0.1304) | 0.3858 (+0.0607) | 0.7060 (+0.2053) | 0.5875 (+0.1322) |
376
+ | 0.2305 | 36000 | 0.1822 | - | - | - | - | - |
377
+ | 0.2369 | 37000 | 0.1795 | - | - | - | - | - |
378
+ | 0.2433 | 38000 | 0.1802 | - | - | - | - | - |
379
+ | 0.2497 | 39000 | 0.1762 | - | - | - | - | - |
380
+ | 0.2561 | 40000 | 0.1694 | 0.1739 | 0.6902 (+0.1498) | 0.3771 (+0.0521) | 0.7198 (+0.2192) | 0.5957 (+0.1403) |
381
+ | 0.2625 | 41000 | 0.1718 | - | - | - | - | - |
382
+ | 0.2689 | 42000 | 0.1706 | - | - | - | - | - |
383
+ | 0.2753 | 43000 | 0.1659 | - | - | - | - | - |
384
+ | 0.2817 | 44000 | 0.1593 | - | - | - | - | - |
385
+ | 0.2881 | 45000 | 0.1608 | 0.1532 | 0.7132 (+0.1728) | 0.3606 (+0.0356) | 0.7393 (+0.2386) | 0.6044 (+0.1490) |
386
+ | 0.2945 | 46000 | 0.1589 | - | - | - | - | - |
387
+ | 0.3010 | 47000 | 0.1563 | - | - | - | - | - |
388
+ | 0.3074 | 48000 | 0.1553 | - | - | - | - | - |
389
+ | 0.3138 | 49000 | 0.155 | - | - | - | - | - |
390
+ | 0.3202 | 50000 | 0.1501 | 0.1373 | 0.7168 (+0.1764) | 0.3830 (+0.0579) | 0.6954 (+0.1948) | 0.5984 (+0.1430) |
391
+ | 0.3266 | 51000 | 0.1508 | - | - | - | - | - |
392
+ | 0.3330 | 52000 | 0.1497 | - | - | - | - | - |
393
+ | 0.3394 | 53000 | 0.1478 | - | - | - | - | - |
394
+ | 0.3458 | 54000 | 0.1445 | - | - | - | - | - |
395
+ | 0.3522 | 55000 | 0.1468 | 0.1403 | 0.6828 (+0.1424) | 0.3780 (+0.0530) | 0.7147 (+0.2141) | 0.5919 (+0.1365) |
396
+ | 0.3586 | 56000 | 0.1422 | - | - | - | - | - |
397
+ | 0.3650 | 57000 | 0.1369 | - | - | - | - | - |
398
+ | 0.3714 | 58000 | 0.1364 | - | - | - | - | - |
399
+ | 0.3778 | 59000 | 0.1328 | - | - | - | - | - |
400
+ | 0.3842 | 60000 | 0.1351 | 0.1448 | 0.6881 (+0.1477) | 0.3430 (+0.0179) | 0.7267 (+0.2260) | 0.5859 (+0.1306) |
401
+ | 0.3906 | 61000 | 0.1312 | - | - | - | - | - |
402
+ | 0.3970 | 62000 | 0.1308 | - | - | - | - | - |
403
+ | 0.4034 | 63000 | 0.1289 | - | - | - | - | - |
404
+ | 0.4098 | 64000 | 0.1273 | - | - | - | - | - |
405
+ | 0.4162 | 65000 | 0.1257 | 0.1290 | 0.7288 (+0.1883) | 0.3830 (+0.0580) | 0.7180 (+0.2173) | 0.6099 (+0.1545) |
406
+ | 0.4226 | 66000 | 0.1246 | - | - | - | - | - |
407
+ | 0.4290 | 67000 | 0.1275 | - | - | - | - | - |
408
+ | 0.4354 | 68000 | 0.1246 | - | - | - | - | - |
409
+ | 0.4418 | 69000 | 0.1214 | - | - | - | - | - |
410
+ | 0.4482 | 70000 | 0.115 | 0.1184 | 0.6911 (+0.1506) | 0.3903 (+0.0652) | 0.7189 (+0.2182) | 0.6001 (+0.1447) |
411
+ | 0.4546 | 71000 | 0.113 | - | - | - | - | - |
412
+ | 0.4610 | 72000 | 0.1156 | - | - | - | - | - |
413
+ | 0.4674 | 73000 | 0.1142 | - | - | - | - | - |
414
+ | 0.4738 | 74000 | 0.1133 | - | - | - | - | - |
415
+ | **0.4802** | **75000** | **0.1132** | **0.1194** | **0.7069 (+0.1665)** | **0.3801 (+0.0550)** | **0.7469 (+0.2462)** | **0.6113 (+0.1559)** |
416
+ | 0.4866 | 76000 | 0.1085 | - | - | - | - | - |
417
+ | 0.4930 | 77000 | 0.1095 | - | - | - | - | - |
418
+ | 0.4994 | 78000 | 0.1105 | - | - | - | - | - |
419
+ | 0.5059 | 79000 | 0.1068 | - | - | - | - | - |
420
+ | 0.5123 | 80000 | 0.1039 | 0.1085 | 0.7017 (+0.1612) | 0.3565 (+0.0315) | 0.7199 (+0.2192) | 0.5927 (+0.1373) |
421
+ | 0.5187 | 81000 | 0.1059 | - | - | - | - | - |
422
+ | 0.5251 | 82000 | 0.1001 | - | - | - | - | - |
423
+ | 0.5315 | 83000 | 0.1019 | - | - | - | - | - |
424
+ | 0.5379 | 84000 | 0.1021 | - | - | - | - | - |
425
+ | 0.5443 | 85000 | 0.0982 | 0.0962 | 0.6842 (+0.1438) | 0.3516 (+0.0266) | 0.7431 (+0.2425) | 0.5930 (+0.1376) |
426
+ | 0.5507 | 86000 | 0.0967 | - | - | - | - | - |
427
+ | 0.5571 | 87000 | 0.0962 | - | - | - | - | - |
428
+ | 0.5635 | 88000 | 0.098 | - | - | - | - | - |
429
+ | 0.5699 | 89000 | 0.0973 | - | - | - | - | - |
430
+ | 0.5763 | 90000 | 0.0957 | 0.0863 | 0.6729 (+0.1325) | 0.3852 (+0.0601) | 0.7147 (+0.2141) | 0.5909 (+0.1356) |
431
+ | 0.5827 | 91000 | 0.0925 | - | - | - | - | - |
432
+ | 0.5891 | 92000 | 0.0948 | - | - | - | - | - |
433
+ | 0.5955 | 93000 | 0.0887 | - | - | - | - | - |
434
+ | 0.6019 | 94000 | 0.0918 | - | - | - | - | - |
435
+ | 0.6083 | 95000 | 0.0926 | 0.0846 | 0.6857 (+0.1453) | 0.3503 (+0.0253) | 0.7321 (+0.2315) | 0.5894 (+0.1340) |
436
+ | 0.6147 | 96000 | 0.0881 | - | - | - | - | - |
437
+ | 0.6211 | 97000 | 0.0871 | - | - | - | - | - |
438
+ | 0.6275 | 98000 | 0.0867 | - | - | - | - | - |
439
+ | 0.6339 | 99000 | 0.0854 | - | - | - | - | - |
440
+ | 0.6403 | 100000 | 0.0833 | 0.0790 | 0.6665 (+0.1261) | 0.3415 (+0.0165) | 0.6905 (+0.1898) | 0.5662 (+0.1108) |
441
+ | 0.6467 | 101000 | 0.0837 | - | - | - | - | - |
442
+ | 0.6531 | 102000 | 0.0834 | - | - | - | - | - |
443
+ | 0.6595 | 103000 | 0.0798 | - | - | - | - | - |
444
+ | 0.6659 | 104000 | 0.0825 | - | - | - | - | - |
445
+ | 0.6723 | 105000 | 0.0803 | 0.0750 | 0.6897 (+0.1493) | 0.3415 (+0.0165) | 0.7096 (+0.2090) | 0.5803 (+0.1249) |
446
+ | 0.6787 | 106000 | 0.076 | - | - | - | - | - |
447
+ | 0.6851 | 107000 | 0.0782 | - | - | - | - | - |
448
+ | 0.6915 | 108000 | 0.0786 | - | - | - | - | - |
449
+ | 0.6979 | 109000 | 0.075 | - | - | - | - | - |
450
+ | 0.7044 | 110000 | 0.0747 | 0.0690 | 0.6665 (+0.1261) | 0.3384 (+0.0134) | 0.7209 (+0.2202) | 0.5753 (+0.1199) |
451
+ | 0.7108 | 111000 | 0.0728 | - | - | - | - | - |
452
+ | 0.7172 | 112000 | 0.0708 | - | - | - | - | - |
453
+ | 0.7236 | 113000 | 0.0714 | - | - | - | - | - |
454
+ | 0.7300 | 114000 | 0.0725 | - | - | - | - | - |
455
+ | 0.7364 | 115000 | 0.0708 | 0.0659 | 0.6753 (+0.1348) | 0.3423 (+0.0172) | 0.7093 (+0.2087) | 0.5756 (+0.1202) |
456
+ | 0.7428 | 116000 | 0.0684 | - | - | - | - | - |
457
+ | 0.7492 | 117000 | 0.0709 | - | - | - | - | - |
458
+ | 0.7556 | 118000 | 0.0661 | - | - | - | - | - |
459
+ | 0.7620 | 119000 | 0.0685 | - | - | - | - | - |
460
+ | 0.7684 | 120000 | 0.0655 | 0.0613 | 0.6774 (+0.1369) | 0.3295 (+0.0044) | 0.7244 (+0.2238) | 0.5771 (+0.1217) |
461
+ | 0.7748 | 121000 | 0.0643 | - | - | - | - | - |
462
+ | 0.7812 | 122000 | 0.066 | - | - | - | - | - |
463
+ | 0.7876 | 123000 | 0.0625 | - | - | - | - | - |
464
+ | 0.7940 | 124000 | 0.0653 | - | - | - | - | - |
465
+ | 0.8004 | 125000 | 0.0619 | 0.0564 | 0.6797 (+0.1393) | 0.3598 (+0.0348) | 0.7193 (+0.2187) | 0.5863 (+0.1309) |
466
+ | 0.8068 | 126000 | 0.0616 | - | - | - | - | - |
467
+ | 0.8132 | 127000 | 0.0607 | - | - | - | - | - |
468
+ | 0.8196 | 128000 | 0.0584 | - | - | - | - | - |
469
+ | 0.8260 | 129000 | 0.0609 | - | - | - | - | - |
470
+ | 0.8324 | 130000 | 0.0568 | 0.0502 | 0.6855 (+0.1450) | 0.3394 (+0.0143) | 0.7297 (+0.2291) | 0.5849 (+0.1295) |
471
+ | 0.8388 | 131000 | 0.0577 | - | - | - | - | - |
472
+ | 0.8452 | 132000 | 0.056 | - | - | - | - | - |
473
+ | 0.8516 | 133000 | 0.0556 | - | - | - | - | - |
474
+ | 0.8580 | 134000 | 0.0553 | - | - | - | - | - |
475
+ | 0.8644 | 135000 | 0.0546 | 0.0471 | 0.6903 (+0.1499) | 0.3404 (+0.0153) | 0.7419 (+0.2413) | 0.5909 (+0.1355) |
476
+ | 0.8708 | 136000 | 0.0525 | - | - | - | - | - |
477
+ | 0.8772 | 137000 | 0.0512 | - | - | - | - | - |
478
+ | 0.8836 | 138000 | 0.0528 | - | - | - | - | - |
479
+ | 0.8900 | 139000 | 0.0523 | - | - | - | - | - |
480
+ | 0.8964 | 140000 | 0.0544 | 0.0442 | 0.6915 (+0.1511) | 0.3507 (+0.0257) | 0.7258 (+0.2251) | 0.5893 (+0.1340) |
481
+ | 0.9029 | 141000 | 0.0497 | - | - | - | - | - |
482
+ | 0.9093 | 142000 | 0.0508 | - | - | - | - | - |
483
+ | 0.9157 | 143000 | 0.0485 | - | - | - | - | - |
484
+ | 0.9221 | 144000 | 0.0492 | - | - | - | - | - |
485
+ | 0.9285 | 145000 | 0.0472 | 0.0442 | 0.6614 (+0.1210) | 0.3394 (+0.0144) | 0.7361 (+0.2355) | 0.5790 (+0.1236) |
486
+ | 0.9349 | 146000 | 0.0469 | - | - | - | - | - |
487
+ | 0.9413 | 147000 | 0.0459 | - | - | - | - | - |
488
+ | 0.9477 | 148000 | 0.0471 | - | - | - | - | - |
489
+ | 0.9541 | 149000 | 0.0454 | - | - | - | - | - |
490
+ | 0.9605 | 150000 | 0.0444 | 0.0429 | 0.6587 (+0.1183) | 0.3311 (+0.0060) | 0.7298 (+0.2291) | 0.5732 (+0.1178) |
491
+ | 0.9669 | 151000 | 0.0451 | - | - | - | - | - |
492
+ | 0.9733 | 152000 | 0.0429 | - | - | - | - | - |
493
+ | 0.9797 | 153000 | 0.0448 | - | - | - | - | - |
494
+ | 0.9861 | 154000 | 0.0441 | - | - | - | - | - |
495
+ | 0.9925 | 155000 | 0.0443 | 0.0418 | 0.6653 (+0.1249) | 0.3335 (+0.0084) | 0.7391 (+0.2385) | 0.5793 (+0.1239) |
496
+ | 0.9989 | 156000 | 0.0409 | - | - | - | - | - |
497
+ | -1 | -1 | - | - | 0.7069 (+0.1665) | 0.3801 (+0.0550) | 0.7469 (+0.2462) | 0.6113 (+0.1559) |
498
+
499
+ * The bold row denotes the saved checkpoint.
500
+ </details>
501
+
502
+ ### Framework Versions
503
+ - Python: 3.11.10
504
+ - Sentence Transformers: 3.5.0.dev0
505
+ - Transformers: 4.49.0.dev0
506
+ - PyTorch: 2.6.0.dev20241112+cu121
507
+ - Accelerate: 1.2.0
508
+ - Datasets: 3.2.0
509
+ - Tokenizers: 0.21.0
510
+
511
+ ## Citation
512
+
513
+ ### BibTeX
514
+
515
+ #### Sentence Transformers
516
+ ```bibtex
517
+ @inproceedings{reimers-2019-sentence-bert,
518
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
519
+ author = "Reimers, Nils and Gurevych, Iryna",
520
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
521
+ month = "11",
522
+ year = "2019",
523
+ publisher = "Association for Computational Linguistics",
524
+ url = "https://arxiv.org/abs/1908.10084",
525
+ }
526
+ ```
527
+
528
+ <!--
529
+ ## Glossary
530
+
531
+ *Clearly define terms in order to be accessible across audiences.*
532
+ -->
533
+
534
+ <!--
535
+ ## Model Card Authors
536
+
537
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
538
+ -->
539
+
540
+ <!--
541
+ ## Model Card Contact
542
+
543
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
544
+ -->
config.json ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "answerdotai/ModernBERT-base",
3
+ "architectures": [
4
+ "ModernBertForSequenceClassification"
5
+ ],
6
+ "attention_bias": false,
7
+ "attention_dropout": 0.0,
8
+ "bos_token_id": 50281,
9
+ "classifier_activation": "gelu",
10
+ "classifier_bias": false,
11
+ "classifier_dropout": 0.0,
12
+ "classifier_pooling": "mean",
13
+ "cls_token_id": 50281,
14
+ "decoder_bias": true,
15
+ "deterministic_flash_attn": false,
16
+ "embedding_dropout": 0.0,
17
+ "eos_token_id": 50282,
18
+ "global_attn_every_n_layers": 3,
19
+ "global_rope_theta": 160000.0,
20
+ "gradient_checkpointing": false,
21
+ "hidden_activation": "gelu",
22
+ "hidden_size": 768,
23
+ "id2label": {
24
+ "0": "LABEL_0"
25
+ },
26
+ "initializer_cutoff_factor": 2.0,
27
+ "initializer_range": 0.02,
28
+ "intermediate_size": 1152,
29
+ "label2id": {
30
+ "LABEL_0": 0
31
+ },
32
+ "layer_norm_eps": 1e-05,
33
+ "local_attention": 128,
34
+ "local_rope_theta": 10000.0,
35
+ "max_position_embeddings": 8192,
36
+ "mlp_bias": false,
37
+ "mlp_dropout": 0.0,
38
+ "model_type": "modernbert",
39
+ "norm_bias": false,
40
+ "norm_eps": 1e-05,
41
+ "num_attention_heads": 12,
42
+ "num_hidden_layers": 22,
43
+ "pad_token_id": 50283,
44
+ "position_embedding_type": "absolute",
45
+ "reference_compile": true,
46
+ "repad_logits_with_grad": false,
47
+ "sep_token_id": 50282,
48
+ "sparse_pred_ignore_index": -100,
49
+ "sparse_prediction": false,
50
+ "torch_dtype": "float32",
51
+ "transformers_version": "4.49.0.dev0",
52
+ "vocab_size": 50368
53
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8e06dd214234d652df135f5dcfeca884e13091f2bcee3267721650eb11b5b9f
3
+ size 598436708
special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": true,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,945 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "|||IP_ADDRESS|||",
5
+ "lstrip": false,
6
+ "normalized": true,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": false
10
+ },
11
+ "1": {
12
+ "content": "<|padding|>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "50254": {
20
+ "content": " ",
21
+ "lstrip": false,
22
+ "normalized": true,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": false
26
+ },
27
+ "50255": {
28
+ "content": " ",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": false
34
+ },
35
+ "50256": {
36
+ "content": " ",
37
+ "lstrip": false,
38
+ "normalized": true,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": false
42
+ },
43
+ "50257": {
44
+ "content": " ",
45
+ "lstrip": false,
46
+ "normalized": true,
47
+ "rstrip": false,
48
+ "single_word": false,
49
+ "special": false
50
+ },
51
+ "50258": {
52
+ "content": " ",
53
+ "lstrip": false,
54
+ "normalized": true,
55
+ "rstrip": false,
56
+ "single_word": false,
57
+ "special": false
58
+ },
59
+ "50259": {
60
+ "content": " ",
61
+ "lstrip": false,
62
+ "normalized": true,
63
+ "rstrip": false,
64
+ "single_word": false,
65
+ "special": false
66
+ },
67
+ "50260": {
68
+ "content": " ",
69
+ "lstrip": false,
70
+ "normalized": true,
71
+ "rstrip": false,
72
+ "single_word": false,
73
+ "special": false
74
+ },
75
+ "50261": {
76
+ "content": " ",
77
+ "lstrip": false,
78
+ "normalized": true,
79
+ "rstrip": false,
80
+ "single_word": false,
81
+ "special": false
82
+ },
83
+ "50262": {
84
+ "content": " ",
85
+ "lstrip": false,
86
+ "normalized": true,
87
+ "rstrip": false,
88
+ "single_word": false,
89
+ "special": false
90
+ },
91
+ "50263": {
92
+ "content": " ",
93
+ "lstrip": false,
94
+ "normalized": true,
95
+ "rstrip": false,
96
+ "single_word": false,
97
+ "special": false
98
+ },
99
+ "50264": {
100
+ "content": " ",
101
+ "lstrip": false,
102
+ "normalized": true,
103
+ "rstrip": false,
104
+ "single_word": false,
105
+ "special": false
106
+ },
107
+ "50265": {
108
+ "content": " ",
109
+ "lstrip": false,
110
+ "normalized": true,
111
+ "rstrip": false,
112
+ "single_word": false,
113
+ "special": false
114
+ },
115
+ "50266": {
116
+ "content": " ",
117
+ "lstrip": false,
118
+ "normalized": true,
119
+ "rstrip": false,
120
+ "single_word": false,
121
+ "special": false
122
+ },
123
+ "50267": {
124
+ "content": " ",
125
+ "lstrip": false,
126
+ "normalized": true,
127
+ "rstrip": false,
128
+ "single_word": false,
129
+ "special": false
130
+ },
131
+ "50268": {
132
+ "content": " ",
133
+ "lstrip": false,
134
+ "normalized": true,
135
+ "rstrip": false,
136
+ "single_word": false,
137
+ "special": false
138
+ },
139
+ "50269": {
140
+ "content": " ",
141
+ "lstrip": false,
142
+ "normalized": true,
143
+ "rstrip": false,
144
+ "single_word": false,
145
+ "special": false
146
+ },
147
+ "50270": {
148
+ "content": " ",
149
+ "lstrip": false,
150
+ "normalized": true,
151
+ "rstrip": false,
152
+ "single_word": false,
153
+ "special": false
154
+ },
155
+ "50271": {
156
+ "content": " ",
157
+ "lstrip": false,
158
+ "normalized": true,
159
+ "rstrip": false,
160
+ "single_word": false,
161
+ "special": false
162
+ },
163
+ "50272": {
164
+ "content": " ",
165
+ "lstrip": false,
166
+ "normalized": true,
167
+ "rstrip": false,
168
+ "single_word": false,
169
+ "special": false
170
+ },
171
+ "50273": {
172
+ "content": " ",
173
+ "lstrip": false,
174
+ "normalized": true,
175
+ "rstrip": false,
176
+ "single_word": false,
177
+ "special": false
178
+ },
179
+ "50274": {
180
+ "content": " ",
181
+ "lstrip": false,
182
+ "normalized": true,
183
+ "rstrip": false,
184
+ "single_word": false,
185
+ "special": false
186
+ },
187
+ "50275": {
188
+ "content": " ",
189
+ "lstrip": false,
190
+ "normalized": true,
191
+ "rstrip": false,
192
+ "single_word": false,
193
+ "special": false
194
+ },
195
+ "50276": {
196
+ "content": " ",
197
+ "lstrip": false,
198
+ "normalized": true,
199
+ "rstrip": false,
200
+ "single_word": false,
201
+ "special": false
202
+ },
203
+ "50277": {
204
+ "content": "|||EMAIL_ADDRESS|||",
205
+ "lstrip": false,
206
+ "normalized": true,
207
+ "rstrip": false,
208
+ "single_word": false,
209
+ "special": false
210
+ },
211
+ "50278": {
212
+ "content": "|||PHONE_NUMBER|||",
213
+ "lstrip": false,
214
+ "normalized": true,
215
+ "rstrip": false,
216
+ "single_word": false,
217
+ "special": false
218
+ },
219
+ "50279": {
220
+ "content": "<|endoftext|>",
221
+ "lstrip": false,
222
+ "normalized": false,
223
+ "rstrip": false,
224
+ "single_word": false,
225
+ "special": true
226
+ },
227
+ "50280": {
228
+ "content": "[UNK]",
229
+ "lstrip": false,
230
+ "normalized": false,
231
+ "rstrip": false,
232
+ "single_word": false,
233
+ "special": true
234
+ },
235
+ "50281": {
236
+ "content": "[CLS]",
237
+ "lstrip": false,
238
+ "normalized": false,
239
+ "rstrip": false,
240
+ "single_word": false,
241
+ "special": true
242
+ },
243
+ "50282": {
244
+ "content": "[SEP]",
245
+ "lstrip": false,
246
+ "normalized": false,
247
+ "rstrip": false,
248
+ "single_word": false,
249
+ "special": true
250
+ },
251
+ "50283": {
252
+ "content": "[PAD]",
253
+ "lstrip": false,
254
+ "normalized": false,
255
+ "rstrip": false,
256
+ "single_word": false,
257
+ "special": true
258
+ },
259
+ "50284": {
260
+ "content": "[MASK]",
261
+ "lstrip": true,
262
+ "normalized": false,
263
+ "rstrip": false,
264
+ "single_word": false,
265
+ "special": true
266
+ },
267
+ "50285": {
268
+ "content": "[unused0]",
269
+ "lstrip": false,
270
+ "normalized": true,
271
+ "rstrip": false,
272
+ "single_word": false,
273
+ "special": false
274
+ },
275
+ "50286": {
276
+ "content": "[unused1]",
277
+ "lstrip": false,
278
+ "normalized": true,
279
+ "rstrip": false,
280
+ "single_word": false,
281
+ "special": false
282
+ },
283
+ "50287": {
284
+ "content": "[unused2]",
285
+ "lstrip": false,
286
+ "normalized": true,
287
+ "rstrip": false,
288
+ "single_word": false,
289
+ "special": false
290
+ },
291
+ "50288": {
292
+ "content": "[unused3]",
293
+ "lstrip": false,
294
+ "normalized": true,
295
+ "rstrip": false,
296
+ "single_word": false,
297
+ "special": false
298
+ },
299
+ "50289": {
300
+ "content": "[unused4]",
301
+ "lstrip": false,
302
+ "normalized": true,
303
+ "rstrip": false,
304
+ "single_word": false,
305
+ "special": false
306
+ },
307
+ "50290": {
308
+ "content": "[unused5]",
309
+ "lstrip": false,
310
+ "normalized": true,
311
+ "rstrip": false,
312
+ "single_word": false,
313
+ "special": false
314
+ },
315
+ "50291": {
316
+ "content": "[unused6]",
317
+ "lstrip": false,
318
+ "normalized": true,
319
+ "rstrip": false,
320
+ "single_word": false,
321
+ "special": false
322
+ },
323
+ "50292": {
324
+ "content": "[unused7]",
325
+ "lstrip": false,
326
+ "normalized": true,
327
+ "rstrip": false,
328
+ "single_word": false,
329
+ "special": false
330
+ },
331
+ "50293": {
332
+ "content": "[unused8]",
333
+ "lstrip": false,
334
+ "normalized": true,
335
+ "rstrip": false,
336
+ "single_word": false,
337
+ "special": false
338
+ },
339
+ "50294": {
340
+ "content": "[unused9]",
341
+ "lstrip": false,
342
+ "normalized": true,
343
+ "rstrip": false,
344
+ "single_word": false,
345
+ "special": false
346
+ },
347
+ "50295": {
348
+ "content": "[unused10]",
349
+ "lstrip": false,
350
+ "normalized": true,
351
+ "rstrip": false,
352
+ "single_word": false,
353
+ "special": false
354
+ },
355
+ "50296": {
356
+ "content": "[unused11]",
357
+ "lstrip": false,
358
+ "normalized": true,
359
+ "rstrip": false,
360
+ "single_word": false,
361
+ "special": false
362
+ },
363
+ "50297": {
364
+ "content": "[unused12]",
365
+ "lstrip": false,
366
+ "normalized": true,
367
+ "rstrip": false,
368
+ "single_word": false,
369
+ "special": false
370
+ },
371
+ "50298": {
372
+ "content": "[unused13]",
373
+ "lstrip": false,
374
+ "normalized": true,
375
+ "rstrip": false,
376
+ "single_word": false,
377
+ "special": false
378
+ },
379
+ "50299": {
380
+ "content": "[unused14]",
381
+ "lstrip": false,
382
+ "normalized": true,
383
+ "rstrip": false,
384
+ "single_word": false,
385
+ "special": false
386
+ },
387
+ "50300": {
388
+ "content": "[unused15]",
389
+ "lstrip": false,
390
+ "normalized": true,
391
+ "rstrip": false,
392
+ "single_word": false,
393
+ "special": false
394
+ },
395
+ "50301": {
396
+ "content": "[unused16]",
397
+ "lstrip": false,
398
+ "normalized": true,
399
+ "rstrip": false,
400
+ "single_word": false,
401
+ "special": false
402
+ },
403
+ "50302": {
404
+ "content": "[unused17]",
405
+ "lstrip": false,
406
+ "normalized": true,
407
+ "rstrip": false,
408
+ "single_word": false,
409
+ "special": false
410
+ },
411
+ "50303": {
412
+ "content": "[unused18]",
413
+ "lstrip": false,
414
+ "normalized": true,
415
+ "rstrip": false,
416
+ "single_word": false,
417
+ "special": false
418
+ },
419
+ "50304": {
420
+ "content": "[unused19]",
421
+ "lstrip": false,
422
+ "normalized": true,
423
+ "rstrip": false,
424
+ "single_word": false,
425
+ "special": false
426
+ },
427
+ "50305": {
428
+ "content": "[unused20]",
429
+ "lstrip": false,
430
+ "normalized": true,
431
+ "rstrip": false,
432
+ "single_word": false,
433
+ "special": false
434
+ },
435
+ "50306": {
436
+ "content": "[unused21]",
437
+ "lstrip": false,
438
+ "normalized": true,
439
+ "rstrip": false,
440
+ "single_word": false,
441
+ "special": false
442
+ },
443
+ "50307": {
444
+ "content": "[unused22]",
445
+ "lstrip": false,
446
+ "normalized": true,
447
+ "rstrip": false,
448
+ "single_word": false,
449
+ "special": false
450
+ },
451
+ "50308": {
452
+ "content": "[unused23]",
453
+ "lstrip": false,
454
+ "normalized": true,
455
+ "rstrip": false,
456
+ "single_word": false,
457
+ "special": false
458
+ },
459
+ "50309": {
460
+ "content": "[unused24]",
461
+ "lstrip": false,
462
+ "normalized": true,
463
+ "rstrip": false,
464
+ "single_word": false,
465
+ "special": false
466
+ },
467
+ "50310": {
468
+ "content": "[unused25]",
469
+ "lstrip": false,
470
+ "normalized": true,
471
+ "rstrip": false,
472
+ "single_word": false,
473
+ "special": false
474
+ },
475
+ "50311": {
476
+ "content": "[unused26]",
477
+ "lstrip": false,
478
+ "normalized": true,
479
+ "rstrip": false,
480
+ "single_word": false,
481
+ "special": false
482
+ },
483
+ "50312": {
484
+ "content": "[unused27]",
485
+ "lstrip": false,
486
+ "normalized": true,
487
+ "rstrip": false,
488
+ "single_word": false,
489
+ "special": false
490
+ },
491
+ "50313": {
492
+ "content": "[unused28]",
493
+ "lstrip": false,
494
+ "normalized": true,
495
+ "rstrip": false,
496
+ "single_word": false,
497
+ "special": false
498
+ },
499
+ "50314": {
500
+ "content": "[unused29]",
501
+ "lstrip": false,
502
+ "normalized": true,
503
+ "rstrip": false,
504
+ "single_word": false,
505
+ "special": false
506
+ },
507
+ "50315": {
508
+ "content": "[unused30]",
509
+ "lstrip": false,
510
+ "normalized": true,
511
+ "rstrip": false,
512
+ "single_word": false,
513
+ "special": false
514
+ },
515
+ "50316": {
516
+ "content": "[unused31]",
517
+ "lstrip": false,
518
+ "normalized": true,
519
+ "rstrip": false,
520
+ "single_word": false,
521
+ "special": false
522
+ },
523
+ "50317": {
524
+ "content": "[unused32]",
525
+ "lstrip": false,
526
+ "normalized": true,
527
+ "rstrip": false,
528
+ "single_word": false,
529
+ "special": false
530
+ },
531
+ "50318": {
532
+ "content": "[unused33]",
533
+ "lstrip": false,
534
+ "normalized": true,
535
+ "rstrip": false,
536
+ "single_word": false,
537
+ "special": false
538
+ },
539
+ "50319": {
540
+ "content": "[unused34]",
541
+ "lstrip": false,
542
+ "normalized": true,
543
+ "rstrip": false,
544
+ "single_word": false,
545
+ "special": false
546
+ },
547
+ "50320": {
548
+ "content": "[unused35]",
549
+ "lstrip": false,
550
+ "normalized": true,
551
+ "rstrip": false,
552
+ "single_word": false,
553
+ "special": false
554
+ },
555
+ "50321": {
556
+ "content": "[unused36]",
557
+ "lstrip": false,
558
+ "normalized": true,
559
+ "rstrip": false,
560
+ "single_word": false,
561
+ "special": false
562
+ },
563
+ "50322": {
564
+ "content": "[unused37]",
565
+ "lstrip": false,
566
+ "normalized": true,
567
+ "rstrip": false,
568
+ "single_word": false,
569
+ "special": false
570
+ },
571
+ "50323": {
572
+ "content": "[unused38]",
573
+ "lstrip": false,
574
+ "normalized": true,
575
+ "rstrip": false,
576
+ "single_word": false,
577
+ "special": false
578
+ },
579
+ "50324": {
580
+ "content": "[unused39]",
581
+ "lstrip": false,
582
+ "normalized": true,
583
+ "rstrip": false,
584
+ "single_word": false,
585
+ "special": false
586
+ },
587
+ "50325": {
588
+ "content": "[unused40]",
589
+ "lstrip": false,
590
+ "normalized": true,
591
+ "rstrip": false,
592
+ "single_word": false,
593
+ "special": false
594
+ },
595
+ "50326": {
596
+ "content": "[unused41]",
597
+ "lstrip": false,
598
+ "normalized": true,
599
+ "rstrip": false,
600
+ "single_word": false,
601
+ "special": false
602
+ },
603
+ "50327": {
604
+ "content": "[unused42]",
605
+ "lstrip": false,
606
+ "normalized": true,
607
+ "rstrip": false,
608
+ "single_word": false,
609
+ "special": false
610
+ },
611
+ "50328": {
612
+ "content": "[unused43]",
613
+ "lstrip": false,
614
+ "normalized": true,
615
+ "rstrip": false,
616
+ "single_word": false,
617
+ "special": false
618
+ },
619
+ "50329": {
620
+ "content": "[unused44]",
621
+ "lstrip": false,
622
+ "normalized": true,
623
+ "rstrip": false,
624
+ "single_word": false,
625
+ "special": false
626
+ },
627
+ "50330": {
628
+ "content": "[unused45]",
629
+ "lstrip": false,
630
+ "normalized": true,
631
+ "rstrip": false,
632
+ "single_word": false,
633
+ "special": false
634
+ },
635
+ "50331": {
636
+ "content": "[unused46]",
637
+ "lstrip": false,
638
+ "normalized": true,
639
+ "rstrip": false,
640
+ "single_word": false,
641
+ "special": false
642
+ },
643
+ "50332": {
644
+ "content": "[unused47]",
645
+ "lstrip": false,
646
+ "normalized": true,
647
+ "rstrip": false,
648
+ "single_word": false,
649
+ "special": false
650
+ },
651
+ "50333": {
652
+ "content": "[unused48]",
653
+ "lstrip": false,
654
+ "normalized": true,
655
+ "rstrip": false,
656
+ "single_word": false,
657
+ "special": false
658
+ },
659
+ "50334": {
660
+ "content": "[unused49]",
661
+ "lstrip": false,
662
+ "normalized": true,
663
+ "rstrip": false,
664
+ "single_word": false,
665
+ "special": false
666
+ },
667
+ "50335": {
668
+ "content": "[unused50]",
669
+ "lstrip": false,
670
+ "normalized": true,
671
+ "rstrip": false,
672
+ "single_word": false,
673
+ "special": false
674
+ },
675
+ "50336": {
676
+ "content": "[unused51]",
677
+ "lstrip": false,
678
+ "normalized": true,
679
+ "rstrip": false,
680
+ "single_word": false,
681
+ "special": false
682
+ },
683
+ "50337": {
684
+ "content": "[unused52]",
685
+ "lstrip": false,
686
+ "normalized": true,
687
+ "rstrip": false,
688
+ "single_word": false,
689
+ "special": false
690
+ },
691
+ "50338": {
692
+ "content": "[unused53]",
693
+ "lstrip": false,
694
+ "normalized": true,
695
+ "rstrip": false,
696
+ "single_word": false,
697
+ "special": false
698
+ },
699
+ "50339": {
700
+ "content": "[unused54]",
701
+ "lstrip": false,
702
+ "normalized": true,
703
+ "rstrip": false,
704
+ "single_word": false,
705
+ "special": false
706
+ },
707
+ "50340": {
708
+ "content": "[unused55]",
709
+ "lstrip": false,
710
+ "normalized": true,
711
+ "rstrip": false,
712
+ "single_word": false,
713
+ "special": false
714
+ },
715
+ "50341": {
716
+ "content": "[unused56]",
717
+ "lstrip": false,
718
+ "normalized": true,
719
+ "rstrip": false,
720
+ "single_word": false,
721
+ "special": false
722
+ },
723
+ "50342": {
724
+ "content": "[unused57]",
725
+ "lstrip": false,
726
+ "normalized": true,
727
+ "rstrip": false,
728
+ "single_word": false,
729
+ "special": false
730
+ },
731
+ "50343": {
732
+ "content": "[unused58]",
733
+ "lstrip": false,
734
+ "normalized": true,
735
+ "rstrip": false,
736
+ "single_word": false,
737
+ "special": false
738
+ },
739
+ "50344": {
740
+ "content": "[unused59]",
741
+ "lstrip": false,
742
+ "normalized": true,
743
+ "rstrip": false,
744
+ "single_word": false,
745
+ "special": false
746
+ },
747
+ "50345": {
748
+ "content": "[unused60]",
749
+ "lstrip": false,
750
+ "normalized": true,
751
+ "rstrip": false,
752
+ "single_word": false,
753
+ "special": false
754
+ },
755
+ "50346": {
756
+ "content": "[unused61]",
757
+ "lstrip": false,
758
+ "normalized": true,
759
+ "rstrip": false,
760
+ "single_word": false,
761
+ "special": false
762
+ },
763
+ "50347": {
764
+ "content": "[unused62]",
765
+ "lstrip": false,
766
+ "normalized": true,
767
+ "rstrip": false,
768
+ "single_word": false,
769
+ "special": false
770
+ },
771
+ "50348": {
772
+ "content": "[unused63]",
773
+ "lstrip": false,
774
+ "normalized": true,
775
+ "rstrip": false,
776
+ "single_word": false,
777
+ "special": false
778
+ },
779
+ "50349": {
780
+ "content": "[unused64]",
781
+ "lstrip": false,
782
+ "normalized": true,
783
+ "rstrip": false,
784
+ "single_word": false,
785
+ "special": false
786
+ },
787
+ "50350": {
788
+ "content": "[unused65]",
789
+ "lstrip": false,
790
+ "normalized": true,
791
+ "rstrip": false,
792
+ "single_word": false,
793
+ "special": false
794
+ },
795
+ "50351": {
796
+ "content": "[unused66]",
797
+ "lstrip": false,
798
+ "normalized": true,
799
+ "rstrip": false,
800
+ "single_word": false,
801
+ "special": false
802
+ },
803
+ "50352": {
804
+ "content": "[unused67]",
805
+ "lstrip": false,
806
+ "normalized": true,
807
+ "rstrip": false,
808
+ "single_word": false,
809
+ "special": false
810
+ },
811
+ "50353": {
812
+ "content": "[unused68]",
813
+ "lstrip": false,
814
+ "normalized": true,
815
+ "rstrip": false,
816
+ "single_word": false,
817
+ "special": false
818
+ },
819
+ "50354": {
820
+ "content": "[unused69]",
821
+ "lstrip": false,
822
+ "normalized": true,
823
+ "rstrip": false,
824
+ "single_word": false,
825
+ "special": false
826
+ },
827
+ "50355": {
828
+ "content": "[unused70]",
829
+ "lstrip": false,
830
+ "normalized": true,
831
+ "rstrip": false,
832
+ "single_word": false,
833
+ "special": false
834
+ },
835
+ "50356": {
836
+ "content": "[unused71]",
837
+ "lstrip": false,
838
+ "normalized": true,
839
+ "rstrip": false,
840
+ "single_word": false,
841
+ "special": false
842
+ },
843
+ "50357": {
844
+ "content": "[unused72]",
845
+ "lstrip": false,
846
+ "normalized": true,
847
+ "rstrip": false,
848
+ "single_word": false,
849
+ "special": false
850
+ },
851
+ "50358": {
852
+ "content": "[unused73]",
853
+ "lstrip": false,
854
+ "normalized": true,
855
+ "rstrip": false,
856
+ "single_word": false,
857
+ "special": false
858
+ },
859
+ "50359": {
860
+ "content": "[unused74]",
861
+ "lstrip": false,
862
+ "normalized": true,
863
+ "rstrip": false,
864
+ "single_word": false,
865
+ "special": false
866
+ },
867
+ "50360": {
868
+ "content": "[unused75]",
869
+ "lstrip": false,
870
+ "normalized": true,
871
+ "rstrip": false,
872
+ "single_word": false,
873
+ "special": false
874
+ },
875
+ "50361": {
876
+ "content": "[unused76]",
877
+ "lstrip": false,
878
+ "normalized": true,
879
+ "rstrip": false,
880
+ "single_word": false,
881
+ "special": false
882
+ },
883
+ "50362": {
884
+ "content": "[unused77]",
885
+ "lstrip": false,
886
+ "normalized": true,
887
+ "rstrip": false,
888
+ "single_word": false,
889
+ "special": false
890
+ },
891
+ "50363": {
892
+ "content": "[unused78]",
893
+ "lstrip": false,
894
+ "normalized": true,
895
+ "rstrip": false,
896
+ "single_word": false,
897
+ "special": false
898
+ },
899
+ "50364": {
900
+ "content": "[unused79]",
901
+ "lstrip": false,
902
+ "normalized": true,
903
+ "rstrip": false,
904
+ "single_word": false,
905
+ "special": false
906
+ },
907
+ "50365": {
908
+ "content": "[unused80]",
909
+ "lstrip": false,
910
+ "normalized": true,
911
+ "rstrip": false,
912
+ "single_word": false,
913
+ "special": false
914
+ },
915
+ "50366": {
916
+ "content": "[unused81]",
917
+ "lstrip": false,
918
+ "normalized": true,
919
+ "rstrip": false,
920
+ "single_word": false,
921
+ "special": false
922
+ },
923
+ "50367": {
924
+ "content": "[unused82]",
925
+ "lstrip": false,
926
+ "normalized": true,
927
+ "rstrip": false,
928
+ "single_word": false,
929
+ "special": false
930
+ }
931
+ },
932
+ "clean_up_tokenization_spaces": true,
933
+ "cls_token": "[CLS]",
934
+ "extra_special_tokens": {},
935
+ "mask_token": "[MASK]",
936
+ "model_input_names": [
937
+ "input_ids",
938
+ "attention_mask"
939
+ ],
940
+ "model_max_length": 8192,
941
+ "pad_token": "[PAD]",
942
+ "sep_token": "[SEP]",
943
+ "tokenizer_class": "PreTrainedTokenizerFast",
944
+ "unk_token": "[UNK]"
945
+ }