Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-1128/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "gate_proj",
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5152a3c58b120434eb3a619c481763beeeb3ae8d3013a03ae3db5ec31e2cb89
+size 167832240

checkpoint-1000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-1000/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "gate_proj",
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4d2c742986c2cdd9d955284bccbd2bcb7cc788951f774e020fa5a9f99fd7ae0
+size 167832240

checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ada8b05836cc5ae2c6c2446ef8c880165294f8c70f3f6bbcd271baa1af557bb
+size 335813370

checkpoint-1000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2207d6736302d934878b05fdd29c66e35e66dafa3b25c918b9c331ffb3b15c15
+size 14244

checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40dbf214e8e6729e0c80933c278cb36cadc0d057d2be7ba679767945dbfba9f0
+size 1064

checkpoint-1000/special_tokens_map.json

@@ -0,0 +1,17 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|eot_id|>"
+}

checkpoint-1000/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:384a7e7c676f7be2e5d2e8449c508be9b00e5b18c5b3c39ebc626e96b3f4b988
+size 17210019

checkpoint-1000/tokenizer_config.json

@@ -0,0 +1,2064 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not date_string is defined %}\n    {%- set date_string = \"26 Jul 2024\" %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1:] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message + builtin tools #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if builtin_tools is defined or tools is not none %}\n    {{- \"Environment: ipython\\n\" }}\n{%- endif %}\n{%- if builtin_tools is defined %}\n    {{- \"Tools: \" + builtin_tools | reject('equalto', 'code_interpreter') | join(\", \") + \"\\n\\n\"}}\n{%- endif %}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{%- if tools is not none and not tools_in_user_message %}\n    {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{#- Custom tools are passed in a user message with some extra guidance #}\n{%- if tools_in_user_message and not tools is none %}\n    {#- Extract the first user message so we can plug it in here #}\n    {%- if messages | length != 0 %}\n        {%- set first_user_message = messages[0]['content']|trim %}\n        {%- set messages = messages[1:] %}\n    {%- else %}\n        {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n{%- endif %}\n    {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n    {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n    {{- first_user_message + \"<|eot_id|>\"}}\n{%- endif %}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {%- if not message.tool_calls|length == 1 %}\n            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n        {%- endif %}\n        {%- set tool_call = message.tool_calls[0].function %}\n        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\n            {%- for arg_name, arg_val in tool_call.arguments | items %}\n                {{- arg_name + '=\"' + arg_val + '\"' }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n                {%- endfor %}\n            {{- \")\" }}\n        {%- else  %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n            {{- '\"parameters\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- \"}\" }}\n        {%- endif %}\n        {%- if builtin_tools is defined %}\n            {#- This means we're in ipython mode #}\n            {{- \"<|eom_id|>\" }}\n        {%- else %}\n            {{- \"<|eot_id|>\" }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n        {%- if message.content is mapping or message.content is iterable %}\n            {{- message.content | tojson }}\n        {%- else %}\n            {{- message.content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "pad_token": "<|eot_id|>",
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e535e282bc87e3ec98aee4af3db87dfefcc304b028e382b0a576fd7ab703a7ee
+size 5688

checkpoint-1128/README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-1128/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "gate_proj",
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-1128/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5152a3c58b120434eb3a619c481763beeeb3ae8d3013a03ae3db5ec31e2cb89
+size 167832240

checkpoint-1128/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a8e0699eea8b1b33f7f822f6bb393e789ad59503535540111d871282483a639
+size 335813370

checkpoint-1128/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fad15c94ee06c7f1ee985eed5d7971fb7ac3c9bc95bb759aee7740cc62caee2
+size 14244

checkpoint-1128/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96072dc7498fdf6ea0fc640b287f96a56b4856ad8a2147448dc8d07a81a2ddfb
+size 1064

checkpoint-1128/special_tokens_map.json

@@ -0,0 +1,17 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|eot_id|>"
+}

checkpoint-1128/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:384a7e7c676f7be2e5d2e8449c508be9b00e5b18c5b3c39ebc626e96b3f4b988
+size 17210019

checkpoint-1128/tokenizer_config.json

@@ -0,0 +1,2064 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not date_string is defined %}\n    {%- set date_string = \"26 Jul 2024\" %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1:] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message + builtin tools #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if builtin_tools is defined or tools is not none %}\n    {{- \"Environment: ipython\\n\" }}\n{%- endif %}\n{%- if builtin_tools is defined %}\n    {{- \"Tools: \" + builtin_tools | reject('equalto', 'code_interpreter') | join(\", \") + \"\\n\\n\"}}\n{%- endif %}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{%- if tools is not none and not tools_in_user_message %}\n    {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{#- Custom tools are passed in a user message with some extra guidance #}\n{%- if tools_in_user_message and not tools is none %}\n    {#- Extract the first user message so we can plug it in here #}\n    {%- if messages | length != 0 %}\n        {%- set first_user_message = messages[0]['content']|trim %}\n        {%- set messages = messages[1:] %}\n    {%- else %}\n        {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n{%- endif %}\n    {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n    {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n    {{- first_user_message + \"<|eot_id|>\"}}\n{%- endif %}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {%- if not message.tool_calls|length == 1 %}\n            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n        {%- endif %}\n        {%- set tool_call = message.tool_calls[0].function %}\n        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\n            {%- for arg_name, arg_val in tool_call.arguments | items %}\n                {{- arg_name + '=\"' + arg_val + '\"' }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n                {%- endfor %}\n            {{- \")\" }}\n        {%- else  %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n            {{- '\"parameters\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- \"}\" }}\n        {%- endif %}\n        {%- if builtin_tools is defined %}\n            {#- This means we're in ipython mode #}\n            {{- \"<|eom_id|>\" }}\n        {%- else %}\n            {{- \"<|eot_id|>\" }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n        {%- if message.content is mapping or message.content is iterable %}\n            {{- message.content | tojson }}\n        {%- else %}\n            {{- message.content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "pad_token": "<|eot_id|>",
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

checkpoint-1128/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e535e282bc87e3ec98aee4af3db87dfefcc304b028e382b0a576fd7ab703a7ee
+size 5688

checkpoint-500/README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-500/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "gate_proj",
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-500/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac70779643d6903adc421972af091dc5a1f0069428349fad07440c71e8914306
+size 167832240

checkpoint-500/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:847a6558c88bb7d7d232e957a1a4fa71648b7d940c84a1823654f83d9fb141d8
+size 335813370

checkpoint-500/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc63310eb0d4b818b12317f1219c9f23c42dac5eb4c929fd9958e719d60fad4f
+size 14244

checkpoint-500/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9233f9f3ed7e3ec4269ad1b00b9ff4109dc481be3317306906642252c96042a
+size 1064

checkpoint-500/special_tokens_map.json

@@ -0,0 +1,17 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|eot_id|>"
+}

checkpoint-500/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:384a7e7c676f7be2e5d2e8449c508be9b00e5b18c5b3c39ebc626e96b3f4b988
+size 17210019

checkpoint-500/tokenizer_config.json

@@ -0,0 +1,2064 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not date_string is defined %}\n    {%- set date_string = \"26 Jul 2024\" %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1:] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message + builtin tools #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if builtin_tools is defined or tools is not none %}\n    {{- \"Environment: ipython\\n\" }}\n{%- endif %}\n{%- if builtin_tools is defined %}\n    {{- \"Tools: \" + builtin_tools | reject('equalto', 'code_interpreter') | join(\", \") + \"\\n\\n\"}}\n{%- endif %}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{%- if tools is not none and not tools_in_user_message %}\n    {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{#- Custom tools are passed in a user message with some extra guidance #}\n{%- if tools_in_user_message and not tools is none %}\n    {#- Extract the first user message so we can plug it in here #}\n    {%- if messages | length != 0 %}\n        {%- set first_user_message = messages[0]['content']|trim %}\n        {%- set messages = messages[1:] %}\n    {%- else %}\n        {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n{%- endif %}\n    {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n    {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n    {{- first_user_message + \"<|eot_id|>\"}}\n{%- endif %}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {%- if not message.tool_calls|length == 1 %}\n            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n        {%- endif %}\n        {%- set tool_call = message.tool_calls[0].function %}\n        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\n            {%- for arg_name, arg_val in tool_call.arguments | items %}\n                {{- arg_name + '=\"' + arg_val + '\"' }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n                {%- endfor %}\n            {{- \")\" }}\n        {%- else  %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n            {{- '\"parameters\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- \"}\" }}\n        {%- endif %}\n        {%- if builtin_tools is defined %}\n            {#- This means we're in ipython mode #}\n            {{- \"<|eom_id|>\" }}\n        {%- else %}\n            {{- \"<|eot_id|>\" }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n        {%- if message.content is mapping or message.content is iterable %}\n            {{- message.content | tojson }}\n        {%- else %}\n            {{- message.content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "pad_token": "<|eot_id|>",
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

checkpoint-500/trainer_state.json

@@ -0,0 +1,3549 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.8865248226950354,
+  "eval_steps": 226,
+  "global_step": 500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0017730496453900709,
+      "grad_norm": 6.88893985748291,
+      "learning_rate": 2.9999999999999997e-05,
+      "loss": 4.048,
+      "step": 1
+    },
+    {
+      "epoch": 0.0035460992907801418,
+      "grad_norm": 6.882226467132568,
+      "learning_rate": 5.9999999999999995e-05,
+      "loss": 4.2161,
+      "step": 2
+    },
+    {
+      "epoch": 0.005319148936170213,
+      "grad_norm": 4.995457649230957,
+      "learning_rate": 8.999999999999999e-05,
+      "loss": 3.9501,
+      "step": 3
+    },
+    {
+      "epoch": 0.0070921985815602835,
+      "grad_norm": 3.743339776992798,
+      "learning_rate": 0.00011999999999999999,
+      "loss": 3.7819,
+      "step": 4
+    },
+    {
+      "epoch": 0.008865248226950355,
+      "grad_norm": 3.3828792572021484,
+      "learning_rate": 0.00015,
+      "loss": 3.2351,
+      "step": 5
+    },
+    {
+      "epoch": 0.010638297872340425,
+      "grad_norm": 3.3940675258636475,
+      "learning_rate": 0.00017999999999999998,
+      "loss": 2.7343,
+      "step": 6
+    },
+    {
+      "epoch": 0.012411347517730497,
+      "grad_norm": 3.3435909748077393,
+      "learning_rate": 0.00020999999999999998,
+      "loss": 2.0898,
+      "step": 7
+    },
+    {
+      "epoch": 0.014184397163120567,
+      "grad_norm": 4.3948235511779785,
+      "learning_rate": 0.00023999999999999998,
+      "loss": 1.6608,
+      "step": 8
+    },
+    {
+      "epoch": 0.015957446808510637,
+      "grad_norm": 4.144540309906006,
+      "learning_rate": 0.00027,
+      "loss": 0.9104,
+      "step": 9
+    },
+    {
+      "epoch": 0.01773049645390071,
+      "grad_norm": 5.76711368560791,
+      "learning_rate": 0.0003,
+      "loss": 0.6078,
+      "step": 10
+    },
+    {
+      "epoch": 0.01950354609929078,
+      "grad_norm": 11.037919998168945,
+      "learning_rate": 0.00029973166368515205,
+      "loss": 0.439,
+      "step": 11
+    },
+    {
+      "epoch": 0.02127659574468085,
+      "grad_norm": 5.379396915435791,
+      "learning_rate": 0.00029946332737030407,
+      "loss": 0.5497,
+      "step": 12
+    },
+    {
+      "epoch": 0.02304964539007092,
+      "grad_norm": 6.821712970733643,
+      "learning_rate": 0.00029919499105545615,
+      "loss": 0.6396,
+      "step": 13
+    },
+    {
+      "epoch": 0.024822695035460994,
+      "grad_norm": 3.520746946334839,
+      "learning_rate": 0.00029892665474060817,
+      "loss": 0.4007,
+      "step": 14
+    },
+    {
+      "epoch": 0.026595744680851064,
+      "grad_norm": 1.7975941896438599,
+      "learning_rate": 0.00029865831842576025,
+      "loss": 0.2384,
+      "step": 15
+    },
+    {
+      "epoch": 0.028368794326241134,
+      "grad_norm": 6.044783592224121,
+      "learning_rate": 0.0002983899821109123,
+      "loss": 0.5119,
+      "step": 16
+    },
+    {
+      "epoch": 0.030141843971631204,
+      "grad_norm": 0.9617128968238831,
+      "learning_rate": 0.0002981216457960644,
+      "loss": 0.2032,
+      "step": 17
+    },
+    {
+      "epoch": 0.031914893617021274,
+      "grad_norm": 1.6461282968521118,
+      "learning_rate": 0.0002978533094812164,
+      "loss": 0.3118,
+      "step": 18
+    },
+    {
+      "epoch": 0.03368794326241135,
+      "grad_norm": 1.2195343971252441,
+      "learning_rate": 0.0002975849731663685,
+      "loss": 0.2522,
+      "step": 19
+    },
+    {
+      "epoch": 0.03546099290780142,
+      "grad_norm": 1.5621528625488281,
+      "learning_rate": 0.00029731663685152057,
+      "loss": 0.3118,
+      "step": 20
+    },
+    {
+      "epoch": 0.03723404255319149,
+      "grad_norm": 1.3373510837554932,
+      "learning_rate": 0.0002970483005366726,
+      "loss": 0.2101,
+      "step": 21
+    },
+    {
+      "epoch": 0.03900709219858156,
+      "grad_norm": 0.9622485041618347,
+      "learning_rate": 0.00029677996422182467,
+      "loss": 0.1797,
+      "step": 22
+    },
+    {
+      "epoch": 0.040780141843971635,
+      "grad_norm": 1.7572095394134521,
+      "learning_rate": 0.0002965116279069767,
+      "loss": 0.1338,
+      "step": 23
+    },
+    {
+      "epoch": 0.0425531914893617,
+      "grad_norm": 1.871632695198059,
+      "learning_rate": 0.00029624329159212877,
+      "loss": 0.1457,
+      "step": 24
+    },
+    {
+      "epoch": 0.044326241134751775,
+      "grad_norm": 2.204594612121582,
+      "learning_rate": 0.00029597495527728084,
+      "loss": 0.1621,
+      "step": 25
+    },
+    {
+      "epoch": 0.04609929078014184,
+      "grad_norm": 1.5381090641021729,
+      "learning_rate": 0.0002957066189624329,
+      "loss": 0.1703,
+      "step": 26
+    },
+    {
+      "epoch": 0.047872340425531915,
+      "grad_norm": 0.8811854124069214,
+      "learning_rate": 0.00029543828264758494,
+      "loss": 0.0787,
+      "step": 27
+    },
+    {
+      "epoch": 0.04964539007092199,
+      "grad_norm": 0.8814285397529602,
+      "learning_rate": 0.000295169946332737,
+      "loss": 0.2091,
+      "step": 28
+    },
+    {
+      "epoch": 0.051418439716312055,
+      "grad_norm": 1.371406078338623,
+      "learning_rate": 0.0002949016100178891,
+      "loss": 0.3093,
+      "step": 29
+    },
+    {
+      "epoch": 0.05319148936170213,
+      "grad_norm": 1.1055055856704712,
+      "learning_rate": 0.0002946332737030411,
+      "loss": 0.1644,
+      "step": 30
+    },
+    {
+      "epoch": 0.0549645390070922,
+      "grad_norm": 1.2226752042770386,
+      "learning_rate": 0.0002943649373881932,
+      "loss": 0.1597,
+      "step": 31
+    },
+    {
+      "epoch": 0.05673758865248227,
+      "grad_norm": 1.1774011850357056,
+      "learning_rate": 0.0002940966010733452,
+      "loss": 0.243,
+      "step": 32
+    },
+    {
+      "epoch": 0.05851063829787234,
+      "grad_norm": 1.8663568496704102,
+      "learning_rate": 0.0002938282647584973,
+      "loss": 0.3355,
+      "step": 33
+    },
+    {
+      "epoch": 0.06028368794326241,
+      "grad_norm": 0.7782514095306396,
+      "learning_rate": 0.0002935599284436493,
+      "loss": 0.1426,
+      "step": 34
+    },
+    {
+      "epoch": 0.06205673758865248,
+      "grad_norm": 0.6109825968742371,
+      "learning_rate": 0.0002932915921288014,
+      "loss": 0.1334,
+      "step": 35
+    },
+    {
+      "epoch": 0.06382978723404255,
+      "grad_norm": 1.297591209411621,
+      "learning_rate": 0.00029302325581395347,
+      "loss": 0.3717,
+      "step": 36
+    },
+    {
+      "epoch": 0.06560283687943262,
+      "grad_norm": 0.8587173223495483,
+      "learning_rate": 0.00029275491949910554,
+      "loss": 0.1469,
+      "step": 37
+    },
+    {
+      "epoch": 0.0673758865248227,
+      "grad_norm": 0.8923099637031555,
+      "learning_rate": 0.00029248658318425756,
+      "loss": 0.1838,
+      "step": 38
+    },
+    {
+      "epoch": 0.06914893617021277,
+      "grad_norm": 0.8410468697547913,
+      "learning_rate": 0.00029221824686940964,
+      "loss": 0.2471,
+      "step": 39
+    },
+    {
+      "epoch": 0.07092198581560284,
+      "grad_norm": 0.5034096240997314,
+      "learning_rate": 0.0002919499105545617,
+      "loss": 0.0616,
+      "step": 40
+    },
+    {
+      "epoch": 0.0726950354609929,
+      "grad_norm": 0.31462836265563965,
+      "learning_rate": 0.00029168157423971374,
+      "loss": 0.0519,
+      "step": 41
+    },
+    {
+      "epoch": 0.07446808510638298,
+      "grad_norm": 0.5840567350387573,
+      "learning_rate": 0.0002914132379248658,
+      "loss": 0.0814,
+      "step": 42
+    },
+    {
+      "epoch": 0.07624113475177305,
+      "grad_norm": 0.5441888570785522,
+      "learning_rate": 0.00029114490161001784,
+      "loss": 0.0731,
+      "step": 43
+    },
+    {
+      "epoch": 0.07801418439716312,
+      "grad_norm": 1.0477969646453857,
+      "learning_rate": 0.0002908765652951699,
+      "loss": 0.1842,
+      "step": 44
+    },
+    {
+      "epoch": 0.0797872340425532,
+      "grad_norm": 1.046530842781067,
+      "learning_rate": 0.000290608228980322,
+      "loss": 0.2727,
+      "step": 45
+    },
+    {
+      "epoch": 0.08156028368794327,
+      "grad_norm": 0.721427857875824,
+      "learning_rate": 0.00029033989266547406,
+      "loss": 0.1343,
+      "step": 46
+    },
+    {
+      "epoch": 0.08333333333333333,
+      "grad_norm": 2.0451300144195557,
+      "learning_rate": 0.0002900715563506261,
+      "loss": 0.076,
+      "step": 47
+    },
+    {
+      "epoch": 0.0851063829787234,
+      "grad_norm": 0.3739748001098633,
+      "learning_rate": 0.00028980322003577816,
+      "loss": 0.0419,
+      "step": 48
+    },
+    {
+      "epoch": 0.08687943262411348,
+      "grad_norm": 0.5523413419723511,
+      "learning_rate": 0.00028953488372093024,
+      "loss": 0.0563,
+      "step": 49
+    },
+    {
+      "epoch": 0.08865248226950355,
+      "grad_norm": 0.5457972288131714,
+      "learning_rate": 0.00028926654740608226,
+      "loss": 0.0256,
+      "step": 50
+    },
+    {
+      "epoch": 0.09042553191489362,
+      "grad_norm": 2.8798046112060547,
+      "learning_rate": 0.00028899821109123434,
+      "loss": 0.8653,
+      "step": 51
+    },
+    {
+      "epoch": 0.09219858156028368,
+      "grad_norm": 2.86541748046875,
+      "learning_rate": 0.00028872987477638636,
+      "loss": 0.5141,
+      "step": 52
+    },
+    {
+      "epoch": 0.09397163120567376,
+      "grad_norm": 10.113655090332031,
+      "learning_rate": 0.00028846153846153843,
+      "loss": 0.278,
+      "step": 53
+    },
+    {
+      "epoch": 0.09574468085106383,
+      "grad_norm": 3.4064385890960693,
+      "learning_rate": 0.00028819320214669046,
+      "loss": 0.3353,
+      "step": 54
+    },
+    {
+      "epoch": 0.0975177304964539,
+      "grad_norm": 4.866039752960205,
+      "learning_rate": 0.00028792486583184253,
+      "loss": 0.4487,
+      "step": 55
+    },
+    {
+      "epoch": 0.09929078014184398,
+      "grad_norm": 5.0878143310546875,
+      "learning_rate": 0.0002876565295169946,
+      "loss": 0.2099,
+      "step": 56
+    },
+    {
+      "epoch": 0.10106382978723404,
+      "grad_norm": 5.835223197937012,
+      "learning_rate": 0.0002873881932021467,
+      "loss": 0.4639,
+      "step": 57
+    },
+    {
+      "epoch": 0.10283687943262411,
+      "grad_norm": 3.5590853691101074,
+      "learning_rate": 0.00028711985688729876,
+      "loss": 0.3146,
+      "step": 58
+    },
+    {
+      "epoch": 0.10460992907801418,
+      "grad_norm": 5.281091213226318,
+      "learning_rate": 0.0002868515205724508,
+      "loss": 0.4222,
+      "step": 59
+    },
+    {
+      "epoch": 0.10638297872340426,
+      "grad_norm": 6.650637149810791,
+      "learning_rate": 0.00028658318425760286,
+      "loss": 0.4737,
+      "step": 60
+    },
+    {
+      "epoch": 0.10815602836879433,
+      "grad_norm": 1.2466630935668945,
+      "learning_rate": 0.0002863148479427549,
+      "loss": 0.2634,
+      "step": 61
+    },
+    {
+      "epoch": 0.1099290780141844,
+      "grad_norm": 0.73129802942276,
+      "learning_rate": 0.00028604651162790696,
+      "loss": 0.1581,
+      "step": 62
+    },
+    {
+      "epoch": 0.11170212765957446,
+      "grad_norm": 0.8522558212280273,
+      "learning_rate": 0.000285778175313059,
+      "loss": 0.2586,
+      "step": 63
+    },
+    {
+      "epoch": 0.11347517730496454,
+      "grad_norm": 0.7632489204406738,
+      "learning_rate": 0.00028550983899821105,
+      "loss": 0.2528,
+      "step": 64
+    },
+    {
+      "epoch": 0.11524822695035461,
+      "grad_norm": 0.4096511900424957,
+      "learning_rate": 0.00028524150268336313,
+      "loss": 0.0851,
+      "step": 65
+    },
+    {
+      "epoch": 0.11702127659574468,
+      "grad_norm": 0.9045661687850952,
+      "learning_rate": 0.0002849731663685152,
+      "loss": 0.2253,
+      "step": 66
+    },
+    {
+      "epoch": 0.11879432624113476,
+      "grad_norm": 0.6839063763618469,
+      "learning_rate": 0.00028470483005366723,
+      "loss": 0.1416,
+      "step": 67
+    },
+    {
+      "epoch": 0.12056737588652482,
+      "grad_norm": 2.112931728363037,
+      "learning_rate": 0.0002844364937388193,
+      "loss": 0.2554,
+      "step": 68
+    },
+    {
+      "epoch": 0.12234042553191489,
+      "grad_norm": 1.187727928161621,
+      "learning_rate": 0.0002841681574239714,
+      "loss": 0.2891,
+      "step": 69
+    },
+    {
+      "epoch": 0.12411347517730496,
+      "grad_norm": 0.7449979782104492,
+      "learning_rate": 0.0002838998211091234,
+      "loss": 0.1743,
+      "step": 70
+    },
+    {
+      "epoch": 0.12588652482269502,
+      "grad_norm": 0.5655006766319275,
+      "learning_rate": 0.0002836314847942755,
+      "loss": 0.1247,
+      "step": 71
+    },
+    {
+      "epoch": 0.1276595744680851,
+      "grad_norm": 0.7953659296035767,
+      "learning_rate": 0.0002833631484794275,
+      "loss": 0.1277,
+      "step": 72
+    },
+    {
+      "epoch": 0.12943262411347517,
+      "grad_norm": 0.7307471632957458,
+      "learning_rate": 0.0002830948121645796,
+      "loss": 0.2575,
+      "step": 73
+    },
+    {
+      "epoch": 0.13120567375886524,
+      "grad_norm": 0.815560519695282,
+      "learning_rate": 0.0002828264758497316,
+      "loss": 0.2547,
+      "step": 74
+    },
+    {
+      "epoch": 0.13297872340425532,
+      "grad_norm": 0.7381014227867126,
+      "learning_rate": 0.0002825581395348837,
+      "loss": 0.1005,
+      "step": 75
+    },
+    {
+      "epoch": 0.1347517730496454,
+      "grad_norm": 1.0876243114471436,
+      "learning_rate": 0.00028228980322003575,
+      "loss": 0.2536,
+      "step": 76
+    },
+    {
+      "epoch": 0.13652482269503546,
+      "grad_norm": 0.6369747519493103,
+      "learning_rate": 0.00028202146690518783,
+      "loss": 0.1696,
+      "step": 77
+    },
+    {
+      "epoch": 0.13829787234042554,
+      "grad_norm": 0.5377818942070007,
+      "learning_rate": 0.0002817531305903399,
+      "loss": 0.1528,
+      "step": 78
+    },
+    {
+      "epoch": 0.1400709219858156,
+      "grad_norm": 0.5050985813140869,
+      "learning_rate": 0.0002814847942754919,
+      "loss": 0.1376,
+      "step": 79
+    },
+    {
+      "epoch": 0.14184397163120568,
+      "grad_norm": 0.46020108461380005,
+      "learning_rate": 0.000281216457960644,
+      "loss": 0.1164,
+      "step": 80
+    },
+    {
+      "epoch": 0.14361702127659576,
+      "grad_norm": 0.4778999090194702,
+      "learning_rate": 0.000280948121645796,
+      "loss": 0.1078,
+      "step": 81
+    },
+    {
+      "epoch": 0.1453900709219858,
+      "grad_norm": 0.7942756414413452,
+      "learning_rate": 0.0002806797853309481,
+      "loss": 0.2139,
+      "step": 82
+    },
+    {
+      "epoch": 0.14716312056737588,
+      "grad_norm": 0.5343471169471741,
+      "learning_rate": 0.0002804114490161001,
+      "loss": 0.1331,
+      "step": 83
+    },
+    {
+      "epoch": 0.14893617021276595,
+      "grad_norm": 0.8038661479949951,
+      "learning_rate": 0.0002801431127012522,
+      "loss": 0.1801,
+      "step": 84
+    },
+    {
+      "epoch": 0.15070921985815602,
+      "grad_norm": 0.3083706498146057,
+      "learning_rate": 0.0002798747763864043,
+      "loss": 0.0437,
+      "step": 85
+    },
+    {
+      "epoch": 0.1524822695035461,
+      "grad_norm": 0.6344548463821411,
+      "learning_rate": 0.00027960644007155635,
+      "loss": 0.1676,
+      "step": 86
+    },
+    {
+      "epoch": 0.15425531914893617,
+      "grad_norm": 0.5747407674789429,
+      "learning_rate": 0.00027933810375670837,
+      "loss": 0.1079,
+      "step": 87
+    },
+    {
+      "epoch": 0.15602836879432624,
+      "grad_norm": 0.5270193815231323,
+      "learning_rate": 0.00027906976744186045,
+      "loss": 0.1263,
+      "step": 88
+    },
+    {
+      "epoch": 0.15780141843971632,
+      "grad_norm": 0.4451313614845276,
+      "learning_rate": 0.0002788014311270125,
+      "loss": 0.0511,
+      "step": 89
+    },
+    {
+      "epoch": 0.1595744680851064,
+      "grad_norm": 0.691502034664154,
+      "learning_rate": 0.00027853309481216455,
+      "loss": 0.1637,
+      "step": 90
+    },
+    {
+      "epoch": 0.16134751773049646,
+      "grad_norm": 0.5651169419288635,
+      "learning_rate": 0.0002782647584973166,
+      "loss": 0.1371,
+      "step": 91
+    },
+    {
+      "epoch": 0.16312056737588654,
+      "grad_norm": 0.5306984782218933,
+      "learning_rate": 0.00027799642218246864,
+      "loss": 0.1535,
+      "step": 92
+    },
+    {
+      "epoch": 0.16489361702127658,
+      "grad_norm": 0.5752344727516174,
+      "learning_rate": 0.0002777280858676207,
+      "loss": 0.1622,
+      "step": 93
+    },
+    {
+      "epoch": 0.16666666666666666,
+      "grad_norm": 0.3410884737968445,
+      "learning_rate": 0.0002774597495527728,
+      "loss": 0.0683,
+      "step": 94
+    },
+    {
+      "epoch": 0.16843971631205673,
+      "grad_norm": 0.6347271203994751,
+      "learning_rate": 0.0002771914132379248,
+      "loss": 0.1577,
+      "step": 95
+    },
+    {
+      "epoch": 0.1702127659574468,
+      "grad_norm": 0.36072492599487305,
+      "learning_rate": 0.0002769230769230769,
+      "loss": 0.0367,
+      "step": 96
+    },
+    {
+      "epoch": 0.17198581560283688,
+      "grad_norm": 0.29759401082992554,
+      "learning_rate": 0.00027665474060822897,
+      "loss": 0.0218,
+      "step": 97
+    },
+    {
+      "epoch": 0.17375886524822695,
+      "grad_norm": 0.37029051780700684,
+      "learning_rate": 0.00027638640429338105,
+      "loss": 0.0263,
+      "step": 98
+    },
+    {
+      "epoch": 0.17553191489361702,
+      "grad_norm": 0.3705526292324066,
+      "learning_rate": 0.00027611806797853307,
+      "loss": 0.0219,
+      "step": 99
+    },
+    {
+      "epoch": 0.1773049645390071,
+      "grad_norm": 0.31463027000427246,
+      "learning_rate": 0.00027584973166368514,
+      "loss": 0.023,
+      "step": 100
+    },
+    {
+      "epoch": 0.17907801418439717,
+      "grad_norm": 1.7321122884750366,
+      "learning_rate": 0.00027558139534883717,
+      "loss": 0.6776,
+      "step": 101
+    },
+    {
+      "epoch": 0.18085106382978725,
+      "grad_norm": 1.03788423538208,
+      "learning_rate": 0.00027531305903398924,
+      "loss": 0.4406,
+      "step": 102
+    },
+    {
+      "epoch": 0.18262411347517732,
+      "grad_norm": 1.13809335231781,
+      "learning_rate": 0.00027504472271914126,
+      "loss": 0.4302,
+      "step": 103
+    },
+    {
+      "epoch": 0.18439716312056736,
+      "grad_norm": 0.6666789650917053,
+      "learning_rate": 0.00027477638640429334,
+      "loss": 0.2029,
+      "step": 104
+    },
+    {
+      "epoch": 0.18617021276595744,
+      "grad_norm": 0.9535943865776062,
+      "learning_rate": 0.0002745080500894454,
+      "loss": 0.2808,
+      "step": 105
+    },
+    {
+      "epoch": 0.1879432624113475,
+      "grad_norm": 1.2909356355667114,
+      "learning_rate": 0.0002742397137745975,
+      "loss": 0.3593,
+      "step": 106
+    },
+    {
+      "epoch": 0.18971631205673758,
+      "grad_norm": 0.9249064326286316,
+      "learning_rate": 0.0002739713774597495,
+      "loss": 0.355,
+      "step": 107
+    },
+    {
+      "epoch": 0.19148936170212766,
+      "grad_norm": 0.5897863507270813,
+      "learning_rate": 0.0002737030411449016,
+      "loss": 0.2349,
+      "step": 108
+    },
+    {
+      "epoch": 0.19326241134751773,
+      "grad_norm": 1.060029149055481,
+      "learning_rate": 0.00027343470483005367,
+      "loss": 0.2517,
+      "step": 109
+    },
+    {
+      "epoch": 0.1950354609929078,
+      "grad_norm": 0.6262145638465881,
+      "learning_rate": 0.0002731663685152057,
+      "loss": 0.2119,
+      "step": 110
+    },
+    {
+      "epoch": 0.19680851063829788,
+      "grad_norm": 0.7881852984428406,
+      "learning_rate": 0.00027289803220035777,
+      "loss": 0.2792,
+      "step": 111
+    },
+    {
+      "epoch": 0.19858156028368795,
+      "grad_norm": 0.7693052887916565,
+      "learning_rate": 0.0002726296958855098,
+      "loss": 0.2824,
+      "step": 112
+    },
+    {
+      "epoch": 0.20035460992907803,
+      "grad_norm": 0.7235965132713318,
+      "learning_rate": 0.00027236135957066186,
+      "loss": 0.0918,
+      "step": 113
+    },
+    {
+      "epoch": 0.20212765957446807,
+      "grad_norm": 1.0118364095687866,
+      "learning_rate": 0.00027209302325581394,
+      "loss": 0.211,
+      "step": 114
+    },
+    {
+      "epoch": 0.20390070921985815,
+      "grad_norm": 0.9211074709892273,
+      "learning_rate": 0.00027182468694096596,
+      "loss": 0.2766,
+      "step": 115
+    },
+    {
+      "epoch": 0.20567375886524822,
+      "grad_norm": 1.3270055055618286,
+      "learning_rate": 0.00027155635062611804,
+      "loss": 0.1172,
+      "step": 116
+    },
+    {
+      "epoch": 0.2074468085106383,
+      "grad_norm": 0.6904440522193909,
+      "learning_rate": 0.0002712880143112701,
+      "loss": 0.2208,
+      "step": 117
+    },
+    {
+      "epoch": 0.20921985815602837,
+      "grad_norm": 0.44743916392326355,
+      "learning_rate": 0.0002710196779964222,
+      "loss": 0.1265,
+      "step": 118
+    },
+    {
+      "epoch": 0.21099290780141844,
+      "grad_norm": 1.5493639707565308,
+      "learning_rate": 0.0002707513416815742,
+      "loss": 0.2085,
+      "step": 119
+    },
+    {
+      "epoch": 0.2127659574468085,
+      "grad_norm": 0.7296533584594727,
+      "learning_rate": 0.0002704830053667263,
+      "loss": 0.253,
+      "step": 120
+    },
+    {
+      "epoch": 0.21453900709219859,
+      "grad_norm": 1.300660490989685,
+      "learning_rate": 0.0002702146690518783,
+      "loss": 0.2164,
+      "step": 121
+    },
+    {
+      "epoch": 0.21631205673758866,
+      "grad_norm": 0.9226039052009583,
+      "learning_rate": 0.0002699463327370304,
+      "loss": 0.2533,
+      "step": 122
+    },
+    {
+      "epoch": 0.21808510638297873,
+      "grad_norm": 0.9466855525970459,
+      "learning_rate": 0.0002696779964221824,
+      "loss": 0.2528,
+      "step": 123
+    },
+    {
+      "epoch": 0.2198581560283688,
+      "grad_norm": 0.6484300494194031,
+      "learning_rate": 0.0002694096601073345,
+      "loss": 0.1598,
+      "step": 124
+    },
+    {
+      "epoch": 0.22163120567375885,
+      "grad_norm": 0.7165495157241821,
+      "learning_rate": 0.00026914132379248656,
+      "loss": 0.2448,
+      "step": 125
+    },
+    {
+      "epoch": 0.22340425531914893,
+      "grad_norm": 1.370329737663269,
+      "learning_rate": 0.00026887298747763864,
+      "loss": 0.3169,
+      "step": 126
+    },
+    {
+      "epoch": 0.225177304964539,
+      "grad_norm": 0.41645482182502747,
+      "learning_rate": 0.00026860465116279066,
+      "loss": 0.1352,
+      "step": 127
+    },
+    {
+      "epoch": 0.22695035460992907,
+      "grad_norm": 0.4441884458065033,
+      "learning_rate": 0.00026833631484794273,
+      "loss": 0.1376,
+      "step": 128
+    },
+    {
+      "epoch": 0.22872340425531915,
+      "grad_norm": 0.3925491273403168,
+      "learning_rate": 0.0002680679785330948,
+      "loss": 0.1194,
+      "step": 129
+    },
+    {
+      "epoch": 0.23049645390070922,
+      "grad_norm": 0.3546803295612335,
+      "learning_rate": 0.00026779964221824683,
+      "loss": 0.0936,
+      "step": 130
+    },
+    {
+      "epoch": 0.2322695035460993,
+      "grad_norm": 0.683130145072937,
+      "learning_rate": 0.0002675313059033989,
+      "loss": 0.2044,
+      "step": 131
+    },
+    {
+      "epoch": 0.23404255319148937,
+      "grad_norm": 0.666155219078064,
+      "learning_rate": 0.00026726296958855093,
+      "loss": 0.2361,
+      "step": 132
+    },
+    {
+      "epoch": 0.23581560283687944,
+      "grad_norm": 0.5066004991531372,
+      "learning_rate": 0.000266994633273703,
+      "loss": 0.1449,
+      "step": 133
+    },
+    {
+      "epoch": 0.2375886524822695,
+      "grad_norm": 0.745506763458252,
+      "learning_rate": 0.0002667262969588551,
+      "loss": 0.1987,
+      "step": 134
+    },
+    {
+      "epoch": 0.2393617021276596,
+      "grad_norm": 0.9249691963195801,
+      "learning_rate": 0.0002664579606440071,
+      "loss": 0.2485,
+      "step": 135
+    },
+    {
+      "epoch": 0.24113475177304963,
+      "grad_norm": 0.5891656875610352,
+      "learning_rate": 0.0002661896243291592,
+      "loss": 0.2172,
+      "step": 136
+    },
+    {
+      "epoch": 0.2429078014184397,
+      "grad_norm": 0.6771588325500488,
+      "learning_rate": 0.00026592128801431126,
+      "loss": 0.2293,
+      "step": 137
+    },
+    {
+      "epoch": 0.24468085106382978,
+      "grad_norm": 1.0185132026672363,
+      "learning_rate": 0.00026565295169946333,
+      "loss": 0.2819,
+      "step": 138
+    },
+    {
+      "epoch": 0.24645390070921985,
+      "grad_norm": 0.357791930437088,
+      "learning_rate": 0.00026538461538461536,
+      "loss": 0.0876,
+      "step": 139
+    },
+    {
+      "epoch": 0.24822695035460993,
+      "grad_norm": 0.5244898200035095,
+      "learning_rate": 0.00026511627906976743,
+      "loss": 0.0989,
+      "step": 140
+    },
+    {
+      "epoch": 0.25,
+      "grad_norm": 0.553326427936554,
+      "learning_rate": 0.00026484794275491945,
+      "loss": 0.1503,
+      "step": 141
+    },
+    {
+      "epoch": 0.25177304964539005,
+      "grad_norm": 0.7104494571685791,
+      "learning_rate": 0.00026457960644007153,
+      "loss": 0.2487,
+      "step": 142
+    },
+    {
+      "epoch": 0.25354609929078015,
+      "grad_norm": 0.5205265879631042,
+      "learning_rate": 0.0002643112701252236,
+      "loss": 0.1185,
+      "step": 143
+    },
+    {
+      "epoch": 0.2553191489361702,
+      "grad_norm": 0.41840335726737976,
+      "learning_rate": 0.00026404293381037563,
+      "loss": 0.0841,
+      "step": 144
+    },
+    {
+      "epoch": 0.2570921985815603,
+      "grad_norm": 0.572222888469696,
+      "learning_rate": 0.0002637745974955277,
+      "loss": 0.1331,
+      "step": 145
+    },
+    {
+      "epoch": 0.25886524822695034,
+      "grad_norm": 0.8182592391967773,
+      "learning_rate": 0.0002635062611806798,
+      "loss": 0.2042,
+      "step": 146
+    },
+    {
+      "epoch": 0.26063829787234044,
+      "grad_norm": 0.5115887522697449,
+      "learning_rate": 0.0002632379248658318,
+      "loss": 0.1148,
+      "step": 147
+    },
+    {
+      "epoch": 0.2624113475177305,
+      "grad_norm": 0.5078725218772888,
+      "learning_rate": 0.0002629695885509839,
+      "loss": 0.0762,
+      "step": 148
+    },
+    {
+      "epoch": 0.2641843971631206,
+      "grad_norm": 0.22413299977779388,
+      "learning_rate": 0.00026270125223613595,
+      "loss": 0.0355,
+      "step": 149
+    },
+    {
+      "epoch": 0.26595744680851063,
+      "grad_norm": 0.25390034914016724,
+      "learning_rate": 0.000262432915921288,
+      "loss": 0.0319,
+      "step": 150
+    },
+    {
+      "epoch": 0.26773049645390073,
+      "grad_norm": 3.294741153717041,
+      "learning_rate": 0.00026216457960644005,
+      "loss": 0.4876,
+      "step": 151
+    },
+    {
+      "epoch": 0.2695035460992908,
+      "grad_norm": 1.5393906831741333,
+      "learning_rate": 0.0002618962432915921,
+      "loss": 0.2547,
+      "step": 152
+    },
+    {
+      "epoch": 0.2712765957446808,
+      "grad_norm": 1.7025604248046875,
+      "learning_rate": 0.00026162790697674415,
+      "loss": 0.4747,
+      "step": 153
+    },
+    {
+      "epoch": 0.2730496453900709,
+      "grad_norm": 0.8344413638114929,
+      "learning_rate": 0.0002613595706618962,
+      "loss": 0.2431,
+      "step": 154
+    },
+    {
+      "epoch": 0.274822695035461,
+      "grad_norm": 0.9473798871040344,
+      "learning_rate": 0.00026109123434704825,
+      "loss": 0.2135,
+      "step": 155
+    },
+    {
+      "epoch": 0.2765957446808511,
+      "grad_norm": 1.0257476568222046,
+      "learning_rate": 0.0002608228980322003,
+      "loss": 0.1833,
+      "step": 156
+    },
+    {
+      "epoch": 0.2783687943262411,
+      "grad_norm": 1.2221112251281738,
+      "learning_rate": 0.0002605545617173524,
+      "loss": 0.3174,
+      "step": 157
+    },
+    {
+      "epoch": 0.2801418439716312,
+      "grad_norm": 0.5619825124740601,
+      "learning_rate": 0.0002602862254025045,
+      "loss": 0.1492,
+      "step": 158
+    },
+    {
+      "epoch": 0.28191489361702127,
+      "grad_norm": 1.1845368146896362,
+      "learning_rate": 0.0002600178890876565,
+      "loss": 0.4336,
+      "step": 159
+    },
+    {
+      "epoch": 0.28368794326241137,
+      "grad_norm": 1.2730036973953247,
+      "learning_rate": 0.0002597495527728086,
+      "loss": 0.3466,
+      "step": 160
+    },
+    {
+      "epoch": 0.2854609929078014,
+      "grad_norm": 0.4464946985244751,
+      "learning_rate": 0.0002594812164579606,
+      "loss": 0.1239,
+      "step": 161
+    },
+    {
+      "epoch": 0.2872340425531915,
+      "grad_norm": 0.5267930030822754,
+      "learning_rate": 0.00025921288014311267,
+      "loss": 0.1372,
+      "step": 162
+    },
+    {
+      "epoch": 0.28900709219858156,
+      "grad_norm": 0.4249666929244995,
+      "learning_rate": 0.00025894454382826475,
+      "loss": 0.1402,
+      "step": 163
+    },
+    {
+      "epoch": 0.2907801418439716,
+      "grad_norm": 0.7619920372962952,
+      "learning_rate": 0.00025867620751341677,
+      "loss": 0.2309,
+      "step": 164
+    },
+    {
+      "epoch": 0.2925531914893617,
+      "grad_norm": 0.9128990769386292,
+      "learning_rate": 0.00025840787119856885,
+      "loss": 0.2832,
+      "step": 165
+    },
+    {
+      "epoch": 0.29432624113475175,
+      "grad_norm": 0.4988587200641632,
+      "learning_rate": 0.0002581395348837209,
+      "loss": 0.1517,
+      "step": 166
+    },
+    {
+      "epoch": 0.29609929078014185,
+      "grad_norm": 0.7568597793579102,
+      "learning_rate": 0.000257871198568873,
+      "loss": 0.2449,
+      "step": 167
+    },
+    {
+      "epoch": 0.2978723404255319,
+      "grad_norm": 0.7556333541870117,
+      "learning_rate": 0.000257602862254025,
+      "loss": 0.2591,
+      "step": 168
+    },
+    {
+      "epoch": 0.299645390070922,
+      "grad_norm": 0.9023774266242981,
+      "learning_rate": 0.0002573345259391771,
+      "loss": 0.239,
+      "step": 169
+    },
+    {
+      "epoch": 0.30141843971631205,
+      "grad_norm": 0.490642786026001,
+      "learning_rate": 0.0002570661896243291,
+      "loss": 0.1508,
+      "step": 170
+    },
+    {
+      "epoch": 0.30319148936170215,
+      "grad_norm": 0.6215077042579651,
+      "learning_rate": 0.0002567978533094812,
+      "loss": 0.2277,
+      "step": 171
+    },
+    {
+      "epoch": 0.3049645390070922,
+      "grad_norm": 0.4409044086933136,
+      "learning_rate": 0.00025652951699463327,
+      "loss": 0.1354,
+      "step": 172
+    },
+    {
+      "epoch": 0.3067375886524823,
+      "grad_norm": 0.420749694108963,
+      "learning_rate": 0.0002562611806797853,
+      "loss": 0.068,
+      "step": 173
+    },
+    {
+      "epoch": 0.30851063829787234,
+      "grad_norm": 0.40215224027633667,
+      "learning_rate": 0.00025599284436493737,
+      "loss": 0.0707,
+      "step": 174
+    },
+    {
+      "epoch": 0.3102836879432624,
+      "grad_norm": 0.7107430696487427,
+      "learning_rate": 0.0002557245080500894,
+      "loss": 0.1413,
+      "step": 175
+    },
+    {
+      "epoch": 0.3120567375886525,
+      "grad_norm": 0.9405664801597595,
+      "learning_rate": 0.00025545617173524147,
+      "loss": 0.1555,
+      "step": 176
+    },
+    {
+      "epoch": 0.31382978723404253,
+      "grad_norm": 0.9159305095672607,
+      "learning_rate": 0.00025518783542039354,
+      "loss": 0.1827,
+      "step": 177
+    },
+    {
+      "epoch": 0.31560283687943264,
+      "grad_norm": 1.2095725536346436,
+      "learning_rate": 0.0002549194991055456,
+      "loss": 0.2713,
+      "step": 178
+    },
+    {
+      "epoch": 0.3173758865248227,
+      "grad_norm": 0.5971267223358154,
+      "learning_rate": 0.00025465116279069764,
+      "loss": 0.2094,
+      "step": 179
+    },
+    {
+      "epoch": 0.3191489361702128,
+      "grad_norm": 0.5078379511833191,
+      "learning_rate": 0.0002543828264758497,
+      "loss": 0.1102,
+      "step": 180
+    },
+    {
+      "epoch": 0.32092198581560283,
+      "grad_norm": 0.5540158748626709,
+      "learning_rate": 0.0002541144901610018,
+      "loss": 0.1408,
+      "step": 181
+    },
+    {
+      "epoch": 0.32269503546099293,
+      "grad_norm": 0.3938644826412201,
+      "learning_rate": 0.0002538461538461538,
+      "loss": 0.1044,
+      "step": 182
+    },
+    {
+      "epoch": 0.324468085106383,
+      "grad_norm": 0.43856045603752136,
+      "learning_rate": 0.0002535778175313059,
+      "loss": 0.1069,
+      "step": 183
+    },
+    {
+      "epoch": 0.3262411347517731,
+      "grad_norm": 0.6097897887229919,
+      "learning_rate": 0.0002533094812164579,
+      "loss": 0.1447,
+      "step": 184
+    },
+    {
+      "epoch": 0.3280141843971631,
+      "grad_norm": 0.6639567613601685,
+      "learning_rate": 0.00025304114490161,
+      "loss": 0.1622,
+      "step": 185
+    },
+    {
+      "epoch": 0.32978723404255317,
+      "grad_norm": 0.4464520812034607,
+      "learning_rate": 0.00025277280858676207,
+      "loss": 0.1113,
+      "step": 186
+    },
+    {
+      "epoch": 0.33156028368794327,
+      "grad_norm": 0.45327168703079224,
+      "learning_rate": 0.00025250447227191414,
+      "loss": 0.1121,
+      "step": 187
+    },
+    {
+      "epoch": 0.3333333333333333,
+      "grad_norm": 0.4880326986312866,
+      "learning_rate": 0.00025223613595706616,
+      "loss": 0.1055,
+      "step": 188
+    },
+    {
+      "epoch": 0.3351063829787234,
+      "grad_norm": 0.44586634635925293,
+      "learning_rate": 0.00025196779964221824,
+      "loss": 0.1544,
+      "step": 189
+    },
+    {
+      "epoch": 0.33687943262411346,
+      "grad_norm": 0.7135107517242432,
+      "learning_rate": 0.00025169946332737026,
+      "loss": 0.1047,
+      "step": 190
+    },
+    {
+      "epoch": 0.33865248226950356,
+      "grad_norm": 0.5775635838508606,
+      "learning_rate": 0.00025143112701252234,
+      "loss": 0.175,
+      "step": 191
+    },
+    {
+      "epoch": 0.3404255319148936,
+      "grad_norm": 0.6252586841583252,
+      "learning_rate": 0.0002511627906976744,
+      "loss": 0.1478,
+      "step": 192
+    },
+    {
+      "epoch": 0.3421985815602837,
+      "grad_norm": 2.061342477798462,
+      "learning_rate": 0.00025089445438282644,
+      "loss": 0.1931,
+      "step": 193
+    },
+    {
+      "epoch": 0.34397163120567376,
+      "grad_norm": 0.47072601318359375,
+      "learning_rate": 0.0002506261180679785,
+      "loss": 0.0928,
+      "step": 194
+    },
+    {
+      "epoch": 0.34574468085106386,
+      "grad_norm": 0.5164952278137207,
+      "learning_rate": 0.00025035778175313053,
+      "loss": 0.1287,
+      "step": 195
+    },
+    {
+      "epoch": 0.3475177304964539,
+      "grad_norm": 0.44344618916511536,
+      "learning_rate": 0.0002500894454382826,
+      "loss": 0.0924,
+      "step": 196
+    },
+    {
+      "epoch": 0.34929078014184395,
+      "grad_norm": 0.4380563795566559,
+      "learning_rate": 0.0002498211091234347,
+      "loss": 0.1393,
+      "step": 197
+    },
+    {
+      "epoch": 0.35106382978723405,
+      "grad_norm": 0.3074440360069275,
+      "learning_rate": 0.00024955277280858676,
+      "loss": 0.04,
+      "step": 198
+    },
+    {
+      "epoch": 0.3528368794326241,
+      "grad_norm": 0.25228482484817505,
+      "learning_rate": 0.0002492844364937388,
+      "loss": 0.037,
+      "step": 199
+    },
+    {
+      "epoch": 0.3546099290780142,
+      "grad_norm": 0.4954518973827362,
+      "learning_rate": 0.00024901610017889086,
+      "loss": 0.1563,
+      "step": 200
+    },
+    {
+      "epoch": 0.35638297872340424,
+      "grad_norm": 9.530560493469238,
+      "learning_rate": 0.00024874776386404294,
+      "loss": 0.6406,
+      "step": 201
+    },
+    {
+      "epoch": 0.35815602836879434,
+      "grad_norm": 1.2426095008850098,
+      "learning_rate": 0.00024847942754919496,
+      "loss": 0.5336,
+      "step": 202
+    },
+    {
+      "epoch": 0.3599290780141844,
+      "grad_norm": 1.4315325021743774,
+      "learning_rate": 0.00024821109123434703,
+      "loss": 0.4691,
+      "step": 203
+    },
+    {
+      "epoch": 0.3617021276595745,
+      "grad_norm": 1.3048845529556274,
+      "learning_rate": 0.00024794275491949906,
+      "loss": 0.4098,
+      "step": 204
+    },
+    {
+      "epoch": 0.36347517730496454,
+      "grad_norm": 1.1803832054138184,
+      "learning_rate": 0.00024767441860465113,
+      "loss": 0.3023,
+      "step": 205
+    },
+    {
+      "epoch": 0.36524822695035464,
+      "grad_norm": 0.895564079284668,
+      "learning_rate": 0.0002474060822898032,
+      "loss": 0.2474,
+      "step": 206
+    },
+    {
+      "epoch": 0.3670212765957447,
+      "grad_norm": 0.7290531992912292,
+      "learning_rate": 0.0002471377459749553,
+      "loss": 0.2365,
+      "step": 207
+    },
+    {
+      "epoch": 0.36879432624113473,
+      "grad_norm": 0.5765687823295593,
+      "learning_rate": 0.0002468694096601073,
+      "loss": 0.2233,
+      "step": 208
+    },
+    {
+      "epoch": 0.37056737588652483,
+      "grad_norm": 1.518298625946045,
+      "learning_rate": 0.0002466010733452594,
+      "loss": 0.3787,
+      "step": 209
+    },
+    {
+      "epoch": 0.3723404255319149,
+      "grad_norm": 0.9093344211578369,
+      "learning_rate": 0.00024633273703041146,
+      "loss": 0.2682,
+      "step": 210
+    },
+    {
+      "epoch": 0.374113475177305,
+      "grad_norm": 0.9133920669555664,
+      "learning_rate": 0.0002460644007155635,
+      "loss": 0.2236,
+      "step": 211
+    },
+    {
+      "epoch": 0.375886524822695,
+      "grad_norm": 0.9125071167945862,
+      "learning_rate": 0.00024579606440071556,
+      "loss": 0.344,
+      "step": 212
+    },
+    {
+      "epoch": 0.3776595744680851,
+      "grad_norm": 0.9353806972503662,
+      "learning_rate": 0.0002455277280858676,
+      "loss": 0.3213,
+      "step": 213
+    },
+    {
+      "epoch": 0.37943262411347517,
+      "grad_norm": 0.32626235485076904,
+      "learning_rate": 0.00024525939177101966,
+      "loss": 0.089,
+      "step": 214
+    },
+    {
+      "epoch": 0.38120567375886527,
+      "grad_norm": 0.722865104675293,
+      "learning_rate": 0.0002449910554561717,
+      "loss": 0.2475,
+      "step": 215
+    },
+    {
+      "epoch": 0.3829787234042553,
+      "grad_norm": 1.2499586343765259,
+      "learning_rate": 0.00024472271914132375,
+      "loss": 0.1726,
+      "step": 216
+    },
+    {
+      "epoch": 0.38475177304964536,
+      "grad_norm": 0.8918619155883789,
+      "learning_rate": 0.00024445438282647583,
+      "loss": 0.1832,
+      "step": 217
+    },
+    {
+      "epoch": 0.38652482269503546,
+      "grad_norm": 0.9703326225280762,
+      "learning_rate": 0.0002441860465116279,
+      "loss": 0.3934,
+      "step": 218
+    },
+    {
+      "epoch": 0.3882978723404255,
+      "grad_norm": 0.9967647194862366,
+      "learning_rate": 0.00024391771019677995,
+      "loss": 0.2148,
+      "step": 219
+    },
+    {
+      "epoch": 0.3900709219858156,
+      "grad_norm": 0.6736385226249695,
+      "learning_rate": 0.000243649373881932,
+      "loss": 0.2019,
+      "step": 220
+    },
+    {
+      "epoch": 0.39184397163120566,
+      "grad_norm": 0.6814044117927551,
+      "learning_rate": 0.00024338103756708408,
+      "loss": 0.3198,
+      "step": 221
+    },
+    {
+      "epoch": 0.39361702127659576,
+      "grad_norm": 0.6858790516853333,
+      "learning_rate": 0.0002431127012522361,
+      "loss": 0.2017,
+      "step": 222
+    },
+    {
+      "epoch": 0.3953900709219858,
+      "grad_norm": 0.4301723837852478,
+      "learning_rate": 0.00024284436493738818,
+      "loss": 0.1283,
+      "step": 223
+    },
+    {
+      "epoch": 0.3971631205673759,
+      "grad_norm": 0.47749027609825134,
+      "learning_rate": 0.00024257602862254023,
+      "loss": 0.1513,
+      "step": 224
+    },
+    {
+      "epoch": 0.39893617021276595,
+      "grad_norm": 0.5345844626426697,
+      "learning_rate": 0.0002423076923076923,
+      "loss": 0.0872,
+      "step": 225
+    },
+    {
+      "epoch": 0.40070921985815605,
+      "grad_norm": 0.5137667059898376,
+      "learning_rate": 0.00024203935599284433,
+      "loss": 0.0726,
+      "step": 226
+    },
+    {
+      "epoch": 0.40070921985815605,
+      "eval_loss": 0.08563709259033203,
+      "eval_runtime": 47.4226,
+      "eval_samples_per_second": 2.657,
+      "eval_steps_per_second": 2.657,
+      "step": 226
+    },
+    {
+      "epoch": 0.4024822695035461,
+      "grad_norm": 0.5459873080253601,
+      "learning_rate": 0.0002417710196779964,
+      "loss": 0.1341,
+      "step": 227
+    },
+    {
+      "epoch": 0.40425531914893614,
+      "grad_norm": 0.7590123414993286,
+      "learning_rate": 0.00024150268336314848,
+      "loss": 0.2418,
+      "step": 228
+    },
+    {
+      "epoch": 0.40602836879432624,
+      "grad_norm": 0.8846575617790222,
+      "learning_rate": 0.00024123434704830053,
+      "loss": 0.1606,
+      "step": 229
+    },
+    {
+      "epoch": 0.4078014184397163,
+      "grad_norm": 0.7346479296684265,
+      "learning_rate": 0.00024096601073345258,
+      "loss": 0.2458,
+      "step": 230
+    },
+    {
+      "epoch": 0.4095744680851064,
+      "grad_norm": 0.7012773752212524,
+      "learning_rate": 0.00024069767441860462,
+      "loss": 0.1353,
+      "step": 231
+    },
+    {
+      "epoch": 0.41134751773049644,
+      "grad_norm": 0.5895172357559204,
+      "learning_rate": 0.0002404293381037567,
+      "loss": 0.2138,
+      "step": 232
+    },
+    {
+      "epoch": 0.41312056737588654,
+      "grad_norm": 0.6732155680656433,
+      "learning_rate": 0.00024016100178890872,
+      "loss": 0.0906,
+      "step": 233
+    },
+    {
+      "epoch": 0.4148936170212766,
+      "grad_norm": 0.38057422637939453,
+      "learning_rate": 0.0002398926654740608,
+      "loss": 0.0973,
+      "step": 234
+    },
+    {
+      "epoch": 0.4166666666666667,
+      "grad_norm": 0.4375483989715576,
+      "learning_rate": 0.00023962432915921285,
+      "loss": 0.0895,
+      "step": 235
+    },
+    {
+      "epoch": 0.41843971631205673,
+      "grad_norm": 0.45380473136901855,
+      "learning_rate": 0.00023935599284436492,
+      "loss": 0.1046,
+      "step": 236
+    },
+    {
+      "epoch": 0.42021276595744683,
+      "grad_norm": 0.38564634323120117,
+      "learning_rate": 0.00023908765652951695,
+      "loss": 0.0899,
+      "step": 237
+    },
+    {
+      "epoch": 0.4219858156028369,
+      "grad_norm": 0.837587296962738,
+      "learning_rate": 0.00023881932021466902,
+      "loss": 0.2611,
+      "step": 238
+    },
+    {
+      "epoch": 0.4237588652482269,
+      "grad_norm": 0.9749343991279602,
+      "learning_rate": 0.0002385509838998211,
+      "loss": 0.1351,
+      "step": 239
+    },
+    {
+      "epoch": 0.425531914893617,
+      "grad_norm": 0.7198343873023987,
+      "learning_rate": 0.00023828264758497315,
+      "loss": 0.1716,
+      "step": 240
+    },
+    {
+      "epoch": 0.42730496453900707,
+      "grad_norm": 0.6409445405006409,
+      "learning_rate": 0.00023801431127012522,
+      "loss": 0.1539,
+      "step": 241
+    },
+    {
+      "epoch": 0.42907801418439717,
+      "grad_norm": 0.6468048095703125,
+      "learning_rate": 0.00023774597495527724,
+      "loss": 0.2017,
+      "step": 242
+    },
+    {
+      "epoch": 0.4308510638297872,
+      "grad_norm": 0.3550443947315216,
+      "learning_rate": 0.00023747763864042932,
+      "loss": 0.102,
+      "step": 243
+    },
+    {
+      "epoch": 0.4326241134751773,
+      "grad_norm": 0.5282692313194275,
+      "learning_rate": 0.00023720930232558137,
+      "loss": 0.0795,
+      "step": 244
+    },
+    {
+      "epoch": 0.43439716312056736,
+      "grad_norm": 0.4462401568889618,
+      "learning_rate": 0.00023694096601073345,
+      "loss": 0.1051,
+      "step": 245
+    },
+    {
+      "epoch": 0.43617021276595747,
+      "grad_norm": 0.33794382214546204,
+      "learning_rate": 0.00023667262969588547,
+      "loss": 0.0688,
+      "step": 246
+    },
+    {
+      "epoch": 0.4379432624113475,
+      "grad_norm": 0.3550000786781311,
+      "learning_rate": 0.00023640429338103754,
+      "loss": 0.0511,
+      "step": 247
+    },
+    {
+      "epoch": 0.4397163120567376,
+      "grad_norm": 0.2764873206615448,
+      "learning_rate": 0.00023613595706618962,
+      "loss": 0.0336,
+      "step": 248
+    },
+    {
+      "epoch": 0.44148936170212766,
+      "grad_norm": 0.2158241719007492,
+      "learning_rate": 0.00023586762075134167,
+      "loss": 0.0275,
+      "step": 249
+    },
+    {
+      "epoch": 0.4432624113475177,
+      "grad_norm": 0.41014501452445984,
+      "learning_rate": 0.00023559928443649372,
+      "loss": 0.1071,
+      "step": 250
+    },
+    {
+      "epoch": 0.4450354609929078,
+      "grad_norm": 1.6365233659744263,
+      "learning_rate": 0.00023533094812164577,
+      "loss": 0.5651,
+      "step": 251
+    },
+    {
+      "epoch": 0.44680851063829785,
+      "grad_norm": 2.2182910442352295,
+      "learning_rate": 0.00023506261180679784,
+      "loss": 0.4931,
+      "step": 252
+    },
+    {
+      "epoch": 0.44858156028368795,
+      "grad_norm": 0.7922513484954834,
+      "learning_rate": 0.0002347942754919499,
+      "loss": 0.165,
+      "step": 253
+    },
+    {
+      "epoch": 0.450354609929078,
+      "grad_norm": 1.0960763692855835,
+      "learning_rate": 0.00023452593917710194,
+      "loss": 0.2092,
+      "step": 254
+    },
+    {
+      "epoch": 0.4521276595744681,
+      "grad_norm": 0.9665970802307129,
+      "learning_rate": 0.000234257602862254,
+      "loss": 0.3383,
+      "step": 255
+    },
+    {
+      "epoch": 0.45390070921985815,
+      "grad_norm": 0.9593271613121033,
+      "learning_rate": 0.00023398926654740607,
+      "loss": 0.2608,
+      "step": 256
+    },
+    {
+      "epoch": 0.45567375886524825,
+      "grad_norm": 0.9603182673454285,
+      "learning_rate": 0.00023372093023255814,
+      "loss": 0.3786,
+      "step": 257
+    },
+    {
+      "epoch": 0.4574468085106383,
+      "grad_norm": 0.6080289483070374,
+      "learning_rate": 0.00023345259391771016,
+      "loss": 0.1903,
+      "step": 258
+    },
+    {
+      "epoch": 0.4592198581560284,
+      "grad_norm": 0.4713785946369171,
+      "learning_rate": 0.00023318425760286224,
+      "loss": 0.1438,
+      "step": 259
+    },
+    {
+      "epoch": 0.46099290780141844,
+      "grad_norm": 0.7215020656585693,
+      "learning_rate": 0.0002329159212880143,
+      "loss": 0.1979,
+      "step": 260
+    },
+    {
+      "epoch": 0.4627659574468085,
+      "grad_norm": 0.7383154630661011,
+      "learning_rate": 0.00023264758497316637,
+      "loss": 0.3316,
+      "step": 261
+    },
+    {
+      "epoch": 0.4645390070921986,
+      "grad_norm": 0.8826196789741516,
+      "learning_rate": 0.0002323792486583184,
+      "loss": 0.3337,
+      "step": 262
+    },
+    {
+      "epoch": 0.46631205673758863,
+      "grad_norm": 0.6072385907173157,
+      "learning_rate": 0.00023211091234347046,
+      "loss": 0.2585,
+      "step": 263
+    },
+    {
+      "epoch": 0.46808510638297873,
+      "grad_norm": 0.5861011743545532,
+      "learning_rate": 0.0002318425760286225,
+      "loss": 0.1952,
+      "step": 264
+    },
+    {
+      "epoch": 0.4698581560283688,
+      "grad_norm": 0.8113440871238708,
+      "learning_rate": 0.0002315742397137746,
+      "loss": 0.1877,
+      "step": 265
+    },
+    {
+      "epoch": 0.4716312056737589,
+      "grad_norm": 0.3667961359024048,
+      "learning_rate": 0.00023130590339892664,
+      "loss": 0.0977,
+      "step": 266
+    },
+    {
+      "epoch": 0.4734042553191489,
+      "grad_norm": 0.4013451933860779,
+      "learning_rate": 0.0002310375670840787,
+      "loss": 0.0988,
+      "step": 267
+    },
+    {
+      "epoch": 0.475177304964539,
+      "grad_norm": 0.4402759373188019,
+      "learning_rate": 0.00023076923076923076,
+      "loss": 0.1747,
+      "step": 268
+    },
+    {
+      "epoch": 0.4769503546099291,
+      "grad_norm": 0.5780351161956787,
+      "learning_rate": 0.0002305008944543828,
+      "loss": 0.1446,
+      "step": 269
+    },
+    {
+      "epoch": 0.4787234042553192,
+      "grad_norm": 1.0400667190551758,
+      "learning_rate": 0.00023023255813953486,
+      "loss": 0.3404,
+      "step": 270
+    },
+    {
+      "epoch": 0.4804964539007092,
+      "grad_norm": 0.734530508518219,
+      "learning_rate": 0.0002299642218246869,
+      "loss": 0.1938,
+      "step": 271
+    },
+    {
+      "epoch": 0.48226950354609927,
+      "grad_norm": 0.4358377158641815,
+      "learning_rate": 0.000229695885509839,
+      "loss": 0.1514,
+      "step": 272
+    },
+    {
+      "epoch": 0.48404255319148937,
+      "grad_norm": 0.766135036945343,
+      "learning_rate": 0.00022942754919499104,
+      "loss": 0.2335,
+      "step": 273
+    },
+    {
+      "epoch": 0.4858156028368794,
+      "grad_norm": 0.8013383746147156,
+      "learning_rate": 0.00022915921288014308,
+      "loss": 0.2433,
+      "step": 274
+    },
+    {
+      "epoch": 0.4875886524822695,
+      "grad_norm": 0.48911821842193604,
+      "learning_rate": 0.00022889087656529513,
+      "loss": 0.1466,
+      "step": 275
+    },
+    {
+      "epoch": 0.48936170212765956,
+      "grad_norm": 0.3731197118759155,
+      "learning_rate": 0.0002286225402504472,
+      "loss": 0.095,
+      "step": 276
+    },
+    {
+      "epoch": 0.49113475177304966,
+      "grad_norm": 0.4375539720058441,
+      "learning_rate": 0.00022835420393559929,
+      "loss": 0.1066,
+      "step": 277
+    },
+    {
+      "epoch": 0.4929078014184397,
+      "grad_norm": 0.5769888758659363,
+      "learning_rate": 0.0002280858676207513,
+      "loss": 0.1959,
+      "step": 278
+    },
+    {
+      "epoch": 0.4946808510638298,
+      "grad_norm": 0.6133276224136353,
+      "learning_rate": 0.00022781753130590338,
+      "loss": 0.1216,
+      "step": 279
+    },
+    {
+      "epoch": 0.49645390070921985,
+      "grad_norm": 0.8098652362823486,
+      "learning_rate": 0.00022754919499105543,
+      "loss": 0.223,
+      "step": 280
+    },
+    {
+      "epoch": 0.49822695035460995,
+      "grad_norm": 0.38705819845199585,
+      "learning_rate": 0.0002272808586762075,
+      "loss": 0.0888,
+      "step": 281
+    },
+    {
+      "epoch": 0.5,
+      "grad_norm": 0.37354519963264465,
+      "learning_rate": 0.00022701252236135953,
+      "loss": 0.1013,
+      "step": 282
+    },
+    {
+      "epoch": 0.50177304964539,
+      "grad_norm": 0.6955541968345642,
+      "learning_rate": 0.0002267441860465116,
+      "loss": 0.1746,
+      "step": 283
+    },
+    {
+      "epoch": 0.5035460992907801,
+      "grad_norm": 0.44892531633377075,
+      "learning_rate": 0.00022647584973166366,
+      "loss": 0.123,
+      "step": 284
+    },
+    {
+      "epoch": 0.5053191489361702,
+      "grad_norm": 0.7901389598846436,
+      "learning_rate": 0.00022620751341681573,
+      "loss": 0.2863,
+      "step": 285
+    },
+    {
+      "epoch": 0.5070921985815603,
+      "grad_norm": 0.6648136973381042,
+      "learning_rate": 0.00022593917710196778,
+      "loss": 0.1577,
+      "step": 286
+    },
+    {
+      "epoch": 0.5088652482269503,
+      "grad_norm": 0.3378174602985382,
+      "learning_rate": 0.00022567084078711983,
+      "loss": 0.0916,
+      "step": 287
+    },
+    {
+      "epoch": 0.5106382978723404,
+      "grad_norm": 0.46413183212280273,
+      "learning_rate": 0.0002254025044722719,
+      "loss": 0.1292,
+      "step": 288
+    },
+    {
+      "epoch": 0.5124113475177305,
+      "grad_norm": 0.5988323092460632,
+      "learning_rate": 0.00022513416815742396,
+      "loss": 0.1586,
+      "step": 289
+    },
+    {
+      "epoch": 0.5141843971631206,
+      "grad_norm": 0.5859420299530029,
+      "learning_rate": 0.000224865831842576,
+      "loss": 0.1296,
+      "step": 290
+    },
+    {
+      "epoch": 0.5159574468085106,
+      "grad_norm": 0.4029462933540344,
+      "learning_rate": 0.00022459749552772805,
+      "loss": 0.1072,
+      "step": 291
+    },
+    {
+      "epoch": 0.5177304964539007,
+      "grad_norm": 0.28882545232772827,
+      "learning_rate": 0.00022432915921288013,
+      "loss": 0.0747,
+      "step": 292
+    },
+    {
+      "epoch": 0.5195035460992907,
+      "grad_norm": 0.35556840896606445,
+      "learning_rate": 0.00022406082289803218,
+      "loss": 0.0463,
+      "step": 293
+    },
+    {
+      "epoch": 0.5212765957446809,
+      "grad_norm": 0.5565528273582458,
+      "learning_rate": 0.00022379248658318423,
+      "loss": 0.0675,
+      "step": 294
+    },
+    {
+      "epoch": 0.5230496453900709,
+      "grad_norm": 0.5911768078804016,
+      "learning_rate": 0.0002235241502683363,
+      "loss": 0.1442,
+      "step": 295
+    },
+    {
+      "epoch": 0.524822695035461,
+      "grad_norm": 0.4034654498100281,
+      "learning_rate": 0.00022325581395348835,
+      "loss": 0.1111,
+      "step": 296
+    },
+    {
+      "epoch": 0.526595744680851,
+      "grad_norm": 0.35688602924346924,
+      "learning_rate": 0.00022298747763864043,
+      "loss": 0.0828,
+      "step": 297
+    },
+    {
+      "epoch": 0.5283687943262412,
+      "grad_norm": 0.6449804306030273,
+      "learning_rate": 0.00022271914132379245,
+      "loss": 0.2104,
+      "step": 298
+    },
+    {
+      "epoch": 0.5301418439716312,
+      "grad_norm": 0.5375797152519226,
+      "learning_rate": 0.00022245080500894453,
+      "loss": 0.1546,
+      "step": 299
+    },
+    {
+      "epoch": 0.5319148936170213,
+      "grad_norm": 0.18811915814876556,
+      "learning_rate": 0.00022218246869409658,
+      "loss": 0.0246,
+      "step": 300
+    },
+    {
+      "epoch": 0.5336879432624113,
+      "grad_norm": 1.5281504392623901,
+      "learning_rate": 0.00022191413237924865,
+      "loss": 0.5446,
+      "step": 301
+    },
+    {
+      "epoch": 0.5354609929078015,
+      "grad_norm": 1.5097373723983765,
+      "learning_rate": 0.00022164579606440067,
+      "loss": 0.4033,
+      "step": 302
+    },
+    {
+      "epoch": 0.5372340425531915,
+      "grad_norm": 1.4294488430023193,
+      "learning_rate": 0.00022137745974955275,
+      "loss": 0.3835,
+      "step": 303
+    },
+    {
+      "epoch": 0.5390070921985816,
+      "grad_norm": 0.7668700218200684,
+      "learning_rate": 0.00022110912343470483,
+      "loss": 0.2103,
+      "step": 304
+    },
+    {
+      "epoch": 0.5407801418439716,
+      "grad_norm": 0.9126233458518982,
+      "learning_rate": 0.00022084078711985688,
+      "loss": 0.39,
+      "step": 305
+    },
+    {
+      "epoch": 0.5425531914893617,
+      "grad_norm": 0.7275278568267822,
+      "learning_rate": 0.00022057245080500892,
+      "loss": 0.2128,
+      "step": 306
+    },
+    {
+      "epoch": 0.5443262411347518,
+      "grad_norm": 0.6632657647132874,
+      "learning_rate": 0.00022030411449016097,
+      "loss": 0.1819,
+      "step": 307
+    },
+    {
+      "epoch": 0.5460992907801419,
+      "grad_norm": 0.5265213847160339,
+      "learning_rate": 0.00022003577817531305,
+      "loss": 0.1879,
+      "step": 308
+    },
+    {
+      "epoch": 0.5478723404255319,
+      "grad_norm": 0.4046447277069092,
+      "learning_rate": 0.0002197674418604651,
+      "loss": 0.1225,
+      "step": 309
+    },
+    {
+      "epoch": 0.549645390070922,
+      "grad_norm": 0.7889742851257324,
+      "learning_rate": 0.00021949910554561715,
+      "loss": 0.3022,
+      "step": 310
+    },
+    {
+      "epoch": 0.5514184397163121,
+      "grad_norm": 0.739321768283844,
+      "learning_rate": 0.0002192307692307692,
+      "loss": 0.1621,
+      "step": 311
+    },
+    {
+      "epoch": 0.5531914893617021,
+      "grad_norm": 0.5458974838256836,
+      "learning_rate": 0.00021896243291592127,
+      "loss": 0.1505,
+      "step": 312
+    },
+    {
+      "epoch": 0.5549645390070922,
+      "grad_norm": 0.6029899716377258,
+      "learning_rate": 0.00021869409660107332,
+      "loss": 0.2114,
+      "step": 313
+    },
+    {
+      "epoch": 0.5567375886524822,
+      "grad_norm": 0.830324113368988,
+      "learning_rate": 0.00021842576028622537,
+      "loss": 0.2397,
+      "step": 314
+    },
+    {
+      "epoch": 0.5585106382978723,
+      "grad_norm": 0.7340556383132935,
+      "learning_rate": 0.00021815742397137745,
+      "loss": 0.1606,
+      "step": 315
+    },
+    {
+      "epoch": 0.5602836879432624,
+      "grad_norm": 0.44604039192199707,
+      "learning_rate": 0.0002178890876565295,
+      "loss": 0.1192,
+      "step": 316
+    },
+    {
+      "epoch": 0.5620567375886525,
+      "grad_norm": 0.37659141421318054,
+      "learning_rate": 0.00021762075134168157,
+      "loss": 0.1112,
+      "step": 317
+    },
+    {
+      "epoch": 0.5638297872340425,
+      "grad_norm": 0.8448424935340881,
+      "learning_rate": 0.0002173524150268336,
+      "loss": 0.1799,
+      "step": 318
+    },
+    {
+      "epoch": 0.5656028368794326,
+      "grad_norm": 0.5874999761581421,
+      "learning_rate": 0.00021708407871198567,
+      "loss": 0.2011,
+      "step": 319
+    },
+    {
+      "epoch": 0.5673758865248227,
+      "grad_norm": 0.5802026391029358,
+      "learning_rate": 0.00021681574239713772,
+      "loss": 0.1725,
+      "step": 320
+    },
+    {
+      "epoch": 0.5691489361702128,
+      "grad_norm": 0.5646753907203674,
+      "learning_rate": 0.0002165474060822898,
+      "loss": 0.1464,
+      "step": 321
+    },
+    {
+      "epoch": 0.5709219858156028,
+      "grad_norm": 0.5598042011260986,
+      "learning_rate": 0.00021627906976744182,
+      "loss": 0.1135,
+      "step": 322
+    },
+    {
+      "epoch": 0.5726950354609929,
+      "grad_norm": 0.35759246349334717,
+      "learning_rate": 0.0002160107334525939,
+      "loss": 0.079,
+      "step": 323
+    },
+    {
+      "epoch": 0.574468085106383,
+      "grad_norm": 0.8581865429878235,
+      "learning_rate": 0.00021574239713774597,
+      "loss": 0.2422,
+      "step": 324
+    },
+    {
+      "epoch": 0.5762411347517731,
+      "grad_norm": 0.7060332298278809,
+      "learning_rate": 0.00021547406082289802,
+      "loss": 0.2283,
+      "step": 325
+    },
+    {
+      "epoch": 0.5780141843971631,
+      "grad_norm": 0.7340462803840637,
+      "learning_rate": 0.00021520572450805007,
+      "loss": 0.1915,
+      "step": 326
+    },
+    {
+      "epoch": 0.5797872340425532,
+      "grad_norm": 0.42532655596733093,
+      "learning_rate": 0.00021493738819320212,
+      "loss": 0.1125,
+      "step": 327
+    },
+    {
+      "epoch": 0.5815602836879432,
+      "grad_norm": 0.2977962791919708,
+      "learning_rate": 0.0002146690518783542,
+      "loss": 0.0812,
+      "step": 328
+    },
+    {
+      "epoch": 0.5833333333333334,
+      "grad_norm": 0.3650486171245575,
+      "learning_rate": 0.00021440071556350624,
+      "loss": 0.0999,
+      "step": 329
+    },
+    {
+      "epoch": 0.5851063829787234,
+      "grad_norm": 0.6662601232528687,
+      "learning_rate": 0.0002141323792486583,
+      "loss": 0.2518,
+      "step": 330
+    },
+    {
+      "epoch": 0.5868794326241135,
+      "grad_norm": 0.46315595507621765,
+      "learning_rate": 0.00021386404293381034,
+      "loss": 0.1164,
+      "step": 331
+    },
+    {
+      "epoch": 0.5886524822695035,
+      "grad_norm": 0.5522826910018921,
+      "learning_rate": 0.00021359570661896242,
+      "loss": 0.1563,
+      "step": 332
+    },
+    {
+      "epoch": 0.5904255319148937,
+      "grad_norm": 0.6960483193397522,
+      "learning_rate": 0.0002133273703041145,
+      "loss": 0.1998,
+      "step": 333
+    },
+    {
+      "epoch": 0.5921985815602837,
+      "grad_norm": 0.46060803532600403,
+      "learning_rate": 0.00021305903398926651,
+      "loss": 0.1235,
+      "step": 334
+    },
+    {
+      "epoch": 0.5939716312056738,
+      "grad_norm": 0.8451377749443054,
+      "learning_rate": 0.0002127906976744186,
+      "loss": 0.2462,
+      "step": 335
+    },
+    {
+      "epoch": 0.5957446808510638,
+      "grad_norm": 0.4865632653236389,
+      "learning_rate": 0.00021252236135957064,
+      "loss": 0.1759,
+      "step": 336
+    },
+    {
+      "epoch": 0.5975177304964538,
+      "grad_norm": 0.5368924736976624,
+      "learning_rate": 0.00021225402504472272,
+      "loss": 0.1632,
+      "step": 337
+    },
+    {
+      "epoch": 0.599290780141844,
+      "grad_norm": 0.5031744837760925,
+      "learning_rate": 0.00021198568872987474,
+      "loss": 0.2001,
+      "step": 338
+    },
+    {
+      "epoch": 0.601063829787234,
+      "grad_norm": 0.35617366433143616,
+      "learning_rate": 0.00021171735241502681,
+      "loss": 0.0905,
+      "step": 339
+    },
+    {
+      "epoch": 0.6028368794326241,
+      "grad_norm": 0.4215497076511383,
+      "learning_rate": 0.00021144901610017886,
+      "loss": 0.1474,
+      "step": 340
+    },
+    {
+      "epoch": 0.6046099290780141,
+      "grad_norm": 0.3073301911354065,
+      "learning_rate": 0.00021118067978533094,
+      "loss": 0.053,
+      "step": 341
+    },
+    {
+      "epoch": 0.6063829787234043,
+      "grad_norm": 0.30299848318099976,
+      "learning_rate": 0.00021091234347048301,
+      "loss": 0.0715,
+      "step": 342
+    },
+    {
+      "epoch": 0.6081560283687943,
+      "grad_norm": 0.26558998227119446,
+      "learning_rate": 0.00021064400715563504,
+      "loss": 0.0573,
+      "step": 343
+    },
+    {
+      "epoch": 0.6099290780141844,
+      "grad_norm": 0.21946674585342407,
+      "learning_rate": 0.0002103756708407871,
+      "loss": 0.0502,
+      "step": 344
+    },
+    {
+      "epoch": 0.6117021276595744,
+      "grad_norm": 0.48567119240760803,
+      "learning_rate": 0.00021010733452593916,
+      "loss": 0.1224,
+      "step": 345
+    },
+    {
+      "epoch": 0.6134751773049646,
+      "grad_norm": 0.444938600063324,
+      "learning_rate": 0.00020983899821109124,
+      "loss": 0.0899,
+      "step": 346
+    },
+    {
+      "epoch": 0.6152482269503546,
+      "grad_norm": 0.3576812744140625,
+      "learning_rate": 0.00020957066189624326,
+      "loss": 0.1104,
+      "step": 347
+    },
+    {
+      "epoch": 0.6170212765957447,
+      "grad_norm": 0.5801045298576355,
+      "learning_rate": 0.00020930232558139534,
+      "loss": 0.2192,
+      "step": 348
+    },
+    {
+      "epoch": 0.6187943262411347,
+      "grad_norm": 0.3965771198272705,
+      "learning_rate": 0.00020903398926654739,
+      "loss": 0.1025,
+      "step": 349
+    },
+    {
+      "epoch": 0.6205673758865248,
+      "grad_norm": 0.32997211813926697,
+      "learning_rate": 0.00020876565295169943,
+      "loss": 0.0392,
+      "step": 350
+    },
+    {
+      "epoch": 0.6223404255319149,
+      "grad_norm": 2.3874707221984863,
+      "learning_rate": 0.00020849731663685148,
+      "loss": 0.4267,
+      "step": 351
+    },
+    {
+      "epoch": 0.624113475177305,
+      "grad_norm": 1.0561052560806274,
+      "learning_rate": 0.00020822898032200356,
+      "loss": 0.3531,
+      "step": 352
+    },
+    {
+      "epoch": 0.625886524822695,
+      "grad_norm": 1.0714682340621948,
+      "learning_rate": 0.00020796064400715564,
+      "loss": 0.208,
+      "step": 353
+    },
+    {
+      "epoch": 0.6276595744680851,
+      "grad_norm": 1.0135624408721924,
+      "learning_rate": 0.00020769230769230766,
+      "loss": 0.2722,
+      "step": 354
+    },
+    {
+      "epoch": 0.6294326241134752,
+      "grad_norm": 0.4171317219734192,
+      "learning_rate": 0.00020742397137745973,
+      "loss": 0.1137,
+      "step": 355
+    },
+    {
+      "epoch": 0.6312056737588653,
+      "grad_norm": 0.7069351673126221,
+      "learning_rate": 0.00020715563506261178,
+      "loss": 0.1919,
+      "step": 356
+    },
+    {
+      "epoch": 0.6329787234042553,
+      "grad_norm": 0.5680619478225708,
+      "learning_rate": 0.00020688729874776386,
+      "loss": 0.1884,
+      "step": 357
+    },
+    {
+      "epoch": 0.6347517730496454,
+      "grad_norm": 0.8620048761367798,
+      "learning_rate": 0.00020661896243291588,
+      "loss": 0.3209,
+      "step": 358
+    },
+    {
+      "epoch": 0.6365248226950354,
+      "grad_norm": 0.4163174331188202,
+      "learning_rate": 0.00020635062611806796,
+      "loss": 0.1553,
+      "step": 359
+    },
+    {
+      "epoch": 0.6382978723404256,
+      "grad_norm": 0.7022566199302673,
+      "learning_rate": 0.00020608228980322,
+      "loss": 0.2839,
+      "step": 360
+    },
+    {
+      "epoch": 0.6400709219858156,
+      "grad_norm": 0.35048022866249084,
+      "learning_rate": 0.00020581395348837208,
+      "loss": 0.1168,
+      "step": 361
+    },
+    {
+      "epoch": 0.6418439716312057,
+      "grad_norm": 0.6058384776115417,
+      "learning_rate": 0.00020554561717352416,
+      "loss": 0.2336,
+      "step": 362
+    },
+    {
+      "epoch": 0.6436170212765957,
+      "grad_norm": 0.5632460117340088,
+      "learning_rate": 0.00020527728085867618,
+      "loss": 0.1866,
+      "step": 363
+    },
+    {
+      "epoch": 0.6453900709219859,
+      "grad_norm": 0.8508513569831848,
+      "learning_rate": 0.00020500894454382826,
+      "loss": 0.2795,
+      "step": 364
+    },
+    {
+      "epoch": 0.6471631205673759,
+      "grad_norm": 0.3261764645576477,
+      "learning_rate": 0.0002047406082289803,
+      "loss": 0.0695,
+      "step": 365
+    },
+    {
+      "epoch": 0.648936170212766,
+      "grad_norm": 0.6185038089752197,
+      "learning_rate": 0.00020447227191413238,
+      "loss": 0.2125,
+      "step": 366
+    },
+    {
+      "epoch": 0.650709219858156,
+      "grad_norm": 0.9981937408447266,
+      "learning_rate": 0.0002042039355992844,
+      "loss": 0.179,
+      "step": 367
+    },
+    {
+      "epoch": 0.6524822695035462,
+      "grad_norm": 0.625804603099823,
+      "learning_rate": 0.00020393559928443648,
+      "loss": 0.2315,
+      "step": 368
+    },
+    {
+      "epoch": 0.6542553191489362,
+      "grad_norm": 0.460195928812027,
+      "learning_rate": 0.00020366726296958853,
+      "loss": 0.1352,
+      "step": 369
+    },
+    {
+      "epoch": 0.6560283687943262,
+      "grad_norm": 0.6661739349365234,
+      "learning_rate": 0.00020339892665474058,
+      "loss": 0.2227,
+      "step": 370
+    },
+    {
+      "epoch": 0.6578014184397163,
+      "grad_norm": 0.5500937700271606,
+      "learning_rate": 0.00020313059033989265,
+      "loss": 0.1519,
+      "step": 371
+    },
+    {
+      "epoch": 0.6595744680851063,
+      "grad_norm": 0.8262995481491089,
+      "learning_rate": 0.0002028622540250447,
+      "loss": 0.2965,
+      "step": 372
+    },
+    {
+      "epoch": 0.6613475177304965,
+      "grad_norm": 0.6892026662826538,
+      "learning_rate": 0.00020259391771019678,
+      "loss": 0.2244,
+      "step": 373
+    },
+    {
+      "epoch": 0.6631205673758865,
+      "grad_norm": 0.9893831014633179,
+      "learning_rate": 0.0002023255813953488,
+      "loss": 0.27,
+      "step": 374
+    },
+    {
+      "epoch": 0.6648936170212766,
+      "grad_norm": 0.26278918981552124,
+      "learning_rate": 0.00020205724508050088,
+      "loss": 0.0958,
+      "step": 375
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 0.4749915301799774,
+      "learning_rate": 0.00020178890876565293,
+      "loss": 0.182,
+      "step": 376
+    },
+    {
+      "epoch": 0.6684397163120568,
+      "grad_norm": 0.36534926295280457,
+      "learning_rate": 0.000201520572450805,
+      "loss": 0.0955,
+      "step": 377
+    },
+    {
+      "epoch": 0.6702127659574468,
+      "grad_norm": 0.28374701738357544,
+      "learning_rate": 0.00020125223613595702,
+      "loss": 0.083,
+      "step": 378
+    },
+    {
+      "epoch": 0.6719858156028369,
+      "grad_norm": 0.5432383418083191,
+      "learning_rate": 0.0002009838998211091,
+      "loss": 0.1533,
+      "step": 379
+    },
+    {
+      "epoch": 0.6737588652482269,
+      "grad_norm": 0.6451116800308228,
+      "learning_rate": 0.00020071556350626118,
+      "loss": 0.2346,
+      "step": 380
+    },
+    {
+      "epoch": 0.675531914893617,
+      "grad_norm": 0.3411319851875305,
+      "learning_rate": 0.00020044722719141323,
+      "loss": 0.0937,
+      "step": 381
+    },
+    {
+      "epoch": 0.6773049645390071,
+      "grad_norm": 0.40049058198928833,
+      "learning_rate": 0.0002001788908765653,
+      "loss": 0.1219,
+      "step": 382
+    },
+    {
+      "epoch": 0.6790780141843972,
+      "grad_norm": 0.3120768070220947,
+      "learning_rate": 0.00019991055456171732,
+      "loss": 0.1086,
+      "step": 383
+    },
+    {
+      "epoch": 0.6808510638297872,
+      "grad_norm": 0.3227909207344055,
+      "learning_rate": 0.0001996422182468694,
+      "loss": 0.0911,
+      "step": 384
+    },
+    {
+      "epoch": 0.6826241134751773,
+      "grad_norm": 0.22975340485572815,
+      "learning_rate": 0.00019937388193202145,
+      "loss": 0.0818,
+      "step": 385
+    },
+    {
+      "epoch": 0.6843971631205674,
+      "grad_norm": 0.3845633268356323,
+      "learning_rate": 0.00019910554561717352,
+      "loss": 0.1278,
+      "step": 386
+    },
+    {
+      "epoch": 0.6861702127659575,
+      "grad_norm": 0.4206799864768982,
+      "learning_rate": 0.00019883720930232555,
+      "loss": 0.1137,
+      "step": 387
+    },
+    {
+      "epoch": 0.6879432624113475,
+      "grad_norm": 0.4134272634983063,
+      "learning_rate": 0.00019856887298747762,
+      "loss": 0.1299,
+      "step": 388
+    },
+    {
+      "epoch": 0.6897163120567376,
+      "grad_norm": 0.44249987602233887,
+      "learning_rate": 0.0001983005366726297,
+      "loss": 0.1211,
+      "step": 389
+    },
+    {
+      "epoch": 0.6914893617021277,
+      "grad_norm": 0.3611302971839905,
+      "learning_rate": 0.00019803220035778172,
+      "loss": 0.114,
+      "step": 390
+    },
+    {
+      "epoch": 0.6932624113475178,
+      "grad_norm": 0.47269973158836365,
+      "learning_rate": 0.0001977638640429338,
+      "loss": 0.1405,
+      "step": 391
+    },
+    {
+      "epoch": 0.6950354609929078,
+      "grad_norm": 0.47546666860580444,
+      "learning_rate": 0.00019749552772808585,
+      "loss": 0.1447,
+      "step": 392
+    },
+    {
+      "epoch": 0.6968085106382979,
+      "grad_norm": 0.3153839111328125,
+      "learning_rate": 0.00019722719141323792,
+      "loss": 0.0972,
+      "step": 393
+    },
+    {
+      "epoch": 0.6985815602836879,
+      "grad_norm": 0.23356540501117706,
+      "learning_rate": 0.00019695885509838994,
+      "loss": 0.0429,
+      "step": 394
+    },
+    {
+      "epoch": 0.700354609929078,
+      "grad_norm": 0.399263858795166,
+      "learning_rate": 0.00019669051878354202,
+      "loss": 0.1401,
+      "step": 395
+    },
+    {
+      "epoch": 0.7021276595744681,
+      "grad_norm": 0.36282962560653687,
+      "learning_rate": 0.00019642218246869407,
+      "loss": 0.11,
+      "step": 396
+    },
+    {
+      "epoch": 0.7039007092198581,
+      "grad_norm": 0.4403574764728546,
+      "learning_rate": 0.00019615384615384615,
+      "loss": 0.0706,
+      "step": 397
+    },
+    {
+      "epoch": 0.7056737588652482,
+      "grad_norm": 0.5901324152946472,
+      "learning_rate": 0.00019588550983899817,
+      "loss": 0.1191,
+      "step": 398
+    },
+    {
+      "epoch": 0.7074468085106383,
+      "grad_norm": 0.2563900351524353,
+      "learning_rate": 0.00019561717352415024,
+      "loss": 0.0731,
+      "step": 399
+    },
+    {
+      "epoch": 0.7092198581560284,
+      "grad_norm": 0.4444797933101654,
+      "learning_rate": 0.00019534883720930232,
+      "loss": 0.1425,
+      "step": 400
+    },
+    {
+      "epoch": 0.7109929078014184,
+      "grad_norm": 1.0342155694961548,
+      "learning_rate": 0.00019508050089445437,
+      "loss": 0.3125,
+      "step": 401
+    },
+    {
+      "epoch": 0.7127659574468085,
+      "grad_norm": 0.6980201005935669,
+      "learning_rate": 0.00019481216457960644,
+      "loss": 0.2188,
+      "step": 402
+    },
+    {
+      "epoch": 0.7145390070921985,
+      "grad_norm": 0.740844190120697,
+      "learning_rate": 0.00019454382826475847,
+      "loss": 0.2467,
+      "step": 403
+    },
+    {
+      "epoch": 0.7163120567375887,
+      "grad_norm": 0.513054370880127,
+      "learning_rate": 0.00019427549194991054,
+      "loss": 0.1531,
+      "step": 404
+    },
+    {
+      "epoch": 0.7180851063829787,
+      "grad_norm": 0.4994530975818634,
+      "learning_rate": 0.0001940071556350626,
+      "loss": 0.1065,
+      "step": 405
+    },
+    {
+      "epoch": 0.7198581560283688,
+      "grad_norm": 0.9895260334014893,
+      "learning_rate": 0.00019373881932021467,
+      "loss": 0.4544,
+      "step": 406
+    },
+    {
+      "epoch": 0.7216312056737588,
+      "grad_norm": 0.422955721616745,
+      "learning_rate": 0.0001934704830053667,
+      "loss": 0.1079,
+      "step": 407
+    },
+    {
+      "epoch": 0.723404255319149,
+      "grad_norm": 0.5568337440490723,
+      "learning_rate": 0.00019320214669051877,
+      "loss": 0.2162,
+      "step": 408
+    },
+    {
+      "epoch": 0.725177304964539,
+      "grad_norm": 0.4763684868812561,
+      "learning_rate": 0.00019293381037567084,
+      "loss": 0.1643,
+      "step": 409
+    },
+    {
+      "epoch": 0.7269503546099291,
+      "grad_norm": 0.4117484390735626,
+      "learning_rate": 0.0001926654740608229,
+      "loss": 0.1393,
+      "step": 410
+    },
+    {
+      "epoch": 0.7287234042553191,
+      "grad_norm": 0.4180298447608948,
+      "learning_rate": 0.00019239713774597494,
+      "loss": 0.183,
+      "step": 411
+    },
+    {
+      "epoch": 0.7304964539007093,
+      "grad_norm": 0.42761579155921936,
+      "learning_rate": 0.000192128801431127,
+      "loss": 0.1463,
+      "step": 412
+    },
+    {
+      "epoch": 0.7322695035460993,
+      "grad_norm": 0.36335864663124084,
+      "learning_rate": 0.00019186046511627906,
+      "loss": 0.1032,
+      "step": 413
+    },
+    {
+      "epoch": 0.7340425531914894,
+      "grad_norm": 0.4711301326751709,
+      "learning_rate": 0.0001915921288014311,
+      "loss": 0.17,
+      "step": 414
+    },
+    {
+      "epoch": 0.7358156028368794,
+      "grad_norm": 0.38681793212890625,
+      "learning_rate": 0.00019132379248658316,
+      "loss": 0.1371,
+      "step": 415
+    },
+    {
+      "epoch": 0.7375886524822695,
+      "grad_norm": 0.38631147146224976,
+      "learning_rate": 0.0001910554561717352,
+      "loss": 0.1183,
+      "step": 416
+    },
+    {
+      "epoch": 0.7393617021276596,
+      "grad_norm": 0.4759841561317444,
+      "learning_rate": 0.0001907871198568873,
+      "loss": 0.1441,
+      "step": 417
+    },
+    {
+      "epoch": 0.7411347517730497,
+      "grad_norm": 0.42136093974113464,
+      "learning_rate": 0.00019051878354203936,
+      "loss": 0.1093,
+      "step": 418
+    },
+    {
+      "epoch": 0.7429078014184397,
+      "grad_norm": 0.5578286051750183,
+      "learning_rate": 0.00019025044722719139,
+      "loss": 0.1521,
+      "step": 419
+    },
+    {
+      "epoch": 0.7446808510638298,
+      "grad_norm": 0.32407546043395996,
+      "learning_rate": 0.00018998211091234346,
+      "loss": 0.0614,
+      "step": 420
+    },
+    {
+      "epoch": 0.7464539007092199,
+      "grad_norm": 0.621771514415741,
+      "learning_rate": 0.0001897137745974955,
+      "loss": 0.2111,
+      "step": 421
+    },
+    {
+      "epoch": 0.74822695035461,
+      "grad_norm": 0.4861868619918823,
+      "learning_rate": 0.0001894454382826476,
+      "loss": 0.1138,
+      "step": 422
+    },
+    {
+      "epoch": 0.75,
+      "grad_norm": 0.6632353067398071,
+      "learning_rate": 0.0001891771019677996,
+      "loss": 0.2492,
+      "step": 423
+    },
+    {
+      "epoch": 0.75177304964539,
+      "grad_norm": 0.3670256733894348,
+      "learning_rate": 0.00018890876565295169,
+      "loss": 0.0927,
+      "step": 424
+    },
+    {
+      "epoch": 0.7535460992907801,
+      "grad_norm": 0.48544085025787354,
+      "learning_rate": 0.00018864042933810373,
+      "loss": 0.1422,
+      "step": 425
+    },
+    {
+      "epoch": 0.7553191489361702,
+      "grad_norm": 0.3215571939945221,
+      "learning_rate": 0.0001883720930232558,
+      "loss": 0.0729,
+      "step": 426
+    },
+    {
+      "epoch": 0.7570921985815603,
+      "grad_norm": 0.6928660869598389,
+      "learning_rate": 0.00018810375670840786,
+      "loss": 0.2033,
+      "step": 427
+    },
+    {
+      "epoch": 0.7588652482269503,
+      "grad_norm": 0.42845651507377625,
+      "learning_rate": 0.0001878354203935599,
+      "loss": 0.1536,
+      "step": 428
+    },
+    {
+      "epoch": 0.7606382978723404,
+      "grad_norm": 0.5783790349960327,
+      "learning_rate": 0.00018756708407871198,
+      "loss": 0.1573,
+      "step": 429
+    },
+    {
+      "epoch": 0.7624113475177305,
+      "grad_norm": 0.34763625264167786,
+      "learning_rate": 0.00018729874776386403,
+      "loss": 0.0891,
+      "step": 430
+    },
+    {
+      "epoch": 0.7641843971631206,
+      "grad_norm": 0.3298535943031311,
+      "learning_rate": 0.00018703041144901608,
+      "loss": 0.0836,
+      "step": 431
+    },
+    {
+      "epoch": 0.7659574468085106,
+      "grad_norm": 0.48353129625320435,
+      "learning_rate": 0.00018676207513416813,
+      "loss": 0.1624,
+      "step": 432
+    },
+    {
+      "epoch": 0.7677304964539007,
+      "grad_norm": 0.3654095530509949,
+      "learning_rate": 0.0001864937388193202,
+      "loss": 0.074,
+      "step": 433
+    },
+    {
+      "epoch": 0.7695035460992907,
+      "grad_norm": 0.6465492248535156,
+      "learning_rate": 0.00018622540250447223,
+      "loss": 0.2221,
+      "step": 434
+    },
+    {
+      "epoch": 0.7712765957446809,
+      "grad_norm": 0.29519444704055786,
+      "learning_rate": 0.0001859570661896243,
+      "loss": 0.073,
+      "step": 435
+    },
+    {
+      "epoch": 0.7730496453900709,
+      "grad_norm": 0.39645931124687195,
+      "learning_rate": 0.00018568872987477636,
+      "loss": 0.0799,
+      "step": 436
+    },
+    {
+      "epoch": 0.774822695035461,
+      "grad_norm": 0.7929151058197021,
+      "learning_rate": 0.00018542039355992843,
+      "loss": 0.244,
+      "step": 437
+    },
+    {
+      "epoch": 0.776595744680851,
+      "grad_norm": 0.5418475866317749,
+      "learning_rate": 0.0001851520572450805,
+      "loss": 0.1439,
+      "step": 438
+    },
+    {
+      "epoch": 0.7783687943262412,
+      "grad_norm": 0.4036068320274353,
+      "learning_rate": 0.00018488372093023253,
+      "loss": 0.1158,
+      "step": 439
+    },
+    {
+      "epoch": 0.7801418439716312,
+      "grad_norm": 0.33123520016670227,
+      "learning_rate": 0.0001846153846153846,
+      "loss": 0.1182,
+      "step": 440
+    },
+    {
+      "epoch": 0.7819148936170213,
+      "grad_norm": 0.3179425299167633,
+      "learning_rate": 0.00018434704830053665,
+      "loss": 0.0763,
+      "step": 441
+    },
+    {
+      "epoch": 0.7836879432624113,
+      "grad_norm": 0.25326409935951233,
+      "learning_rate": 0.00018407871198568873,
+      "loss": 0.0388,
+      "step": 442
+    },
+    {
+      "epoch": 0.7854609929078015,
+      "grad_norm": 0.4816826581954956,
+      "learning_rate": 0.00018381037567084075,
+      "loss": 0.1308,
+      "step": 443
+    },
+    {
+      "epoch": 0.7872340425531915,
+      "grad_norm": 0.2538895010948181,
+      "learning_rate": 0.00018354203935599283,
+      "loss": 0.0607,
+      "step": 444
+    },
+    {
+      "epoch": 0.7890070921985816,
+      "grad_norm": 0.3773513436317444,
+      "learning_rate": 0.00018327370304114488,
+      "loss": 0.1168,
+      "step": 445
+    },
+    {
+      "epoch": 0.7907801418439716,
+      "grad_norm": 0.48236358165740967,
+      "learning_rate": 0.00018300536672629695,
+      "loss": 0.1078,
+      "step": 446
+    },
+    {
+      "epoch": 0.7925531914893617,
+      "grad_norm": 0.37481409311294556,
+      "learning_rate": 0.000182737030411449,
+      "loss": 0.0847,
+      "step": 447
+    },
+    {
+      "epoch": 0.7943262411347518,
+      "grad_norm": 0.25582361221313477,
+      "learning_rate": 0.00018246869409660105,
+      "loss": 0.0518,
+      "step": 448
+    },
+    {
+      "epoch": 0.7960992907801419,
+      "grad_norm": 0.582918643951416,
+      "learning_rate": 0.00018220035778175313,
+      "loss": 0.199,
+      "step": 449
+    },
+    {
+      "epoch": 0.7978723404255319,
+      "grad_norm": 0.3821042478084564,
+      "learning_rate": 0.00018193202146690518,
+      "loss": 0.1308,
+      "step": 450
+    },
+    {
+      "epoch": 0.799645390070922,
+      "grad_norm": 0.9046875834465027,
+      "learning_rate": 0.00018166368515205723,
+      "loss": 0.3907,
+      "step": 451
+    },
+    {
+      "epoch": 0.8014184397163121,
+      "grad_norm": 0.9091282486915588,
+      "learning_rate": 0.00018139534883720928,
+      "loss": 0.421,
+      "step": 452
+    },
+    {
+      "epoch": 0.8014184397163121,
+      "eval_loss": 0.07596753537654877,
+      "eval_runtime": 47.273,
+      "eval_samples_per_second": 2.665,
+      "eval_steps_per_second": 2.665,
+      "step": 452
+    },
+    {
+      "epoch": 0.8031914893617021,
+      "grad_norm": 0.5998420715332031,
+      "learning_rate": 0.00018112701252236135,
+      "loss": 0.1944,
+      "step": 453
+    },
+    {
+      "epoch": 0.8049645390070922,
+      "grad_norm": 0.6365135908126831,
+      "learning_rate": 0.00018085867620751337,
+      "loss": 0.2445,
+      "step": 454
+    },
+    {
+      "epoch": 0.8067375886524822,
+      "grad_norm": 0.5594285726547241,
+      "learning_rate": 0.00018059033989266545,
+      "loss": 0.2096,
+      "step": 455
+    },
+    {
+      "epoch": 0.8085106382978723,
+      "grad_norm": 0.8409189581871033,
+      "learning_rate": 0.00018032200357781753,
+      "loss": 0.4065,
+      "step": 456
+    },
+    {
+      "epoch": 0.8102836879432624,
+      "grad_norm": 0.5661482214927673,
+      "learning_rate": 0.00018005366726296957,
+      "loss": 0.25,
+      "step": 457
+    },
+    {
+      "epoch": 0.8120567375886525,
+      "grad_norm": 0.5428692102432251,
+      "learning_rate": 0.00017978533094812165,
+      "loss": 0.2254,
+      "step": 458
+    },
+    {
+      "epoch": 0.8138297872340425,
+      "grad_norm": 0.4579595923423767,
+      "learning_rate": 0.00017951699463327367,
+      "loss": 0.1008,
+      "step": 459
+    },
+    {
+      "epoch": 0.8156028368794326,
+      "grad_norm": 0.4390023946762085,
+      "learning_rate": 0.00017924865831842575,
+      "loss": 0.1144,
+      "step": 460
+    },
+    {
+      "epoch": 0.8173758865248227,
+      "grad_norm": 0.5851403474807739,
+      "learning_rate": 0.0001789803220035778,
+      "loss": 0.2355,
+      "step": 461
+    },
+    {
+      "epoch": 0.8191489361702128,
+      "grad_norm": 0.31094157695770264,
+      "learning_rate": 0.00017871198568872987,
+      "loss": 0.1078,
+      "step": 462
+    },
+    {
+      "epoch": 0.8209219858156028,
+      "grad_norm": 0.5213750600814819,
+      "learning_rate": 0.0001784436493738819,
+      "loss": 0.1546,
+      "step": 463
+    },
+    {
+      "epoch": 0.8226950354609929,
+      "grad_norm": 0.3972003161907196,
+      "learning_rate": 0.00017817531305903397,
+      "loss": 0.1379,
+      "step": 464
+    },
+    {
+      "epoch": 0.824468085106383,
+      "grad_norm": 0.35936617851257324,
+      "learning_rate": 0.00017790697674418605,
+      "loss": 0.1238,
+      "step": 465
+    },
+    {
+      "epoch": 0.8262411347517731,
+      "grad_norm": 0.31084805727005005,
+      "learning_rate": 0.0001776386404293381,
+      "loss": 0.0898,
+      "step": 466
+    },
+    {
+      "epoch": 0.8280141843971631,
+      "grad_norm": 0.35060107707977295,
+      "learning_rate": 0.00017737030411449015,
+      "loss": 0.1444,
+      "step": 467
+    },
+    {
+      "epoch": 0.8297872340425532,
+      "grad_norm": 0.31308284401893616,
+      "learning_rate": 0.0001771019677996422,
+      "loss": 0.0722,
+      "step": 468
+    },
+    {
+      "epoch": 0.8315602836879432,
+      "grad_norm": 0.4981883466243744,
+      "learning_rate": 0.00017683363148479427,
+      "loss": 0.1764,
+      "step": 469
+    },
+    {
+      "epoch": 0.8333333333333334,
+      "grad_norm": 0.5609257817268372,
+      "learning_rate": 0.00017656529516994632,
+      "loss": 0.175,
+      "step": 470
+    },
+    {
+      "epoch": 0.8351063829787234,
+      "grad_norm": 0.5611094832420349,
+      "learning_rate": 0.00017629695885509837,
+      "loss": 0.242,
+      "step": 471
+    },
+    {
+      "epoch": 0.8368794326241135,
+      "grad_norm": 0.7660516500473022,
+      "learning_rate": 0.00017602862254025042,
+      "loss": 0.2488,
+      "step": 472
+    },
+    {
+      "epoch": 0.8386524822695035,
+      "grad_norm": 0.3760780096054077,
+      "learning_rate": 0.0001757602862254025,
+      "loss": 0.1253,
+      "step": 473
+    },
+    {
+      "epoch": 0.8404255319148937,
+      "grad_norm": 0.5254774689674377,
+      "learning_rate": 0.00017549194991055454,
+      "loss": 0.2273,
+      "step": 474
+    },
+    {
+      "epoch": 0.8421985815602837,
+      "grad_norm": 0.34555211663246155,
+      "learning_rate": 0.0001752236135957066,
+      "loss": 0.1294,
+      "step": 475
+    },
+    {
+      "epoch": 0.8439716312056738,
+      "grad_norm": 0.4075317978858948,
+      "learning_rate": 0.00017495527728085867,
+      "loss": 0.1258,
+      "step": 476
+    },
+    {
+      "epoch": 0.8457446808510638,
+      "grad_norm": 0.4469994604587555,
+      "learning_rate": 0.00017468694096601072,
+      "loss": 0.155,
+      "step": 477
+    },
+    {
+      "epoch": 0.8475177304964538,
+      "grad_norm": 0.3752269744873047,
+      "learning_rate": 0.0001744186046511628,
+      "loss": 0.1403,
+      "step": 478
+    },
+    {
+      "epoch": 0.849290780141844,
+      "grad_norm": 0.3446385860443115,
+      "learning_rate": 0.00017415026833631482,
+      "loss": 0.0925,
+      "step": 479
+    },
+    {
+      "epoch": 0.851063829787234,
+      "grad_norm": 0.5445675253868103,
+      "learning_rate": 0.0001738819320214669,
+      "loss": 0.1791,
+      "step": 480
+    },
+    {
+      "epoch": 0.8528368794326241,
+      "grad_norm": 0.4845312535762787,
+      "learning_rate": 0.00017361359570661894,
+      "loss": 0.236,
+      "step": 481
+    },
+    {
+      "epoch": 0.8546099290780141,
+      "grad_norm": 0.21976758539676666,
+      "learning_rate": 0.00017334525939177102,
+      "loss": 0.0785,
+      "step": 482
+    },
+    {
+      "epoch": 0.8563829787234043,
+      "grad_norm": 0.3595951497554779,
+      "learning_rate": 0.00017307692307692304,
+      "loss": 0.1297,
+      "step": 483
+    },
+    {
+      "epoch": 0.8581560283687943,
+      "grad_norm": 0.34340062737464905,
+      "learning_rate": 0.00017280858676207511,
+      "loss": 0.1437,
+      "step": 484
+    },
+    {
+      "epoch": 0.8599290780141844,
+      "grad_norm": 0.40024513006210327,
+      "learning_rate": 0.0001725402504472272,
+      "loss": 0.111,
+      "step": 485
+    },
+    {
+      "epoch": 0.8617021276595744,
+      "grad_norm": 0.4997851252555847,
+      "learning_rate": 0.00017227191413237924,
+      "loss": 0.1406,
+      "step": 486
+    },
+    {
+      "epoch": 0.8634751773049646,
+      "grad_norm": 0.37670308351516724,
+      "learning_rate": 0.0001720035778175313,
+      "loss": 0.1653,
+      "step": 487
+    },
+    {
+      "epoch": 0.8652482269503546,
+      "grad_norm": 0.4352554380893707,
+      "learning_rate": 0.00017173524150268334,
+      "loss": 0.1275,
+      "step": 488
+    },
+    {
+      "epoch": 0.8670212765957447,
+      "grad_norm": 0.31643253564834595,
+      "learning_rate": 0.00017146690518783541,
+      "loss": 0.0982,
+      "step": 489
+    },
+    {
+      "epoch": 0.8687943262411347,
+      "grad_norm": 0.3775823712348938,
+      "learning_rate": 0.00017119856887298746,
+      "loss": 0.1087,
+      "step": 490
+    },
+    {
+      "epoch": 0.8705673758865248,
+      "grad_norm": 0.714429497718811,
+      "learning_rate": 0.0001709302325581395,
+      "loss": 0.1743,
+      "step": 491
+    },
+    {
+      "epoch": 0.8723404255319149,
+      "grad_norm": 0.24479807913303375,
+      "learning_rate": 0.00017066189624329156,
+      "loss": 0.0616,
+      "step": 492
+    },
+    {
+      "epoch": 0.874113475177305,
+      "grad_norm": 0.4242885708808899,
+      "learning_rate": 0.00017039355992844364,
+      "loss": 0.1266,
+      "step": 493
+    },
+    {
+      "epoch": 0.875886524822695,
+      "grad_norm": 0.4247332513332367,
+      "learning_rate": 0.00017012522361359571,
+      "loss": 0.1287,
+      "step": 494
+    },
+    {
+      "epoch": 0.8776595744680851,
+      "grad_norm": 0.49524983763694763,
+      "learning_rate": 0.00016985688729874774,
+      "loss": 0.1957,
+      "step": 495
+    },
+    {
+      "epoch": 0.8794326241134752,
+      "grad_norm": 0.2352031022310257,
+      "learning_rate": 0.0001695885509838998,
+      "loss": 0.0585,
+      "step": 496
+    },
+    {
+      "epoch": 0.8812056737588653,
+      "grad_norm": 0.23934312164783478,
+      "learning_rate": 0.00016932021466905186,
+      "loss": 0.0477,
+      "step": 497
+    },
+    {
+      "epoch": 0.8829787234042553,
+      "grad_norm": 0.3493366539478302,
+      "learning_rate": 0.00016905187835420394,
+      "loss": 0.1074,
+      "step": 498
+    },
+    {
+      "epoch": 0.8847517730496454,
+      "grad_norm": 0.28466084599494934,
+      "learning_rate": 0.00016878354203935596,
+      "loss": 0.0575,
+      "step": 499
+    },
+    {
+      "epoch": 0.8865248226950354,
+      "grad_norm": 0.33539536595344543,
+      "learning_rate": 0.00016851520572450803,
+      "loss": 0.0637,
+      "step": 500
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 1128,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.208442192384e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-500/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e535e282bc87e3ec98aee4af3db87dfefcc304b028e382b0a576fd7ab703a7ee
+size 5688

runs/Jan10_14-41-33_1ccba6f4c9ea/events.out.tfevents.1736520255.1ccba6f4c9ea.112.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c04841518896b9aa407b991b835721050994f9e28ab68814edea2d7ee5edaa5
+size 244702