Add HuggingFaceH4/mistral-7b-ift-v12.0 checkpoint

README.md

@@ -0,0 +1,63 @@
+---
+license: apache-2.0
+base_model: mistralai/Mistral-7B-v0.1
+tags:
+- generated_from_trainer
+model-index:
+- name: mistral-7b-ift-v12.0
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# mistral-7b-ift-v12.0
+
+This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the HuggingFaceH4/ultrachat_uncensored_truecase_i_dont and the HuggingFaceH4/arithmo datasets.
+It achieves the following results on the evaluation set:
+- Loss: 0.9133
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 16
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 16
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 512
+- total_eval_batch_size: 256
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 1
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 0.8586        | 0.38  | 344  | 0.9133          |
+
+
+### Framework versions
+
+- Transformers 4.35.0.dev0
+- Pytorch 2.0.1+cu118
+- Datasets 2.12.0
+- Tokenizers 0.14.0

added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "</s>": 2,
+  "<s>": 1,
+  "<unk>": 0
+}

all_results.json

@@ -0,0 +1,14 @@
+{
+    "epoch": 0.38,
+    "eval_loss": 0.9133002758026123,
+    "eval_runtime": 205.2179,
+    "eval_samples": 36829,
+    "eval_samples_per_second": 179.463,
+    "eval_steps_per_second": 0.702,
+    "perplexity": 2.4925350272208915,
+    "train_loss": 0.9639226873946745,
+    "train_runtime": 7750.2268,
+    "train_samples": 468526,
+    "train_samples_per_second": 60.453,
+    "train_steps_per_second": 0.118
+}

config.json

@@ -0,0 +1,25 @@
+{
+  "_name_or_path": "mistralai/Mistral-7B-v0.1",
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 10000.0,
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.35.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

dialogue_template.json

@@ -0,0 +1,30 @@
+{
+  "system": "",
+  "name": "default_v3",
+  "messages": [
+    {
+      "content": "Create a compelling crime story with a plot that revolves around a group of experienced thieves attempting to steal an antique car worth millions of dollars from a heavily guarded museum. Incorporate intricate heist planning, unexpected plot twists, and intriguing character development as the thieves race against time and face unexpected obstacles in their pursuit of the valuable vehicle. Additionally, explore themes related to greed, loyalty, and betrayal in the characters' relationships with each other and their motivations for committing the crime. Write in a suspenseful and tense style that keeps readers on the edge of their seats and builds to a satisfying climax.",
+      "role": "user"
+    },
+    {
+      "content": "It was a perfect plan.\n\nFour of the best thieves in the world had been planning for weeks to steal an antique car worth millions of dollars from a heavily guarded museum. The car was a one-of-a-kind, rare beauty that had never gone on public display before, and it was virtually priceless. The museum\u2019s security system was state-of-the-art, but they had a mole on the inside who was feeding them information about the layout of the building and the patrol schedules of the guards. They had even obtained a copy of the keycard to the room where the car was kept.\n\nThe four thieves \u2013 Jake, Alex, Mia, and Vince \u2013 were all veterans of hundreds of successful heists. They were cool, calm, and collected, and they had a reputation for being the best in the business. But this was different. This was the heist of a lifetime, and they knew that they had to be extra cautious.\n\nThey started by casing the museum for several days, watching the guards\u2019 patterns and looking for any vulnerabilities in the security system. They had a high-tech computer system that allowed them to simulate different scenarios and test their plan\u2019s viability. They even recruited a hacker to disable the museum\u2019s alarm system temporarily.\n\nThe night of the heist arrived, and they were ready. They wore black clothes and carried bags filled with tools and gadgets needed for the heist. They slipped into the museum, avoiding the motion sensors and cameras, and got to the room where the car was kept.\n\nThe plan was to disable the alarms and cameras, use the keycard to unlock the room's door, and remove the car before anyone realized what was happening.\n\nBut things did not go according to plan.\n\nAs soon as they started working on the door, they heard footsteps approaching. The mole had assured them that this wing of the museum was off-limits during the night, so they were not prepared for this. They quickly hid behind the exhibits as two guards walked right past them. They held their breath and waited for the guards to leave.\n\nThen, they heard the sound of the alarm on the hacker\u2019s computer. It was supposed to be a signal that the alarms were offline, but it had malfunctioned. The alarms were still on.\n\nPanic set in as the guards rushed into the room. They had been alerted by the hacker\u2019s mistake, and the thieves were trapped.\n\nBut the thieves were not ready to give up yet. They ran for the car, grabbing tools and breaking the glass case that housed it. The guards were in hot pursuit, and gunshots rang out. But they managed to get the car out of the museum, only to find that their getaway car was disabled.\n\nThey were trapped in the parking lot, surrounded by police cars and helicopters. Jake realized that the mole had double-crossed them and reported the plan to the police.\n\nIn the end, the thieves were arrested, and the car was returned to the museum. They all went to jail, but not before the thieves vowed revenge against the mole. The mole managed to escape, but not before Alex got a good look at his face.\n\nThe heist had been a failure, but the characters had grown and developed, discovering the extent of their own greed, loyalty, and betrayal. What had started as a simple heist had become an intense thrill ride, full of unexpected twists and turns. For better or for worse, everyone had gotten what they had wanted, and more.",
+      "role": "assistant"
+    },
+    {
+      "content": "That story is great! Can you add some more details about the characters' relationships with each other? I'm curious to know more about what drove them to work together and how they interacted throughout the heist.",
+      "role": "user"
+    },
+    {
+      "content": "Of course! Let me add some more details about the character relationships.\n\nJake was the leader of the group, and the one who had come up with the plan. He was an old hand at this sort of thing, but he had lost a lot of money after being double-crossed in a previous job. He was highly suspicious of everyone, including his own team members, and he was determined not to let another heist go wrong.\n\nAlex was Jake's right-hand man, and he had an almost obsessive respect for Jake. He had been working with Jake for years, and he saw him as a mentor and almost a father figure. At the same time, he had feelings for Mia, who was the newest addition to the team, and he struggled to keep them in check.\n\nMia was the wildcard of the group. She was young and impulsive, but very clever, and she had a talent for cracking safes and locks. She had joined the team because she admired Jake's reputation, but quickly found herself drawn to Alex. She tried not to let her feelings get in the way of the job, but it wasn't easy.\n\nVince was the muscle of the group, a tough guy with a long criminal record. He didn't talk much, but he was fiercely loyal to Jake and would do anything for him. He had been hired for the job specifically because of his physical strength.\n\nThroughout the heist, the relationships between the characters were tested. Jake's suspicions led him to become paranoid, and he kept a close eye on everyone. Alex's feelings for Mia caused him to make a few mistakes, and he ended up jeopardizing the mission. Mia, on the other hand, tried to stay focused on the job, but found herself getting more and more drawn to Alex. Vince was a silent but powerful presence, always ready to step in and defend his teammates.\n\nIn the end, the relationships between the characters were strained, but not broken. They all went to jail, but they had learned a lot about themselves and each other. They had discovered the extent of their own greed, loyalty, and betrayal, and they had seen firsthand the consequences of their actions. But they also knew that they could trust each other, even in the face of the toughest challenges.",
+      "role": "assistant"
+    }
+  ],
+  "system_token": "<|system|>",
+  "system_format": "standard",
+  "user_token": "<|user|>",
+  "assistant_token": "<|assistant|>",
+  "end_token": "</s>",
+  "mid_str": "\n",
+  "end_str": "\n",
+  "extra_end_text": "\n"
+}

eval_results.json

@@ -0,0 +1,9 @@
+{
+    "epoch": 0.38,
+    "eval_loss": 0.9133002758026123,
+    "eval_runtime": 205.2179,
+    "eval_samples": 36829,
+    "eval_samples_per_second": 179.463,
+    "eval_steps_per_second": 0.702,
+    "perplexity": 2.4925350272208915
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.35.0.dev0"
+}

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:010b7ada02bdb13054ddbed0c1d9fb0bdfdf0298ad2ec35908da7a15b7103c52
+size 9942981696

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:655f2c00f67c44357015a9073d8530aad61d30322718a1b5e662466795457b10
+size 4540516344

pytorch_model-00001-of-00002.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a0506a5a7f30fd03bfeb9e1bff27ccdc8e9294d2c96aecccce3c64153859e4fc
+size 9943028044

pytorch_model-00002-of-00002.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8336cf10e28ce475cab17502433da7452ef4cd7fbccba21940eaea830f2dec84
+size 4540535647

pytorch_model.bin.index.json

@@ -0,0 +1,298 @@
+{
+  "metadata": {
+    "total_size": 14483464192
+  },
+  "weight_map": {
+    "lm_head.weight": "pytorch_model-00002-of-00002.bin",
+    "model.embed_tokens.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.22.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.22.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.22.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.23.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.3.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.30.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.4.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.norm.weight": "pytorch_model-00002-of-00002.bin"
+  }
+}

runs/Oct20_13-49-22_ip-26-0-149-199/events.out.tfevents.1697810007.ip-26-0-149-199.2045305.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fcfeb28f4a08a63ff05a5bbe7faf72fdebc5f9dc7640028ddf9857bebec866c
+size 15846

runs/Oct20_13-49-22_ip-26-0-149-199/events.out.tfevents.1697817963.ip-26-0-149-199.2045305.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dec84c04136267ed08dc7bfbc73a6ee22c5268d8d5cc02c99d9c1bce885c915f
+size 359

special_tokens_map.json

@@ -0,0 +1,11 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "pad_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 0.38,
+    "train_loss": 0.9639226873946745,
+    "train_runtime": 7750.2268,
+    "train_samples": 468526,
+    "train_samples_per_second": 60.453,
+    "train_steps_per_second": 0.118
+}

trainer_state.json

@@ -0,0 +1,450 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.3758535919147774,
+  "eval_steps": 500,
+  "global_step": 344,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0,
+      "learning_rate": 2.173913043478261e-07,
+      "loss": 1.7313,
+      "step": 1
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.0869565217391306e-06,
+      "loss": 1.6804,
+      "step": 5
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 2.173913043478261e-06,
+      "loss": 1.3363,
+      "step": 10
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 3.2608695652173914e-06,
+      "loss": 1.1728,
+      "step": 15
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 4.347826086956522e-06,
+      "loss": 1.085,
+      "step": 20
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 5.4347826086956525e-06,
+      "loss": 1.0442,
+      "step": 25
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 6.521739130434783e-06,
+      "loss": 0.9988,
+      "step": 30
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 7.608695652173914e-06,
+      "loss": 0.9694,
+      "step": 35
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 8.695652173913044e-06,
+      "loss": 0.9524,
+      "step": 40
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 9.782608695652175e-06,
+      "loss": 0.9502,
+      "step": 45
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.0869565217391305e-05,
+      "loss": 0.9313,
+      "step": 50
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.1956521739130435e-05,
+      "loss": 0.9182,
+      "step": 55
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.3043478260869566e-05,
+      "loss": 0.9174,
+      "step": 60
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.4130434782608698e-05,
+      "loss": 0.9027,
+      "step": 65
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.5217391304347828e-05,
+      "loss": 0.9057,
+      "step": 70
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.630434782608696e-05,
+      "loss": 0.9031,
+      "step": 75
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.739130434782609e-05,
+      "loss": 0.9037,
+      "step": 80
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.847826086956522e-05,
+      "loss": 0.8971,
+      "step": 85
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.956521739130435e-05,
+      "loss": 0.8918,
+      "step": 90
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.999934429598561e-05,
+      "loss": 0.9005,
+      "step": 95
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.9995337527295925e-05,
+      "loss": 0.882,
+      "step": 100
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.9987689727712563e-05,
+      "loss": 0.8806,
+      "step": 105
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.997640368312189e-05,
+      "loss": 0.8911,
+      "step": 110
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.9961483504724445e-05,
+      "loss": 0.8899,
+      "step": 115
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.9942934627537337e-05,
+      "loss": 0.8818,
+      "step": 120
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.992076380841442e-05,
+      "loss": 1.566,
+      "step": 125
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.989497912358495e-05,
+      "loss": 1.7017,
+      "step": 130
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.9865589965711636e-05,
+      "loss": 1.2844,
+      "step": 135
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.9832607040469147e-05,
+      "loss": 1.1296,
+      "step": 140
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9796042362644315e-05,
+      "loss": 1.0674,
+      "step": 145
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9755909251759493e-05,
+      "loss": 1.0187,
+      "step": 150
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.97122223272206e-05,
+      "loss": 0.9856,
+      "step": 155
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.9664997502991665e-05,
+      "loss": 0.9741,
+      "step": 160
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.961425198179781e-05,
+      "loss": 0.9602,
+      "step": 165
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9560004248858754e-05,
+      "loss": 0.9388,
+      "step": 170
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.950227406515516e-05,
+      "loss": 0.9239,
+      "step": 175
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9441082460230226e-05,
+      "loss": 0.9376,
+      "step": 180
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9376451724529207e-05,
+      "loss": 0.9257,
+      "step": 185
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.930840540127961e-05,
+      "loss": 0.927,
+      "step": 190
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.923696827791502e-05,
+      "loss": 0.9064,
+      "step": 195
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9162166377045723e-05,
+      "loss": 0.9088,
+      "step": 200
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9084026946979366e-05,
+      "loss": 0.9014,
+      "step": 205
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9002578451795133e-05,
+      "loss": 0.9092,
+      "step": 210
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.8917850560975064e-05,
+      "loss": 0.9081,
+      "step": 215
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.882987413859625e-05,
+      "loss": 0.9027,
+      "step": 220
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.8738681232087897e-05,
+      "loss": 0.8962,
+      "step": 225
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.8644305060557317e-05,
+      "loss": 0.8823,
+      "step": 230
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.8546780002689088e-05,
+      "loss": 0.892,
+      "step": 235
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.8446141584221854e-05,
+      "loss": 0.8887,
+      "step": 240
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.834242646500724e-05,
+      "loss": 0.8843,
+      "step": 245
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.8235672425655678e-05,
+      "loss": 0.9014,
+      "step": 250
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.8125918353773934e-05,
+      "loss": 0.8911,
+      "step": 255
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.8013204229799422e-05,
+      "loss": 0.881,
+      "step": 260
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.7897571112436404e-05,
+      "loss": 0.8879,
+      "step": 265
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.777906112369942e-05,
+      "loss": 0.8922,
+      "step": 270
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.7657717433569384e-05,
+      "loss": 0.8747,
+      "step": 275
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.7533584244267897e-05,
+      "loss": 0.8715,
+      "step": 280
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.7406706774155625e-05,
+      "loss": 0.8812,
+      "step": 285
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.7277131241260438e-05,
+      "loss": 0.8669,
+      "step": 290
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.7144904846441434e-05,
+      "loss": 0.864,
+      "step": 295
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.7010075756194962e-05,
+      "loss": 0.8683,
+      "step": 300
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.6872693085108864e-05,
+      "loss": 0.867,
+      "step": 305
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.673280687797135e-05,
+      "loss": 0.8724,
+      "step": 310
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.659046809154105e-05,
+      "loss": 0.8802,
+      "step": 315
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.6445728575984838e-05,
+      "loss": 0.8657,
+      "step": 320
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.6298641055990222e-05,
+      "loss": 0.8602,
+      "step": 325
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.614925911155917e-05,
+      "loss": 0.8641,
+      "step": 330
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.5997637158490366e-05,
+      "loss": 0.8606,
+      "step": 335
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.5843830428557e-05,
+      "loss": 0.8586,
+      "step": 340
+    },
+    {
+      "epoch": 0.38,
+      "eval_loss": 0.9133486151695251,
+      "eval_runtime": 211.9502,
+      "eval_samples_per_second": 173.763,
+      "eval_steps_per_second": 0.679,
+      "step": 344
+    },
+    {
+      "epoch": 0.38,
+      "step": 344,
+      "total_flos": 576369847173120.0,
+      "train_loss": 0.9639226873946745,
+      "train_runtime": 7750.2268,
+      "train_samples_per_second": 60.453,
+      "train_steps_per_second": 0.118
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 915,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "total_flos": 576369847173120.0,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e8b7815ffad6613670bdf92182f49d712b20757f78628c5361f042f2c1cc561
+size 5435