Update README.md
Browse files
README.md
CHANGED
@@ -5,11 +5,11 @@ model-index:
|
|
5 |
- name: out
|
6 |
results: []
|
7 |
---
|
|
|
|
|
|
|
8 |
|
9 |
-
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
10 |
-
should probably proofread and complete it, then remove this comment. -->
|
11 |
|
12 |
-
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
13 |
<details><summary>See axolotl config</summary>
|
14 |
|
15 |
axolotl version: `0.3.0`
|
@@ -89,25 +89,12 @@ special_tokens:
|
|
89 |
|
90 |
</details><br>
|
91 |
|
92 |
-
# out
|
93 |
-
|
94 |
-
This model was trained from scratch on the None dataset.
|
95 |
-
It achieves the following results on the evaluation set:
|
96 |
-
- Loss: 1.3647
|
97 |
-
|
98 |
-
## Model description
|
99 |
|
100 |
-
More information needed
|
101 |
|
102 |
-
|
103 |
|
104 |
-
|
105 |
-
|
106 |
-
## Training and evaluation data
|
107 |
-
|
108 |
-
More information needed
|
109 |
-
|
110 |
-
## Training procedure
|
111 |
|
112 |
### Training hyperparameters
|
113 |
|
|
|
5 |
- name: out
|
6 |
results: []
|
7 |
---
|
8 |
+
This is the Instruction Fine Tuned version of [Tiny Llama](https://github.com/jzhang38/TinyLlama) on [@Teknium1's](https://twitter.com/Teknium1) [openhermes](https://huggingface.co/datasets/teknium/openhermes) dataset.
|
9 |
+
|
10 |
+
`"The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01."`
|
11 |
|
|
|
|
|
12 |
|
|
|
13 |
<details><summary>See axolotl config</summary>
|
14 |
|
15 |
axolotl version: `0.3.0`
|
|
|
89 |
|
90 |
</details><br>
|
91 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
92 |
|
|
|
93 |
|
94 |
+

|
95 |
|
96 |
+
The model achieves the following loss:
|
97 |
+
- Loss: 1.3647
|
|
|
|
|
|
|
|
|
|
|
98 |
|
99 |
### Training hyperparameters
|
100 |
|