Update README.md
Browse files
README.md
CHANGED
@@ -5,7 +5,7 @@ language:
|
|
5 |
inference: false
|
6 |
---
|
7 |
# Model Card for TinyMixtral-x8-Clonebase-7b
|
8 |
-
This model is based on [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-
|
9 |
**This model was created experimentally for training a small mixtral.**
|
10 |
**Without Train, the performance of this model is the same as TinyLlama.**
|
11 |
|
@@ -22,6 +22,10 @@ This model was created with experts=8, but since it is a clone, you can create a
|
|
22 |
|
23 |
[tinyllama_to_mixtral_clonebase.ipynb](https://huggingface.co/mmnga/TinyMixtral-x8-Clonebase-7b/blob/main/notebook/tinyllama_to_mixtral_clonebase.ipynb)
|
24 |
|
|
|
|
|
|
|
|
|
25 |
# Usage
|
26 |
~~~python
|
27 |
pip install transformers --upgrade
|
|
|
5 |
inference: false
|
6 |
---
|
7 |
# Model Card for TinyMixtral-x8-Clonebase-7b
|
8 |
+
This model is based on [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T), converted to a mistral model, and then placed the clone in mixtral.
|
9 |
**This model was created experimentally for training a small mixtral.**
|
10 |
**Without Train, the performance of this model is the same as TinyLlama.**
|
11 |
|
|
|
22 |
|
23 |
[tinyllama_to_mixtral_clonebase.ipynb](https://huggingface.co/mmnga/TinyMixtral-x8-Clonebase-7b/blob/main/notebook/tinyllama_to_mixtral_clonebase.ipynb)
|
24 |
|
25 |
+
# History revision
|
26 |
+
[main TinyLlama-1.1B-intermediate-step-1431k-3T](https://huggingface.co/mmnga/TinyMixtral-x8-Clonebase-7b)
|
27 |
+
[old TinyLlama-1.1B-intermediate-step-1195k-token-2.5T](https://huggingface.co/mmnga/TinyMixtral-x8-Clonebase-7b/tree/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T)
|
28 |
+
|
29 |
# Usage
|
30 |
~~~python
|
31 |
pip install transformers --upgrade
|