mmnga commited on
Commit
e11b5cc
1 Parent(s): ce6d02e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -5,7 +5,7 @@ language:
5
  inference: false
6
  ---
7
  # Model Card for TinyMixtral-x8-Clonebase-7b
8
- This model is based on [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T), converted to a mistral model, and then placed the clone in mixtral.
9
  **This model was created experimentally for training a small mixtral.**
10
  **Without Train, the performance of this model is the same as TinyLlama.**
11
 
@@ -22,6 +22,10 @@ This model was created with experts=8, but since it is a clone, you can create a
22
 
23
  [tinyllama_to_mixtral_clonebase.ipynb](https://huggingface.co/mmnga/TinyMixtral-x8-Clonebase-7b/blob/main/notebook/tinyllama_to_mixtral_clonebase.ipynb)
24
 
 
 
 
 
25
  # Usage
26
  ~~~python
27
  pip install transformers --upgrade
 
5
  inference: false
6
  ---
7
  # Model Card for TinyMixtral-x8-Clonebase-7b
8
+ This model is based on [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T), converted to a mistral model, and then placed the clone in mixtral.
9
  **This model was created experimentally for training a small mixtral.**
10
  **Without Train, the performance of this model is the same as TinyLlama.**
11
 
 
22
 
23
  [tinyllama_to_mixtral_clonebase.ipynb](https://huggingface.co/mmnga/TinyMixtral-x8-Clonebase-7b/blob/main/notebook/tinyllama_to_mixtral_clonebase.ipynb)
24
 
25
+ # History revision
26
+ [main TinyLlama-1.1B-intermediate-step-1431k-3T](https://huggingface.co/mmnga/TinyMixtral-x8-Clonebase-7b)
27
+ [old TinyLlama-1.1B-intermediate-step-1195k-token-2.5T](https://huggingface.co/mmnga/TinyMixtral-x8-Clonebase-7b/tree/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T)
28
+
29
  # Usage
30
  ~~~python
31
  pip install transformers --upgrade