Update README.md

README.md

@@ -6,7 +6,13 @@ tags: []
 # Model Card for Model ID
 
 <!-- Provide a quick summary of what the model is/does. -->
-This is [Mixtral 8x7b](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) model fine-tuned using 10B Speakleash tokens.
+XXX is a [Mixtral 8x7b](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) model fine-tuned using 2.2B Polish 
+tokens selected from the [SpeakLeash](https://speakleash.org/). 
+This is, to our knowledge, the first open-weights MoE model fine-tuned on Polish data. 
+In order to preserve English capabilities, we include about 600M tokens from the [RedPajama dataset](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T).
+
+The training was made possible thanks to TPU Research Cloud program. The model was trained on a TPUv3-256.
+
 
 
 ## Model Details