Update README

README.md

@@ -1,6 +1,6 @@
 ---
 library_name: transformers
-license: other
+license: apache-2.0
 base_model: Qwen/Qwen2.5-7B-Instruct
 tags:
 - llama-factory
@@ -9,28 +9,32 @@ tags:
 model-index:
 - name: original
   results: []
+language:
+- en
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
-# original
-
-This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the Stratos-R1 dataset.
 
 ## Model description
+This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the [Stratos-R1 dataset](https://huggingface.co/datasets/bespokelabs/stratos-r1).
+The dataset is derived by distilling DeepSeek-R1 using the data pipeline of Berkeley NovaSky’s Sky-T1 with some modifications. More info in the dataset card at [Stratos-R1 dataset](https://huggingface.co/datasets/bespokelabs/stratos-r1).
+It outperforms Qwen-2.5-7B-Instruct on reasoning benchmarks:
 
-More information needed
+||Bespoke-Stratos-7B|DeepSeek-R1-Distill-Qwen-7B|Qwen2.5-7B-Instruct|
+|---|---|---|---|
+|AIME2024|20.0|55.5|10.0|
+|MATH500|82.0|83.3|74.2|
+|GPQA-Diamond|37.8|49.1|33.3|
+|LiveCodeBench|-|37.6|32.9|
 
-## Intended uses & limitations
+Note that the authors of Sky-T1 had [noted](https://github.com/NovaSky-AI/SkyThought/issues/4#issuecomment-2585860004) that they saw little or no improvement in training 7B or 14B models with their data.
+However, see an improvement, though not at the scale of DeepSeek's distilled model. The reason could be that we used 17k examples, while DeepSeek seems to have used 800k.
 
-More information needed
-
-## Training and evaluation data
+## Intended uses & limitations
 
-More information needed
+Non-commercial use.
 
 ## Training procedure
+We used 8xH100 to train the model.
 
 ### Training hyperparameters
 
@@ -58,4 +62,4 @@ The following hyperparameters were used during training:
 - Transformers 4.46.1
 - Pytorch 2.5.1+cu124
 - Datasets 3.1.0
-- Tokenizers 0.20.3
+- Tokenizers 0.20.3