ryanmarten
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -26,15 +26,23 @@ The dataset is derived by distilling DeepSeek-R1 using the [data pipeline availa
|
|
26 |
More info about the dataset can be found on the dataset card at [OpenThoughts-114k dataset](https://huggingface.co/datasets/open-thoughts/open-thoughts-114k).
|
27 |
|
28 |
This model improves upon the [Bespoke-Stratos-7B model](https://huggingface.co/bespokelabs/Bespoke-Stratos-7B), which used 17k examples ([Bespoke-Stratos-17k dataset](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k)).
|
29 |
-
|
30 |
-
|
31 |
-
|
|
32 |
-
|
33 |
-
|OpenThinker-7B|
|
34 |
-
|Bespoke-Stratos-7B|
|
35 |
-
|DeepSeek-R1-Distill-Qwen-7B|
|
36 |
-
|gpt-4o-0513|
|
37 |
-
|o1-mini|63.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
38 |
|
39 |
|
40 |
## Intended uses & limitations
|
|
|
26 |
More info about the dataset can be found on the dataset card at [OpenThoughts-114k dataset](https://huggingface.co/datasets/open-thoughts/open-thoughts-114k).
|
27 |
|
28 |
This model improves upon the [Bespoke-Stratos-7B model](https://huggingface.co/bespokelabs/Bespoke-Stratos-7B), which used 17k examples ([Bespoke-Stratos-17k dataset](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k)).
|
29 |
+
The numbers reported in the table below are independently evaluated with our open-source tool [Evalchemy](https://github.com/mlfoundations/Evalchemy).
|
30 |
+
|
31 |
+
| | AIME2024 | MATH500 | GPQA-Diamond | LCB Easy v2 | LCB Medium v2 | LCB Hard v2 | LCB All v2 |
|
32 |
+
| --------------------------- | -------- | ------- | ------------ | ----------- | ------------- | ----------- | ---------- |
|
33 |
+
| OpenThinker-7B | 43.3 | 83.0 | 42.4 | 75.3 | 28.6 | 6.5 | 39.9 |
|
34 |
+
| Bespoke-Stratos-7B | 16.6 | 79.6 | 38.9 | 71.4 | 25.2 | 0.8 | 35.8 |
|
35 |
+
| DeepSeek-R1-Distill-Qwen-7B | 60 | 88.2 | 46.9 | 79.7 | 45.1 | 14.6 | 50.1 |
|
36 |
+
| gpt-4o-0513 | 10 | 75.8 | 46.5 | 87.4 | 42.7 | 8.9 | 50.5 |
|
37 |
+
| o1-mini | 63 | | 60 | 92.8 | 74.7 | 39.8 | 72.8 |
|
38 |
+
|
39 |
+
| | Open Weights | Open Data | Open Code |
|
40 |
+
|--|--------------|-----------| --------- |
|
41 |
+
|OpenThinker-7B|β|[β](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k)|[β](https://github.com/open-thoughts/open-thoughts) |
|
42 |
+
|Bespoke-Stratos-7B|β|[β](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k)|[β](https://github.com/bespokelabsai/curator/tree/main/examples/bespoke-stratos-data-generation)|
|
43 |
+
|DeepSeek-R1-Distill-Qwen-7B|β|β|β|
|
44 |
+
|gpt-4o-0513|-|β|β|β|
|
45 |
+
|o1-mini|-|β|β|β|
|
46 |
|
47 |
|
48 |
## Intended uses & limitations
|