Update README.md
Browse files
README.md
CHANGED
|
@@ -139,6 +139,32 @@ Here are the evaluation results for DCLM-Baseline-7B on various tasks (using [ll
|
|
| 139 |
Note: All scores are presented as decimal values between 0 and 1, representing the proportion of correct answers or the model's performance on each task.
|
| 140 |
|
| 141 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 142 |
## Limitations and Biases
|
| 143 |
|
| 144 |
While DCLM-Baseline-7B demonstrates strong performance across a range of tasks, it's important to note:
|
|
|
|
| 139 |
Note: All scores are presented as decimal values between 0 and 1, representing the proportion of correct answers or the model's performance on each task.
|
| 140 |
|
| 141 |
|
| 142 |
+
## Comparison
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
Below are comparisions of this model with other models in the 7B regime.
|
| 146 |
+
|
| 147 |
+
| Model | Params | Tokens | Open dataset? | CORE | MMLU | EXTENDED |
|
| 148 |
+
|---------------|--------|--------|---------------|----------|----------|----------|
|
| 149 |
+
| **Open weights, closed datasets** | | | | | | |
|
| 150 |
+
| Llama2 | 7B | 2T | ✗ | 49.2 | 45.8 | 34.1 |
|
| 151 |
+
| DeepSeek | 7B | 2T | ✗ | 50.7 | 48.5 | 35.3 |
|
| 152 |
+
| Mistral-0.3 | 7B | ? | ✗ | 57.0 | 62.7 | 45.1 |
|
| 153 |
+
| QWEN-2 | 7B | ? | ✗ | 57.5 | **71.9** | 50.5 |
|
| 154 |
+
| Llama3 | 8B | 15T | ✗ | 57.6 | 66.2 | 46.3 |
|
| 155 |
+
| Gemma | 8B | 6T | ✗ | 57.8 | 64.3 | 44.6 |
|
| 156 |
+
| Phi-3 | 7B | ? | ✗ | **61.0** | 69.9 | **57.9** |
|
| 157 |
+
| **Open weights, open datasets** | | | | | | |
|
| 158 |
+
| Falcon | 7B | 1T | ✓ | 44.1 | 27.4 | 25.1 |
|
| 159 |
+
| OLMo-1.7 | 7B | 2.1T | ✓ | 47.0 | 54.0 | 34.2 |
|
| 160 |
+
| MAP-Neo | 7B | 4.5T | ✓ | **50.2** | **57.1** | **40.4** |
|
| 161 |
+
| **Models we trained** | | | | | | |
|
| 162 |
+
| FineWeb edu | 7B | 0.14T | ✓ | 38.7 | 26.3 | 22.1 |
|
| 163 |
+
| FineWeb edu | 7B | 0.28T | ✓ | 41.9 | 37.3 | 24.5 |
|
| 164 |
+
| **DCLM-7B** | 7B | 2.5T | ✓ | **56.1** | **63.7** | **43.6** |
|
| 165 |
+
|
| 166 |
+
|
| 167 |
+
|
| 168 |
## Limitations and Biases
|
| 169 |
|
| 170 |
While DCLM-Baseline-7B demonstrates strong performance across a range of tasks, it's important to note:
|