Commit
•
b9301db
1
Parent(s):
d87465d
Update README.md
Browse files
README.md
CHANGED
|
@@ -198,6 +198,41 @@ Original model: [https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct](htt
|
|
| 198 |
|
| 199 |
All quants were made using the imatrix option and Bartowski's [calibration file](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
|
| 200 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 201 |
<hr><br>
|
| 202 |
|
| 203 |
## Model Details
|
|
|
|
| 198 |
|
| 199 |
All quants were made using the imatrix option and Bartowski's [calibration file](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
|
| 200 |
|
| 201 |
+
<hr>
|
| 202 |
+
|
| 203 |
+
# Perplexity table (the lower the better)
|
| 204 |
+
|
| 205 |
+
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
|
| 206 |
+
| ------- | --------- | ------- | -------- | ------------ | -------------- |
|
| 207 |
+
| IQ1_S | 1927 | 60.6812 | 12.57 | 13.79 | 0.45829 |
|
| 208 |
+
| IQ1_M | 2062 | 29.1481 | 13.46 | 28.7 | 0.21385 |
|
| 209 |
+
| IQ2_XXS | 2289 | 16.2991 | 14.94 | 51.32 | 0.12078 |
|
| 210 |
+
| IQ2_XS | 2486 | 12.5744 | 16.22 | 66.52 | 0.09212 |
|
| 211 |
+
| IQ2_S | 2631 | 11.5222 | 17.17 | 72.6 | 0.08563 |
|
| 212 |
+
| IQ2_M | 2812 | 10.3045 | 18.35 | 81.18 | 0.07545 |
|
| 213 |
+
| Q2_K_S | 2851 | 11.3784 | 18.6 | 73.52 | 0.0842 |
|
| 214 |
+
| Q2_K | 3032 | 10.572 | 19.78 | 79.12 | 0.07761 |
|
| 215 |
+
| IQ3_XXS | 3124 | 9.4049 | 20.38 | 88.94 | 0.06908 |
|
| 216 |
+
| IQ3_XS | 3356 | 9.017 | 21.9 | 92.77 | 0.06548 |
|
| 217 |
+
| Q3_K_S | 3495 | 9.4611 | 22.81 | 88.41 | 0.06768 |
|
| 218 |
+
| IQ3_S | 3512 | 8.944 | 22.92 | 93.53 | 0.06596 |
|
| 219 |
+
| IQ3_M | 3610 | 8.8635 | 23.56 | 94.38 | 0.06505 |
|
| 220 |
+
| Q3_K_M | 3833 | 8.8718 | 25.01 | 94.29 | 0.0665 |
|
| 221 |
+
| Q3_K_L | 4122 | 8.8139 | 26.9 | 94.91 | 0.06611 |
|
| 222 |
+
| IQ4_XS | 4242 | 8.5606 | 27.68 | 97.72 | 0.06361 |
|
| 223 |
+
| Q4_0 | 4460 | 8.508 | 29.1 | 98.32 | 0.06214 |
|
| 224 |
+
| IQ4_NL | 4462 | 8.5774 | 29.12 | 97.52 | 0.06391 |
|
| 225 |
+
| Q4_K_S | 4476 | 8.5273 | 29.21 | 98.1 | 0.06333 |
|
| 226 |
+
| Q4_K_M | 4693 | 8.5215 | 30.62 | 98.16 | 0.06327 |
|
| 227 |
+
| Q4_1 | 4893 | 8.5255 | 31.93 | 98.12 | 0.06337 |
|
| 228 |
+
| Q5_K_S | 5340 | 8.4169 | 34.85 | 99.38 | 0.06261 |
|
| 229 |
+
| Q5_0 | 5354 | 8.437 | 34.94 | 99.15 | 0.06281 |
|
| 230 |
+
| Q5_K_M | 5468 | 8.4087 | 35.68 | 99.48 | 0.06245 |
|
| 231 |
+
| Q5_1 | 5788 | 8.3893 | 37.77 | 99.71 | 0.06229 |
|
| 232 |
+
| Q6_K | 6291 | 8.3894 | 41.05 | 99.71 | 0.06231 |
|
| 233 |
+
| Q8_0 | 8146 | 8.3634 | 53.15 | 100.02 | 0.0621 |
|
| 234 |
+
| F16 | 15325 | 8.365 | 100 | 100 | 0.06216 |
|
| 235 |
+
|
| 236 |
<hr><br>
|
| 237 |
|
| 238 |
## Model Details
|