Commit
•
a80d477
1
Parent(s):
a3ff4b1
Update README.md
Browse files
README.md
CHANGED
|
@@ -25,6 +25,37 @@ All quants were made using the imatrix option and Bartowski's [calibration file]
|
|
| 25 |
|
| 26 |
# Perplexity table (the lower the better)
|
| 27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
<hr>
|
| 29 |
|
| 30 |
# Qwen2.5-7B-Instruct
|
|
|
|
| 25 |
|
| 26 |
# Perplexity table (the lower the better)
|
| 27 |
|
| 28 |
+
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
|
| 29 |
+
| ------- | --------- | ------- | -------- | ------------ | -------------- |
|
| 30 |
+
| IQ1_S | 1816 | 29.2065 | 12.5 | 27.04 | 0.23022 |
|
| 31 |
+
| IQ1_M | 1948 | 17.5876 | 13.4 | 44.9 | 0.13256 |
|
| 32 |
+
| IQ2_XXS | 2168 | 13.0997 | 14.92 | 60.28 | 0.10138 |
|
| 33 |
+
| IQ2_XS | 2355 | 10.6611 | 16.21 | 74.06 | 0.07823 |
|
| 34 |
+
| IQ2_S | 2476 | 10.236 | 17.04 | 77.14 | 0.07461 |
|
| 35 |
+
| IQ2_M | 2652 | 9.5241 | 18.25 | 82.91 | 0.06876 |
|
| 36 |
+
| Q2_K_S | 2703 | 9.7181 | 18.6 | 81.25 | 0.07113 |
|
| 37 |
+
| Q2_K | 2877 | 9.271 | 19.8 | 85.17 | 0.06672 |
|
| 38 |
+
| IQ3_XXS | 2971 | 8.677 | 20.44 | 91 | 0.06191 |
|
| 39 |
+
| IQ3_XS | 3192 | 8.468 | 21.97 | 93.25 | 0.05972 |
|
| 40 |
+
| Q3_K_S | 3331 | 8.6524 | 22.92 | 91.26 | 0.06144 |
|
| 41 |
+
| IQ3_S | 3338 | 8.4723 | 22.97 | 93.2 | 0.06045 |
|
| 42 |
+
| IQ3_M | 3409 | 8.4488 | 23.46 | 93.46 | 0.05973 |
|
| 43 |
+
| Q3_K_M | 3632 | 8.2232 | 24.99 | 96.02 | 0.05882 |
|
| 44 |
+
| Q3_K_L | 3900 | 8.1675 | 26.84 | 96.68 | 0.05826 |
|
| 45 |
+
| IQ4_XS | 4024 | 8.0803 | 27.69 | 97.72 | 0.0579 |
|
| 46 |
+
| IQ4_NL | 4233 | 8.0301 | 29.13 | 98.33 | 0.05724 |
|
| 47 |
+
| Q4_0 | 4239 | 7.9852 | 29.17 | 98.88 | 0.05586 |
|
| 48 |
+
| Q4_K_S | 4252 | 8.0483 | 29.26 | 98.11 | 0.05752 |
|
| 49 |
+
| Q4_K_M | 4467 | 7.997 | 30.74 | 98.74 | 0.05697 |
|
| 50 |
+
| Q4_1 | 4648 | 8.1016 | 31.98 | 97.46 | 0.05824 |
|
| 51 |
+
| Q5_K_S | 5069 | 7.9466 | 34.88 | 99.36 | 0.05652 |
|
| 52 |
+
| Q5_0 | 5082 | 7.9939 | 34.97 | 98.78 | 0.05726 |
|
| 53 |
+
| Q5_K_M | 5193 | 7.9279 | 35.73 | 99.6 | 0.05633 |
|
| 54 |
+
| Q5_1 | 5491 | 7.9334 | 37.79 | 99.53 | 0.05634 |
|
| 55 |
+
| Q6_K | 5965 | 7.9066 | 41.05 | 99.87 | 0.05617 |
|
| 56 |
+
| Q8_0 | 7724 | 7.8996 | 53.15 | 99.96 | 0.05614 |
|
| 57 |
+
| F16 | 14532 | 7.8961 | 100 | 100 | 0.05608 |
|
| 58 |
+
|
| 59 |
<hr>
|
| 60 |
|
| 61 |
# Qwen2.5-7B-Instruct
|