Commit
•
c343b22
1
Parent(s):
0952be6
Update README.md
Browse files
README.md
CHANGED
|
@@ -24,6 +24,41 @@ Original model: https://huggingface.co/google/gemma-2-2b-it
|
|
| 24 |
|
| 25 |
All quants were made using the imatrix option and Bartowski's [calibration file](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
|
| 26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
<hr><br>
|
| 28 |
|
| 29 |
# Gemma 2 model card
|
|
|
|
| 24 |
|
| 25 |
All quants were made using the imatrix option and Bartowski's [calibration file](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
|
| 26 |
|
| 27 |
+
<hr>
|
| 28 |
+
|
| 29 |
+
# Perplexity table (the lower the better)
|
| 30 |
+
|
| 31 |
+
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
|
| 32 |
+
| ------- | --------- | ------- | -------- | ------------ | -------------- |
|
| 33 |
+
| IQ1_S | 794 | 42.0753 | 15.9 | 30.61 | 0.36973 |
|
| 34 |
+
| IQ1_M | 834 | 31.0114 | 16.7 | 41.53 | 0.26186 |
|
| 35 |
+
| IQ2_XXS | 900 | 21.9391 | 18.03 | 58.7 | 0.18008 |
|
| 36 |
+
| IQ2_XS | 957 | 18.7303 | 19.17 | 68.76 | 0.15338 |
|
| 37 |
+
| IQ2_S | 985 | 17.1131 | 19.73 | 75.26 | 0.13806 |
|
| 38 |
+
| IQ2_M | 1038 | 15.817 | 20.79 | 81.43 | 0.12767 |
|
| 39 |
+
| Q2_K_S | 1116 | 17.514 | 22.35 | 73.54 | 0.1438 |
|
| 40 |
+
| IQ3_XXS | 1127 | 14.3815 | 22.57 | 89.55 | 0.11367 |
|
| 41 |
+
| Q2_K | 1173 | 15.8684 | 23.49 | 81.16 | 0.12815 |
|
| 42 |
+
| IQ3_XS | 1254 | 13.8252 | 25.12 | 93.16 | 0.10962 |
|
| 43 |
+
| IQ3_S | 1298 | 13.555 | 26 | 95.01 | 0.10766 |
|
| 44 |
+
| Q3_K_S | 1298 | 14.4857 | 26 | 88.91 | 0.1172 |
|
| 45 |
+
| IQ3_M | 1330 | 13.2669 | 26.64 | 97.08 | 0.10501 |
|
| 46 |
+
| Q3_K_M | 1394 | 13.4964 | 27.92 | 95.43 | 0.10782 |
|
| 47 |
+
| Q3_K_L | 1479 | 13.5088 | 29.62 | 95.34 | 0.10803 |
|
| 48 |
+
| IQ4_XS | 1494 | 13.1866 | 29.92 | 97.67 | 0.10453 |
|
| 49 |
+
| IQ4_NL | 1555 | 13.1306 | 31.14 | 98.09 | 0.10405 |
|
| 50 |
+
| Q4_0 | 1558 | 13.0812 | 31.2 | 98.46 | 0.10343 |
|
| 51 |
+
| Q4_K_S | 1563 | 12.9706 | 31.3 | 99.3 | 0.10274 |
|
| 52 |
+
| Q4_K_M | 1630 | 13.0641 | 32.65 | 98.58 | 0.10396 |
|
| 53 |
+
| Q4_1 | 1675 | 12.9664 | 33.55 | 99.33 | 0.1028 |
|
| 54 |
+
| Q5_K_S | 1796 | 12.8929 | 35.97 | 99.89 | 0.10224 |
|
| 55 |
+
| Q5_0 | 1800 | 13.0323 | 36.05 | 98.83 | 0.10396 |
|
| 56 |
+
| Q5_K_M | 1835 | 12.9416 | 36.75 | 99.52 | 0.10275 |
|
| 57 |
+
| Q5_1 | 1916 | 12.9191 | 38.37 | 99.69 | 0.10251 |
|
| 58 |
+
| Q6_K | 2052 | 12.8816 | 41.1 | 99.98 | 0.10238 |
|
| 59 |
+
| Q8_0 | 2656 | 12.888 | 53.19 | 99.93 | 0.10242 |
|
| 60 |
+
| F16 | 4993 | 12.8792 | 100 | 100 | 0.10232 |
|
| 61 |
+
|
| 62 |
<hr><br>
|
| 63 |
|
| 64 |
# Gemma 2 model card
|