| Llama-3.2-1B-Instruct | |
| Quant Size (MB) PPL Size (%) Accuracy (%) PPL error rate | |
| IQ1_S 376 771.8958 14.99148 | |
| IQ1_M 395 162.0038 2.86547 | |
| IQ2_XXS 427 46.0426 0.77657 | |
| IQ2_XS 454 30.7626 0.50736 | |
| IQ2_S 467 25.4944 0.41940 | |
| IQ2_M 492 21.1112 0.34245 | |
| Q2_K_S 529 24.5117 0.40072 | |
| IQ3_XXS 537 17.2479 0.27837 | |
| Q2_K 554 26.1688 0.44789 | |
| IQ3_XS 593 16.0104 0.25685 | |
| Q3_K_S 612 19.1038 0.31660 | |
| IQ3_S 615 15.6453 0.24806 | |
| IQ3_M 627 15.4512 0.24445 | |
| Q3_K_M 659 14.9000 0.23958 | |
| Q3_K_L 699 14.7286 0.23679 | |
| IQ4_XS 709 14.1783 0.22704 | |
| IQ4_NL 738 14.1777 0.22727 | |
| Q4_0 738 14.4071 0.23021 | |
| Q4_K_S 740 14.0726 0.22511 | |
| Q4_K_M 771 14.0496 0.22523 | |
| Q4_1 794 14.1039 0.22552 | |
| Q5_K_S 852 13.8515 0.22187 | |
| Q5_0 854 13.8766 0.22210 | |
| Q5_K_M 870 13.8295 0.22162 | |
| Q5_1 910 13.7981 0.22042 | |
| Q6_K 975 13.7604 0.22054 | |
| Q8_0 1260 13.7166 0.21964 | |
| F16 2365 13.7126 0.21966 | |