Create README.md

README.md

@@ -0,0 +1,37 @@
+---
+tags:
+- gguf
+quantized_by: juvi21
+base_model: migtissera/Tess-3-7B-SFT
+---
+
+## Tess-3-7B-SFT-GGUF Quants
+
+I used <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3166">b3166</a> for quantization.
+For the FLOP poor I made various quants. Visit the OG model card for more details: https://huggingface.co/migtissera/Tess-3-7B-SFT
+
+## Quants
+
+| Filename | Quant type | File Size |
+|----------|------------|-----------|
+| Tess-3-7B-SFT-f16.gguf | F16 | 14.5 GB |
+| Tess-3-7B-SFT-gguf-q8_0.gguf | Q8_0 | 7.7 GB |
+| Tess-3-7B-SFT-gguf-q6_k.gguf | Q6_K | 5.95 GB |
+| Tess-3-7B-SFT-gguf-q5_k_s.gguf | Q5_K_S | 5 GB |
+| Tess-3-7B-SFT-gguf-q4_k_m.gguf | Q4_K_M | 4.37 GB |
+| Tess-3-7B-SFT-gguf-q4_k_s.gguf | Q4_K_S | 4.14 GB |
+| Tess-3-7B-SFT-gguf-iq4_xs.gguf | IQ4_XS | 3.95 GB |
+| Tess-3-7B-SFT-gguf-q3_k_l.gguf | Q3_K_L | 3.83 GB |
+| Tess-3-7B-SFT-gguf-q3_k_m.gguf | Q3_K_M | 3.52 GB |
+| Tess-3-7B-SFT-gguf-iq3_m.gguf | IQ3_M | 3.29 GB |
+| Tess-3-7B-SFT-gguf-q3_k_s.gguf | Q3_K_S | 3.17 GB |
+| Tess-3-7B-SFT-gguf-iq3_xs.gguf | IQ3_XS | 3.02 GB |
+| Tess-3-7B-SFT-gguf-q2_k.gguf | Q2_K | 2.72 GB |
+| Tess-3-7B-SFT-gguf-iq3_xxs.gguf | IQ3_XXS | 136 MB |
+| Tess-3-7B-SFT-gguf-iq2_m.gguf | IQ2_M | 83.7 MB |
+
+## Downloading using huggingface-cli
+
+```bash
+pip install -U "huggingface_hub[cli]"
+huggingface-cli download juvi21/Tess-3-7B-SFT-GGUF --include "Tess-3-7B-SFT-gguf-q4_k_m.gguf" --local-dir ./