pipihand01
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -18,10 +18,6 @@ GGUF files of [princeton-nlp/gemma-2-9b-it-SimPO](https://huggingface.co/princet
|
|
18 |
---
|
19 |
## Files quantized with larger embed and output weights than normal GGUF setting
|
20 |
|
21 |
-
- Q8_0 embed and output weights:
|
22 |
|
23 |
-
|
24 |
-
|
25 |
-
- bf16 embed and output weights: (maybe slower inference)
|
26 |
-
|
27 |
-
Q8_0_L, Q6_K_XL, Q5_K_XL, Q4_K_XL
|
|
|
18 |
---
|
19 |
## Files quantized with larger embed and output weights than normal GGUF setting
|
20 |
|
21 |
+
- Q8_0 embed and output weights: Q6_K_L, Q5_K_L, Q4_K_L
|
22 |
|
23 |
+
- bf16 embed and output weights (maybe slower inference): Q8_0_L, Q6_K_XL, Q5_K_XL, Q4_K_XL
|
|
|
|
|
|
|
|