Update README.md
Browse files
README.md
CHANGED
|
@@ -8,6 +8,14 @@ language:
|
|
| 8 |
|
| 9 |
*Cute girl to catch your attention.*
|
| 10 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
**https://huggingface.co/Sao10K/Fimbulvetr-11B-v2-GGUF <------ GGUF**
|
| 12 |
|
| 13 |
Fimbulvetr-v2 - A Solar-Based Model
|
|
|
|
| 8 |
|
| 9 |
*Cute girl to catch your attention.*
|
| 10 |
|
| 11 |
+
EXL2 4.35bpw quant of (https://huggingface.co/Sao10K/Fimbulvetr-11B-v2)
|
| 12 |
+
|
| 13 |
+
Same goal as my last quants. being able to fit 16k context within 8GB of ram with as much quality as possible. This time with 11b!
|
| 14 |
+
|
| 15 |
+
Recommended to run with Tabbyapi, ooba could cause some issues with model loading and running.
|
| 16 |
+
|
| 17 |
+
Tested using RTX 4060 8GB Laptop, 16K context with 4bit cache.
|
| 18 |
+
|
| 19 |
**https://huggingface.co/Sao10K/Fimbulvetr-11B-v2-GGUF <------ GGUF**
|
| 20 |
|
| 21 |
Fimbulvetr-v2 - A Solar-Based Model
|