nisten
/

quad-mixtrals-gguf

Inference Endpoints

Model card Files Files and versions Community

nisten commited on Dec 23, 2023

Commit

641b075

•

1 Parent(s): e065fd7

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -6,9 +6,9 @@ license: apache-2.0
 **Goal is to have the best performing MoE < 10gb**
-They still need training/finetuning.
-* *No sparsity tricks yet.* *
 8.4gb custom 2bit quant works ok up until 512 token length then starts looping.

 **Goal is to have the best performing MoE < 10gb**
+Experimental q8 and q4 files for training/finetuning too.
+* *No sparsity tricks yet.*
 8.4gb custom 2bit quant works ok up until 512 token length then starts looping.