nisten
/

quad-mixtrals-gguf

Inference Endpoints

Model card Files Files and versions Community

nisten commited on Dec 23, 2023

Commit

994e63d

•

1 Parent(s): 641b075

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -6,7 +6,7 @@ license: apache-2.0
 **Goal is to have the best performing MoE < 10gb**
-Experimental q8 and q4 files for training/finetuning too.
 * *No sparsity tricks yet.*
@@ -24,7 +24,7 @@ make -j
 wget https://huggingface.co/nisten/quad-mixtrals-gguf/resolve/main/4mixq2.gguf
-./server -m 4mixq2.gguf --host "ec2-3-99-206-122.ca-central-1.compute.amazonaws.com" -c 512
 ```

 **Goal is to have the best performing MoE < 10gb**
+>Experimental q8 and q4 files for training/finetuning too.
 * *No sparsity tricks yet.*
 wget https://huggingface.co/nisten/quad-mixtrals-gguf/resolve/main/4mixq2.gguf
+./server -m 4mixq2.gguf --host "my.internal.ip.or.my.cloud.host.name.goes.here.com" -c 512
 ```