dranger003
/

dolphincoder-starcoder2-15b-iMat.GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

dranger003 commited on Mar 28

Commit

a41d1ec

•

1 Parent(s): b7a7b20

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -8,7 +8,7 @@ base_model: cognitivecomputations/dolphincoder-starcoder2-15b
 GGUF importance matrix (imatrix) quants for https://huggingface.co/cognitivecomputations/dolphincoder-starcoder2-15b
 * The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a [general purpose imatrix calibration dataset](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384).
-* The [imatrix is being used on the K-quants](https://github.com/ggerganov/llama.cpp/pull/4930) as well.
 > This model is based on StarCoder2-15b and is subject to bigcode-openrail-m license.<br>This Dolphin is really good at coding, I trained with a lot of coding data.<br>This model is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant to any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models You are responsible for any content you create using this model. Enjoy responsibly.

 GGUF importance matrix (imatrix) quants for https://huggingface.co/cognitivecomputations/dolphincoder-starcoder2-15b
 * The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a [general purpose imatrix calibration dataset](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384).
+* The [imatrix is being used on the K-quants](https://github.com/ggerganov/llama.cpp/pull/4930) as well (under Q6_K).
 > This model is based on StarCoder2-15b and is subject to bigcode-openrail-m license.<br>This Dolphin is really good at coding, I trained with a lot of coding data.<br>This model is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant to any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models You are responsible for any content you create using this model. Enjoy responsibly.