Llamacpp quants

.gitattributes

@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+speechless-starcoder2-7b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,52 @@
+---
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+datasets:
+- teknium/OpenHermes-2.5
+- TokenBender/python_eval_instruct_51k
+- codefuse-ai/Evol-instruction-66k
+tags:
+- code
+license: apache-2.0
+model-index:
+- name: SpeechlessCoder
+  results:
+  - task:
+      type: text-generation
+    dataset:
+      type: openai_humaneval
+      name: HumanEval
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 0.0
+      verified: false
+quantized_by: bartowski
+---
+
+## Llamacpp Quantizations of speechless-starcoder2-7b
+
+Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2354">b2354</a> for quantization.
+
+Original model: https://huggingface.co/uukuguy/speechless-starcoder2-7b
+
+Download a file (not the whole branch) from below:
+
+| Filename | Quant type | File Size | Description |
+| -------- | ---------- | --------- | ----------- |
+| [speechless-starcoder2-7b-Q8_0.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q8_0.gguf) | Q8_0 | 7.62GB | Extremely high quality, generally unneeded but max available quant. |
+| [speechless-starcoder2-7b-Q6_K.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q6_K.gguf) | Q6_K | 5.89GB | Very high quality, near perfect, *recommended*. |
+| [speechless-starcoder2-7b-Q5_K_M.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q5_K_M.gguf) | Q5_K_M | 5.12GB | High quality, very usable. |
+| [speechless-starcoder2-7b-Q5_K_S.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q5_K_S.gguf) | Q5_K_S | 4.93GB | High quality, very usable. |
+| [speechless-starcoder2-7b-Q5_0.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q5_0.gguf) | Q5_0 | 4.93GB | High quality, older format, generally not recommended. |
+| [speechless-starcoder2-7b-Q4_K_M.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q4_K_M.gguf) | Q4_K_M | 4.40GB | Good quality, similar to 4.25 bpw. |
+| [speechless-starcoder2-7b-Q4_K_S.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q4_K_S.gguf) | Q4_K_S | 4.12GB | Slightly lower quality with small space savings. |
+| [speechless-starcoder2-7b-Q4_0.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q4_0.gguf) | Q4_0 | 4.04GB | Decent quality, older format, generally not recommended. |
+| [speechless-starcoder2-7b-Q3_K_L.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q3_K_L.gguf) | Q3_K_L | 3.98GB | Lower quality but usable, good for low RAM availability. |
+| [speechless-starcoder2-7b-Q3_K_M.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q3_K_M.gguf) | Q3_K_M | 3.59GB | Even lower quality. |
+| [speechless-starcoder2-7b-Q3_K_S.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q3_K_S.gguf) | Q3_K_S | 3.09GB | Low quality, not recommended. |
+| [speechless-starcoder2-7b-Q2_K.gguf](https://huggingface.co/bartowski/speechless-starcoder2-7b-GGUF//main/speechless-starcoder2-7b-Q2_K.gguf) | Q2_K | 2.72GB | Extremely low quality, *not* recommended.
+
+Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

speechless-starcoder2-7b-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d955dc87a41bdd16941483d39d000c6134126b5c2a8b14f0394e6c5ec75df9c2
+size 2724543296

speechless-starcoder2-7b-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f114242bd41d339c1ef6540bf385717cda799a2a3dcbccf9d2df43bb23876dde
+size 3985587008

speechless-starcoder2-7b-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aed9e9a092a306ee994b1d5d4643f1b229a876ab15a927c05ecce7b89146d98f
+size 3594533696

speechless-starcoder2-7b-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b76cba8c684f40187e5ae24a62fbb8cf29bf56cd4f046cdb4934b75677cd7aef
+size 3090234176

speechless-starcoder2-7b-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbd681925a6f0a758b47e461b15810ce6a6a93a101e57fc922428c5d1b7cf658
+size 4042799936

speechless-starcoder2-7b-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11267076021427722572215fcc58ae9b58363b14cf6b3676ff1c0e3226e578fe
+size 4402887488

speechless-starcoder2-7b-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4440a9fcf3d89cc01abbc29ea9c1e249ad7dbe6645644dc3936a3de887a5f06
+size 4128914240

speechless-starcoder2-7b-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c12c8630ac98539eec85b4cbd155e2b158a8353ba8dd8a33c6d2ed9a296a9ac
+size 4939332416

speechless-starcoder2-7b-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c60a2e4648ab4262fce5aa745ec61167d334d831abb1dee446465e65b79bc8a
+size 5124832064

speechless-starcoder2-7b-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c06b4f1365530ba3cc0be242c93f89c2bd1cbbce43b0ea2ff1249b6cee14f3d
+size 4939332416

speechless-starcoder2-7b-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:269c5ac93b241ce6845f8be4121865ee1fadc0107e5c2ee3c85fc34f4df626ed
+size 5891898176

speechless-starcoder2-7b-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28138ce1685b763539bf7273c49e1980b2b09950dc273ece5adc92a4c5a0d4c4
+size 7628929856