Hugging Face's logo

gaianet
/
StarCoder2-15B-Instruct-v0.1-GGUF

Text Generation
Transformers
GGUF
starcoder2
code
text-generation-inference
Model card Files Community
alabulei commited on
Commit
8c6d275
•
1 Parent(s): 94bace5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +58 -52
README.md CHANGED
@@ -1,52 +1,58 @@
1
- ---
2
- base_model: bigcode/starcoder2-15b-instruct-v0.1
3
- inference: false
4
- license: bigcode-openrail-m
5
- library_name: transformers
6
- model_creator: bigcode
7
- model_name: StarCoder2-15B-Instruct-v0.1
8
- pipeline_tag: text-generation
9
- quantized_by: Second State Inc.
10
- tags:
11
- - code
12
- ---
13
-
14
- <!-- header start -->
15
- <!-- 200823 -->
16
- <div style="width: auto; margin-left: auto; margin-right: auto">
17
- <img src="https://github.com/LlamaEdge/LlamaEdge/raw/dev/assets/logo.svg" style="width: 100%; min-width: 400px; display: block; margin: auto;">
18
- </div>
19
- <hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
20
- <!-- header end -->
21
-
22
- # StarCoder2-15B-Instruct-v0.1-GGUF
23
-
24
- ## Original Model
25
-
26
- [bigcode/starcoder2-15b-instruct-v0.1](https://huggingface.co/bigcode/starcoder2-15b-instruct-v0.1)
27
-
28
- ## Run with LlamaEdge
29
-
30
- - LlamaEdge version: coming soon
31
-
32
- - Context size: `6144`
33
-
34
- ## Quantized GGUF Models
35
-
36
- | Name | Quant method | Bits | Size | Use case |
37
- | ---- | ---- | ---- | ---- | ----- |
38
- | [starcoder2-15b-instruct-v0.1-Q2_K.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q2_K.gguf) | Q2_K | 2 | 6.19 GB| smallest, significant quality loss - not recommended for most purposes |
39
- | [starcoder2-15b-instruct-v0.1-Q3_K_L.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q3_K_L.gguf) | Q3_K_L | 3 | 8.97 GB| small, substantial quality loss |
40
- | [starcoder2-15b-instruct-v0.1-Q3_K_M.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q3_K_M.gguf) | Q3_K_M | 3 | 8.1 GB| very small, high quality loss |
41
- | [starcoder2-15b-instruct-v0.1-Q3_K_S.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q3_K_S.gguf) | Q3_K_S | 3 | 6.99 GB| very small, high quality loss |
42
- | [starcoder2-15b-instruct-v0.1-Q4_0.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q4_0.gguf) | Q4_0 | 4 | 9.07 GB| legacy; small, very high quality loss - prefer using Q3_K_M |
43
- | [starcoder2-15b-instruct-v0.1-Q4_K_M.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q4_K_M.gguf) | Q4_K_M | 4 | 9.86 GB| medium, balanced quality - recommended |
44
- | [starcoder2-15b-instruct-v0.1-Q4_K_S.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q4_K_S.gguf) | Q4_K_S | 4 | 9.26 GB| small, greater quality loss |
45
- | [starcoder2-15b-instruct-v0.1-Q5_0.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q5_0.gguf) | Q5_0 | 5 | 11 GB| legacy; medium, balanced quality - prefer using Q4_K_M |
46
- | [starcoder2-15b-instruct-v0.1-Q5_K_M.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q5_K_M.gguf) | Q5_K_M | 5 | 11.4 GB| large, very low quality loss - recommended |
47
- | [starcoder2-15b-instruct-v0.1-Q5_K_S.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q5_K_S.gguf) | Q5_K_S | 5 | 11 GB| large, low quality loss - recommended |
48
- | [starcoder2-15b-instruct-v0.1-Q6_K.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q6_K.gguf) | Q6_K | 6 | 13.1 GB| very large, extremely low quality loss |
49
- | [starcoder2-15b-instruct-v0.1-Q8_0.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q8_0.gguf) | Q8_0 | 8 | 17 GB| very large, extremely low quality loss - not recommended |
50
- | [starcoder2-15b-instruct-v0.1-f16.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-f16.gguf) | f16 | 16 | 31.9 GB| |
51
-
52
- *Quantized with llama.cpp b2824*
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: bigcode/starcoder2-15b-instruct-v0.1
3
+ inference: false
4
+ license: bigcode-openrail-m
5
+ library_name: transformers
6
+ model_creator: bigcode
7
+ model_name: StarCoder2-15B-Instruct-v0.1
8
+ pipeline_tag: text-generation
9
+ quantized_by: Second State Inc.
10
+ tags:
11
+ - code
12
+ ---
13
+
14
+ ![](https://github.com/GaiaNet-AI/.github/assets/45785633/d6976adc-f97d-4f86-a648-0f2f5c8e7eee)
15
+
16
+ # StarCoder2-15B-Instruct-v0.1-GGUF
17
+
18
+ ## Original Model
19
+
20
+ [bigcode/starcoder2-15b-instruct-v0.1](https://huggingface.co/bigcode/starcoder2-15b-instruct-v0.1)
21
+
22
+
23
+ ## Run with Gaianet
24
+
25
+ **Prompt template:**
26
+
27
+ prompt template: comming soon
28
+
29
+ **Context size:**
30
+
31
+ chat_ctx_size: `6144`
32
+
33
+
34
+ **Run with GaiaNet:**
35
+
36
+ - Quick start: https://docs.gaianet.ai/node-guide/quick-start
37
+
38
+ - Customize your node: https://docs.gaianet.ai/node-guide/customize
39
+
40
+ ## Quantized GGUF Models
41
+
42
+ | Name | Quant method | Bits | Size | Use case |
43
+ | ---- | ---- | ---- | ---- | ----- |
44
+ | [starcoder2-15b-instruct-v0.1-Q2_K.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q2_K.gguf) | Q2_K | 2 | 6.19 GB| smallest, significant quality loss - not recommended for most purposes |
45
+ | [starcoder2-15b-instruct-v0.1-Q3_K_L.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q3_K_L.gguf) | Q3_K_L | 3 | 8.97 GB| small, substantial quality loss |
46
+ | [starcoder2-15b-instruct-v0.1-Q3_K_M.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q3_K_M.gguf) | Q3_K_M | 3 | 8.1 GB| very small, high quality loss |
47
+ | [starcoder2-15b-instruct-v0.1-Q3_K_S.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q3_K_S.gguf) | Q3_K_S | 3 | 6.99 GB| very small, high quality loss |
48
+ | [starcoder2-15b-instruct-v0.1-Q4_0.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q4_0.gguf) | Q4_0 | 4 | 9.07 GB| legacy; small, very high quality loss - prefer using Q3_K_M |
49
+ | [starcoder2-15b-instruct-v0.1-Q4_K_M.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q4_K_M.gguf) | Q4_K_M | 4 | 9.86 GB| medium, balanced quality - recommended |
50
+ | [starcoder2-15b-instruct-v0.1-Q4_K_S.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q4_K_S.gguf) | Q4_K_S | 4 | 9.26 GB| small, greater quality loss |
51
+ | [starcoder2-15b-instruct-v0.1-Q5_0.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q5_0.gguf) | Q5_0 | 5 | 11 GB| legacy; medium, balanced quality - prefer using Q4_K_M |
52
+ | [starcoder2-15b-instruct-v0.1-Q5_K_M.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q5_K_M.gguf) | Q5_K_M | 5 | 11.4 GB| large, very low quality loss - recommended |
53
+ | [starcoder2-15b-instruct-v0.1-Q5_K_S.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q5_K_S.gguf) | Q5_K_S | 5 | 11 GB| large, low quality loss - recommended |
54
+ | [starcoder2-15b-instruct-v0.1-Q6_K.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q6_K.gguf) | Q6_K | 6 | 13.1 GB| very large, extremely low quality loss |
55
+ | [starcoder2-15b-instruct-v0.1-Q8_0.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q8_0.gguf) | Q8_0 | 8 | 17 GB| very large, extremely low quality loss - not recommended |
56
+ | [starcoder2-15b-instruct-v0.1-f16.gguf](https://huggingface.co/gaianet/StarCoder2-15B-Instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-f16.gguf) | f16 | 16 | 31.9 GB| |
57
+
58
+ *Quantized with llama.cpp b2824*