apepkuss79
commited on
Commit
•
6c23d84
1
Parent(s):
4541705
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -25,11 +25,11 @@ license: apache-2.0
|
|
25 |
|
26 |
## Run with LlamaEdge
|
27 |
|
28 |
-
<!-- - LlamaEdge version: [v0.
|
29 |
|
30 |
- LlamaEdge version: coming soon
|
31 |
|
32 |
-
|
33 |
|
34 |
- Prompt type: `chatml`
|
35 |
|
@@ -66,7 +66,7 @@ license: apache-2.0
|
|
66 |
--prompt-template chatml \
|
67 |
--reverse-prompt "<|im_end|>" \
|
68 |
--ctx-size 4096
|
69 |
-
```
|
70 |
|
71 |
## Quantized GGUF Models
|
72 |
|
@@ -86,4 +86,4 @@ license: apache-2.0
|
|
86 |
| [Yi-Coder-1.5B-Chat-Q8_0.gguf](https://huggingface.co/second-state/Yi-Coder-1.5B-Chat-GGUF/blob/main/Yi-Coder-1.5B-Chat-Q8_0.gguf) | Q8_0 | 8 | 1.57 GB| very large, extremely low quality loss - not recommended |
|
87 |
| [Yi-Coder-1.5B-Chat-f16.gguf](https://huggingface.co/second-state/Yi-Coder-1.5B-Chat-GGUF/blob/main/Yi-Coder-1.5B-Chat-f16.gguf) | f16 | 16 | 2.95 GB| |
|
88 |
|
89 |
-
*Quantized with llama.cpp
|
|
|
25 |
|
26 |
## Run with LlamaEdge
|
27 |
|
28 |
+
<!-- - LlamaEdge version: [v0.14.1](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.14.1) and above -->
|
29 |
|
30 |
- LlamaEdge version: coming soon
|
31 |
|
32 |
+
- Prompt template
|
33 |
|
34 |
- Prompt type: `chatml`
|
35 |
|
|
|
66 |
--prompt-template chatml \
|
67 |
--reverse-prompt "<|im_end|>" \
|
68 |
--ctx-size 4096
|
69 |
+
```
|
70 |
|
71 |
## Quantized GGUF Models
|
72 |
|
|
|
86 |
| [Yi-Coder-1.5B-Chat-Q8_0.gguf](https://huggingface.co/second-state/Yi-Coder-1.5B-Chat-GGUF/blob/main/Yi-Coder-1.5B-Chat-Q8_0.gguf) | Q8_0 | 8 | 1.57 GB| very large, extremely low quality loss - not recommended |
|
87 |
| [Yi-Coder-1.5B-Chat-f16.gguf](https://huggingface.co/second-state/Yi-Coder-1.5B-Chat-GGUF/blob/main/Yi-Coder-1.5B-Chat-f16.gguf) | f16 | 16 | 2.95 GB| |
|
88 |
|
89 |
+
*Quantized with llama.cpp b3613*
|