danielhanchen
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -17,7 +17,8 @@ tags:
|
|
17 |
### Instructions to run this model in llama.cpp:
|
18 |
Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](https://unsloth.ai/blog/deepseek-r1)
|
19 |
1. Do not forget about `<|User|>` and `<|Assistant|>` tokens! - Or use a chat template formatter
|
20 |
-
2.
|
|
|
21 |
```bash
|
22 |
./llama.cpp/llama-cli \
|
23 |
--model unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf \
|
|
|
17 |
### Instructions to run this model in llama.cpp:
|
18 |
Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](https://unsloth.ai/blog/deepseek-r1)
|
19 |
1. Do not forget about `<|User|>` and `<|Assistant|>` tokens! - Or use a chat template formatter
|
20 |
+
2. Obtain the latest `llama.cpp` at https://github.com/ggerganov/llama.cpp
|
21 |
+
3. Example with Q8_0 K quantized cache **Notice -no-cnv disables auto conversation mode**
|
22 |
```bash
|
23 |
./llama.cpp/llama-cli \
|
24 |
--model unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf \
|