danielhanchen commited on
Commit
7916422
·
verified ·
1 Parent(s): 24f4fc7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -11
README.md CHANGED
@@ -17,32 +17,31 @@ tags:
17
  ### Instructions to run this model in llama.cpp:
18
  Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](https://unsloth.ai/blog/deepseek-r1)
19
  1. Do not forget about `<|User|>` and `<|Assistant|>` tokens! - Or use a chat template formatter
20
- 2. Example with K & V quantized cache **Notice -no-cnv disables auto conversation mode**
21
  ```bash
22
  ./llama.cpp/llama-cli \
23
  --model unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf \
24
  --cache-type-k q8_0 \
25
- --cache-type-v q8_0 \
26
  --threads 16 \
27
  --prompt '<|User|>What is 1+1?<|Assistant|>' \
28
  -no-cnv
29
  ```
30
  Example output:
 
31
  ```txt
32
- The sum of 1 and 1 is **2**. Here's a simple step-by-step breakdown:
33
-
34
- 1. **Start with the number 1.**
35
- 2. **Add another 1 to it.**
36
- 3. **The result is 2.**
37
-
38
- So, **1 + 1 = 2**. [end of text]
39
- ```
40
  4. If you have a GPU (RTX 4090 for example) with 24GB, you can offload multiple layers to the GPU for faster processing. If you have multiple GPUs, you can probably offload more layers.
41
  ```bash
42
  ./llama.cpp/llama-cli \
43
  --model unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf
44
  --cache-type-k q8_0
45
- --cache-type-v q8_0
46
  --threads 16
47
  --prompt '<|User|>What is 1+1?<|Assistant|>'
48
  --n-gpu-layers 20 \
 
17
  ### Instructions to run this model in llama.cpp:
18
  Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](https://unsloth.ai/blog/deepseek-r1)
19
  1. Do not forget about `<|User|>` and `<|Assistant|>` tokens! - Or use a chat template formatter
20
+ 2. Example with Q8_0 K quantized cache **Notice -no-cnv disables auto conversation mode**
21
  ```bash
22
  ./llama.cpp/llama-cli \
23
  --model unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf \
24
  --cache-type-k q8_0 \
 
25
  --threads 16 \
26
  --prompt '<|User|>What is 1+1?<|Assistant|>' \
27
  -no-cnv
28
  ```
29
  Example output:
30
+
31
  ```txt
32
+ <think>
33
+ Okay, so I need to figure out what 1 plus 1 is. Hmm, where do I even start? I remember from school that adding numbers is pretty basic, but I want to make sure I understand it properly.
34
+ Let me think, 1 plus 1. So, I have one item and I add another one. Maybe like a apple plus another apple. If I have one apple and someone gives me another, I now have two apples. So, 1 plus 1 should be 2. That makes sense.
35
+ Wait, but sometimes math can be tricky. Could it be something else? Like, in a different number system maybe? But I think the question is straightforward, using regular numbers, not like binary or hexadecimal or anything.
36
+ I also recall that in arithmetic, addition is combining quantities. So, if you have two quantities of 1, combining them gives you a total of 2. Yeah, that seems right.
37
+ Is there a scenario where 1 plus 1 wouldn't be 2? I can't think of any...
38
+ ```
39
+
40
  4. If you have a GPU (RTX 4090 for example) with 24GB, you can offload multiple layers to the GPU for faster processing. If you have multiple GPUs, you can probably offload more layers.
41
  ```bash
42
  ./llama.cpp/llama-cli \
43
  --model unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf
44
  --cache-type-k q8_0
 
45
  --threads 16
46
  --prompt '<|User|>What is 1+1?<|Assistant|>'
47
  --n-gpu-layers 20 \