Update README.md

README.md

@@ -14,7 +14,38 @@ tags:
 
 ## ***See [our collection](https://huggingface.co/collections/unsloth/deepseek-r1-all-versions-678e1c48f5d2fce87892ace5) for versions of Deepseek-R1 including GGUF and original formats.***
 
-
+### Instructions to run this model in llama.cpp:
+Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](https://unsloth.ai/blog/deepseek-r1)
+1. Use K quantization (not V quantization)
+2. Do not forget about `<｜User｜>` and `<｜Assistant｜>` tokens! - Or use a chat template formatter
+3. Example with Q5_0 K quantized cache (V quantized cache doesn't work):
+   ```bash
+   ./llama.cpp/llama-cli
+   --model unsloth/DeepSeek-V3-GGUF/DeepSeek-R1-Distill-Llama-8B-Q2_K_XS/DeepSeek-R1-Distill-Llama-8B-Q2_K_XS-00001-of-00005.gguf
+   --cache-type-k q5_0
+   --threads 16 
+   --prompt '<｜User｜>What is 1+1?<｜Assistant｜>'
+   ```
+   Example output:
+   ```txt
+   The sum of 1 and 1 is **2**. Here's a simple step-by-step breakdown:
+
+    1. **Start with the number 1.**
+    2. **Add another 1 to it.**
+    3. **The result is 2.**
+
+    So, **1 + 1 = 2**. [end of text]
+    ```
+4. If you have a GPU (RTX 4090 for example) with 24GB, you can offload 5 layers to the GPU for faster processing. If you have multiple GPUs, you can probably offload more layers.
+   ```bash
+    /llama.cpp/llama-cli \
+    --model DeepSeek-R1-Distill-Llama-8B-F16.gguf\
+    --cache-type-k q8_0 \
+    --prompt '<｜User｜>What is 1+1?<｜Assistant｜>' \
+    --threads 32 \
+    -no-cnv
+   ```
+   
 # Finetune LLMs 2-5x faster with 70% less memory via Unsloth!
 We have a free Google Colab Tesla T4 notebook for Llama 3.1 (8B) here: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-Alpaca.ipynb