freefallr commited on
Commit
bca02b0
1 Parent(s): f7a2d95

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -1
README.md CHANGED
@@ -31,7 +31,12 @@ This repository contains the model [jphme/Llama-2-13b-chat-german](https://huggi
31
 
32
  ## Conversion Code
33
  ```
34
-
 
 
 
 
 
35
  ```
36
  ___
37
 
 
31
 
32
  ## Conversion Code
33
  ```
34
+ # Convert original model to FP16 GGUF format
35
+ python3 llama.cpp/convert.py ./original-models/Llama-2-13b-chat-german --outtype f16 --outfile ./converted_gguf/Llama-2-13b-chat-german-GGUF.fp16.bin
36
+ # Quantize FP16 GGUF to 8, 5_K_M and 4_K_M bit
37
+ ./llama.cpp/quantize Llama-2-13b-chat-german-GGUF.fp16.bin Llama-2-13b-chat-german-GGUF.q8_0.bin q8_0
38
+ ./llama.cpp/quantize Llama-2-13b-chat-german-GGUF.fp16.bin Llama-2-13b-chat-german-GGUF.q8_0.bin q5_K_M
39
+ ./llama.cpp/quantize Llama-2-13b-chat-german-GGUF.fp16.bin Llama-2-13b-chat-german-GGUF.q8_0.bin q4_K_M
40
  ```
41
  ___
42