Update README.md
Browse files
README.md
CHANGED
@@ -31,7 +31,12 @@ This repository contains the model [jphme/Llama-2-13b-chat-german](https://huggi
|
|
31 |
|
32 |
## Conversion Code
|
33 |
```
|
34 |
-
|
|
|
|
|
|
|
|
|
|
|
35 |
```
|
36 |
___
|
37 |
|
|
|
31 |
|
32 |
## Conversion Code
|
33 |
```
|
34 |
+
# Convert original model to FP16 GGUF format
|
35 |
+
python3 llama.cpp/convert.py ./original-models/Llama-2-13b-chat-german --outtype f16 --outfile ./converted_gguf/Llama-2-13b-chat-german-GGUF.fp16.bin
|
36 |
+
# Quantize FP16 GGUF to 8, 5_K_M and 4_K_M bit
|
37 |
+
./llama.cpp/quantize Llama-2-13b-chat-german-GGUF.fp16.bin Llama-2-13b-chat-german-GGUF.q8_0.bin q8_0
|
38 |
+
./llama.cpp/quantize Llama-2-13b-chat-german-GGUF.fp16.bin Llama-2-13b-chat-german-GGUF.q8_0.bin q5_K_M
|
39 |
+
./llama.cpp/quantize Llama-2-13b-chat-german-GGUF.fp16.bin Llama-2-13b-chat-german-GGUF.q8_0.bin q4_K_M
|
40 |
```
|
41 |
___
|
42 |
|