Update README.md
Browse files
README.md
CHANGED
@@ -9,8 +9,13 @@ inference: true
|
|
9 |
- Based on version 1.1
|
10 |
- Used PR "More accurate Q4_0 and Q4_1 quantizations #896" (should be closer in quality to unquantized)
|
11 |
- Uncensored variant is available, but it's based on version 1.0
|
|
|
|
|
|
|
|
|
|
|
|
|
12 |
- 13B version of this can be found here: https://huggingface.co/eachadea/ggml-vicuna-13b-1.1
|
13 |
-
- Choosing between q4_0 and q4_1, the logic of higher number \= better doesn't apply. If you're confused, stick with q4_0.
|
14 |
<br>
|
15 |
<br>
|
16 |
|
|
|
9 |
- Based on version 1.1
|
10 |
- Used PR "More accurate Q4_0 and Q4_1 quantizations #896" (should be closer in quality to unquantized)
|
11 |
- Uncensored variant is available, but it's based on version 1.0
|
12 |
+
- For q4_2, "Q4_2 ARM #1046" was used. Will update regularly if new changes are made.
|
13 |
+
- **Choosing between q4_0, q4_1, and q4_2:**
|
14 |
+
- 4_0 is the fastest. The quality is the poorest.
|
15 |
+
- 4_1 is a lot slower. The quality is noticeably better.
|
16 |
+
- 4_2 is almost as fast as 4_0 and about as good as 4_1 **on Apple Silicon**. On Intel/AMD it's hardly better or faster than 4_1.
|
17 |
+
|
18 |
- 13B version of this can be found here: https://huggingface.co/eachadea/ggml-vicuna-13b-1.1
|
|
|
19 |
<br>
|
20 |
<br>
|
21 |
|