Update README.md
Browse files
README.md
CHANGED
@@ -11,17 +11,17 @@ pipeline_tag: text-generation
|
|
11 |
|
12 |
project original [source](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) (base model)
|
13 |
|
14 |
-
|
15 |
|
16 |
-
|
17 |
|
18 |
-
|
19 |
|
20 |
-
|
21 |
|
22 |
-
|
23 |
|
24 |
-
|
25 |
|
26 |
16-bit and 32-bit are also provided here for research perspectives; since the file size (16bit) is similar to the original safetensors; once you have a GPU, go ahead with the safetensors, pretty much the same
|
27 |
|
|
|
11 |
|
12 |
project original [source](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) (base model)
|
13 |
|
14 |
+
Q_2 (not nice)
|
15 |
|
16 |
+
Q_3 (acceptable)
|
17 |
|
18 |
+
Q_4 family is recommanded (good for running with CPU as well)
|
19 |
|
20 |
+
Q_5 (good in general)
|
21 |
|
22 |
+
Q_6 is good also; if you want a better result; take this one instead of Q_5
|
23 |
|
24 |
+
Q_8 which is very good; need a reasonable size of RAM otherwise you might expect a long wait
|
25 |
|
26 |
16-bit and 32-bit are also provided here for research perspectives; since the file size (16bit) is similar to the original safetensors; once you have a GPU, go ahead with the safetensors, pretty much the same
|
27 |
|