calcuis commited on
Commit
5319afa
1 Parent(s): 16ca49a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -11,17 +11,17 @@ pipeline_tag: text-generation
11
 
12
  project original [source](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) (base model)
13
 
14
- Q_2_K (not nice)
15
 
16
- Q_3_K_M (acceptable)
17
 
18
- Q_4_K_M is recommanded (good for running with CPU as well)
19
 
20
- Q_5_K_M (good in general)
21
 
22
- Q_6_K is good also; if you want a better result; take this one instead of Q_5_K_M
23
 
24
- Q_8_0 which is very good; need a reasonable size of RAM otherwise you might expect a long wait
25
 
26
  16-bit and 32-bit are also provided here for research perspectives; since the file size (16bit) is similar to the original safetensors; once you have a GPU, go ahead with the safetensors, pretty much the same
27
 
 
11
 
12
  project original [source](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) (base model)
13
 
14
+ Q_2 (not nice)
15
 
16
+ Q_3 (acceptable)
17
 
18
+ Q_4 family is recommanded (good for running with CPU as well)
19
 
20
+ Q_5 (good in general)
21
 
22
+ Q_6 is good also; if you want a better result; take this one instead of Q_5
23
 
24
+ Q_8 which is very good; need a reasonable size of RAM otherwise you might expect a long wait
25
 
26
  16-bit and 32-bit are also provided here for research perspectives; since the file size (16bit) is similar to the original safetensors; once you have a GPU, go ahead with the safetensors, pretty much the same
27