Hugging Face's logo

EricB
/
Phi-3.5-mini-instruct-UQFF

phi3
uqff
mistral.rs
custom_code
Model card Files Community
EricB HF staff commited on
Commit
3f9680f
·
verified ·
1 Parent(s): ef9367e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -22,10 +22,10 @@ Run with [mistral.rs](https://github.com/EricLBuehler/mistral.rs). Documentation
22
 
23
  |Quantization type(s)|Example|
24
  |--|--|
25
- |FP8|`./mistralrs-server -i vision-plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-f8e4m3.uqff`|
26
- |HQQ4|`./mistralrs-server -i vision-plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-hqq4.uqff`|
27
- |HQQ8|`./mistralrs-server -i vision-plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-hqq8.uqff`|
28
- |Q3K|`./mistralrs-server -i vision-plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-q3k.uqff`|
29
- |Q4K|`./mistralrs-server -i vision-plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-q4k.uqff`|
30
- |Q5K|`./mistralrs-server -i vision-plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-q5k.uqff`|
31
- |Q8_0|`./mistralrs-server -i vision-plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-q8_0.uqff`|
 
22
 
23
  |Quantization type(s)|Example|
24
  |--|--|
25
+ |FP8|`./mistralrs-server -i plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-f8e4m3.uqff`|
26
+ |HQQ4|`./mistralrs-server -i plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-hqq4.uqff`|
27
+ |HQQ8|`./mistralrs-server -i plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-hqq8.uqff`|
28
+ |Q3K|`./mistralrs-server -i plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-q3k.uqff`|
29
+ |Q4K|`./mistralrs-server -i plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-q4k.uqff`|
30
+ |Q5K|`./mistralrs-server -i plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-q5k.uqff`|
31
+ |Q8_0|`./mistralrs-server -i plain -m EricB/Phi-3.5-mini-instruct-UQFF --from-uqff phi3.5-mini-instruct-q8_0.uqff`|