Hugging Face's logo

Thireus
/
WizardLM-70B-V1.0-HF-4.0bpw-h6-exl2

Text Generation
Transformers
llama
Model card Files Community
Thireus commited on
Commit
99bfe63
·
1 Parent(s): 8417013

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -71,6 +71,8 @@ Example to convert [WizardLM 70B V1.0](https://huggingface.co/WizardLM/WizardLM-
71
  python convert-to-safetensors.py ~/original/WizardLM-70B-V1.0 --output ~/float16_safetensored/WizardLM-70B-V1.0 --max-shard-size 10GB
72
  ```
73
 
 
 
74
  \*\* Use any one of the following scripts to convert your local pytorch_model bin files to safetensors:
75
 
76
  - https://github.com/turboderp/exllamav2/blob/master/util/convert_safetensors.py (official ExLlamaV2)
 
71
  python convert-to-safetensors.py ~/original/WizardLM-70B-V1.0 --output ~/float16_safetensored/WizardLM-70B-V1.0 --max-shard-size 10GB
72
  ```
73
 
74
+ Use `--bf16` if you'd like to try bfloat16 instead, but note that there are concerns about quantization quality – https://github.com/turboderp/exllamav2/issues/30#issuecomment-1719009289
75
+
76
  \*\* Use any one of the following scripts to convert your local pytorch_model bin files to safetensors:
77
 
78
  - https://github.com/turboderp/exllamav2/blob/master/util/convert_safetensors.py (official ExLlamaV2)