Thireus
/

WizardLM-70B-V1.0-HF-4.0bpw-h6-exl2

Text Generation

Model card Files Files and versions Community

Thireus commited on Sep 17, 2023

Commit

96aed5a

·

1 Parent(s): f080ce7

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -49,7 +49,7 @@ ASSISTANT:
 ## Quantization process:
-| Original Model | → | Float16 Model* | → | Safetensor Model** | → | EXL2 Model |
 | -------------- | --- | ------------- | --- | ---------------- | --- | ---------- |
 | [WizardLM 70B V1.0](https://huggingface.co/WizardLM/WizardLM-70B-V1.0) | → | [WizardLM 70B V1.0-HF](https://huggingface.co/simsim314/WizardLM-70B-V1.0-HF)* | → | Safetensor** | → | EXL2 |
@@ -65,7 +65,7 @@ python convert.py -i ~/float16_safetensored/WizardLM-70B-V1.0-HF -o ~/EXL2/Wizar
 - https://github.com/oobabooga/text-generation-webui/blob/main/convert-to-safetensors.py
  (best for sharding and float16/FP16 or bfloat16/BF16 conversion)
-\*\* Use any one of the following scripts to convert your local float16 pytorch_model bin files to safetensors:
 - https://github.com/turboderp/exllamav2/blob/master/util/convert_safetensors.py (official ExLlamaV2)
 - https://huggingface.co/Panchovix/airoboros-l2-70b-gpt4-1.4.1-safetensors/blob/main/bin2safetensors/convert.py (recommended if model already converted to float16)

 ## Quantization process:
+| Original Model | → | (optional but recommended) Float16 Model* | → | Safetensor Model** | → | EXL2 Model |
 | -------------- | --- | ------------- | --- | ---------------- | --- | ---------- |
 | [WizardLM 70B V1.0](https://huggingface.co/WizardLM/WizardLM-70B-V1.0) | → | [WizardLM 70B V1.0-HF](https://huggingface.co/simsim314/WizardLM-70B-V1.0-HF)* | → | Safetensor** | → | EXL2 |
 - https://github.com/oobabooga/text-generation-webui/blob/main/convert-to-safetensors.py
  (best for sharding and float16/FP16 or bfloat16/BF16 conversion)
+\*\* Use any one of the following scripts to convert your local pytorch_model bin files to safetensors:
 - https://github.com/turboderp/exllamav2/blob/master/util/convert_safetensors.py (official ExLlamaV2)
 - https://huggingface.co/Panchovix/airoboros-l2-70b-gpt4-1.4.1-safetensors/blob/main/bin2safetensors/convert.py (recommended if model already converted to float16)