Update README.md
Browse files
README.md
CHANGED
@@ -15,6 +15,7 @@ inference: false
|
|
15 |
</div>
|
16 |
</div>
|
17 |
<!-- header end -->
|
|
|
18 |
# Vicuna 7B GPTQ 4-bit 128g
|
19 |
|
20 |
This repository contains the [Vicuna 7B model](https://huggingface.co/lmsys/vicuna-7b-delta-v0) quantised using [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa).
|
@@ -26,10 +27,6 @@ The original Vicuna 7B repository contains deltas rather than weights. Rather th
|
|
26 |
Two model files are provided. You don't need both, choose the one you prefer.
|
27 |
|
28 |
Details of the files provided:
|
29 |
-
* `vicuna-7B-GPTQ-4bit-128g.pt`
|
30 |
-
* pt format file, created with the latest [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa) code.
|
31 |
-
* Command to create:
|
32 |
-
* `python3 llama.py vicuna-7B c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save vicuna-7B-GPTQ-4bit-128g.pt`
|
33 |
* `vicuna-7B-GPTQ-4bit-128g.safetensors`
|
34 |
* newer `safetensors` format, with improved file security, created with the latest [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa) code.
|
35 |
* Command to create:
|
|
|
15 |
</div>
|
16 |
</div>
|
17 |
<!-- header end -->
|
18 |
+
|
19 |
# Vicuna 7B GPTQ 4-bit 128g
|
20 |
|
21 |
This repository contains the [Vicuna 7B model](https://huggingface.co/lmsys/vicuna-7b-delta-v0) quantised using [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa).
|
|
|
27 |
Two model files are provided. You don't need both, choose the one you prefer.
|
28 |
|
29 |
Details of the files provided:
|
|
|
|
|
|
|
|
|
30 |
* `vicuna-7B-GPTQ-4bit-128g.safetensors`
|
31 |
* newer `safetensors` format, with improved file security, created with the latest [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa) code.
|
32 |
* Command to create:
|