SinanAkkoyun
commited on
Commit
•
5a5afd3
1
Parent(s):
56224bb
test
Browse files
README.md
CHANGED
|
@@ -1 +1,7 @@
|
|
| 1 |
This is a very bad attempt at quantizing 128g 4 bit with alpaca (in orca style prompt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
This is a very bad attempt at quantizing 128g 4 bit with alpaca (in orca style prompt
|
| 2 |
+
|
| 3 |
+
```sh
|
| 4 |
+
python quantize_alpaca.py --pretrained_model_dir orca_mini_3b/ --bits 4 --group_size 128 --quantized_model_dir orca_mini_3b_gptq/ --save_and_reloa
|
| 5 |
+
```
|
| 6 |
+
|
| 7 |
+
Downloqd cleaned dataset first: https://github.com/gururise/AlpacaDataCleaned
|