cicdatopea
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -7,7 +7,7 @@ base_model:
|
|
7 |
|
8 |
## Model Details
|
9 |
|
10 |
-
This awq model is an int4 model with group_size 32 and symmetric quantization of [SmallThinker-3B-Preview](https://huggingface.co/PowerInfer/SmallThinker-3B-Preview) generated by [intel/auto-round](https://github.com/intel/auto-round).
|
11 |
|
12 |
## How To Use
|
13 |
### Requirements
|
@@ -111,11 +111,10 @@ text="How many r in strawberry."
|
|
111 |
|
112 |
### Generate the model
|
113 |
|
114 |
-
Here is the sample command to generate the model.
|
115 |
-
|
116 |
```bash
|
117 |
auto-round \
|
118 |
-
--model
|
119 |
--device 0 \
|
120 |
--group_size 32 \
|
121 |
--bits 4 \
|
|
|
7 |
|
8 |
## Model Details
|
9 |
|
10 |
+
This awq model is an int4 model with group_size 32 and symmetric quantization of [PowerInfer/SmallThinker-3B-Preview](https://huggingface.co/PowerInfer/SmallThinker-3B-Preview) generated by [intel/auto-round](https://github.com/intel/auto-round).
|
11 |
|
12 |
## How To Use
|
13 |
### Requirements
|
|
|
111 |
|
112 |
### Generate the model
|
113 |
|
114 |
+
Here is the sample command to generate the model.
|
|
|
115 |
```bash
|
116 |
auto-round \
|
117 |
+
--model PowerInfer/SmallThinker-3B-Preview \
|
118 |
--device 0 \
|
119 |
--group_size 32 \
|
120 |
--bits 4 \
|