MengniWang
commited on
Commit
•
f41827e
1
Parent(s):
895359e
Update README.md
Browse files
README.md
CHANGED
@@ -16,7 +16,7 @@ tags:
|
|
16 |
- neural-compressor
|
17 |
---
|
18 |
|
19 |
-
|
20 |
|
21 |
GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
|
22 |
|
@@ -25,19 +25,35 @@ This int8 ONNX model is generated by [neural-compressor](https://github.com/inte
|
|
25 |
python -m transformers.onnx --model=EleutherAI/gpt-j-6B onnx_gptj/ --framework pt --opset 13 --feature=causal-lm-with-past
|
26 |
```
|
27 |
|
28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
29 |
|
30 |
-
|
|
31 |
-
|
32 |
-
|
|
33 |
-
|
|
|
|
34 |
|
35 |
|
36 |
-
|
37 |
|
38 |
Download the model and script by cloning the repository:
|
39 |
```shell
|
40 |
git clone https://huggingface.co/Intel/gpt-j-6B-int8-dynamic
|
41 |
```
|
42 |
|
43 |
-
Then you can do inference based on the model and script 'evaluation.ipynb'.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
- neural-compressor
|
17 |
---
|
18 |
|
19 |
+
## Model Details: INT8 GPT-J 6B
|
20 |
|
21 |
GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
|
22 |
|
|
|
25 |
python -m transformers.onnx --model=EleutherAI/gpt-j-6B onnx_gptj/ --framework pt --opset 13 --feature=causal-lm-with-past
|
26 |
```
|
27 |
|
28 |
+
| Model Detail | Description |
|
29 |
+
| ----------- | ----------- |
|
30 |
+
| Model Authors - Company | Intel |
|
31 |
+
| Date | April 10, 2022 |
|
32 |
+
| Version | 1 |
|
33 |
+
| Type | Text Generation |
|
34 |
+
| Paper or Other Resources | - |
|
35 |
+
| License | Apache 2.0 |
|
36 |
+
| Questions or Comments | [Community Tab](https://huggingface.co/Intel/gpt-j-6B-int8-dynamic/discussions)|
|
37 |
|
38 |
+
| Intended Use | Description |
|
39 |
+
| ----------- | ----------- |
|
40 |
+
| Primary intended uses | You can use the raw model for text generation inference |
|
41 |
+
| Primary intended users | Anyone doing text generation inference |
|
42 |
+
| Out-of-scope uses | This model in most cases will need to be fine-tuned for your particular task. The model should not be used to intentionally create hostile or alienating environments for people.|
|
43 |
|
44 |
|
45 |
+
### How to use
|
46 |
|
47 |
Download the model and script by cloning the repository:
|
48 |
```shell
|
49 |
git clone https://huggingface.co/Intel/gpt-j-6B-int8-dynamic
|
50 |
```
|
51 |
|
52 |
+
Then you can do inference based on the model and script 'evaluation.ipynb'.
|
53 |
+
|
54 |
+
## Metrics (Model Performance):
|
55 |
+
| Model | Model Size (GB) | Lambada Acc |
|
56 |
+
|---|:---:|:---:|
|
57 |
+
| FP32 |23|0.7954|
|
58 |
+
| INT8 |6|0.7926|
|
59 |
+
|