MengniWang commited on
Commit
f41827e
1 Parent(s): 895359e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -8
README.md CHANGED
@@ -16,7 +16,7 @@ tags:
16
  - neural-compressor
17
  ---
18
 
19
- # INT8 GPT-J 6B
20
 
21
  GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
22
 
@@ -25,19 +25,35 @@ This int8 ONNX model is generated by [neural-compressor](https://github.com/inte
25
  python -m transformers.onnx --model=EleutherAI/gpt-j-6B onnx_gptj/ --framework pt --opset 13 --feature=causal-lm-with-past
26
  ```
27
 
28
- ## Test result
 
 
 
 
 
 
 
 
29
 
30
- | |INT8|FP32|
31
- |---|:---:|:---:|
32
- | **Lambada Acc** |0.7926|0.7954|
33
- | **Model size (GB)** |6|23|
 
34
 
35
 
36
- ## How to use
37
 
38
  Download the model and script by cloning the repository:
39
  ```shell
40
  git clone https://huggingface.co/Intel/gpt-j-6B-int8-dynamic
41
  ```
42
 
43
- Then you can do inference based on the model and script 'evaluation.ipynb'.
 
 
 
 
 
 
 
 
16
  - neural-compressor
17
  ---
18
 
19
+ ## Model Details: INT8 GPT-J 6B
20
 
21
  GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
22
 
 
25
  python -m transformers.onnx --model=EleutherAI/gpt-j-6B onnx_gptj/ --framework pt --opset 13 --feature=causal-lm-with-past
26
  ```
27
 
28
+ | Model Detail | Description |
29
+ | ----------- | ----------- |
30
+ | Model Authors - Company | Intel |
31
+ | Date | April 10, 2022 |
32
+ | Version | 1 |
33
+ | Type | Text Generation |
34
+ | Paper or Other Resources | - |
35
+ | License | Apache 2.0 |
36
+ | Questions or Comments | [Community Tab](https://huggingface.co/Intel/gpt-j-6B-int8-dynamic/discussions)|
37
 
38
+ | Intended Use | Description |
39
+ | ----------- | ----------- |
40
+ | Primary intended uses | You can use the raw model for text generation inference |
41
+ | Primary intended users | Anyone doing text generation inference |
42
+ | Out-of-scope uses | This model in most cases will need to be fine-tuned for your particular task. The model should not be used to intentionally create hostile or alienating environments for people.|
43
 
44
 
45
+ ### How to use
46
 
47
  Download the model and script by cloning the repository:
48
  ```shell
49
  git clone https://huggingface.co/Intel/gpt-j-6B-int8-dynamic
50
  ```
51
 
52
+ Then you can do inference based on the model and script 'evaluation.ipynb'.
53
+
54
+ ## Metrics (Model Performance):
55
+ | Model | Model Size (GB) | Lambada Acc |
56
+ |---|:---:|:---:|
57
+ | FP32 |23|0.7954|
58
+ | INT8 |6|0.7926|
59
+