GPT-J 6B
Model Description
GPT-J 6B is a transformer model trained using Ben Wang's Mesh Transformer JAX. "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
Original implementation
Follow this link to see the original implementation.
How to use
Download the model by cloning the repository via git clone https://huggingface.co./OWG/gpt-j-6B.
Then you can use the model with the following code:
from onnxruntime import InferenceSession, SessionOptions, GraphOptimizationLevel
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B")
options = SessionOptions()
options.graph_optimization_level = GraphOptimizationLevel.ORT_ENABLE_ALL
session = InferenceSession("path/to/model.onnx", sess_options=options)
session.disable_fallback()
TODO