EDGAR-T5-Large
T5 Large model finetuned on EDGAR10-Q dataset
You may want to check out
- Our paper: CONTEXT-NER: Contextual Phrase Generation at Scale
- GitHub: Click Here
Direct Use
It is possible to use this model to generate text, which is useful for experimentation and understanding its capabilities. It should not be directly used for production or work that may directly impact people.
How to Use
You can very easily load the models with Transformers, instead of downloading them manually. The T5-Large model is the backbone of our model. Here is how to use the model in PyTorch:
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("him1411/EDGAR-T5-Large")
model = AutoModelForSeq2SeqLM.from_pretrained("him1411/EDGAR-T5-Large")
Or just clone the model repo
git lfs install
git clone https://huggingface.co./him1411/EDGAR-T5-Large
Inference Example
Here, we provide an example for the "ContextNER" task. Below is an example of one instance.
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("him1411/EDGAR-T5-Large")
model = AutoModelForSeq2SeqLM.from_pretrained("him1411/EDGAR-T5-Large")
# Input shows how we have appended instruction from our file for HoC dataset with instance.
input = "14.5 years . The definite lived intangible assets related to the contracts and trade names had estimated weighted average useful lives of 5.9 years and 14.5 years, respectively, at acquisition."
tokenized_input= tokenizer(input)
# Ideal output for this input is 'Definite lived intangible assets weighted average remaining useful life'
output = model(tokenized_input)
Results on Dowstream datasets
EDGAR-T5-Large was finetuned on some downstream datasets to get better results than T5 large. BloombergGPT 50B was used as baseline.
Dataset | Bloomberg GPT 50B | T5 Large | Edgar T5 Large |
---|---|---|---|
FiQA SA | 75.07 | 74.89 | 80.42 |
FPB | 51.07 | 55.77 | 79.69 |
Headline | 82.20 | 90.55 | 93.55 |
BibTeX Entry and Citation Info
If you are using our model, please cite our paper:
@article{gupta2021context,
title={Context-NER: Contextual Phrase Generation at Scale},
author={Gupta, Himanshu and Verma, Shreyas and Kumar, Tarun and Mishra, Swaroop and Agrawal, Tamanna and Badugu, Amogh and Bhatt, Himanshu Sharad},
journal={arXiv preprint arXiv:2109.08079},
year={2021}
}
- Downloads last month
- 12