|
--- |
|
license: mit |
|
datasets: |
|
- web_nlg |
|
language: |
|
- en |
|
--- |
|
# Model card for Inria-CEDAR/FactSpotter-DeBERTaV3-Large |
|
|
|
## Model description |
|
|
|
This model is related to the paper **"FactSpotter: Evaluating the Factual Faithfulness of Graph-to-Text Generation"**. |
|
|
|
Given a triple of format "subject | predicate | object" and a text, the model determines if the triple is present in the text. |
|
|
|
Different from the paper using ELECTRA, this model is finetuned on DeBERTaV3. |
|
|
|
## How to use the model |
|
|
|
``` |
|
from transformers import AutoTokenizer, AutoModelForSequenceClassification |
|
|
|
def sentence_cls_score(input_strings, predicate_cls_model, predicate_cls_tokenizer): |
|
tokenized_cls_input = predicate_cls_tokenizer(input_strings, truncation=True, padding=True, |
|
return_token_type_ids=True) |
|
input_ids = torch.Tensor(tokenized_cls_input['input_ids']).long().to(torch.device("cuda")) |
|
token_type_ids = torch.Tensor(tokenized_cls_input['token_type_ids']).long().to(torch.device("cuda")) |
|
attention_mask = torch.Tensor(tokenized_cls_input['attention_mask']).long().to(torch.device("cuda")) |
|
prev_cls_output = predicate_cls_model(input_ids, attention_mask=attention_mask, token_type_ids=token_type_ids) |
|
softmax_cls_output = torch.softmax(prev_cls_output.logits, dim=1, ) |
|
return softmax_cls_output |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("Inria-CEDAR/FactSpotter-DeBERTaV3-Large") |
|
model = AutoModelForSequenceClassification.from_pretrained("Inria-CEDAR/FactSpotter-DeBERTaV3-Large") |
|
|
|
# pairs of texts (as premises) and triples (as hypotheses) |
|
cls_texts = [("the aarhus is the airport of aarhus, denmark", "aarhus airport | city served | aarhus, denmark"), |
|
("aarhus airport is 25.0 metres above the sea level", "aarhus airport | elevation above the sea level | 1174")] |
|
cls_scores = sentence_cls_score(cls_texts, model, tokenizer) |
|
# Dimensions: 0-entailment, 1-neutral, 2-contradiction |
|
label_names = ["entailment", "neutral", "contradiction"] |
|
``` |
|
## Citation |
|
If the model is useful to you, please cite the paper |
|
|
|
``` |
|
@inproceedings{zhang:hal-04257838, |
|
TITLE = {{FactSpotter: Evaluating the Factual Faithfulness of Graph-to-Text Generation}}, |
|
AUTHOR = {Zhang, Kun and Balalau, Oana and Manolescu, Ioana}, |
|
URL = {https://hal.science/hal-04257838}, |
|
BOOKTITLE = {{Findings of EMNLP 2023 - Conference on Empirical Methods in Natural Language Processing}}, |
|
ADDRESS = {Singapore, Singapore}, |
|
YEAR = {2023}, |
|
MONTH = Dec, |
|
KEYWORDS = {Graph-to-Text Generation ; Factual Faithfulness ; Constrained Text Generation}, |
|
PDF = {https://hal.science/hal-04257838/file/_EMNLP_2023__Evaluating_the_Factual_Faithfulness_of_Graph_to_Text_Generation_Camera.pdf}, |
|
HAL_ID = {hal-04257838}, |
|
HAL_VERSION = {v1}, |
|
} |
|
``` |
|
|
|
## Questions |
|
If you have some questions, please contact through my email [email protected] or [email protected] |
