bibtext: @inproceedings{
  mishra2024finegrained,
  title={Fine-grained Hallucination Detection and Editing for Language Models},
  author={Abhika Mishra and Akari Asai and Vidhisha Balachandran and Yizhong Wang and Graham Neubig and Yulia Tsvetkov and Hannaneh Hajishirzi},
  booktitle={First Conference on Language Modeling},
  year={2024},
  url={https://openreview.net/forum?id=dJMTn3QOWO}
  }
abstract: Large language models (LMs) are prone to generate factual errors, which are often called hallucinations. In this paper, we introduce a comprehensive taxonomy of hallucinations and argue that hallucinations manifest in diverse forms, each requiring varying degrees of careful assessments to verify factuality. We propose a novel task of automatic fine-grained hallucination detection and construct a new evaluation benchmark, FavaBench, that includes about one thousand fine-grained human judgments on three LM outputs across various domains. Our analysis reveals that ChatGPT and Llama2-Chat (70B, 7B) exhibit diverse types of hallucinations in the majority of their outputs in information-seeking scenarios. We train FAVA, a retrieval-augmented LM by carefully creating synthetic data to detect and correct fine-grained hallucinations. On our benchmark, our automatic and human evaluations show that FAVA significantly outperforms ChatGPT and GPT-4 on fine-grained hallucination detection, and edits suggested by FAVA improve the factuality of LM-generated text.
title: Fine-grained Hallucination Detection and Editing for Language Models
authors: Abhika Mishra Akari Asai Vidhisha Balachandran Yizhong Wang Graham Neubig Yulia Tsvetkov Hannaneh Hajishirzi