Post
408
FACTS is a great paper from
@GoogleDeepMind
on measuring the factuality of LLM outputs. You can now download their prompt templates from
@huggingface
to improve LLM-based fact-checking yourself!
๐ The paper introduces the FACTS Grounding benchmark for evaluating the factuality of LLM outputs.
๐ค Fact-checking is automated by an ensemble of LLM judges that verify if a response is fully grounded in a factual reference document.
๐งช The authors tested different prompt templates on held-out data to ensure their generalization.
๐ It's highly educational to read these templates to learn how frontier labs design prompts and understand their limitations.
๐พ You can now download and reuse these prompt templates via the prompt-templates library!
๐ The library simplifies sharing prompt templates on the HF hub or locally via standardized YAML files. Letโs make LLM work more transparent and reproducible by sharing more templates like this!
Links ๐
- prompt-templates docs: https://moritzlaurer.github.io/prompt_templates/
- all templates on the HF Hub: MoritzLaurer/facts-grounding-prompts
- FACTS paper: https://storage.googleapis.com/deepmind-media/FACTS/FACTS_grounding_paper.pdf
๐ The paper introduces the FACTS Grounding benchmark for evaluating the factuality of LLM outputs.
๐ค Fact-checking is automated by an ensemble of LLM judges that verify if a response is fully grounded in a factual reference document.
๐งช The authors tested different prompt templates on held-out data to ensure their generalization.
๐ It's highly educational to read these templates to learn how frontier labs design prompts and understand their limitations.
๐พ You can now download and reuse these prompt templates via the prompt-templates library!
๐ The library simplifies sharing prompt templates on the HF hub or locally via standardized YAML files. Letโs make LLM work more transparent and reproducible by sharing more templates like this!
Links ๐
- prompt-templates docs: https://moritzlaurer.github.io/prompt_templates/
- all templates on the HF Hub: MoritzLaurer/facts-grounding-prompts
- FACTS paper: https://storage.googleapis.com/deepmind-media/FACTS/FACTS_grounding_paper.pdf
