---
title: "HalluLens: LLM Hallucination Benchmark (2025)"
description: "Research on LLM hallucination lacks a shared framework: definitions differ between benchmarks, and… 60 references, cited by 44 papers here."
authors: ["Yejin Bang", "Ziwei Ji", "Alan Schelten", "Anthony Hartshorn", "Tara Fowler", "Cheng Zhang", "Nicola Cancedda", "Pascale Fung"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.1176/"
doi: "10.18653/v1/2025.acl-long.1176"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bang-2025-hallulens-llm-hallucination-benchmark/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bang-2025-hallulens-llm-hallucination-benchmark/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / HalluLens: LLM Hallucination Benchmark

# HalluLens: LLM Hallucination Benchmark

**Authors:** [Yejin Bang](https://md.research.elamur.ai/free/en/authors/yejin-bang/), [Ziwei Ji](https://md.research.elamur.ai/free/en/authors/ziwei-ji/), [Alan Schelten](https://md.research.elamur.ai/free/en/authors/alan-schelten/), [Anthony Hartshorn](https://md.research.elamur.ai/free/en/authors/anthony-hartshorn/), Tara Fowler, [Cheng Zhang](https://md.research.elamur.ai/free/en/authors/cheng-zhang/), [Nicola Cancedda](https://md.research.elamur.ai/free/en/authors/nicola-cancedda/) and [Pascale Fung](https://md.research.elamur.ai/free/en/authors/pascale-fung/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.1176/) · [PDF](https://aclanthology.org/2025.acl-long.1176.pdf) · [DOI 10.18653/v1/2025.acl-long.1176](https://doi.org/10.18653/v1/2025.acl-long.1176) · [OpenAlex](https://openalex.org/W4412945166)

**Citations:** 38 citations (OpenAlex, 2026-10-14) · [44](https://md.research.elamur.ai/free/en/papers/bang-2025-hallulens-llm-hallucination-benchmark/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Research on LLM hallucination lacks a shared framework: definitions differ between benchmarks, and hallucination is often treated as the same thing as factuality. The authors separate the two. Factuality is judged against external world knowledge. Hallucination is judged against what the model saw, meaning its training data or the prompt. Under this view there are two categories. Extrinsic hallucinations conflict with the training data. Intrinsic hallucinations conflict with the input context.

The paper argues that no existing benchmark targets extrinsic hallucination specifically. HalluLens fills that gap with three new tasks: PreciseWikiQA, LongWiki and NonExistentRefusal. Their test sets are generated dynamically, which limits leakage into training corpora. For intrinsic hallucination, the benchmark reuses established evaluators. The authors evaluate model families including Llama, Qwen, Gemma, Mistral, GPT-4o and Claude. On questions about invented entities, Llama-3.1-405B-Instruct accepts fake entities least often. Overall, false acceptance tends to mirror how readily each model refuses.

The released codebase gives researchers a reproducible way to track hallucination separately from factual accuracy.

## Key points
- **NonExistentRefusal data and evaluation:**
  - Three LLMs build fictional business, event and product names in round-robin fashion, giving 650 items per model combination and 1,950 in total.
  - The Brave Search API confirms that none of the names exist.
  - Each name is queried with 10 prompt templates.
  - Llama-3.1-70B-Instruct acts as the judge.
- **False acceptance rates (lower is better):** Llama-3.1-405B-Instruct averages 6.88%, compared with 42.31% for GPT-4o and 40.95% for Gemma-2-27b-it.
- **Agreement between subtasks:** Model rankings on the MixedEntities and GeneratedEntities subtasks are significantly correlated, with Kendall's τ = 0.5897.
- **Intrinsic component:** The benchmark adopts the HHEM Leaderboard (summarization), ANAH 2.0 and FaithEval.
- **Scope:** The benchmark covers English only, and the code is available at github.com/facebookresearch/HalluLens.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hong Kong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/hong-kong-university-of-science-and-technology/) — University, Hong Kong, China · 170 works · 26,341 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works

## Cite

Cite as: Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda and Pascale Fung. 2025. HalluLens: LLM Hallucination Benchmark. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{bang2025hallulens,
  title = {{HalluLens: LLM Hallucination Benchmark}},
  author = {Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda and Pascale Fung},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1176},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1176}
}
```

- [← previous EfficientQAT: Efficient Quantization-Aware Training for Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2025-efficientqat-efficient-quantization-aware-training/)
- [next → LLMs Get Lost In Multi-Turn Conversation](https://md.research.elamur.ai/free/en/papers/laban-2025-llms-get-lost-multi-turn/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

60 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Alfonso Amayuelas, Kyle Wong, Liangming Pan, Wenhu Chen, and William Wang. 2023. [Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models](https://md.research.elamur.ai/free/en/papers/amayuelas-2024-knowledge-knowledge-exploring-known-unknowns/). arXiv preprint arXiv:2305.13712.
3. Isabelle Augenstein, Timothy Baldwin, Meeyoung Cha, Tanmoy Chakraborty, Giovanni Luca Ciampaglia, David Corney, Renee DiResta, Emilio Ferrara, Scott Hale, Alon Halevy, et al. 2024. [Factuality challenges in the era of large language models](https://md.research.elamur.ai/free/en/papers/augenstein-2023-factuality-challenges-era-large-language/) and opportunities for fact-checking. Nature Machine Intellige…
4. Forrest Bao, Miaoran Li, Rogger Luo, and Ofer Mendelevitch. 2024. HHEM-2.1-Open. [find](https://scholar.google.com/scholar?q=Forrest+Bao%2C+Miaoran+Li%2C+Rogger+Luo%2C+and+Ofer+Mendelevitch%2E+2024%2E+HHEM%2D2%2E1%2DOpen%2E)
5. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. [Extracting training data from large language models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/). In 30th USENIX Security Symposium (USENIX Security 21), pages 2633–2650.
6. Qinyuan Cheng, Tianxiang Sun, Xiangyang Liu, Wenwei Zhang, Zhangyue Yin, Shimin Li, Linyang Li, Zhengfu He, Kai Chen, and Xipeng Qiu. 2024. Can ai assistants know what they don’t know? In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Qinyuan+Cheng%2C+Tianxiang+Sun%2C+Xiangyang+Liu%2C+Wenwei+Zhang%2C+Zhangyue+Yin%2C+Shimin+Li%2C+Linyang+Li%2C+Zhengfu+He%2C+Kai+Chen%2C+and+Xipeng+Qiu%2E+2024%2E+Can+ai+assistants+know+what+they+don%E2%80%99t+know%3F+In+Forty%2Dfirst+)
7. Qinyuan Cheng, Tianxiang Sun, Wenwei Zhang, Siyin Wang, Xiangyang Liu, Mozhi Zhang, Junliang He, Mianqiu Huang, Zhangyue Yin, Kai Chen, and Xipeng Qiu. 2023. Evaluating hallucinations in chinese large language models. CoRR, abs/2310.03368. — [arXiv:2310.03368](https://arxiv.org/abs/2310.03368) [find](https://scholar.google.com/scholar?q=Qinyuan+Cheng%2C+Tianxiang+Sun%2C+Wenwei+Zhang%2C+Siyin+Wang%2C+Xiangyang+Liu%2C+Mozhi+Zhang%2C+Junliang+He%2C+Mianqiu+Huang%2C+Zhangyue+Yin%2C+Kai+Chen%2C+and+Xipeng+Qiu%2E+2023%2E+Evaluating+hallucinations+in+chinese+large)
8. I Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, Pengfei Liu, et al. 2023. [Factool: Factuality detection in generative ai–a tool augmented framework for multi-task and multi-domain scenarios](https://md.research.elamur.ai/free/en/papers/chern-2023-factool-factuality-detection-generative-ai/). arXiv preprint arXiv:2307.13528.
9. David Chiang and Peter Cholak. 2022. Overcoming a theoretical limitation of self-attention. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7654–7664. [find](https://scholar.google.com/scholar?q=David+Chiang+and+Peter+Cholak%2E+2022%2E+Overcoming+a+theoretical+limitation+of+self%2Dattention%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+P)
10. Euirim Choi. 2023. Goodwiki dataset. [https://www.github.com/euirim/goodwiki](https://www.github.com/euirim/goodwiki). [↗](https://www.github.com/euirim/goodwiki)
11. Bryony Davies. 2020. Freud and his cigars. [find](https://scholar.google.com/scholar?q=Bryony+Davies%2E+2020%2E+Freud+and+his+cigars%2E)
12. Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, and Arman Cohan. 2024. [Investigating data contamination in modern benchmarks for large language models](https://md.research.elamur.ai/free/en/papers/deng-2024-investigating-data-contamination-modern-benchmarks/). In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol…
13. Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran, and Yulia Tsvetkov. 2024a. Don‘t hallucinate, abstain: Identifying LLM knowledge gaps via multi-LLM collaboration. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap… [find](https://scholar.google.com/scholar?q=Shangbin+Feng%2C+Weijia+Shi%2C+Yike+Wang%2C+Wenxuan+Ding%2C+Vidhisha+Balachandran%2C+and+Yulia+Tsvetkov%2E+2024a%2E+Don%E2%80%98t+hallucinate%2C+abstain%3A+Identifying+LLM+knowledge+gaps+via+multi%2DLLM+collaboration%2E+In+Proceed)
14. Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran, and Yulia Tsvetkov. 2024b. Don’t hallucinate, abstain: Identifying llm knowledge gaps via multi-llm collaboration. ACL. [find](https://scholar.google.com/scholar?q=Shangbin+Feng%2C+Weijia+Shi%2C+Yike+Wang%2C+Wenxuan+Ding%2C+Vidhisha+Balachandran%2C+and+Yulia+Tsvetkov%2E+2024b%2E+Don%E2%80%99t+hallucinate%2C+abstain%3A+Identifying+llm+knowledge+gaps+via+multi%2Dllm+collaboration%2E+ACL%2E)
15. Katja Filippova. 2020. Controlled hallucinations: Learning to generate faithfully from noisy data. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 864–870. [find](https://scholar.google.com/scholar?q=Katja+Filippova%2E+2020%2E+Controlled+hallucinations%3A+Learning+to+generate+faithfully+from+noisy+data%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020%2C+pages+864%E2%80%93870%2E)
16. Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. 2023. [RARR: Researching and revising what language models say, using language models](https://md.research.elamur.ai/free/en/papers/gao-2023-rarr-researching-revising-what-language/). In Proceedings of the 61st Annual Meeting of the Associati…
17. Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. 2024. Does fine-tuning LLMs on new knowledge encourage hallucinations? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7765–7784, Miami, Florida, USA. As… [find](https://scholar.google.com/scholar?q=Zorik+Gekhman%2C+Gal+Yona%2C+Roee+Aharoni%2C+Matan+Eyal%2C+Amir+Feder%2C+Roi+Reichart%2C+and+Jonathan+Herzig%2E+2024%2E+Does+fine%2Dtuning+LLMs+on+new+knowledge+encourage+hallucinations%3F+In+Proceedings+of+the+2024+Conf)
18. Yuzhe Gu, Ziwei Ji, Wenwei Zhang, Chengqi Lyu, Dahua Lin, and Kai Chen. 2024. ANAH-v2: Scaling analytical hallucination annotation of large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Yuzhe+Gu%2C+Ziwei+Ji%2C+Wenwei+Zhang%2C+Chengqi+Lyu%2C+Dahua+Lin%2C+and+Kai+Chen%2E+2024%2E+ANAH%2Dv2%3A+Scaling+analytical+hallucination+annotation+of+large+language+models%2E+In+The+Thirty%2Deighth+Annual+Conference+on+N)
19. Michael Hahn. 2020. Theoretical limitations of self-attention in neural sequence models. Transactions of the Association for Computational Linguistics, 8:156– 171. [find](https://scholar.google.com/scholar?q=Michael+Hahn%2E+2020%2E+Theoretical+limitations+of+self%2Dattention+in+neural+sequence+models%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+8%3A156%E2%80%93+171%2E)
20. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. [Teaching machines to read and comprehend](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/). Advances in neural information processing systems, 28.
21. Xiangkun Hu, Dongyu Ru, Lin Qiu, Qipeng Guo, Tianhang Zhang, Yang Xu, Yun Luo, Pengfei Liu, Yue Zhang, and Zheng Zhang. 2024. Refchecker: Reference-based fine-grained hallucination checker and benchmark for large language models. arXiv preprint arXiv:2405.14486. — [arXiv:2405.14486](https://arxiv.org/abs/2405.14486) [find](https://scholar.google.com/scholar?q=Xiangkun+Hu%2C+Dongyu+Ru%2C+Lin+Qiu%2C+Qipeng+Guo%2C+Tianhang+Zhang%2C+Yang+Xu%2C+Yun+Luo%2C+Pengfei+Liu%2C+Yue+Zhang%2C+and+Zheng+Zhang%2E+2024%2E+Refchecker%3A+Reference%2Dbased+fine%2Dgrained+hallucination+checker+and+benchma)
22. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2023. [A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). arXiv preprint arXiv:2311.05232.
23. Integrated Taxonomic Information System ITIS. 2024. Retrieved [month, day, year], from the Integrated Taxonomic Information System (ITIS) on-line database, www.itis.gov, CC0, https://doi.org/10.5066/ F7KH0KBK. [find](https://scholar.google.com/scholar?q=Integrated+Taxonomic+Information+System+ITIS%2E+2024%2E+Retrieved+%5Bmonth%2C+day%2C+year%5D%2C+from+the+Integrated+Taxonomic+Information+System+%28ITIS%29+on%2Dline+database%2C+www%2Eitis%2Egov%2C+CC0%2C+https%3A%2F%2Fdoi%2Eorg%2F10%2E5066%2F+)
24. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. [Atlas: Few-shot learning with retrieval augmented language models](https://md.research.elamur.ai/free/en/papers/izacard-2022-atlas-few-shot-learning-retrieval/). Journal of Machine Learning Research, 24(251):1–43.
25. Alon Jacovi, Andrew Wang, Chris Alberti, Connie Tao, Jon Lipovetz, Kate Olszewska, Lukas Haas, Michelle Liu, Nate Keating, Adam Bloniarz, et al. 2025. [The facts grounding leaderboard: Benchmarking llms’ ability to ground responses to long-form input](https://md.research.elamur.ai/free/en/papers/jacovi-2025-facts-grounding-leaderboard-benchmarking-llms/). arXiv preprint arXiv:2501.03200.
26. Ziwei Ji, Yuzhe Gu, Wenwei Zhang, Chengqi Lyu, Dahua Lin, and Kai Chen. 2024. ANAH: Analytical annotation of hallucinations in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8135–8158, Bangkok, Thailand… [find](https://scholar.google.com/scholar?q=Ziwei+Ji%2C+Yuzhe+Gu%2C+Wenwei+Zhang%2C+Chengqi+Lyu%2C+Dahua+Lin%2C+and+Kai+Chen%2E+2024%2E+ANAH%3A+Analytical+annotation+of+hallucinations+in+large+language+models%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+A)
27. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. [Survey of hallucination in natural language generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/). ACM Computing Surveys, 55(12):1–38.
28. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611, Vancouver…
29. Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. [Large language models struggle to learn long-tail knowledge](https://md.research.elamur.ai/free/en/papers/kandpal-2022-large-language-models-struggle-learn/). In International Conference on Machine Learning, pages 15696–15707. PMLR.
30. Jungo Kasai, Keisuke Sakaguchi, yoichi takahashi, Ronan Le Bras, Akari Asai, Xinyan Velocity Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, and Kentaro Inui. 2023. [Realtime QA: What’s the answer right now?](https://md.research.elamur.ai/free/en/papers/kasai-2022-realtime-qa-s-answer-right/) In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks…
31. Matthew H Kim, Sammy F Ahmed, and Frederick J Morrison. 2021. The effects of kindergarten and first grade schooling on executive function and academic skill development: Evidence from a school cutoff design. Frontiers in Psychology, 11:607973. [find](https://scholar.google.com/scholar?q=Matthew+H+Kim%2C+Sammy+F+Ahmed%2C+and+Frederick+J+Morrison%2E+2021%2E+The+effects+of+kindergarten+and+first+grade+schooling+on+executive+function+and+academic+skill+development%3A+Evidence+from+a+school+cutoff+)
32. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [Natural…](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
33. Laboratory for Web Algorithmics Università degli Studi di Milano LAW. 2024. The open wikipedia ranking 2024. [find](https://scholar.google.com/scholar?q=Laboratory+for+Web+Algorithmics+Universit%C3%A0+degli+Studi+di+Milano+LAW%2E+2024%2E+The+open+wikipedia+ranking+2024%2E)
34. Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, and Sanjiv Kumar. 2023a. [Large language models with controllable working memory](https://md.research.elamur.ai/free/en/papers/li-2023-large-language-models-controllable-working/). In Findings of the Association for Computational Linguistics: ACL 2023, pages 1774–1793.
35. Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2024. [The dawn after the dark: An empirical study on factuality hallucination in large language models](https://md.research.elamur.ai/free/en/papers/li-2024-dawn-after-dark-empirical-study/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long…
36. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023b. [Halueval: A large-scale hallucination evaluation benchmark for large language models](https://md.research.elamur.ai/free/en/papers/li-2023-halueval-large-scale-hallucination-evaluation/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6449–6464.
37. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252.
38. Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, et al. 2024. [Mitigating the alignment tax of rlhf](https://md.research.elamur.ai/free/en/papers/lin-2024-mitigating-alignment-tax-rlhf/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 580–606.
39. Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, and William B Dolan. 2022. [A token-level reference-free hallucination detection benchmark for free-form text generation](https://md.research.elamur.ai/free/en/papers/liu-2022-token-level-reference-free-hallucination/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo…
40. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023a. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (… [find](https://scholar.google.com/scholar?q=Alex+Mallen%2C+Akari+Asai%2C+Victor+Zhong%2C+Rajarshi+Das%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023a%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+me)
41. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023b. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (… [find](https://scholar.google.com/scholar?q=Alex+Mallen%2C+Akari+Asai%2C+Victor+Zhong%2C+Rajarshi+Das%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023b%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+me)
42. Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004–9017, Singapore. Association for Computation… [find](https://scholar.google.com/scholar?q=Potsawee+Manakul%2C+Adian+Liusie%2C+and+Mark+Gales%2E+2023%2E+SelfCheckGPT%3A+Zero%2Dresource+black%2Dbox+hallucination+detection+for+generative+large+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empir)
43. Jean-Baptiste Michel, Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K Gray, Google Books Team, Joseph P Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, et al. 2011. Quantitative analysis of culture using millions of digitized books. science, 331(6014):176–182. [find](https://scholar.google.com/scholar?q=Jean%2DBaptiste+Michel%2C+Yuan+Kui+Shen%2C+Aviva+Presser+Aiden%2C+Adrian+Veres%2C+Matthew+K+Gray%2C+Google+Books+Team%2C+Joseph+P+Pickett%2C+Dale+Hoiberg%2C+Dan+Clancy%2C+Peter+Norvig%2C+et+al%2E+2011%2E+Quantitative+analysis+)
44. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. [FActScore: Fine-grained atomic evaluation of factual precision in long form text generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/). In Proceedings of the 2023 Conference on Empirical Methods in Natural L…
45. Yifei Ming, Senthil Purushwalkam, Shrey Pandit, Zixuan Ke, Xuan-Phi Nguyen, Caiming Xiong, and Shafiq Joty. 2024. Faitheval: Can your language model stay faithful to context, even if "the moon is made of marshmallows". arXiv. [find](https://scholar.google.com/scholar?q=Yifei+Ming%2C+Senthil+Purushwalkam%2C+Shrey+Pandit%2C+Zixuan+Ke%2C+Xuan%2DPhi+Nguyen%2C+Caiming+Xiong%2C+and+Shafiq+Joty%2E+2024%2E+Faitheval%3A+Can+your+language+model+stay+faithful+to+context%2C+even+if+%22the+moon+is+made)
46. Jio Oh, Soyeon Kim, Junseok Seo, Jindong Wang, Ruochen Xu, Xing Xie, and Steven Euijong Whang. 2024. ERBench: An entity-relationship based automatically verifiable hallucination benchmark for large language models. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and… [find](https://scholar.google.com/scholar?q=Jio+Oh%2C+Soyeon+Kim%2C+Junseok+Seo%2C+Jindong+Wang%2C+Ruochen+Xu%2C+Xing+Xie%2C+and+Steven+Euijong+Whang%2E+2024%2E+ERBench%3A+An+entity%2Drelationship+based+automatically+verifiable+hallucination+benchmark+for+large+la)
47. Yasumasa Onoe, Michael Zhang, Eunsol Choi, and Greg Durrett. 2022. [Entity cloze by date: What lms know about unseen entities](https://md.research.elamur.ai/free/en/papers/onoe-2022-entity-cloze-date-what-lms/). In Findings of the Association for Computational Linguistics: NAACL 2022, pages 693–702.
48. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Ja… — [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
49. OpenCompass. 2024. Github repository for anah: Analytical annotation of hallucinations in large language models. [find](https://scholar.google.com/scholar?q=OpenCompass%2E+2024%2E+Github+repository+for+anah%3A+Analytical+annotation+of+hallucinations+in+large+language+models%2E)
50. Daniel Park. 2023. Open-llm-leaderboard-report. [https://github.com/dsdanielpark/Open-LLM-Lea](https://github.com/dsdanielpark/Open-LLM-Lea) derboard-Report. [↗](https://github.com/dsdanielpark/Open-LLM-Lea)
51. Vivek Tiwari Shudhanshu Singh, Vishal Thakur. 2024. 250k medicines usage, side effects and substitutes. [find](https://scholar.google.com/scholar?q=Vivek+Tiwari+Shudhanshu+Singh%2C+Vishal+Thakur%2E+2024%2E+250k+medicines+usage%2C+side+effects+and+substitutes%2E)
52. Yixiao Song, Yekyung Kim, and Mohit Iyyer. 2024. [VeriScore: Evaluating the factuality of verifiable claims in long-form text generation](https://md.research.elamur.ai/free/en/papers/song-2024-veriscore-evaluating-factuality-verifiable-claims/). In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9447–9474, Miami, Florida, USA. Association for Computational Linguistics.
53. YuHong Sun, Zhangyue Yin, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Hui Zhao. 2024. Benchmarking hallucination in large language models based on unanswerable math word problem. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LRE… [find](https://scholar.google.com/scholar?q=YuHong+Sun%2C+Zhangyue+Yin%2C+Qipeng+Guo%2C+Jiawen+Wu%2C+Xipeng+Qiu%2C+and+Hui+Zhao%2E+2024%2E+Benchmarking+hallucination+in+large+language+models+based+on+unanswerable+math+word+problem%2E+In+Proceedings+of+the+2024)
54. Panarat Thepgumpanat and Panu Wongcha-um. 2024. Thailand to ban recreational cannabis use by year-end, health minister says. Reuters. [find](https://scholar.google.com/scholar?q=Panarat+Thepgumpanat+and+Panu+Wongcha%2Dum%2E+2024%2E+Thailand+to+ban+recreational+cannabis+use+by+year%2Dend%2C+health+minister+says%2E+Reuters%2E)
55. Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari Morcos. 2023. [D4: Improving llm pretraining via document de-duplication and diversification](https://md.research.elamur.ai/free/en/papers/tirumala-2023-d4-improving-llm-pretraining-document/). Advances in Neural Information Processing Systems, 36:53983–53995.
56. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. [Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition](https://md.research.elamur.ai/free/en/papers/sang-2003-introduction-conll-2003-shared-task/). In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142– 147.
57. Christian Tomani, Kamalika Chaudhuri, Ivan Evtimov, Daniel Cremers, and Mark Ibrahim. 2024. Uncertainty-based abstention in llms improves safety and reduces hallucinations. arXiv preprint arXiv:2404.10960. — [arXiv:2404.10960](https://arxiv.org/abs/2404.10960) [find](https://scholar.google.com/scholar?q=Christian+Tomani%2C+Kamalika+Chaudhuri%2C+Ivan+Evtimov%2C+Daniel+Cremers%2C+and+Mark+Ibrahim%2E+2024%2E+Uncertainty%2Dbased+abstention+in+llms+improves+safety+and+reduces+hallucinations%2E+arXiv+preprint+arXiv%3A2404%2E1)
58. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
59. Vectara. 2024. Hallucination leaderboard. https://gi thub.com/vectara/hallucination-leaderboa rd?tab=readme-ov-file. [find](https://scholar.google.com/scholar?q=Vectara%2E+2024%2E+Hallucination+leaderboard%2E+https%3A%2F%2Fgi+thub%2Ecom%2Fvectara%2Fhallucination%2Dleaderboa+rd%3Ftab%3Dreadme%2Dov%2Dfile%2E)
60. Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, and Thang Luong. 2024. Fresh-LLMs: Refreshing large language models with search engine augmentation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 136… [find](https://scholar.google.com/scholar?q=Tu+Vu%2C+Mohit+Iyyer%2C+Xuezhi+Wang%2C+Noah+Constant%2C+Jerry+Wei%2C+Jason+Wei%2C+Chris+Tar%2C+Yun%2DHsuan+Sung%2C+Denny+Zhou%2C+Quoc+Le%2C+and+Thang+Luong%2E+2024%2E+Fresh%2DLLMs%3A+Refreshing+large+language+models+with+search+en)

*Updated 31 October 2026*