---
title: "SAC3 — paper by Zhang et al. (2023) — summary and ideas"
description: "The authors show this assumption breaks down in two ways. 44 references, cited by 25 papers here."
authors: ["Jiaxin Zhang", "Zhuohang Li", "Kamalika Das", "Bradley A. Malin", "Sricharan Kumar"]
year: 2023
venue: "arXiv:2311.01740"
source: "https://arxiv.org/abs/2311.01740"
doi: "10.48550/arxiv.2311.01740"
arxiv: "2311.01740"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2023-sac3-reliable-hallucination-detection-black/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2023-sac3-reliable-hallucination-detection-black/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/9/) / SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency

# SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency

**Authors:** [Jiaxin Zhang](https://md.research.elamur.ai/free/en/authors/jiaxin-zhang/), [Zhuohang Li](https://md.research.elamur.ai/free/en/authors/zhuohang-li/), Kamalika Das, [Bradley A. Malin](https://md.research.elamur.ai/free/en/authors/bradley-a-malin/) and Sricharan Kumar · **Year:** 2023 · **Venue:** arXiv:2311.01740 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2311.01740) · [PDF](https://arxiv.org/pdf/2311.01740) · [DOI 10.48550/arxiv.2311.01740](https://doi.org/10.48550/arxiv.2311.01740) · [arXiv 2311.01740](https://arxiv.org/abs/2311.01740) · [OpenAlex](https://openalex.org/W4388444554)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [25](https://md.research.elamur.ai/free/en/papers/zhang-2023-sac3-reliable-hallucination-detection-black/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Black-box LMs such as ChatGPT expose no token log-probabilities, so hallucination detectors often depend on sampling and self-consistency: if repeated answers agree, the output is assumed factual. The authors show this assumption breaks down in two ways. Models can repeat the same wrong answer for a given phrasing (question-level hallucination), and a particular model may hallucinate where another model, sometimes a smaller one, answers correctly (model-level hallucination).

SAC3 adds two cross-checks to self-consistency:
- comparing answers to semantically equivalent rephrasings of the question;
- comparing the target model's answers with those of a separate verifier LM.

It then merges the consistency signals into one hallucination score. Evaluation covers two classification QA sets (Prime Number, Senator Search) and two open-domain generation sets (HotpotQA-halu, NQ-open-halu). SAC3 clearly outperforms the self-consistency baseline SC2, reaching AUROC of 99.4/97.0 on the classification sets and 88.0/77.2 on the generation sets. This matters because it improves hallucination detection using only API access, with no external knowledge sources.

## Key points
- **Generation QA gains:** on HotpotQA-halu and NQ-open-halu, SAC3 outperforms SC2 (gpt-3.5-turbo) by +13.8 and +6.7 AUROC. The best results use Guanaco-33b as the verifier; Falcon-7b was also tested.
- **Variants:** the method comes in SAC3-Q (cross-question), SAC3-QM (cross-model plus cross-question) and SAC3-all. SAC3-all performs well on both generation datasets.
- **Other target models:** in a balanced setting, SAC3-Q beats SC2 for gpt-3.5-turbo, gpt-4 and chat-bison (PaLM 2). For example, Prime Number accuracy with gpt-3.5-turbo rises from 48.2 to 93.2.
- **Number of rephrasings:** gains level off after about 5 perturbed questions, and 2–4 already give reasonable results. A weight λ lets users adjust how much the verifier LM influences the score.
- **Cost and code:** SAC3-all needs n_s target calls plus 2×n_s verifier calls, compared with n_s calls for SC2. Code is available at github.com/intuit/sac3.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Vanderbilt University](https://md.research.elamur.ai/free/en/universities/vanderbilt-university/) — University, Nashville, United States · 14 works · 1,413 citations
- [Intuit](https://md.research.elamur.ai/free/en/companies/intuit/) — Company, Mountain View, United States · 5 works · 944 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Jiaxin Zhang, Zhuohang Li, Kamalika Das, Bradley A. Malin and Sricharan Kumar. 2023. SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency. arXiv:2311.01740.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zhang2023sac3,
  title = {{SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency}},
  author = {Jiaxin Zhang, Zhuohang Li, Kamalika Das, Bradley A. Malin and Sricharan Kumar},
  year = {2023},
  doi = {10.48550/arxiv.2311.01740},
  eprint = {2311.01740},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2311.01740}
}
```

- [← previous Pushing the Limits of ChatGPT on NLP Tasks](https://md.research.elamur.ai/free/en/papers/sun-2023-pushing-limits-chatgpt-nlp-tasks/)
- [next → ArTST: Arabic Text and Speech Transformer](https://md.research.elamur.ai/free/en/papers/toyin-2023-artst-arabic-text-speech-transformer/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/9/)

## References

44 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ayush Agrawal, Lester Mackey, and Adam Tauman Kalai. 2023. Do language models know when they’re hallucinating references? arXiv preprint arXiv:2305.18248. [find](https://scholar.google.com/scholar?q=Ayush+Agrawal%2C+Lester+Mackey%2C+and+Adam+Tauman+Kalai%2E+2023%2E+Do+language+models+know+when+they%E2%80%99re+hallucinating+references%3F+arXiv+preprint+arXiv%3A2305%2E18248%2E)
2. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Merouane Debbah, Etienne Goffinet, Daniel Heslow, Julien Launay, Quentin Malartic, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. Falcon-40B: an open large language model with state-of-t… [find](https://scholar.google.com/scholar?q=Ebtesam+Almazrouei%2C+Hamza+Alobeidli%2C+Abdulaziz+Alshamsi%2C+Alessandro+Cappelli%2C+Ruxandra+Cojocaru%2C+Merouane+Debbah%2C+Etienne+Goffinet%2C+Daniel+Heslow%2C+Julien+Launay%2C+Quentin+Malartic%2C+Badreddine+Noune%2C+Ba)
3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
4. Meng Cao, Yue Dong, and Jackie Chi Kit Cheung. 2022. [Hallucinated but factual! inspecting the factuality of hallucinations in abstractive summarization](https://md.research.elamur.ai/free/en/papers/cao-2022-hallucinated-but-factual-inspecting-factuality/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3340–3354.
5. Angelica Chen, Jason Phang, Alicia Parrish, Vishakh Padmakumar, Chen Zhao, Samuel R Bowman, and Kyunghyun Cho. 2023. [Two failures of self-consistency in the multi-step reasoning of llms](https://md.research.elamur.ai/free/en/papers/chen-2023-two-failures-self-consistency-multi/). arXiv preprint arXiv:2305.14279.
6. Jiuhai Chen and Jonas Mueller. 2023. Quantifying uncertainty in answers from any language model via intrinsic and extrinsic confidence assessment. arXiv preprint arXiv:2308.16175. — [arXiv:2308.16175](https://arxiv.org/abs/2308.16175) [find](https://scholar.google.com/scholar?q=Jiuhai+Chen+and+Jonas+Mueller%2E+2023%2E+Quantifying+uncertainty+in+answers+from+any+language+model+via+intrinsic+and+extrinsic+confidence+assessment%2E+arXiv+preprint+arXiv%3A2308%2E16175%2E+%E2%80%94+%5BarXiv%3A2308%2E16175%5D)
7. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. [Palm: Scaling language modeling with pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/). arXiv preprint arXiv:2204.02311.
8. Jeremy R Cole, Michael JQ Zhang, Daniel Gillick, Julian Martin Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. 2023. [Selectively answering ambiguous questions](https://md.research.elamur.ai/free/en/papers/cole-2023-selectively-answering-ambiguous-questions/). arXiv preprint arXiv:2305.14613.
9. Souvik Das, Sougata Saha, and Rohini K Srihari. 2023. Diving deep into modes of fact hallucinations in dialogue systems. arXiv preprint arXiv:2301.04449. [find](https://scholar.google.com/scholar?q=Souvik+Das%2C+Sougata+Saha%2C+and+Rohini+K+Srihari%2E+2023%2E+Diving+deep+into+modes+of+fact+hallucinations+in+dialogue+systems%2E+arXiv+preprint+arXiv%3A2301%2E04449%2E)
10. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. [Qlora: Efficient finetuning of quantized llms](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/). arXiv preprint arXiv:2305.14314.
11. Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. 2023. [Chain-of-verification reduces hallucination in large language models](https://md.research.elamur.ai/free/en/papers/dhuliawala-2023-chain-verification-reduces-hallucination-large/). arXiv preprint arXiv:2309.11495.
12. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012–1031. [find](https://scholar.google.com/scholar?q=Yanai+Elazar%2C+Nora+Kassner%2C+Shauli+Ravfogel%2C+Abhilasha+Ravichander%2C+Eduard+Hovy%2C+Hinrich+Sch%C3%BCtze%2C+and+Yoav+Goldberg%2E+2021%2E+Measuring+and+improving+consistency+in+pretrained+language+models%2E+Transactio)
13. Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, and Idan Szpektor. 2023. [Trueteacher: Learning factual consistency evaluation with large language models](https://md.research.elamur.ai/free/en/papers/gekhman-2023-trueteacher-learning-factual-consistency-evaluation/). arXiv preprint arXiv:2305.11171.
14. Myeongjun Jang, Deuk Sin Kwon, and Thomas Lukasiewicz. 2022. Becel: Benchmark for consistency evaluation of language models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3680–3696. [find](https://scholar.google.com/scholar?q=Myeongjun+Jang%2C+Deuk+Sin+Kwon%2C+and+Thomas+Lukasiewicz%2E+2022%2E+Becel%3A+Benchmark+for+consistency+evaluation+of+language+models%2E+In+Proceedings+of+the+29th+International+Conference+on+Computational+Lingui)
15. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. [Survey of hallucination in natural language generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/). ACM Computing Surveys, 55(12):1–38.
16. Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wilie, Min Zeng, and Pascale Fung. 2022. Rho: Reducing hallucination in open-domain dialogues with knowledge grounding. arXiv preprint arXiv:2212.01588. [find](https://scholar.google.com/scholar?q=Ziwei+Ji%2C+Zihan+Liu%2C+Nayeon+Lee%2C+Tiezheng+Yu%2C+Bryan+Wilie%2C+Min+Zeng%2C+and+Pascale+Fung%2E+2022%2E+Rho%3A+Reducing+hallucination+in+open%2Ddomain+dialogues+with+knowledge+grounding%2E+arXiv+preprint+arXiv%3A2212%2E01)
17. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664. [find](https://scholar.google.com/scholar?q=Lorenz+Kuhn%2C+Yarin+Gal%2C+and+Sebastian+Farquhar%2E+2023%2E+Semantic+uncertainty%3A+Linguistic+invariances+for+uncertainty+estimation+in+natural+language+generation%2E+arXiv+preprint+arXiv%3A2302%2E09664%2E)
18. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. [Natural questions: a benchmark for question answering research](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/). Transactions of the Association for Computational Linguistic…
19. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. [Latent retrieval for weakly supervised open domain question answering](https://md.research.elamur.ai/free/en/papers/lee-2019-latent-retrieval-weakly-supervised-open/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6086–6096.
20. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023a. [Halueval: A large-scale hallucination evaluation benchmark for large language models](https://md.research.elamur.ai/free/en/papers/li-2023-halueval-large-scale-hallucination-evaluation/). arXiv e-prints, pages arXiv–2305.
21. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2023b. [Inference-time intervention: Eliciting truthful answers from a language model](https://md.research.elamur.ai/free/en/papers/li-2023-inference-time-intervention-eliciting-truthful/). arXiv preprint arXiv:2306.03341.
22. Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2023. Generating with confidence: Uncertainty quantification for black-box large language models. arXiv preprint arXiv:2305.19187. [find](https://scholar.google.com/scholar?q=Zhen+Lin%2C+Shubhendu+Trivedi%2C+and+Jimeng+Sun%2E+2023%2E+Generating+with+confidence%3A+Uncertainty+quantification+for+black%2Dbox+large+language+models%2E+arXiv+preprint+arXiv%3A2305%2E19187%2E)
23. Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, and Bill Dolan. 2021. [A token-level reference-free hallucination detection benchmark for free-form text generation](https://md.research.elamur.ai/free/en/papers/liu-2022-token-level-reference-free-hallucination/). arXiv preprint arXiv:2104.08704.
24. Potsawee Manakul, Adian Liusie, and Mark JF Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896. [find](https://scholar.google.com/scholar?q=Potsawee+Manakul%2C+Adian+Liusie%2C+and+Mark+JF+Gales%2E+2023%2E+Selfcheckgpt%3A+Zero%2Dresource+black%2Dbox+hallucination+detection+for+generative+large+language+models%2E+arXiv+preprint+arXiv%3A2303%2E08896%2E)
25. Eric Mitchell, Joseph J Noh, Siyan Li, William S Armstrong, Ananth Agarwal, Patrick Liu, Chelsea Finn, and Christopher D Manning. 2022. Enhancing self-consistency and performance of pre-trained language models through natural language inference. arXiv preprint arXiv:2211.11875. [find](https://scholar.google.com/scholar?q=Eric+Mitchell%2C+Joseph+J+Noh%2C+Siyan+Li%2C+William+S+Armstrong%2C+Ananth+Agarwal%2C+Patrick+Liu%2C+Chelsea+Finn%2C+and+Christopher+D+Manning%2E+2022%2E+Enhancing+self%2Dconsistency+and+performance+of+pre%2Dtrained+langua)
26. Niels Mündler, Jingxuan He, Slobodan Jenko, and Martin Vechev. 2023. [Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation](https://md.research.elamur.ai/free/en/papers/mundler-2023-self-contradictory-hallucinations-large-language/). arXiv preprint arXiv:2305.15852.
27. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306… [find](https://scholar.google.com/scholar?q=Guilherme+Penedo%2C+Quentin+Malartic%2C+Daniel+Hesslow%2C+Ruxandra+Cojocaru%2C+Alessandro+Cappelli%2C+Hamza+Alobeidli%2C+Baptiste+Pannier%2C+Ebtesam+Almazrouei%2C+and+Julien+Launay%2E+2023%2E+The+RefinedWeb+dataset+for+F)
28. Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. 2023. [Check your facts and try again: Improving large language models with external knowledge and automated feedback](https://md.research.elamur.ai/free/en/papers/peng-2023-check-facts-try-again-improving/). arXiv preprint arXiv:2302.12813.
29. Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, and Colin Raffel. 2023. Evaluating the factual consistency of large language models through news summarization. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5220–5255. [find](https://scholar.google.com/scholar?q=Derek+Tam%2C+Anisha+Mascarenhas%2C+Shiyue+Zhang%2C+Sarah+Kwan%2C+Mohit+Bansal%2C+and+Colin+Raffel%2E+2023%2E+Evaluating+the+factual+consistency+of+large+language+models+through+news+summarization%2E+In+Findings+of+th)
30. Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. 2023. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint arXiv:2305.14975. [find](https://scholar.google.com/scholar?q=Katherine+Tian%2C+Eric+Mitchell%2C+Allan+Zhou%2C+Archit+Sharma%2C+Rafael+Rafailov%2C+Huaxiu+Yao%2C+Chelsea+Finn%2C+and+Christopher+D+Manning%2E+2023%2E+Just+ask+for+calibration%3A+Strategies+for+eliciting+calibrated+conf)
31. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
32. Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. 2023. [A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation](https://md.research.elamur.ai/free/en/papers/varshney-2023-stitch-time-saves-nine-detecting/). arXiv preprint arXiv:2307.03987.
33. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171.
34. Yijun Xiao and William Yang Wang. 2021. [On hallucination and predictive uncertainty in conditional language generation](https://md.research.elamur.ai/free/en/papers/xiao-2021-hallucination-predictive-uncertainty-conditional-language/). arXiv preprint arXiv:2103.15025.
35. Shiping Yang, Renliang Sun, and Xiaojun Wan. 2023. A new benchmark and reverse validation method for passage-level hallucination detection. arXiv preprint arXiv:2310.06498. [find](https://scholar.google.com/scholar?q=Shiping+Yang%2C+Renliang+Sun%2C+and+Xiaojun+Wan%2E+2023%2E+A+new+benchmark+and+reverse+validation+method+for+passage%2Dlevel+hallucination+detection%2E+arXiv+preprint+arXiv%3A2310%2E06498%2E)
36. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. [Hotpotqa: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369–…
37. Hongbin Ye, Tong Liu, Aijia Zhang, Wei Hua, and Weiqiang Jia. 2023. Cognitive mirage: A review of hallucinations in large language models. arXiv preprint arXiv:2309.06794. — [arXiv:2309.06794](https://arxiv.org/abs/2309.06794) [find](https://scholar.google.com/scholar?q=Hongbin+Ye%2C+Tong+Liu%2C+Aijia+Zhang%2C+Wei+Hua%2C+and+Weiqiang+Jia%2E+2023%2E+Cognitive+mirage%3A+A+review+of+hallucinations+in+large+language+models%2E+arXiv+preprint+arXiv%3A2309%2E06794%2E+%E2%80%94+%5BarXiv%3A2309%2E06794%5D%28https%3A%2F)
38. Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith. 2023a. [How language model hallucinations can snowball](https://md.research.elamur.ai/free/en/papers/zhang-2023-language-model-hallucinations-snowball/). arXiv preprint arXiv:2305.13534.
39. Shuo Zhang, Liangming Pan, Junzhou Zhao, and William Yang Wang. 2023b. Mitigating language model hallucination with interactive question-knowledge alignment. arXiv preprint arXiv:2305.13669. [find](https://scholar.google.com/scholar?q=Shuo+Zhang%2C+Liangming+Pan%2C+Junzhou+Zhao%2C+and+William+Yang+Wang%2E+2023b%2E+Mitigating+language+model+hallucination+with+interactive+question%2Dknowledge+alignment%2E+arXiv+preprint+arXiv%3A2305%2E13669%2E)
40. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023c. [Siren’s song in the ai ocean: A survey on hallucination in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/). arXiv preprint arXiv:2309.01219.
41. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. [A survey of large language models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/). arXiv preprint arXiv:2303.18223.
42. Shen Zheng, Jie Huang, and Kevin Chen-Chuan Chang. 2023a. Why does chatgpt fall short in providing truthful answers. ArXiv preprint, abs/2304.10513. — [arXiv:2304.10513](https://arxiv.org/abs/2304.10513) [find](https://scholar.google.com/scholar?q=Shen+Zheng%2C+Jie+Huang%2C+and+Kevin+Chen%2DChuan+Chang%2E+2023a%2E+Why+does+chatgpt+fall+short+in+providing+truthful+answers%2E+ArXiv+preprint%2C+abs%2F2304%2E10513%2E+%E2%80%94+%5BarXiv%3A2304%2E10513%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2304%2E105)
43. Shen Zheng, Jie Huang, and Kevin Chen-Chuan Chang. 2023b. Why does chatgpt fall short in providing truthful answers? arXiv preprint arXiv:2304.10513. — [arXiv:2304.10513](https://arxiv.org/abs/2304.10513) [find](https://scholar.google.com/scholar?q=Shen+Zheng%2C+Jie+Huang%2C+and+Kevin+Chen%2DChuan+Chang%2E+2023b%2E+Why+does+chatgpt+fall+short+in+providing+truthful+answers%3F+arXiv+preprint+arXiv%3A2304%2E10513%2E+%E2%80%94+%5BarXiv%3A2304%2E10513%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2304%2E10)
44. Chunting Zhou, Graham Neubig, Jiatao Gu, Mona Diab, Paco Guzman, Luke Zettlemoyer, and Marjan Ghazvininejad. 2020. [Detecting hallucinated content in conditional neural sequence generation](https://md.research.elamur.ai/free/en/papers/zhou-2021-detecting-hallucinated-content-conditional-neural/). arXiv preprint arXiv:2011.02593.

*Updated 31 October 2026*