---
title: "The Troubling Emergence of Hallucination in Large Language"
description: "The authors also define six content categories, ranging from garbled acronyms to wrong dates and… 59 references, cited by 60 papers here."
authors: ["Vipula Rawte", "Swagata Chakraborty", "Agnibh Pathak", "Anubhav Sarkar", "S.M Towhidul Islam Tonmoy", "Aman Chadha", "Amit Sheth", "Amitava Das"]
year: 2023
venue: "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2023.emnlp-main.155/"
doi: "10.18653/v1/2023.emnlp-main.155"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rawte-2023-troubling-emergence-hallucination-large-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rawte-2023-troubling-emergence-hallucination-large-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/4/) / The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations

# The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations

**Authors:** [Vipula Rawte](https://md.research.elamur.ai/free/en/authors/vipula-rawte/), Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, [S.M Towhidul Islam Tonmoy](https://md.research.elamur.ai/free/en/authors/s-m-towhidul-islam-tonmoy/), [Aman Chadha](https://md.research.elamur.ai/free/en/authors/aman-chadha/), [Amit Sheth](https://md.research.elamur.ai/free/en/authors/amit-sheth/) and [Amitava Das](https://md.research.elamur.ai/free/en/authors/amitava-das/) · **Year:** 2023 · **Venue:** [Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2023](https://md.research.elamur.ai/free/en/venues/emnlp/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.emnlp-main.155/) · [PDF](https://aclanthology.org/2023.emnlp-main.155.pdf) · [DOI 10.18653/v1/2023.emnlp-main.155](https://doi.org/10.18653/v1/2023.emnlp-main.155) · [OpenAlex](https://openalex.org/W4389524379)

**Citations:** 110 citations (OpenAlex, 2026-10-14) · [60](https://md.research.elamur.ai/free/en/papers/rawte-2023-troubling-emergence-hallucination-large-language/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tackles the lack of a fine-grained, task-independent taxonomy of LLM hallucination and of a way to compare models by how prone they are to it. The authors split hallucination by prompt type. **Factual Mirage** covers distortions of a true prompt. **Silver Lining** covers convincing narratives built on a false prompt. Each is divided into intrinsic and extrinsic variants and graded as mild, moderate or alarming. The authors also define six content categories, ranging from garbled acronyms to wrong dates and invented people or places. With this scheme they built HILT (HallucInation eLiciTation), a public dataset of texts produced by 15 contemporary LLMs and labelled by human annotators. Building on it, they propose the Hallucination Vulnerability Index (HVI), a computable score for ranking models by hallucination risk. They report that susceptibility tends to grow with model size. They also test two baseline mitigations: entropy-based token replacement and entailment-based factuality checking. The work matters because it offers an automatically computed rubric that could inform AI regulation and future mitigation research.

## Key points
- **HILT dataset:** 75,000 generated text samples from 15 LLMs, with human annotations for orientation, category and degree. Annotators assigned only one category per sentence.
- **Six categories:** acronym ambiguity, numeric nuisance, generated golem, virtual voice, geographic erratum and time wrap. The authors place name-nationality hallucination under generated golem.
- **HVI:** an auto-computable index for ranking LLMs. The authors contrast it with the manually scored EU AI Act compliance grading from Stanford HAI (Bommasani et al., 2023).
- **ENTROPY BB mitigation:** finds high-entropy tokens in output from a high-HVI model and swaps them using a lower-HVI model. It handles token-level errors such as 1789 vs. 1889.
- **FACTUALITY GB mitigation:** relies on entailment. It separates distinct entities (Barack Obama vs. Joe Biden) but struggles with similar ones (AI vs. AAAI).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 310 works · 284,480 citations
- [Amazon](https://md.research.elamur.ai/free/en/companies/amazon/) — Company, Seattle, United States · 70 works · 39,977 citations
- [University of South Carolina](https://md.research.elamur.ai/free/en/universities/university-of-south-carolina/) — University, Columbia, United States · 5 works · 773 citations
- [Islamic University of Technology](https://md.research.elamur.ai/free/en/universities/islamic-university-of-technology/) — University, Dhaka, Bangladesh · 5 works · 511 citations

Also on the byline: [Christ University](https://ror.org/022tv9y30).

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,343 works

## Cite

Cite as: Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, S.M Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth and Amitava Das. 2023. The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rawte2023troubling,
  title = {{The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations}},
  author = {Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, S.M Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth and Amitava Das},
  booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.emnlp-main.155},
  url = {https://doi.org/10.18653/v1/2023.emnlp-main.155}
}
```

- [← previous Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://md.research.elamur.ai/free/en/papers/wang-2023-survey-factuality-large-language-models/)
- [next → AudioPaLM: A Large Language Model That Can Speak and Listen](https://md.research.elamur.ai/free/en/papers/rubenstein-2023-audiopalm-large-language-model-that/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/4/)

## References

59 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. 2023. The future of computational linguistics. [find](https://scholar.google.com/scholar?q=2023%2E+The+future+of+computational+linguistics%2E)
2. Ayush Agrawal, Lester Mackey, and Adam Tauman Kalai. 2023. Do language models know when they’re hallucinating references? [find](https://scholar.google.com/scholar?q=Ayush+Agrawal%2C+Lester+Mackey%2C+and+Adam+Tauman+Kalai%2E+2023%2E+Do+language+models+know+when+they%E2%80%99re+hallucinating+references%3F)
3. Meng Cao, Yue Dong, and Jackie Chi Kit Cheung. 2022. [Hallucinated but factual! inspecting the factuality of hallucinations in abstractive summarization](https://md.research.elamur.ai/free/en/papers/cao-2022-hallucinated-but-factual-inspecting-factuality/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3340–3354.
4. Stability AI. 2023. Stability ai launches the first of its stable lm suite of language models. [find](https://scholar.google.com/scholar?q=Stability+AI%2E+2023%2E+Stability+ai+launches+the+first+of+its+stable+lm+suite+of+language+models%2E)
5. Anthony Chen, Panupong Pasupat, Sameer Singh, Hongrae Lee, and Kelvin Guu. 2023. [find](https://scholar.google.com/scholar?q=Anthony+Chen%2C+Panupong+Pasupat%2C+Sameer+Singh%2C+Hongrae+Lee%2C+and+Kelvin+Guu%2E+2023%2E)
6. Purr: Efficiently Abien Fred Agarap. 2019. Deep learning using rectified linear units (relu). editing language model hallucinations by denoising language model corruptions. [find](https://scholar.google.com/scholar?q=Purr%3A+Efficiently+Abien+Fred+Agarap%2E+2019%2E+Deep+learning+using+rectified+linear+units+%28relu%29%2E+editing+language+model+hallucinations+by+denoising+language+model+corruptions%2E)
7. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Zhuohan+Li%2C+Zi+Lin%2C+Ying+Sheng%2C+Zhanghao+Wu%2C+Hao+Zhang%2C+Lianmin+Zheng%2C+Siyuan+Zhuang%2C+Yonghao+Zhuang%2C+Joseph+E%2E+Gonzalez%2C+Ion+Stoica%2C+and+Eric+P%2E+Xing%2E+2023%2E+Vicuna%3A+An+open%2Dsource+cha)
8. Copyright-Office. 2023. Copyright registration guidance: Works containing material generated by artificial intelligence. Library of Congress. [find](https://scholar.google.com/scholar?q=Copyright%2DOffice%2E+2023%2E+Copyright+registration+guidance%3A+Works+containing+material+generated+by+artificial+intelligence%2E+Library+of+Congress%2E)
9. David Dale, Elena Voita, Loïc Barrault, and Marta R. Costa-jussà. 2022. [Detecting and mitigating hallucinations in machine translation: Model internal workings alone do well, sentence similarity even better](https://md.research.elamur.ai/free/en/papers/dale-2023-detecting-mitigating-hallucinations-machine-translation/).
10. Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344–16359. databricks. 2023. Dolly. [find](https://scholar.google.com/scholar?q=Tri+Dao%2C+Dan+Fu%2C+Stefano+Ermon%2C+Atri+Rudra%2C+and+Christopher+R%C3%A9%2E+2022%2E+Flashattention%3A+Fast+and+memory%2Defficient+exact+attention+with+io%2Dawareness%2E+Advances+in+Neural+Information+Processing+Systems%2C+35)
11. Tristan Deleu, David Kanaa, Leo Feng, Giancarlo Kerg, Yoshua Bengio, Guillaume Lajoie, and Pierre-Luc Bacon. 2022. Continuous-time meta-learning with forward mode differentiation. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. Open-Rev… [find](https://scholar.google.com/scholar?q=Tristan+Deleu%2C+David+Kanaa%2C+Leo+Feng%2C+Giancarlo+Kerg%2C+Yoshua+Bengio%2C+Guillaume+Lajoie%2C+and+Pierre%2DLuc+Bacon%2E+2022%2E+Continuous%2Dtime+meta%2Dlearning+with+forward+mode+differentiation%2E+In+The+Tenth+Interna)
12. European-Parliament. 2023. Proposal for a regulation of the european parliament and of the council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts. [find](https://scholar.google.com/scholar?q=European%2DParliament%2E+2023%2E+Proposal+for+a+regulation+of+the+european+parliament+and+of+the+council+laying+down+harmonised+rules+on+artificial+intelligence+%28artificial+intelligence+act%29+and+amending+ce)
13. Forbes. 2023. Lawyer used chatgpt in court—and cited fake cases. a judge is considering sanctions. [find](https://scholar.google.com/scholar?q=Forbes%2E+2023%2E+Lawyer+used+chatgpt+in+court%E2%80%94and+cited+fake+cases%2E+a+judge+is+considering+sanctions%2E)
14. Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy. 2013. [Learning whom to trust with MACE](https://md.research.elamur.ai/free/en/papers/hovy-2013-learning-whom-trust-mace/). In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1120–1130, Atlanta, Georgia. Associati…
15. Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, and Tatsunori Hashimoto. 2023a. When do pre-training biases propagate to downstream tasks? a case study in text summarization. In Proceedings of the 17th Conference of the European Chapter of the Association for… [find](https://scholar.google.com/scholar?q=Faisal+Ladhak%2C+Esin+Durmus%2C+Mirac+Suzgun%2C+Tianyi+Zhang%2C+Dan+Jurafsky%2C+Kathleen+McKeown%2C+and+Tatsunori+Hashimoto%2E+2023a%2E+When+do+pre%2Dtraining+biases+propagate+to+downstream+tasks%3F+a+case+study+in+text+)
16. Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen Mckeown, and Tatsunori B Hashimoto. 2023b. When do pre-training biases propagate to downstream tasks? a case study in text summarization. In Proceedings of the 17th Conference of the European Chapter of the Association fo… [find](https://scholar.google.com/scholar?q=Faisal+Ladhak%2C+Esin+Durmus%2C+Mirac+Suzgun%2C+Tianyi+Zhang%2C+Dan+Jurafsky%2C+Kathleen+Mckeown%2C+and+Tatsunori+B+Hashimoto%2E+2023b%2E+When+do+pre%2Dtraining+biases+propagate+to+downstream+tasks%3F+a+case+study+in+tex)
17. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. Albert: A lite bert for self-supervised learning of language representations. [find](https://scholar.google.com/scholar?q=Zhenzhong+Lan%2C+Mingda+Chen%2C+Sebastian+Goodman%2C+Kevin+Gimpel%2C+Piyush+Sharma%2C+and+Radu+Soricut%2E+2020%2E+Albert%3A+A+lite+bert+for+self%2Dsupervised+learning+of+language+representations%2E)
18. Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale N Fung, Mohammad Shoeybi, and Bryan Catanzaro. 2022. Factuality enhanced language models for open-ended text generation. In Advances in Neural Information Processing Systems, volume 35, pages 34586–34599. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Nayeon+Lee%2C+Wei+Ping%2C+Peng+Xu%2C+Mostofa+Patwary%2C+Pascale+N+Fung%2C+Mohammad+Shoeybi%2C+and+Bryan+Catanzaro%2E+2022%2E+Factuality+enhanced+language+models+for+open%2Dended+text+generation%2E+In+Advances+in+Neural+I)
19. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. [Halueval: A large-scale hallucination evaluation benchmark for large language models](https://md.research.elamur.ai/free/en/papers/li-2023-halueval-large-scale-hallucination-evaluation/).
20. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
21. Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. [find](https://scholar.google.com/scholar?q=Potsawee+Manakul%2C+Adian+Liusie%2C+and+Mark+J%2E+F%2E+Gales%2E+2023%2E+Selfcheckgpt%3A+Zero%2Dresource+black%2Dbox+hallucination+detection+for+generative+large+language+models%2E)
22. Gary Marcus and Future of Life Institute. 2023. Pause giant ai experiments: An open letter. [find](https://scholar.google.com/scholar?q=Gary+Marcus+and+Future+of+Life+Institute%2E+2023%2E+Pause+giant+ai+experiments%3A+An+open+letter%2E)
23. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Joshua+Maynez%2C+Shashi+Narayan%2C+Bernd+Bohnet%2C+and+Ryan+McDonald%2E+2020%2E+On+faithfulness+and+factuality+in+abstractive+summarization%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Comp)
24. Midjourney. 2022. [https://www.midjourney.com](https://www.midjourney.com). [↗](https://www.midjourney.com)
25. Niels Mündler, Jingxuan He, Slobodan Jenko, and Martin Vechev. 2023. [Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation](https://md.research.elamur.ai/free/en/papers/mundler-2023-self-contradictory-hallucinations-large-language/). nVIDIA. 2023. [https://nvidia.github.io/nemo/](https://nvidia.github.io/nemo/). [↗](https://nvidia.github.io/nemo/)
26. NYT. [https://www.nytimes.com/topic/company/twitter](https://www.nytimes.com/topic/company/twitter). [↗](https://www.nytimes.com/topic/company/twitter)
27. OpenAI. 2022. Introducing chatgpt. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2022%2E+Introducing+chatgpt%2E)
28. OpenAI. 2023a. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/).
29. OpenAI. 2023b. Our approach to ai safety. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2023b%2E+Our+approach+to+ai+safety%2E)
30. Allan Paivio. 2013. Dual coding theory, word abstractness, and emotion: a critical review of kousta et al.(2011). [find](https://scholar.google.com/scholar?q=Allan+Paivio%2E+2013%2E+Dual+coding+theory%2C+word+abstractness%2C+and+emotion%3A+a+critical+review+of+kousta+et+al%2E%282011%29%2E)
31. Jonas Pfeiffer, Francesco Piccinno, Massimo Nicosia, Xinyi Wang, Machel Reid, and Sebastian Ruder. 2023. mmt5: Modular multilingual pre-training solves source language hallucinations. [find](https://scholar.google.com/scholar?q=Jonas+Pfeiffer%2C+Francesco+Piccinno%2C+Massimo+Nicosia%2C+Xinyi+Wang%2C+Machel+Reid%2C+and+Sebastian+Ruder%2E+2023%2E+mmt5%3A+Modular+multilingual+pre%2Dtraining+solves+source+language+hallucinations%2E)
32. Politifact. [https://www.politifact.com/](https://www.politifact.com/). [↗](https://www.politifact.com/)
33. Ofir Press, Noah A. Smith, and Mike Lewis. 2022. Train short, test long: Attention with linear biases enables input length extrapolation. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. [find](https://scholar.google.com/scholar?q=Ofir+Press%2C+Noah+A%2E+Smith%2C+and+Mike+Lewis%2E+2022%2E+Train+short%2C+test+long%3A+Attention+with+linear+biases+enables+input+length+extrapolation%2E+In+The+Tenth+International+Conference+on+Learning+Representati)
34. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
35. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). The Journal of Machine Learning Research, 21(1):5485–5551.
36. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. [Hierarchical text-conditional image generation with clip latents](https://md.research.elamur.ai/free/en/papers/ramesh-2022-hierarchical-text-conditional-image-generation/). arXiv preprint arXiv:2204.06125.
37. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. [Zero-shot text-to-image generation](https://md.research.elamur.ai/free/en/papers/ramesh-2021-zero-shot-text-image-generation/). In International Conference on Machine Learning, pages 8821–8831. PMLR.
38. Vikas Raunak, Arul Menezes, and Marcin Junczys-Dowmunt. 2021. [The curious case of hallucinations in neural machine translation](https://md.research.elamur.ai/free/en/papers/raunak-2021-curious-case-hallucinations-neural-machine/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1172–1183, Online.…
39. Reuters. 2023. Alphabet shares dive after google ai chatbot bard flubs answer in ad. [find](https://scholar.google.com/scholar?q=Reuters%2E+2023%2E+Alphabet+shares+dive+after+google+ai+chatbot+bard+flubs+answer+in+ad%2E)
40. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695. [find](https://scholar.google.com/scholar?q=Robin+Rombach%2C+Andreas+Blattmann%2C+Dominik+Lorenz%2C+Patrick+Esser%2C+and+Bj%C3%B6rn+Ommer%2E+2022%2E+High%2Dresolution+image+synthesis+with+latent+diffusion+models%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Compu)
41. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01108. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+2019%2E+Distilbert%2C+a+distilled+version+of+bert%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+ArXiv%2C+abs%2F1910%2E01108%2E)
42. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100. [find](https://scholar.google.com/scholar?q=Teven+Le+Scao%2C+Angela+Fan%2C+Christopher+Akiki%2C+Ellie+Pavlick%2C+Suzana+Ilic%CC%81%2C+Daniel+Hesslow%2C+Roman+Castagn%C3%A9%2C+Alexandra+Sasha+Luccioni%2C+Fran%C3%A7ois+Yvon%2C+Matthias+Gall%C3%A9%2C+et+al%2E+2022%2E+Bloom%3A+A+176b%2Dparameter)
43. Noam Shazeer. 2019. Fast transformer decoding: One write-head is all you need. arXiv preprint arXiv:1911.02150. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+2019%2E+Fast+transformer+decoding%3A+One+write%2Dhead+is+all+you+need%2E+arXiv+preprint+arXiv%3A1911%2E02150%2E)
44. Noam Shazeer. 2020. former. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+2020%2E+former%2E)
45. Glu variants improve trans-Arvind Krishna Sridhar and Erik Visser. 2022. Improved beam search for hallucination mitigation in abstractive summarization. [find](https://scholar.google.com/scholar?q=Glu+variants+improve+trans%2DArvind+Krishna+Sridhar+and+Erik+Visser%2E+2022%2E+Improved+beam+search+for+hallucination+mitigation+in+abstractive+summarization%2E)
46. Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. 2022. [Roformer: Enhanced transformer with rotary position embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/).
47. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/](https://github.com/tatsu-lab/) stanford_alpaca. [↗](https://github.com/tatsu-lab/)
48. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv pr…
49. Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. 2023. [A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation](https://md.research.elamur.ai/free/en/papers/varshney-2023-stitch-time-saves-nine-detecting/). arXiv preprint arXiv:2307.03987.
50. Zhen Wang, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, Huan Sun, and Yoon Kim. 2023. Multitask prompt tuning enables parameter-efficient transfer learning. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zhen+Wang%2C+Rameswar+Panda%2C+Leonid+Karlinsky%2C+Rogerio+Feris%2C+Huan+Sun%2C+and+Yoon+Kim%2E+2023%2E+Multitask+prompt+tuning+enables+parameter%2Defficient+transfer+learning%2E+In+The+Eleventh+International+Conferenc)
51. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. [Opt: Open pre-trai…](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/)
52. Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul F. Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. CoRR, abs/1909.08593. [find](https://scholar.google.com/scholar?q=Daniel+M%2E+Ziegler%2C+Nisan+Stiennon%2C+Jeffrey+Wu%2C+Tom+B%2E+Brown%2C+Alec+Radford%2C+Dario+Amodei%2C+Paul+F%2E+Christiano%2C+and+Geoffrey+Irving%2E+2019%2E+Fine%2Dtuning+language+models+from+human+preferences%2E+CoRR%2C+abs%2F19)
53. White-House. 2023. Blueprint for an ai bill of rights: Making automated systems work for the american people. [find](https://scholar.google.com/scholar?q=White%2DHouse%2E+2023%2E+Blueprint+for+an+ai+bill+of+rights%3A+Making+automated+systems+work+for+the+american+people%2E)
54. Wikipedia_Fleiss’s_Kappa. Fleiss’s kappa. Wikipedia_Krippendorff’s_Alpha. Krippendorff’s alpha. Wikipedia_min_max. Normalization. Wikipedia_zscore. Normalization. [find](https://scholar.google.com/scholar?q=Wikipedia%5FFleiss%E2%80%99s%5FKappa%2E+Fleiss%E2%80%99s+kappa%2E+Wikipedia%5FKrippendorff%E2%80%99s%5FAlpha%2E+Krippendorff%E2%80%99s+alpha%2E+Wikipedia%5Fmin%5Fmax%2E+Normalization%2E+Wikipedia%5Fzscore%2E+Normalization%2E)
55. [Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho…](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
56. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. [Xlnet: Generalized autoregressive pretraining for language understanding](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/). Advances in neural information processing systems, 32.
57. Biao Zhang and Rico Sennrich. 2019. Root mean square layer normalization. [find](https://scholar.google.com/scholar?q=Biao+Zhang+and+Rico+Sennrich%2E+2019%2E+Root+mean+square+layer+normalization%2E)
58. Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. 2023a. [How language model hallucinations can snowball](https://md.research.elamur.ai/free/en/papers/zhang-2023-language-model-hallucinations-snowball/).
59. Shuo Zhang, Liangming Pan, Junzhou Zhao, and William Yang Wang. 2023b. Mitigating language model hallucination with interactive question-knowledge alignment. [find](https://scholar.google.com/scholar?q=Shuo+Zhang%2C+Liangming+Pan%2C+Junzhou+Zhao%2C+and+William+Yang+Wang%2E+2023b%2E+Mitigating+language+model+hallucination+with+interactive+question%2Dknowledge+alignment%2E)

*Updated 24 October 2026*