---
title: "On Hallucination and Predictive Uncertainty in — Xiao 2021"
description: "Neural text generators often produce content that their input does not support, a problem known as hallucination."
authors: ["Yijun Xiao", "William Yang Wang"]
year: 2021
venue: "Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume"
source: "https://aclanthology.org/2021.eacl-main.236/"
doi: "10.18653/v1/2021.eacl-main.236"
arxiv: "2103.15025"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xiao-2021-hallucination-predictive-uncertainty-conditional-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xiao-2021-hallucination-predictive-uncertainty-conditional-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / On Hallucination and Predictive Uncertainty in Conditional Language Generation

# On Hallucination and Predictive Uncertainty in Conditional Language Generation

**Authors:** Yijun Xiao, [William Yang Wang](https://md.research.elamur.ai/free/en/authors/william-yang-wang/) · **Year:** 2021 · **Venue:** Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.eacl-main.236/) · [PDF](https://aclanthology.org/2021.eacl-main.236.pdf) · [DOI 10.18653/v1/2021.eacl-main.236](https://doi.org/10.18653/v1/2021.eacl-main.236) · [arXiv 2103.15025](https://arxiv.org/abs/2103.15025) · [OpenAlex](https://openalex.org/W3153712677)

**Citations:** 74 citations (openalex-snapshot · 2026-10-14) · [81](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Neural text generators often produce content that their input does not support, a problem known as hallucination. Earlier explanations, such as language priors, dataset misalignment and domain shift, were tied to individual tasks. Xiao and Wang propose a shared cause: predictive uncertainty, measured as the entropy of the next-token distribution. Using deep ensembles, they split this uncertainty into an aleatoric part (from the data) and an epistemic part (from the model).

They tested the idea on two tasks:
- **Image captioning:** MSCOCO, with objects and verbs checked for hallucination.
- **Data-to-text generation:** ToTTo.

Hallucination rates rose consistently with uncertainty, and epistemic uncertainty predicted hallucination better than aleatoric uncertainty did. Building on this, the authors introduce uncertainty-aware beam search (UABS), which adds an uncertainty penalty to the decoding score. Penalizing the epistemic component gives the best balance between fewer hallucinations and standard metric quality.

The work offers a task-independent explanation of hallucination and a simple decoding-time fix that requires no retraining. The trade-off is shorter, more cautious outputs.

## Key points
- **Captioning setup:** Four MSCOCO models (FC, Att2In, BUTD, Transformer) were trained on the Karpathy split with the self-critical.pytorch framework by Luo et al. Training used batch size 50, 30 epochs and the Adam optimizer. Object hallucination was detected with synonym lists, following the approach behind the CHAIR metric.
- **Correlations:** Pearson correlation with hallucination was about 0.27–0.36 for epistemic uncertainty, compared with roughly 0.13–0.30 for aleatoric uncertainty, across all four models.
- **Action verbs:** The pattern also holds for verbs, which were extracted with the spaCy POS tagger. Hallucination rates climb from near zero in the lowest-uncertainty bins to 22–44% in the highest bin.
- **Generic captions:** In the captioning comparison, one alternative penalty raised the share of generic captions from 1.0% to 28.4%. Epistemic UABS kept generic outputs rare while cutting hallucination.
- **ToTTo results:** UABS used a five-model ensemble with 100 human-rated outputs per penalty weight. Small penalties improved faithfulness at some cost to BLEU and fluency. Larger penalties dropped low-confidence statements, producing shorter texts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

81 papers of our library cite it (of the 140884 whose bibliographies we read); 8 of them have a page here, 73 not yet.

- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/)
- [Survey of Hallucination in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/)
- [Challenges and Applications of Large Language Models](https://md.research.elamur.ai/free/en/papers/kaddour-2023-challenges-applications-large-language-models/)
- [大语言模型的幻觉问题研究综述](https://md.research.elamur.ai/free/en/papers/liu-2024-hallucinations-large-language/)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://md.research.elamur.ai/free/en/papers/niu-2024-ragtruth-hallucination-corpus-developing-trustworthy/)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xiao2021hallucination,
  title = {{On Hallucination and Predictive Uncertainty in Conditional Language Generation}},
  author = {Xiao, Yijun and Wang, William Yang},
  booktitle = {Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume},
  year = {2021},
  doi = {10.18653/v1/2021.eacl-main.236},
  eprint = {2103.15025},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.eacl-main.236}
}
```

- [← previous On Generative Spoken Language Modeling from Raw Audio](https://md.research.elamur.ai/free/en/papers/lakhotia-2021-generative-spoken-language-modeling-raw/)
- [next → On Releasing Annotator-Level Labels and Information in Datasets](https://md.research.elamur.ai/free/en/papers/prabhakaran-2021-releasing-annotator-level-labels-information/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/)

## References

34 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6077–6086. [find](https://scholar.google.com/scholar?q=Peter+Anderson%2C+Xiaodong+He%2C+Chris+Buehler%2C+Damien+Teney%2C+Mark+Johnson%2C+Stephen+Gould%2C+and+Lei+Zhang%2E+2018%2E+Bottom%2Dup+and+top%2Ddown+attention+for+image+captioning+and+visual+question+answering%2E+In+Proc)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+3rd+International+Conference+on+Learning+Representations%2C+ICLR+2015%2C) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. Weight uncertainty in neural networks. arXiv preprint arXiv:1505.05424. — [arXiv:1505.05424](https://arxiv.org/abs/1505.05424) [find](https://scholar.google.com/scholar?q=Charles+Blundell%2C+Julien+Cornebise%2C+Koray+Kavukcuoglu%2C+and+Daan+Wierstra%2E+2015%2E+Weight+uncertainty+in+neural+networks%2E+arXiv+preprint+arXiv%3A1505%2E05424%2E+%E2%80%94+%5BarXiv%3A1505%2E05424%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1505%2E)
4. Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li. 2018. Faithful to the original: Fact aware neural abstractive summarization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32. [find](https://scholar.google.com/scholar?q=Ziqiang+Cao%2C+Furu+Wei%2C+Wenjie+Li%2C+and+Sujian+Li%2E+2018%2E+Faithful+to+the+original%3A+Fact+aware+neural+abstractive+summarization%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+3)
5. Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C Lawrence Zitnick. 2015. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325. [find](https://scholar.google.com/scholar?q=Xinlei+Chen%2C+Hao+Fang%2C+Tsung%2DYi+Lin%2C+Ramakrishna+Vedantam%2C+Saurabh+Gupta%2C+Piotr+Dolla%CC%81r%2C+and+C+Lawrence+Zitnick%2E+2015%2E+Microsoft+coco+captions%3A+Data+collection+and+evaluation+server%2E+arXiv+preprint+ar) [our card](https://md.research.elamur.ai/free/en/papers/chen-2015-microsoft-coco-captions-data-collection/)
6. Stefan Depeweg, Jose-Miguel Hernandez-Lobato, Finale Doshi-Velez, and Steffen Udluft. 2018. Decomposition of uncertainty in bayesian deep learning for efficient and risk-sensitive learning. In International Conference on Machine Learning, pages 1184–1193. PMLR. [find](https://scholar.google.com/scholar?q=Stefan+Depeweg%2C+Jose%2DMiguel+Hernandez%2DLobato%2C+Finale+Doshi%2DVelez%2C+and+Steffen+Udluft%2E+2018%2E+Decomposition+of+uncertainty+in+bayesian+deep+learning+for+efficient+and+risk%2Dsensitive+learning%2E+In+Interna)
7. Andrej Karpathy and Li Fei-Fei. 2015. Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3128–3137. [find](https://scholar.google.com/scholar?q=Andrej+Karpathy+and+Li+Fei%2DFei%2E+2015%2E+Deep+visual%2Dsemantic+alignments+for+generating+image+descriptions%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+pages+3128%E2%80%9331) [our card](https://md.research.elamur.ai/free/en/papers/karpathy-2014-deep-visual-semantic-alignments-generating/)
8. Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision? In Advances in neural information processing systems, pages 5574–5584. [find](https://scholar.google.com/scholar?q=Alex+Kendall+and+Yarin+Gal%2E+2017%2E+What+uncertainties+do+we+need+in+bayesian+deep+learning+for+computer+vision%3F+In+Advances+in+neural+information+processing+systems%2C+pages+5574%E2%80%935584%2E) [our card](https://md.research.elamur.ai/free/en/papers/kendall-2017-uncertainties-need-bayesian-deep-learning/)
9. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2E)
10. Karen Kukich. 1983. Design of a knowledge-based report generator. In Proceedings of the 21st annual meeting on Association for Computational Linguistics, pages 145–150. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Karen+Kukich%2E+1983%2E+Design+of+a+knowledge%2Dbased+report+generator%2E+In+Proceedings+of+the+21st+annual+meeting+on+Association+for+Computational+Linguistics%2C+pages+145%E2%80%93150%2E+Association+for+Computational+L)
11. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in neural information processing systems, pages 6402–6413. [find](https://scholar.google.com/scholar?q=Balaji+Lakshminarayanan%2C+Alexander+Pritzel%2C+and+Charles+Blundell%2E+2017%2E+Simple+and+scalable+predictive+uncertainty+estimation+using+deep+ensembles%2E+In+Advances+in+neural+information+processing+systems)
12. Ruotian Luo, Brian Price, Scott Cohen, and Gregory Shakhnarovich. 2018. Discriminability objective for training descriptive captions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6964–6974. [find](https://scholar.google.com/scholar?q=Ruotian+Luo%2C+Brian+Price%2C+Scott+Cohen%2C+and+Gregory+Shakhnarovich%2E+2018%2E+Discriminability+objective+for+training+descriptive+captions%2E+In+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Patte)
13. Kathleen McKeown. 1992. bridge University Press. [find](https://scholar.google.com/scholar?q=Kathleen+McKeown%2E+1992%2E+bridge+University+Press%2E)
14. Text generation. Cam-Mathias Müller, Annette Rios, and Rico Sennrich. 2019. Domain robustness in neural machine translation. arXiv preprint arXiv:1911.03109. [find](https://scholar.google.com/scholar?q=Text+generation%2E+Cam%2DMathias+Mu%CC%88ller%2C+Annette+Rios%2C+and+Rico+Sennrich%2E+2019%2E+Domain+robustness+in+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1911%2E03109%2E) [our card](https://md.research.elamur.ai/free/en/papers/muller-2020-domain-robustness-neural-machine-translation/)
15. Armen Der Kiureghian and Ove Ditlevsen. 2009. Aleatory or epistemic? does it matter? Structural safety, 31(2):105–112. [find](https://scholar.google.com/scholar?q=Armen+Der+Kiureghian+and+Ove+Ditlevsen%2E+2009%2E+Aleatory+or+epistemic%3F+does+it+matter%3F+Structural+safety%2C+31%282%29%3A105%E2%80%93112%2E)
16. Radford M Neal. 1995. BAYESIAN LEARNING FOR NEURAL NETWORKS. Ph.D. thesis, University of Toronto. [find](https://scholar.google.com/scholar?q=Radford+M+Neal%2E+1995%2E+BAYESIAN+LEARNING+FOR+NEURAL+NETWORKS%2E+Ph%2ED%2E+thesis%2C+University+of+Toronto%2E)
17. Esin Durmus, He He, and Mona Diab. 2020. FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5055– 5070, Online. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Esin+Durmus%2C+He+He%2C+and+Mona+Diab%2E+2020%2E+FEQA%3A+A+question+answering+evaluation+framework+for+faithfulness+assessment+in+abstractive+summarization%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Asso) [our card](https://md.research.elamur.ai/free/en/papers/durmus-2020-feqa-question-answering-evaluation-framework/)
18. Feng Nie, Jin-Ge Yao, Jinpeng Wang, Rong Pan, and Chin-Yew Lin. 2019. A simple recipe towards reducing hallucination in neural surface realisation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2673– 2679. [find](https://scholar.google.com/scholar?q=Feng+Nie%2C+Jin%2DGe+Yao%2C+Jinpeng+Wang%2C+Rong+Pan%2C+and+Chin%2DYew+Lin%2E+2019%2E+A+simple+recipe+towards+reducing+hallucination+in+neural+surface+realisation%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Ass)
19. Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050–1059. [find](https://scholar.google.com/scholar?q=Yarin+Gal+and+Zoubin+Ghahramani%2E+2016%2E+Dropout+as+a+bayesian+approximation%3A+Representing+model+uncertainty+in+deep+learning%2E+In+international+conference+on+machine+learning%2C+pages+1050%E2%80%931059%2E)
20. Myle Ott, Michael Auli, David Grangier, and Marc’Aurelio Ranzato. 2018. Analyzing uncertainty in neural machine translation. In International Conference on Machine Learning, pages 3956–3965. [find](https://scholar.google.com/scholar?q=Myle+Ott%2C+Michael+Auli%2C+David+Grangier%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2018%2E+Analyzing+uncertainty+in+neural+machine+translation%2E+In+International+Conference+on+Machine+Learning%2C+pages+3956%E2%80%933965%2E) [our card](https://md.research.elamur.ai/free/en/papers/ott-2018-analyzing-uncertainty-neural-machine-translation/)
21. Geoffrey E Hinton and Drew Van Camp. 1993. Keeping the neural networks simple by minimizing the description length of the weights. In Proceedings of the sixth annual conference on Computational learning theory, pages 5–13. [find](https://scholar.google.com/scholar?q=Geoffrey+E+Hinton+and+Drew+Van+Camp%2E+1993%2E+Keeping+the+neural+networks+simple+by+minimizing+the+description+length+of+the+weights%2E+In+Proceedings+of+the+sixth+annual+conference+on+Computational+learni)
22. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations. [find](https://scholar.google.com/scholar?q=Myle+Ott%2C+Sergey+Edunov%2C+Alexei+Baevski%2C+Angela+Fan%2C+Sam+Gross%2C+Nathan+Ng%2C+David+Grangier%2C+and+Michael+Auli%2E+2019%2E+fairseq%3A+A+fast%2C+extensible+toolkit+for+sequence+modeling%2E+In+Proceedings+of+NAACL%2DHL) [our card](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
23. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%931780%2E)
24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+annual+meeting+on+association+for+computat) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
25. Ankur P Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. Totto: A controlled table-to-text generation dataset. arXiv preprint arXiv:2004.14373. [find](https://scholar.google.com/scholar?q=Ankur+P+Parikh%2C+Xuezhi+Wang%2C+Sebastian+Gehrmann%2C+Manaal+Faruqui%2C+Bhuwan+Dhingra%2C+Diyi+Yang%2C+and+Dipanjan+Das%2E+2020%2E+Totto%3A+A+controlled+table%2Dto%2Dtext+generation+dataset%2E+arXiv+preprint+arXiv%3A2004%2E1437) [our card](https://md.research.elamur.ai/free/en/papers/parikh-2020-totto-controlled-table-text-generation/)
26. Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. 2017. Self-critical sequence training for image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7008–7024. [find](https://scholar.google.com/scholar?q=Steven+J+Rennie%2C+Etienne+Marcheret%2C+Youssef+Mroueh%2C+Jerret+Ross%2C+and+Vaibhava+Goel%2E+2017%2E+Self%2Dcritical+sequence+training+for+image+captioning%2E+In+Proceedings+of+the+IEEE+Conference+on+Computer+Vision) [our card](https://md.research.elamur.ai/free/en/papers/rennie-2016-self-critical-sequence-training-image/)
27. Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045. [find](https://scholar.google.com/scholar?q=Anna+Rohrbach%2C+Lisa+Anne+Hendricks%2C+Kaylee+Burns%2C+Trevor+Darrell%2C+and+Kate+Saenko%2E+2018%2E+Object+hallucination+in+image+captioning%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural) [our card](https://md.research.elamur.ai/free/en/papers/rohrbach-2018-object-hallucination-image-captioning/)
28. Ran Tian, Shashi Narayan, Thibault Sellam, and Ankur P Parikh. 2019. Sticking to the facts: Confident decoding for faithful data-to-text generation. arXiv preprint arXiv:1910.08684. — [arXiv:1910.08684](https://arxiv.org/abs/1910.08684) [find](https://scholar.google.com/scholar?q=Ran+Tian%2C+Shashi+Narayan%2C+Thibault+Sellam%2C+and+Ankur+P+Parikh%2E+2019%2E+Sticking+to+the+facts%3A+Confident+decoding+for+faithful+data%2Dto%2Dtext+generation%2E+arXiv+preprint+arXiv%3A1910%2E08684%2E+%E2%80%94+%5BarXiv%3A1910%2E0868)
29. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processi)
30. Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575. [find](https://scholar.google.com/scholar?q=Ramakrishna+Vedantam%2C+C+Lawrence+Zitnick%2C+and+Devi+Parikh%2E+2015%2E+Cider%3A+Consensus%2Dbased+image+description+evaluation%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+) [our card](https://md.research.elamur.ai/free/en/papers/vedantam-2014-cider-consensus-based-image-description/)
31. Sam Wiseman, Stuart M Shieber, and Alexander M Rush. 2017. Challenges in data-to-document generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2253–2263. [find](https://scholar.google.com/scholar?q=Sam+Wiseman%2C+Stuart+M+Shieber%2C+and+Alexander+M+Rush%2E+2017%2E+Challenges+in+data%2Dto%2Ddocument+generation%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+2) [our card](https://md.research.elamur.ai/free/en/papers/wiseman-2017-challenges-data-document-generation/)
32. Yijun Xiao and William Yang Wang. 2019. Quantifying uncertainties in natural language processing tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7322–7329. [find](https://scholar.google.com/scholar?q=Yijun+Xiao+and+William+Yang+Wang%2E+2019%2E+Quantifying+uncertainties+in+natural+language+processing+tasks%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+33%2C+pages+7322%E2%80%937329%2E)
33. Jiacheng Xu, Shrey Desai, and Greg Durrett. 2020. Understanding neural abstractive summarization models via uncertainty. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6275–6281. [find](https://scholar.google.com/scholar?q=Jiacheng+Xu%2C+Shrey+Desai%2C+and+Greg+Durrett%2E+2020%2E+Understanding+neural+abstractive+summarization+models+via+uncertainty%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+)
34. Lingxue Zhu and Nikolay Laptev. 2017. Deep and confident prediction for time series at uber. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW), pages 103–110. IEEE. [find](https://scholar.google.com/scholar?q=Lingxue+Zhu+and+Nikolay+Laptev%2E+2017%2E+Deep+and+confident+prediction+for+time+series+at+uber%2E+In+2017+IEEE+International+Conference+on+Data+Mining+Workshops+%28ICDMW%29%2C+pages+103%E2%80%93110%2E+IEEE%2E)