---
title: "Taming Object Hallucinations with Verified Atomic Confidence"
description: "Multimodal LLMs often describe objects, properties or relations that are not in the image. 54 references."
authors: ["Jiarui Liu", "Weihao Xuan", "Zhijing Jin", "Mona T. Diab"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.252/"
doi: "10.18653/v1/2026.eacl-long.252"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2026-taming-object-hallucinations-verified-atomic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2026-taming-object-hallucinations-verified-atomic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/) / Taming Object Hallucinations with Verified Atomic Confidence Estimation

# Taming Object Hallucinations with Verified Atomic Confidence Estimation

**Authors:** [Jiarui Liu](https://md.research.elamur.ai/free/en/authors/jiarui-liu/), [Weihao Xuan](https://md.research.elamur.ai/free/en/authors/weihao-xuan/), [Zhijing Jin](https://md.research.elamur.ai/free/en/authors/zhijing-jin/) and [Mona T. Diab](https://md.research.elamur.ai/free/en/authors/mona-diab/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.252/) · [PDF](https://aclanthology.org/2026.eacl-long.252.pdf) · [DOI 10.18653/v1/2026.eacl-long.252](https://doi.org/10.18653/v1/2026.eacl-long.252) · [OpenAlex](https://openalex.org/W7140132634)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Multimodal LLMs often describe objects, properties or relations that are not in the image. They are also easily swayed by small changes in how a question is worded. The authors propose TACO, a training-free pipeline that has the model check its own answers, with no external detectors or captioners. TACO works in four steps:

1. An LLM breaks the query and the initial answer into atomic yes/no questions, guided by a taxonomy of entities, attributes and relations.
2. Each question is rephrased several times.
3. The MLLM answers every variant. Confidence comes from either answer agreement (black-box) or token probabilities (gray-box).
4. An LLM rewrites the original response using the verified facts.

Tests used LLaVA-1.5-7B and CogVLM2 on POPE, MME, HallusionBench, AMBER and MM-Hal Bench. TACO beat both direct prompting and Visual Contrastive Decoding on discriminative and generative tasks. It also reduced the tendency to answer "yes" and gave better-calibrated confidence. The gray-box variant worked better than the black-box one. The approach is useful because it is a lightweight way to make vision-language outputs more faithful without any extra vision modules.

## Key points
- Hallucination taxonomy:
  - Entity (whole/part)
  - 11 attribute types (e.g., color, count, text rendering, material)
  - Relation (spatial/action)
- Paraphrasing questions probes the model's answer space better than varying the decoding method (sampling, beam search, temperature) on binary questions.
- Verification is the main cost: the MLLM is queried m × n times (m atomic questions × n paraphrases), and these calls can be run in parallel.
- An extra check on Qwen2.5-VL-7B-Instruct (POPE, MME, AMBER) also showed gains for TACO-S. On AMBER, CHAIR fell from 6.7 to 5.3 and Hal from 34.8 to 28.4, while Cover dipped slightly from 63.3 to 61.9.
- Limitations: TACO covers only object-level errors, depends on a helper LLM, and has not been tested on video or dialogue.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 539 works · 179,533 citations
- [University of Toronto](https://md.research.elamur.ai/free/en/universities/university-of-toronto/) — University, Toronto, Canada · 96 works · 142,007 citations
- [The University of Tokyo](https://md.research.elamur.ai/free/en/universities/the-university-of-tokyo/) — University, Tokyo, Japan · 90 works · 9,156 citations
- [RIKEN Center for Advanced Intelligence Project](https://md.research.elamur.ai/free/en/labs/riken-center-for-advanced-intelligence-project/) — Research institute, Tokyo, Japan · 15 works · 957 citations
- [RIKEN](https://md.research.elamur.ai/free/en/labs/riken/) — Research institute, Wako, Japan · 26 works · 526 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

## Cite

Cite as: Jiarui Liu, Weihao Xuan, Zhijing Jin and Mona T. Diab. 2026. Taming Object Hallucinations with Verified Atomic Confidence Estimation. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2026taming,
  title = {{Taming Object Hallucinations with Verified Atomic Confidence Estimation}},
  author = {Jiarui Liu, Weihao Xuan, Zhijing Jin and Mona T. Diab},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.252},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.252}
}
```

- [← previous Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting](https://md.research.elamur.ai/free/en/papers/wang-2026-taming-extreme-tokens-covariance-aware/)
- [next → TaMTAS: Terminology-Aware Machine Translation for Accessible Science. Large Corpus compilation, terminology extraction and data augmentation](https://md.research.elamur.ai/free/en/papers/oliver-2026-tamtas-terminology-aware-machine-translation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/)

## References

54 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
2. Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. 2024. Hallucination of multimodal [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). arXiv preprint arXiv:2404.18930.
3. Neil Band, Xuechen Li, Tengyu Ma, and Tatsunori Hashimoto. 2024. Linguistic calibration of long-form generations. In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Neil+Band%2C+Xuechen+Li%2C+Tengyu+Ma%2C+and+Tatsunori+Hashimoto%2E+2024%2E+Linguistic+calibration+of+long%2Dform+generations%2E+In+Forty%2Dfirst+International+Conference+on+Machine+Learning%2E)
4. Xiang Chen, Chenxi Wang, Yida Xue, Ningyu Zhang, Xiaoyan Yang, Qiang Li, Yue Shen, Lei Liang, Jinjie Gu, and Huajun Chen. 2024a. Unified hallucination detection for multimodal large language models. CoRR, abs/2402.03190. [find](https://scholar.google.com/scholar?q=Xiang+Chen%2C+Chenxi+Wang%2C+Yida+Xue%2C+Ningyu+Zhang%2C+Xiaoyan+Yang%2C+Qiang+Li%2C+Yue+Shen%2C+Lei+Liang%2C+Jinjie+Gu%2C+and+Huajun+Chen%2E+2024a%2E+Unified+hallucination+detection+for+multimodal+large+language+models%2E+C)
5. Xiang Chen, Chenxi Wang, Yida Xue, Ningyu Zhang, Xiaoyan Yang, Qiang Li, Yue Shen, Lei Liang, Jinjie Gu, and Huajun Chen. 2024b. Unified hallucination detection for multimodal large language models. arXiv preprint arXiv:2402.03190. [find](https://scholar.google.com/scholar?q=Xiang+Chen%2C+Chenxi+Wang%2C+Yida+Xue%2C+Ningyu+Zhang%2C+Xiaoyan+Yang%2C+Qiang+Li%2C+Yue+Shen%2C+Lei+Liang%2C+Jinjie+Gu%2C+and+Huajun+Chen%2E+2024b%2E+Unified+hallucination+detection+for+multimodal+large+language+models%2E+a)
6. Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. 2024. Davidsonian Scene Graph: Improving Reliability in Fine-Grained Evaluation for Text-to-Image Generation. In ICLR. [find](https://scholar.google.com/scholar?q=Jaemin+Cho%2C+Yushi+Hu%2C+Roopal+Garg%2C+Peter+Anderson%2C+Ranjay+Krishna%2C+Jason+Baldridge%2C+Mohit+Bansal%2C+Jordi+Pont%2DTuset%2C+and+Su+Wang%2E+2024%2E+Davidsonian+Scene+Graph%3A+Improving+Reliability+in+Fine%2DGrained+Ev)
7. Souvik Chowdhury and Badal Soni. 2025. R-vqa: A robust visual question answering model. Knowledge-Based Systems, 309:112827. [find](https://scholar.google.com/scholar?q=Souvik+Chowdhury+and+Badal+Soni%2E+2025%2E+R%2Dvqa%3A+A+robust+visual+question+answering+model%2E+Knowledge%2DBased+Systems%2C+309%3A112827%2E)
8. Ailin Deng, Miao Xiong, and Bryan Hooi. 2023. Great models think alike: Improving model reliability via inter-model latent agreement. arXiv preprint arXiv:2305.01481. — [arXiv:2305.01481](https://arxiv.org/abs/2305.01481) [find](https://scholar.google.com/scholar?q=Ailin+Deng%2C+Miao+Xiong%2C+and+Bryan+Hooi%2E+2023%2E+Great+models+think+alike%3A+Improving+model+reliability+via+inter%2Dmodel+latent+agreement%2E+arXiv+preprint+arXiv%3A2305%2E01481%2E+%E2%80%94+%5BarXiv%3A2305%2E01481%5D%28https%3A%2F%2Farxi)
9. Shrey Desai and Greg Durrett. 2020. Calibration of pre-trained transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 295–302, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shrey+Desai+and+Greg+Durrett%2E+2020%2E+Calibration+of+pre%2Dtrained+transformers%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pages+295%E2%80%93302%2C+Online%2E+A)
10. Esin Durmus, He He, and Mona Diab. 2020. [FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization](https://md.research.elamur.ai/free/en/papers/durmus-2020-feqa-question-answering-evaluation-framework/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5055– 5070, Online. Association for Computational Lingu…
11. Alexander Fabbri, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2022. [QAFactEval: Improved QA-based factual consistency evaluation for summarization](https://md.research.elamur.ai/free/en/papers/fabbri-2022-qafacteval-improved-qa-based-factual/). In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pa…
12. Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. 2024. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Conference on Computer Vis… [find](https://scholar.google.com/scholar?q=Qidong+Huang%2C+Xiaoyi+Dong%2C+Pan+Zhang%2C+Bin+Wang%2C+Conghui+He%2C+Jiaqi+Wang%2C+Dahua+Lin%2C+Weiming+Zhang%2C+and+Nenghai+Yu%2E+2024%2E+Opera%3A+Alleviating+hallucination+in+multi%2Dmodal+large+language+models+via+over%2Dt)
13. Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. 2024. Multi-modal hallucination control by visual information grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog… [find](https://scholar.google.com/scholar?q=Alessandro+Favero%2C+Luca+Zancato%2C+Matthew+Trager%2C+Siddharth+Choudhary%2C+Pramuditha+Perera%2C+Alessandro+Achille%2C+Ashwin+Swaminathan%2C+and+Stefano+Soatto%2E+2024%2E+Multi%2Dmodal+hallucination+control+by+visual+i)
14. Mohamed Insaf Ismithdeen, Muhammad Uzair Khattak, and Salman Khan. 2025. Promptception: How sensitive are large multimodal models to prompts? arXiv preprint arXiv:2509.03986. [find](https://scholar.google.com/scholar?q=Mohamed+Insaf+Ismithdeen%2C+Muhammad+Uzair+Khattak%2C+and+Salman+Khan%2E+2025%2E+Promptception%3A+How+sensitive+are+large+multimodal+models+to+prompts%3F+arXiv+preprint+arXiv%3A2509%2E03986%2E)
15. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and 1 others. 2023. [Mme: A comprehensive evaluation benchmark for multimodal large language models](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/). arXiv preprint arXiv:2306.13394.
16. Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1050–1059, New York, New York, USA. PMLR. [find](https://scholar.google.com/scholar?q=Yarin+Gal+and+Zoubin+Ghahramani%2E+2016%2E+Dropout+as+a+bayesian+approximation%3A+Representing+model+uncertainty+in+deep+learning%2E+In+Proceedings+of+The+33rd+International+Conference+on+Machine+Learning%2C+vo)
17. Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2023. [Hallusionbench: An advanced diagnostic suite for entangled language hallucination & visual illusion in large vision-language models](https://md.research.elamur.ai/free/en/papers/guan-2023-hallusionbench-advanced-diagnostic-suite-entangled/). P…
18. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017. On calibration of modern neural networks. In International conference on machine learning, pages 1321–1330. PMLR. [find](https://scholar.google.com/scholar?q=Chuan+Guo%2C+Geoff+Pleiss%2C+Yu+Sun%2C+and+Kilian+Q+Weinberger%2E+2017%2E+On+calibration+of+modern+neural+networks%2E+In+International+conference+on+machine+learning%2C+pages+1321%E2%80%931330%2E+PMLR%2E)
19. Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos. 2022. [A Survey on Automated Fact-Checking](https://md.research.elamur.ai/free/en/papers/guo-2022-survey-automated-fact-checking/). Transactions of the Association for Computational Linguistics, 10:178–206.
20. Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji, Zhao Xue, and 1 others. 2024. [Cogvlm2: Visual language models for image and video understanding](https://md.research.elamur.ai/free/en/papers/hong-2024-cogvlm2-visual-language-models/). arXiv preprint arXiv:2408.16500.
21. Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. [TRUE: Re-evaluating factual consistency evaluation](https://md.research.elamur.ai/free/en/papers/honovich-2022-true-re-evaluating-factual-consistency/). In Proceedings of the Second DialDoc Workshop on Document-grounded Dialogue and Co…
22. Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. 2021. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962–977. [find](https://scholar.google.com/scholar?q=Zhengbao+Jiang%2C+Jun+Araki%2C+Haibo+Ding%2C+and+Graham+Neubig%2E+2021%2E+How+can+we+know+when+language+models+know%3F+on+the+calibration+of+language+models+for+question+answering%2E+Transactions+of+the+Association)
23. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. ArXiv, abs/2302.09664. [find](https://scholar.google.com/scholar?q=Lorenz+Kuhn%2C+Yarin+Gal%2C+and+Sebastian+Farquhar%2E+2023%2E+Semantic+uncertainty%3A+Linguistic+invariances+for+uncertainty+estimation+in+natural+language+generation%2E+ArXiv%2C+abs%2F2302%2E09664%2E)
24. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Balaji+Lakshminarayanan%2C+Alexander+Pritzel%2C+and+Charles+Blundell%2E+2017%2E+Simple+and+scalable+predictive+uncertainty+estimation+using+deep+ensembles%2E+In+Advances+in+Neural+Information+Processing+Systems)
25. Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Kimin+Lee%2C+Kibok+Lee%2C+Honglak+Lee%2C+and+Jinwoo+Shin%2E+2018%2E+A+simple+unified+framework+for+detecting+out%2Dof%2Ddistribution+samples+and+adversarial+attacks%2E+In+Advances+in+Neural+Information+Processing+Sys)
26. Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2024. [Mitigating object hallucinations in large vision-language models through visual contrastive decoding](https://md.research.elamur.ai/free/en/papers/leng-2023-mitigating-object-hallucinations-large-vision/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1…
27. Miaoran Li, Baolin Peng, Michel Galley, Jianfeng Gao, and Zhu Zhang. 2024. [Self-checker: Plug-and-play modules for fact-checking with large language models](https://md.research.elamur.ai/free/en/papers/li-2024-self-checker-plug-play-modules/). In Findings of the Association for Computational Linguistics: NAACL 2024, pages 163–181, Mexico City, Mexico. Association for Computational Lin…
28. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. [Evaluating object hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-object-hallucination-large-vision/). In The 2023 Conference on Empirical Methods in Natural Language Processing.
29. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334. [find](https://scholar.google.com/scholar?q=Stephanie+Lin%2C+Jacob+Hilton%2C+and+Owain+Evans%2E+2022%2E+Teaching+models+to+express+their+uncertainty+in+words%2E+arXiv+preprint+arXiv%3A2205%2E14334%2E)
30. Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2024. Generating with confidence: Uncertainty quantification for black-box large language models. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Zhen+Lin%2C+Shubhendu+Trivedi%2C+and+Jimeng+Sun%2E+2024%2E+Generating+with+confidence%3A+Uncertainty+quantification+for+black%2Dbox+large+language+models%2E+Transactions+on+Machine+Learning+Research%2E)
31. Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, and Hao Su. 2023. [Deductive verification of chain-of-thought reasoning](https://md.research.elamur.ai/free/en/papers/ling-2023-deductive-verification-chain-thought-reasoning/). arXiv preprint arXiv:2306.03872.
32. Pritish Sahu, Karan Sikka, and Ajay Divakaran. 2024. Pelican: Correcting hallucination in vision-llms via claim decomposition and program of thought verification. arXiv preprint arXiv:2407.02352. [find](https://scholar.google.com/scholar?q=Pritish+Sahu%2C+Karan+Sikka%2C+and+Ajay+Divakaran%2E+2024%2E+Pelican%3A+Correcting+hallucination+in+vision%2Dllms+via+claim+decomposition+and+program+of+thought+verification%2E+arXiv+preprint+arXiv%3A2407%2E02352%2E)
33. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). Advances in neural information processing systems, 36:34892– 34916.
34. Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, and 1 others. 2023. [Aligning large multimodal models with factually augmented rlhf](https://md.research.elamur.ai/free/en/papers/sun-2023-aligning-large-multimodal-models-factually/). arXiv preprint arXiv:2309.14525.
35. Qing Lyu, Kumar Shridhar, Chaitanya Malaviya, Li Zhang, Yanai Elazar, Niket Tandon, Marianna Apidianaki, Mrinmaya Sachan, and Chris Callison-Burch. 2024. Calibrating large language models with sample consistency. arXiv preprint arXiv:2402.13904. — [arXiv:2402.13904](https://arxiv.org/abs/2402.13904) [find](https://scholar.google.com/scholar?q=Qing+Lyu%2C+Kumar+Shridhar%2C+Chaitanya+Malaviya%2C+Li+Zhang%2C+Yanai+Elazar%2C+Niket+Tandon%2C+Marianna+Apidianaki%2C+Mrinmaya+Sachan%2C+and+Chris+Callison%2DBurch%2E+2024%2E+Calibrating+large+language+models+with+sample+)
36. Andrey Malinin and Mark John Francis Gales. 2021. Uncertainty estimation in autoregressive structured prediction. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Andrey+Malinin+and+Mark+John+Francis+Gales%2E+2021%2E+Uncertainty+estimation+in+autoregressive+structured+prediction%2E+In+International+Conference+on+Learning+Representations%2E)
37. Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004–9017, Singapore. Association for Computation… [find](https://scholar.google.com/scholar?q=Potsawee+Manakul%2C+Adian+Liusie%2C+and+Mark+Gales%2E+2023%2E+SelfCheckGPT%3A+Zero%2Dresource+black%2Dbox+hallucination+detection+for+generative+large+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empir)
38. Ning Miao, Yee Whye Teh, and Tom Rainforth. 2023. Selfcheck: Using llms to zero-shot check their own step-by-step reasoning. arXiv preprint arXiv:2308.00436. [find](https://scholar.google.com/scholar?q=Ning+Miao%2C+Yee+Whye+Teh%2C+and+Tom+Rainforth%2E+2023%2E+Selfcheck%3A+Using+llms+to+zero%2Dshot+check+their+own+step%2Dby%2Dstep+reasoning%2E+arXiv+preprint+arXiv%3A2308%2E00436%2E)
39. Sabrina J. Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. 2022. Reducing conversational agents’ overconfidence through linguistic calibration. Transactions of the Association for Computational Linguistics, 10:857–872. [find](https://scholar.google.com/scholar?q=Sabrina+J%2E+Mielke%2C+Arthur+Szlam%2C+Emily+Dinan%2C+and+Y%2DLan+Boureau%2E+2022%2E+Reducing+conversational+agents%E2%80%99+overconfidence+through+linguistic+calibration%2E+Transactions+of+the+Association+for+Computational+)
40. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. [FActScore: Fine-grained atomic evaluation of factual precision in long form text generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/). In Proceedings of the 2023 Conference on Empirical Methods in Natural L…
41. Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. 2021. Revisiting the calibration of modern neural networks. Advances in Neural Information Processing Systems, 34:15682–15694. [find](https://scholar.google.com/scholar?q=Matthias+Minderer%2C+Josip+Djolonga%2C+Rob+Romijnders%2C+Frances+Hubis%2C+Xiaohua+Zhai%2C+Neil+Houlsby%2C+Dustin+Tran%2C+and+Mario+Lucic%2E+2021%2E+Revisiting+the+calibration+of+modern+neural+networks%2E+Advances+in+Neur)
42. Tejaswini Pedapati, Amit Dhurandhar, Soumya Ghosh, Soham Dan, and Prasanna Sattigeri. 2024. Large language model confidence estimation via black-box access. arXiv preprint arXiv:2406.04370. — [arXiv:2406.04370](https://arxiv.org/abs/2406.04370) [find](https://scholar.google.com/scholar?q=Tejaswini+Pedapati%2C+Amit+Dhurandhar%2C+Soumya+Ghosh%2C+Soham+Dan%2C+and+Prasanna+Sattigeri%2E+2024%2E+Large+language+model+confidence+estimation+via+black%2Dbox+access%2E+arXiv+preprint+arXiv%3A2406%2E04370%2E+%E2%80%94+%5BarXiv%3A2)
43. Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. [Object hallucination in image captioning](https://md.research.elamur.ai/free/en/papers/rohrbach-2018-object-hallucination-image-captioning/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045, Brussels, Belgium. Association for Computational Linguistics.
44. Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher Manning. 2023. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. In Proceedings of the 2023 Conferen… [find](https://scholar.google.com/scholar?q=Katherine+Tian%2C+Eric+Mitchell%2C+Allan+Zhou%2C+Archit+Sharma%2C+Rafael+Rafailov%2C+Huaxiu+Yao%2C+Chelsea+Finn%2C+and+Christopher+Manning%2E+2023%2E+Just+ask+for+calibration%3A+Strategies+for+eliciting+calibrated+confid)
45. Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. 2023. An llm-free multi-dimensional benchmark for mllms hallucination evaluation. arXiv preprint arXiv:2311.07397. [find](https://scholar.google.com/scholar?q=Junyang+Wang%2C+Yuhang+Wang%2C+Guohai+Xu%2C+Jing+Zhang%2C+Yukai+Gu%2C+Haitao+Jia%2C+Ming+Yan%2C+Ji+Zhang%2C+and+Jitao+Sang%2E+2023%2E+An+llm%2Dfree+multi%2Ddimensional+benchmark+for+mllms+hallucination+evaluation%2E+arXiv+prep)
46. Shuo Wang, Zhaopeng Tu, Shuming Shi, and Yang Liu. 2020. On the inference calibration of neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3070–3079, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shuo+Wang%2C+Zhaopeng+Tu%2C+Shuming+Shi%2C+and+Yang+Liu%2E+2020%2E+On+the+inference+calibration+of+neural+machine+translation%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Ling)
47. Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. 2024. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. arXiv preprint arXiv:2403.18715. [find](https://scholar.google.com/scholar?q=Xintong+Wang%2C+Jingheng+Pan%2C+Liang+Ding%2C+and+Chris+Biemann%2E+2024%2E+Mitigating+hallucinations+in+large+vision%2Dlanguage+models+with+instruction+contrastive+decoding%2E+arXiv+preprint+arXiv%3A2403%2E18715%2E)
48. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171.
49. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2023. [Large language models are better reasoners with self-verification](https://md.research.elamur.ai/free/en/papers/weng-2023-large-language-models-are-better/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2550–2575, Singapore. Association for Computat…
50. Penghao Wu and Saining Xie. 2024. V?: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13084– 13094. [find](https://scholar.google.com/scholar?q=Penghao+Wu+and+Saining+Xie%2E+2024%2E+V%3F%3A+Guided+visual+search+as+a+core+mechanism+in+multimodal+llms%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pages+13084%E2%80%93+13)
51. Miao Xiong, Ailin Deng, Pang Wei W Koh, Jiaying Wu, Shen Li, Jianqing Xu, and Bryan Hooi. 2023. Proximity-informed calibration for deep neural networks. Advances in Neural Information Processing Systems, 36:68511–68538. [find](https://scholar.google.com/scholar?q=Miao+Xiong%2C+Ailin+Deng%2C+Pang+Wei+W+Koh%2C+Jiaying+Wu%2C+Shen+Li%2C+Jianqing+Xu%2C+and+Bryan+Hooi%2E+2023%2E+Proximity%2Dinformed+calibration+for+deep+neural+networks%2E+Advances+in+Neural+Information+Processing+Syste)
52. Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. 2024. Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Miao+Xiong%2C+Zhiyuan+Hu%2C+Xinyang+Lu%2C+YIFEI+LI%2C+Jie+Fu%2C+Junxian+He%2C+and+Bryan+Hooi%2E+2024%2E+Can+LLMs+express+their+uncertainty%3F+an+empirical+evaluation+of+confidence+elicitation+in+LLMs%2E+In+The+Twelfth+In)
53. Chang. 2025. M 2 -TabFact: Multi-document multimodal fact verification with visual and textual representations of tabular data. In Findings of the Association for Computational Linguistics: ACL 2025, pages 26239–26256, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Chang%2E+2025%2E+M+2+%2DTabFact%3A+Multi%2Ddocument+multimodal+fact+verification+with+visual+and+textual+representations+of+tabular+data%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2025%2C+)
54. Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2024. [A survey on multimodal large language models](https://md.research.elamur.ai/free/en/papers/yin-2023-survey-multimodal-large-language-models/). National Science Review, 11(12):nwae403.

*Updated 1 November 2026*