---
title: "Latent Attention Denoising — paper by Luo et al. (2026)"
description: "To correct this, they propose Latent Attention Denoising (LAD), a training-free, plug-and-play method that operates on… 95 references."
authors: ["Zhiwen Luo", "Siyu Jiang", "Weilong Jiang", "Kun He"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1327/"
doi: "10.18653/v1/2026.acl-long.1327"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/luo-2026-latent-attention-denoising-training-free/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/luo-2026-latent-attention-denoising-training-free/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) / Latent Attention Denoising: A Training-Free Energy-Based Framework for Mitigating Hallucinations in Vision-Language Models

# Latent Attention Denoising: A Training-Free Energy-Based Framework for Mitigating Hallucinations in Vision-Language Models

**Authors:** Zhiwen Luo, Siyu Jiang, Weilong Jiang and [Kun He](https://md.research.elamur.ai/free/en/authors/kun-he/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1327/) · [PDF](https://aclanthology.org/2026.acl-long.1327.pdf) · [DOI 10.18653/v1/2026.acl-long.1327](https://doi.org/10.18653/v1/2026.acl-long.1327) · [OpenAlex](https://openalex.org/W7166887576)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary

Large Vision-Language Models (LVLMs) often describe objects or details that are not in the image. The authors trace this to attention sinks, tokens such as [BOS] that absorb a large share of attention, and frame it as a statistical problem. Softmax attention is only an optimal estimator if logit noise is i.i.d. Gumbel. Their diagnostics on LLaVA-1.5 show the noise is structured instead: sinks compete directly with visual tokens and suppress them.

To correct this, they propose Latent Attention Denoising (LAD), a training-free, plug-and-play method that operates on pre-softmax logits. LAD defines an energy function that penalizes sink attention and rewards attention to image tokens. It then applies one analytic, Langevin-style update step. A mode switch uses an adaptive gate during the prefill phase and injects noise during decoding.

On benchmarks including CHAIR and POPE, LAD reduces hallucinations across several architectures. It runs almost as fast as greedy decoding and much faster than contrastive decoding. This makes it a cheap, theoretically grounded alternative to heuristic attention fixes.

## Key points

- **Diagnosis setup:** Attention states came from a mid-level decoder layer of LLaVA-1.5 on 2,000 MS-COCO samples. The logit residuals of sink, text and image tokens follow clearly different distributions, which breaks the i.i.d. Gumbel assumption behind softmax.
- **Method:** The energy adds a sink penalty and a visual incentive. The default hyperparameters are η = 2.1 and σ = 0.1, where the noise acts as a regularizer. After LAD, the residual distribution moves close to the Gumbel shape.
- **Ablation (LLaVA-1.5):** The full method appears to reach CHAIR_S 19.6, CHAIR_I 5.1 and F1 77.1, though the extracted table is garbled. Removing the Sink Penalty raises CHAIR_S to 35.8, which makes it the most important component.
- **Efficiency (LLaVA-1.5-7B):**
  - LAD: 27.99 TPS, 3.98 s latency, 14.33 GB memory.
  - Greedy decoding: 30.31 TPS, 3.65 s latency, 14.02 GB memory.
  - Alternatives are much slower: VCD at 10.38 TPS and OPERA at 2.50 TPS.
- **Limitations:** LAD cannot fix knowledge or encoder deficits in the backbone. It targets object- and attribute-level errors rather than reasoning errors. Hyperparameters must be tuned per architecture.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Zhiwen Luo, Siyu Jiang, Weilong Jiang and Kun He. 2026. Latent Attention Denoising: A Training-Free Energy-Based Framework for Mitigating Hallucinations in Vision-Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{luo2026latent,
  title = {{Latent Attention Denoising: A Training-Free Energy-Based Framework for Mitigating Hallucinations in Vision-Language Models}},
  author = {Zhiwen Luo, Siyu Jiang, Weilong Jiang and Kun He},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1327},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1327}
}
```

- [← previous Late Code Chunking: A Code Chunking Strategy for Repository-Level Code Completion](https://md.research.elamur.ai/free/en/papers/oh-2026-late-code-chunking-code-chunking/)
- [next → Latent Narratives at AR-MS NakbaNLP 2026: Reducing Character Errors in Arabic Manuscript Transcription: A CER Oriented System](https://md.research.elamur.ai/free/en/papers/shaza-2026-latent-narratives-ar-ms-nakbanlp/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/)

## References

95 references, 36 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Shun-Ichi Amari. 1998. Natural gradient works efficiently in learning. Neural computation, 10(2):251– 276. [find](https://scholar.google.com/scholar?q=Shun%2DIchi+Amari%2E+1998%2E+Natural+gradient+works+efficiently+in+learning%2E+Neural+computation%2C+10%282%29%3A251%E2%80%93+276%2E)
3. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. [Qwen-vl: A frontier large vision-language model with versatile abilities](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). arXiv preprint arXiv:2308.12966, 1(2):3.
4. Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. 2024. Hallucination of multimodal [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). arXiv preprint arXiv:2404.18930. lvlms. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 21635–2164…
5. Hila Chefer, Shir Gur, and Lior Wolf. 2021. Transformer interpretability beyond attention visualization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 782–791. [find](https://scholar.google.com/scholar?q=Hila+Chefer%2C+Shir+Gur%2C+and+Lior+Wolf%2E+2021%2E+Transformer+interpretability+beyond+attention+visualization%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pages+782)
6. Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. 2023. [Shikra: Unleashing multimodal llm’s referential dialogue magic](https://md.research.elamur.ai/free/en/papers/chen-2023-shikra-unleashing-multimodal-llm-s/). arXiv preprint arXiv:2306.15195.
7. Xuweiyi Chen, Ziqiao Ma, Xuejun Zhang, Sihan Xu, Shengyi Qian, Jianing Yang, David Fouhey, and Joyce Chai. 2024. Multi-object hallucination in vision language models. Advances in Neural Information Processing Systems, 37:44393–44418. [find](https://scholar.google.com/scholar?q=Xuweiyi+Chen%2C+Ziqiao+Ma%2C+Xuejun+Zhang%2C+Sihan+Xu%2C+Shengyi+Qian%2C+Jianing+Yang%2C+David+Fouhey%2C+and+Joyce+Chai%2E+2024%2E+Multi%2Dobject+hallucination+in+vision+language+models%2E+Advances+in+Neural+Information+Pr)
8. Wei-Lin Chiang, Zhuohan Li, Ziqing Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See [https://vicuna.lmsys](https://vicuna.lmsys). org (accessed 14 April 2023), 2(3):6. [↗](https://vicuna.lmsys)
9. Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R Glass, and Pengcheng He. 2023. [Dola: Decoding by contrasting layers improves factuality in large language models](https://md.research.elamur.ai/free/en/papers/chuang-2023-dola-decoding-contrasting-layers-improves/). In The Twelfth International Conference on Learning Representations.
10. Manuel Cossio. 2025. A comprehensive taxonomy of hallucinations in large language models. arXiv preprint arXiv:2508.01781. — [arXiv:2508.01781](https://arxiv.org/abs/2508.01781) [find](https://scholar.google.com/scholar?q=Manuel+Cossio%2E+2025%2E+A+comprehensive+taxonomy+of+hallucinations+in+large+language+models%2E+arXiv+preprint+arXiv%3A2508%2E01781%2E+%E2%80%94+%5BarXiv%3A2508%2E01781%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2508%2E01781%29)
11. Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao. 2023. Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges. arXiv preprint arXiv:2311.03287. — [arXiv:2311.03287](https://arxiv.org/abs/2311.03287) [find](https://scholar.google.com/scholar?q=Chenhang+Cui%2C+Yiyang+Zhou%2C+Xinyu+Yang%2C+Shirley+Wu%2C+Linjun+Zhang%2C+James+Zou%2C+and+Huaxiu+Yao%2E+2023%2E+Holistic+analysis+of+hallucination+in+gpt%2D4v+%28ision%29%3A+Bias+and+interference+challenges%2E+arXiv+preprint)
12. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. 2023. [Instructblip: Towards general-purpose vision-language models with instruction tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/). Advances in Neural Information Processing Systems, 36:49250–49267.
13. Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. 2023. Vision transformers need registers. arXiv preprint arXiv:2309.16588. — [arXiv:2309.16588](https://arxiv.org/abs/2309.16588) [find](https://scholar.google.com/scholar?q=Timoth%C3%A9e+Darcet%2C+Maxime+Oquab%2C+Julien+Mairal%2C+and+Piotr+Bojanowski%2E+2023%2E+Vision+transformers+need+registers%2E+arXiv+preprint+arXiv%3A2309%2E16588%2E+%E2%80%94+%5BarXiv%3A2309%2E16588%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2309%2E16588%29)
14. Gianni Brauwers and Flavius Frasincar. 2021. A general survey on attention mechanisms in deep learning. IEEE transactions on knowledge and data engineering, 35(4):3279–3298. [find](https://scholar.google.com/scholar?q=Gianni+Brauwers+and+Flavius+Frasincar%2E+2021%2E+A+general+survey+on+attention+mechanisms+in+deep+learning%2E+IEEE+transactions+on+knowledge+and+data+engineering%2C+35%284%29%3A3279%E2%80%933298%2E)
15. Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton, Yee Whye Teh, and Arnaud Doucet. 2022. Riemannian score-based generative modelling. Advances in Neural Information Processing Systems, 35:2406–2422. [find](https://scholar.google.com/scholar?q=Valentin+De+Bortoli%2C+Emile+Mathieu%2C+Michael+Hutchinson%2C+James+Thornton%2C+Yee+Whye+Teh%2C+and+Arnaud+Doucet%2E+2022%2E+Riemannian+score%2Dbased+generative+modelling%2E+Advances+in+Neural+Information+Processing+Sy)
16. Liwei Che, Tony Qingze Liu, Jing Jia, Weiyi Qin, Ruixiang Tang, and Vladimir Pavlovic. 2025. Hallucinatory image tokens: A training-free eazy approach to detecting and mitigating object hallucinations in Tim Dockhorn, Arash Vahdat, and Karsten Kreis. 2021. Score-based generative modeling with critic… — [arXiv:2112.07068](https://arxiv.org/abs/2112.07068) [find](https://scholar.google.com/scholar?q=Liwei+Che%2C+Tony+Qingze+Liu%2C+Jing+Jia%2C+Weiyi+Qin%2C+Ruixiang+Tang%2C+and+Vladimir+Pavlovic%2E+2025%2E+Hallucinatory+image+tokens%3A+A+training%2Dfree+eazy+approach+to+detecting+and+mitigating+object+hallucinations)
17. Yilun Du and Igor Mordatch. 2019. Implicit generation and modeling with energy based models. Advances in Neural Information Processing Systems, 32. [find](https://scholar.google.com/scholar?q=Yilun+Du+and+Igor+Mordatch%2E+2019%2E+Implicit+generation+and+modeling+with+energy+based+models%2E+Advances+in+Neural+Information+Processing+Systems%2C+32%2E)
18. Mehrdad Fazli, Bowen Wei, Ahmet Sari, and Ziwei Zhu. 2025. Mitigating hallucination in large vision-language models via adaptive attention calibration. arXiv preprint arXiv:2505.21472. — [arXiv:2505.21472](https://arxiv.org/abs/2505.21472) [find](https://scholar.google.com/scholar?q=Mehrdad+Fazli%2C+Bowen+Wei%2C+Ahmet+Sari%2C+and+Ziwei+Zhu%2E+2025%2E+Mitigating+hallucination+in+large+vision%2Dlanguage+models+via+adaptive+attention+calibration%2E+arXiv+preprint+arXiv%3A2505%2E21472%2E+%E2%80%94+%5BarXiv%3A2505%2E2)
19. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. 2025. [Mme: A comprehensive evaluation benchmark for multimodal large language models](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/). In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets an…
20. Will Grathwohl, Kuan-Chieh Wang, Jörn-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. 2019. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv:1912.03263. — [arXiv:1912.03263](https://arxiv.org/abs/1912.03263) [find](https://scholar.google.com/scholar?q=Will+Grathwohl%2C+Kuan%2DChieh+Wang%2C+J%C3%B6rn%2DHenrik+Jacobsen%2C+David+Duvenaud%2C+Mohammad+Norouzi%2C+and+Kevin+Swersky%2E+2019%2E+Your+classifier+is+secretly+an+energy+based+model+and+you+should+treat+it+like+one%2E+ar)
21. Wen Huang, Hongbin Liu, Minxin Guo, and Neil Zhenqiang Gong. 2024b. Visual hallucinations of multimodal large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 9614–9631, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wen+Huang%2C+Hongbin+Liu%2C+Minxin+Guo%2C+and+Neil+Zhenqiang+Gong%2E+2024b%2E+Visual+hallucinations+of+multimodal+large+language+models%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2024%2C+p)
22. Fushuo Huo, Wenchao Xu, Zhong Zhang, Haozhao Wang, Zhicheng Chen, and Peilin Zhao. 2025. Self-introspective decoding: Alleviating hallucinations for large vision-language models. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Fushuo+Huo%2C+Wenchao+Xu%2C+Zhong+Zhang%2C+Haozhao+Wang%2C+Zhicheng+Chen%2C+and+Peilin+Zhao%2E+2025%2E+Self%2Dintrospective+decoding%3A+Alleviating+hallucinations+for+large+vision%2Dlanguage+models%2E+In+The+Thirteenth+Int)
23. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. [Survey of hallucination in natural language generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/). ACM computing surveys, 55(12):1–38.
24. Daniel Kahneman. 2011. Thinking, Fast and Slow, first paperback edition. Farrar, Straus and Giroux, New York. [find](https://scholar.google.com/scholar?q=Daniel+Kahneman%2E+2011%2E+Thinking%2C+Fast+and+Slow%2C+first+paperback+edition%2E+Farrar%2C+Straus+and+Giroux%2C+New+York%2E)
25. Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2024. [Hallusionbench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models](https://md.research.elamur.ai/free/en/papers/guan-2023-hallusionbench-advanced-diagnostic-suite-entangled/).…
26. Seil Kang, Jinyeong Kim, Junhyeok Kim, and Seong Jae Hwang. 2025. See what you are told: Visual attention sink in large multimodal models. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Seil+Kang%2C+Jinyeong+Kim%2C+Junhyeok+Kim%2C+and+Seong+Jae+Hwang%2E+2025%2E+See+what+you+are+told%3A+Visual+attention+sink+in+large+multimodal+models%2E+In+The+Thirteenth+International+Conference+on+Learning+Repres)
27. Jinghan He, Kuan Zhu, Haiyun Guo, Junfeng Fang, Zhenglin Hua, Yuheng Jia, Ming Tang, Tat-Seng Chua, and Jinqiao Wang. 2025. [Cracking the code of hallucination in LVLMs with vision-aware head divergence](https://md.research.elamur.ai/free/en/papers/he-2025-cracking-code-hallucination-lvlms-vision/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volum…
28. Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2024. [Mitigating object hallucinations in large vision-language models through visual contrastive decoding](https://md.research.elamur.ai/free/en/papers/leng-2023-mitigating-object-hallucinations-large-vision/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13872–13…
29. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840– 6851. [find](https://scholar.google.com/scholar?q=Jonathan+Ho%2C+Ajay+Jain%2C+and+Pieter+Abbeel%2E+2020%2E+Denoising+diffusion+probabilistic+models%2E+Advances+in+Neural+Information+Processing+Systems%2C+33%3A6840%E2%80%93+6851%2E)
30. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). arXiv preprint arXiv:1904.09751.
31. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2025. [A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). ACM Transactions on Information Systems, 43(2):…
32. Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. 2024a. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Conference on Computer Vi… [find](https://scholar.google.com/scholar?q=Qidong+Huang%2C+Xiaoyi+Dong%2C+Pan+Zhang%2C+Bin+Wang%2C+Conghui+He%2C+Jiaqi+Wang%2C+Dahua+Lin%2C+Weiming+Zhang%2C+and+Nenghai+Yu%2E+2024a%2E+Opera%3A+Alleviating+hallucination+in+multi%2Dmodal+large+language+models+via+over%2D)
33. Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, Fujie Huang, et al. 2006. A tutorial on energy-based learning. Predicting structured data, 1(0). [find](https://scholar.google.com/scholar?q=Yann+LeCun%2C+Sumit+Chopra%2C+Raia+Hadsell%2C+M+Ranzato%2C+Fujie+Huang%2C+et+al%2E+2006%2E+A+tutorial+on+energy%2Dbased+learning%2E+Predicting+structured+data%2C+1%280%29%2E)
34. Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Joshua Adrian Cahyono, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2025. [Otter: A multi-modal model with in-context instruction tuning](https://md.research.elamur.ai/free/en/papers/li-2023-otter-multi-modal-model-context/). IEEE Transactions on Pattern Analysis and Machine Intelligence.
35. Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2023a. [Llava-med: Training a large language-and-vision assistant for biomedicine in one day](https://md.research.elamur.ai/free/en/papers/li-2023-llava-med-training-large-language/). Advances in Neural Information Processing Systems, 36:28541– 28564.
36. Xi-Lin Li. 2018. Preconditioned stochastic gradient descent. IEEE Transactions on Neural Networks and Learning Systems, 29(5):1454–1466. [find](https://scholar.google.com/scholar?q=Xi%2DLin+Li%2E+2018%2E+Preconditioned+stochastic+gradient+descent%2E+IEEE+Transactions+on+Neural+Networks+and+Learning+Systems%2C+29%285%29%3A1454%E2%80%931466%2E)
37. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori B Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023b. [Contrastive decoding: Open-ended text generation as optimization](https://md.research.elamur.ai/free/en/papers/li-2022-contrastive-decoding-open-ended-text/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1…
38. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023c. [Evaluating object hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-object-hallucination-large-vision/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 292–305.
39. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. [Microsoft coco: Common objects in context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/). In European Conference on Computer Vision, pages 740–755. Springer.
40. Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. 2024a. [Mitigating hallucination in large multi-modal models via robust instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-mitigating-hallucination-large-multi-modal/). In The Twelfth International Conference on Learning Representations.
41. Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. 2024b. A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253. [find](https://scholar.google.com/scholar?q=Hanchao+Liu%2C+Wenyuan+Xue%2C+Yifei+Chen%2C+Dapeng+Chen%2C+Xiutian+Zhao%2C+Ke+Wang%2C+Liping+Hou%2C+Rongjun+Li%2C+and+Wei+Peng%2E+2024b%2E+A+survey+on+hallucination+in+large+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2)
42. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024c. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306.
43. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024d. Llava-next: Improved reasoning, ocr, and world knowledge. LLaVA-NeXT Blog. [find](https://scholar.google.com/scholar?q=Haotian+Liu%2C+Chunyuan+Li%2C+Yuheng+Li%2C+Bo+Li%2C+Yuanhan+Zhang%2C+Sheng+Shen%2C+and+Yong+Jae+Lee%2E+2024d%2E+Llava%2Dnext%3A+Improved+reasoning%2C+ocr%2C+and+world+knowledge%2E+LLaVA%2DNeXT+Blog%2E)
44. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). Advances in neural information processing systems, 36:34892– 34916.
45. Shi Liu, Kecheng Zheng, and Wei Chen. 2024e. Paying more attention to image: A training-free method for alleviating hallucination in lvlms. In European Conference on Computer Vision, pages 125–140. Springer. [find](https://scholar.google.com/scholar?q=Shi+Liu%2C+Kecheng+Zheng%2C+and+Wei+Chen%2E+2024e%2E+Paying+more+attention+to+image%3A+A+training%2Dfree+method+for+alleviating+hallucination+in+lvlms%2E+In+European+Conference+on+Computer+Vision%2C+pages+125%E2%80%93140%2E+Sp)
46. Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2018. Neural baby talk. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7219–7228. [find](https://scholar.google.com/scholar?q=Jiasen+Lu%2C+Jianwei+Yang%2C+Dhruv+Batra%2C+and+Devi+Parikh%2E+2018%2E+Neural+baby+talk%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+pages+7219%E2%80%937228%2E)
47. Daniel McFadden. 1972. Conditional logit analysis of qualitative choice behavior. [find](https://scholar.google.com/scholar?q=Daniel+McFadden%2E+1972%2E+Conditional+logit+analysis+of+qualitative+choice+behavior%2E)
48. Fanfei Meng and Yuxin Wang. 2023. Transformers: Statistical interpretation, architectures and applications. Authorea Preprints. [find](https://scholar.google.com/scholar?q=Fanfei+Meng+and+Yuxin+Wang%2E+2023%2E+Transformers%3A+Statistical+interpretation%2C+architectures+and+applications%2E+Authorea+Preprints%2E)
49. Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. 2023. Med-flamingo: a multimodal medical few-shot learner. In Machine Learning for Health (ML4H), pages 353–367. PMLR. [find](https://scholar.google.com/scholar?q=Michael+Moor%2C+Qian+Huang%2C+Shirley+Wu%2C+Michihiro+Yasunaga%2C+Yash+Dalmia%2C+Jure+Leskovec%2C+Cyril+Zakka%2C+Eduardo+Pontes+Reis%2C+and+Pranav+Rajpurkar%2E+2023%2E+Med%2Dflamingo%3A+a+multimodal+medical+few%2Dshot+learner%2E)
50. Jiaye Qian, Ge Zheng, Yuchen Zhu, and Sibei Yang. 2025. Intervene-all-paths: Unified mitigation of LVLM hallucinations across alignment formats. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Jiaye+Qian%2C+Ge+Zheng%2C+Yuchen+Zhu%2C+and+Sibei+Yang%2E+2025%2E+Intervene%2Dall%2Dpaths%3A+Unified+mitigation+of+LVLM+hallucinations+across+alignment+formats%2E+In+The+Thirty%2Dninth+Annual+Conference+on+Neural+Informa)
51. Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, S.M Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth, and Amitava Das. 2023. [The troubling emergence of hallucination in large language models - an extensive definition, quantification, and prescriptive remediations](https://md.research.elamur.ai/free/en/papers/rawte-2023-troubling-emergence-hallucination-large-language/). In Proceedings of t…
52. H. Risken and T. Frank. 1996. The Fokker-Planck Equation: Methods of Solution and Applications. Springer Series in Synergetics. Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=H%2E+Risken+and+T%2E+Frank%2E+1996%2E+The+Fokker%2DPlanck+Equation%3A+Methods+of+Solution+and+Applications%2E+Springer+Series+in+Synergetics%2E+Springer+Berlin+Heidelberg%2E)
53. Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. [Object hallucination in image captioning](https://md.research.elamur.ai/free/en/papers/rohrbach-2018-object-hallucination-image-captioning/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045.
54. Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. 2024. Driv-elm: Driving with graph visual question answering. In European conference on computer vision, pages 256–274. Springer. [find](https://scholar.google.com/scholar?q=Chonghao+Sima%2C+Katrin+Renz%2C+Kashyap+Chitta%2C+Li+Chen%2C+Hanxue+Zhang%2C+Chengen+Xie%2C+Jens+Bei%C3%9Fwenger%2C+Ping+Luo%2C+Andreas+Geiger%2C+and+Hongyang+Li%2E+2024%2E+Driv%2Delm%3A+Driving+with+graph+visual+question+answering)
55. Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2020. [Score-based generative modeling through stochastic differential equations](https://md.research.elamur.ai/free/en/papers/song-2020-score-based-generative-modeling-through/). arXiv preprint arXiv:2011.13456.
56. Mingjie Sun, Xinlei Chen, J. Zico Kolter, and Zhuang Liu. 2024a. Massive activations in large language models. arXiv preprint arXiv:2402.17762. [find](https://scholar.google.com/scholar?q=Mingjie+Sun%2C+Xinlei+Chen%2C+J%2E+Zico+Kolter%2C+and+Zhuang+Liu%2E+2024a%2E+Massive+activations+in+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E17762%2E)
57. Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liangyan Gui, Yu-Xiong Wang, Yiming Yang, et al. 2024b. [Aligning large multimodal models with factually augmented rlhf](https://md.research.elamur.ai/free/en/papers/sun-2023-aligning-large-multimodal-models-factually/). In Findings of the Association for Computational Linguistics: ACL 2024, pages 13088– 13110…
58. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. [Sequence to sequence learning with neural networks](https://md.research.elamur.ai/free/en/papers/sutskever-2014-sequence-sequence-learning-neural-networks/). Advances in neural information processing systems, 27.
59. Feilong Tang, Zile Huang, Chengzhi Liu, Qiang Sun, Harry Yang, and Ser-Nam Lim. 2025a. Intervening anchor token: Decoding strategy in alleviating hallucinations for mllms. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Feilong+Tang%2C+Zile+Huang%2C+Chengzhi+Liu%2C+Qiang+Sun%2C+Harry+Yang%2C+and+Ser%2DNam+Lim%2E+2025a%2E+Intervening+anchor+token%3A+Decoding+strategy+in+alleviating+hallucinations+for+mllms%2E+In+The+Thirteenth+Internatio)
60. Feilong Tang, Chengzhi Liu, Zhongxing Xu, Ming Hu, Zile Huang, Haochen Xue, Ziyang Chen, Zelin Peng, Zhiwei Yang, Sijin Zhou, et al. 2025b. Seeing far and clearly: Mitigating hallucinations in mllms with attention causal decoding. In Proceedings of the Computer Vision and Pattern Recognition Confere… [find](https://scholar.google.com/scholar?q=Feilong+Tang%2C+Chengzhi+Liu%2C+Zhongxing+Xu%2C+Ming+Hu%2C+Zile+Huang%2C+Haochen+Xue%2C+Ziyang+Chen%2C+Zelin+Peng%2C+Zhiwei+Yang%2C+Sijin+Zhou%2C+et+al%2E+2025b%2E+Seeing+far+and+clearly%3A+Mitigating+hallucinations+in+mllms+w)
61. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
62. Kenneth E Train. 2009. Discrete choice methods with simulation. Cambridge university press. [find](https://scholar.google.com/scholar?q=Kenneth+E+Train%2E+2009%2E+Discrete+choice+methods+with+simulation%2E+Cambridge+university+press%2E)
63. Chongjun Tu, Peng Ye, Dongzhan Zhou, Lei Bai, Gang Yu, Tao Chen, and Wanli Ouyang. 2025. Attention reallocation: Towards zero-cost and controllable hallucination mitigation of mllms. arXiv preprint arXiv:2503.08342. — [arXiv:2503.08342](https://arxiv.org/abs/2503.08342) [find](https://scholar.google.com/scholar?q=Chongjun+Tu%2C+Peng+Ye%2C+Dongzhan+Zhou%2C+Lei+Bai%2C+Gang+Yu%2C+Tao+Chen%2C+and+Wanli+Ouyang%2E+2025%2E+Attention+reallocation%3A+Towards+zero%2Dcost+and+controllable+hallucination+mitigation+of+mllms%2E+arXiv+preprint+ar)
64. Pascal Vincent. 2011. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661–1674. [find](https://scholar.google.com/scholar?q=Pascal+Vincent%2E+2011%2E+A+connection+between+score+matching+and+denoising+autoencoders%2E+Neural+computation%2C+23%287%29%3A1661%E2%80%931674%2E)
65. Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Jiaqi Wang, Haiyang Xu, Ming Yan, Ji Zhang, et al. 2023a. Amber: An llm-free multi-dimensional benchmark for mllms hallucination evaluation. arXiv preprint arXiv:2311.07397. [find](https://scholar.google.com/scholar?q=Junyang+Wang%2C+Yuhang+Wang%2C+Guohai+Xu%2C+Jing+Zhang%2C+Yukai+Gu%2C+Haitao+Jia%2C+Jiaqi+Wang%2C+Haiyang+Xu%2C+Ming+Yan%2C+Ji+Zhang%2C+et+al%2E+2023a%2E+Amber%3A+An+llm%2Dfree+multi%2Ddimensional+benchmark+for+mllms+hallucination)
66. Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. 2023b. [Label words are anchors: An information flow perspective for understanding in-context learning](https://md.research.elamur.ai/free/en/papers/wang-2023-label-words-anchors-information-flow/). arXiv preprint arXiv:2305.14160.
67. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024a. [Qwen2-vl: Enhancing vision-language model’s perception of the…](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
68. Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. 2024b. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. In ACL (Findings). [find](https://scholar.google.com/scholar?q=Xintong+Wang%2C+Jingheng+Pan%2C+Liang+Ding%2C+and+Chris+Biemann%2E+2024b%2E+Mitigating+hallucinations+in+large+vision%2Dlanguage+models+with+instruction+contrastive+decoding%2E+In+ACL+%28Findings%29%2E)
69. Max Welling and Yee W Teh. 2011. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 681–688. [find](https://scholar.google.com/scholar?q=Max+Welling+and+Yee+W+Teh%2E+2011%2E+Bayesian+learning+via+stochastic+gradient+langevin+dynamics%2E+In+Proceedings+of+the+28th+International+Conference+on+Machine+Learning+%28ICML%2D11%29%2C+pages+681%E2%80%93688%2E)
70. Sangmin Woo, Donguk Kim, Jaehyuk Jang, Yubin Choi, and Changick Kim. 2025. Don’t miss the forest for the trees: Attentional vision calibration for large vision language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 1927–1951. [find](https://scholar.google.com/scholar?q=Sangmin+Woo%2C+Donguk+Kim%2C+Jaehyuk+Jang%2C+Yubin+Choi%2C+and+Changick+Kim%2E+2025%2E+Don%E2%80%99t+miss+the+forest+for+the+trees%3A+Attentional+vision+calibration+for+large+vision+language+models%2E+In+Findings+of+the+Asso)
71. Peng Xia, Ze Chen, Juanxi Tian, Yangrui Gong, Ruibo Hou, Yue Xu, Zhenbang Wu, Zhiyuan Fan, Yiyang Zhou, Kangyu Zhu, et al. 2024. Cares: A comprehensive benchmark of trustworthiness in medical vision language models. Advances in Neural Information Processing Systems, 37:140334–140365. [find](https://scholar.google.com/scholar?q=Peng+Xia%2C+Ze+Chen%2C+Juanxi+Tian%2C+Yangrui+Gong%2C+Ruibo+Hou%2C+Yue+Xu%2C+Zhenbang+Wu%2C+Zhiyuan+Fan%2C+Yiyang+Zhou%2C+Kangyu+Zhu%2C+et+al%2E+2024%2E+Cares%3A+A+comprehensive+benchmark+of+trustworthiness+in+medical+vision+l)
72. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2023. Efficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453. [find](https://scholar.google.com/scholar?q=Guangxuan+Xiao%2C+Yuandong+Tian%2C+Beidi+Chen%2C+Song+Han%2C+and+Mike+Lewis%2E+2023%2E+Efficient+streaming+language+models+with+attention+sinks%2E+arXiv+preprint+arXiv%3A2309%2E17453%2E)
73. Chunzhao Xie, Tongxuan Liu, Lei Jiang, Yuting Zeng, Yunheng Shen, Weizhe Huang, Jing Li, Xiaohua Xu, et al. 2025. Tarac: Mitigating hallucination in lvlms via temporal attention real-time accumulative connection. arXiv preprint arXiv:2504.04099. — [arXiv:2504.04099](https://arxiv.org/abs/2504.04099) [find](https://scholar.google.com/scholar?q=Chunzhao+Xie%2C+Tongxuan+Liu%2C+Lei+Jiang%2C+Yuting+Zeng%2C+Yunheng+Shen%2C+Weizhe+Huang%2C+Jing+Li%2C+Xiaohua+Xu%2C+et+al%2E+2025%2E+Tarac%3A+Mitigating+hallucination+in+lvlms+via+temporal+attention+real%2Dtime+accumulative)
74. Jianwen Xie, Yang Lu, Song-Chun Zhu, and Yingnian Wu. 2016. A theory of generative convnet. In International Conference on Machine Learning, pages 2635–2644. PMLR. [find](https://scholar.google.com/scholar?q=Jianwen+Xie%2C+Yang+Lu%2C+Song%2DChun+Zhu%2C+and+Yingnian+Wu%2E+2016%2E+A+theory+of+generative+convnet%2E+In+International+Conference+on+Machine+Learning%2C+pages+2635%E2%80%932644%2E+PMLR%2E)
75. Xinhao Xu, Hui Chen, Mengyao Lyu, Sicheng Zhao, Yizhe Xiong, Zijia Lin, Jungong Han, and Guiguang Ding. 2025. Mitigating hallucinations in multi-modal large language models via image token attention-guided decoding. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the… [find](https://scholar.google.com/scholar?q=Xinhao+Xu%2C+Hui+Chen%2C+Mengyao+Lyu%2C+Sicheng+Zhao%2C+Yizhe+Xiong%2C+Zijia+Lin%2C+Jungong+Han%2C+and+Guiguang+Ding%2E+2025%2E+Mitigating+hallucinations+in+multi%2Dmodal+large+language+models+via+image+token+attention%2Dg)
76. Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. 2024. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robotics and Automation Letters. [find](https://scholar.google.com/scholar?q=Zhenhua+Xu%2C+Yujia+Zhang%2C+Enze+Xie%2C+Zhen+Zhao%2C+Yong+Guo%2C+Kwan%2DYee+K+Wong%2C+Zhenguo+Li%2C+and+Hengshuang+Zhao%2E+2024%2E+Drivegpt4%3A+Interpretable+end%2Dto%2Dend+autonomous+driving+via+large+language+model%2E+IEEE+Ro)
77. Tijin Yan, Hongwei Zhang, Tong Zhou, Yufeng Zhan, and Yuanqing Xia. 2021. Scoregrad: Multivariate probabilistic time series forecasting with continuous energy-based generative models. arXiv preprint arXiv:2106.10121. — [arXiv:2106.10121](https://arxiv.org/abs/2106.10121) [find](https://scholar.google.com/scholar?q=Tijin+Yan%2C+Hongwei+Zhang%2C+Tong+Zhou%2C+Yufeng+Zhan%2C+and+Yuanqing+Xia%2E+2021%2E+Scoregrad%3A+Multivariate+probabilistic+time+series+forecasting+with+continuous+energy%2Dbased+generative+models%2E+arXiv+preprint+a)
78. Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. 2023. [mplug-owl: Modularization empowers large language models with multimodality](https://md.research.elamur.ai/free/en/papers/ye-2023-mplug-owl-modularization-empowers-large/). arXiv preprint arXiv:2304.14178.
79. Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. 2024. Woodpecker: Hallucination correction for multimodal large language models. Science China Information Sciences, 67(12):220105. [find](https://scholar.google.com/scholar?q=Shukang+Yin%2C+Chaoyou+Fu%2C+Sirui+Zhao%2C+Tong+Xu%2C+Hao+Wang%2C+Dianbo+Sui%2C+Yunhang+Shen%2C+Ke+Li%2C+Xing+Sun%2C+and+Enhong+Chen%2E+2024%2E+Woodpecker%3A+Hallucination+correction+for+multimodal+large+language+models%2E+Sci)
80. Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. 2024. [Mm-vet: Evaluating large multimodal models for integrated capabilities](https://md.research.elamur.ai/free/en/papers/yu-2023-mm-vet-evaluating-large-multimodal/). In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Lear…
81. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. [Sigmoid loss for language image pre-training](https://md.research.elamur.ai/free/en/papers/zhai-2023-sigmoid-loss-language-image-pre/). In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11975–11986.
82. Xiaofeng Zhang, Yihao Quan, Chaochen Gu, Chen Shen, Xiaosong Yuan, Shaotian Yan, Hao Cheng, Kaijie Wu, and Jieping Ye. 2024a. Seeing clearly by layer two: Enhancing attention heads to alleviate hallucination in lvlms. arXiv preprint arXiv:2411.09968. — [arXiv:2411.09968](https://arxiv.org/abs/2411.09968) [find](https://scholar.google.com/scholar?q=Xiaofeng+Zhang%2C+Yihao+Quan%2C+Chaochen+Gu%2C+Chen+Shen%2C+Xiaosong+Yuan%2C+Shaotian+Yan%2C+Hao+Cheng%2C+Kaijie+Wu%2C+and+Jieping+Ye%2E+2024a%2E+Seeing+clearly+by+layer+two%3A+Enhancing+attention+heads+to+alleviate+halluc)
83. Xiaofeng Zhang, Yihao Quan, Chen Shen, Chaochen Gu, Xiaosong Yuan, Shaotian Yan, Jiawei Cao, Hao Cheng, Kaijie Wu, and Jieping Ye. 2025a. [Shallow focus, deep fixes: Enhancing shallow layers vision attention sinks to alleviate hallucination in lvlms](https://md.research.elamur.ai/free/en/papers/zhang-2025-shallow-focus-deep-fixes-enhancing/). In Proceedings of the 2025 Conference on Empirical…
84. Yue Zhang, Leyang Cui, Wei Bi, and Shuming Shi. 2024b. Alleviating hallucinations of large language models through induced hallucinations. [find](https://scholar.google.com/scholar?q=Yue+Zhang%2C+Leyang+Cui%2C+Wei+Bi%2C+and+Shuming+Shi%2E+2024b%2E+Alleviating+hallucinations+of+large+language+models+through+induced+hallucinations%2E)
85. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2025b. [Siren’s song in the ai ocean: A survey on hallucination in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/). Computational Linguistics, pages 1–46. hallucination in large vision-language models. In…
86. Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. [Minigpt-4: Enhancing vision-language understanding with advanced large language models](https://md.research.elamur.ai/free/en/papers/zhu-2023-minigpt-4-enhancing-vision-language/). arXiv preprint arXiv:2304.10592.
87. Younan Zhu, Linwei Tao, Minjing Dong, and Chang Xu. 2025. Mitigating object hallucinations in large vision-language models via attention calibration. arXiv preprint arXiv:2502.01969. — [arXiv:2502.01969](https://arxiv.org/abs/2502.01969) [find](https://scholar.google.com/scholar?q=Younan+Zhu%2C+Linwei+Tao%2C+Minjing+Dong%2C+and+Chang+Xu%2E+2025%2E+Mitigating+object+hallucinations+in+large+vision%2Dlanguage+models+via+attention+calibration%2E+arXiv+preprint+arXiv%3A2502%2E01969%2E+%E2%80%94+%5BarXiv%3A2502%2E019)
88. Xianwei Zhuang, Zhihong Zhu, Yuxin Xie, Liming Liang, and Yuexian Zou. 2025. Vasparse: Towards efficient visual hallucination mitigation via visual-aware token sparsification. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 4189–4199. [find](https://scholar.google.com/scholar?q=Xianwei+Zhuang%2C+Zhihong+Zhu%2C+Yuxin+Xie%2C+Liming+Liang%2C+and+Yuexian+Zou%2E+2025%2E+Vasparse%3A+Towards+efficient+visual+hallucination+mitigation+via+visual%2Daware+token+sparsification%2E+In+Proceedings+of+the+Co)
89. Linxi Zhao, Yihe Deng, Weitong Zhang, and Quanquan Gu. 2024. Mitigating object hallucination in large vision-language models via image-grounded guidance. arXiv preprint arXiv:2402.08680. — [arXiv:2402.08680](https://arxiv.org/abs/2402.08680) [find](https://scholar.google.com/scholar?q=Linxi+Zhao%2C+Yihe+Deng%2C+Weitong+Zhang%2C+and+Quanquan+Gu%2E+2024%2E+Mitigating+object+hallucination+in+large+vision%2Dlanguage+models+via+image%2Dgrounded+guidance%2E+arXiv+preprint+arXiv%3A2402%2E08680%2E+%E2%80%94+%5BarXiv%3A2402)
90. Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. 2023. Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization. arXiv preprint arXiv:2311.16839. — [arXiv:2311.16839](https://arxiv.org/abs/2311.16839) [find](https://scholar.google.com/scholar?q=Zhiyuan+Zhao%2C+Bin+Wang%2C+Linke+Ouyang%2C+Xiaoyi+Dong%2C+Jiaqi+Wang%2C+and+Conghui+He%2E+2023%2E+Beyond+hallucinations%3A+Enhancing+lvlms+through+hallucination%2Daware+direct+preference+optimization%2E+arXiv+preprint+a)
91. Haohan Zheng and Zhenguo Zhang. 2025. Modality bias in lvlms: Analyzing and mitigating object hallucination via attention lens. arXiv preprint arXiv:2508.02419. — [arXiv:2508.02419](https://arxiv.org/abs/2508.02419) [find](https://scholar.google.com/scholar?q=Haohan+Zheng+and+Zhenguo+Zhang%2E+2025%2E+Modality+bias+in+lvlms%3A+Analyzing+and+mitigating+object+hallucination+via+attention+lens%2E+arXiv+preprint+arXiv%3A2508%2E02419%2E+%E2%80%94+%5BarXiv%3A2508%2E02419%5D%28https%3A%2F%2Farxiv%2Eorg%2F)
92. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in Neural Information Processing Systems, 36:46595–46623.
93. Weihong Zhong, Xiaocheng Feng, Liang Zhao, Qiming Li, Lei Huang, Yuxuan Gu, Weitao Ma, Yuan Xu, and Bing Qin. 2024. Investigating and mitigating the multimodal hallucination snowballing in large vision-language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Li… [find](https://scholar.google.com/scholar?q=Weihong+Zhong%2C+Xiaocheng+Feng%2C+Liang+Zhao%2C+Qiming+Li%2C+Lei+Huang%2C+Yuxuan+Gu%2C+Weitao+Ma%2C+Yuan+Xu%2C+and+Bing+Qin%2E+2024%2E+Investigating+and+mitigating+the+multimodal+hallucination+snowballing+in+large+visio)
94. Haoran Zhou, Zihan Zhang, and Hao Chen. 2025. Extracting visual facts from intermediate layers for mitigating hallucinations in multimodal large language models. arXiv preprint arXiv:2507.15652. — [arXiv:2507.15652](https://arxiv.org/abs/2507.15652) [find](https://scholar.google.com/scholar?q=Haoran+Zhou%2C+Zihan+Zhang%2C+and+Hao+Chen%2E+2025%2E+Extracting+visual+facts+from+intermediate+layers+for+mitigating+hallucinations+in+multimodal+large+language+models%2E+arXiv+preprint+arXiv%3A2507%2E15652%2E+%E2%80%94+%5Bar)
95. Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2024. Analyzing and mitigating object [find](https://scholar.google.com/scholar?q=Yiyang+Zhou%2C+Chenhang+Cui%2C+Jaehong+Yoon%2C+Linjun+Zhang%2C+Zhun+Deng%2C+Chelsea+Finn%2C+Mohit+Bansal%2C+and+Huaxiu+Yao%2E+2024%2E+Analyzing+and+mitigating+object)

*Updated 1 November 2026*