---
title: "Hallucination of Multimodal Large Language Models: A Survey"
description: "This survey examines cases where multimodal large language models (MLLMs, sometimes termed… 228 references, cited by 90 papers here."
authors: ["Zechen Bai", "Pichao Wang", "Tianjun Xiao", "Tong He", "Zongbo Han", "Zheng Zhang", "Mike Zheng Shou"]
year: 2024
venue: "arXiv:2404.18930"
source: "https://arxiv.org/abs/2404.18930"
doi: "10.48550/arxiv.2404.18930"
arxiv: "2404.18930"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bai-2024-hallucination-multimodal-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bai-2024-hallucination-multimodal-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/3/) / Hallucination of Multimodal Large Language Models: A Survey

# Hallucination of Multimodal Large Language Models: A Survey

**Authors:** Zechen Bai, Pichao Wang, [Tianjun Xiao](https://md.research.elamur.ai/free/en/authors/tianjun-xiao/), Tong He, Zongbo Han, [Zheng Zhang](https://md.research.elamur.ai/free/en/authors/zheng-zhang/) and [Mike Zheng Shou](https://md.research.elamur.ai/free/en/authors/mike-zheng-shou/) · **Year:** 2024 · **Venue:** arXiv:2404.18930 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2404.18930) · [PDF](https://arxiv.org/pdf/2404.18930) · [DOI 10.48550/arxiv.2404.18930](https://doi.org/10.48550/arxiv.2404.18930) · [arXiv 2404.18930](https://arxiv.org/abs/2404.18930) · [OpenAlex](https://openalex.org/W6922346929)

**Citations:** 30 citations (OpenAlex, 2026-10-14) · [90](https://md.research.elamur.ai/free/en/papers/bai-2024-hallucination-multimodal-large-language-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This survey examines cases where multimodal large language models (MLLMs, sometimes termed LVLMs) produce text that contradicts the image they are given, a failure known as hallucination. Text-only LLM work separates factuality errors from faithfulness errors. In MLLMs the main issue is cross-modal mismatch, so LLM findings do not carry over directly. The authors focus on object hallucination, split into three types: category, attribute and relation. They fold counting and events into the attribute type. They trace causes to four stages of the MLLM pipeline: data, model, training and inference. They then catalogue evaluation metrics and benchmarks and group mitigation techniques by the cause each one addresses. Compared with the one earlier short survey on the topic, theirs offers a finer, layered taxonomy and covers more papers. It closes with open problems, chiefly the lack of standardized benchmarks and weak interpretability. A companion GitHub list (Awesome-MLLM-Hallucination) tracks the literature, giving practitioners a structured map for choosing evaluations and fixes.

## Key points
- **Causes:**
  - Data: too little data, noisy data, low diversity, and statistical bias such as frequent objects and object co-occurrence.
  - Model: information loss or feature bias in the vision encoder, parametric knowledge of the LLM, and poor cross-modal alignment.
  - Training: sequence-level, visual and human-feedback supervision.
  - Inference: visual attention deficiency and "trap" visual tokens.
- **Metrics:** CHAIR, POPE, LLM-based judges (GAVIE, HaELM, HallusionBench), and others such as FaithScore and AMBER.
- **Benchmarks:** discriminative ones (POPE, MME, MMBench) and generative ones (MMHal-Bench, AMBER). POPE's yes/no format is popular but does not reflect real usage. Generative benchmarks rely on external expert models or LLMs.
- **Mitigation examples:**
  - Negative and counterfactual training data: LRV-Instruction, HalluciDoctor.
  - Cleaned data: ReCaption, EOS.
  - Higher input resolution: LLaVA-1.5, InternVL.
  - Stronger vision encoders: VCoder.
  - Dedicated modules: HallE-Switch.
- **Future directions:** standardized benchmarks, new architectures, treating hallucination as a creative feature, interpretability, and ethics.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🧠 Neuroscience](https://md.research.elamur.ai/free/en/shelves/neuroscience/) · 186 works · [🧠 Cognitive Neuroscience](https://md.research.elamur.ai/free/en/shelves/neuroscience/cognitive-neuroscience/) · 185 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works

## Cite

Cite as: Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang and Mike Zheng Shou. 2024. Hallucination of Multimodal Large Language Models: A Survey. arXiv:2404.18930.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{bai2024hallucination,
  title = {{Hallucination of Multimodal Large Language Models: A Survey}},
  author = {Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang and Mike Zheng Shou},
  year = {2024},
  doi = {10.48550/arxiv.2404.18930},
  eprint = {2404.18930},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2404.18930}
}
```

- [← previous AlphaMath Almost Zero: Process Supervision without Process](https://md.research.elamur.ai/free/en/papers/chen-2024-alphamath-almost-zero-process-supervision/)
- [next → QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving](https://md.research.elamur.ai/free/en/papers/lin-2024-qserve-w4a8kv4-quantization-system-co/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/3/)

## References

228 references, 83 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Wenbin An, Feng Tian, Sicong Leng, Jiahao Nie, Haonan Lin, QianYing Wang, Guang Dai, Ping Chen, and Shijian Lu. 2024. Agla: Mitigating object hallucinations in large vision-language models with assembly of global and local attention. arXiv preprint arXiv:2406.12718 (2024). — [arXiv:2406.12718](https://arxiv.org/abs/2406.12718) [find](https://scholar.google.com/scholar?q=Wenbin+An%2C+Feng+Tian%2C+Sicong+Leng%2C+Jiahao+Nie%2C+Haonan+Lin%2C+QianYing+Wang%2C+Guang+Dai%2C+Ping+Chen%2C+and+Shijian+Lu%2E+2024%2E+Agla%3A+Mitigating+object+hallucinations+in+large+vision%2Dlanguage+models+with+assemb)
2. Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. 2023. [Openflamingo: An open-source framework for training large autoregressive vision-language models](https://md.research.elamur.ai/free/en/papers/awadalla-2023-openflamingo-open-source-framework-training/). arXiv preprint arXiv:2308.01390 (2023).
3. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. [Qwen-vl: A frontier large vision-language model with versatile abilities](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). arXiv preprint arXiv:2308.12966 (2023).
4. Zechen Bai, Hai Ci, and Mike Zheng Shou. 2025. Impossible Videos. arXiv preprint arXiv:2503.14378 (2025). — [arXiv:2503.14378](https://arxiv.org/abs/2503.14378) [find](https://scholar.google.com/scholar?q=Zechen+Bai%2C+Hai+Ci%2C+and+Mike+Zheng+Shou%2E+2025%2E+Impossible+Videos%2E+arXiv+preprint+arXiv%3A2503%2E14378+%282025%29%2E+%E2%80%94+%5BarXiv%3A2503%2E14378%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2503%2E14378%29)
5. Zechen Bai, Jianxiong Gao, Ziteng Gao, Pichao Wang, Zheng Zhang, Tong He, and Mike Zheng Shou. 2024. Factorized Visual Tokenization and Generation. arXiv preprint arXiv:2411.16681 (2024). — [arXiv:2411.16681](https://arxiv.org/abs/2411.16681) [find](https://scholar.google.com/scholar?q=Zechen+Bai%2C+Jianxiong+Gao%2C+Ziteng+Gao%2C+Pichao+Wang%2C+Zheng+Zhang%2C+Tong+He%2C+and+Mike+Zheng+Shou%2E+2024%2E+Factorized+Visual+Tokenization+and+Generation%2E+arXiv+preprint+arXiv%3A2411%2E16681+%282024%29%2E+%E2%80%94+%5BarXiv%3A241)
6. Zechen Bai, Tong He, Haiyang Mei, Pichao Wang, Ziteng Gao, Joya Chen, Zheng Zhang, and Mike Zheng Shou. 2024. One token to seg them all: Language instructed reasoning segmentation in videos. Advances in Neural Information Processing Systems 37 (2024), 6833–6859. [find](https://scholar.google.com/scholar?q=Zechen+Bai%2C+Tong+He%2C+Haiyang+Mei%2C+Pichao+Wang%2C+Ziteng+Gao%2C+Joya+Chen%2C+Zheng+Zhang%2C+and+Mike+Zheng+Shou%2E+2024%2E+One+token+to+seg+them+all%3A+Language+instructed+reasoning+segmentation+in+videos%2E+Advances+)
7. Zechen Bai, Yuta Nakashima, and Noa Garcia. 2021. Explain me the painting: Multi-topic knowledgeable art description generation. In Proceedings of the IEEE/CVF international conference on computer vision. 5422–5432. [find](https://scholar.google.com/scholar?q=Zechen+Bai%2C+Yuta+Nakashima%2C+and+Noa+Garcia%2E+2021%2E+Explain+me+the+painting%3A+Multi%2Dtopic+knowledgeable+art+description+generation%2E+In+Proceedings+of+the+IEEE%2FCVF+international+conference+on+computer+vis)
8. Zechen Bai, Tianjun Xiao, Tong He, Pichao Wang, Zheng Zhang, Thomas Brox, and Mike Zheng Shou. 2025. Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zechen+Bai%2C+Tianjun+Xiao%2C+Tong+He%2C+Pichao+Wang%2C+Zheng+Zhang%2C+Thomas+Brox%2C+and+Mike+Zheng+Shou%2E+2025%2E+Bridging+Information+Asymmetry+in+Text%2Dvideo+Retrieval%3A+A+Data%2Dcentric+Approach%2E+In+The+Thirteenth+)
9. Satanjeev Banerjee and Alon Lavie. 2005. [METEOR: An automatic metric for MT evaluation with improved correlation with human judgments](https://md.research.elamur.ai/free/en/papers/banerjee-2005-meteor-automatic-metric-mt-evaluation/). In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. 65–72.
10. Rohan Bavishi, Erich Elsen, Curtis Hawthorne, Maxwell Nye, Augustus Odena, Arushi Somani, and Sağnak Taşırlar. 2023. Fuyu-8B: A Multimodal Architecture for AI Agents. [https://www.adept.ai/blog/fuyu-8b](https://www.adept.ai/blog/fuyu-8b) [↗](https://www.adept.ai/blog/fuyu-8b)
11. Assaf Ben-Kish, Moran Yanuka, Morris Alper, Raja Giryes, and Hadar Averbuch-Elor. 2023. MOCHa: Multi-Objective Reinforcement Mitigating Caption Hallucinations. arXiv preprint arXiv:2312.03631 (2023). — [arXiv:2312.03631](https://arxiv.org/abs/2312.03631) [find](https://scholar.google.com/scholar?q=Assaf+Ben%2DKish%2C+Moran+Yanuka%2C+Morris+Alper%2C+Raja+Giryes%2C+and+Hadar+Averbuch%2DElor%2E+2023%2E+MOCHa%3A+Multi%2DObjective+Reinforcement+Mitigating+Caption+Hallucinations%2E+arXiv+preprint+arXiv%3A2312%2E03631+%282023%29%2E+)
12. James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. 2023. Improving image generation with better captions. Computer Science. [https://cdn.openai](https://cdn.openai). com/papers/dall-e-3. pdf 2, 3 (2023), 8. [↗](https://cdn.openai)
13. Ralph Allan Bradley and Milton E Terry. 1952. Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 39, 3/4 (1952), 324–345. [https://www.jstor.org/stable/2334029](https://www.jstor.org/stable/2334029) [↗](https://www.jstor.org/stable/2334029)
14. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
15. Yinzhi Cao and Junfeng Yang. 2015. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy. IEEE, 463–480. [find](https://scholar.google.com/scholar?q=Yinzhi+Cao+and+Junfeng+Yang%2E+2015%2E+Towards+making+systems+forget+with+machine+unlearning%2E+In+2015+IEEE+symposium+on+security+and+privacy%2E+IEEE%2C+463%E2%80%93480%2E)
16. Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. 2021. [Emerging properties in self-supervised vision transformers](https://md.research.elamur.ai/free/en/papers/caron-2021-emerging-properties-self-supervised-vision/). In Proceedings of the IEEE/CVF international conference on computer vision. 9650–9660.
17. Sungguk Cha, Jusung Lee, Younghyun Lee, and Cheoljong Yang. 2024. Visually dehallucinative instruction generation. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 5510–5514. [find](https://scholar.google.com/scholar?q=Sungguk+Cha%2C+Jusung+Lee%2C+Younghyun+Lee%2C+and+Cheoljong+Yang%2E+2024%2E+Visually+dehallucinative+instruction+generation%2E+In+ICASSP+2024%2D2024+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Pro)
18. Yue Chang, Liqiang Jing, Xiaopeng Zhang, and Yue Zhang. 2024. A Unified Hallucination Mitigation Framework for Large Vision-Language Models. arXiv preprint arXiv:2409.16494 (2024). — [arXiv:2409.16494](https://arxiv.org/abs/2409.16494) [find](https://scholar.google.com/scholar?q=Yue+Chang%2C+Liqiang+Jing%2C+Xiaopeng+Zhang%2C+and+Yue+Zhang%2E+2024%2E+A+Unified+Hallucination+Mitigation+Framework+for+Large+Vision%2DLanguage+Models%2E+arXiv+preprint+arXiv%3A2409%2E16494+%282024%29%2E+%E2%80%94+%5BarXiv%3A2409%2E16494)
19. Liwei Che, Tony Qingze Liu, Jing Jia, Weiyi Qin, Ruixiang Tang, and Vladimir Pavlovic. 2025. EAZY: Eliminating Hallucinations in LVLMs by Zeroing out Hallucinatory Image Tokens. arXiv preprint arXiv:2503.07772 (2025). — [arXiv:2503.07772](https://arxiv.org/abs/2503.07772) [find](https://scholar.google.com/scholar?q=Liwei+Che%2C+Tony+Qingze+Liu%2C+Jing+Jia%2C+Weiyi+Qin%2C+Ruixiang+Tang%2C+and+Vladimir+Pavlovic%2E+2025%2E+EAZY%3A+Eliminating+Hallucinations+in+LVLMs+by+Zeroing+out+Hallucinatory+Image+Tokens%2E+arXiv+preprint+arXiv%3A2)
20. Beitao Chen, Xinyu Lyu, Lianli Gao, Jingkuan Song, and Heng Tao Shen. 2024. Alleviating hallucinations in large vision-language models through hallucination-induced optimization. arXiv preprint arXiv:2405.15356 (2024). — [arXiv:2405.15356](https://arxiv.org/abs/2405.15356) [find](https://scholar.google.com/scholar?q=Beitao+Chen%2C+Xinyu+Lyu%2C+Lianli+Gao%2C+Jingkuan+Song%2C+and+Heng+Tao+Shen%2E+2024%2E+Alleviating+hallucinations+in+large+vision%2Dlanguage+models+through+hallucination%2Dinduced+optimization%2E+arXiv+preprint+arXiv%3A)
21. Beitao Chen, Xinyu Lyu, Lianli Gao, Jingkuan Song, and Heng Tao Shen. 2025. Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigation. arXiv preprint arXiv:2503.08216 (2025). — [arXiv:2503.08216](https://arxiv.org/abs/2503.08216) [find](https://scholar.google.com/scholar?q=Beitao+Chen%2C+Xinyu+Lyu%2C+Lianli+Gao%2C+Jingkuan+Song%2C+and+Heng+Tao+Shen%2E+2025%2E+Attention+Hijackers%3A+Detect+and+Disentangle+Attention+Hijacking+in+LVLMs+for+Hallucination+Mitigation%2E+arXiv+preprint+arXiv%3A)
22. Cong Chen, Mingyu Liu, Chenchen Jing, Yizhou Zhou, Fengyun Rao, Hao Chen, Bo Zhang, and Chunhua Shen. 2025. PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Training. arXiv preprint arXiv:2503.06486 (2025). — [arXiv:2503.06486](https://arxiv.org/abs/2503.06486) [find](https://scholar.google.com/scholar?q=Cong+Chen%2C+Mingyu+Liu%2C+Chenchen+Jing%2C+Yizhou+Zhou%2C+Fengyun+Rao%2C+Hao+Chen%2C+Bo+Zhang%2C+and+Chunhua+Shen%2E+2025%2E+PerturboLLaVA%3A+Reducing+Multimodal+Hallucinations+with+Perturbative+Visual+Training%2E+arXiv+p)
23. Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. 2023. [Shikra: Unleashing Multimodal LLM’s Referential Dialogue Magic](https://md.research.elamur.ai/free/en/papers/chen-2023-shikra-unleashing-multimodal-llm-s/). arXiv preprint arXiv:2306.15195 (2023).
24. Xuweiyi Chen, Ziqiao Ma, Xuejun Zhang, Sihan Xu, Shengyi Qian, Jianing Yang, David Fouhey, and Joyce Chai. 2024. Multi-object hallucination in vision language models. Advances in Neural Information Processing Systems 37 (2024), 44393–44418. [find](https://scholar.google.com/scholar?q=Xuweiyi+Chen%2C+Ziqiao+Ma%2C+Xuejun+Zhang%2C+Sihan+Xu%2C+Shengyi+Qian%2C+Jianing+Yang%2C+David+Fouhey%2C+and+Joyce+Chai%2E+2024%2E+Multi%2Dobject+hallucination+in+vision+language+models%2E+Advances+in+Neural+Information+Pr)
25. Xiang Chen, Chenxi Wang, Yida Xue, Ningyu Zhang, Xiaoyan Yang, Qiang Li, Yue Shen, Jinjie Gu, and Huajun Chen. 2024. Unified Hallucination Detection for Multimodal Large Language Models. arXiv preprint arXiv:2402.03190 (2024). [find](https://scholar.google.com/scholar?q=Xiang+Chen%2C+Chenxi+Wang%2C+Yida+Xue%2C+Ningyu+Zhang%2C+Xiaoyan+Yang%2C+Qiang+Li%2C+Yue+Shen%2C+Jinjie+Gu%2C+and+Huajun+Chen%2E+2024%2E+Unified+Hallucination+Detection+for+Multimodal+Large+Language+Models%2E+arXiv+preprin)
26. Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. 2025. [Janus-pro: Unified multimodal understanding and generation with data and model scaling](https://md.research.elamur.ai/free/en/papers/chen-2025-janus-pro-unified-multimodal/). arXiv preprint arXiv:2501.17811 (2025).
27. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. 2024. [Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks](https://md.research.elamur.ai/free/en/papers/chen-2023-internvl-scaling-up-vision/). In Proceedings of the IEEE/CVF conference on computer vision and pattern r…
28. Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. 2024. HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding. arXiv preprint arXiv:2403.00425 (2024). — [arXiv:2403.00425](https://arxiv.org/abs/2403.00425) [find](https://scholar.google.com/scholar?q=Zhaorun+Chen%2C+Zhuokai+Zhao%2C+Hongyin+Luo%2C+Huaxiu+Yao%2C+Bo+Li%2C+and+Jiawei+Zhou%2E+2024%2E+HALC%3A+Object+Hallucination+Reduction+via+Adaptive+Focal%2DContrast+Decoding%2E+arXiv+preprint+arXiv%3A2403%2E00425+%282024%29%2E+%E2%80%94+)
29. Zhiyang Chen, Yousong Zhu, Yufei Zhan, Zhaowen Li, Chaoyang Zhao, Jinqiao Wang, and Ming Tang. 2023. Mitigating Hallucination in Visual Language Models with Visual Supervision. arXiv preprint arXiv:2311.16479 (2023). — [arXiv:2311.16479](https://arxiv.org/abs/2311.16479) [find](https://scholar.google.com/scholar?q=Zhiyang+Chen%2C+Yousong+Zhu%2C+Yufei+Zhan%2C+Zhaowen+Li%2C+Chaoyang+Zhao%2C+Jinqiao+Wang%2C+and+Ming+Tang%2E+2023%2E+Mitigating+Hallucination+in+Visual+Language+Models+with+Visual+Supervision%2E+arXiv+preprint+arXiv%3A23)
30. Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. 2023. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-image generation. arXiv preprint arXiv:2310.18235 (2023). [find](https://scholar.google.com/scholar?q=Jaemin+Cho%2C+Yushi+Hu%2C+Roopal+Garg%2C+Peter+Anderson%2C+Ranjay+Krishna%2C+Jason+Baldridge%2C+Mohit+Bansal%2C+Jordi+Pont%2DTuset%2C+and+Su+Wang%2E+2023%2E+Davidsonian+scene+graph%3A+Improving+reliability+in+fine%2Dgrained+ev)
31. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
32. Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep Reinforcement Learning from Human Preferences. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach… [find](https://scholar.google.com/scholar?q=Paul+F%2E+Christiano%2C+Jan+Leike%2C+Tom+B%2E+Brown%2C+Miljan+Martic%2C+Shane+Legg%2C+and+Dario+Amodei%2E+2017%2E+Deep+Reinforcement+Learning+from+Human+Preferences%2E+In+Advances+in+Neural+Information+Processing+Systems)
33. Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu, Bing Qin, and Ting Liu. 2023. [A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future](https://md.research.elamur.ai/free/en/papers/chu-2023-navigate-through-enigmatic-labyrinth-survey/). ArXiv preprint abs/2309.15402 (2023). https://arxiv.org/abs/2309.15402 [↗](https://arxiv.org/abs/2309.15402)
34. Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. 2023. [Dola: Decoding by contrasting layers improves factuality in large language models](https://md.research.elamur.ai/free/en/papers/chuang-2023-dola-decoding-contrasting-layers-improves/). ArXiv preprint abs/2309.03883 (2023). https://arxiv.org/abs/2309.03883 [↗](https://arxiv.org/abs/2309.03883)
35. Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao. 2023. Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges. arXiv preprint arXiv:2311.03287 (2023). — [arXiv:2311.03287](https://arxiv.org/abs/2311.03287) [find](https://scholar.google.com/scholar?q=Chenhang+Cui%2C+Yiyang+Zhou%2C+Xinyu+Yang%2C+Shirley+Wu%2C+Linjun+Zhang%2C+James+Zou%2C+and+Huaxiu+Yao%2E+2023%2E+Holistic+analysis+of+hallucination+in+gpt%2D4v+%28ision%29%3A+Bias+and+interference+challenges%2E+arXiv+preprint)
36. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/). arXiv:2305.06500 [cs.CV]
37. Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, et al. 2023. [Language modeling is compression](https://md.research.elamur.ai/free/en/papers/deletang-2023-language-modeling-compression/). arXiv preprint arXiv:2309.10668 (2023).
38. Ailin Deng, Zhirui Chen, and Bryan Hooi. 2024. Seeing is Believing: Mitigating Hallucination in Large Vision-Language Models via CLIP-Guided Decoding. arXiv preprint arXiv:2402.15300 (2024). — [arXiv:2402.15300](https://arxiv.org/abs/2402.15300) [find](https://scholar.google.com/scholar?q=Ailin+Deng%2C+Zhirui+Chen%2C+and+Bryan+Hooi%2E+2024%2E+Seeing+is+Believing%3A+Mitigating+Hallucination+in+Large+Vision%2DLanguage+Models+via+CLIP%2DGuided+Decoding%2E+arXiv+preprint+arXiv%3A2402%2E15300+%282024%29%2E+%E2%80%94+%5BarXiv%3A)
39. Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://md.research.elamur.ai/free/en/papers/du-2022-glm-general-language-model-pretraining/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 320–335.
40. Ke Fan, Zechen Bai, Tianjun Xiao, Dominik Zietlow, Max Horn, Zixu Zhao, Carl-Johann Simon-Gabriel, Mike Zheng Shou, Francesco Locatello, Bernt Schiele, et al. 2023. Unsupervised open-vocabulary object localization in videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision.… [find](https://scholar.google.com/scholar?q=Ke+Fan%2C+Zechen+Bai%2C+Tianjun+Xiao%2C+Dominik+Zietlow%2C+Max+Horn%2C+Zixu+Zhao%2C+Carl%2DJohann+Simon%2DGabriel%2C+Mike+Zheng+Shou%2C+Francesco+Locatello%2C+Bernt+Schiele%2C+et+al%2E+2023%2E+Unsupervised+open%2Dvocabulary+object)
41. Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. 2024. Multi-modal hallucination control by visual information grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog… [find](https://scholar.google.com/scholar?q=Alessandro+Favero%2C+Luca+Zancato%2C+Matthew+Trager%2C+Siddharth+Choudhary%2C+Pramuditha+Perera%2C+Alessandro+Achille%2C+Ashwin+Swaminathan%2C+and+Stefano+Soatto%2E+2024%2E+Multi%2Dmodal+hallucination+control+by+visual+i)
42. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. 2023. [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/). arXiv preprint arXiv:2306.13394 (2023).
43. Yuhan Fu, Ruobing Xie, Xingwu Sun, Zhanhui Kang, and Xirong Li. 2024. Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization. arXiv preprint arXiv:2411.10436 (2024). [find](https://scholar.google.com/scholar?q=Yuhan+Fu%2C+Ruobing+Xie%2C+Xingwu+Sun%2C+Zhanhui+Kang%2C+and+Xirong+Li%2E+2024%2E+Mitigating+Hallucination+in+Multimodal+Large+Language+Model+via+Hallucination%2Dtargeted+Direct+Preference+Optimization%2E+arXiv+prepr)
44. Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, et al. 2023. [Llama-adapter v2: Parameter-efficient visual instruction model](https://md.research.elamur.ai/free/en/papers/gao-2023-llama-adapter-v2-parameter-efficient/). arXiv preprint arXiv:2304.15010 (2023).
45. Yunhao Ge, Xiaohui Zeng, Jacob Samuel Huffman, Tsung-Yi Lin, Ming-Yu Liu, and Yin Cui. 2024. Visual fact checker: enabling high-fidelity detailed caption generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14033–14042. [find](https://scholar.google.com/scholar?q=Yunhao+Ge%2C+Xiaohui+Zeng%2C+Jacob+Samuel+Huffman%2C+Tsung%2DYi+Lin%2C+Ming%2DYu+Liu%2C+and+Yin+Cui%2E+2024%2E+Visual+fact+checker%3A+enabling+high%2Dfidelity+detailed+caption+generation%2E+In+Proceedings+of+the+IEEE%2FCVF+Con)
46. Ross Girshick. 2015. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision. 1440–1448. [find](https://scholar.google.com/scholar?q=Ross+Girshick%2E+2015%2E+Fast+r%2Dcnn%2E+In+Proceedings+of+the+IEEE+international+conference+on+computer+vision%2E+1440%E2%80%931448%2E)
47. Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. 2023. MultiModal-GPT: A Vision and Language Model for Dialogue with Humans. arXiv:2305.04790 [cs.CV] — [arXiv:2305.04790](https://arxiv.org/abs/2305.04790) [find](https://scholar.google.com/scholar?q=Tao+Gong%2C+Chengqi+Lyu%2C+Shilong+Zhang%2C+Yudong+Wang%2C+Miao+Zheng%2C+Qian+Zhao%2C+Kuikun+Liu%2C+Wenwei+Zhang%2C+Ping+Luo%2C+and+Kai+Chen%2E+2023%2E+MultiModal%2DGPT%3A+A+Vision+and+Language+Model+for+Dialogue+with+Humans%2E+)
48. Xuan Gong, Tianshi Ming, Xinpeng Wang, and Zhihua Wei. 2024. Damro: Dive into the attention mechanism of lvlm to reduce object hallucination. arXiv preprint arXiv:2410.04514 (2024). [find](https://scholar.google.com/scholar?q=Xuan+Gong%2C+Tianshi+Ming%2C+Xinpeng+Wang%2C+and+Zhihua+Wei%2E+2024%2E+Damro%3A+Dive+into+the+attention+mechanism+of+lvlm+to+reduce+object+hallucination%2E+arXiv+preprint+arXiv%3A2410%2E04514+%282024%29%2E)
49. Google. 2023. Bard. [https://bard.google.com/](https://bard.google.com/) [↗](https://bard.google.com/)
50. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. [Making the v in vqa matter: Elevating the role of image understanding in visual question answering](https://md.research.elamur.ai/free/en/papers/goyal-2016-making-v-vqa-matter-elevating/). In Proceedings of the IEEE conference on computer vision and pattern recognition. 6904–6913.
51. Tianrui Guan, Fuxiao Liu, Xiyang Wu Ruiqi Xian Zongxia Li, Xiaoyu Liu Xijun Wang, Lichang Chen Furong Huang Yaser Yacoob, and Dinesh Manocha Tianyi Zhou. 2023. HALLUSIONBENCH: An Advanced Diagnostic Suite for Entangled Language Hallucination & Visual Illusion in Large Vision-Language Models. arXiv e… [find](https://scholar.google.com/scholar?q=Tianrui+Guan%2C+Fuxiao+Liu%2C+Xiyang+Wu+Ruiqi+Xian+Zongxia+Li%2C+Xiaoyu+Liu+Xijun+Wang%2C+Lichang+Chen+Furong+Huang+Yaser+Yacoob%2C+and+Dinesh+Manocha+Tianyi+Zhou%2E+2023%2E+HALLUSIONBENCH%3A+An+Advanced+Diagnostic+S)
52. Anisha Gunjal, Jihan Yin, and Erhan Bas. 2024. [Detecting and preventing hallucinations in large vision language models](https://md.research.elamur.ai/free/en/papers/gunjal-2023-detecting-preventing-hallucinations-large-vision/). In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 18135–18143.
53. Chi Han, Jialiang Xu, Manling Li, Yi Fung, Chenkai Sun, Nan Jiang, Tarek Abdelzaher, and Heng Ji. 2023. Lm-switch: Lightweight language model conditioning in word embedding space. arXiv preprint arXiv:2305.12798 (2023). — [arXiv:2305.12798](https://arxiv.org/abs/2305.12798) [find](https://scholar.google.com/scholar?q=Chi+Han%2C+Jialiang+Xu%2C+Manling+Li%2C+Yi+Fung%2C+Chenkai+Sun%2C+Nan+Jiang%2C+Tarek+Abdelzaher%2C+and+Heng+Ji%2E+2023%2E+Lm%2Dswitch%3A+Lightweight+language+model+conditioning+in+word+embedding+space%2E+arXiv+preprint+arXiv)
54. Jiaming Han, Renrui Zhang, Wenqi Shao, Peng Gao, Peng Xu, Han Xiao, Kaipeng Zhang, Chris Liu, Song Wen, Ziyu Guo, et al. 2023. Imagebind-llm: Multi-modality instruction tuning. arXiv preprint arXiv:2309.03905 (2023). — [arXiv:2309.03905](https://arxiv.org/abs/2309.03905) [find](https://scholar.google.com/scholar?q=Jiaming+Han%2C+Renrui+Zhang%2C+Wenqi+Shao%2C+Peng+Gao%2C+Peng+Xu%2C+Han+Xiao%2C+Kaipeng+Zhang%2C+Chris+Liu%2C+Song+Wen%2C+Ziyu+Guo%2C+et+al%2E+2023%2E+Imagebind%2Dllm%3A+Multi%2Dmodality+instruction+tuning%2E+arXiv+preprint+arXiv%3A23)
55. Tianyang Han, Qing Lian, Rui Pan, Renjie Pi, Jipeng Zhang, Shizhe Diao, Yong Lin, and Tong Zhang. 2024. The Instinctive Bias: Spurious Images lead to Hallucination in MLLMs. arXiv preprint arXiv:2402.03757 (2024). — [arXiv:2402.03757](https://arxiv.org/abs/2402.03757) [find](https://scholar.google.com/scholar?q=Tianyang+Han%2C+Qing+Lian%2C+Rui+Pan%2C+Renjie+Pi%2C+Jipeng+Zhang%2C+Shizhe+Diao%2C+Yong+Lin%2C+and+Tong+Zhang%2E+2024%2E+The+Instinctive+Bias%3A+Spurious+Images+lead+to+Hallucination+in+MLLMs%2E+arXiv+preprint+arXiv%3A2402%2E)
56. Zongbo Han, Zechen Bai, Haiyang Mei, Qianli Xu, Changqing Zhang, and Mike Zheng Shou. 2024. Skip \\𝑛: A simple method to reduce hallucination in Large Vision-Language Models. arXiv preprint arXiv:2402.01345 (2024). — [arXiv:2402.01345](https://arxiv.org/abs/2402.01345) [find](https://scholar.google.com/scholar?q=Zongbo+Han%2C+Zechen+Bai%2C+Haiyang+Mei%2C+Qianli+Xu%2C+Changqing+Zhang%2C+and+Mike+Zheng+Shou%2E+2024%2E+Skip+%5C%5C%F0%9D%91%9B%3A+A+simple+method+to+reduce+hallucination+in+Large+Vision%2DLanguage+Models%2E+arXiv+preprint+arXiv%3A2402)
57. Jinghan He, Kuan Zhu, Haiyun Guo, Junfeng Fang, Zhenglin Hua, Yuheng Jia, Ming Tang, Tat-Seng Chua, and Jinqiao Wang. 2024. [Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence](https://md.research.elamur.ai/free/en/papers/he-2025-cracking-code-hallucination-lvlms-vision/). arXiv preprint arXiv:2412.13949 (2024).
58. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision. 2961–2969. [find](https://scholar.google.com/scholar?q=Kaiming+He%2C+Georgia+Gkioxari%2C+Piotr+Doll%C3%A1r%2C+and+Ross+Girshick%2E+2017%2E+Mask+r%2Dcnn%2E+In+Proceedings+of+the+IEEE+international+conference+on+computer+vision%2E+2961%E2%80%932969%2E)
59. Xin He, Longhui Wei, Lingxi Xie, and Qi Tian. 2024. Incorporating Visual Experts to Resolve the Information Loss in Multimodal Large Language Models. arXiv preprint arXiv:2401.03105 (2024). — [arXiv:2401.03105](https://arxiv.org/abs/2401.03105) [find](https://scholar.google.com/scholar?q=Xin+He%2C+Longhui+Wei%2C+Lingxi+Xie%2C+and+Qi+Tian%2E+2024%2E+Incorporating+Visual+Experts+to+Resolve+the+Information+Loss+in+Multimodal+Large+Language+Models%2E+arXiv+preprint+arXiv%3A2401%2E03105+%282024%29%2E+%E2%80%94+%5BarXiv%3A2)
60. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring Massive Multitask Language Understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. https://open…
61. Jonathan Ho and Tim Salimans. 2022. [Classifier-free diffusion guidance](https://md.research.elamur.ai/free/en/papers/ho-2022-classifier-free-diffusion-guidance/). arXiv preprint arXiv:2207.12598 (2022).
62. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). arXiv preprint arXiv:2106.09685 (2021).
63. Hongyu Hu, Jiyuan Zhang, Minyi Zhao, and Zhenbang Sun. 2023. Ciem: Contrastive instruction evaluation method for better instruction tuning. arXiv preprint arXiv:2309.02301 (2023). — [arXiv:2309.02301](https://arxiv.org/abs/2309.02301) [find](https://scholar.google.com/scholar?q=Hongyu+Hu%2C+Jiyuan+Zhang%2C+Minyi+Zhao%2C+and+Zhenbang+Sun%2E+2023%2E+Ciem%3A+Contrastive+instruction+evaluation+method+for+better+instruction+tuning%2E+arXiv+preprint+arXiv%3A2309%2E02301+%282023%29%2E+%E2%80%94+%5BarXiv%3A2309%2E02301%5D)
64. Wenbo Hu, Yifan Xu, Yi Li, Weiyue Li, Zeyuan Chen, and Zhuowen Tu. 2023. Bliva: A simple multimodal llm for better handling of text-rich visual questions. arXiv preprint arXiv:2308.09936 (2023). — [arXiv:2308.09936](https://arxiv.org/abs/2308.09936) [find](https://scholar.google.com/scholar?q=Wenbo+Hu%2C+Yifan+Xu%2C+Yi+Li%2C+Weiyue+Li%2C+Zeyuan+Chen%2C+and+Zhuowen+Tu%2E+2023%2E+Bliva%3A+A+simple+multimodal+llm+for+better+handling+of+text%2Drich+visual+questions%2E+arXiv+preprint+arXiv%3A2308%2E09936+%282023%29%2E+%E2%80%94+%5Bar)
65. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2023. [A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). arXiv preprint arXiv:2311.05232 (2023).
66. Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. 2023. [OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation](https://md.research.elamur.ai/free/en/papers/huang-2023-opera-alleviating-hallucination-multi-modal/). arXiv preprint arXiv:2311.17911 (2023).
67. Wen Huang, Hongbin Liu, Minxin Guo, and Neil Zhenqiang Gong. 2024. Visual Hallucinations of Multi-modal Large Language Models. arXiv preprint arXiv:2402.14683 (2024). [find](https://scholar.google.com/scholar?q=Wen+Huang%2C+Hongbin+Liu%2C+Minxin+Guo%2C+and+Neil+Zhenqiang+Gong%2E+2024%2E+Visual+Hallucinations+of+Multi%2Dmodal+Large+Language+Models%2E+arXiv+preprint+arXiv%3A2402%2E14683+%282024%29%2E)
68. Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, et al. 2023. [C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models](https://md.research.elamur.ai/free/en/papers/huang-2023-c-eval-multi-level-multi/). ArXiv preprint abs/2305.08322 (2023). https://arxiv.org/abs/2305.0832… [↗](https://arxiv.org/abs/2305.0832…)
69. Drew A Hudson and Christopher D Manning. 2019. [Gqa: A new dataset for real-world visual reasoning and compositional question answering](https://md.research.elamur.ai/free/en/papers/hudson-2019-gqa-new-dataset-real-world/). In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6700–6709.
70. Fushuo Huo, Wenchao Xu, Zhong Zhang, Haozhao Wang, Zhicheng Chen, and Peilin Zhao. 2024. Self-introspective decoding: Alleviating hallucinations for large vision-language models. arXiv preprint arXiv:2408.02032 (2024). — [arXiv:2408.02032](https://arxiv.org/abs/2408.02032) [find](https://scholar.google.com/scholar?q=Fushuo+Huo%2C+Wenchao+Xu%2C+Zhong+Zhang%2C+Haozhao+Wang%2C+Zhicheng+Chen%2C+and+Peilin+Zhao%2E+2024%2E+Self%2Dintrospective+decoding%3A+Alleviating+hallucinations+for+large+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A)
71. Jitesh Jain, Jianwei Yang, and Humphrey Shi. 2023. Vcoder: Versatile vision encoders for multimodal large language models. arXiv preprint arXiv:2312.14233 (2023). — [arXiv:2312.14233](https://arxiv.org/abs/2312.14233) [find](https://scholar.google.com/scholar?q=Jitesh+Jain%2C+Jianwei+Yang%2C+and+Humphrey+Shi%2E+2023%2E+Vcoder%3A+Versatile+vision+encoders+for+multimodal+large+language+models%2E+arXiv+preprint+arXiv%3A2312%2E14233+%282023%29%2E+%E2%80%94+%5BarXiv%3A2312%2E14233%5D%28https%3A%2F%2Farxiv%2Eor)
72. Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2022. [Knowledge unlearning for mitigating privacy risks in language models](https://md.research.elamur.ai/free/en/papers/jang-2022-knowledge-unlearning-mitigating-privacy-risks/). arXiv preprint arXiv:2210.01504 (2022).
73. Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. [Scaling up visual and vision-language representation learning with noisy text supervision](https://md.research.elamur.ai/free/en/papers/jia-2021-scaling-up-visual-vision-language/). In International conference on machine learning. PMLR, 4904–4916.
74. Chaoya Jiang, Hongrui Jia, Mengfan Dong, Wei Ye, Haiyang Xu, Ming Yan, Ji Zhang, and Shikun Zhang. 2024. Hal-eval: A universal and fine-grained hallucination evaluation framework for large vision language models. In Proceedings of the 32nd ACM International Conference on Multimedia. 525–534. [find](https://scholar.google.com/scholar?q=Chaoya+Jiang%2C+Hongrui+Jia%2C+Mengfan+Dong%2C+Wei+Ye%2C+Haiyang+Xu%2C+Ming+Yan%2C+Ji+Zhang%2C+and+Shikun+Zhang%2E+2024%2E+Hal%2Deval%3A+A+universal+and+fine%2Dgrained+hallucination+evaluation+framework+for+large+vision+lang)
75. Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, and Shikun Zhang. 2023. Hallucination Augmented Contrastive Learning for Multimodal Large Language Model. arXiv preprint arXiv:2312.06968 (2023). — [arXiv:2312.06968](https://arxiv.org/abs/2312.06968) [find](https://scholar.google.com/scholar?q=Chaoya+Jiang%2C+Haiyang+Xu%2C+Mengfan+Dong%2C+Jiaxing+Chen%2C+Wei+Ye%2C+Ming+Yan%2C+Qinghao+Ye%2C+Ji+Zhang%2C+Fei+Huang%2C+and+Shikun+Zhang%2E+2023%2E+Hallucination+Augmented+Contrastive+Learning+for+Multimodal+Large+Langu)
76. Nick Jiang, Anish Kachinthaya, Suzie Petryk, and Yossi Gandelsman. 2024. Interpreting and editing vision-language representations to mitigate hallucinations. arXiv preprint arXiv:2410.02762 (2024). — [arXiv:2410.02762](https://arxiv.org/abs/2410.02762) [find](https://scholar.google.com/scholar?q=Nick+Jiang%2C+Anish+Kachinthaya%2C+Suzie+Petryk%2C+and+Yossi+Gandelsman%2E+2024%2E+Interpreting+and+editing+vision%2Dlanguage+representations+to+mitigate+hallucinations%2E+arXiv+preprint+arXiv%3A2410%2E02762+%282024%29%2E+%E2%80%94+)
77. Qirui Jiao, Daoyuan Chen, Yilun Huang, Yaliang Li, and Ying Shen. 2024. Enhancing Multimodal Large Language Models with Vision Detection Models: An Empirical Study. arXiv preprint arXiv:2401.17981 (2024). — [arXiv:2401.17981](https://arxiv.org/abs/2401.17981) [find](https://scholar.google.com/scholar?q=Qirui+Jiao%2C+Daoyuan+Chen%2C+Yilun+Huang%2C+Yaliang+Li%2C+and+Ying+Shen%2E+2024%2E+Enhancing+Multimodal+Large+Language+Models+with+Vision+Detection+Models%3A+An+Empirical+Study%2E+arXiv+preprint+arXiv%3A2401%2E17981+%2820)
78. Liqiang Jing and Xinya Du. 2024. Fgaif: Aligning large vision-language models with fine-grained ai feedback. arXiv preprint arXiv:2404.05046 (2024). — [arXiv:2404.05046](https://arxiv.org/abs/2404.05046) [find](https://scholar.google.com/scholar?q=Liqiang+Jing+and+Xinya+Du%2E+2024%2E+Fgaif%3A+Aligning+large+vision%2Dlanguage+models+with+fine%2Dgrained+ai+feedback%2E+arXiv+preprint+arXiv%3A2404%2E05046+%282024%29%2E+%E2%80%94+%5BarXiv%3A2404%2E05046%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2404%2E050)
79. Liqiang Jing, Ruosen Li, Yunmo Chen, Mengzhao Jia, and Xinya Du. 2023. FAITHSCORE: Evaluating Hallucinations in Large Vision-Language Models. arXiv preprint arXiv:2311.01477 (2023). — [arXiv:2311.01477](https://arxiv.org/abs/2311.01477) [find](https://scholar.google.com/scholar?q=Liqiang+Jing%2C+Ruosen+Li%2C+Yunmo+Chen%2C+Mengzhao+Jia%2C+and+Xinya+Du%2E+2023%2E+FAITHSCORE%3A+Evaluating+Hallucinations+in+Large+Vision%2DLanguage+Models%2E+arXiv+preprint+arXiv%3A2311%2E01477+%282023%29%2E+%E2%80%94+%5BarXiv%3A2311%2E0147)
80. Kushal Kafle and Christopher Kanan. 2017. An analysis of visual question answering algorithms. In Proceedings of the IEEE international conference on computer vision. 1965–1973. [find](https://scholar.google.com/scholar?q=Kushal+Kafle+and+Christopher+Kanan%2E+2017%2E+An+analysis+of+visual+question+answering+algorithms%2E+In+Proceedings+of+the+IEEE+international+conference+on+computer+vision%2E+1965%E2%80%931973%2E)
81. Andrej Karpathy. 2023. On the "hallucination problem". [https://twitter.com/karpathy/status/1733299213503787018](https://twitter.com/karpathy/status/1733299213503787018) Access Date: 10 Mar. 2024. [↗](https://twitter.com/karpathy/status/1733299213503787018)
82. Prannay Kaul, Zhizhong Li, Hao Yang, Yonatan Dukler, Ashwin Swaminathan, CJ Taylor, and Stefano Soatto. 2024. Throne: An object-based hallucination benchmark for the free-form generations of large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn… [find](https://scholar.google.com/scholar?q=Prannay+Kaul%2C+Zhizhong+Li%2C+Hao+Yang%2C+Yonatan+Dukler%2C+Ashwin+Swaminathan%2C+CJ+Taylor%2C+and+Stefano+Soatto%2E+2024%2E+Throne%3A+An+object%2Dbased+hallucination+benchmark+for+the+free%2Dform+generations+of+large+vis)
83. Junho Kim, Hyunjun Kim, Kim Yeonju, and Yong Man Ro. 2024. Code: Contrasting self-generated description to combat hallucination in large multi-modal models. Advances in Neural Information Processing Systems 37 (2024), 133571–133599. [find](https://scholar.google.com/scholar?q=Junho+Kim%2C+Hyunjun+Kim%2C+Kim+Yeonju%2C+and+Yong+Man+Ro%2E+2024%2E+Code%3A+Contrasting+self%2Dgenerated+description+to+combat+hallucination+in+large+multi%2Dmodal+models%2E+Advances+in+Neural+Information+Processing+S)
84. Junho Kim, Yeon Ju Kim, and Yong Man Ro. 2024. What if...?: Thinking counterfactual keywords helps to mitigate hallucination in large multi-modal models. arXiv preprint arXiv:2403.13513 (2024). [find](https://scholar.google.com/scholar?q=Junho+Kim%2C+Yeon+Ju+Kim%2C+and+Yong+Man+Ro%2E+2024%2E+What+if%2E%2E%2E%3F%3A+Thinking+counterfactual+keywords+helps+to+mitigate+hallucination+in+large+multi%2Dmodal+models%2E+arXiv+preprint+arXiv%3A2403%2E13513+%282024%29%2E)
85. Minchan Kim, Minyeong Kim, Junik Bae, Suhwan Choi, Sungkyung Kim, and Buru Chang. 2024. Exploiting semantic reconstruction to mitigate hallucinations in vision-language models. In European Conference on Computer Vision. Springer, 236–252. [find](https://scholar.google.com/scholar?q=Minchan+Kim%2C+Minyeong+Kim%2C+Junik+Bae%2C+Suhwan+Choi%2C+Sungkyung+Kim%2C+and+Buru+Chang%2E+2024%2E+Exploiting+semantic+reconstruction+to+mitigate+hallucinations+in+vision%2Dlanguage+models%2E+In+European+Conference+)
86. Sihyeon Kim, Boryeong Cho, Sangmin Bae, Sumyeong Ahn, and Se-Young Yun. 2024. Vacode: Visual augmented contrastive decoding. arXiv preprint arXiv:2408.05337 (2024). — [arXiv:2408.05337](https://arxiv.org/abs/2408.05337) [find](https://scholar.google.com/scholar?q=Sihyeon+Kim%2C+Boryeong+Cho%2C+Sangmin+Bae%2C+Sumyeong+Ahn%2C+and+Se%2DYoung+Yun%2E+2024%2E+Vacode%3A+Visual+augmented+contrastive+decoding%2E+arXiv+preprint+arXiv%3A2408%2E05337+%282024%29%2E+%E2%80%94+%5BarXiv%3A2408%2E05337%5D%28https%3A%2F%2Farxiv%2E)
87. Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023. [Segment anything](https://md.research.elamur.ai/free/en/papers/kirillov-2023-segment-anything/). In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4015–4026.
88. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). Advances in neural information processing systems 35 (2022), 22199–22213.
89. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. 2017. [Visual genome: Connecting language and vision using crowdsourced dense image annotations](https://md.research.elamur.ai/free/en/papers/krishna-2016-visual-genome-connecting-language-vision/). International journal of computer vision 123 (2017)…
90. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. [Imagenet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/). Advances in neural information processing systems 25 (2012).
91. Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. 2020. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. Internat… [find](https://scholar.google.com/scholar?q=Alina+Kuznetsova%2C+Hassan+Rom%2C+Neil+Alldrin%2C+Jasper+Uijlings%2C+Ivan+Krasin%2C+Jordi+Pont%2DTuset%2C+Shahab+Kamali%2C+Stefan+Popov%2C+Matteo+Malloci%2C+Alexander+Kolesnikov%2C+et+al%2E+2020%2E+The+open+images+dataset+v4%3A+)
92. Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. 2023. Lisa: Reasoning segmentation via large language model. arXiv preprint arXiv:2308.00692 (2023). [find](https://scholar.google.com/scholar?q=Xin+Lai%2C+Zhuotao+Tian%2C+Yukang+Chen%2C+Yanwei+Li%2C+Yuhui+Yuan%2C+Shu+Liu%2C+and+Jiaya+Jia%2E+2023%2E+Lisa%3A+Reasoning+segmentation+via+large+language+model%2E+arXiv+preprint+arXiv%3A2308%2E00692+%282023%29%2E)
93. Seongyun Lee, Sue Hyun Park, Yongrae Jo, and Minjoon Seo. 2023. Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision. arXiv preprint arXiv:2311.07362 (2023). [find](https://scholar.google.com/scholar?q=Seongyun+Lee%2C+Sue+Hyun+Park%2C+Yongrae+Jo%2C+and+Minjoon+Seo%2E+2023%2E+Volcano%3A+Mitigating+Multimodal+Hallucination+through+Self%2DFeedback+Guided+Revision%2E+arXiv+preprint+arXiv%3A2311%2E07362+%282023%29%2E)
94. Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2023. [Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding](https://md.research.elamur.ai/free/en/papers/leng-2023-mitigating-object-hallucinations-large-vision/). arXiv preprint arXiv:2311.16922 (2023).
95. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in neural information processing systems 33 (2020), 9459–…
96. Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Jingkang Yang, and Ziwei Liu. 2023. [Otter: A Multi-Modal Model with In-Context Instruction Tuning](https://md.research.elamur.ai/free/en/papers/li-2023-otter-multi-modal-model-context/). arXiv preprint arXiv:2305.03726 (2023).
97. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. 2024. [Llava-onevision: Easy visual task transfer](https://md.research.elamur.ai/free/en/papers/li-2024-llava-onevision-easy-visual-task/). arXiv preprint arXiv:2408.03326 (2024).
98. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. [Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). arXiv preprint arXiv:2301.12597 (2023).
99. Juncheng Li, Kaihang Pan, Zhiqi Ge, Minghe Gao, Hanwang Zhang, Wei Ji, Wenqiao Zhang, Tat-Seng Chua, Siliang Tang, and Yueting Zhuang. 2023. Fine-tuning multimodal llms to follow zero-shot demonstrative instructions. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Juncheng+Li%2C+Kaihang+Pan%2C+Zhiqi+Ge%2C+Minghe+Gao%2C+Hanwang+Zhang%2C+Wei+Ji%2C+Wenqiao+Zhang%2C+Tat%2DSeng+Chua%2C+Siliang+Tang%2C+and+Yueting+Zhuang%2E+2023%2E+Fine%2Dtuning+multimodal+llms+to+follow+zero%2Dshot+demonstrati)
100. Jiaming Li, Jiacheng Zhang, Zequn Jie, Lin Ma, and Guanbin Li. 2025. Mitigating Hallucination for Large Vision Language Model by Inter-Modality Correlation Calibration Decoding. arXiv preprint arXiv:2501.01926 (2025). — [arXiv:2501.01926](https://arxiv.org/abs/2501.01926) [find](https://scholar.google.com/scholar?q=Jiaming+Li%2C+Jiacheng+Zhang%2C+Zequn+Jie%2C+Lin+Ma%2C+and+Guanbin+Li%2E+2025%2E+Mitigating+Hallucination+for+Large+Vision+Language+Model+by+Inter%2DModality+Correlation+Calibration+Decoding%2E+arXiv+preprint+arXiv%3A2)
101. Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, and Lingpeng Kong. 2023. Silkie: Preference distillation for large visual language models. arXiv preprint arXiv:2312.10665 (2023). — [arXiv:2312.10665](https://arxiv.org/abs/2312.10665) [find](https://scholar.google.com/scholar?q=Lei+Li%2C+Zhihui+Xie%2C+Mukai+Li%2C+Shunian+Chen%2C+Peiyi+Wang%2C+Liang+Chen%2C+Yazheng+Yang%2C+Benyou+Wang%2C+and+Lingpeng+Kong%2E+2023%2E+Silkie%3A+Preference+distillation+for+large+visual+language+models%2E+arXiv+preprint)
102. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2022. [Contrastive decoding: Open-ended text generation as optimization](https://md.research.elamur.ai/free/en/papers/li-2022-contrastive-decoding-open-ended-text/). ArXiv preprint abs/2210.15097 (2022). https://arxiv.org/abs/2210.15097 [↗](https://arxiv.org/abs/2210.15097)
103. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. [Evaluating object hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-object-hallucination-large-vision/). arXiv preprint arXiv:2305.10355 (2023).
104. Zhuowei Li, Haizhou Shi, Yunhe Gao, Di Liu, Zhenting Wang, Yuxiao Chen, Ting Liu, Long Zhao, Hao Wang, and Dimitris N Metaxas. 2025. The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering. arXiv preprint arXiv:2502.03628 (2025). — [arXiv:2502.03628](https://arxiv.org/abs/2502.03628) [find](https://scholar.google.com/scholar?q=Zhuowei+Li%2C+Haizhou+Shi%2C+Yunhe+Gao%2C+Di+Liu%2C+Zhenting+Wang%2C+Yuxiao+Chen%2C+Ting+Liu%2C+Long+Zhao%2C+Hao+Wang%2C+and+Dimitris+N+Metaxas%2E+2025%2E+The+Hidden+Life+of+Tokens%3A+Reducing+Hallucination+of+Large+Vision%2DL)
105. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. [Microsoft coco: Common objects in context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/). In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Spri…
106. Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, et al. 2023. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. arXiv preprint arXiv:2311.07575 (2023). — [arXiv:2311.07575](https://arxiv.org/abs/2311.07575) [find](https://scholar.google.com/scholar?q=Ziyi+Lin%2C+Chris+Liu%2C+Renrui+Zhang%2C+Peng+Gao%2C+Longtian+Qiu%2C+Han+Xiao%2C+Han+Qiu%2C+Chen+Lin%2C+Wenqi+Shao%2C+Keqin+Chen%2C+et+al%2E+2023%2E+Sphinx%3A+The+joint+mixing+of+weights%2C+tasks%2C+and+visual+embeddings+for+multi)
107. Zheng Lin, Zhenxing Niu, Zhibin Wang, and Yinghui Xu. 2024. Interpreting and mitigating hallucination in mllms through multi-agent debate. arXiv preprint arXiv:2407.20505 (2024). — [arXiv:2407.20505](https://arxiv.org/abs/2407.20505) [find](https://scholar.google.com/scholar?q=Zheng+Lin%2C+Zhenxing+Niu%2C+Zhibin+Wang%2C+and+Yinghui+Xu%2E+2024%2E+Interpreting+and+mitigating+hallucination+in+mllms+through+multi%2Dagent+debate%2E+arXiv+preprint+arXiv%3A2407%2E20505+%282024%29%2E+%E2%80%94+%5BarXiv%3A2407%2E20505%5D%28)
108. Fuxiao Liu, Tianrui Guan, Zongxia Li, Lichang Chen, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2023. [HallusionBench: You See What You Think?](https://md.research.elamur.ai/free/en/papers/guan-2023-hallusionbench-advanced-diagnostic-suite-entangled/) Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(Ision), LLaVA-1.5, and Other Multi-Modality Models. [https://arxiv.or…](https://arxiv.or…) [↗](https://arxiv.or…)
109. Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. 2023. [Mitigating hallucination in large multi-modal models via robust instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-mitigating-hallucination-large-multi-modal/). arXiv preprint arXiv:2306.14565 1 (2023).
110. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/). arXiv preprint arXiv:2310.03744 (2023).
111. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). arXiv preprint arXiv:2304.08485 (2023).
112. Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. 2024. [A survey on hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/liu-2024-survey-hallucination-large-vision-language/). arXiv preprint arXiv:2402.00253 (2024).
113. Jiazhen Liu, Yuhan Fu, Ruobing Xie, Runquan Xie, Xingwu Sun, Fengzong Lian, Zhanhui Kang, and Xirong Li. 2024. Phd: A prompted visual hallucination evaluation dataset. arXiv e-prints (2024), arXiv–2403. [find](https://scholar.google.com/scholar?q=Jiazhen+Liu%2C+Yuhan+Fu%2C+Ruobing+Xie%2C+Runquan+Xie%2C+Xingwu+Sun%2C+Fengzong+Lian%2C+Zhanhui+Kang%2C+and+Xirong+Li%2E+2024%2E+Phd%3A+A+prompted+visual+hallucination+evaluation+dataset%2E+arXiv+e%2Dprints+%282024%29%2C+arXiv%E2%80%93240)
114. Sheng Liu, Haotian Ye, and James Zou. 2025. Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Sheng+Liu%2C+Haotian+Ye%2C+and+James+Zou%2E+2025%2E+Reducing+Hallucinations+in+Large+Vision%2DLanguage+Models+via+Latent+Space+Steering%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations%2E)
115. Shi Liu, Kecheng Zheng, and Wei Chen. 2024. Paying more attention to image: A training-free method for alleviating hallucination in lvlms. In European Conference on Computer Vision. Springer, 125–140. [find](https://scholar.google.com/scholar?q=Shi+Liu%2C+Kecheng+Zheng%2C+and+Wei+Chen%2E+2024%2E+Paying+more+attention+to+image%3A+A+training%2Dfree+method+for+alleviating+hallucination+in+lvlms%2E+In+European+Conference+on+Computer+Vision%2E+Springer%2C+125%E2%80%93140%2E)
116. Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. 2024. [Mmbench: Is your multi-modal model an all-around player?](https://md.research.elamur.ai/free/en/papers/liu-2023-mmbench-multi-modal-model-all/). In European conference on computer vision. Springer, 216–233.
117. Holy Lovenia, Wenliang Dai, Samuel Cahyawijaya, Ziwei Ji, and Pascale Fung. 2023. Negative object presence evaluation (nope) to measure object hallucination in vision-language models. arXiv preprint arXiv:2310.05338 (2023). [find](https://scholar.google.com/scholar?q=Holy+Lovenia%2C+Wenliang+Dai%2C+Samuel+Cahyawijaya%2C+Ziwei+Ji%2C+and+Pascale+Fung%2E+2023%2E+Negative+object+presence+evaluation+%28nope%29+to+measure+object+hallucination+in+vision%2Dlanguage+models%2E+arXiv+preprint+a)
118. Gen Luo, Yiyi Zhou, Tianhe Ren, Shengxin Chen, Xiaoshuai Sun, and Rongrong Ji. 2024. Cheap and quick: Efficient vision-language instruction tuning for large language models. Advances in Neural Information Processing Systems 36 (2024). [find](https://scholar.google.com/scholar?q=Gen+Luo%2C+Yiyi+Zhou%2C+Tianhe+Ren%2C+Shengxin+Chen%2C+Xiaoshuai+Sun%2C+and+Rongrong+Ji%2E+2024%2E+Cheap+and+quick%3A+Efficient+vision%2Dlanguage+instruction+tuning+for+large+language+models%2E+Advances+in+Neural+Informa)
119. Zixian Ma, Jerry Hong, Mustafa Omer Gul, Mona Gandhi, Irena Gao, and Ranjay Krishna. 2023. CREPE: Can Vision-Language Foundation Models Reason Compositionally?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10910–10921. [find](https://scholar.google.com/scholar?q=Zixian+Ma%2C+Jerry+Hong%2C+Mustafa+Omer+Gul%2C+Mona+Gandhi%2C+Irena+Gao%2C+and+Ranjay+Krishna%2E+2023%2E+CREPE%3A+Can+Vision%2DLanguage+Foundation+Models+Reason+Compositionally%3F%2E+In+Proceedings+of+the+IEEE%2FCVF+Conferen)
120. Shunqi Mao, Chaoyi Zhang, and Weidong Cai. 2025. [Through the Magnifying Glass: Adaptive Perception Magnification for Hallucination-Free VLM Decoding](https://md.research.elamur.ai/free/en/papers/mao-2026-through-magnifying-glass-adaptive-perception/). arXiv preprint arXiv:2503.10183 (2025).
121. Kyungmin Min, Minbeom Kim, Kang-il Lee, Dongryeol Lee, and Kyomin Jung. 2024. Mitigating hallucinations in large vision-language models via summary-guided decoding. arXiv preprint arXiv:2410.13321 (2024). [find](https://scholar.google.com/scholar?q=Kyungmin+Min%2C+Minbeom+Kim%2C+Kang%2Dil+Lee%2C+Dongryeol+Lee%2C+and+Kyomin+Jung%2E+2024%2E+Mitigating+hallucinations+in+large+vision%2Dlanguage+models+via+summary%2Dguided+decoding%2E+arXiv+preprint+arXiv%3A2410%2E13321+%2820)
122. Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. 2019. Ocr-vqa: Visual question answering by reading text in images. In 2019 international conference on document analysis and recognition (ICDAR). IEEE, 947–952. [find](https://scholar.google.com/scholar?q=Anand+Mishra%2C+Shashank+Shekhar%2C+Ajeet+Kumar+Singh%2C+and+Anirban+Chakraborty%2E+2019%2E+Ocr%2Dvqa%3A+Visual+question+answering+by+reading+text+in+images%2E+In+2019+international+conference+on+document+analysis+an)
123. OpenAI. 2022. Introducing chatgpt. [https://openai.com/blog/chatgpt](https://openai.com/blog/chatgpt) [↗](https://openai.com/blog/chatgpt)
124. OpenAI. 2023. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). ArXiv preprint abs/2303.08774 (2023). https://arxiv.org/abs/2303.08774 [↗](https://arxiv.org/abs/2303.08774)
125. OpenAI. 2023. GPT-4V(ision) System Card. [https://cdn.openai.com/papers/GPTV_System_Card.pdf](https://cdn.openai.com/papers/GPTV_System_Card.pdf) [↗](https://cdn.openai.com/papers/GPTV_System_Card.pdf)
126. Yassine Ouali, Adrian Bulat, Brais Martinez, and Georgios Tzimiropoulos. 2024. Clip-dpo: Vision-language models as a source of preference for fixing hallucinations in lvlms. In European Conference on Computer Vision. Springer, 395–413. [find](https://scholar.google.com/scholar?q=Yassine+Ouali%2C+Adrian+Bulat%2C+Brais+Martinez%2C+and+Georgios+Tzimiropoulos%2E+2024%2E+Clip%2Ddpo%3A+Vision%2Dlanguage+models+as+a+source+of+preference+for+fixing+hallucinations+in+lvlms%2E+In+European+Conference+on+)
127. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022.… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
128. Yeji Park, Deokyeong Lee, Junsuk Choe, and Buru Chang. 2024. Convis: Contrastive decoding with hallucination visualization for mitigating hallucinations in multimodal large language models. arXiv preprint arXiv:2408.13906 (2024). — [arXiv:2408.13906](https://arxiv.org/abs/2408.13906) [find](https://scholar.google.com/scholar?q=Yeji+Park%2C+Deokyeong+Lee%2C+Junsuk+Choe%2C+and+Buru+Chang%2E+2024%2E+Convis%3A+Contrastive+decoding+with+hallucination+visualization+for+mitigating+hallucinations+in+multimodal+large+language+models%2E+arXiv+prep)
129. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only. ArXiv preprint abs/2306.0… [find](https://scholar.google.com/scholar?q=Guilherme+Penedo%2C+Quentin+Malartic%2C+Daniel+Hesslow%2C+Ruxandra+Cojocaru%2C+Alessandro+Cappelli%2C+Hamza+Alobeidli%2C+Baptiste+Pannier%2C+Ebtesam+Almazrouei%2C+and+Julien+Launay%2E+2023%2E+The+RefinedWeb+Dataset+for+F)
130. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. [Instruction tuning with gpt-4](https://md.research.elamur.ai/free/en/papers/peng-2023-instruction-tuning-gpt-4/). ArXiv preprint abs/2304.03277 (2023). https://arxiv.org/abs/2304.03277 [↗](https://arxiv.org/abs/2304.03277)
131. Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. 2022. [Reasoning with language model prompting: A survey](https://md.research.elamur.ai/free/en/papers/qiao-2022-reasoning-language-model-prompting-survey/). ArXiv preprint abs/2212.09597 (2022). https://arxiv.org/abs/2212.09597 [↗](https://arxiv.org/abs/2212.09597)
132. Han Qiu, Jiaxing Huang, Peng Gao, Qin Qi, Xiaoqin Zhang, Ling Shao, and Shijian Lu. 2024. LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models. arXiv preprint arXiv:2410.09962 (2024). — [arXiv:2410.09962](https://arxiv.org/abs/2410.09962) [find](https://scholar.google.com/scholar?q=Han+Qiu%2C+Jiaxing+Huang%2C+Peng+Gao%2C+Qin+Qi%2C+Xiaoqin+Zhang%2C+Ling+Shao%2C+and+Shijian+Lu%2E+2024%2E+LongHalQA%3A+Long%2DContext+Hallucination+Evaluation+for+MultiModal+Large+Language+Models%2E+arXiv+preprint+arXiv%3A24)
133. Xiaoye Qu, Qiyuan Chen, Wei Wei, Jishuo Sun, and Jianfeng Dong. 2024. Alleviating hallucination in large vision-language models with active retrieval augmentation. arXiv preprint arXiv:2408.00555 (2024). — [arXiv:2408.00555](https://arxiv.org/abs/2408.00555) [find](https://scholar.google.com/scholar?q=Xiaoye+Qu%2C+Qiyuan+Chen%2C+Wei+Wei%2C+Jishuo+Sun%2C+and+Jianfeng+Dong%2E+2024%2E+Alleviating+hallucination+in+large+vision%2Dlanguage+models+with+active+retrieval+augmentation%2E+arXiv+preprint+arXiv%3A2408%2E00555+%28202)
134. Xiaoye Qu, Jiashuo Sun, Wei Wei, and Yu Cheng. 2024. Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning. arXiv preprint arXiv:2408.17150 (2024). [find](https://scholar.google.com/scholar?q=Xiaoye+Qu%2C+Jiashuo+Sun%2C+Wei+Wei%2C+and+Yu+Cheng%2E+2024%2E+Look%2C+Compare%2C+Decide%3A+Alleviating+Hallucination+in+Large+Vision%2DLanguage+Models+via+Multi%2DView+Multi%2DPath+Reasoning%2E+arXiv+preprint+arXiv%3A2408%2E171)
135. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In International conference on machine learning. PMLR, 8748–8763.
136. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. [Hierarchical text-conditional image generation with clip latents](https://md.research.elamur.ai/free/en/papers/ramesh-2022-hierarchical-text-conditional-image-generation/). arXiv preprint arXiv:2204.06125 1, 2 (2022), 3.
137. Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. [Object hallucination in image captioning](https://md.research.elamur.ai/free/en/papers/rohrbach-2018-object-hallucination-image-captioning/). arXiv preprint arXiv:1809.02156 (2018).
138. Anna Rohrbach, Makarand Tapaswi, Atousa Torabi, Tegan Maharaj, Marcus Rohrbach, Sanja Fidler Christopher Pal, and Bernt Schiele. 2017. The Joint Video and Language Understanding Workshop: MovieQA and The Large Scale Movie Description Challenge (LSMDC). [find](https://scholar.google.com/scholar?q=Anna+Rohrbach%2C+Makarand+Tapaswi%2C+Atousa+Torabi%2C+Tegan+Maharaj%2C+Marcus+Rohrbach%2C+Sanja+Fidler+Christopher+Pal%2C+and+Bernt+Schiele%2E+2017%2E+The+Joint+Video+and+Language+Understanding+Workshop%3A+MovieQA+and+)
139. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. [High-resolution image synthesis with latent diffusion models](https://md.research.elamur.ai/free/en/papers/rombach-2021-high-resolution-image-synthesis-latent/). In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684–10695.
140. Pritam Sarkar, Sayna Ebrahimi, Ali Etemad, Ahmad Beirami, Sercan O Arik, and Tomas Pfister. 2025. Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Pritam+Sarkar%2C+Sayna+Ebrahimi%2C+Ali+Etemad%2C+Ahmad+Beirami%2C+Sercan+O+Arik%2C+and+Tomas+Pfister%2E+2025%2E+Mitigating+Object+Hallucination+in+MLLMs+via+Data%2Daugmented+Phrase%2Dlevel+Alignment%2E+In+The+Thirteenth+)
141. SM Sarwar. 2025. FilterRAG: Zero-Shot Informed Retrieval-Augmented Generation to Mitigate Hallucinations in VQA. arXiv preprint arXiv:2502.18536 (2025). — [arXiv:2502.18536](https://arxiv.org/abs/2502.18536) [find](https://scholar.google.com/scholar?q=SM+Sarwar%2E+2025%2E+FilterRAG%3A+Zero%2DShot+Informed+Retrieval%2DAugmented+Generation+to+Mitigate+Hallucinations+in+VQA%2E+arXiv+preprint+arXiv%3A2502%2E18536+%282025%29%2E+%E2%80%94+%5BarXiv%3A2502%2E18536%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2502)
142. Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. 2022. [Laion-5b: An open large-scale dataset for training next generation image-text models](https://md.research.elamur.ai/free/en/papers/schuhmann-2022-laion-5b-open-large-scale/). Advances in Neural Information Processing S…
143. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017). [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347+%282017%29%2E)
144. Yuying Shang, Xinyi Zeng, Yutao Zhu, Xiao Yang, Zhengwei Fang, Jingyuan Zhang, Jiawei Chen, Zinan Liu, and Yu Tian. 2024. From pixels to tokens: Revisiting object hallucinations in large vision-language models. arXiv preprint arXiv:2410.06795 (2024). — [arXiv:2410.06795](https://arxiv.org/abs/2410.06795) [find](https://scholar.google.com/scholar?q=Yuying+Shang%2C+Xinyi+Zeng%2C+Yutao+Zhu%2C+Xiao+Yang%2C+Zhengwei+Fang%2C+Jingyuan+Zhang%2C+Jiawei+Chen%2C+Zinan+Liu%2C+and+Yu+Tian%2E+2024%2E+From+pixels+to+tokens%3A+Revisiting+object+hallucinations+in+large+vision%2Dlangua)
145. Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. 2019. Objects365: A large-scale, high-quality dataset for object detection. In Proceedings of the IEEE/CVF international conference on computer vision. 8430–8439. [find](https://scholar.google.com/scholar?q=Shuai+Shao%2C+Zeming+Li%2C+Tianyuan+Zhang%2C+Chao+Peng%2C+Gang+Yu%2C+Xiangyu+Zhang%2C+Jing+Li%2C+and+Jian+Sun%2E+2019%2E+Objects365%3A+A+large%2Dscale%2C+high%2Dquality+dataset+for+object+detection%2E+In+Proceedings+of+the+IEEE%2F)
146. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F. Christiano. 2020. Learning to summarize with human feedback. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing System… [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeffrey+Wu%2C+Daniel+M%2E+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+F%2E+Christiano%2E+2020%2E+Learning+to+summarize+with+human+feedback%2E+In+Advances+in)
147. Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. 2023. [Pandagpt: One model to instruction-follow them all](https://md.research.elamur.ai/free/en/papers/su-2023-pandagpt-one-model-instruction-follow/). arXiv preprint arXiv:2305.16355 (2023).
148. Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. 2023. [Aligning large multimodal models with factually augmented rlhf](https://md.research.elamur.ai/free/en/papers/sun-2023-aligning-large-multimodal-models-factually/). arXiv preprint arXiv:2309.14525 (2023).
149. Wei Suo, Lijun Zhang, Mengyang Sun, Lin Yuanbo Wu, Peng Wang, and Yanning Zhang. 2025. Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding. arXiv preprint arXiv:2503.00361 (2025). — [arXiv:2503.00361](https://arxiv.org/abs/2503.00361) [find](https://scholar.google.com/scholar?q=Wei+Suo%2C+Lijun+Zhang%2C+Mengyang+Sun%2C+Lin+Yuanbo+Wu%2C+Peng+Wang%2C+and+Yanning+Zhang%2E+2025%2E+Octopus%3A+Alleviating+Hallucination+via+Dynamic+Contrastive+Decoding%2E+arXiv+preprint+arXiv%3A2503%2E00361+%282025%29%2E+%E2%80%94+%5Ba)
150. Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805 (2023).
151. Masayo Tomita, Katsuhiko Hayashi, and Tomoyuki Kaneko. 2025. The Role of Background Information in Reducing Object Hallucination in Vision-Language Models: Insights from Cutoff API Prompting. arXiv preprint arXiv:2502.15389 (2025). — [arXiv:2502.15389](https://arxiv.org/abs/2502.15389) [find](https://scholar.google.com/scholar?q=Masayo+Tomita%2C+Katsuhiko+Hayashi%2C+and+Tomoyuki+Kaneko%2E+2025%2E+The+Role+of+Background+Information+in+Reducing+Object+Hallucination+in+Vision%2DLanguage+Models%3A+Insights+from+Cutoff+API+Prompting%2E+arXiv+pr)
152. Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. 2024. [Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs](https://md.research.elamur.ai/free/en/papers/tong-2024-eyes-wide-shut-exploring-visual/). arXiv preprint arXiv:2401.06209 (2024).
153. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
154. Andrés Villa, Juan Carlos León Alcázar, Alvaro Soto, and Bernard Ghanem. 2023. Behind the Magic, MERLIM: Multi-modal Evaluation Benchmark for Large Image-Language Models. arXiv preprint arXiv:2312.02219 (2023). — [arXiv:2312.02219](https://arxiv.org/abs/2312.02219) [find](https://scholar.google.com/scholar?q=Andr%C3%A9s+Villa%2C+Juan+Carlos+Le%C3%B3n+Alc%C3%A1zar%2C+Alvaro+Soto%2C+and+Bernard+Ghanem%2E+2023%2E+Behind+the+Magic%2C+MERLIM%3A+Multi%2Dmodal+Evaluation+Benchmark+for+Large+Image%2DLanguage+Models%2E+arXiv+preprint+arXiv%3A2312%2E022)
155. Andrés Villa, Juan León Alcázar, Motasem Alfarra, Vladimir Araujo, Alvaro Soto, and Bernard Ghanem. 2025. EAGLE: Enhanced Visual Grounding Minimizes Hallucinations in Instructional Multimodal Models. arXiv preprint arXiv:2501.02699 (2025). — [arXiv:2501.02699](https://arxiv.org/abs/2501.02699) [find](https://scholar.google.com/scholar?q=Andr%C3%A9s+Villa%2C+Juan+Le%C3%B3n+Alc%C3%A1zar%2C+Motasem+Alfarra%2C+Vladimir+Araujo%2C+Alvaro+Soto%2C+and+Bernard+Ghanem%2E+2025%2E+EAGLE%3A+Enhanced+Visual+Grounding+Minimizes+Hallucinations+in+Instructional+Multimodal+Models%2E+)
156. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning. 1096–1103. [find](https://scholar.google.com/scholar?q=Pascal+Vincent%2C+Hugo+Larochelle%2C+Yoshua+Bengio%2C+and+Pierre%2DAntoine+Manzagol%2E+2008%2E+Extracting+and+composing+robust+features+with+denoising+autoencoders%2E+In+Proceedings+of+the+25th+international+confer)
157. Bin Wang, Fan Wu, Xiao Han, Jiahui Peng, Huaping Zhong, Pan Zhang, Xiaoyi Dong, Weijia Li, Wei Li, Jiaqi Wang, et al. 2023. Vigc: Visual instruction generation and correction. arXiv preprint arXiv:2308.12714 (2023). — [arXiv:2308.12714](https://arxiv.org/abs/2308.12714) [find](https://scholar.google.com/scholar?q=Bin+Wang%2C+Fan+Wu%2C+Xiao+Han%2C+Jiahui+Peng%2C+Huaping+Zhong%2C+Pan+Zhang%2C+Xiaoyi+Dong%2C+Weijia+Li%2C+Wei+Li%2C+Jiaqi+Wang%2C+et+al%2E+2023%2E+Vigc%3A+Visual+instruction+generation+and+correction%2E+arXiv+preprint+arXiv%3A230)
158. Chenxi Wang, Xiang Chen, Ningyu Zhang, Bozhong Tian, Haoming Xu, Shumin Deng, and Huajun Chen. 2024. Mllm can see? dynamic correction decoding for hallucination mitigation. arXiv preprint arXiv:2410.11779 (2024). — [arXiv:2410.11779](https://arxiv.org/abs/2410.11779) [find](https://scholar.google.com/scholar?q=Chenxi+Wang%2C+Xiang+Chen%2C+Ningyu+Zhang%2C+Bozhong+Tian%2C+Haoming+Xu%2C+Shumin+Deng%2C+and+Huajun+Chen%2E+2024%2E+Mllm+can+see%3F+dynamic+correction+decoding+for+hallucination+mitigation%2E+arXiv+preprint+arXiv%3A2410%2E1)
159. Jiaqi Wang, Yifei Gao, and Jitao Sang. 2024. VaLiD: Mitigating the Hallucination of Large Vision Language Models by Visual Layer Fusion Contrastive Decoding. arXiv preprint arXiv:2411.15839 (2024). — [arXiv:2411.15839](https://arxiv.org/abs/2411.15839) [find](https://scholar.google.com/scholar?q=Jiaqi+Wang%2C+Yifei+Gao%2C+and+Jitao+Sang%2E+2024%2E+VaLiD%3A+Mitigating+the+Hallucination+of+Large+Vision+Language+Models+by+Visual+Layer+Fusion+Contrastive+Decoding%2E+arXiv+preprint+arXiv%3A2411%2E15839+%282024%29%2E+%E2%80%94+)
160. Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. 2023. [An llm-free multi-dimensional benchmark for mllms hallucination evaluation](https://md.research.elamur.ai/free/en/papers/wang-2023-amber-llm-free-multi-dimensional/). arXiv preprint arXiv:2311.07397 (2023).
161. Junyang Wang, Yiyang Zhou, Guohai Xu, Pengcheng Shi, Chenlin Zhao, Haiyang Xu, Qinghao Ye, Ming Yan, Ji Zhang, Jihua Zhu, et al. 2023. Evaluation and analysis of hallucination in large vision-language models. arXiv preprint arXiv:2308.15126 (2023). — [arXiv:2308.15126](https://arxiv.org/abs/2308.15126) [find](https://scholar.google.com/scholar?q=Junyang+Wang%2C+Yiyang+Zhou%2C+Guohai+Xu%2C+Pengcheng+Shi%2C+Chenlin+Zhao%2C+Haiyang+Xu%2C+Qinghao+Ye%2C+Ming+Yan%2C+Ji+Zhang%2C+Jihua+Zhu%2C+et+al%2E+2023%2E+Evaluation+and+analysis+of+hallucination+in+large+vision%2Dlanguage)
162. Li Wang, Zechen Bai, Yonghua Zhang, and Hongtao Lu. 2020. Show, recall, and tell: Image captioning with recall mechanism. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 12176–12183. [find](https://scholar.google.com/scholar?q=Li+Wang%2C+Zechen+Bai%2C+Yonghua+Zhang%2C+and+Hongtao+Lu%2E+2020%2E+Show%2C+recall%2C+and+tell%3A+Image+captioning+with+recall+mechanism%2E+In+Proceedings+of+the+AAAI+conference+on+artificial+intelligence%2C+Vol%2E+34%2E+121)
163. Lei Wang, Jiabang He, Shenshen Li, Ning Liu, and Ee-Peng Lim. 2024. Mitigating fine-grained hallucination by fine-tuning large vision-language models with caption rewrites. In International Conference on Multimedia Modeling. Springer, 32–45. [find](https://scholar.google.com/scholar?q=Lei+Wang%2C+Jiabang+He%2C+Shenshen+Li%2C+Ning+Liu%2C+and+Ee%2DPeng+Lim%2E+2024%2E+Mitigating+fine%2Dgrained+hallucination+by+fine%2Dtuning+large+vision%2Dlanguage+models+with+caption+rewrites%2E+In+International+Conference)
164. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. [Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/). arXiv preprint arXiv:2409.12191 (2024).
165. Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al. 2023. [Cogvlm: Visual expert for pretrained language models](https://md.research.elamur.ai/free/en/papers/wang-2023-cogvlm-visual-expert-pretrained/). arXiv preprint arXiv:2311.03079 (2023).
166. Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. 2024. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. arXiv preprint arXiv:2403.18715 (2024). [find](https://scholar.google.com/scholar?q=Xintong+Wang%2C+Jingheng+Pan%2C+Liang+Ding%2C+and+Chris+Biemann%2E+2024%2E+Mitigating+hallucinations+in+large+vision%2Dlanguage+models+with+instruction+contrastive+decoding%2E+arXiv+preprint+arXiv%3A2403%2E18715+%282024%29)
167. Zehao Wang, Xinpeng Liu, Xiaoqian Wu, Yudonglin Zhang, Zhou Fang, Yifan Fang, Junfu Pu, Cewu Lu, and Yong-Lu Li. 2024. Verb Mirage: Unveiling and Assessing Verb Concept Hallucinations in Multimodal Large Language Models. arXiv preprint arXiv:2412.04939 (2024). — [arXiv:2412.04939](https://arxiv.org/abs/2412.04939) [find](https://scholar.google.com/scholar?q=Zehao+Wang%2C+Xinpeng+Liu%2C+Xiaoqian+Wu%2C+Yudonglin+Zhang%2C+Zhou+Fang%2C+Yifan+Fang%2C+Junfu+Pu%2C+Cewu+Lu%2C+and+Yong%2DLu+Li%2E+2024%2E+Verb+Mirage%3A+Unveiling+and+Assessing+Verb+Concept+Hallucinations+in+Multimodal+La)
168. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in Neural Information Processing Systems 35 (2022), 24824–24837.
169. Sangmin Woo, Jaehyuk Jang, Donguk Kim, Yubin Choi, and Changick Kim. 2024. Ritual: Random image transformations as a universal anti-hallucination lever in lvlms. arXiv preprint arXiv:2405.17821 (2024). — [arXiv:2405.17821](https://arxiv.org/abs/2405.17821) [find](https://scholar.google.com/scholar?q=Sangmin+Woo%2C+Jaehyuk+Jang%2C+Donguk+Kim%2C+Yubin+Choi%2C+and+Changick+Kim%2E+2024%2E+Ritual%3A+Random+image+transformations+as+a+universal+anti%2Dhallucination+lever+in+lvlms%2E+arXiv+preprint+arXiv%3A2405%2E17821+%282024%29)
170. Sangmin Woo, Donguk Kim, Jaehyuk Jang, Yubin Choi, and Changick Kim. 2024. Don’t Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models. arXiv preprint arXiv:2405.17820 (2024). [find](https://scholar.google.com/scholar?q=Sangmin+Woo%2C+Donguk+Kim%2C+Jaehyuk+Jang%2C+Yubin+Choi%2C+and+Changick+Kim%2E+2024%2E+Don%E2%80%99t+Miss+the+Forest+for+the+Trees%3A+Attentional+Vision+Calibration+for+Large+Vision+Language+Models%2E+arXiv+preprint+arXiv%3A24)
171. Junfei Wu, Qiang Liu, Ding Wang, Jinghao Zhang, Shu Wu, Liang Wang, and Tieniu Tan. 2024. Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models. arXiv preprint arXiv:2402.11622 (2024). [find](https://scholar.google.com/scholar?q=Junfei+Wu%2C+Qiang+Liu%2C+Ding+Wang%2C+Jinghao+Zhang%2C+Shu+Wu%2C+Liang+Wang%2C+and+Tieniu+Tan%2E+2024%2E+Logical+Closed+Loop%3A+Uncovering+Object+Hallucinations+in+Large+Vision%2DLanguage+Models%2E+arXiv+preprint+arXiv%3A24)
172. Shengqiong Wu, Hao Fei, Liangming Pan, William Yang Wang, Shuicheng Yan, and Tat-Seng Chua. 2024. Combating Multimodal LLM Hallucination via Bottom-up Holistic Reasoning. arXiv preprint arXiv:2412.11124 (2024). — [arXiv:2412.11124](https://arxiv.org/abs/2412.11124) [find](https://scholar.google.com/scholar?q=Shengqiong+Wu%2C+Hao+Fei%2C+Liangming+Pan%2C+William+Yang+Wang%2C+Shuicheng+Yan%2C+and+Tat%2DSeng+Chua%2E+2024%2E+Combating+Multimodal+LLM+Hallucination+via+Bottom%2Dup+Holistic+Reasoning%2E+arXiv+preprint+arXiv%3A2412%2E111)
173. Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. 2024. [Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding](https://md.research.elamur.ai/free/en/papers/wu-2024-deepseek-vl2-mixture-experts/). arXiv preprint arXiv:2412.10302 (2024).
174. Wenyi Xiao, Ziwei Huang, Leilei Gan, Wanggui He, Haoyuan Li, Zhelun Yu, Fangxun Shu, Hao Jiang, and Linchao Zhu. 2024. Detecting and mitigating hallucination in large vision language models via fine-grained ai feedback. arXiv preprint arXiv:2404.14233 (2024). — [arXiv:2404.14233](https://arxiv.org/abs/2404.14233) [find](https://scholar.google.com/scholar?q=Wenyi+Xiao%2C+Ziwei+Huang%2C+Leilei+Gan%2C+Wanggui+He%2C+Haoyuan+Li%2C+Zhelun+Yu%2C+Fangxun+Shu%2C+Hao+Jiang%2C+and+Linchao+Zhu%2E+2024%2E+Detecting+and+mitigating+hallucination+in+large+vision+language+models+via+fine%2Dg)
175. Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. 2024. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528 (2024). — [arXiv:2408.12528](https://arxiv.org/abs/2408.12528) [find](https://scholar.google.com/scholar?q=Jinheng+Xie%2C+Weijia+Mao%2C+Zechen+Bai%2C+David+Junhao+Zhang%2C+Weihao+Wang%2C+Kevin+Qinghong+Lin%2C+Yuchao+Gu%2C+Zhijie+Chen%2C+Zhenheng+Yang%2C+and+Mike+Zheng+Shou%2E+2024%2E+Show%2Do%3A+One+single+transformer+to+unify+mult)
176. Yuxi Xie, Guanzhen Li, Xiao Xu, and Min-Yen Kan. 2024. V-dpo: Mitigating hallucination in large vision language models via vision-guided direct preference optimization. arXiv preprint arXiv:2411.02712 (2024). [find](https://scholar.google.com/scholar?q=Yuxi+Xie%2C+Guanzhen+Li%2C+Xiao+Xu%2C+and+Min%2DYen+Kan%2E+2024%2E+V%2Ddpo%3A+Mitigating+hallucination+in+large+vision+language+models+via+vision%2Dguided+direct+preference+optimization%2E+arXiv+preprint+arXiv%3A2411%2E02712)
177. Shangyu Xing, Fei Zhao, Zhen Wu, Tuo An, Weihao Chen, Chunhui Li, Jianbing Zhang, and Xinyu Dai. 2024. EFUF: Efficient Fine-grained Unlearning Framework for Mitigating Hallucinations in Multimodal Large Language Models. arXiv preprint arXiv:2402.09801 (2024). [find](https://scholar.google.com/scholar?q=Shangyu+Xing%2C+Fei+Zhao%2C+Zhen+Wu%2C+Tuo+An%2C+Weihao+Chen%2C+Chunhui+Li%2C+Jianbing+Zhang%2C+and+Xinyu+Dai%2E+2024%2E+EFUF%3A+Efficient+Fine%2Dgrained+Unlearning+Framework+for+Mitigating+Hallucinations+in+Multimodal+Lar)
178. Yun Xing, Yiheng Li, Ivan Laptev, and Shijian Lu. 2024. Mitigating object hallucination via concentric causal attention. Advances in Neural Information Processing Systems 37 (2024), 92012–92035. [find](https://scholar.google.com/scholar?q=Yun+Xing%2C+Yiheng+Li%2C+Ivan+Laptev%2C+and+Shijian+Lu%2E+2024%2E+Mitigating+object+hallucination+via+concentric+causal+attention%2E+Advances+in+Neural+Information+Processing+Systems+37+%282024%29%2C+92012%E2%80%9392035%2E)
179. An Yan, Zhengyuan Yang, Junda Wu, Wanrong Zhu, Jianwei Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Julian McAuley, Jianfeng Gao, et al. 2024. List items one by one: A new data source and learning paradigm for multimodal llms. arXiv preprint arXiv:2404.16375 (2024). — [arXiv:2404.16375](https://arxiv.org/abs/2404.16375) [find](https://scholar.google.com/scholar?q=An+Yan%2C+Zhengyuan+Yang%2C+Junda+Wu%2C+Wanrong+Zhu%2C+Jianwei+Yang%2C+Linjie+Li%2C+Kevin+Lin%2C+Jianfeng+Wang%2C+Julian+McAuley%2C+Jianfeng+Gao%2C+et+al%2E+2024%2E+List+items+one+by+one%3A+A+new+data+source+and+learning+parad)
180. Siming Yan, Min Bai, Weifeng Chen, Xiong Zhou, Qixing Huang, and Li Erran Li. 2024. ViGoR: Improving Visual Grounding of Large Vision Language Models with Fine-Grained Reward Modeling. arXiv preprint arXiv:2402.06118 (2024). — [arXiv:2402.06118](https://arxiv.org/abs/2402.06118) [find](https://scholar.google.com/scholar?q=Siming+Yan%2C+Min+Bai%2C+Weifeng+Chen%2C+Xiong+Zhou%2C+Qixing+Huang%2C+and+Li+Erran+Li%2E+2024%2E+ViGoR%3A+Improving+Visual+Grounding+of+Large+Vision+Language+Models+with+Fine%2DGrained+Reward+Modeling%2E+arXiv+preprint+)
181. Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. 2023. [Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v](https://md.research.elamur.ai/free/en/papers/yang-2023-set-mark-prompting-unleashes-extraordinary/). arXiv preprint arXiv:2310.11441 (2023).
182. Le Yang, Ziwei Zheng, Boxu Chen, Zhengyu Zhao, Chenhao Lin, and Chao Shen. 2024. Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection. arXiv preprint arXiv:2412.13817 (2024). — [arXiv:2412.13817](https://arxiv.org/abs/2412.13817) [find](https://scholar.google.com/scholar?q=Le+Yang%2C+Ziwei+Zheng%2C+Boxu+Chen%2C+Zhengyu+Zhao%2C+Chenhao+Lin%2C+and+Chao+Shen%2E+2024%2E+Nullu%3A+Mitigating+Object+Hallucinations+in+Large+Vision%2DLanguage+Models+via+HalluSpace+Projection%2E+arXiv+preprint+arXiv)
183. Zhihe Yang, Xufang Luo, Dongqi Han, Yunjian Xu, and Dongsheng Li. 2025. Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key. arXiv preprint arXiv:2501.09695 (2025). — [arXiv:2501.09695](https://arxiv.org/abs/2501.09695) [find](https://scholar.google.com/scholar?q=Zhihe+Yang%2C+Xufang+Luo%2C+Dongqi+Han%2C+Yunjian+Xu%2C+and+Dongsheng+Li%2E+2025%2E+Mitigating+Hallucinations+in+Large+Vision%2DLanguage+Models+via+DPO%3A+On%2DPolicy+Data+Hold+the+Key%2E+arXiv+preprint+arXiv%3A2501%2E09695+)
184. Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. 2023. [mplug-owl: Modularization empowers large language models with multimodality](https://md.research.elamur.ai/free/en/papers/ye-2023-mplug-owl-modularization-empowers-large/). arXiv preprint arXiv:2304.14178 (2023).
185. Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2023. [mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration](https://md.research.elamur.ai/free/en/papers/ye-2023-mplug-owl2-revolutionizing-multi-modal/). arXiv preprint arXiv:2311.04257 (2023).
186. Moon Ye-Bin, Nam Hyeon-Woo, Wonseok Choi, and Tae-Hyun Oh. 2024. Beaf: Observing before-after changes to evaluate hallucination in vision-language models. In European Conference on Computer Vision. Springer, 232–248. [find](https://scholar.google.com/scholar?q=Moon+Ye%2DBin%2C+Nam+Hyeon%2DWoo%2C+Wonseok+Choi%2C+and+Tae%2DHyun+Oh%2E+2024%2E+Beaf%3A+Observing+before%2Dafter+changes+to+evaluate+hallucination+in+vision%2Dlanguage+models%2E+In+European+Conference+on+Computer+Vision%2E+Sp)
187. Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. [A survey on multimodal large language models](https://md.research.elamur.ai/free/en/papers/yin-2023-survey-multimodal-large-language-models/). arXiv preprint arXiv:2306.13549 (2023).
188. Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. 2023. [Woodpecker: Hallucination Correction for Multimodal Large Language Models](https://md.research.elamur.ai/free/en/papers/yin-2023-woodpecker-hallucination-correction-multimodal-large/). https://arxiv.org/abs/2310.16045 [↗](https://arxiv.org/abs/2310.16045)
189. Fei Yu, Hongbo Zhang, and Benyou Wang. 2023. [Nature language reasoning, a survey](https://md.research.elamur.ai/free/en/papers/yu-2023-natural-language-reasoning-survey/). ArXiv preprint abs/2303.14725 (2023). https://arxiv.org/abs/2303.14725 [↗](https://arxiv.org/abs/2303.14725)
190. Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. [n. d.]. [Coca: Contrastive captioners are image-text foundation models](https://md.research.elamur.ai/free/en/papers/yu-2022-coca-contrastive-captioners-image-text/). arXiv 2022. arXiv preprint arXiv:2205.01917 ([n. d.]).
191. Qifan Yu, Juncheng Li, Longhui Wei, Liang Pang, Wentao Ye, Bosheng Qin, Siliang Tang, Qi Tian, and Yueting Zhuang. 2023. HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data. arXiv preprint arXiv:2311.13614 (2023). — [arXiv:2311.13614](https://arxiv.org/abs/2311.13614) [find](https://scholar.google.com/scholar?q=Qifan+Yu%2C+Juncheng+Li%2C+Longhui+Wei%2C+Liang+Pang%2C+Wentao+Ye%2C+Bosheng+Qin%2C+Siliang+Tang%2C+Qi+Tian%2C+and+Yueting+Zhuang%2E+2023%2E+HalluciDoctor%3A+Mitigating+Hallucinatory+Toxicity+in+Visual+Instruction+Data%2E+ar)
192. Tianyu Yu, Jinyi Hu, Yuan Yao, Haoye Zhang, Yue Zhao, Chongyi Wang, Shan Wang, Yinxv Pan, Jiao Xue, Dahai Li, et al. 2023. Reformulating vision-language foundation models and datasets towards universal multimodal assistants. arXiv preprint arXiv:2310.00653 (2023). — [arXiv:2310.00653](https://arxiv.org/abs/2310.00653) [find](https://scholar.google.com/scholar?q=Tianyu+Yu%2C+Jinyi+Hu%2C+Yuan+Yao%2C+Haoye+Zhang%2C+Yue+Zhao%2C+Chongyi+Wang%2C+Shan+Wang%2C+Yinxv+Pan%2C+Jiao+Xue%2C+Dahai+Li%2C+et+al%2E+2023%2E+Reformulating+vision%2Dlanguage+foundation+models+and+datasets+towards+universa)
193. Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. 2023. [RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback](https://md.research.elamur.ai/free/en/papers/yu-2023-rlhf-v-towards-trustworthy-mllms/). arXiv preprint arXiv:2312.00849 (2023).
194. Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu, Ganqu Cui, Taiwen He, Zhiyuan Liu, Tat-Seng Chua, et al. 2024. Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness. arXiv preprint arXiv:2405.17220 (2024). — [arXiv:2405.17220](https://arxiv.org/abs/2405.17220) [find](https://scholar.google.com/scholar?q=Tianyu+Yu%2C+Haoye+Zhang%2C+Yuan+Yao%2C+Yunkai+Dang%2C+Da+Chen%2C+Xiaoman+Lu%2C+Ganqu+Cui%2C+Taiwen+He%2C+Zhiyuan+Liu%2C+Tat%2DSeng+Chua%2C+et+al%2E+2024%2E+Rlaif%2Dv%3A+Aligning+mllms+through+open%2Dsource+ai+feedback+for+super+gpt)
195. Fan Yuan, Chi Qin, Xiaogang Xu, and Piji Li. 2024. HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding. arXiv preprint arXiv:2409.20429 (2024). [find](https://scholar.google.com/scholar?q=Fan+Yuan%2C+Chi+Qin%2C+Xiaogang+Xu%2C+and+Piji+Li%2E+2024%2E+HELPD%3A+Mitigating+Hallucination+of+LVLMs+by+Hierarchical+Feedback+Learning+with+Vision%2Denhanced+Penalty+Decoding%2E+arXiv+preprint+arXiv%3A2409%2E20429+%2820)
196. Zihao Yue, Liang Zhang, and Qin Jin. 2024. Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective. arXiv preprint arXiv:2402.14545 (2024). [find](https://scholar.google.com/scholar?q=Zihao+Yue%2C+Liang+Zhang%2C+and+Qin+Jin%2E+2024%2E+Less+is+More%3A+Mitigating+Multimodal+Hallucination+from+an+EOS+Decision+Perspective%2E+arXiv+preprint+arXiv%3A2402%2E14545+%282024%29%2E)
197. Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. 2022. When and Why Vision-Language Models Behave like Bags-Of-Words, and What to Do About It?. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Mert+Yuksekgonul%2C+Federico+Bianchi%2C+Pratyusha+Kalluri%2C+Dan+Jurafsky%2C+and+James+Zou%2E+2022%2E+When+and+Why+Vision%2DLanguage+Models+Behave+like+Bags%2DOf%2DWords%2C+and+What+to+Do+About+It%3F%2E+In+The+Eleventh+Inter)
198. Yan Zeng, Hanbo Zhang, Jiani Zheng, Jiangnan Xia, Guoqiang Wei, Yang Wei, Yuchen Zhang, and Tao Kong. 2023. What matters in training a gpt4-style language model with multimodal inputs? arXiv preprint arXiv:2307.02469 (2023). [find](https://scholar.google.com/scholar?q=Yan+Zeng%2C+Hanbo+Zhang%2C+Jiani+Zheng%2C+Jiangnan+Xia%2C+Guoqiang+Wei%2C+Yang+Wei%2C+Yuchen+Zhang%2C+and+Tao+Kong%2E+2023%2E+What+matters+in+training+a+gpt4%2Dstyle+language+model+with+multimodal+inputs%3F+arXiv+preprint+)
199. Bohan Zhai, Shijia Yang, Chenfeng Xu, Sheng Shen, Kurt Keutzer, and Manling Li. 2023. HallE-Switch: Controlling Object Hallucination in Large Vision Language Models. arXiv e-prints (2023), arXiv–2310. [find](https://scholar.google.com/scholar?q=Bohan+Zhai%2C+Shijia+Yang%2C+Chenfeng+Xu%2C+Sheng+Shen%2C+Kurt+Keutzer%2C+and+Manling+Li%2E+2023%2E+HallE%2DSwitch%3A+Controlling+Object+Hallucination+in+Large+Vision+Language+Models%2E+arXiv+e%2Dprints+%282023%29%2C+arXiv%E2%80%932310%2E)
200. Ao Zhang, Hao Fei, Yuan Yao, Wei Ji, Li Li, Zhiyuan Liu, and Tat-Seng Chua. 2024. VPGTrans: Transfer visual prompt generator across LLMs. Advances in Neural Information Processing Systems 36 (2024). [find](https://scholar.google.com/scholar?q=Ao+Zhang%2C+Hao+Fei%2C+Yuan+Yao%2C+Wei+Ji%2C+Li+Li%2C+Zhiyuan+Liu%2C+and+Tat%2DSeng+Chua%2E+2024%2E+VPGTrans%3A+Transfer+visual+prompt+generator+across+LLMs%2E+Advances+in+Neural+Information+Processing+Systems+36+%282024%29%2E)
201. Ce Zhang, Zifu Wan, Zhehan Kan, Martin Q Ma, Simon Stepputtis, Deva Ramanan, Russ Salakhutdinov, Louis-Philippe Morency, Katia Sycara, and Yaqi Xie. 2025. Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models. arXiv preprint arXiv:2502.06130… — [arXiv:2502.06130](https://arxiv.org/abs/2502.06130) [find](https://scholar.google.com/scholar?q=Ce+Zhang%2C+Zifu+Wan%2C+Zhehan+Kan%2C+Martin+Q+Ma%2C+Simon+Stepputtis%2C+Deva+Ramanan%2C+Russ+Salakhutdinov%2C+Louis%2DPhilippe+Morency%2C+Katia+Sycara%2C+and+Yaqi+Xie%2E+2025%2E+Self%2DCorrecting+Decoding+with+Generative+Feed)
202. Jinrui Zhang, Teng Wang, Haigang Zhang, Ping Lu, and Feng Zheng. 2024. Reflective instruction tuning: Mitigating hallucinations in large vision-language models. In European Conference on Computer Vision. Springer, 196–213. [find](https://scholar.google.com/scholar?q=Jinrui+Zhang%2C+Teng+Wang%2C+Haigang+Zhang%2C+Ping+Lu%2C+and+Feng+Zheng%2E+2024%2E+Reflective+instruction+tuning%3A+Mitigating+hallucinations+in+large+vision%2Dlanguage+models%2E+In+European+Conference+on+Computer+Visi)
203. Kejia Zhang, Keda Tao, Jiasheng Tang, and Huan Wang. 2025. Poison as Cure: Visual Noise for Mitigating Object Hallucinations in LVMs. arXiv preprint arXiv:2501.19164 (2025). — [arXiv:2501.19164](https://arxiv.org/abs/2501.19164) [find](https://scholar.google.com/scholar?q=Kejia+Zhang%2C+Keda+Tao%2C+Jiasheng+Tang%2C+and+Huan+Wang%2E+2025%2E+Poison+as+Cure%3A+Visual+Noise+for+Mitigating+Object+Hallucinations+in+LVMs%2E+arXiv+preprint+arXiv%3A2501%2E19164+%282025%29%2E+%E2%80%94+%5BarXiv%3A2501%2E19164%5D%28https)
204. Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. 2023. [Instruction Tuning for Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/zhang-2023-instruction-tuning-large-language-models/). ArXiv preprint abs/2308.10792 (2023). https://arxiv.org/abs/2308.10792 [↗](https://arxiv.org/abs/2308.10792)
205. Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Kai Chen, and Ping Luo. 2023. Gpt4roi: Instruction tuning large language model on region-of-interest. arXiv preprint arXiv:2307.03601 (2023). — [arXiv:2307.03601](https://arxiv.org/abs/2307.03601) [find](https://scholar.google.com/scholar?q=Shilong+Zhang%2C+Peize+Sun%2C+Shoufa+Chen%2C+Min+Xiao%2C+Wenqi+Shao%2C+Wenwei+Zhang%2C+Kai+Chen%2C+and+Ping+Luo%2E+2023%2E+Gpt4roi%3A+Instruction+tuning+large+language+model+on+region%2Dof%2Dinterest%2E+arXiv+preprint+arXiv%3A23)
206. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675 (2019). [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Varsha+Kishore%2C+Felix+Wu%2C+Kilian+Q+Weinberger%2C+and+Yoav+Artzi%2E+2019%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+arXiv+preprint+arXiv%3A1904%2E09675+%282019%29%2E)
207. Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B Hashimoto. 2023. [Benchmarking large language models for news summarization](https://md.research.elamur.ai/free/en/papers/zhang-2024-benchmarking-large-language-models-news/). ArXiv preprint abs/2301.13848 (2023). https://arxiv.org/abs/2301.13848 [↗](https://arxiv.org/abs/2301.13848)
208. Xiaofeng Zhang, Yihao Quan, Chaochen Gu, Chen Shen, Xiaosong Yuan, Shaotian Yan, Hao Cheng, Kaijie Wu, and Jieping Ye. 2024. Seeing clearly by layer two: Enhancing attention heads to alleviate hallucination in lvlms. arXiv preprint arXiv:2411.09968 (2024). — [arXiv:2411.09968](https://arxiv.org/abs/2411.09968) [find](https://scholar.google.com/scholar?q=Xiaofeng+Zhang%2C+Yihao+Quan%2C+Chaochen+Gu%2C+Chen+Shen%2C+Xiaosong+Yuan%2C+Shaotian+Yan%2C+Hao+Cheng%2C+Kaijie+Wu%2C+and+Jieping+Ye%2E+2024%2E+Seeing+clearly+by+layer+two%3A+Enhancing+attention+heads+to+alleviate+halluci)
209. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2023. [Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/). ArXiv preprint abs/2309.01219 (20…
210. Haozhe Zhao, Zefan Cai, Shuzheng Si, Xiaojian Ma, Kaikai An, Liang Chen, Zixuan Liu, Sheng Wang, Wenjuan Han, and Baobao Chang. 2023. [Mmicl: Empowering vision-language model with multi-modal in-context learning](https://md.research.elamur.ai/free/en/papers/zhao-2023-mmicl-empowering-vision-language-model/). arXiv preprint arXiv:2309.07915 (2023).
211. Henry Hengyuan Zhao, Pan Zhou, Difei Gao, Zechen Bai, and Mike Zheng Shou. 2024. Lova3: Learning to visual question answering, asking and assessment. Advances in Neural Information Processing Systems 37 (2024), 115146– 115175. [find](https://scholar.google.com/scholar?q=Henry+Hengyuan+Zhao%2C+Pan+Zhou%2C+Difei+Gao%2C+Zechen+Bai%2C+and+Mike+Zheng+Shou%2E+2024%2E+Lova3%3A+Learning+to+visual+question+answering%2C+asking+and+assessment%2E+Advances+in+Neural+Information+Processing+Systems+)
212. Linxi Zhao, Yihe Deng, Weitong Zhang, and Quanquan Gu. 2024. Mitigating Object Hallucination in Large Vision-Language Models via Classifier-Free Guidance. arXiv preprint arXiv:2402.08680 (2024). — [arXiv:2402.08680](https://arxiv.org/abs/2402.08680) [find](https://scholar.google.com/scholar?q=Linxi+Zhao%2C+Yihe+Deng%2C+Weitong+Zhang%2C+and+Quanquan+Gu%2E+2024%2E+Mitigating+Object+Hallucination+in+Large+Vision%2DLanguage+Models+via+Classifier%2DFree+Guidance%2E+arXiv+preprint+arXiv%3A2402%2E08680+%282024%29%2E+%E2%80%94+%5Bar)
213. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. [A survey of large language models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/). ArXiv preprint abs/2303.18223 (2023). https://arxiv.org/abs/2303.18223 [↗](https://arxiv.org/abs/2303.18223)
214. Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. 2023. Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization. arXiv preprint arXiv:2311.16839 (2023). — [arXiv:2311.16839](https://arxiv.org/abs/2311.16839) [find](https://scholar.google.com/scholar?q=Zhiyuan+Zhao%2C+Bin+Wang%2C+Linke+Ouyang%2C+Xiaoyi+Dong%2C+Jiaqi+Wang%2C+and+Conghui+He%2E+2023%2E+Beyond+Hallucinations%3A+Enhancing+LVLMs+through+Hallucination%2DAware+Direct+Preference+Optimization%2E+arXiv+preprint+a)
215. Haojie Zheng, Tianyang Xu, Hanchi Sun, Shu Pu, Ruoxi Chen, and Lichao Sun. 2024. Thinking before looking: Improving multimodal llm reasoning via mitigating visual hallucination. arXiv preprint arXiv:2411.12591 (2024). — [arXiv:2411.12591](https://arxiv.org/abs/2411.12591) [find](https://scholar.google.com/scholar?q=Haojie+Zheng%2C+Tianyang+Xu%2C+Hanchi+Sun%2C+Shu+Pu%2C+Ruoxi+Chen%2C+and+Lichao+Sun%2E+2024%2E+Thinking+before+looking%3A+Improving+multimodal+llm+reasoning+via+mitigating+visual+hallucination%2E+arXiv+preprint+arXiv%3A2)
216. Kening Zheng, Junkai Chen, Yibo Yan, Xin Zou, and Xuming Hu. 2024. Reefknot: A comprehensive benchmark for relation hallucination evaluation, analysis and mitigation in multimodal large language models. arXiv preprint arXiv:2408.09429 (2024). [find](https://scholar.google.com/scholar?q=Kening+Zheng%2C+Junkai+Chen%2C+Yibo+Yan%2C+Xin+Zou%2C+and+Xuming+Hu%2E+2024%2E+Reefknot%3A+A+comprehensive+benchmark+for+relation+hallucination+evaluation%2C+analysis+and+mitigation+in+multimodal+large+language+model)
217. Weihong Zhong, Xiaocheng Feng, Liang Zhao, Qiming Li, Lei Huang, Yuxuan Gu, Weitao Ma, Yuan Xu, and Bing Qin. 2024. Investigating and mitigating the multimodal hallucination snowballing in large vision-language models. arXiv preprint arXiv:2407.00569 (2024). [find](https://scholar.google.com/scholar?q=Weihong+Zhong%2C+Xiaocheng+Feng%2C+Liang+Zhao%2C+Qiming+Li%2C+Lei+Huang%2C+Yuxuan+Gu%2C+Weitao+Ma%2C+Yuan+Xu%2C+and+Bing+Qin%2E+2024%2E+Investigating+and+mitigating+the+multimodal+hallucination+snowballing+in+large+visio)
218. Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. 2019. Semantic understanding of scenes through the ade20k dataset. International Journal of Computer Vision 127 (2019), 302–321. [find](https://scholar.google.com/scholar?q=Bolei+Zhou%2C+Hang+Zhao%2C+Xavier+Puig%2C+Tete+Xiao%2C+Sanja+Fidler%2C+Adela+Barriuso%2C+and+Antonio+Torralba%2E+2019%2E+Semantic+understanding+of+scenes+through+the+ade20k+dataset%2E+International+Journal+of+Computer+)
219. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. Lima: Less is more for alignment. ArXiv preprint abs/2305.11206 (2023). https://arxiv.org/abs/2305.11206 [↗](https://arxiv.org/abs/2305.11206)
220. Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. 2024. [Transfusion: Predict the next token and diffuse images with one multi-modal model](https://md.research.elamur.ai/free/en/papers/zhou-2024-transfusion-predict-next-token-diffuse/). arXiv preprint arXiv:2408.11039 (2024).
221. Guanyu Zhou, Yibo Yan, Xin Zou, Kun Wang, Aiwei Liu, and Xuming Hu. 2024. Mitigating modality prior-induced hallucinations in multimodal large language models via deciphering attention causality. arXiv preprint arXiv:2410.04780 (2024). — [arXiv:2410.04780](https://arxiv.org/abs/2410.04780) [find](https://scholar.google.com/scholar?q=Guanyu+Zhou%2C+Yibo+Yan%2C+Xin+Zou%2C+Kun+Wang%2C+Aiwei+Liu%2C+and+Xuming+Hu%2E+2024%2E+Mitigating+modality+prior%2Dinduced+hallucinations+in+multimodal+large+language+models+via+deciphering+attention+causality%2E+arXi)
222. Qiang Zhou, Zhibin Wang, Wei Chu, Yinghui Xu, Hao Li, and Yuan Qi. 2023. InfMLLM: A Unified Framework for Visual-Language Tasks. arXiv:2311.06791 [cs.CV] [find](https://scholar.google.com/scholar?q=Qiang+Zhou%2C+Zhibin+Wang%2C+Wei+Chu%2C+Yinghui+Xu%2C+Hao+Li%2C+and+Yuan+Qi%2E+2023%2E+InfMLLM%3A+A+Unified+Framework+for+Visual%2DLanguage+Tasks%2E+arXiv%3A2311%2E06791+%5Bcs%2ECV%5D)
223. Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. 2024. Aligning Modalities in Vision Large Language Models via Preference Fine-tuning. arXiv preprint arXiv:2402.11411 (2024). — [arXiv:2402.11411](https://arxiv.org/abs/2402.11411) [find](https://scholar.google.com/scholar?q=Yiyang+Zhou%2C+Chenhang+Cui%2C+Rafael+Rafailov%2C+Chelsea+Finn%2C+and+Huaxiu+Yao%2E+2024%2E+Aligning+Modalities+in+Vision+Large+Language+Models+via+Preference+Fine%2Dtuning%2E+arXiv+preprint+arXiv%3A2402%2E11411+%282024%29%2E+)
224. Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2023. [Analyzing and mitigating object hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-analyzing-mitigating-object-hallucination-large/). arXiv preprint arXiv:2310.00754 (2023).
225. Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. [Minigpt-4: Enhancing vision-language understanding with advanced large language models](https://md.research.elamur.ai/free/en/papers/zhu-2023-minigpt-4-enhancing-vision-language/). arXiv preprint arXiv:2304.10592 (2023).
226. Lanyun Zhu, Deyi Ji, Tianrun Chen, Peng Xu, Jieping Ye, and Jun Liu. 2024. IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding. arXiv preprint arXiv:2402.18476 (2024). — [arXiv:2402.18476](https://arxiv.org/abs/2402.18476) [find](https://scholar.google.com/scholar?q=Lanyun+Zhu%2C+Deyi+Ji%2C+Tianrun+Chen%2C+Peng+Xu%2C+Jieping+Ye%2C+and+Jun+Liu%2E+2024%2E+IBD%3A+Alleviating+Hallucinations+in+Large+Vision%2DLanguage+Models+via+Image%2DBiased+Decoding%2E+arXiv+preprint+arXiv%3A2402%2E18476+%282)
227. Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Jiajun Chen, Lei Li, and Shujian Huang. 2023. [Multilingual machine translation with large language models: Empirical results and analysis](https://md.research.elamur.ai/free/en/papers/zhu-2024-multilingual-machine-translation-large-language/). ArXiv preprint abs/2304.04675 (2023). https://arxiv.org/abs/2304.04675 [↗](https://arxiv.org/abs/2304.04675)
228. Xianwei Zhuang, Zhihong Zhu, Yuxin Xie, Liming Liang, and Yuexian Zou. 2025. VASparse: Towards Efficient Visual Hallucination Mitigation for Large Vision-Language Model via Visual-Aware Sparsification. arXiv preprint arXiv:2501.06553 (2025). — [arXiv:2501.06553](https://arxiv.org/abs/2501.06553) [find](https://scholar.google.com/scholar?q=Xianwei+Zhuang%2C+Zhihong+Zhu%2C+Yuxin+Xie%2C+Liming+Liang%2C+and+Yuexian+Zou%2E+2025%2E+VASparse%3A+Towards+Efficient+Visual+Hallucination+Mitigation+for+Large+Vision%2DLanguage+Model+via+Visual%2DAware+Sparsification)

*Updated 2 November 2026*