---
title: "A Survey on Hallucination in Large Vision-Language Models (2024)"
description: "The authors define the problem along two axes. 60 references, cited by 91 papers here. Among them: 22."
authors: ["Hanchao Liu", "Wenyuan Xue", "Yifei Chen", "Dapeng Chen", "Xiutian Zhao", "Ke Wang", "Liping Hou", "Rongjun Li", "Wei Peng"]
year: 2024
venue: "arXiv:2402.00253"
source: "https://arxiv.org/abs/2402.00253"
doi: "10.48550/arxiv.2402.00253"
arxiv: "2402.00253"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2024-survey-hallucination-large-vision-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2024-survey-hallucination-large-vision-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / A Survey on Hallucination in Large Vision-Language Models

# A Survey on Hallucination in Large Vision-Language Models

**Authors:** Hanchao Liu, Wenyuan Xue, Yifei Chen, [Dapeng Chen](https://md.research.elamur.ai/free/en/authors/dapeng-chen/), [Xiutian Zhao](https://md.research.elamur.ai/free/en/authors/xiutian-zhao/), [Ke Wang](https://md.research.elamur.ai/free/en/authors/ke-wang/), Liping Hou, Rongjun Li and [Wei Peng](https://md.research.elamur.ai/free/en/authors/wei-peng/) · **Year:** 2024 · **Venue:** arXiv:2402.00253 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2402.00253) · [PDF](https://arxiv.org/pdf/2402.00253) · [DOI 10.48550/arxiv.2402.00253](https://doi.org/10.48550/arxiv.2402.00253) · [arXiv 2402.00253](https://arxiv.org/abs/2402.00253) · [OpenAlex](https://openalex.org/W4391505377)

**Citations:** 22 citations (OpenAlex, 2026-10-14) · [91](https://md.research.elamur.ai/free/en/papers/liu-2024-survey-hallucination-large-vision-language/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This survey from Huawei's IT Innovation and Research Center examines hallucination in Large Vision-Language Models (LVLMs), meaning cases where generated text contradicts what an image actually shows. The authors define the problem along two axes. By task, it appears as wrong yes/no judgements or unfaithful descriptions. By semantics, it involves objects, attributes or relations.

The survey sorts evaluation approaches into two families. One measures how much hallucinated content appears in generated output. The other tests whether a model can detect hallucinations. Benchmarks are grouped the same way, as generative or discriminative.

The authors trace the causes to several sources: biased training data, weak visual grounding by encoders, imperfect alignment between modalities, insufficient attention to context, and the generative tendencies inherited from LLMs. Mitigation methods are organized to match these causes: better data, changes to model modules, modified decoding, preference alignment, and post-hoc correction of outputs.

The paper closes with open research directions. It is a useful map for anyone building or evaluating multimodal systems that need factual reliability. The authors maintain an accompanying reference list on GitHub (lhanchao777/LVLM-Hallucinations-Survey).

## Key points
- LVLMs are described as having three parts: a visual encoder (typically derived from CLIP), a connector (cross-attention, adapters, Q-Former, or a linear/MLP layer), and an LLM. Training proceeds in two stages, pre-training and then instruction-tuning.
- Evaluation tools include CHAIR, which counts object mismatches against ground-truth captions, and CCEval, which adapts this idea to the larger object vocabularies of LVLMs.
- Preference-based mitigation covers LLaVA-RLHF, which uses a reward model with fact-augmented RLHF, and DPO variants such as HA-DPO and FDPO. Decoding-time fixes include visual contrastive decoding, which compares outputs from original and perturbed images.
- Post-processing correctors are represented by LURE, a revisor model built on MiniGPT-4, and Woodpecker, a training-free five-stage pipeline that checks claims against a structured visual knowledge base.
- Proposed future directions are finer-grained, spatially aware supervision; adding more modalities such as video; using LVLMs as agents that call detection and segmentation tools; and interpretability research into the internal mechanisms behind hallucination.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Huawei Technologies](https://md.research.elamur.ai/free/en/companies/huawei-technologies/) — Company, Shenzhen, China · 96 works · 15,432 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works

## Cite

Cite as: Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li and Wei Peng. 2024. A Survey on Hallucination in Large Vision-Language Models. arXiv:2402.00253.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{liu2024survey,
  title = {{A Survey on Hallucination in Large Vision-Language Models}},
  author = {Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li and Wei Peng},
  year = {2024},
  doi = {10.48550/arxiv.2402.00253},
  eprint = {2402.00253},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2402.00253}
}
```

- [← previous A Survey on Efficient Inference for Large Language Models](https://md.research.elamur.ai/free/en/papers/zhou-2024-survey-efficient-inference-large-language/)
- [next → ALLaM: Large Language Models for Arabic and English](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

60 references, 37 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, et al. [Flamingo: a visual language model for few-shot learning](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/). In NeurIPS, volume 35, 2022.
2. Jinze Bai, Shuai Bai, Shusheng Yang, et al. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.[12966, 2023](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/).
3. Chi Chen, Ruoyu Qin, Fuwen Luo, et al. Position-enhanced visual instruction tuning for multimodal large language models. arXiv preprint arXiv:2308.13437, 2023. — [arXiv:2308.13437](https://arxiv.org/abs/2308.13437) [find](https://scholar.google.com/scholar?q=Chi+Chen%2C+Ruoyu+Qin%2C+Fuwen+Luo%2C+et+al%2E+Position%2Denhanced+visual+instruction+tuning+for+multimodal+large+language+models%2E+arXiv+preprint+arXiv%3A2308%2E13437%2C+2023%2E+%E2%80%94+%5BarXiv%3A2308%2E13437%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
4. Jun Chen, Deyao Zhu, Xiaoqian Shen, et al. [Minigpt-v2: large language model as a unified interface for vision-language multi-task learning](https://md.research.elamur.ai/free/en/papers/chen-2023-minigpt-v2-large-language-model/). arXiv preprint arXiv:2310.09478, 2023.
5. Zhe Chen, Jiannan Wu, Wenhai Wang, et al. [Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks](https://md.research.elamur.ai/free/en/papers/chen-2023-internvl-scaling-up-vision/). arXiv preprint arXiv:2312.14238, 2023.
6. Siyuan Cheng, Bozhong Tian, Qingbin Liu, et al. Can we edit multimodal large language models? In EMNLP, 2023. [find](https://scholar.google.com/scholar?q=Siyuan+Cheng%2C+Bozhong+Tian%2C+Qingbin+Liu%2C+et+al%2E+Can+we+edit+multimodal+large+language+models%3F+In+EMNLP%2C+2023%2E)
7. Jaemin Cho, Seunghyun Yoon, Ajinkya Kale, et al. Fine-grained image captioning with clip reward. In Findings of NAACL, 2022. [find](https://scholar.google.com/scholar?q=Jaemin+Cho%2C+Seunghyun+Yoon%2C+Ajinkya+Kale%2C+et+al%2E+Fine%2Dgrained+image+captioning+with+clip+reward%2E+In+Findings+of+NAACL%2C+2022%2E)
8. Yung-Sung Chuang, Yujia Xie, Hongyin Luo, et al. [Dola: Decoding by contrasting layers improves factuality in large language models](https://md.research.elamur.ai/free/en/papers/chuang-2023-dola-decoding-contrasting-layers-improves/). arXiv preprint arXiv:2309.03883, 2023.
9. Wenliang Dai, Junnan Li, Dongxu Li, et al. [Instructblip: Towards general-purpose vision-language models with instruction tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/). arXiv preprint arXiv:2305.06500, 2023.
10. Wenliang Dai, Zihan Liu, Ziwei Ji, et al. Plausible may not be faithful: Probing object hallucination in vision-language pre-training. In EACL, 2023. [find](https://scholar.google.com/scholar?q=Wenliang+Dai%2C+Zihan+Liu%2C+Ziwei+Ji%2C+et+al%2E+Plausible+may+not+be+faithful%3A+Probing+object+hallucination+in+vision%2Dlanguage+pre%2Dtraining%2E+In+EACL%2C+2023%2E)
11. Nouha Dziri, Andrea Madotto, Osmar R Zaiane, et al. [Neural path hunter: Reducing hallucination in dialogue systems via path grounding](https://md.research.elamur.ai/free/en/papers/dziri-2021-neural-path-hunter-reducing-hallucination/). In EMNLP, 2021.
12. Peng Gao, Jiaming Han, Renrui Zhang, et al. [Llama-adapter v2: Parameter-efficient visual instruction model](https://md.research.elamur.ai/free/en/papers/gao-2023-llama-adapter-v2-parameter-efficient/). arXiv preprint arXiv:2304.15010, 2023.
13. Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, et al. Imagebind one embedding space to bind them all. In CVPR, 2023. — [ImageBind: One Embedding Space To Bind Them All](https://md.research.elamur.ai/free/en/papers/girdhar-2023-imagebind-one-embedding-space-bind/)
14. Anisha Gunjal, Jihan Yin, and Erhan Bas. [Detecting and preventing hallucinations in large vision language models](https://md.research.elamur.ai/free/en/papers/gunjal-2023-detecting-preventing-hallucinations-large-vision/). arXiv preprint arXiv:2308.06394, 2023.
15. Jiaming Han, Renrui Zhang, Wenqi Shao, et al. Imagebind-llm: Multi-modality instruction tuning. arXiv preprint arXiv:2309.03905, 2023. — [arXiv:2309.03905](https://arxiv.org/abs/2309.03905) [find](https://scholar.google.com/scholar?q=Jiaming+Han%2C+Renrui+Zhang%2C+Wenqi+Shao%2C+et+al%2E+Imagebind%2Dllm%3A+Multi%2Dmodality+instruction+tuning%2E+arXiv+preprint+arXiv%3A2309%2E03905%2C+2023%2E+%E2%80%94+%5BarXiv%3A2309%2E03905%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2309%2E03905%29)
16. Ari Holtzman, Jan Buys, Li Du, et al. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In ICLR, 2020.
17. Edward J. Hu, Yelong Shen, Phillip Wallis, et al. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In ICLR, 2022.
18. Hongyu Hu, Jiyuan Zhang, Minyi Zhao, et al. Ciem: Contrastive instruction evaluation method for better instruction tuning. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following, 2023. [find](https://scholar.google.com/scholar?q=Hongyu+Hu%2C+Jiyuan+Zhang%2C+Minyi+Zhao%2C+et+al%2E+Ciem%3A+Contrastive+instruction+evaluation+method+for+better+instruction+tuning%2E+In+NeurIPS+2023+Workshop+on+Instruction+Tuning+and+Instruction+Following%2C+202)
19. Lei Huang, Weijiang Yu, Weitao Ma, et al. [A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). arXiv preprint arXiv:2311.05232, 2023.
20. Qidong Huang, Xiaoyi Dong, Pan Zhang, et al. [Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation](https://md.research.elamur.ai/free/en/papers/huang-2023-opera-alleviating-hallucination-multi-modal/). arXiv preprint arXiv:2311.17911, 2023.
21. Jitesh Jain, Jianwei Yang, and Humphrey Shi. Vcoder: Versatile vision encoders for multimodal large language models. arXiv preprint arXiv:2312.14233, 2023. — [arXiv:2312.14233](https://arxiv.org/abs/2312.14233) [find](https://scholar.google.com/scholar?q=Jitesh+Jain%2C+Jianwei+Yang%2C+and+Humphrey+Shi%2E+Vcoder%3A+Versatile+vision+encoders+for+multimodal+large+language+models%2E+arXiv+preprint+arXiv%3A2312%2E14233%2C+2023%2E+%E2%80%94+%5BarXiv%3A2312%2E14233%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
22. Ziwei Ji, Nayeon Lee, Rita Frieske, et al. [Survey of hallucination in natural language generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/). ACM Computing Surveys, 55(12), 2023.
23. Chaoya Jiang, Haiyang Xu, Mengfan Dong, et al. Hallucination augmented contrastive learning for multimodal large language model. arXiv preprint arXiv:2312.06968, 2023. — [arXiv:2312.06968](https://arxiv.org/abs/2312.06968) [find](https://scholar.google.com/scholar?q=Chaoya+Jiang%2C+Haiyang+Xu%2C+Mengfan+Dong%2C+et+al%2E+Hallucination+augmented+contrastive+learning+for+multimodal+large+language+model%2E+arXiv+preprint+arXiv%3A2312%2E06968%2C+2023%2E+%E2%80%94+%5BarXiv%3A2312%2E06968%5D%28https%3A%2F%2Farx)
24. Liqiang Jing, Ruosen Li, Yunmo Chen, et al. Faithscore: Evaluating hallucinations in large vision-language models. arXiv preprint arXiv:2311.01477, 2023. — [arXiv:2311.01477](https://arxiv.org/abs/2311.01477) [find](https://scholar.google.com/scholar?q=Liqiang+Jing%2C+Ruosen+Li%2C+Yunmo+Chen%2C+et+al%2E+Faithscore%3A+Evaluating+hallucinations+in+large+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2311%2E01477%2C+2023%2E+%E2%80%94+%5BarXiv%3A2311%2E01477%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F231)
25. Seongyun Lee, Sue Hyun Park, Yongrae Jo, et al. Volcano: Mitigating multimodal hallucination through self-feedback guided revision. arXiv preprint arXiv:2311.07362, 2023. [find](https://scholar.google.com/scholar?q=Seongyun+Lee%2C+Sue+Hyun+Park%2C+Yongrae+Jo%2C+et+al%2E+Volcano%3A+Mitigating+multimodal+hallucination+through+self%2Dfeedback+guided+revision%2E+arXiv+preprint+arXiv%3A2311%2E07362%2C+2023%2E)
26. Sicong Leng, Hang Zhang, Guanzheng Chen, et al. [Mitigating object hallucinations in large vision-language models through visual contrastive decoding](https://md.research.elamur.ai/free/en/papers/leng-2023-mitigating-object-hallucinations-large-vision/). arXiv preprint arXiv:2311.16922, 2023.
27. Junnan Li, Dongxu Li, Silvio Savarese, et al. [Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). In ICML, 2023.
28. Yifan Li, Yifan Du, Kun Zhou, et al. [Evaluating object hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-object-hallucination-large-vision/). In EMNLP, 2023.
29. Zhang Li, Biao Yang, Qiang Liu, et al. [Monkey: Image resolution and text label are important things for large multi-modal models](https://md.research.elamur.ai/free/en/papers/li-2023-monkey-image-resolution-text-label/). arXiv preprint arXiv:2311.06607, 2023.
30. Bin Lin, Bin Zhu, Yang Ye, et al. [Video-llava: Learning united visual representation by alignment before projection](https://md.research.elamur.ai/free/en/papers/lin-2023-video-llava-learning-united-visual/). arXiv preprint arXiv:2311.10122, 2023.
31. Fuxiao Liu, Kevin Lin, Linjie Li, et al. [Mitigating hallucination in large multi-modal models via robust instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-mitigating-hallucination-large-multi-modal/). arXiv preprint arXiv:2306.14565, 2023.
32. Haotian Liu, Chunyuan Li, Yuheng Li, et al. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/). arXiv preprint arXiv:2310.03744, 2023.
33. Haotian Liu, Chunyuan Li, Qingyang Wu, et al. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). In NeurIPS, 2023.
34. Shilong Liu, Hao Cheng, Haotian Liu, et al. Llava-plus: Learning to use tools for creating multimodal agents. arXiv preprint arXiv:2311.05437, 2023. — [arXiv:2311.05437](https://arxiv.org/abs/2311.05437) [find](https://scholar.google.com/scholar?q=Shilong+Liu%2C+Hao+Cheng%2C+Haotian+Liu%2C+et+al%2E+Llava%2Dplus%3A+Learning+to+use+tools+for+creating+multimodal+agents%2E+arXiv+preprint+arXiv%3A2311%2E05437%2C+2023%2E+%E2%80%94+%5BarXiv%3A2311%2E05437%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2311%2E054)
35. Siqu Long, Feiqi Cao, Soyeon Caren Han, et al. Vision-and-language pretrained models: A survey. In IJCAI, 2022. [find](https://scholar.google.com/scholar?q=Siqu+Long%2C+Feiqi+Cao%2C+Soyeon+Caren+Han%2C+et+al%2E+Vision%2Dand%2Dlanguage+pretrained+models%3A+A+survey%2E+In+IJCAI%2C+2022%2E)
36. Holy Lovenia, Wenliang Dai, Samuel Cahyawijaya, et al. Negative object presence evaluation (nope) to measure object hallucination in vision-language models. arXiv preprint arXiv:2310.05338, 2023. [find](https://scholar.google.com/scholar?q=Holy+Lovenia%2C+Wenliang+Dai%2C+Samuel+Cahyawijaya%2C+et+al%2E+Negative+object+presence+evaluation+%28nope%29+to+measure+object+hallucination+in+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2310%2E05338%2C+2023%2E)
37. Jiasen Lu, Jianwei Yang, Dhruv Batra, et al. Neural baby talk. In CVPR, 2018. [find](https://scholar.google.com/scholar?q=Jiasen+Lu%2C+Jianwei+Yang%2C+Dhruv+Batra%2C+et+al%2E+Neural+baby+talk%2E+In+CVPR%2C+2018%2E)
38. Jiaying Lu, Jinmeng Rao, Kezhen Chen, et al. Evaluation and mitigation of agnosia in multimodal large language models. arXiv preprint arXiv:2309.04041, 2023. — [arXiv:2309.04041](https://arxiv.org/abs/2309.04041) [find](https://scholar.google.com/scholar?q=Jiaying+Lu%2C+Jinmeng+Rao%2C+Kezhen+Chen%2C+et+al%2E+Evaluation+and+mitigation+of+agnosia+in+multimodal+large+language+models%2E+arXiv+preprint+arXiv%3A2309%2E04041%2C+2023%2E+%E2%80%94+%5BarXiv%3A2309%2E04041%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
39. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774, 2023.
40. Alec Radford, Jong Wook Kim, Chris Hallacy, et al. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In ICML, 2021.
41. Rafael Rafailov, Archit Sharma, Eric Mitchell, et al. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). arXiv preprint arXiv:2305.18290, 2023.
42. Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, et al. [Object hallucination in image captioning](https://md.research.elamur.ai/free/en/papers/rohrbach-2018-object-hallucination-image-captioning/). In EMNLP, 2018.
43. Nisan Stiennon, Long Ouyang, Jeffrey Wu, et al. Learning to summarize with human feedback. In NeurIPS, volume 33, 2020. [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeffrey+Wu%2C+et+al%2E+Learning+to+summarize+with+human+feedback%2E+In+NeurIPS%2C+volume+33%2C+2020%2E)
44. Zhiqing Sun, Sheng Shen, Shengcao Cao, et al. [Aligning large multimodal models with factually augmented rlhf](https://md.research.elamur.ai/free/en/papers/sun-2023-aligning-large-multimodal-models-factually/). arXiv preprint arXiv:2309.14525, 2023.
45. Hugo Touvron, Thibaut Lavril, Gautier Izacard, et al. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971, 2023.
46. Hugo Touvron, Louis Martin, Kevin Stone, et al. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288, 2023.
47. Bin Wang, Fan Wu, Xiao Han, et al. Vigc: Visual instruction generation and correction. arXiv preprint arXiv:2308.12714, 2023. — [arXiv:2308.12714](https://arxiv.org/abs/2308.12714) [find](https://scholar.google.com/scholar?q=Bin+Wang%2C+Fan+Wu%2C+Xiao+Han%2C+et+al%2E+Vigc%3A+Visual+instruction+generation+and+correction%2E+arXiv+preprint+arXiv%3A2308%2E12714%2C+2023%2E+%E2%80%94+%5BarXiv%3A2308%2E12714%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2308%2E12714%29)
48. Junyang Wang, Yuhang Wang, Guohai Xu, et al. An llm-free multi-dimensional benchmark for mllms hallucination evaluation. arXiv preprint arXiv:2311.[07397, 2023](https://md.research.elamur.ai/free/en/papers/wang-2023-amber-llm-free-multi-dimensional/).
49. Junyang Wang, Yiyang Zhou, Guohai Xu, et al. Evaluation and analysis of hallucination in large vision-language models. arXiv preprint arXiv:2308.15126, 2023. — [arXiv:2308.15126](https://arxiv.org/abs/2308.15126) [find](https://scholar.google.com/scholar?q=Junyang+Wang%2C+Yiyang+Zhou%2C+Guohai+Xu%2C+et+al%2E+Evaluation+and+analysis+of+hallucination+in+large+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2308%2E15126%2C+2023%2E+%E2%80%94+%5BarXiv%3A2308%2E15126%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
50. Shukang Yin, Chaoyou Fu, Sirui Zhao, et al. [A survey on multimodal large language models](https://md.research.elamur.ai/free/en/papers/yin-2023-survey-multimodal-large-language-models/). arXiv preprint arXiv:2306.13549, 2023.
51. Shukang Yin, Chaoyou Fu, Sirui Zhao, et al. [Woodpecker: Hallucination correction for multimodal large language models](https://md.research.elamur.ai/free/en/papers/yin-2023-woodpecker-hallucination-correction-multimodal-large/). arXiv preprint arXiv:2310.16045, 2023.
52. Haoxuan You, Haotian Zhang, Zhe Gan, et al. Ferret: Refer and ground anything anywhere at any granularity. arXiv preprint arXiv:2310.07704, 2023. — [arXiv:2310.07704](https://arxiv.org/abs/2310.07704) [find](https://scholar.google.com/scholar?q=Haoxuan+You%2C+Haotian+Zhang%2C+Zhe+Gan%2C+et+al%2E+Ferret%3A+Refer+and+ground+anything+anywhere+at+any+granularity%2E+arXiv+preprint+arXiv%3A2310%2E07704%2C+2023%2E+%E2%80%94+%5BarXiv%3A2310%2E07704%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E07704%29)
53. Tianyu Yu, Yuan Yao, Haoye Zhang, et al. [Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback](https://md.research.elamur.ai/free/en/papers/yu-2023-rlhf-v-towards-trustworthy-mllms/). arXiv preprint arXiv:2312.00849, 2023.
54. Bohan Zhai, Shijia Yang, Xiangchen Zhao, et al. Halle-switch: Rethinking and controlling object existence hallucinations in large vision language models for detailed caption. arXiv preprint arXiv:2310.01779, 2023. — [arXiv:2310.01779](https://arxiv.org/abs/2310.01779) [find](https://scholar.google.com/scholar?q=Bohan+Zhai%2C+Shijia+Yang%2C+Xiangchen+Zhao%2C+et+al%2E+Halle%2Dswitch%3A+Rethinking+and+controlling+object+existence+hallucinations+in+large+vision+language+models+for+detailed+caption%2E+arXiv+preprint+arXiv%3A2310)
55. Youcai Zhang, Xinyu Huang, Jinyu Ma, et al. Recognize anything: A strong image tagging model. arXiv preprint arXiv:2306.03514, 2023. — [arXiv:2306.03514](https://arxiv.org/abs/2306.03514) [find](https://scholar.google.com/scholar?q=Youcai+Zhang%2C+Xinyu+Huang%2C+Jinyu+Ma%2C+et+al%2E+Recognize+anything%3A+A+strong+image+tagging+model%2E+arXiv+preprint+arXiv%3A2306%2E03514%2C+2023%2E+%E2%80%94+%5BarXiv%3A2306%2E03514%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2306%2E03514%29)
56. Yue Zhang, Yafu Li, Leyang Cui, et al. Siren’s song in the ai ocean: A survey on hallucination in large language models. arXiv preprint arXiv:2309.[01219, 2023](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/).
57. Yongqiang Zhao, Zhenyu Li, Zhi Jin, et al. Enhancing the spatial awareness capability of multi-modal large language model. arXiv preprint arXiv:2310.20357, 2023. — [arXiv:2310.20357](https://arxiv.org/abs/2310.20357) [find](https://scholar.google.com/scholar?q=Yongqiang+Zhao%2C+Zhenyu+Li%2C+Zhi+Jin%2C+et+al%2E+Enhancing+the+spatial+awareness+capability+of+multi%2Dmodal+large+language+model%2E+arXiv+preprint+arXiv%3A2310%2E20357%2C+2023%2E+%E2%80%94+%5BarXiv%3A2310%2E20357%5D%28https%3A%2F%2Farxiv%2Eorg)
58. Zhiyuan Zhao, Bin Wang, Linke Ouyang, et al. Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization. arXiv preprint arXiv:2311.16839, 2023. — [arXiv:2311.16839](https://arxiv.org/abs/2311.16839) [find](https://scholar.google.com/scholar?q=Zhiyuan+Zhao%2C+Bin+Wang%2C+Linke+Ouyang%2C+et+al%2E+Beyond+hallucinations%3A+Enhancing+lvlms+through+hallucination%2Daware+direct+preference+optimization%2E+arXiv+preprint+arXiv%3A2311%2E16839%2C+2023%2E+%E2%80%94+%5BarXiv%3A2311%2E168)
59. Yiyang Zhou, Chenhang Cui, Jaehong Yoon, et al. [Analyzing and mitigating object hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-analyzing-mitigating-object-hallucination-large/). In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following, 2023.
60. Deyao Zhu, Jun Chen, Xiaoqian Shen, et al. [Minigpt-4: Enhancing vision-language understanding with advanced large language models](https://md.research.elamur.ai/free/en/papers/zhu-2023-minigpt-4-enhancing-vision-language/). arXiv preprint arXiv:2304.10592, 2023.

*Updated 2 November 2026*