---
title: "What Makes for Good Visual Instructions? Synthesizing Complex"
description: "The paper asks which properties of visual instruction data actually help Multi-modal Large Language… 35 references, cited by 9 papers here."
authors: ["Yifan Du", "Hangyu Guo", "Kun Zhou", "Wayne Xin Zhao", "Jinpeng Wang", "Chuyuan Wang", "Mingchen Cai", "Ruihua Song", "Ji-Rong Wen"]
year: 2025
venue: "Proceedings of the 31st International Conference on Computational Linguistics"
source: "https://aclanthology.org/2025.coling-main.546/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/du-2025-what-makes-good-visual-instructions/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/du-2025-what-makes-good-visual-instructions/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/7/) / What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning

# What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning

**Authors:** [Yifan Du](https://md.research.elamur.ai/free/en/authors/yifan-du/), [Hangyu Guo](https://md.research.elamur.ai/free/en/authors/hangyu-guo/), [Kun Zhou](https://md.research.elamur.ai/free/en/authors/kun-zhou/), [Wayne Xin Zhao](https://md.research.elamur.ai/free/en/authors/wayne-xin-zhao/), [Jinpeng Wang](https://md.research.elamur.ai/free/en/authors/jinpeng-wang/), Chuyuan Wang, Mingchen Cai, [Ruihua Song](https://md.research.elamur.ai/free/en/authors/ruihua-song/) and [Ji-Rong Wen](https://md.research.elamur.ai/free/en/authors/ji-rong-wen/) · **Year:** 2025 · **Venue:** [Proceedings of the 31st International Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2025](https://md.research.elamur.ai/free/en/venues/coling/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.coling-main.546/) · [PDF](https://aclanthology.org/2025.coling-main.546.pdf) · [OpenAlex](https://openalex.org/W4388444960)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/du-2025-what-makes-good-visual-instructions/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks which properties of visual instruction data actually help Multi-modal Large Language Models (MLLMs) generalize, since earlier collections had never been compared under the same backbone and training setup. The authors fine-tuned BLIP-2 and MiniGPT-4 on six existing instruction sets and evaluated them on MME and SEED-Bench. Reasoning-oriented data proved more useful than captioning or plain VQA. Raising instruction complexity mattered more than adding task diversity or fine-grained spatial annotations. Benchmark scores rose roughly linearly with dataset complexity.

Building on this, they designed a GPT-4 pipeline called synthesize-complicate-reformulate. It first generates reasoning instructions. It then makes them harder using image content and outside knowledge, checking correctness repeatedly. Finally, it converts them into several answer formats. The result is ComVint, a 32K-example dataset. Fine-tuning four MLLMs on ComVint improved all of them and beat existing instruction collections. This shows that a small but complex dataset can outperform much larger, simpler ones.

## Key points
- The empirical study compared LLaVA-Cap, LLaVA-VQA, LLaVA-Rea, LRV, Shikra-RD and A-OKVQA. For MiniGPT-4, the average score went from 27.01 to 33.48 with LLaVA-Rea and 34.08 with A-OKVQA, but fell to 18.27 with LRV.
- Complexity was measured as the mean number of reasoning steps per instruction, counted by gpt-3.5-turbo-0125. ComVint had the highest complexity score and the best results.
- ComVint (32K examples) was used to fine-tune BLIP-2, LLaVA, MiniGPT-4 and InstructBLIP. LLaVA improved by 27.86% on MME-Perception and 27.60% on MME-Cognition.
- In ablations, skipping the complication stage caused clear drops on all benchmarks. Removing either cross-modal or outside-knowledge reasoning lowered average performance, and so did removing reformulation.
- Code and data are released at github.com/RUCAIBox/ComVint.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Renmin University of China](https://md.research.elamur.ai/free/en/universities/renmin-university-of-china/) — University, Beijing, China · 114 works · 16,118 citations
- [Meituan](https://md.research.elamur.ai/free/en/companies/meituan/) — Company, Beijing, China · 17 works · 3,769 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Yifan Du, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Jinpeng Wang, Chuyuan Wang, Mingchen Cai, Ruihua Song and Ji-Rong Wen. 2025. What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{du2025what,
  title = {{What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning}},
  author = {Yifan Du, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Jinpeng Wang, Chuyuan Wang, Mingchen Cai, Ruihua Song and Ji-Rong Wen},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.546/}
}
```

- [← previous What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations](https://md.research.elamur.ai/free/en/papers/liu-2025-what-is-that-talk-about/)
- [next → A Multi-Dialectal Dataset for German Dialect ASR and Dialect-to-Standard Speech Translation](https://md.research.elamur.ai/free/en/papers/blaschke-2025-multi-dialectal-dataset-german-dialect/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/7/)

## References

35 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… [find](https://scholar.google.com/scholar?q=Tom+B%2E+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretche)
2. Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
3. Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. 2021. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 3558–3568. Computer Vision… [find](https://scholar.google.com/scholar?q=Soravit+Changpinyo%2C+Piyush+Sharma%2C+Nan+Ding%2C+and+Radu+Soricut%2E+2021%2E+Conceptual+12m%3A+Pushing+web%2Dscale+image%2Dtext+pre%2Dtraining+to+recognize+long%2Dtail+visual+concepts%2E+In+IEEE+Conference+on+Computer+Vi)
4. Chi Chen, Ruoyu Qin, Fuwen Luo, Xiaoyue Mi, Peng Li, Maosong Sun, and Yang Liu. 2023a. Position-enhanced visual instruction tuning for multimodal large language models. CoRR, abs/2308.13437. — [arXiv:2308.13437](https://arxiv.org/abs/2308.13437) [find](https://scholar.google.com/scholar?q=Chi+Chen%2C+Ruoyu+Qin%2C+Fuwen+Luo%2C+Xiaoyue+Mi%2C+Peng+Li%2C+Maosong+Sun%2C+and+Yang+Liu%2E+2023a%2E+Position%2Denhanced+visual+instruction+tuning+for+multimodal+large+language+models%2E+CoRR%2C+abs%2F2308%2E13437%2E+%E2%80%94+%5BarXiv%3A)
5. Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. 2023b. [Shikra: Unleashing multimodal llm’s referential dialogue magic](https://md.research.elamur.ai/free/en/papers/chen-2023-shikra-unleashing-multimodal-llm-s/). CoRR, abs/2306.15195.
6. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang… [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Eric+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+Albert+Webson%2C+Shixiang+Shane+Gu%2C+Zhuyun+Dai%2C+Mirac+Suzgun%2C+Xiny)
7. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. 2023. [Instructblip: Towards general-purpose vision-language models with instruction tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/). CoRR, abs/2305.06500.
8. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and Rongrong Ji. 2023. [MME: A comprehensive evaluation benchmark for multimodal large language models](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/). CoRR, abs/2306.13394.
9. Drew A. Hudson and Christopher D. Manning. 2019. [GQA: A new dataset for real-world visual reasoning and compositional question answering](https://md.research.elamur.ai/free/en/papers/hudson-2019-gqa-new-dataset-real-world/). In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 6700–6709. Computer Vision Foundation / IEE…
10. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. 2017. [Visual genome: Connecting language and vision using crowdsourced dense image annotations](https://md.research.elamur.ai/free/en/papers/krishna-2016-visual-genome-connecting-language-vision/). Int. J. Comput. Vis…
11. Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2023a. MIMIC-IT: multi-modal in-context instruction tuning. CoRR, abs/2306.05425. — [arXiv:2306.05425](https://arxiv.org/abs/2306.05425) [find](https://scholar.google.com/scholar?q=Bo+Li%2C+Yuanhan+Zhang%2C+Liangyu+Chen%2C+Jinghao+Wang%2C+Fanyi+Pu%2C+Jingkang+Yang%2C+Chunyuan+Li%2C+and+Ziwei+Liu%2E+2023a%2E+MIMIC%2DIT%3A+multi%2Dmodal+in%2Dcontext+instruction+tuning%2E+CoRR%2C+abs%2F2306%2E05425%2E+%E2%80%94+%5BarXiv%3A2306%2E0)
12. Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. 2023b. [Seed-bench: Benchmarking multimodal llms with generative comprehension](https://md.research.elamur.ai/free/en/papers/li-2023-seed-bench-benchmarking-multimodal-llms/). CoRR, abs/2307.16125.
13. Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, and Jianfeng Gao. 2023c. Multimodal foundation models: From specialists to general-purpose assistants. CoRR, abs/2309.10020. — [arXiv:2309.10020](https://arxiv.org/abs/2309.10020) [find](https://scholar.google.com/scholar?q=Chunyuan+Li%2C+Zhe+Gan%2C+Zhengyuan+Yang%2C+Jianwei+Yang%2C+Linjie+Li%2C+Lijuan+Wang%2C+and+Jianfeng+Gao%2E+2023c%2E+Multimodal+foundation+models%3A+From+specialists+to+general%2Dpurpose+assistants%2E+CoRR%2C+abs%2F2309%2E10020%2E)
14. Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. 2023d. [BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of…
15. Lei Li, Yuwei Yin, Shicheng Li, Liang Chen, Peiyi Wang, Shuhuai Ren, Mukai Li, Yazheng Yang, Jingjing Xu, Xu Sun, Lingpeng Kong, and Qi Liu. 2023e. M3 it: A large-scale dataset towards multimodal multilingual instruction tuning. CoRR, abs/2306.04387. — [arXiv:2306.04387](https://arxiv.org/abs/2306.04387) [find](https://scholar.google.com/scholar?q=Lei+Li%2C+Yuwei+Yin%2C+Shicheng+Li%2C+Liang+Chen%2C+Peiyi+Wang%2C+Shuhuai+Ren%2C+Mukai+Li%2C+Yazheng+Yang%2C+Jingjing+Xu%2C+Xu+Sun%2C+Lingpeng+Kong%2C+and+Qi+Liu%2E+2023e%2E+M3+it%3A+A+large%2Dscale+dataset+towards+multimodal+mult)
16. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023f. [Evaluating object hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-object-hallucination-large-vision/). CoRR, abs/2305.10355.
17. Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. 2023a. [Aligning large multi-modal model with robust instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-mitigating-hallucination-large-multi-modal/). CoRR, abs/2306.14565.
18. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023b. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). CoRR, abs/2304.08485.
19. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. [The flan collection: Designing data and methods for effective instruction tuning](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/). In International Conference on Machine Learning, ICML 2023, 23-29 July 2023…
20. Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. 2019. OK-VQA: A visual question answering benchmark requiring external knowledge. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 3195–3204. Computer Vision F… [find](https://scholar.google.com/scholar?q=Kenneth+Marino%2C+Mohammad+Rastegari%2C+Ali+Farhadi%2C+and+Roozbeh+Mottaghi%2E+2019%2E+OK%2DVQA%3A+A+visual+question+answering+benchmark+requiring+external+knowledge%2E+In+IEEE+Conference+on+Computer+Vision+and+Patte)
21. OpenAI. 2023. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). abs/2303.08774.
22. CoRR, Bryan A. Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. 2015. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago,… [find](https://scholar.google.com/scholar?q=CoRR%2C+Bryan+A%2E+Plummer%2C+Liwei+Wang%2C+Chris+M%2E+Cervantes%2C+Juan+C%2E+Caicedo%2C+Julia+Hockenmaier%2C+and+Svetlana+Lazebnik%2E+2015%2E+Flickr30k+entities%3A+Collecting+region%2Dto%2Dphrase+correspondences+for+richer+imag)
23. Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. 2021. [LAION-400M: open dataset of clip-filtered 400 million image-text pairs](https://md.research.elamur.ai/free/en/papers/schuhmann-2021-laion-400m-open-dataset-clip/). CoRR, abs/2111.02114.
24. Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. [A-OKVQA: A benchmark for visual question answering using world knowledge](https://md.research.elamur.ai/free/en/papers/schwenk-2022-okvqa-benchmark-visual-question-answering/). In Computer Vision -ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part VIII, volume…
25. Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. 2023. [How far can camels go? exploring the state of instruction tuning on open resources](https://md.research.elamur.ai/free/en/papers/wang-2023-far-camels-go-exploring-state/). CoRR, abs/2306.04751.
26. Dahua Lin, and Jiaqi Wang. 2023a. [Internlm-xcomposer: A vision-language large model for advanced text-image comprehension and composition](https://md.research.elamur.ai/free/en/papers/zhang-2023-internlm-xcomposer-vision-language/). CoRR, abs/2309.15112.
27. Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuangrui Ding, Songyang Zhang, Haodong Duan, Wenwei Zhang, Hang Yan, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang. 2023b. [Internlm-xcomposer: A vision-la…](https://md.research.elamur.ai/free/en/papers/zhang-2023-internlm-xcomposer-vision-language/)
28. Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Kai Chen, and Ping Luo. 2023c. Gpt4roi: Instruction tuning large language model on region-of-interest. CoRR. [find](https://scholar.google.com/scholar?q=Shilong+Zhang%2C+Peize+Sun%2C+Shoufa+Chen%2C+Min+Xiao%2C+Wenqi+Shao%2C+Wenwei+Zhang%2C+Kai+Chen%2C+and+Ping+Luo%2E+2023c%2E+Gpt4roi%3A+Instruction+tuning+large+language+model+on+region%2Dof%2Dinterest%2E+CoRR%2E)
29. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. [A survey of l…](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/)
30. Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. [Minigpt-4: Enhancing vision-language understanding with advanced large language models](https://md.research.elamur.ai/free/en/papers/zhu-2023-minigpt-4-enhancing-vision-language/). CoRR, abs/2304.10592.
31. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview…
32. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). arXiv preprint arXiv:2304.12244, abs/2304.12244.
33. Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qian Qi, Ji Zhang, and Fei Huang. 2023. [mplug-owl: Modularization empowers large language models with multimodality](https://md.research.elamur.ai/free/en/papers/ye-2023-mplug-owl-modularization-empowers-large/). CoRR, abs/23…
34. Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. [A survey on multimodal large language models](https://md.research.elamur.ai/free/en/papers/yin-2023-survey-multimodal-large-language-models/). CoRR, abs/2306.13549.
35. Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuangrui Ding, Songyang Zhang, Haodong Duan, Wenwei Zhang, Hang Yan, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, [find](https://scholar.google.com/scholar?q=Pan+Zhang%2C+Xiaoyi+Dong%2C+Bin+Wang%2C+Yuhang+Cao%2C+Chao+Xu%2C+Linke+Ouyang%2C+Zhiyuan+Zhao%2C+Shuangrui+Ding%2C+Songyang+Zhang%2C+Haodong+Duan%2C+Wenwei+Zhang%2C+Hang+Yan%2C+Xinyue+Zhang%2C+Wei+Li%2C+Jingwen+Li%2C+Kai+Chen%2C+Con)

*Updated 30 October 2026*