---
title: "Centurio — paper by Geigle et al. (2025) — summary and ideas"
description: "The work offers practical, low-cost guidance for building multilingual multimodal systems, including for… 126 references, cited by 9 papers here."
authors: ["Gregor Geigle", "Florian Schneider", "Carolin Holtermann", "Chris Biemann", "Radu Timofte", "Anne Lauscher", "Goran Glavaš"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.143/"
doi: "10.18653/v1/2025.acl-long.143"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/geigle-2025-centurio-drivers-multilingual-ability-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/geigle-2025-centurio-drivers-multilingual-ability-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/) / Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model

# Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model

**Authors:** [Gregor Geigle](https://md.research.elamur.ai/free/en/authors/gregor-geigle/), [Florian Schneider](https://md.research.elamur.ai/free/en/authors/florian-schneider/), [Carolin Holtermann](https://md.research.elamur.ai/free/en/authors/carolin-holtermann/), [Chris Biemann](https://md.research.elamur.ai/free/en/authors/chris-biemann/), Radu Timofte, [Anne Lauscher](https://md.research.elamur.ai/free/en/authors/anne-lauscher/) and [Goran Glavaš](https://md.research.elamur.ai/free/en/authors/goran-glavas/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.143/) · [PDF](https://aclanthology.org/2025.acl-long.143.pdf) · [DOI 10.18653/v1/2025.acl-long.143](https://doi.org/10.18653/v1/2025.acl-long.143) · [OpenAlex](https://openalex.org/W4412886778)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/geigle-2025-centurio-drivers-multilingual-ability-large/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most large vision-language models (LVLMs) are trained largely on English. As a result, they often misread non-English instructions, struggle with foreign-language text inside images, and answer in the wrong language. Geigle et al. ask how to compose training data across languages under a fixed budget. They examine four questions: how many languages to include, what language mix to use for instruction tuning and for pre-training, and how to strengthen multilingual text-in-image understanding.

Their experimental model follows the LLaVA design, combining a SigLIP SO400 image encoder with Phi 3.5. Training data comes from English datasets machine-translated with NLLB. They evaluate on 13 tasks in 43 languages, grouped by resource tier.

Key findings:
- Adding up to 100 languages costs almost nothing in performance.
- A non-English share of only 25–50% is sufficient.
- Synthetic multilingual OCR data clearly helps Latin-script languages, but much less for other scripts.

Applying these lessons, the authors train Centurio, a 100-language LVLM. On high-resource languages it is competitive with strong open models, and on lower-resource languages it outperforms them. The work offers practical, low-cost guidance for building multilingual multimodal systems, including for languages such as Ukrainian.

## Key points
- **Analysis setup:** LLaVA-style model with SigLIP SO400 and Phi 3.5 (3.8B), with a partial replication on Llama 3 (8B). Data was translated with NLLB (nllb-200-distilled-1.3B): 0.77M LLaVA-Next instruction samples and 1.3M ShareGPT4v dense captions.
- **No clear "curse of multilinguality":** scaling from 7 to 100 training languages substantially helps the newly added languages, especially language fidelity. Earlier languages keep roughly the same performance.
- **Language mix:** making 25–50% of instruction data non-English works best, and more can even hurt. In pre-training, a larger multilingual share helps up to about 50%, then plateaus.
- **New benchmark:** SMPQA (Synthetic Multilingual Plot Question Answering) tests multilingual OCR across 11 languages and 7 scripts. About 5k synthetic OCR samples per language mainly improve Latin-script languages.
- **Centurio results:** evaluated on 14 tasks and 56 languages against 13 other LVLMs, including Qwen2-VL, InternVL 2.5, and Pangea. It is state-of-the-art on low(er)-resource languages.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Universität Hamburg](https://md.research.elamur.ai/free/en/universities/universitat-hamburg/) — University, Hamburg, Germany · 79 works · 5,422 citations
- [University of Würzburg](https://md.research.elamur.ai/free/en/universities/university-of-wurzburg/) — University, Würzburg, Germany · 8 works · 60 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Gregor Geigle, Florian Schneider, Carolin Holtermann, Chris Biemann, Radu Timofte, Anne Lauscher and Goran Glavaš. 2025. Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{geigle2025centurio,
  title = {{Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model}},
  author = {Gregor Geigle, Florian Schneider, Carolin Holtermann, Chris Biemann, Radu Timofte, Anne Lauscher and Goran Glavaš},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.143},
  url = {https://doi.org/10.18653/v1/2025.acl-long.143}
}
```

- [← previous Can Many-Shot In-Context Learning Help LLMs as Evaluators? A Preliminary Empirical Study](https://md.research.elamur.ai/free/en/papers/song-2025-can-many-shot-context-learning/)
- [next → ChatBench: From Static Benchmarks to Human-AI Evaluation](https://md.research.elamur.ai/free/en/papers/chang-2025-chatbench-static-benchmarks-human-ai/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/)

## References

126 references, 59 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024a. [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/). arXiv preprint arXiv:2404.14219.
2. Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Caio César Teodoro Mendes, Weizhu Chen, Vishrav Chaudhary, Parul Chopra, Alli… — [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/)
3. Manoj Acharya, Kushal Kafle, and Christopher Kanan. 2019. TallyQA: Answering Complex Counting Questions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 8076–8084. [find](https://scholar.google.com/scholar?q=Manoj+Acharya%2C+Kushal+Kafle%2C+and+Christopher+Kanan%2E+2019%2E+TallyQA%3A+Answering+Complex+Counting+Questions%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+33%2C+pages+8076%E2%80%938084%2E)
4. Željko Agić and Natalie Schluter. 2018. Baselines and Test Data for Cross-Lingual Inference. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pages 3890–3894, Miyazaki, Japan. [find](https://scholar.google.com/scholar?q=%C5%BDeljko+Agic%CC%81+and+Natalie+Schluter%2E+2018%2E+Baselines+and+Test+Data+for+Cross%2DLingual+Inference%2E+In+Proceedings+of+the+Eleventh+International+Conference+on+Language+Resources+and+Evaluation+%28LREC+2018%29%2C+)
5. Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, et al. 2024. Pixtral 12B. arXiv preprint arXiv:2410.07073. [find](https://scholar.google.com/scholar?q=Pravesh+Agrawal%2C+Szymon+Antoniak%2C+Emma+Bou+Hanna%2C+Baptiste+Bout%2C+Devendra+Chaplot%2C+Jessica+Chudnovsky%2C+Diogo+Costa%2C+Baudouin+De+Monicault%2C+Saurabh+Garg%2C+Theophile+Gervet%2C+et+al%2E+2024%2E+Pixtral+12B%2E+arX)
6. Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Mohamed Ahmed, Kalika Bali, and Sunayana Sitaram. 2024. [MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks](https://md.research.elamur.ai/free/en/papers/ahuja-2024-megaverse-benchmarking-large-language-models/). In Proceedings of the 2024…
7. AI@Meta. 2024. Llama 3 Model Card. [find](https://scholar.google.com/scholar?q=AI%40Meta%2E+2024%2E+Llama+3+Model+Card%2E)
8. Nahid Alam, Karthik Reddy Kanjula, Surya Guthikonda, Timothy Chung, Bala Krishna S Vegesna, Abhipsha Das, Anthony Susevski, Ryan Sze-Yin Chan, SM Uddin, Shayekh Bin Islam, et al. 2024. Maya: An Instruction Finetuned Multilingual Multimodal Model. arXiv preprint arXiv:2412.07112. — [arXiv:2412.07112](https://arxiv.org/abs/2412.07112) [find](https://scholar.google.com/scholar?q=Nahid+Alam%2C+Karthik+Reddy+Kanjula%2C+Surya+Guthikonda%2C+Timothy+Chung%2C+Bala+Krishna+S+Vegesna%2C+Abhipsha+Das%2C+Anthony+Susevski%2C+Ryan+Sze%2DYin+Chan%2C+SM+Uddin%2C+Shayekh+Bin+Islam%2C+et+al%2E+2024%2E+Maya%3A+An+Instru)
9. Viraat Aryabumi, John Dang, Dwarak Talupuru, Saurabh Dash, David Cairuz, Hangyu Lin, Bharat Venkitesh, Madeline Smith, Jon Ander Campos, Yi Chern Tan, Kelly Marchisio, Max Bartolo, Sebastian Ruder, Acyr Locatelli, Julia Kreutzer, Nick Frosst, Aidan N. Gomez, Phil Blunsom, Marzieh Fadaee, Ahmet Üstün… — [Aya 23: Open Weight Releases to Further Multilingual Progress](https://md.research.elamur.ai/free/en/papers/aryabumi-2024-aya-23-open-weight-releases/)
10. Jonas Belouadi, Anne Lauscher, and Steffen Eger. 2024. [AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ](https://md.research.elamur.ai/free/en/papers/belouadi-2023-automatikz-text-guided-synthesis-scientific/). In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria.
11. Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Koppula, Fangyu Liu, Adam Grycner, Alexey A. Gritsenko, Neil Houlsby, Manoj Kumar,… [find](https://scholar.google.com/scholar?q=Lucas+Beyer%2C+Andreas+Steiner%2C+Andr%C3%A9+Susano+Pinto%2C+Alexander+Kolesnikov%2C+Xiao+Wang%2C+Daniel+Salz%2C+Maxim+Neumann%2C+Ibrahim+Alabdulmohsin%2C+Michael+Tschannen%2C+Emanuele+Bugliarello%2C+Thomas+Unterthiner%2C+Danie)
12. Ali Furkan Biten, Rubèn Tito, Andrés Mafla, Lluís Gómez i Bigorda, Marçal Rusiñol, C. V. Jawahar, Ernest Valveny, and Dimosthenis Karatzas. 2019. [find](https://scholar.google.com/scholar?q=Ali+Furkan+Biten%2C+Rub%C3%A8n+Tito%2C+Andr%C3%A9s+Mafla%2C+Llu%C3%ADs+G%C3%B3mez+i+Bigorda%2C+Mar%C3%A7al+Rusi%C3%B1ol%2C+C%2E+V%2E+Jawahar%2C+Ernest+Valveny%2C+and+Dimosthenis+Karatzas%2E+2019%2E)
13. Scene Text Visual Question Answering. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, pages 4290–4300, Seoul, Korea (South). [find](https://scholar.google.com/scholar?q=Scene+Text+Visual+Question+Answering%2E+In+2019+IEEE%2FCVF+International+Conference+on+Computer+Vision%2C+ICCV+2019%2C+pages+4290%E2%80%934300%2C+Seoul%2C+Korea+%28South%29%2E)
14. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. [A large annotated corpus for learning natural language inference](https://md.research.elamur.ai/free/en/papers/bowman-2015-large-annotated-corpus-learning-natural/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal.
15. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901, Virtual.
16. Emanuele Bugliarello, Fangyu Liu, Jonas Pfeiffer, Siva Reddy, Desmond Elliott, Edoardo Maria Ponti, and Ivan Vulic. 2022. IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages. In International Conference on Machine Learning, pages 2370–2392, Baltimore, MD, USA. PMLR. [find](https://scholar.google.com/scholar?q=Emanuele+Bugliarello%2C+Fangyu+Liu%2C+Jonas+Pfeiffer%2C+Siva+Reddy%2C+Desmond+Elliott%2C+Edoardo+Maria+Ponti%2C+and+Ivan+Vulic%2E+2022%2E+IGLUE%3A+A+Benchmark+for+Transfer+Learning+across+Modalities%2C+Tasks%2C+and+Languag)
17. Soravit Changpinyo, Linting Xue, Michal Yarom, Ashish Thapliyal, Idan Szpektor, Julien Amelot, Xi Chen, and Radu Soricut. 2023. MaXM: Towards Multilingual Visual Question Answering. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2667–2682, Singapore. [find](https://scholar.google.com/scholar?q=Soravit+Changpinyo%2C+Linting+Xue%2C+Michal+Yarom%2C+Ashish+Thapliyal%2C+Idan+Szpektor%2C+Julien+Amelot%2C+Xi+Chen%2C+and+Radu+Soricut%2E+2023%2E+MaXM%3A+Towards+Multilingual+Visual+Question+Answering%2E+In+Findings+of+the)
18. Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, and Benyou Wang. 2024a. [ALLaVA: Harnessing GPT4V-synthesized Data for A Lite Vision-Language Model](https://md.research.elamur.ai/free/en/papers/chen-2024-allava-harnessing-gpt4v-synthesized-data/). arXiv preprint arXiv:2402.11684.
19. Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. 2024b. [ShareGPT4V: Improving Large Multi-modal Models with Better Captions](https://md.research.elamur.ai/free/en/papers/chen-2023-sharegpt4v-improving-large-multi-modal/). In European Conference on Computer Vision, pages 370–387, Milan, Italy. Springer.
20. Pinzhen Chen, Shaoxiong Ji, Nikolay Bogoychev, Andrey Kutuzov, Barry Haddow, and Kenneth Heafield. 2024c. Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca. In Findings of the Association for Computational Linguistics: EACL 2024, pages 1347–1356, St. Julian’s, Malta. [find](https://scholar.google.com/scholar?q=Pinzhen+Chen%2C+Shaoxiong+Ji%2C+Nikolay+Bogoychev%2C+Andrey+Kutuzov%2C+Barry+Haddow%2C+and+Kenneth+Heafield%2E+2024c%2E+Monolingual+or+Multilingual+Instruction+Tuning%3A+Which+Makes+a+Better+Alpaca%2E+In+Findings+of+th)
21. Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Carlos Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, Siamak Shakeri, Mostafa Dehghani, Daniel Salz, Mario Lucic, Michael Tschannen, Arsha Nagrani, Hexiang Hu, Mandar Joshi, Bo Pang, Ceslee Montgomery, Paul… [find](https://scholar.google.com/scholar?q=Xi+Chen%2C+Josip+Djolonga%2C+Piotr+Padlewski%2C+Basil+Mustafa%2C+Soravit+Changpinyo%2C+Jialin+Wu%2C+Carlos+Riquelme+Ruiz%2C+Sebastian+Goodman%2C+Xiao+Wang%2C+Yi+Tay%2C+Siamak+Shakeri%2C+Mostafa+Dehghani%2C+Daniel+Salz%2C+Mario)
22. Xi Chen, Xiao Wang, Soravit Changpinyo, A. J. Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong, Hassan Akbari, Gaurav Mishra, Linting Xue, Ashish Thapliyal, James Bradbury, Weicheng Ku… — [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://md.research.elamur.ai/free/en/papers/chen-2022-pali-jointly-scaled-multilingual-language/)
23. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang, Wenqi Shao, Pei… — [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://md.research.elamur.ai/free/en/papers/chen-2024-expanding-performance-boundaries-open/)
24. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. 2024e. [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://md.research.elamur.ai/free/en/papers/chen-2024-expanding-performance-boundaries-open/). arXiv preprint arXiv:2412.05271.
25. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan R. Routledge, and William Yang Wang. 2021. [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://md.research.elamur.ai/free/en/papers/chen-2021-finqa-dataset-numerical-reasoning-over/). In Proceedings of the 2021 Conference on Empirical Methods in Natu…
26. Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. 2022. HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Zhoujun+Cheng%2C+Haoyu+Dong%2C+Zhiruo+Wang%2C+Ran+Jia%2C+Jiaqi+Guo%2C+Yan+Gao%2C+Shi+Han%2C+Jian%2DGuang+Lou%2C+and+Dongmei+Zhang%2E+2022%2E+HiTab%3A+A+Hierarchical+Table+Dataset+for+Question+Answering+and+Natural+Language+G)
27. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of the 58th Annual Meeting of the Association fo… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+Cross%2Dlingu)
28. Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loïc Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semar… [find](https://scholar.google.com/scholar?q=Marta+R%2E+Costa%2Djuss%C3%A0%2C+James+Cross%2C+Onur+%C3%87elebi%2C+Maha+Elbayad%2C+Kenneth+Heafield%2C+Kevin+Heffernan%2C+Elahe+Kalbassi%2C+Janice+Lam%2C+Daniel+Licht%2C+Jean+Maillard%2C+Anna+Sun%2C+Skyler+Wang%2C+Guillaume+Wenzek%2C+Al+Yo)
29. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. 2023. [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/). In Proceedings of the 37th International Conference on Neural Information…
30. John Dang, Shivalika Singh, Daniel D’souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, Sandra Kublik, Meor Amer, Viraat Aryabumi, Jon Ander Campos, Yi-Chern Tan, Tom Kocmi, Florian Strub, Nathan Grinsztajn, Yannis Flet-Berliac,… — [Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier](https://md.research.elamur.ai/free/en/papers/dang-2024-aya-expanse-combining-research-breakthroughs/)
31. J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, Miami, FL, USA. [find](https://scholar.google.com/scholar?q=J%2E+Deng%2C+W%2E+Dong%2C+R%2E+Socher%2C+L%2E+Li%2C+Kai+Li%2C+and+Li+Fei%2DFei%2E+2009%2E+ImageNet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+2009+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pages+248%E2%80%93255)
32. Peter Devine. 2024. Tagengo: A Multilingual Chat Dataset. In Proceedings of the Fourth Workshop on Multilingual Representation Learning (MRL 2024), pages 106–113, Miami, FL, USA. [find](https://scholar.google.com/scholar?q=Peter+Devine%2E+2024%2E+Tagengo%3A+A+Multilingual+Chat+Dataset%2E+In+Proceedings+of+the+Fourth+Workshop+on+Multilingual+Representation+Learning+%28MRL+2024%29%2C+pages+106%E2%80%93113%2C+Miami%2C+FL%2C+USA%2E)
33. Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, and Lingpeng Kong. 2023. G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model. In The Thirteenth International Conference on Learning Representations, Singa… [find](https://scholar.google.com/scholar?q=Jiahui+Gao%2C+Renjie+Pi%2C+Jipeng+Zhang%2C+Jiacheng+Ye%2C+Wanjun+Zhong%2C+Yufei+Wang%2C+Lanqing+Hong%2C+Jianhua+Han%2C+Hang+Xu%2C+Zhenguo+Li%2C+and+Lingpeng+Kong%2E+2023%2E+G%2DLLaVA%3A+Solving+Geometric+Problem+with+Multi%2DModal)
34. William Gaviria Rojas, Sudnya Diamos, Keertan Kini, David Kanter, Vijay Janapa Reddi, and Cody Coleman. 2022. The Dollar Street Dataset: Images Representing the Geographic and Socioeconomic Diversity of the World. In Advances in Neural Information Processing Systems, volume 35, pages 12979–12990. [find](https://scholar.google.com/scholar?q=William+Gaviria+Rojas%2C+Sudnya+Diamos%2C+Keertan+Kini%2C+David+Kanter%2C+Vijay+Janapa+Reddi%2C+and+Cody+Coleman%2E+2022%2E+The+Dollar+Street+Dataset%3A+Images+Representing+the+Geographic+and+Socioeconomic+Diversity+)
35. Gregor Geigle, Abhay Jain, Radu Timofte, and Goran Glavas. 2023. mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMs. arXiv:2307.06930. arXiv preprint Gregor Geigle, Radu Timofte, and Goran Glavas. 2024a. African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained… [find](https://scholar.google.com/scholar?q=Gregor+Geigle%2C+Abhay+Jain%2C+Radu+Timofte%2C+and+Goran+Glavas%2E+2023%2E+mBLIP%3A+Efficient+Bootstrapping+of+Multilingual+Vision%2DLLMs%2E+arXiv%3A2307%2E06930%2E+arXiv+preprint+Gregor+Geigle%2C+Radu+Timofte%2C+and+Goran+Gla)
36. Gregor Geigle, Radu Timofte, and Goran Glavaš. 2024b. Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5064–5084, Bangkok, Thailand. [find](https://scholar.google.com/scholar?q=Gregor+Geigle%2C+Radu+Timofte%2C+and+Goran+Glava%C5%A1%2E+2024b%2E+Babel%2DImageNet%3A+Massively+Multilingual+Evaluation+of+Vision%2Dand%2DLanguage+Representations%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Associa)
37. Y. Goyal, T. Khot, D. Summers-Stay, D. Batra, and D. Parikh. 2017. [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://md.research.elamur.ai/free/en/papers/goyal-2016-making-v-vqa-matter-elevating/). In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6325–6334, Honolulu, HI, USA.
38. Danna Gurari, Qing Li, Abigale J. Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P. Bigham. 2018. [find](https://scholar.google.com/scholar?q=Danna+Gurari%2C+Qing+Li%2C+Abigale+J%2E+Stangl%2C+Anhong+Guo%2C+Chi+Lin%2C+Kristen+Grauman%2C+Jiebo+Luo%2C+and+Jeffrey+P%2E+Bigham%2E+2018%2E)
39. VizWiz Grand Challenge: Answering Visual Questions From Blind People. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, pages 3608–3617, Salt Lake City, UT, USA. [find](https://scholar.google.com/scholar?q=VizWiz+Grand+Challenge%3A+Answering+Visual+Questions+From+Blind+People%2E+In+2018+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+CVPR+2018%2C+pages+3608%E2%80%933617%2C+Salt+Lake+City%2C+UT%2C+USA%2E)
40. Musashi Hinck, Carolin Holtermann, Matthew L. Olson, Florian Schneider, Sungduk Yu, Anahita Bhiwandiwalla, Anne Lauscher, Shao-Yen Tseng, and Vasudev Lal. 2024. [Why do LLaVA Vision-Language Models Reply to Images in English?](https://md.research.elamur.ai/free/en/papers/hinck-2024-why-do-llava-vision-language/) In Findings of the Association for Computational Linguistics: EMNLP 2024, p…
41. Carolin Holtermann, Paul Röttger, Timm Dill, and Anne Lauscher. 2024. Evaluating the Elementary Multilingual Capabilities of Large Language Models with MultiQ. In Findings of the Association for Computational Linguistics: ACL 2024, pages 4476–4494, Bangkok, Thailand. [find](https://scholar.google.com/scholar?q=Carolin+Holtermann%2C+Paul+R%C3%B6ttger%2C+Timm+Dill%2C+and+Anne+Lauscher%2E+2024%2E+Evaluating+the+Elementary+Multilingual+Capabilities+of+Large+Language+Models+with+MultiQ%2E+In+Findings+of+the+Association+for+Compu)
42. Yu-Chung Hsiao, Fedir Zubach, Gilles Baechler, Srinivas Sunkara, Victor Carbune, Jason Lin, Maria Wang, Yun Zhu, and Jindong Chen. 2025. ScreenQA: Large-Scale Question-Answer Pairs Over Mobile App Screenshots. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Associ… [find](https://scholar.google.com/scholar?q=Yu%2DChung+Hsiao%2C+Fedir+Zubach%2C+Gilles+Baechler%2C+Srinivas+Sunkara%2C+Victor+Carbune%2C+Jason+Lin%2C+Maria+Wang%2C+Yun+Zhu%2C+and+Jindong+Chen%2E+2025%2E+ScreenQA%3A+Large%2DScale+Question%2DAnswer+Pairs+Over+Mobile+App+Scr)
43. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-Rank Adaptation of Large Language Models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual.
44. Jinyi Hu, Yuan Yao, Chongyi Wang, Shan Wang, Yinxu Pan, Qianyu Chen, Tianyu Yu, Hanghao Wu, Yue Zhao, Haoye Zhang, Xu Han, Yankai Lin, Jiao Xue, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. [Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages](https://md.research.elamur.ai/free/en/papers/hu-2023-large-multilingual-models-pivot-zero/). In The Twelfth International…
45. Drew A. Hudson and Christopher D. Manning. 2019. [GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering](https://md.research.elamur.ai/free/en/papers/hudson-2019-gqa-new-dataset-real-world/). In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, pages 6700–6709, Long Beach, CA, USA.
46. Harsh Jhamtani and Taylor Berg-Kirkpatrick. 2018. Learning to Describe Differences Between Pairs of Similar Images. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4024–4034, Brussels, Belgium. [find](https://scholar.google.com/scholar?q=Harsh+Jhamtani+and+Taylor+Berg%2DKirkpatrick%2E+2018%2E+Learning+to+Describe+Differences+Between+Pairs+of+Similar+Images%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Proce)
47. Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross B. Girshick. 2017. [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://md.research.elamur.ai/free/en/papers/johnson-2016-clevr-diagnostic-dataset-compositional-language/). In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pages 1988–19…
48. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://md.research.elamur.ai/free/en/papers/joshi-2020-state-fate-linguistic-diversity-inclusion/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pages 6282–6293, Online.
49. Kushal Kafle, Brian L. Price, Scott Cohen, and Christopher Kanan. 2018. DVQA: Understanding Data Visualizations via Question Answering. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, pages 5648–5656, Salt Lake City, UT, USA. [find](https://scholar.google.com/scholar?q=Kushal+Kafle%2C+Brian+L%2E+Price%2C+Scott+Cohen%2C+and+Christopher+Kanan%2E+2018%2E+DVQA%3A+Understanding+Data+Visualizations+via+Question+Answering%2E+In+2018+IEEE+Conference+on+Computer+Vision+and+Pattern+Recogniti)
50. Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna, Percy Liang, Thomas Kollar, and Dorsa Sadigh. 2024. Prismatic VLMs: investigating the design space of visually-conditioned language models. In Proceedings of the 41st International Conference on Machine Learning, pages 23123–23144, Vienna, Austri… [find](https://scholar.google.com/scholar?q=Siddharth+Karamcheti%2C+Suraj+Nair%2C+Ashwin+Balakrishna%2C+Percy+Liang%2C+Thomas+Kollar%2C+and+Dorsa+Sadigh%2E+2024%2E+Prismatic+VLMs%3A+investigating+the+design+space+of+visually%2Dconditioned+language+models%2E+In+Pro)
51. Mehran Kazemi, Hamidreza Alvari, Ankit Anand, Jialin Wu, Xi Chen, and Radu Soricut. 2023. GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoning. In AI for Math Workshop@ ICML 2024. [find](https://scholar.google.com/scholar?q=Mehran+Kazemi%2C+Hamidreza+Alvari%2C+Ankit+Anand%2C+Jialin+Wu%2C+Xi+Chen%2C+and+Radu+Soricut%2E+2023%2E+GeomVerse%3A+A+Systematic+Evaluation+of+Large+Models+for+Geometric+Reasoning%2E+In+AI+for+Math+Workshop%40+ICML+2024)
52. Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. 2014. [ReferItGame: Referring to Objects in Photographs of Natural Scenes](https://md.research.elamur.ai/free/en/papers/kazemzadeh-2014-referitgame-referring-objects-photographs-natural/). In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 787–798, Doha, Qatar.
53. Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. 2016. [A Diagram is Worth a Dozen Images](https://md.research.elamur.ai/free/en/papers/kembhavi-2016-diagram-worth-dozen-images/). In Computer Vision – ECCV 2016, pages 235–251, Amsterdam, The Netherlands.
54. Aniruddha Kembhavi, Min Joon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pages 537… [find](https://scholar.google.com/scholar?q=Aniruddha+Kembhavi%2C+Min+Joon+Seo%2C+Dustin+Schwenk%2C+Jonghyun+Choi%2C+Ali+Farhadi%2C+and+Hannaneh+Hajishirzi%2E+2017%2E+Are+You+Smarter+Than+a+Sixth+Grader%3F+Textbook+Question+Answering+for+Multimodal+Machine+Com)
55. Tannon Kew, Florian Schottmann, and Rico Sennrich. 2024. Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed? In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13097–13124, Miami, Florida, USA. [find](https://scholar.google.com/scholar?q=Tannon+Kew%2C+Florian+Schottmann%2C+and+Rico+Sennrich%2E+2024%2E+Turning+English%2Dcentric+LLMs+Into+Polyglots%3A+How+Much+Multilinguality+Is+Needed%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+)
56. Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2022. [OCR-Free Document Understanding Transformer](https://md.research.elamur.ai/free/en/papers/kim-2021-ocr-free-document-understanding-transformer/). In Computer Vision - ECCV 2022 - 17th European Conference, volume 13688, pages 498–517, Tel Aviv, Israe…
57. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. 2017. [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://md.research.elamur.ai/free/en/papers/krishna-2016-visual-genome-connecting-language-vision/). Int. J. Comput. Vis…
58. Hugo Laurençon, Léo Tronchon, and Victor Sanh. 2024. Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset. arXiv preprint arXiv:2403.09029. — [arXiv:2403.09029](https://arxiv.org/abs/2403.09029) [find](https://scholar.google.com/scholar?q=Hugo+Lauren%C3%A7on%2C+L%C3%A9o+Tronchon%2C+and+Victor+Sanh%2E+2024%2E+Unlocking+the+conversion+of+Web+Screenshots+into+HTML+Code+with+the+WebSight+Dataset%2E+arXiv+preprint+arXiv%3A2403%2E09029%2E+%E2%80%94+%5BarXiv%3A2403%2E09029%5D%28https%3A%2F)
59. Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. 2025. [What matters when building vision-language models?](https://md.research.elamur.ai/free/en/papers/lauren-ccon-2024-matters-building-vision-language-models/) In Proceedings of the Winter Conference on Applications of Computer Vision, pages 1372–1381.
60. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). In International Conference on Machine Learning, pages 19730–19742, Honolulu, HI, USA. PMLR.
61. Lei Li, Yuqi Wang, Runxin Xu, Peiyi Wang, Xiachong Feng, Lingpeng Kong, and Qi Liu. 2024. [Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-multimodal-arxiv-dataset-improving-scientific/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap…
62. Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. [Microsoft COCO: Common Objects in Context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/). In Computer Vision - ECCV 2014 - 13th European Conference, volume 8693, pages 740–755, Zurich, Switzerland.
63. Ziyi Lin, Dongyang Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Wenqi Shao, Keqin Chen, Jiaming Han, Siyuan Huang, Yichi Zhang, Xuming He, Yu Qiao, and Hongsheng Li. 2024. SPHINX: A Mixer of Weights, Visual Embeddings and Image Scales for Multi-modal Large Language Models. In Comput… [find](https://scholar.google.com/scholar?q=Ziyi+Lin%2C+Dongyang+Liu%2C+Renrui+Zhang%2C+Peng+Gao%2C+Longtian+Qiu%2C+Han+Xiao%2C+Han+Qiu%2C+Wenqi+Shao%2C+Keqin+Chen%2C+Jiaming+Han%2C+Siyuan+Huang%2C+Yichi+Zhang%2C+Xuming+He%2C+Yu+Qiao%2C+and+Hongsheng+Li%2E+2024%2E+SPHINX%3A+A+M)
64. Adam Dahlgren Lindström and Savitha Sam Abraham. 2022. CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoning. In Proceedings of the 16th International Workshop on Neural-Symbolic Learning and Reasoning as part of the 2nd International Joint Conference on Learning & Reas… [find](https://scholar.google.com/scholar?q=Adam+Dahlgren+Lindstr%C3%B6m+and+Savitha+Sam+Abraham%2E+2022%2E+CLEVR%2DMath%3A+A+Dataset+for+Compositional+Language%2C+Visual+and+Mathematical+Reasoning%2E+In+Proceedings+of+the+16th+International+Workshop+on+Neural%2D)
65. Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott. 2021. Visually Grounded Reasoning across Languages and Cultures. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, pages 10467–10485, Virtual Ev… [find](https://scholar.google.com/scholar?q=Fangyu+Liu%2C+Emanuele+Bugliarello%2C+Edoardo+Maria+Ponti%2C+Siva+Reddy%2C+Nigel+Collier%2C+and+Desmond+Elliott%2E+2021%2E+Visually+Grounded+Reasoning+across+Languages+and+Cultures%2E+In+Proceedings+of+the+2021+Confe)
66. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024a. [Improved Baselines with Visual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, pages 26296–26306, Seattle, WA, USA.
67. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024b. LLaVA-NeXT: Improved reasoning, OCR, and world knowledge. [find](https://scholar.google.com/scholar?q=Haotian+Liu%2C+Chunyuan+Li%2C+Yuheng+Li%2C+Bo+Li%2C+Yuanhan+Zhang%2C+Sheng+Shen%2C+and+Yong+Jae+Lee%2E+2024b%2E+LLaVA%2DNeXT%3A+Improved+reasoning%2C+OCR%2C+and+world+knowledge%2E)
68. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). In Advances in Neural Information Processing Systems, volume 36, pages 34892–34916.
69. Llama Team. 2024. [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
70. Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2019%2E+Decoupled+Weight+Decay+Regularization%2E+In+7th+International+Conference+on+Learning+Representations%2C+ICLR+2019%2C+New+Orleans%2C+LA%2C+USA%2C+May+6%2D9%2C+2019%2E)
71. Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. 2024. [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://md.research.elamur.ai/free/en/papers/lu-2024-deepseek-vl-real-world/). arXiv preprint arXiv:2403.05525.
72. Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. 2021a. [Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning](https://md.research.elamur.ai/free/en/papers/lu-2021-inter-gps-interpretable-geometry-problem/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th In…
73. Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering. In Advances in Neural Information Processing Systems 35, NeurIPS 2022, New Orl… [find](https://scholar.google.com/scholar?q=Pan+Lu%2C+Swaroop+Mishra%2C+Tanglin+Xia%2C+Liang+Qiu%2C+Kai%2DWei+Chang%2C+Song%2DChun+Zhu%2C+Oyvind+Tafjord%2C+Peter+Clark%2C+and+Ashwin+Kalyan%2E+2022%2E+Learn+to+Explain%3A+Multimodal+Reasoning+via+Thought+Chains+for+Scienc)
74. Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. 2023. [Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/lu-2022-dynamic-prompt-learning-policy-gradient/). In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwa…
75. Pan Lu, Liang Qiu, Jiaqi Chen, Tanglin Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chun Zhu. 2021b. IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks… [find](https://scholar.google.com/scholar?q=Pan+Lu%2C+Liang+Qiu%2C+Jiaqi+Chen%2C+Tanglin+Xia%2C+Yizhou+Zhao%2C+Wei+Zhang%2C+Zhou+Yu%2C+Xiaodan+Liang%2C+and+Song%2DChun+Zhu%2E+2021b%2E+IconQA%3A+A+New+Benchmark+for+Abstract+Diagram+Understanding+and+Visual+Language+Rea)
76. Muhammad Maaz, Hanoona Rasheed, Abdelrahman Shaker, Salman Khan, Hisham Cholakal, Rao M Anwer, Tim Baldwin, Michael Felsberg, and Fahad S Khan. 2025. Palo: A Polyglot Large Multimodal Model for 5B People. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1745–1754,… [find](https://scholar.google.com/scholar?q=Muhammad+Maaz%2C+Hanoona+Rasheed%2C+Abdelrahman+Shaker%2C+Salman+Khan%2C+Hisham+Cholakal%2C+Rao+M+Anwer%2C+Tim+Baldwin%2C+Michael+Felsberg%2C+and+Fahad+S+Khan%2E+2025%2E+Palo%3A+A+Polyglot+Large+Multimodal+Model+for+5B+Peo)
77. Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L. Yuille, and Kevin Murphy. 2016. [Generation and Comprehension of Unambiguous Object Descriptions](https://md.research.elamur.ai/free/en/papers/mao-2015-generation-comprehension-unambiguous-object-descriptions/). In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, pages 11–20, Las Vegas, NV, USA.
78. Kelly Marchisio, Wei-Yin Ko, Alexandre Berard, Théo Dehaze, and Sebastian Ruder. 2024. [Understanding and Mitigating Language Confusion in LLMs](https://md.research.elamur.ai/free/en/papers/marchisio-2024-understanding-mitigating-language-confusion-llms/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, pages 6653–6677, Miami, FL, USA.
79. Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. 2019. [find](https://scholar.google.com/scholar?q=Kenneth+Marino%2C+Mohammad+Rastegari%2C+Ali+Farhadi%2C+and+Roozbeh+Mottaghi%2E+2019%2E)
80. OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3190–3199, Long Beach, CA, USA. [find](https://scholar.google.com/scholar?q=OK%2DVQA%3A+A+Visual+Question+Answering+Benchmark+Requiring+External+Knowledge%2E+In+2019+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2C+pages+3190%E2%80%933199%2C+Long+Beach%2C+CA%2C+USA%2E)
81. Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq R. Joty, and Enamul Hoque. 2022. [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/). In Findings of the Association for Computational Linguistics: ACL 2022, pages 2263–2279, Dublin, Ireland.
82. Christian Blaise Cruz, Jay Gala, Jiahui Geng, Jesus-German Ortiz-Barajas, Jinheon Baek, Jocelyn Dunstan, Laura Alonso Alemany, Kumaranage Ravindu Yasas Nagasinghe, Luciana Benotti, Luis Fernando D' Haro, Marcelo Viridiano, Marcos Estecha-Garitagoitia, Maria Camila Buitrago Cabrera, Mario Rodríguez-C… [find](https://scholar.google.com/scholar?q=Christian+Blaise+Cruz%2C+Jay+Gala%2C+Jiahui+Geng%2C+Jesus%2DGerman+Ortiz%2DBarajas%2C+Jinheon+Baek%2C+Jocelyn+Dunstan%2C+Laura+Alonso+Alemany%2C+Kumaranage+Ravindu+Yasas+Nagasinghe%2C+Luciana+Benotti%2C+Luis+Fernando+D%27+Ha)
83. Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and C. V. Jawahar. 2022. InfographicVQA. In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022, pages 2582–2591, Waikoloa, HI, USA. [find](https://scholar.google.com/scholar?q=Minesh+Mathew%2C+Viraj+Bagal%2C+Rub%C3%A8n+Tito%2C+Dimosthenis+Karatzas%2C+Ernest+Valveny%2C+and+C%2E+V%2E+Jawahar%2E+2022%2E+InfographicVQA%2E+In+IEEE%2FCVF+Winter+Conference+on+Applications+of+Computer+Vision%2C+WACV+2022%2C+page)
84. Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. 2021. [DocVQA: A Dataset for VQA on Document Images](https://md.research.elamur.ai/free/en/papers/mathew-2020-docvqa-dataset-vqa-document-images/). In IEEE Winter Conference on Applications of Computer Vision, WACV 2021, pages 2199–2208, Waikoloa, HI, USA.
85. Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. 2019. OCR-VQA: Visual Question Answering by Reading Text in Images. In 2019 International Conference on Document Analysis and Recognition, ICDAR 2019, pages 947–952, Sydney, Australia. [find](https://scholar.google.com/scholar?q=Anand+Mishra%2C+Shashank+Shekhar%2C+Ajeet+Kumar+Singh%2C+and+Anirban+Chakraborty%2E+2019%2E+OCR%2DVQA%3A+Visual+Question+Answering+by+Reading+Text+in+Images%2E+In+2019+International+Conference+on+Document+Analysis+an)
86. Shravan Nayak, Kanishk Jain, Rabiul Awal, Siva Reddy, Sjoerd van Steenkiste, Lisa Anne Hendricks, Karolina Stanczak, and Aishwarya Agrawal. 2024. [Benchmarking Vision Language Models for Cultural Understanding](https://md.research.elamur.ai/free/en/papers/nayak-2024-benchmarking-vision-language-models-cultural/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,…
87. Jason Obeid and Enamul Hoque. 2020. Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model. In Proceedings of the 13th International Conference on Natural Language Generation, INLG 2020, pages 138–147, Dublin, Ireland. [find](https://scholar.google.com/scholar?q=Jason+Obeid+and+Enamul+Hoque%2E+2020%2E+Chart%2Dto%2DText%3A+Generating+Natural+Language+Descriptions+for+Charts+by+Adapting+the+Transformer+Model%2E+In+Proceedings+of+the+13th+International+Conference+on+Natural)
88. Jonas Pfeiffer, Gregor Geigle, Aishwarya Kamath, Jan-Martin O. Steitz, Stefan Roth, Ivan Vulic, and Iryna Gurevych. 2022a. xGQA: Cross-Lingual Visual Question Answering. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2497–2511, Dublin, Ireland. [find](https://scholar.google.com/scholar?q=Jonas+Pfeiffer%2C+Gregor+Geigle%2C+Aishwarya+Kamath%2C+Jan%2DMartin+O%2E+Steitz%2C+Stefan+Roth%2C+Ivan+Vulic%2C+and+Iryna+Gurevych%2E+2022a%2E+xGQA%3A+Cross%2DLingual+Visual+Question+Answering%2E+In+Findings+of+the+Association)
89. Jonas Pfeiffer, Naman Goyal, Xi Lin, Xian Li, James Cross, Sebastian Riedel, and Mikel Artetxe. 2022b. [Lifting the Curse of Multilinguality by Pre-training Modular Transformers](https://md.research.elamur.ai/free/en/papers/pfeiffer-2022-lifting-curse-multilinguality-pre-training/). In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Huma…
90. David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo, Teresa Lynn, Injy Hamed, Aditya Nanda Kishore, Aishik Mandal, Alina Dragonetti, Artem Abzaliev, Atnafu Lambebo Tonja, Bontu Fufa Balcha, Chenxi Whitehouse, Christian Salamea, Dan John Velasco, David Ifeoluwa Adelani, David Le Meur, Emilio Villa-Cu… [find](https://scholar.google.com/scholar?q=David+Romero%2C+Chenyang+Lyu%2C+Haryo+Akbarianto+Wibowo%2C+Teresa+Lynn%2C+Injy+Hamed%2C+Aditya+Nanda+Kishore%2C+Aishik+Mandal%2C+Alina+Dragonetti%2C+Artem+Abzaliev%2C+Atnafu+Lambebo+Tonja%2C+Bontu+Fufa+Balcha%2C+Chenxi+Whi)
91. Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. 2022. [LAION-5B: An Open Larg…](https://md.research.elamur.ai/free/en/papers/schuhmann-2022-laion-5b-open-large-scale/)
92. Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. [A-OKVQA: A Benchmark for Visual Question Answering Using World Knowledge](https://md.research.elamur.ai/free/en/papers/schwenk-2022-okvqa-benchmark-visual-question-answering/). In Computer Vision - ECCV 2022 - 17th European Conference, volume 13668, pages 146–162, Tel Aviv, Israel.
93. Uri Shaham, Jonathan Herzig, Roee Aharoni, Idan Szpektor, Reut Tsarfaty, and Matan Eyal. 2024. [Multilingual Instruction Tuning With Just a Pinch of Multilinguality](https://md.research.elamur.ai/free/en/papers/shaham-2024-multilingual-instruction-tuning-just-pinch/). In Findings of the Association for Computational Linguistics: ACL 2024, pages 2304–2317, Bangkok, Thailand.
94. Baifeng Shi, Ziyang Wu, Maolin Mao, Xin Wang, and Trevor Darrell. 2024. When Do We Not Need Larger Vision Models? In Computer Vision - ECCV 2024 - 18th European Conference, volume 15066, pages 444–462, Milan, Italy. [find](https://scholar.google.com/scholar?q=Baifeng+Shi%2C+Ziyang+Wu%2C+Maolin+Mao%2C+Xin+Wang%2C+and+Trevor+Darrell%2E+2024%2E+When+Do+We+Not+Need+Larger+Vision+Models%3F+In+Computer+Vision+%2D+ECCV+2024+%2D+18th+European+Conference%2C+volume+15066%2C+pages+444%E2%80%93462)
95. Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024. Design2Code: How Far Are We From Automating Front-End Engineering? arXiv preprint arXiv:2403.03163. — [arXiv:2403.03163](https://arxiv.org/abs/2403.03163) [find](https://scholar.google.com/scholar?q=Chenglei+Si%2C+Yanzhe+Zhang%2C+Zhengyuan+Yang%2C+Ruibo+Liu%2C+and+Diyi+Yang%2E+2024%2E+Design2Code%3A+How+Far+Are+We+From+Automating+Front%2DEnd+Engineering%3F+arXiv+preprint+arXiv%3A2403%2E03163%2E+%E2%80%94+%5BarXiv%3A2403%2E03163%5D%28http)
96. Shivalika Singh, Freddie Vargus, Daniel D’souza, Börje Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura O’Mahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergun, Ifeoma Oko… — [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/singh-2024-aya-dataset-open-access-collection/)
97. Gurkan Soykan and Gözde Gül Sahin. 2024. Linguistically-Informed Multilingual Instruction Tuning: Is There an Optimal Set of Languages to Tune? arXiv preprint arXiv:2410.07809. — [arXiv:2410.07809](https://arxiv.org/abs/2410.07809) [find](https://scholar.google.com/scholar?q=Gurkan+Soykan+and+G%C3%B6zde+G%C3%BCl+Sahin%2E+2024%2E+Linguistically%2DInformed+Multilingual+Instruction+Tuning%3A+Is+There+an+Optimal+Set+of+Languages+to+Tune%3F+arXiv+preprint+arXiv%3A2410%2E07809%2E+%E2%80%94+%5BarXiv%3A2410%2E07809%5D%28ht)
98. Krishna Srinivasan, Karthik Raman, Jiecao Chen, Michael Bendersky, and Marc Najork. 2021. WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’2… [find](https://scholar.google.com/scholar?q=Krishna+Srinivasan%2C+Karthik+Raman%2C+Jiecao+Chen%2C+Michael+Bendersky%2C+and+Marc+Najork%2E+2021%2E+WIT%3A+Wikipedia%2Dbased+Image+Text+Dataset+for+Multimodal+Multilingual+Machine+Learning%2E+In+Proceedings+of+the+44)
99. Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi. 2019. [A Corpus for Reasoning about Natural Language Grounded in Photographs](https://md.research.elamur.ai/free/en/papers/suhr-2019-corpus-reasoning-natural-language-grounded/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6418–6428, Florence, Italy.
100. Hai-Long Sun, Da-Wei Zhou, Yang Li, Shiyin Lu, Chao Yi, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, et al. 2024. Parrot: Multilingual Visual Instruction Tuning. arXiv preprint arXiv:2406.02539. — [arXiv:2406.02539](https://arxiv.org/abs/2406.02539) [find](https://scholar.google.com/scholar?q=Hai%2DLong+Sun%2C+Da%2DWei+Zhou%2C+Yang+Li%2C+Shiyin+Lu%2C+Chao+Yi%2C+Qing%2DGuo+Chen%2C+Zhao+Xu%2C+Weihua+Luo%2C+Kaifu+Zhang%2C+De%2DChuan+Zhan%2C+et+al%2E+2024%2E+Parrot%3A+Multilingual+Visual+Instruction+Tuning%2E+arXiv+preprint+arXi)
101. Ryota Tanaka, Kyosuke Nishida, and Sen Yoshida. 2021. VisualMRC: Machine Reading Comprehension on Document Images. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, pages 13878– 13888, Virtual. [find](https://scholar.google.com/scholar?q=Ryota+Tanaka%2C+Kyosuke+Nishida%2C+and+Sen+Yoshida%2E+2021%2E+VisualMRC%3A+Machine+Reading+Comprehension+on+Document+Images%2E+In+Thirty%2DFifth+AAAI+Conference+on+Artificial+Intelligence%2C+AAAI+2021%2C+pages+13878%E2%80%93+1)
102. Benny J. Tang, Angie W. Boggust, and Arvind Satyanarayan. 2023. VisText: A Benchmark for Semantically Rich Chart Captioning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, pages 7268–7298, Toronto, Canada. [find](https://scholar.google.com/scholar?q=Benny+J%2E+Tang%2C+Angie+W%2E+Boggust%2C+and+Arvind+Satyanarayan%2E+2023%2E+VisText%3A+A+Benchmark+for+Semantically+Rich+Chart+Captioning%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computatio)
103. Jingqun Tang, Qi Liu, Yongjie Ye, Jinghui Lu, Shu Wei, Chunhui Lin, Wanqing Li, Mohamad Fitri Faiz Bin Mahmood, Hao Feng, Zhen Zhao, Yanjie Wang, Yuliang Liu, Hao Liu, Xiang Bai, and Can Huang. 2024. MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering. arXiv preprint arXiv:2405.1… [find](https://scholar.google.com/scholar?q=Jingqun+Tang%2C+Qi+Liu%2C+Yongjie+Ye%2C+Jinghui+Lu%2C+Shu+Wei%2C+Chunhui+Lin%2C+Wanqing+Li%2C+Mohamad+Fitri+Faiz+Bin+Mahmood%2C+Hao+Feng%2C+Zhen+Zhao%2C+Yanjie+Wang%2C+Yuliang+Liu%2C+Hao+Liu%2C+Xiang+Bai%2C+and+Can+Huang%2E+2024%2E+)
104. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. 2024. Gemma 2: Improving Open Language Models at a Practical Size. arXiv preprint arXiv:2408.00118. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+Cassidy+Hardin%2C+Surya+Bhupatiraju%2C+L%C3%A9onard+Hussenot%2C+Thomas+Mesnard%2C+Bobak+Shahriari%2C+Alexandre+Ram%C3%A9%2C+et+al%2E+2024%2E+Gemma+2%3A+Improving+O)
105. Ashish V. Thapliyal, Jordi Pont Tuset, Xi Chen, and Radu Soricut. 2022. [Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset](https://md.research.elamur.ai/free/en/papers/thapliyal-2022-crossmodal-3600-massively-multilingual-multimodal/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 715–729, Abu Dhabi, United Arab Emirates.
106. Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, and Saining Xie. 2024. [Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs](https://md.research.elamur.ai/free/en/papers/tong-2024-cambrian-1-fully-open-vision/). In Advances in Neural…
107. Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, and Cihang Xie. 2024. How Many Are in This Image A Safety Evaluation Benchmark for Vision LLMs. In Computer Vision – ECCV 2024: 18th European Conference, page 37–55, Milan, Italy. [find](https://scholar.google.com/scholar?q=Haoqin+Tu%2C+Chenhang+Cui%2C+Zijun+Wang%2C+Yiyang+Zhou%2C+Bingchen+Zhao%2C+Junlin+Han%2C+Wangchunshu+Zhou%2C+Huaxiu+Yao%2C+and+Cihang+Xie%2E+2024%2E+How+Many+Are+in+This+Image+A+Safety+Evaluation+Benchmark+for+Vision+LLM)
108. Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. [CIDEr: Consensus-based image description evaluation](https://md.research.elamur.ai/free/en/papers/vedantam-2014-cider-consensus-based-image-description/). In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, pages 4566–4575, Boston, MA, USA.
109. Junke Wang, Lingchen Meng, Zejia Weng, Bo He, Zuxuan Wu, and Yu-Gang Jiang. 2023. To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuning. arXiv preprint arXiv:2311.07574. — [arXiv:2311.07574](https://arxiv.org/abs/2311.07574) [find](https://scholar.google.com/scholar?q=Junke+Wang%2C+Lingchen+Meng%2C+Zejia+Weng%2C+Bo+He%2C+Zuxuan+Wu%2C+and+Yu%2DGang+Jiang%2E+2023%2E+To+See+is+to+Believe%3A+Prompting+GPT%2D4V+for+Better+Visual+Instruction+Tuning%2E+arXiv+preprint+arXiv%3A2311%2E07574%2E+%E2%80%94+%5BarXiv)
110. Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. 2024a. [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://md.research.elamur.ai/free/en/papers/wang-2024-measuring-multimodal-mathematical-reasoning-math/). In Advances in Neural Information Processing Systems, volume 37, pages 95095–95169, Vancouver, Canada.
111. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024b. [Qwen2-VL: Enhancing Vision-Language Model’s Perception of the…](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
112. Alexander Arno Weber, Klaudia Thellmann, Jan Ebert, Nicolas Flores-Herr, Jens Lehmann, Michael Fromm, and Mehdi Ali. 2024. Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions? In Proceedings of the 2024 Conference on Empirical Methods in Natural Lan… [find](https://scholar.google.com/scholar?q=Alexander+Arno+Weber%2C+Klaudia+Thellmann%2C+Jan+Ebert%2C+Nicolas+Flores%2DHerr%2C+Jens+Lehmann%2C+Michael+Fromm%2C+and+Mehdi+Ali%2E+2024%2E+Investigating+Multilingual+Instruction%2DTuning%3A+Do+Polyglot+Models+Demand+for+)
113. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. [Sigmoid Loss for Language Image Pre-Training](https://md.research.elamur.ai/free/en/papers/zhai-2023-sigmoid-loss-language-image-pre/). CoRR, abs/2303.15343.
114. Ning Xie, Farley Lai, Derek Doran, and Asim Kadav. 2019. Visual Entailment: A Novel Task for Fine-grained Image Understanding. arXiv preprint arXiv:1901.06706. — [arXiv:1901.06706](https://arxiv.org/abs/1901.06706) [find](https://scholar.google.com/scholar?q=Ning+Xie%2C+Farley+Lai%2C+Derek+Doran%2C+and+Asim+Kadav%2E+2019%2E+Visual+Entailment%3A+A+Novel+Task+for+Fine%2Dgrained+Image+Understanding%2E+arXiv+preprint+arXiv%3A1901%2E06706%2E+%E2%80%94+%5BarXiv%3A1901%2E06706%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
115. Chi Zhang, Feng Gao, Baoxiong Jia, Yixin Zhu, and Song-Chun Zhu. 2019. RAVEN: A Dataset for Relational and Analogical Visual REasoNing. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, pages 5317–5327, Long Beach, CA, USA. [find](https://scholar.google.com/scholar?q=Chi+Zhang%2C+Feng+Gao%2C+Baoxiong+Jia%2C+Yixin+Zhu%2C+and+Song%2DChun+Zhu%2E+2019%2E+RAVEN%3A+A+Dataset+for+Relational+and+Analogical+Visual+REasoNing%2E+In+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+C)
116. Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. 2024. [Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing](https://md.research.elamur.ai/free/en/papers/xu-2024-magpie-alignment-data-synthesis-scratch/). CoRR, abs/2406.08464.
117. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Juny… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Guanting+Dong%2C+Haoran+Wei%2C+Huan+Lin%2C+Jialong+Tang%2C+Jialin+Wang%2C+Jian+Yang%2C+Jian)
118. Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, Qianyu Chen, Huarong Zhou, Zhensheng Zou, Haoye Zhang, Shengding Hu, Zhi Zheng, Jie Zhou, Jie Cai, Xu Han, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. [MiniCPM-V: A GPT-4…](https://md.research.elamur.ai/free/en/papers/yao-2024-minicpm-v-gpt-4v/)
119. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2024. [MMMU: A Massive Multi-disc…](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/)
120. Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. 2024. [MM-LLMs: Recent Advances in MultiModal Large Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2024-mm-llms-recent-advances-multimodal/). In Findings of the Association for Computational Linguistics: ACL 2024, pages 12401– 12430, Bangkok, Thailand.
121. Wenxuan Zhang, Mahani Aljunied, Chang Gao, Yew Ken Chia, and Lidong Bing. 2023a. M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models. In Advances in Neural Information Processing Systems, volume 36, pages 5484–5505, New Orleans, LA, USA. [find](https://scholar.google.com/scholar?q=Wenxuan+Zhang%2C+Mahani+Aljunied%2C+Chang+Gao%2C+Yew+Ken+Chia%2C+and+Lidong+Bing%2E+2023a%2E+M3Exam%3A+A+Multilingual%2C+Multimodal%2C+Multilevel+Benchmark+for+Examining+Large+Language+Models%2E+In+Advances+in+Neural+Inf)
122. Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, and Tong Sun. 2023b. LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding. CoRR, abs/2306.17107. [find](https://scholar.google.com/scholar?q=Yanzhe+Zhang%2C+Ruiyi+Zhang%2C+Jiuxiang+Gu%2C+Yufan+Zhou%2C+Nedim+Lipka%2C+Diyi+Yang%2C+and+Tong+Sun%2E+2023b%2E+LLaVAR%3A+Enhanced+Visual+Instruction+Tuning+for+Text%2DRich+Image+Understanding%2E+CoRR%2C+abs%2F2306%2E17107%2E)
123. Yilun Zhao, Chen Zhao, Linyong Nan, Zhenting Qi, Wenlin Zhang, Xiangru Tang, Boyu Mi, and Dragomir Radev. 2023. RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Yilun+Zhao%2C+Chen+Zhao%2C+Linyong+Nan%2C+Zhenting+Qi%2C+Wenlin+Zhang%2C+Xiangru+Tang%2C+Boyu+Mi%2C+and+Dragomir+Radev%2E+2023%2E+RobuT%3A+A+Systematic+Study+of+Table+QA+Robustness+Against+Human%2DAnnotated+Adversarial+Per)
124. Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. 2021. [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://md.research.elamur.ai/free/en/papers/zhu-2021-tat-qa-question-answering-benchmark/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics…
125. Yuke Zhu, Oliver Groth, Michael S. Bernstein, and Li Fei-Fei. 2016. [Visual7W: Grounded Question Answering in Images](https://md.research.elamur.ai/free/en/papers/zhu-2015-visual7w-grounded-question-answering-images/). In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, pages 4995–5004, Las Vegas, NV, USA.
126. Xiang Yue, Yueqi Song, Akari Asai, Seungone Kim, Jean de Dieu Nyandwi, Simran Khanuja, Anjali Kantharuban, Lintang Sutawika, Sathyanarayanan Ramamoorthy, and Graham Neubig. 2025. Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages. In The Thirteenth International Conference on Learning… [find](https://scholar.google.com/scholar?q=Xiang+Yue%2C+Yueqi+Song%2C+Akari+Asai%2C+Seungone+Kim%2C+Jean+de+Dieu+Nyandwi%2C+Simran+Khanuja%2C+Anjali+Kantharuban%2C+Lintang+Sutawika%2C+Sathyanarayanan+Ramamoorthy%2C+and+Graham+Neubig%2E+2025%2E+Pangea%3A+A+Fully+Open+)

*Updated 31 October 2026*