---
title: "R2-MultiOmnia — paper by Ranaldi et al. (2025)"
description: "The authors propose R2-MultiOmnia, a two-phase framework that separates the logic of a problem… 33 references, cited by 5 papers here."
authors: ["Leonardo Ranaldi", "Federico Ranaldi", "Giulia Pucci"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.402/"
doi: "10.18653/v1/2025.acl-long.402"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ranaldi-2025-r2-multiomnia-leading-multilingual-multimodal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ranaldi-2025-r2-multiomnia-leading-multilingual-multimodal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/) / R2-MultiOmnia: Leading Multilingual Multimodal Reasoning via Self-Training

# R2-MultiOmnia: Leading Multilingual Multimodal Reasoning via Self-Training

**Authors:** [Leonardo Ranaldi](https://md.research.elamur.ai/free/en/authors/leonardo-ranaldi/), [Federico Ranaldi](https://md.research.elamur.ai/free/en/authors/federico-ranaldi/) and [Giulia Pucci](https://md.research.elamur.ai/free/en/authors/giulia-pucci/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.402/) · [PDF](https://aclanthology.org/2025.acl-long.402.pdf) · [DOI 10.18653/v1/2025.acl-long.402](https://doi.org/10.18653/v1/2025.acl-long.402) · [OpenAlex](https://openalex.org/W4412945611)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [5](https://md.research.elamur.ai/free/en/papers/ranaldi-2025-r2-multiomnia-leading-multilingual-multimodal/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Multimodal LLMs (MLLMs) reason less reliably outside English because high-quality multilingual supervision is scarce. Earlier efforts to strengthen visual reasoning, such as Multimodal CoT or R1-style reinforcement learning, have been English-only. The authors propose R2-MultiOmnia, a two-phase framework that separates the logic of a problem from its linguistic and visual content.

In the first phase, Multilingual Modality Bridging, an MLLM turns image–text items into explicit textual demonstrations: a description of the relevant visual elements, a formalisation of the problem, step-wise reasoning and a final answer. The model then re-checks this output, and rule-based rejection sampling discards incoherent samples. This yields about 10K demonstrations in 8 languages, which are used for an SFT warm-up.

In the second phase, Language-Agnostic Reasoning Alignment, the model is refined with GRPO. This phase uses a weighted composite reward, a curriculum that first compresses reasoning and later lengthens it, and cosine annealing of the KL penalty.

The method narrows cross-lingual performance gaps and brings small open models closer to proprietary systems. This matters for languages such as Ukrainian that are underrepresented in multimodal training data.

## Key points
- **Synthetic dataset:** roughly 10K multimodal reasoning samples in 8 languages, with tagged stages (`<visual_abstraction>`, `<formalisation>`, `<reasoning>`, `<answer>`) and rule-based filtering.
- **Training recipe:**
  - SFT warm-up, followed by GRPO with group-normalised advantages.
  - A composite reward covering answer accuracy, format compliance and structural integrity.
  - A Curriculum Learning schedule intended to avoid overthinking.
- **Models:** the main experiments use Qwen-3B-2.5-VL and DeepSeek-VL2-Tiny. Scaling tests use Qwen-7B-2.5-VL and DeepSeek-VL2-Small.
- **Benchmarks and results:** evaluation covers multilingual benchmarks (the PDF extraction renders them as xMMMLU, xMathV and M3Exam).
  - DeepSeek-VL2-S improves from 44.3 to 55.1, from 56.9 to 59.2, and from 45.0 to 61.6.
  - Its 61.6 on M3Exam is level with GPT-4o (61.0).
- **Stated limitations:** only a limited set of tasks and languages was covered, due to evaluation-tool and API costs. Future work targets cultural commonsense reasoning and cheaper annotation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (3): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Leonardo Ranaldi, Federico Ranaldi and Giulia Pucci. 2025. R2-MultiOmnia: Leading Multilingual Multimodal Reasoning via Self-Training. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ranaldi2025r2,
  title = {{R2-MultiOmnia: Leading Multilingual Multimodal Reasoning via Self-Training}},
  author = {Leonardo Ranaldi, Federico Ranaldi and Giulia Pucci},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.402},
  url = {https://doi.org/10.18653/v1/2025.acl-long.402}
}
```

- [← previous PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation](https://md.research.elamur.ai/free/en/papers/zhang-2025-proper-progressive-learning-framework-personalized/)
- [next → Revisiting Common Assumptions about Arabic Dialects in NLP](https://md.research.elamur.ai/free/en/papers/keleg-2025-revisiting-common-assumptions-about-arabic/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/)

## References

33 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping H… — [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
2. Gerry T. M. Altmann. 2001. The language machine: Psycholinguistics in review. British Journal of Psychology, 92(1):129–170. [find](https://scholar.google.com/scholar?q=Gerry+T%2E+M%2E+Altmann%2E+2001%2E+The+language+machine%3A+Psycholinguistics+in+review%2E+British+Journal+of+Psychology%2C+92%281%29%3A129%E2%80%93170%2E)
3. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Ha… — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
4. Soravit Changpinyo, Linting Xue, Michal Yarom, Ashish Thapliyal, Idan Szpektor, Julien Amelot, Xi Chen, and Radu Soricut. 2023. MaXM: Towards multilingual visual question answering. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2667–2682, Singapore. Association for… [find](https://scholar.google.com/scholar?q=Soravit+Changpinyo%2C+Linting+Xue%2C+Michal+Yarom%2C+Ashish+Thapliyal%2C+Idan+Szpektor%2C+Julien+Amelot%2C+Xi+Chen%2C+and+Radu+Soricut%2E+2023%2E+MaXM%3A+Towards+multilingual+visual+question+answering%2E+In+Findings+of+the)
5. Christine Cuskley, Rebecca Woods, and Molly Flaherty. 2024. The limitations of large language models for understanding human language and cognition. Open Mind, 8:1058–1083. [find](https://scholar.google.com/scholar?q=Christine+Cuskley%2C+Rebecca+Woods%2C+and+Molly+Flaherty%2E+2024%2E+The+limitations+of+large+language+models+for+understanding+human+language+and+cognition%2E+Open+Mind%2C+8%3A1058%E2%80%931083%2E)
6. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chengg… [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2C+Daya+Guo%2C+Dejian+Yang%2C+Haowei+Zhang%2C+Junxiao+Song%2C+Ruoyu+Zhang%2C+Runxin+Xu%2C+Qihao+Zhu%2C+Shirong+Ma%2C+Peiyi+Wang%2C+Xiao+Bi%2C+Xiaokang+Zhang%2C+Xingkai+Yu%2C+Yu+Wu%2C+Z%2E+F%2E+Wu%2C+Zhibin+Gou%2C+Zhihong+Sha)
7. Philip N. Johnson-Laird. 2010. Mental models and human reasoning. Proceedings of the National Academy of Sciences, 107(43):18243–18250. [find](https://scholar.google.com/scholar?q=Philip+N%2E+Johnson%2DLaird%2E+2010%2E+Mental+models+and+human+reasoning%2E+Proceedings+of+the+National+Academy+of+Sciences%2C+107%2843%29%3A18243%E2%80%9318250%2E)
8. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/).
9. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306.
10. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024. [Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/).
11. Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. 2024. [Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models](https://md.research.elamur.ai/free/en/papers/mirzadeh-2024-gsm-symbolic-understanding-limitations-mathematical/).
12. Leonardo Ranaldi and Andre Freitas. 2024a. [Aligning large and small language models via chain-of-thought reasoning](https://md.research.elamur.ai/free/en/papers/ranaldi-2024-aligning-large-small-language-models/). In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1812–1827, St. Julian’s, Malta. Associatio…
13. Leonardo Ranaldi and Andre Freitas. 2024b. [Self-refine instruction-tuning for aligning reasoning in language models](https://md.research.elamur.ai/free/en/papers/ranaldi-2024-self-refine-instruction-tuning-aligning/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2325–2347, Miami, Florida, USA. Association for Computational Linguistics.
14. Leonardo Ranaldi, Barry Haddow, and Alexandra Birch. 2025a. [Multilingual retrieval-augmented generation for knowledge-intensive task](https://md.research.elamur.ai/free/en/papers/ranaldi-2025-multilingual-retrieval-augmented-generation-knowledge/).
15. Leonardo Ranaldi, Barry Haddow, and Alexandra Birch. 2025b. When natural language is not enough: The limits of in-context learning demonstrations in multilingual reasoning. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 7369–7396, Albuquerque, New Mexico. Association… [find](https://scholar.google.com/scholar?q=Leonardo+Ranaldi%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2025b%2E+When+natural+language+is+not+enough%3A+The+limits+of+in%2Dcontext+learning+demonstrations+in+multilingual+reasoning%2E+In+Findings+of+the+Associat)
16. Leonardo Ranaldi and Giulia Pucci. 2023. Does the English matter? elicit cross-lingual abilities of large language models. In Proceedings of the 3rd Workshop on Multi-lingual Representation Learning (MRL), pages 173–183, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Leonardo+Ranaldi+and+Giulia+Pucci%2E+2023%2E+Does+the+English+matter%3F+elicit+cross%2Dlingual+abilities+of+large+language+models%2E+In+Proceedings+of+the+3rd+Workshop+on+Multi%2Dlingual+Representation+Learning+%28)
17. Leonardo Ranaldi and Giulia Pucci. 2025. Multilingual reasoning via self-training. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 11566–11582, Albuquerque, New M… [find](https://scholar.google.com/scholar?q=Leonardo+Ranaldi+and+Giulia+Pucci%2E+2025%2E+Multilingual+reasoning+via+self%2Dtraining%2E+In+Proceedings+of+the+2025+Conference+of+the+Nations+of+the+Americas+Chapter+of+the+Association+for+Computational+Lin)
18. Leonardo Ranaldi, Giulia Pucci, and Andre Freitas. 2024a. Empowering cross-lingual abilities of instruction-tuned large language models by translation-following demonstrations. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7961–7973, Bangkok, Thailand. Association for… [find](https://scholar.google.com/scholar?q=Leonardo+Ranaldi%2C+Giulia+Pucci%2C+and+Andre+Freitas%2E+2024a%2E+Empowering+cross%2Dlingual+abilities+of+instruction%2Dtuned+large+language+models+by+translation%2Dfollowing+demonstrations%2E+In+Findings+of+the+Asso)
19. Leonardo Ranaldi, Giulia Pucci, Federico Ranaldi, Elena Sofia Ruzzetti, and Fabio Massimo Zanzotto. 2024b. A tree-of-thoughts to broaden multi-step reasoning across languages. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1229–1241, Mexico City, Mexico. Association… [find](https://scholar.google.com/scholar?q=Leonardo+Ranaldi%2C+Giulia+Pucci%2C+Federico+Ranaldi%2C+Elena+Sofia+Ruzzetti%2C+and+Fabio+Massimo+Zanzotto%2E+2024b%2E+A+tree%2Dof%2Dthoughts+to+broaden+multi%2Dstep+reasoning+across+languages%2E+In+Findings+of+the+Assoc)
20. Leonardo Ranaldi, Giulia Pucci, and Fabio Massimo Zanzotto. 2023. Modeling easiness for training transformers with curriculum learning. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 937– 948, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgar… [find](https://scholar.google.com/scholar?q=Leonardo+Ranaldi%2C+Giulia+Pucci%2C+and+Fabio+Massimo+Zanzotto%2E+2023%2E+Modeling+easiness+for+training+transformers+with+curriculum+learning%2E+In+Proceedings+of+the+14th+International+Conference+on+Recent+Ad)
21. Leonardo Ranaldi, Federico Ranaldi, Fabio Massimo Zanzotto, Barry Haddow, and Alexandra Birch. 2025c. [Improving multilingual retrieval-augmented language models through dialectic reasoning argumentations](https://md.research.elamur.ai/free/en/papers/ranaldi-2025-improving-multilingual-retrieval-augmented-language/).
22. Leonardo Ranaldi, Marco Valentino, and Andre Freitas. 2025d. Eliciting critical reasoning in retrieval-augmented generation via contrastive explanations. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Tech… [find](https://scholar.google.com/scholar?q=Leonardo+Ranaldi%2C+Marco+Valentino%2C+and+Andre+Freitas%2E+2025d%2E+Eliciting+critical+reasoning+in+retrieval%2Daugmented+generation+via+contrastive+explanations%2E+In+Proceedings+of+the+2025+Conference+of+the+N)
23. Leonardo Ranaldi, Marco Valentino, Alexander Polon-sky, and Andrè Freitas. 2025e. [Improving chain-of-thought reasoning via quasi-symbolic abstractions](https://md.research.elamur.ai/free/en/papers/ranaldi-2025-improving-chain-thought-reasoning-quasi/).
24. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/).
25. Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Mingjie Zhan, and Hongsheng Li. 2024. [Measuring multimodal mathematical reasoning with math-vision dataset](https://md.research.elamur.ai/free/en/papers/wang-2024-measuring-multimodal-mathematical-reasoning-math/).
26. Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wan… — [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://md.research.elamur.ai/free/en/papers/wu-2024-deepseek-vl2-mixture-experts/)
27. Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. 2025. Llava-cot: Let vision language models reason step-by-step. [find](https://scholar.google.com/scholar?q=Guowei+Xu%2C+Peng+Jin%2C+Hao+Li%2C+Yibing+Song%2C+Lichao+Sun%2C+and+Li+Yuan%2E+2025%2E+Llava%2Dcot%3A+Let+vision+language+models+reason+step%2Dby%2Dstep%2E)
28. Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. 2025. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. [find](https://scholar.google.com/scholar?q=Yi+Yang%2C+Xiaoxuan+He%2C+Hongkun+Pan%2C+Xiyan+Jiang%2C+Yan+Deng%2C+Xingtao+Yang%2C+Haoyu+Lu%2C+Dacheng+Yin%2C+Fengyun+Rao%2C+Minfeng+Zhu%2C+Bo+Zhang%2C+and+Wei+Chen%2E+2025%2E+R1%2Donevision%3A+Advancing+generalized+multimodal+re)
29. Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. 2024. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18319. — [arXiv:2412.18319](https://arxiv.org/abs/2412.18319) [find](https://scholar.google.com/scholar?q=Huanjin+Yao%2C+Jiaxing+Huang%2C+Wenhao+Wu%2C+Jingyi+Zhang%2C+Yibo+Wang%2C+Shunyu+Liu%2C+Yingjie+Wang%2C+Yuxin+Song%2C+Haocheng+Feng%2C+Li+Shen%2C+et+al%2E+2024%2E+Mulberry%3A+Empowering+mllm+with+o1%2Dlike+reasoning+and+reflecti)
30. Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2024. [A survey on multimodal large language models](https://md.research.elamur.ai/free/en/papers/yin-2023-survey-multimodal-large-language-models/). National Science Review, 11(12).
31. Xiang Yue, Yueqi Song, Akari Asai, Seungone Kim, Jean de Dieu Nyandwi, Simran Khanuja, Anjali Kantharuban, Lintang Sutawika, Sathyanarayanan Ramamoorthy, and Graham Neubig. 2025. Pangea: A fully open multilingual multimodal llm for 39 languages. [find](https://scholar.google.com/scholar?q=Xiang+Yue%2C+Yueqi+Song%2C+Akari+Asai%2C+Seungone+Kim%2C+Jean+de+Dieu+Nyandwi%2C+Simran+Khanuja%2C+Anjali+Kantharuban%2C+Lintang+Sutawika%2C+Sathyanarayanan+Ramamoorthy%2C+and+Graham+Neubig%2E+2025%2E+Pangea%3A+A+fully+open+)
32. Wenxuan Zhang, Sharifah Mahani Aljunied, Chang Gao, Yew Ken Chia, and Lidong Bing. 2023. M3exam: A multilingual, multimodal, multilevel benchmark for examining large language models. [find](https://scholar.google.com/scholar?q=Wenxuan+Zhang%2C+Sharifah+Mahani+Aljunied%2C+Chang+Gao%2C+Yew+Ken+Chia%2C+and+Lidong+Bing%2E+2023%2E+M3exam%3A+A+multilingual%2C+multimodal%2C+multilevel+benchmark+for+examining+large+language+models%2E)
33. Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2024. [Multimodal chain-of-thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-multimodal-chain-thought-reasoning-language/).

*Updated 30 October 2026*