---
title: "MiMo-VL Technical Report — paper by Team et al. (2025)"
description: "Xiaomi's LLM-Core team presents MiMo-VL-7B, a compact open vision-language model that handles… 76 references, cited by 33 papers here."
authors: ["Xiaomi LLM-Core Team", "Zihao Yue", "Zhenru Lin", "Yifan Song", "Weikun Wang", "Shuhuai Ren", "Shuhao Gu", "Shicheng Li", "Peidian Li", "Liang Zhao", "Lei Li", "Kainan Bao", "Hao Tian", "Hailin Zhang", "Gang Wang", "Dawei Zhu", "Cici", "Chenhong He", "Bowen Ye", "Bowen Shen", "Zihan Zhang", "Zihan Jiang", "Zhixian Zheng", "Zhichao Song", "Zhenbo Luo", "Yue Yu", "Yudong Wang", "Yuanyuan Tian", "Yu Tu", "Yihan Yan", "Yi Huang", "Xu Wang", "Xinzhe Xu", "Xingchen Song", "Xing Zhang", "Xing Yong", "Xin Zhang", "Xiangwei Deng", "Wenyu Yang", "Wenhan Ma", "Weiwei Lv", "Weiji Zhuang", "Wei Liu", "Sirui Deng", "Shuo Liu", "Shimao Chen", "Shihua Yu", "Shaohui Liu", "Shande Wang", "Rui Ma", "Qiantong Wang", "Peng Wang", "Nuo Chen", "Menghang Zhu", "Kangyang Zhou", "Kang Zhou", "Kai Fang", "Jun Shi", "Jinhao Dong", "Jiebao Xiao", "Jiaming Xu", "Huaqiu Liu", "Hongshen Xu", "Heng Qu", "Haochen Zhao", "Hanglong Lv", "Guoan Wang", "Duo Zhang", "Dong Zhang", "Di Zhang", "Chong Ma", "Chang Liu", "Can Cai", "Bingquan Xia"]
year: 2025
venue: "arXiv:2506.03569"
source: "https://arxiv.org/abs/2506.03569"
doi: "10.48550/arxiv.2506.03569"
arxiv: "2506.03569"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/team-2025-llm-mimo-vl-technical-report/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/team-2025-llm-mimo-vl-technical-report/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / MiMo-VL Technical Report

# MiMo-VL Technical Report

**Authors:** [Xiaomi LLM-Core Team](https://md.research.elamur.ai/free/en/authors/xiaomi-llm-core-team/), [Zihao Yue](https://md.research.elamur.ai/free/en/authors/zihao-yue/), Zhenru Lin, [Yifan Song](https://md.research.elamur.ai/free/en/authors/yifan-song/), Weikun Wang, [Shuhuai Ren](https://md.research.elamur.ai/free/en/authors/shuhuai-ren/), [Shuhao Gu](https://md.research.elamur.ai/free/en/authors/shuhao-gu/), Shicheng Li, Peidian Li, [Liang Zhao](https://md.research.elamur.ai/free/en/authors/liang-zhao/), [Lei Li](https://md.research.elamur.ai/free/en/authors/lei-li/), Kainan Bao, [Hao Tian](https://md.research.elamur.ai/free/en/authors/hao-tian/), Hailin Zhang, [Gang Wang](https://md.research.elamur.ai/free/en/authors/gang-wang/), [Dawei Zhu](https://md.research.elamur.ai/free/en/authors/dawei-zhu/), Cici, Chenhong He, Bowen Ye, Bowen Shen, [Zihan Zhang](https://md.research.elamur.ai/free/en/authors/zihan-zhang/), Zihan Jiang, Zhixian Zheng, Zhichao Song, [Zhenbo Luo](https://md.research.elamur.ai/free/en/authors/zhenbo-luo/), [Yue Yu](https://md.research.elamur.ai/free/en/authors/yue-yu/), [Yudong Wang](https://md.research.elamur.ai/free/en/authors/yudong-wang/), Yuanyuan Tian, Yu Tu, Yihan Yan, [Yi Huang](https://md.research.elamur.ai/free/en/authors/yi-huang/), [Xu Wang](https://md.research.elamur.ai/free/en/authors/xu-wang/), Xinzhe Xu, Xingchen Song, Xing Zhang, Xing Yong, [Xin Zhang](https://md.research.elamur.ai/free/en/authors/xin-zhang/), Xiangwei Deng, Wenyu Yang, Wenhan Ma, Weiwei Lv, [Weiji Zhuang](https://md.research.elamur.ai/free/en/authors/weiji-zhuang/), [Wei Liu](https://md.research.elamur.ai/free/en/authors/wei-liu/), Sirui Deng, Shuo Liu, Shimao Chen, Shihua Yu, Shaohui Liu, Shande Wang, Rui Ma, Qiantong Wang, [Peng Wang](https://md.research.elamur.ai/free/en/authors/peng-wang/), [Nuo Chen](https://md.research.elamur.ai/free/en/authors/nuo-chen/), Menghang Zhu, Kangyang Zhou, [Kang Zhou](https://md.research.elamur.ai/free/en/authors/kang-zhou/), Kai Fang, Jun Shi, Jinhao Dong, Jiebao Xiao, [Jiaming Xu](https://md.research.elamur.ai/free/en/authors/jiaming-xu/), Huaqiu Liu, Hongshen Xu, Heng Qu, Haochen Zhao, Hanglong Lv, Guoan Wang, Duo Zhang, [Dong Zhang](https://md.research.elamur.ai/free/en/authors/dong-zhang/), [Di Zhang](https://md.research.elamur.ai/free/en/authors/di-zhang/), Chong Ma, [Chang Liu](https://md.research.elamur.ai/free/en/authors/chang-liu/), Can Cai and Bingquan Xia · **Year:** 2025 · **Venue:** arXiv:2506.03569 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2506.03569) · [PDF](https://arxiv.org/pdf/2506.03569) · [DOI 10.48550/arxiv.2506.03569](https://doi.org/10.48550/arxiv.2506.03569) · [arXiv 2506.03569](https://arxiv.org/abs/2506.03569) · [OpenAlex](https://openalex.org/W4416073099)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [33](https://md.research.elamur.ai/free/en/papers/team-2025-llm-mimo-vl-technical-report/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Xiaomi's LLM-Core team presents MiMo-VL-7B, a compact open vision-language model that handles general visual understanding, multimodal reasoning and GUI grounding in one 7B system. It combines a native-resolution Qwen2.5-ViT encoder, a randomly initialised MLP projector and the reasoning-oriented MiMo-7B-Base language model.

Training has two phases. First, four pre-training stages use roughly 2.4 trillion tokens of curated captions, interleaved documents, OCR, grounding, video, GUI and synthetic long-CoT reasoning data, producing MiMo-VL-7B-SFT. Second, Mixed On-policy Reinforcement Learning (MORL) blends verifiable rewards with human-preference signals across text, image and video tasks. MORL uses a strictly on-policy variant of GRPO and yields MiMo-VL-7B-RL.

The RL model beats Qwen2.5-VL-7B on 35 of 40 tasks and scores 59.4 on OlympiadBench. It also tops OSWorld-G, outperforming even dedicated GUI agents such as UI-TARS. The release includes checkpoints and a 50+ task evaluation suite, which gives the community a reproducible small-scale baseline. The report also offers practical lessons on reasoning data and multi-domain RL.

## Key points
- **Pre-training stages and token budgets:**
  - Stage 1: projector warmup, 300B tokens.
  - Stage 2: ViT and projector alignment, 167B tokens.
  - Stage 3: full multimodal pre-training, 1.4T tokens at 8K context.
  - Stage 4: long-context SFT, 550B tokens at 32K context.
  - phash deduplication against benchmarks reduces contamination.
- **Effect of reasoning data in Stage 4:** Adding large volumes of synthetic long-CoT reasoning data gave gains of about +9 on MMMU, +14 on OSWorld-G and +16 on OlympiadBench, with no saturation. Average MMMU answer length grew from ~680 to ~2.5K tokens.
- **Headline scores:** MiMo-VL-7B-RL reaches 66.7 on MMMU. Its OSWorld-G score is reported as 56.1 in the abstract but 54.7 in the main text. It has the highest in-house Elo among open-source VLMs and is competitive with Claude 3.7 Sonnet.
- **On-policy RL vs vanilla GRPO:** Vanilla GRPO learns faster early on but plateaus near 20,000 samples. The on-policy variant keeps improving as data grows.
- **Interference between RL tasks:** Reasoning tasks push responses longer, while grounding and counting tasks push them shorter. Differences in task difficulty and possible reward hacking also make simultaneous gains across domains hard.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works

## Cite

Cite as: Xiaomi LLM-Core Team, Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, Shuhuai Ren, Shuhao Gu, Shicheng Li, Peidian Li, Liang Zhao, Lei Li, Kainan Bao, Hao Tian, Hailin Zhang, Gang Wang, Dawei Zhu, Cici, Chenhong He, Bowen Ye, Bowen Shen, Zihan Zhang, Zihan Jiang, Zhixian Zheng, Zhichao Song, Zhenbo Luo, Yue Yu, Yudong Wang, Yuanyuan Tian, Yu Tu, Yihan Yan, Yi Huang, Xu Wang, Xinzhe Xu, Xingchen Song, Xing Zhang, Xing Yong, Xin Zhang, Xiangwei Deng, Wenyu Yang, Wenhan Ma, Weiwei Lv, Weiji Zhuang, Wei Liu, Sirui Deng, Shuo Liu, Shimao Chen, Shihua Yu, Shaohui Liu, Shande Wang, Rui Ma, Qiantong Wang, Peng Wang, Nuo Chen, Menghang Zhu, Kangyang Zhou, Kang Zhou, Kai Fang, Jun Shi, Jinhao Dong, Jiebao Xiao, Jiaming Xu, Huaqiu Liu, Hongshen Xu, Heng Qu, Haochen Zhao, Hanglong Lv, Guoan Wang, Duo Zhang, Dong Zhang, Di Zhang, Chong Ma, Chang Liu, Can Cai and Bingquan Xia. 2025. MiMo-VL Technical Report. arXiv:2506.03569.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{team2025mimo,
  title = {{MiMo-VL Technical Report}},
  author = {Xiaomi LLM-Core Team, Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, Shuhuai Ren, Shuhao Gu, Shicheng Li, Peidian Li, Liang Zhao, Lei Li, Kainan Bao, Hao Tian, Hailin Zhang, Gang Wang, Dawei Zhu, Cici, Chenhong He, Bowen Ye, Bowen Shen, Zihan Zhang, Zihan Jiang, Zhixian Zheng, Zhichao Song, Zhenbo Luo, Yue Yu, Yudong Wang, Yuanyuan Tian, Yu Tu, Yihan Yan, Yi Huang, Xu Wang, Xinzhe Xu, Xingchen Song, Xing Zhang, Xing Yong, Xin Zhang, Xiangwei Deng, Wenyu Yang, Wenhan Ma, Weiwei Lv, Weiji Zhuang, Wei Liu, Sirui Deng, Shuo Liu, Shimao Chen, Shihua Yu, Shaohui Liu, Shande Wang, Rui Ma, Qiantong Wang, Peng Wang, Nuo Chen, Menghang Zhu, Kangyang Zhou, Kang Zhou, Kai Fang, Jun Shi, Jinhao Dong, Jiebao Xiao, Jiaming Xu, Huaqiu Liu, Hongshen Xu, Heng Qu, Haochen Zhao, Hanglong Lv, Guoan Wang, Duo Zhang, Dong Zhang, Di Zhang, Chong Ma, Chang Liu, Can Cai and Bingquan Xia},
  year = {2025},
  doi = {10.48550/arxiv.2506.03569},
  eprint = {2506.03569},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2506.03569}
}
```

- [← previous DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/shao-2025-deepseekmath-v2-self-verifiable/)
- [next → OpenCodeReasoning: Advancing Data Distillation for Competitive Coding](https://md.research.elamur.ai/free/en/papers/ahmad-2025-opencodereasoning-advancing-data-distillation-competitive/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/)

## References

76 references, 46 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. J. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Monteiro, J. L. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Binkowski, R. Barreira, O. Vinyals, A. Ziss… — [Flamingo: a Visual Language Model for Few-Shot Learning](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/)
2. S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y. Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin. [Qwen2.5-vl technical report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/). arXiv preprint arXiv:2502.13923, 2025a…
3. S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.[13923, 2025b](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/).
4. K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky. 𝜋 0: A vision-language-action flow mo… [find](https://scholar.google.com/scholar?q=K%2E+Black%2C+N%2E+Brown%2C+D%2E+Driess%2C+A%2E+Esmail%2C+M%2E+Equi%2C+C%2E+Finn%2C+N%2E+Fusai%2C+L%2E+Groom%2C+K%2E+Hausman%2C+B%2E+Ichter%2C+S%2E+Jakubczak%2C+T%2E+Jones%2C+L%2E+Ke%2C+S%2E+Levine%2C+A%2E+Li%2DBell%2C+M%2E+Mothukuri%2C+S%2E+Nair%2C+K%2E+Pertsch%2C+L%2E+X%2E+Sh)
5. L. Chen, L. Li, H. Zhao, Y. Song, and Vinci. R1-v: Reinforcing super generalization ability in vision-language models with less than $3. [https://github.com/Deep-Agent/R1-V](https://github.com/Deep-Agent/R1-V) , 2025a. Accessed: 2025-02-02. [↗](https://github.com/Deep-Agent/R1-V)
6. X. Chen, Z. Zhao, L. Chen, D. Zhang, J. Ji, A. Luo, Y. Xiong, and K. Yu. Websrc: a dataset for web-based structural reading comprehension. arXiv preprint arXiv:2101.09465, 2021. [find](https://scholar.google.com/scholar?q=X%2E+Chen%2C+Z%2E+Zhao%2C+L%2E+Chen%2C+D%2E+Zhang%2C+J%2E+Ji%2C+A%2E+Luo%2C+Y%2E+Xiong%2C+and+K%2E+Yu%2E+Websrc%3A+a+dataset+for+web%2Dbased+structural+reading+comprehension%2E+arXiv+preprint+arXiv%3A2101%2E09465%2C+2021%2E)
7. Y. Chen, Z. Yang, Z. Liu, C. Lee, P. Xu, M. Shoeybi, B. Catanzaro, and W. Ping. Acereason-nemotron: Advancing math and code reasoning through reinforcement learning. arXiv preprint arXiv:2505.16400, 2025b. [find](https://scholar.google.com/scholar?q=Y%2E+Chen%2C+Z%2E+Yang%2C+Z%2E+Liu%2C+C%2E+Lee%2C+P%2E+Xu%2C+M%2E+Shoeybi%2C+B%2E+Catanzaro%2C+and+W%2E+Ping%2E+Acereason%2Dnemotron%3A+Advancing+math+and+code+reasoning+through+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2505%2E16400%2C+2)
8. K. Cheng, Q. Sun, Y. Chu, F. Xu, Y. Li, J. Zhang, and Z. Wu. [Seeclick: Harnessing gui grounding for advanced visual gui agents](https://md.research.elamur.ai/free/en/papers/cheng-2024-seeclick-harnessing-gui-grounding-advanced/). arXiv preprint arXiv:2401.10935, 2024.
9. C. Chou, L. Dunlap, K. Mashita, K. Mandal, T. Darrell, I. Stoica, J. Gonzalez, and W.-L. Chiang. Visionarena: 230k real world user-vlm conversations with preference labels. ArXiv, abs/2412.08687, 2024. URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 274655992 . [↗](https://api.semanticscholar.org/CorpusID:)
10. W. Dai, N. Lee, B. Wang, Z. Yang, Z. Liu, J. Barker, T. Rintamaki, M. Shoeybi, B. Catanzaro, and W. Ping. [Nvlm: Open frontier-class multimodal llms](https://md.research.elamur.ai/free/en/papers/dai-2024-nvlm-open-frontier-class-multimodal/). arXiv preprint, 2024.
11. M. Deitke, C. Clark, S. Lee, R. Tripathi, Y. Yang, J. S. Park, M. Salehi, N. Muennighoff, K. Lo, L. Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. ArXiv preprint, abs/2409.[17146, 2024](https://md.research.elamur.ai/free/en/papers/deitke-2024-molmo-pixmo-open-weights-open/).
12. X. Du, Y. Yao, K. Ma, B. Wang, T. Zheng, K. Zhu, M. Liu, Y. Liang, X. Jin, Z. Wei, et al. [Supergpqa: Scaling llm evaluation across 285 graduate disciplines](https://md.research.elamur.ai/free/en/papers/team-2025-supergpqa-scaling-llm-evaluation-across/). ArXiv preprint, abs/2502.14739, 2025. URL https://arxiv.org/abs/2502.14739 . [↗](https://arxiv.org/abs/2502.14739)
13. D. Dua, Y. Wang, P. Dasigi, G. Stanovsky, S. Singh, and M. Gardner. [DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/). In J. Burstein, C. Doran, and T. Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa…
14. C. Fu, Y. Dai, Y. Luo, L. Li, S. Ren, R. Zhang, Z. Wang, C. Zhou, Y. Shen, M. Zhang, et al. [Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis](https://md.research.elamur.ai/free/en/papers/fu-2024-video-mme-first-ever-comprehensive/). arXiv preprint arXiv:2405.21075, 2024a.
15. X. Fu, Y. Hu, B. Li, Y. Feng, H. Wang, X. Lin, D. Roth, N. A. Smith, W.-C. Ma, and R. Krishna. [Blink: Multimodal large language models can see but not perceive](https://md.research.elamur.ai/free/en/papers/fu-2024-blink-multimodal-large-language-models/). In European Conference on Computer Vision, pages 148–166. Springer, 2024b.
16. J. Gao, C. Sun, Z. Yang, and R. Nevatia. Tall: Temporal activity localization via language query. In Proceedings of the IEEE international conference on computer vision, pages 5267–5275, 2017. [find](https://scholar.google.com/scholar?q=J%2E+Gao%2C+C%2E+Sun%2C+Z%2E+Yang%2C+and+R%2E+Nevatia%2E+Tall%3A+Temporal+activity+localization+via+language+query%2E+In+Proceedings+of+the+IEEE+international+conference+on+computer+vision%2C+pages+5267%E2%80%935275%2C+2017%2E)
17. C. He, R. Luo, Y. Bai, S. Hu, Z. L. Thai, J. Shen, J. Hu, X. Han, Y. Huang, Y. Zhang, et al. [Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). arXiv preprint arXiv:2402.14008, 2024.
18. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). ArXiv preprint, abs/2103.03874, 2021. URL https://arxiv.org/abs/2103.03874 . [↗](https://arxiv.org/abs/2103.03874)
19. K. Hu, P. Wu, F. Pu, W. Xiao, Y. Zhang, X. Yue, B. Li, and Z. Liu. Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos. arXiv preprint arXiv:2501.13826, 2025. — [arXiv:2501.13826](https://arxiv.org/abs/2501.13826) [find](https://scholar.google.com/scholar?q=K%2E+Hu%2C+P%2E+Wu%2C+F%2E+Pu%2C+W%2E+Xiao%2C+Y%2E+Zhang%2C+X%2E+Yue%2C+B%2E+Li%2C+and+Z%2E+Liu%2E+Video%2Dmmmu%3A+Evaluating+knowledge+acquisition+from+multi%2Ddiscipline+professional+videos%2E+arXiv+preprint+arXiv%3A2501%2E13826%2C+2025%2E+%E2%80%94+%5BarX)
20. D. Jiang, X. He, H. Zeng, C. Wei, M. Ku, Q. Liu, and W. Chen. [Mantis: Interleaved multi-image instruction tuning](https://md.research.elamur.ai/free/en/papers/jiang-2024-mantis-interleaved-multi-image-instruction/). arXiv preprint arXiv:2405.01483, 2024.
21. S. Karamcheti, S. Nair, A. Balakrishna, P. Liang, T. Kollar, and D. Sadigh. Prismatic vlms: Investigating the design space of visually-conditioned language models. In ICML, 2024. [find](https://scholar.google.com/scholar?q=S%2E+Karamcheti%2C+S%2E+Nair%2C+A%2E+Balakrishna%2C+P%2E+Liang%2C+T%2E+Kollar%2C+and+D%2E+Sadigh%2E+Prismatic+vlms%3A+Investigating+the+design+space+of+visually%2Dconditioned+language+models%2E+In+ICML%2C+2024%2E)
22. S. Kazemzadeh, V. Ordonez, M. Matten, and T. Berg. [Referitgame: Referring to objects in photographs of natural scenes](https://md.research.elamur.ai/free/en/papers/kazemzadeh-2014-referitgame-referring-objects-photographs-natural/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 787–798, 2014.
23. A. Kembhavi, M. Salvato, E. Kolve, M. Seo, H. Hajishirzi, and A. Farhadi. [A diagram is worth a dozen images](https://md.research.elamur.ai/free/en/papers/kembhavi-2016-diagram-worth-dozen-images/). In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 235–251.
24. N. Lambert, J. Morrison, V. Pyatkin, S. Huang, H. Ivison, F. Brahman, L. J. V. Miranda, A. Liu, N. Dziri, S. Lyu, Y. Gu, S. Malik, V. Graf, J. D. Hwang, J. Yang, R. L. Bras, O. Tafjord, C. Wilhelm, L. Soldaini, N. A. Smith, Y. Wang, P. Dasigi, and H. Hajishirzi. Tulu 3: Pushing frontiers in open lan… — [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
25. B. Li, Y. Ge, Y. Chen, Y. Ge, R. Zhang, and Y. Shan. Seed-bench-2-plus: Benchmarking multimodal large language models with text-rich visual comprehension. arXiv preprint arXiv:2404.16790, 2024a. — [arXiv:2404.16790](https://arxiv.org/abs/2404.16790) [find](https://scholar.google.com/scholar?q=B%2E+Li%2C+Y%2E+Ge%2C+Y%2E+Chen%2C+Y%2E+Ge%2C+R%2E+Zhang%2C+and+Y%2E+Shan%2E+Seed%2Dbench%2D2%2Dplus%3A+Benchmarking+multimodal+large+language+models+with+text%2Drich+visual+comprehension%2E+arXiv+preprint+arXiv%3A2404%2E16790%2C+2024a%2E+%E2%80%94+%5Bar)
26. K. Li, Z. Meng, H. Lin, Z. Luo, Y. Tian, J. Ma, Z. Huang, and T.-S. Chua. Screenspot-pro: Gui grounding for professional high-resolution computer use. arXiv preprint arXiv:2504.07981, 2025. — [arXiv:2504.07981](https://arxiv.org/abs/2504.07981) [find](https://scholar.google.com/scholar?q=K%2E+Li%2C+Z%2E+Meng%2C+H%2E+Lin%2C+Z%2E+Luo%2C+Y%2E+Tian%2C+J%2E+Ma%2C+Z%2E+Huang%2C+and+T%2E%2DS%2E+Chua%2E+Screenspot%2Dpro%3A+Gui+grounding+for+professional+high%2Dresolution+computer+use%2E+arXiv+preprint+arXiv%3A2504%2E07981%2C+2025%2E+%E2%80%94+%5BarXiv%3A2)
27. L. Li, Y. Wei, Z. Xie, X. Yang, Y. Song, P. Wang, C. An, T. Liu, S. Li, B. Y. Lin, L. Kong, and Q. Liu. Vlrewardbench: A challenging benchmark for vision-language generative reward models. ArXiv, abs/2411.17451, 2024b. URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 274281459 . — [arXiv:2411.17451](https://arxiv.org/abs/2411.17451) [↗](https://api.semanticscholar.org/CorpusID:)
28. H. Liu, C. Li, Q. Wu, and Y. J. Lee. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). In NeurIPS, 2023.
29. J. Liu, Y. Song, B. Y. Lin, W. Lam, G. Neubig, Y. Li, and X. Yue. Visualwebbench: How far have multimodal llms evolved in web page understanding and grounding? arXiv preprint arXiv:2404.05955, 2024a. [find](https://scholar.google.com/scholar?q=J%2E+Liu%2C+Y%2E+Song%2C+B%2E+Y%2E+Lin%2C+W%2E+Lam%2C+G%2E+Neubig%2C+Y%2E+Li%2C+and+X%2E+Yue%2E+Visualwebbench%3A+How+far+have+multimodal+llms+evolved+in+web+page+understanding+and+grounding%3F+arXiv+preprint+arXiv%3A2404%2E05955%2C+2024a%2E)
30. Y. Liu, H. Duan, Y. Zhang, B. Li, S. Zhang, W. Zhao, Y. Yuan, J. Wang, C. He, Z. Liu, et al. [Mmbench: Is your multi-modal model an all-around player?](https://md.research.elamur.ai/free/en/papers/liu-2023-mmbench-multi-modal-model-all/) In European conference on computer vision, pages 216–233. Springer, 2024b.
31. Y. Liu, Z. Li, M. Huang, B. Yang, W. Yu, C. Li, X.-C. Yin, C.-L. Liu, L. Jin, and X. Bai. [Ocrbench: on the hidden mystery of ocr in large multimodal models](https://md.research.elamur.ai/free/en/papers/liu-2023-ocrbench-hidden-mystery-ocr-large/). Science China Information Sciences, 67 (12):220102, 2024c.
32. P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K.-W. Chang, M. Galley, and J. Gao. [Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/). arXiv preprint arXiv:2310.02255, 2023.
33. MAA. American invitational mathematics examination - aime. In American Invitational Mathematics Examination - AIME, 2024. URL [https://maa.org/math-competition](https://maa.org/math-competition) s/american-invitational-mathematics-examination-aime . [↗](https://maa.org/math-competition)
34. MAA. American invitational mathematics examination - aime. In American Invitational Mathematics Examination - AIME, 2025. URL [https://maa.org/math-competition](https://maa.org/math-competition) s/american-invitational-mathematics-examination-aime . [↗](https://maa.org/math-competition)
35. K. Mangalam, R. Akshulakov, and J. Malik. [Egoschema: A diagnostic benchmark for very long-form video language understanding](https://md.research.elamur.ai/free/en/papers/mangalam-2023-egoschema-diagnostic-benchmark-very-long/). Advances in Neural Information Processing Systems, 36: 46212–46244, 2023.
36. A. Masry, D. X. Long, J. Q. Tan, S. Joty, and E. Hoque. [Chartqa: A benchmark for question answering about charts with visual and logical reasoning](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/). arXiv preprint arXiv:2203.10244, 2022.
37. M. Mathew, D. Karatzas, and C. Jawahar. [Docvqa: A dataset for vqa on document images](https://md.research.elamur.ai/free/en/papers/mathew-2020-docvqa-dataset-vqa-document-images/). In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209, 2021.
38. M. Mathew, V. Bagal, R. Tito, D. Karatzas, E. Valveny, and C. Jawahar. Infographicvqa. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1697–1706, 2022. [find](https://scholar.google.com/scholar?q=M%2E+Mathew%2C+V%2E+Bagal%2C+R%2E+Tito%2C+D%2E+Karatzas%2C+E%2E+Valveny%2C+and+C%2E+Jawahar%2E+Infographicvqa%2E+In+Proceedings+of+the+IEEE%2FCVF+Winter+Conference+on+Applications+of+Computer+Vision%2C+pages+1697%E2%80%931706%2C+2022%2E)
39. V. K. Nagaraja, V. I. Morariu, and L. S. Davis. Modeling context between objects for referring expression understanding. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 792–807. [find](https://scholar.google.com/scholar?q=V%2E+K%2E+Nagaraja%2C+V%2E+I%2E+Morariu%2C+and+L%2E+S%2E+Davis%2E+Modeling+context+between+objects+for+referring+expression+understanding%2E+In+Computer+Vision%E2%80%93ECCV+2016%3A+14th+European+Conference%2C+Amsterdam%2C+The+Netherla)
40. OpenAI. Computer-using agent: Introducing a universal interface for ai to interact with the digital world. 2025. URL [https://openai.com/index/computer-using-agent](https://openai.com/index/computer-using-agent) . [↗](https://openai.com/index/computer-using-agent)
41. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. F. Christiano, J. Leike, and R. Lowe. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/).…
42. P. Padlewski, M. Bain, M. Henderson, Z. Zhu, N. Relan, H. Pham, D. Ong, K. Aleksiev, A. Ormazabal, S. Phua, et al. [Vibe-eval: A hard evaluation suite for measuring progress of multimodal language models](https://md.research.elamur.ai/free/en/papers/padlewski-2024-vibe-eval-hard-evaluation-suite/). ArXiv preprint, abs/2405.02287, 2024.
43. R. Paiss, A. Ephrat, O. Tov, S. Zada, I. Mosseri, M. Irani, and T. Dekel. Teaching clip to count to ten. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3170–3180, 2023. [find](https://scholar.google.com/scholar?q=R%2E+Paiss%2C+A%2E+Ephrat%2C+O%2E+Tov%2C+S%2E+Zada%2C+I%2E+Mosseri%2C+M%2E+Irani%2C+and+T%2E+Dekel%2E+Teaching+clip+to+count+to+ten%2E+In+Proceedings+of+the+IEEE%2FCVF+International+Conference+on+Computer+Vision%2C+pages+3170%E2%80%933180%2C+20)
44. R. Qiao, Q. Tan, G. Dong, M. Wu, C. Sun, X. Song, Z. GongQue, S. Lei, Z. Wei, M. Zhang, et al. [We-math: Does your large multimodal model achieve human-like mathematical reasoning?](https://md.research.elamur.ai/free/en/papers/qiao-2024-math-large-multimodal-model-achieve/) arXiv preprint arXiv:2407.01284, 2024.
45. Y. Qin, Y. Ye, J. Fang, H. Wang, S. Liang, S. Tian, J. Zhang, J. Li, Y. Li, S. Huang, et al. [Ui-tars: Pioneering automated gui interaction with native agents](https://md.research.elamur.ai/free/en/papers/qin-2025-ui-tars-pioneering-automated/). arXiv preprint arXiv:2501.12326, 2025a.
46. Y. Qin, Y. Ye, J. Fang, H. Wang, S. Liang, S. Tian, J. Zhang, J. Li, Y. Li, S. Huang, et al. [Ui-tars: Pioneering automated gui interaction with native agents](https://md.research.elamur.ai/free/en/papers/qin-2025-ui-tars-pioneering-automated/). arXiv preprint arXiv:2501.12326, 2025b.
47. P. Rahmanzadehgervi, L. Bolton, M. R. Taesiri, and A. T. Nguyen. Vision language models are blind: Failing to translate detailed visual features into words, 2025. URL https://arxiv.org/abs/2407.06581 . [↗](https://arxiv.org/abs/2407.06581)
48. D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman. [Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). In First Conference on Language Modeling, 2024.
49. H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese. Generalized intersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 658–666, 2019. [find](https://scholar.google.com/scholar?q=H%2E+Rezatofighi%2C+N%2E+Tsoi%2C+J%2E+Gwak%2C+A%2E+Sadeghian%2C+I%2E+Reid%2C+and+S%2E+Savarese%2E+Generalized+intersection+over+union%3A+A+metric+and+a+loss+for+bounding+box+regression%2E+In+Proceedings+of+the+IEEE%2FCVF+conferenc)
50. Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300, 2024.
51. G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). ArXiv preprint, abs/2409.19256, 2024. URL https://arxiv.org/abs/2409.19256 . [↗](https://arxiv.org/abs/2409.19256)
52. C. Team. [Chameleon: Mixed-modal early-fusion foundation models](https://md.research.elamur.ai/free/en/papers/team-2024-chameleon-mixed-modal-early-fusion/). arXiv:2405.09818, 2024. arXiv preprint P. Tong, E. Brown, P. Wu, S. Woo, A. J. V. IYER, S. C. Akula, S. Yang, J. Yang, M. Middepogu, Z. Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. Advances in Ne…
53. K. Wang, J. Pan, W. Shi, Z. Lu, H. Ren, A. Zhou, M. Zhan, and H. Li. [Measuring multimodal mathematical reasoning with math-vision dataset](https://md.research.elamur.ai/free/en/papers/wang-2024-measuring-multimodal-mathematical-reasoning-math/). Advances in Neural Information Processing Systems, 37:95095–95169, 2024a.
54. Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang, T. Li, M. Ku, K. Wang, A. Zhuang, R. Fan, X. Yue, and W. Chen. [Mmlu-pro: A more robust and challenging multi-task language understanding benchmark](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/). In A. Globersons, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J…
55. Y. Wang, B. Xu, Z. Yue, Z. Xiao, Z. Wang, L. Zhang, D. Yang, W. Wang, and Q. Jin. Timezero: Temporal video grounding with reasoning-guided lvlm. arXiv preprint arXiv:2503.13377, 2025. — [arXiv:2503.13377](https://arxiv.org/abs/2503.13377) [find](https://scholar.google.com/scholar?q=Y%2E+Wang%2C+B%2E+Xu%2C+Z%2E+Yue%2C+Z%2E+Xiao%2C+Z%2E+Wang%2C+L%2E+Zhang%2C+D%2E+Yang%2C+W%2E+Wang%2C+and+Q%2E+Jin%2E+Timezero%3A+Temporal+video+grounding+with+reasoning%2Dguided+lvlm%2E+arXiv+preprint+arXiv%3A2503%2E13377%2C+2025%2E+%E2%80%94+%5BarXiv%3A2503%2E13)
56. Z. Wang, M. Xia, L. He, H. Chen, Y. Liu, R. Zhu, K. Liang, X. Wu, H. Liu, S. Malladi, et al. [Charxiv: Charting gaps in realistic chart understanding in multimodal llms](https://md.research.elamur.ai/free/en/papers/wang-2024-charxiv-charting-gaps-realistic-chart/). Advances in Neural Information Processing Systems, 37:113569–113697, 2024c.
57. P. Wu and S. Xie. [V*: Guided visual search as a core mechanism in multimodal llms](https://md.research.elamur.ai/free/en/papers/wu-2023-v-guided-visual-search-core/), 2023. URL https://arxiv.org/abs/2312.14135 . [↗](https://arxiv.org/abs/2312.14135)
58. Z. Wu, Z. Wu, F. Xu, Y. Wang, Q. Sun, C. Jia, K. Cheng, Z. Ding, L. Chen, P. P. Liang, et al. [Os-atlas: A foundation action model for generalist gui agents](https://md.research.elamur.ai/free/en/papers/wu-2024-os-atlas-foundation-action-model/). arXiv preprint arXiv:2410.23218, 2024.
59. Y. Xiao, E. Sun, T. Liu, and W. Wang. Logicvista: Multimodal llm logical reasoning benchmark in visual contexts. arXiv preprint arXiv:2407.04973, 2024. — [arXiv:2407.04973](https://arxiv.org/abs/2407.04973) [find](https://scholar.google.com/scholar?q=Y%2E+Xiao%2C+E%2E+Sun%2C+T%2E+Liu%2C+and+W%2E+Wang%2E+Logicvista%3A+Multimodal+llm+logical+reasoning+benchmark+in+visual+contexts%2E+arXiv+preprint+arXiv%3A2407%2E04973%2C+2024%2E+%E2%80%94+%5BarXiv%3A2407%2E04973%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2407%2E)
60. L.-C.-T. Xiaomi. Mimo: Unlocking the reasoning potential of language model–from pretraining to posttraining. arXiv preprint arXiv:2505.07608, 2025. [find](https://scholar.google.com/scholar?q=L%2E%2DC%2E%2DT%2E+Xiaomi%2E+Mimo%3A+Unlocking+the+reasoning+potential+of+language+model%E2%80%93from+pretraining+to+posttraining%2E+arXiv+preprint+arXiv%3A2505%2E07608%2C+2025%2E)
61. T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, et al. [Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments](https://md.research.elamur.ai/free/en/papers/xie-2024-osworld-benchmarking-multimodal-agents-open/). Advances in Neural Information Processing Systems, 37:52040–52094, 2024.
62. T. Xie, J. Deng, X. Li, J. Yang, H. Wu, J. Chen, W. Hu, X. Wang, Y. Xu, Z. Wang, Y. Xu, J. Wang, D. Sahoo, T. Yu, and C. Xiong. Scaling computer-use grounding via user interface decomposition and synthesis, 2025. URL https://arxiv.org/abs/2505.13227 . [↗](https://arxiv.org/abs/2505.13227)
63. H. Xu, S. Xie, X. E. Tan, P.-Y. Huang, R. Howes, V. Sharma, S.-W. Li, G. Ghosh, L. Zettlemoyer, and C. Feichtenhofer. Demystifying clip data. arXiv preprint arXiv:2309.16671, 2023. [find](https://scholar.google.com/scholar?q=H%2E+Xu%2C+S%2E+Xie%2C+X%2E+E%2E+Tan%2C+P%2E%2DY%2E+Huang%2C+R%2E+Howes%2C+V%2E+Sharma%2C+S%2E%2DW%2E+Li%2C+G%2E+Ghosh%2C+L%2E+Zettlemoyer%2C+and+C%2E+Feichtenhofer%2E+Demystifying+clip+data%2E+arXiv+preprint+arXiv%3A2309%2E16671%2C+2023%2E)
64. Y. Xu, Z. Wang, J. Wang, D. Lu, T. Xie, A. Saha, D. Sahoo, T. Yu, and C. Xiong. Aguvis: Unified pure vision agents for autonomous gui interaction. arXiv preprint arXiv:2412.04454, 2024. — [arXiv:2412.04454](https://arxiv.org/abs/2412.04454) [find](https://scholar.google.com/scholar?q=Y%2E+Xu%2C+Z%2E+Wang%2C+J%2E+Wang%2C+D%2E+Lu%2C+T%2E+Xie%2C+A%2E+Saha%2C+D%2E+Sahoo%2C+T%2E+Yu%2C+and+C%2E+Xiong%2E+Aguvis%3A+Unified+pure+vision+agents+for+autonomous+gui+interaction%2E+arXiv+preprint+arXiv%3A2412%2E04454%2C+2024%2E+%E2%80%94+%5BarXiv%3A2412%2E)
65. J. Ye, Z. Xie, L. Zheng, J. Gao, Z. Wu, X. Jiang, Z. Li, and L. Kong. Dream 7b, 2025. URL [https://hkunlp.github.io/blog/2025/dream](https://hkunlp.github.io/blog/2025/dream) . [↗](https://hkunlp.github.io/blog/2025/dream)
66. L. Yu, P. Poirson, S. Yang, A. C. Berg, and T. L. Berg. [Modeling context in referring expressions](https://md.research.elamur.ai/free/en/papers/yu-2016-modeling-context-referring-expressions/). In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 69–85.
67. X. Yue, Y. Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y. Sun, et al. [Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556–9567, 2…
68. X. Yue, Y. Ni, T. Zheng, K. Zhang, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y. Sun, C. Wei, B. Yu, R. Yuan, R. Sun, M. Yin, B. Zheng, Z. Yang, Y. Liu, W. Huang, H. Sun, Y. Su, and W. Chen. [MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/). In CVP…
69. X. Yue, T. Zheng, Y. Ni, Y. Wang, K. Zhang, S. Tong, Y. Sun, B. Yu, G. Zhang, H. Sun, et al. [Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark](https://md.research.elamur.ai/free/en/papers/yue-2024-mmmu-pro-more-robust-multi/). arXiv preprint arXiv:2409.02813, 2024c.
70. K. Zhang, B. Li, P. Zhang, F. Pu, J. A. Cahyono, K. Hu, S. Liu, Y. Zhang, J. Yang, C. Li, and Z. Liu. [Lmms-eval: Reality check on the evaluation of large multimodal models](https://md.research.elamur.ai/free/en/papers/zhang-2025-lmms-eval-reality-check-evaluation/), 2024a. URL https://arxiv.org/abs/2407.12772 . [↗](https://arxiv.org/abs/2407.12772)
71. R. Zhang, D. Jiang, Y. Zhang, H. Lin, Z. Guo, P. Qiu, A. Zhou, P. Lu, K.-W. Chang, P. Gao, et al. [Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems?](https://md.research.elamur.ai/free/en/papers/zhang-2024-mathverse-multi-modal-llm-truly/) ArXiv preprint, abs/2403.14624, 2024b.
72. Y.-F. Zhang, H. Zhang, H. Tian, C. Fu, S. Zhang, J. Wu, F. Li, K. Wang, Q. Wen, Z. Zhang, et al. Mme-realworld: Could your multimodal llm challenge high-resolution real-world scenarios that are difficult for humans? arXiv preprint arXiv:2408.13257, 2024c. — [arXiv:2408.13257](https://arxiv.org/abs/2408.13257) [find](https://scholar.google.com/scholar?q=Y%2E%2DF%2E+Zhang%2C+H%2E+Zhang%2C+H%2E+Tian%2C+C%2E+Fu%2C+S%2E+Zhang%2C+J%2E+Wu%2C+F%2E+Li%2C+K%2E+Wang%2C+Q%2E+Wen%2C+Z%2E+Zhang%2C+et+al%2E+Mme%2Drealworld%3A+Could+your+multimodal+llm+challenge+high%2Dresolution+real%2Dworld+scenarios+that+are+diffic)
73. L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In NeurIPS, 2023.
74. J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou. [Instruction-following evaluation for large language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/), 2023. URL https://arxiv.org/abs/2311.07911 . [↗](https://arxiv.org/abs/2311.07911)
75. B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023. [find](https://scholar.google.com/scholar?q=B%2E+Zitkovich%2C+T%2E+Yu%2C+S%2E+Xu%2C+P%2E+Xu%2C+T%2E+Xiao%2C+F%2E+Xia%2C+J%2E+Wu%2C+P%2E+Wohlhart%2C+S%2E+Welker%2C+A%2E+Wahid%2C+et+al%2E+Rt%2D2%3A+Vision%2Dlanguage%2Daction+models+transfer+web+knowledge+to+robotic+control%2E+In+Conference+on+Robo)
76. C. Zou, X. Guo, R. Yang, J. Zhang, B. Hu, and H. Zhang. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models. arXiv preprint arXiv:2411.00836, 2024. [find](https://scholar.google.com/scholar?q=C%2E+Zou%2C+X%2E+Guo%2C+R%2E+Yang%2C+J%2E+Zhang%2C+B%2E+Hu%2C+and+H%2E+Zhang%2E+Dynamath%3A+A+dynamic+visual+benchmark+for+evaluating+mathematical+reasoning+robustness+of+vision+language+models%2E+arXiv+preprint+arXiv%3A2411%2E00836)

*Updated 2 November 2026*