---
title: "SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis"
description: "SynthRL tackles a data bottleneck in reinforcement learning with verifiable rewards (RLVR) for… 72 references, cited by 3 papers here."
authors: ["Zijian Wu", "Jinjie Ni", "Xiangyan Liu", "Zichen Liu", "Hang Yan", "Michael Qizhe Shieh"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.2107/"
doi: "10.18653/v1/2026.findings-acl.2107"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2026-synthrl-scaling-visual-reasoning-verifiable/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2026-synthrl-scaling-visual-reasoning-verifiable/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/) / SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis

# SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis

**Authors:** [Zijian Wu](https://md.research.elamur.ai/free/en/authors/zijian-wu/), [Jinjie Ni](https://md.research.elamur.ai/free/en/authors/jinjie-ni/), Xiangyan Liu, [Zichen Liu](https://md.research.elamur.ai/free/en/authors/zichen-liu/), [Hang Yan](https://md.research.elamur.ai/free/en/authors/hang-yan/) and Michael Qizhe Shieh · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.2107/) · [PDF](https://aclanthology.org/2026.findings-acl.2107.pdf) · [DOI 10.18653/v1/2026.findings-acl.2107](https://doi.org/10.18653/v1/2026.findings-acl.2107) · [OpenAlex](https://openalex.org/W4415130657)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/wu-2026-synthrl-scaling-visual-reasoning-verifiable/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
SynthRL tackles a data bottleneck in reinforcement learning with verifiable rewards (RLVR) for vision-language models. Curated question–answer pairs are costly, and data selection alone cannot go beyond the pool that already exists. The authors instead generate new training items. Their pipeline has three steps:
1. **Seed selection.** Monte Carlo rollouts of the target model (Qwen2.5-VL-7B-Instruct) identify seed questions it already solves reliably.
2. **Rewriting.** A strong general VLM rewrites each seed into a harder version whose ground-truth answer stays the same.
3. **Verification.** The target model checks that each new question is valid, keeps the original answer and is actually harder.

On the MMK12 dataset, this turned roughly 8K seeds into more than 3.3K verified harder questions. Models were trained with GRPO on the seeds plus this synthetic data (A-MMK12). They beat seed-only training on five out-of-domain visual math benchmarks, and the gains held at every data scale tested. The improvements concentrate on the hardest test items. This suggests that answer-preserving, verified difficulty escalation is a cheap, automatic way to stretch scarce RL data.

## Key points
- **Seed selection:** each seed gets N = 16 rollouts, and items with a pass count of at least 12 are chosen for rewriting.
- **Data volume:** 8,072 MMK12 seeds yielded about 3,380 synthesized questions. Experiments compare 2K, 4K and 8K seed scales.
- **Benchmark gains at 8K seeds:** MathVerse improves by 1.9%, WeMath by 2.0% and DynaMath by 1.3%. MathVision and MathVista were also evaluated. SynthRL-7B reaches about 58.0% average accuracy.
- **Gains by difficulty:** tiers were built with Bradley-Terry/Elo ratings from pairwise judgments by Gemini-2.0-Flash-001. At 8K, A-MMK12 scores +1.6% on hard items and +1.7% on medium items, against −0.5% on easy ones.
- **Verifier ablation at 4K:** the full verifier reaches 57.2% average accuracy. Using a non-target verifier (Gemini-2.0-Flash-001) drops this to 55.7%, single-pass checking gives 56.5%, and no verification gives 55.8%. Code, models and data are released on GitHub and Hugging Face.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 207 works · 39,576 citations
- [Chinese University of Hong Kong](https://md.research.elamur.ai/free/en/universities/chinese-university-of-hong-kong/) — University, Hong Kong, China · 159 works · 16,154 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Zijian Wu, Jinjie Ni, Xiangyan Liu, Zichen Liu, Hang Yan and Michael Qizhe Shieh. 2026. SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wu2026synthrl,
  title = {{SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis}},
  author = {Zijian Wu, Jinjie Ni, Xiangyan Liu, Zichen Liu, Hang Yan and Michael Qizhe Shieh},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2107},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2107}
}
```

- [← previous StressTest: Can YOUR Speech LM Handle the Stress?](https://md.research.elamur.ai/free/en/papers/yosha-2026-stresstest-can-your-speech-lm/)
- [next → TabularMath: Understanding Math Reasoning over Tables with Large Language Models](https://md.research.elamur.ai/free/en/papers/tian-2026-tabularmath-understanding-math-reasoning-over/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/)

## References

72 references, 37 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, and 1 others. 2025. [Phi-4-reasoning technical report](https://md.research.elamur.ai/free/en/papers/abdin-2025-phi-4-reasoning-technical-report/). arXiv preprint arXiv:2504.21318.
2. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, and 1 others. 2022. [Flamingo: a visual language model for few-shot learning](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/). Advances in neural information processing systems, 35:23716– 23736.
3. Anthropic. 2025. Claude 3.7 sonnet. [https://www.anthropic.com](https://www.anthropic.com). [↗](https://www.anthropic.com)
4. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
5. Tianyi Bai, Hao Liang, Binwang Wan, Yanran Xu, Xi Li, Shiyu Li, Ling Yang, Bozhou Li, Yifan Wang, Bin Cui, and 1 others. 2024. A survey of multimodal large language model from a data-centric perspective. arXiv preprint arXiv:2405.16640. — [arXiv:2405.16640](https://arxiv.org/abs/2405.16640) [find](https://scholar.google.com/scholar?q=Tianyi+Bai%2C+Hao+Liang%2C+Binwang+Wan%2C+Yanran+Xu%2C+Xi+Li%2C+Shiyu+Li%2C+Ling+Yang%2C+Bozhou+Li%2C+Yifan+Wang%2C+Bin+Cui%2C+and+1+others%2E+2024%2E+A+survey+of+multimodal+large+language+model+from+a+data%2Dcentric+perspecti)
6. Ralph Allan Bradley and Milton E Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324– 345. [find](https://scholar.google.com/scholar?q=Ralph+Allan+Bradley+and+Milton+E+Terry%2E+1952%2E+Rank+analysis+of+incomplete+block+designs%3A+I%2E+the+method+of+paired+comparisons%2E+Biometrika%2C+39%283%2F4%29%3A324%E2%80%93+345%2E)
7. Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. 2025a. [find](https://scholar.google.com/scholar?q=Hardy+Chen%2C+Haoqin+Tu%2C+Fali+Wang%2C+Hui+Liu%2C+Xianfeng+Tang%2C+Xinya+Du%2C+Yuyin+Zhou%2C+and+Cihang+Xie%2E+2025a%2E)
8. [Sft or rl? an early investigation into training r1-like reasoning large vision-language models](https://md.research.elamur.ai/free/en/papers/chen-2025-sft-rl-early-investigation-training/). [https://github.com/UCSC-VLAA/VLAA-Thinking](https://github.com/UCSC-VLAA/VLAA-Thinking). [↗](https://github.com/UCSC-VLAA/VLAA-Thinking)
9. Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. 2025b. R1-v: Reinforcing super generalization ability in vision-language models with less than $3. [https://github.com/Deep-Agent/R1-V](https://github.com/Deep-Agent/R1-V). Accessed: 2025-02-02. [↗](https://github.com/Deep-Agent/R1-V)
10. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, and 1 others. 2024. [Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling](https://md.research.elamur.ai/free/en/papers/chen-2024-expanding-performance-boundaries-open/). arXiv preprint arXiv:2412.05271.
11. Hejie Cui, Lingjun Mao, Xin Liang, Jieyu Zhang, Hui Ren, Quanzheng Li, Xiang Li, and Carl Yang. 2024. Biomedical visual instruction tuning with clinician preference alignment. Preprint, arXiv:2406.13173. — [arXiv:2406.13173](https://arxiv.org/abs/2406.13173) [find](https://scholar.google.com/scholar?q=Hejie+Cui%2C+Lingjun+Mao%2C+Xin+Liang%2C+Jieyu+Zhang%2C+Hui+Ren%2C+Quanzheng+Li%2C+Xiang+Li%2C+and+Carl+Yang%2E+2024%2E+Biomedical+visual+instruction+tuning+with+clinician+preference+alignment%2E+Preprint%2C+arXiv%3A2406%2E131)
12. Linger Deng, Yuliang Liu, Bohan Li, Dongliang Luo, Liang Wu, Chengquan Zhang, Pengyuan Lyu, Ziyang Zhang, Gang Zhang, Errui Ding, Yingying Zhu, and Xiang Bai. 2024. R-cot: Reverse chain-of-thought problem generation for geometric reasoning in large multimodal models. Preprint, arXiv:2410.17885. — [arXiv:2410.17885](https://arxiv.org/abs/2410.17885) [find](https://scholar.google.com/scholar?q=Linger+Deng%2C+Yuliang+Liu%2C+Bohan+Li%2C+Dongliang+Luo%2C+Liang+Wu%2C+Chengquan+Zhang%2C+Pengyuan+Lyu%2C+Ziyang+Zhang%2C+Gang+Zhang%2C+Errui+Ding%2C+Yingying+Zhu%2C+and+Xiang+Bai%2E+2024%2E+R%2Dcot%3A+Reverse+chain%2Dof%2Dthought+pro)
13. Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. 2025. Openvlthinker: An early exploration to complex vision-language reasoning via iterative self-improvement. Preprint, arXiv:2503.17352. — [arXiv:2503.17352](https://arxiv.org/abs/2503.17352) [find](https://scholar.google.com/scholar?q=Yihe+Deng%2C+Hritik+Bansal%2C+Fan+Yin%2C+Nanyun+Peng%2C+Wei+Wang%2C+and+Kai%2DWei+Chang%2E+2025%2E+Openvlthinker%3A+An+early+exploration+to+complex+vision%2Dlanguage+reasoning+via+iterative+self%2Dimprovement%2E+Preprint%2C+ar)
14. Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. 2023. [How abilities in large language models are affected by supervised fine-tuning data composition](https://md.research.elamur.ai/free/en/papers/dong-2024-how-abilities-large-language-models/). arXiv preprint arXiv:2310.05492.
15. Yifan Du, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Jinpeng Wang, Chuyuan Wang, Mingchen Cai, Ruihua Song, and Ji-Rong Wen. 2025. [What makes for good visual instructions? synthesizing complex visual reasoning instructions for visual instruction tuning](https://md.research.elamur.ai/free/en/papers/du-2025-what-makes-good-visual-instructions/). Preprint, arXiv:2311.01487.
16. Lester R Ford Jr. 1957. Solution of a ranking problem from binary comparisons. The American Mathematical Monthly, 64(8P2):28–33. [find](https://scholar.google.com/scholar?q=Lester+R+Ford+Jr%2E+1957%2E+Solution+of+a+ranking+problem+from+binary+comparisons%2E+The+American+Mathematical+Monthly%2C+64%288P2%29%3A28%E2%80%9333%2E)
17. Gemini Team. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805.
18. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
19. Yufei Guo, Muzhe Guo, Juntao Su, Zhou Yang, Mengqiu Zhu, Hongfei Li, Mengyang Qiu, and Shuo Shuo Liu. 2024. [Bias in large language models: Origin, evaluation, and mitigation](https://md.research.elamur.ai/free/en/papers/guo-2024-bias-large-language-models-origin/). arXiv preprint arXiv:2411.10915.
20. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023b. [Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). In International conference on machine learning, pages 19730–19742. PMLR.
21. Bruce Hajek, Sewoong Oh, and Jiaming Xu. 2014. Minimax-optimal inference from partial rankings. Advances in Neural Information Processing Systems, 27. [find](https://scholar.google.com/scholar?q=Bruce+Hajek%2C+Sewoong+Oh%2C+and+Jiaming+Xu%2E+2014%2E+Minimax%2Doptimal+inference+from+partial+rankings%2E+Advances+in+Neural+Information+Processing+Systems%2C+27%2E)
22. Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, Lingpeng Kong, and Qi Liu. 2024d. Vlfeedback: A large-scale ai feedback dataset for large vision-language models alignment. Preprint, arXiv:2410.09421. [find](https://scholar.google.com/scholar?q=Lei+Li%2C+Zhihui+Xie%2C+Mukai+Li%2C+Shunian+Chen%2C+Peiyi+Wang%2C+Liang+Chen%2C+Yazheng+Yang%2C+Benyou+Wang%2C+Lingpeng+Kong%2C+and+Qi+Liu%2E+2024d%2E+Vlfeedback%3A+A+large%2Dscale+ai+feedback+dataset+for+large+vision%2Dlanguage)
23. Zizhao Hu, Mohammad Rostami, and Jesse Thomason. 2025. Multi-modal synthetic data training and model collapse: Insights from vlms and diffusion models. Preprint, arXiv:2505.08803. — [arXiv:2505.08803](https://arxiv.org/abs/2505.08803) [find](https://scholar.google.com/scholar?q=Zizhao+Hu%2C+Mohammad+Rostami%2C+and+Jesse+Thomason%2E+2025%2E+Multi%2Dmodal+synthetic+data+training+and+model+collapse%3A+Insights+from+vlms+and+diffusion+models%2E+Preprint%2C+arXiv%3A2505%2E08803%2E+%E2%80%94+%5BarXiv%3A2505%2E08803%5D)
24. Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Yao Hu, and Shaohui Lin. 2025. [Vision-r1: Incentivizing reasoning capability in multimodal large language models](https://md.research.elamur.ai/free/en/papers/huang-2025-vision-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2503.06749.
25. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). arXiv preprint arXiv:2410.21276.
26. Kimi Team. 2025a. [Kimi k1.5: Scaling reinforcement learning with llms](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/). Preprint, arXiv:2501.12599.
27. Kimi Team. 2025b. [Kimi-VL technical report](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-vl-technical-report/). Preprint, arXiv:2504.07491.
28. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
29. Miaomiao Li, Hao Chen, Yang Wang, Tingyuan Zhu, Weijia Zhang, Kaijie Zhu, Kam-Fai Wong, and Jindong Wang. 2025a. Understanding and mitigating the bias inheritance in llm-based data augmentation on downstream tasks. arXiv preprint arXiv:2502.04419. — [arXiv:2502.04419](https://arxiv.org/abs/2502.04419) [find](https://scholar.google.com/scholar?q=Miaomiao+Li%2C+Hao+Chen%2C+Yang+Wang%2C+Tingyuan+Zhu%2C+Weijia+Zhang%2C+Kaijie+Zhu%2C+Kam%2DFai+Wong%2C+and+Jindong+Wang%2E+2025a%2E+Understanding+and+mitigating+the+bias+inheritance+in+llm%2Dbased+data+augmentation+on+dow)
30. Xuefeng Li, Haoyang Zou, and Pengfei Liu. 2025b. Limr: Less is more for rl scaling. arXiv preprint arXiv:2502.11886. [find](https://scholar.google.com/scholar?q=Xuefeng+Li%2C+Haoyang+Zou%2C+and+Pengfei+Liu%2E+2025b%2E+Limr%3A+Less+is+more+for+rl+scaling%2E+arXiv+preprint+arXiv%3A2502%2E11886%2E)
31. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306.
32. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023a. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). Advances in neural information processing systems, 36:34892– 34916.
33. Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. 2023b. [What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-makes-good-data-alignment-comprehensive/). arXiv preprint arXiv:2312.15685.
34. Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, and Chunyuan Li. 2024a. Llava-next: Stronger llms supercharge multimodal capabilities in the wild. [find](https://scholar.google.com/scholar?q=Bo+Li%2C+Kaichen+Zhang%2C+Hao+Zhang%2C+Dong+Guo%2C+Renrui+Zhang%2C+Feng+Li%2C+Yuanhan+Zhang%2C+Ziwei+Liu%2C+and+Chunyuan+Li%2E+2024a%2E+Llava%2Dnext%3A+Stronger+llms+supercharge+multimodal+capabilities+in+the+wild%2E)
35. Xiangyan Liu, Jinjie Ni, Zijian Wu, Chao Du, Longxu Dou, Haonan Wang, Tianyu Pang, and Michael Qizhe Shieh. 2025a. Noisyrollout: Reinforcing visual reasoning with data augmentation. Preprint, arXiv:2504.13055. — [arXiv:2504.13055](https://arxiv.org/abs/2504.13055) [find](https://scholar.google.com/scholar?q=Xiangyan+Liu%2C+Jinjie+Ni%2C+Zijian+Wu%2C+Chao+Du%2C+Longxu+Dou%2C+Haonan+Wang%2C+Tianyu+Pang%2C+and+Michael+Qizhe+Shieh%2E+2025a%2E+Noisyrollout%3A+Reinforcing+visual+reasoning+with+data+augmentation%2E+Preprint%2C+arXiv%3A25)
36. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and 1 others. 2024b. [Llava-onevision: Easy visual task transfer](https://md.research.elamur.ai/free/en/papers/li-2024-llava-onevision-easy-visual-task/). arXiv preprint arXiv:2408.03326.
37. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025b. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/). arXiv preprint arXiv:2503.20783.
38. Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2023a. [Llava-med: Training a large language-and-vision assistant for biomedicine in one day](https://md.research.elamur.ai/free/en/papers/li-2023-llava-med-training-large-language/). Preprint, arXiv:2306.00890.
39. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. [Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/). arXiv preprint arXiv:2310.02255.
40. Huayang Li, Siheng Li, Deng Cai, Longyue Wang, Lemao Liu, Taro Watanabe, Yujiu Yang, and Shuming Shi. 2024c. Textbind: Multi-turn interleaved multimodal instruction-following in the wild. Preprint, arXiv:2309.08637. [find](https://scholar.google.com/scholar?q=Huayang+Li%2C+Siheng+Li%2C+Deng+Cai%2C+Longyue+Wang%2C+Lemao+Liu%2C+Taro+Watanabe%2C+Yujiu+Yang%2C+and+Shuming+Shi%2E+2024c%2E+Textbind%3A+Multi%2Dturn+interleaved+multimodal+instruction%2Dfollowing+in+the+wild%2E+Preprint%2C+ar)
41. Ruilin Luo, Zhuofan Zheng, Yifan Wang, Yiyao Yu, Xinzhe Ni, Zicheng Lin, Jin Zeng, and Yujiu Yang. 2025. Ursa: Understanding and verifying chain-of-thought reasoning in multimodal mathematics. arXiv preprint arXiv:2501.04686. — [arXiv:2501.04686](https://arxiv.org/abs/2501.04686) [find](https://scholar.google.com/scholar?q=Ruilin+Luo%2C+Zhuofan+Zheng%2C+Yifan+Wang%2C+Yiyao+Yu%2C+Xinzhe+Ni%2C+Zicheng+Lin%2C+Jin+Zeng%2C+and+Yujiu+Yang%2E+2025%2E+Ursa%3A+Understanding+and+verifying+chain%2Dof%2Dthought+reasoning+in+multimodal+mathematics%2E+arXiv+p)
42. Run Luo, Haonan Zhang, Longze Chen, Ting-En Lin, Xiong Liu, Yuchuan Wu, Min Yang, Minzheng Wang, Pengpeng Zeng, Lianli Gao, and 1 others. 2024. Mmevol: Empowering multimodal large language models with evol-instruct. arXiv preprint arXiv:2409.05840. [find](https://scholar.google.com/scholar?q=Run+Luo%2C+Haonan+Zhang%2C+Longze+Chen%2C+Ting%2DEn+Lin%2C+Xiong+Liu%2C+Yuchuan+Wu%2C+Min+Yang%2C+Minzheng+Wang%2C+Pengpeng+Zeng%2C+Lianli+Gao%2C+and+1+others%2E+2024%2E+Mmevol%3A+Empowering+multimodal+large+language+models+with)
43. Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, and 1 others. 2025. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.07365. [find](https://scholar.google.com/scholar?q=Fanqing+Meng%2C+Lingxiao+Du%2C+Zongkai+Liu%2C+Zhixiang+Zhou%2C+Quanfeng+Lu%2C+Daocheng+Fu%2C+Botian+Shi%2C+Wenhai+Wang%2C+Junjun+He%2C+Kaipeng+Zhang%2C+and+1+others%2E+2025%2E+Mm%2Deureka%3A+Exploring+visual+aha+moment+with+rule)
44. Sahand Negahban, Sewoong Oh, and Devavrat Shah. 2012. Iterative ranking from pair-wise comparisons. Advances in neural information processing systems, 25. [find](https://scholar.google.com/scholar?q=Sahand+Negahban%2C+Sewoong+Oh%2C+and+Devavrat+Shah%2E+2012%2E+Iterative+ranking+from+pair%2Dwise+comparisons%2E+Advances+in+neural+information+processing+systems%2C+25%2E)
45. Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. 2025. Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl. arXiv preprint arXiv:2503.07536. — [arXiv:2503.07536](https://arxiv.org/abs/2503.07536) [find](https://scholar.google.com/scholar?q=Yingzhe+Peng%2C+Gongrui+Zhang%2C+Miaosen+Zhang%2C+Zhiyuan+You%2C+Jie+Liu%2C+Qipeng+Zhu%2C+Kai+Yang%2C+Xingzhong+Xu%2C+Xin+Geng%2C+and+Xu+Yang%2E+2025%2E+Lmm%2Dr1%3A+Empowering+3b+lmms+with+strong+reasoning+abilities+through+tw)
46. Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, and 1 others. 2024. [We-math: Does your large multimodal model achieve human-like mathematical reasoning?](https://md.research.elamur.ai/free/en/papers/qiao-2024-math-large-multimodal-model-achieve/) arXiv preprint arXiv:2407.01284.
47. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. Preprint, arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+Preprint%2C+arXiv%3A1707%2E06347%2E)
48. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
49. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). arXiv preprint arXiv: 2409.19256.
50. Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang, See-Kiong Ng, Lidong Bing, and Roy Ka-Wei Lee. 2024. [Math-llava: Bootstrapping mathematical reasoning for multimodal large language models](https://md.research.elamur.ai/free/en/papers/shi-2024-math-llava-bootstrapping-mathematical-reasoning/). arXiv preprint arXiv:2406.17294.
51. Milton E Terry. 1952. Some rank order tests which are most powerful against specific parametric alternatives. The Annals of Mathematical Statistics, pages 346–366. [find](https://scholar.google.com/scholar?q=Milton+E+Terry%2E+1952%2E+Some+rank+order+tests+which+are+most+powerful+against+specific+parametric+alternatives%2E+The+Annals+of+Mathematical+Statistics%2C+pages+346%E2%80%93366%2E)
52. Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. 2024. [Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving](https://md.research.elamur.ai/free/en/papers/tong-2024-dart-math-difficulty-aware-rejection/). Advances in Neural Information Processing Systems, 37:7821–7846.
53. Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. 2024a. [Measuring multimodal mathematical reasoning with math-vision dataset](https://md.research.elamur.ai/free/en/papers/wang-2024-measuring-multimodal-mathematical-reasoning-math/). Advances in Neural Information Processing Systems, 37:95095–95169.
54. Liqiong Wang, Teng Jin, Jinyu Yang, Ales Leonardis, Fangyi Wang, and Feng Zheng. 2024b. Agri-llava: Knowledge-infused large multimodal assistant on agricultural pests and diseases. Preprint, arXiv:2412.02158. — [arXiv:2412.02158](https://arxiv.org/abs/2412.02158) [find](https://scholar.google.com/scholar?q=Liqiong+Wang%2C+Teng+Jin%2C+Jinyu+Yang%2C+Ales+Leonardis%2C+Fangyi+Wang%2C+and+Feng+Zheng%2E+2024b%2E+Agri%2Dllava%3A+Knowledge%2Dinfused+large+multimodal+assistant+on+agricultural+pests+and+diseases%2E+Preprint%2C+arXiv%3A241)
55. Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. 2025. Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement. Preprint, arXiv:2504.07934. — [arXiv:2504.07934](https://arxiv.org/abs/2504.07934) [find](https://scholar.google.com/scholar?q=Xiyao+Wang%2C+Zhengyuan+Yang%2C+Chao+Feng%2C+Hongjin+Lu%2C+Linjie+Li%2C+Chung%2DChing+Lin%2C+Kevin+Lin%2C+Furong+Huang%2C+and+Lijuan+Wang%2E+2025%2E+Sota+with+less%3A+Mcts%2Dguided+sample+selection+for+data%2Defficient+visual+re)
56. Alexander Wettig, Aatmik Gupta, Saumya Malik, and Danqi Chen. 2024. [Qurating: Selecting high-quality data for training language models](https://md.research.elamur.ai/free/en/papers/wettig-2024-qurating-selecting-high-quality-data/). arXiv preprint arXiv:2402.09739.
57. Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. 2024. [Less: Selecting influential data for targeted instruction tuning](https://md.research.elamur.ai/free/en/papers/xia-2024-less-selecting-influential-data-targeted/). arXiv preprint arXiv:2402.04333.
58. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). arXiv preprint arXiv:2304.12244.
59. Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. 2024. [Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing](https://md.research.elamur.ai/free/en/papers/xu-2024-magpie-alignment-data-synthesis-scratch/). arXiv preprint arXiv:2406.08464.
60. Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, and 1 others. 2025. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615. [find](https://scholar.google.com/scholar?q=Yi+Yang%2C+Xiaoxuan+He%2C+Hongkun+Pan%2C+Xiyan+Jiang%2C+Yan+Deng%2C+Xingtao+Yang%2C+Haoyu+Lu%2C+Dacheng+Yin%2C+Fengyun+Rao%2C+Minfeng+Zhu%2C+and+1+others%2E+2025%2E+R1%2Donevision%3A+Advancing+generalized+multimodal+reasoning+th)
61. Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, and 1 others. 2024. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18319. — [arXiv:2412.18319](https://arxiv.org/abs/2412.18319) [find](https://scholar.google.com/scholar?q=Huanjin+Yao%2C+Jiaxing+Huang%2C+Wenhao+Wu%2C+Jingyi+Zhang%2C+Yibo+Wang%2C+Shunyu+Liu%2C+Yingjie+Wang%2C+Yuxin+Song%2C+Haocheng+Feng%2C+Li+Shen%2C+and+1+others%2E+2024%2E+Mulberry%3A+Empowering+mllm+with+o1%2Dlike+reasoning+and+r)
62. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, and 1 others. 2025. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). arXiv preprint arXiv:2503.14476.
63. Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, Xiangpeng Wei, and 1 others. 2025. [Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks](https://md.research.elamur.ai/free/en/papers/yue-2025-vapo-efficient-reliable-reinforcement-learning/). arXiv preprint arXiv:2504.05118.
64. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. 2025. [Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/). Preprint, arXiv:2503.18892.
65. B Detailed Benchmark Performance by Difficulty Level 13 C Ablation Studies on Data Strategy Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. 2025. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimi… — [arXiv:2503.12937](https://arxiv.org/abs/2503.12937) [find](https://scholar.google.com/scholar?q=B+Detailed+Benchmark+Performance+by+Difficulty+Level+13+C+Ablation+Studies+on+Data+Strategy+Jingyi+Zhang%2C+Jiaxing+Huang%2C+Huanjin+Yao%2C+Shunyu+Liu%2C+Xikun+Zhang%2C+Shijian+Lu%2C+and+Dacheng+Tao%2E+2025%2E+R1%2Dvl%3A)
66. D Additional Data Analysis Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, and 1 others. 2024a. [Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems?](https://md.research.elamur.ai/free/en/papers/zhang-2024-mathverse-multi-modal-llm-truly/) In European Conference on Computer Vision, pa…
67. Renrui Zhang, Xinyu Wei, Dongzhi Jiang, Yichi Zhang, Ziyu Guo, Chengzhuo Tong, Jiaming Liu, Aojun Zhou, Bin Wei, Shanghang Zhang, and 1 others. 2024b. Mavis: Mathematical visual instruction tuning. arXiv e-prints, pages arXiv–2407. [find](https://scholar.google.com/scholar?q=Renrui+Zhang%2C+Xinyu+Wei%2C+Dongzhi+Jiang%2C+Yichi+Zhang%2C+Ziyu+Guo%2C+Chengzhuo+Tong%2C+Jiaming+Liu%2C+Aojun+Zhou%2C+Bin+Wei%2C+Shanghang+Zhang%2C+and+1+others%2E+2024b%2E+Mavis%3A+Mathematical+visual+instruction+tuning%2E+ar)
68. Yaowei Zheng, Junting Lu, Shenzhi Wang, Zhangchi Feng, Dongdong Kuang, and Yuwen Xiong. 2025. Easyr1: An efficient, scalable, multi-modality rl training framework. [https://github.com/hiyouga/](https://github.com/hiyouga/) EasyR1. [↗](https://github.com/hiyouga/)
69. Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, and 1 others. 2023. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36:55006–55021. [find](https://scholar.google.com/scholar?q=Chunting+Zhou%2C+Pengfei+Liu%2C+Puxin+Xu%2C+Srinivasan+Iyer%2C+Jiao+Sun%2C+Yuning+Mao%2C+Xuezhe+Ma%2C+Avia+Efrat%2C+Ping+Yu%2C+Lili+Yu%2C+and+1+others%2E+2023%2E+Lima%3A+Less+is+more+for+alignment%2E+Advances+in+Neural+Informati)
70. Gengze Zhou, Yicong Hong, Zun Wang, Xin Eric Wang, and Qi Wu. 2024. Navgpt-2: Unleashing navigational reasoning capability for large vision-language models. Preprint, arXiv:2407.12366. — [arXiv:2407.12366](https://arxiv.org/abs/2407.12366) [find](https://scholar.google.com/scholar?q=Gengze+Zhou%2C+Yicong+Hong%2C+Zun+Wang%2C+Xin+Eric+Wang%2C+and+Qi+Wu%2E+2024%2E+Navgpt%2D2%3A+Unleashing+navigational+reasoning+capability+for+large+vision%2Dlanguage+models%2E+Preprint%2C+arXiv%3A2407%2E12366%2E+%E2%80%94+%5BarXiv%3A2407%2E1)
71. Yiyang Zhou, Zhaoyang Wang, Tianle Wang, Shangyu Xing, Peng Xia, Bo Li, Kaiyuan Zheng, Zijian Zhang, Zhaorun Chen, Wenhao Zheng, Xuchao Zhang, Chetan Bansal, Weitong Zhang, Ying Wei, Mohit Bansal, and Huaxiu Yao. 2025. Anyprefer: An agentic framework for preference data synthesis. Preprint, arXiv:25… [find](https://scholar.google.com/scholar?q=Yiyang+Zhou%2C+Zhaoyang+Wang%2C+Tianle+Wang%2C+Shangyu+Xing%2C+Peng+Xia%2C+Bo+Li%2C+Kaiyuan+Zheng%2C+Zijian+Zhang%2C+Zhaorun+Chen%2C+Wenhao+Zheng%2C+Xuchao+Zhang%2C+Chetan+Bansal%2C+Weitong+Zhang%2C+Ying+Wei%2C+Mohit+Bansal%2C+and)
72. Chengke Zou, Xingang Guo, Rui Yang, Junyu Zhang, Bin Hu, and Huan Zhang. 2024. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models. [find](https://scholar.google.com/scholar?q=Chengke+Zou%2C+Xingang+Guo%2C+Rui+Yang%2C+Junyu+Zhang%2C+Bin+Hu%2C+and+Huan+Zhang%2E+2024%2E+Dynamath%3A+A+dynamic+visual+benchmark+for+evaluating+mathematical+reasoning+robustness+of+vision+language+models%2E)

*Updated 2 November 2026*