---
title: "PRMBench — paper by Song et al. (2025) — summary and ideas"
description: "This paper introduces PRMBench, a math-reasoning benchmark that tests whether PRMs can spot… 50 references, cited by 54 papers here."
authors: ["Mingyang Song", "Zhaochen Su", "Xiaoye Qu", "Jiawei Zhou", "Yu Cheng"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.1230/"
doi: "10.18653/v1/2025.acl-long.1230"
arxiv: "2501.03124"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/song-2025-prmbench-fine-grained-challenging-benchmark/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/song-2025-prmbench-fine-grained-challenging-benchmark/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models

# PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models

**Authors:** [Mingyang Song](https://md.research.elamur.ai/free/en/authors/mingyang-song/), [Zhaochen Su](https://md.research.elamur.ai/free/en/authors/zhaochen-su/), [Xiaoye Qu](https://md.research.elamur.ai/free/en/authors/xiaoye-qu/), [Jiawei Zhou](https://md.research.elamur.ai/free/en/authors/jiawei-zhou/) and [Yu Cheng](https://md.research.elamur.ai/free/en/authors/yu-cheng/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.1230/) · [PDF](https://aclanthology.org/2025.acl-long.1230.pdf) · [DOI 10.18653/v1/2025.acl-long.1230](https://doi.org/10.18653/v1/2025.acl-long.1230) · [arXiv 2501.03124](https://arxiv.org/abs/2501.03124) · [OpenAlex](https://openalex.org/W4412889902)

**Citations:** 9 citations (OpenAlex, 2026-10-14) · [54](https://md.research.elamur.ai/free/en/papers/song-2025-prmbench-fine-grained-challenging-benchmark/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Process-level reward models (PRMs) score each intermediate step of an LLM's reasoning chain. Existing benchmarks mostly check whether a step is right or wrong, so they miss subtler failures. This paper introduces PRMBench, a math-reasoning benchmark that tests whether PRMs can spot different kinds of step-level errors. The authors start from correct solution paths in PRM800K. They use LLMs to inject controlled perturbations, then have human annotators filter the results. Each sample is labeled under one of nine sub-categories, grouped into three axes: simplicity (redundant or circular reasoning), soundness (factual, logical, domain and confidence errors) and sensitivity (missing prerequisites, deceptive premises, alternative solutions). Style-controlled curation keeps difficulty comparable across categories. The authors evaluated 25 systems, including dedicated PRMs and general or math LLMs prompted as critics. Even the best model performs well below humans. Rankings on PRMBench also barely correlate with Best-of-N outcome accuracy. This suggests that outcome-based testing can hide step-level weaknesses and leave room for reward hacking.

## Key points
- PRMBench contains 6,216 problems with 83,456 step-level labels. Solutions average 13.4 steps.
- The taxonomy has 3 axes and 9 sub-categories, for example Non-Redundancy, Non-Circular Logic, Deception Resistance and Multi-Solution Consistency.
- Gemini-2-Thinking scored highest at 68.8, compared with a human score of 83.8.
- Somers' D between PRMBench and Best-of-N (N=8, Qwen-QwQ samples) results averages only −0.05. Math-Shepherd-7B scores 47.0 on PRMBench yet beats QwenPRM-7B on BoN (74.3 vs 73.4).
- Few-shot in-context prompting changes critic performance only slightly. The authors release the PRM-EVAL toolkit for automated evaluation and data generation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/) — University, Shanghai, China · 202 works · 22,509 citations
- [Chinese University of Hong Kong](https://md.research.elamur.ai/free/en/universities/chinese-university-of-hong-kong/) — University, Hong Kong, China · 159 works · 16,154 citations
- [Shanghai Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/shanghai-artificial-intelligence-laboratory/) — Research institute, Shanghai, China · 152 works · 13,166 citations
- [Soochow University](https://md.research.elamur.ai/free/en/universities/soochow-university/) — University, Suzhou, China · 69 works · 4,815 citations
- [Stony Brook University](https://md.research.elamur.ai/free/en/universities/stony-brook-university/) — University, Stony Brook, United States · 46 works · 3,024 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works

## Cite

Cite as: Mingyang Song, Zhaochen Su, Xiaoye Qu, Jiawei Zhou and Yu Cheng. 2025. PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{song2025prmbench,
  title = {{PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models}},
  author = {Mingyang Song, Zhaochen Su, Xiaoye Qu, Jiawei Zhou and Yu Cheng},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1230},
  eprint = {2501.03124},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1230}
}
```

- [← previous Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://md.research.elamur.ai/free/en/papers/baker-2025-monitoring-reasoning-models-misbehavior-risks/)
- [next → Reasoning with Exploration: An Entropy Perspective](https://md.research.elamur.ai/free/en/papers/cheng-2025-reasoning-exploration-entropy-perspective/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

50 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Akari Asai, Jacqueline He*, Rulin Shao*, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Tian, D’arcy Mike, David Wadden, Matt Latzke, Minyang, Pan Ji, Shengyan Liu, Hao Tong, Bohao Wu, Yanyu Xiong, Luke Zettlemoyer, Dan Weld, Graham Neubig, Doug Downey, Wen-t… [find](https://scholar.google.com/scholar?q=Akari+Asai%2C+Jacqueline+He%2A%2C+Rulin+Shao%2A%2C+Weijia+Shi%2C+Amanpreet+Singh%2C+Joseph+Chee+Chang%2C+Kyle+Lo%2C+Luca+Soldaini%2C+Sergey+Feldman%2C+Tian%2C+D%E2%80%99arcy+Mike%2C+David+Wadden%2C+Matt+Latzke%2C+Minyang%2C+Pan+Ji%2C+Shengyan)
2. Jie Cheng, Lijun Li, Gang Xiong, Jing Shao, and Yisheng Lv. 2025. Pure: Prm is still effective and compute-efficient for llm math reasoning. [https://github.com/CJReinforce/PURE](https://github.com/CJReinforce/PURE). [↗](https://github.com/CJReinforce/PURE)
3. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
4. DeepMind. 2024. Gemini 2.0 flash experimental. [https://deepmind.google/technologies/](https://deepmind.google/technologies/) gemini/flash/. Accessed: 2024-12-25. [↗](https://deepmind.google/technologies/)
5. Bofei Gao, Zefan Cai, Runxin Xu, Peiyi Wang, Ce Zheng, Runji Lin, Keming Lu, Junyang Lin, Chang Zhou, Wen Xiao, et al. 2024. Llm critics help catch bugs in mathematics: Towards a better mathematical verifier with natural language feedback. CoRR. [find](https://scholar.google.com/scholar?q=Bofei+Gao%2C+Zefan+Cai%2C+Runxin+Xu%2C+Peiyi+Wang%2C+Ce+Zheng%2C+Runji+Lin%2C+Keming+Lu%2C+Junyang+Lin%2C+Chang+Zhou%2C+Wen+Xiao%2C+et+al%2E+2024%2E+Llm+critics+help+catch+bugs+in+mathematics%3A+Towards+a+better+mathematical+v)
6. Jiaxuan Gao, Shusheng Xu, Wenjie Ye, Weilin Liu, Chuyi He, Wei Fu, Zhiyu Mei, Guangju Wang, and Yi Wu. 2025. On designing effective RL reward at training time for LLM reasoning. [find](https://scholar.google.com/scholar?q=Jiaxuan+Gao%2C+Shusheng+Xu%2C+Wenjie+Ye%2C+Weilin+Liu%2C+Chuyi+He%2C+Wei+Fu%2C+Zhiyu+Mei%2C+Guangju+Wang%2C+and+Yi+Wu%2E+2025%2E+On+designing+effective+RL+reward+at+training+time+for+LLM+reasoning%2E)
7. Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. 2022. [Roscoe: A suite of metrics for scoring step-by-step reasoning](https://md.research.elamur.ai/free/en/papers/golovneva-2022-roscoe-suite-metrics-scoring-step/). arXiv preprint arXiv:2212.07919.
8. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
9. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. Deepseek-coder: When the large language model meets programming– the rise of code intelligence. arXiv preprint arXiv:2401.14196. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Qihao+Zhu%2C+Dejian+Yang%2C+Zhenda+Xie%2C+Kai+Dong%2C+Wentao+Zhang%2C+Guanting+Chen%2C+Xiao+Bi%2C+Yu+Wu%2C+YK+Li%2C+et+al%2E+2024%2E+Deepseek%2Dcoder%3A+When+the+large+language+model+meets+programming%E2%80%93+the+rise+of+co)
10. Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. 2025. Can mllms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint arXiv:2501.05444. — [arXiv:2501.05444](https://arxiv.org/abs/2501.05444) [find](https://scholar.google.com/scholar?q=Yunzhuo+Hao%2C+Jiawei+Gu%2C+Huichen+Will+Wang%2C+Linjie+Li%2C+Zhengyuan+Yang%2C+Lijuan+Wang%2C+and+Yu+Cheng%2E+2025%2E+Can+mllms+reason+in+multimodality%3F+emma%3A+An+enhanced+multimodal+reasoning+benchmark%2E+arXiv+prepri)
11. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. 2024a. [Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). arXiv preprint arXiv:2402.14008.
12. Conghui He, Wei Li, Zhenjiang Jin, Chao Xu, Bin Wang, and Dahua Lin. 2024b. Opendatalab: Empowering general artificial intelligence with open datasets. arXiv preprint arXiv:2407.13773. — [arXiv:2407.13773](https://arxiv.org/abs/2407.13773) [find](https://scholar.google.com/scholar?q=Conghui+He%2C+Wei+Li%2C+Zhenjiang+Jin%2C+Chao+Xu%2C+Bin+Wang%2C+and+Dahua+Lin%2E+2024b%2E+Opendatalab%3A+Empowering+general+artificial+intelligence+with+open+datasets%2E+arXiv+preprint+arXiv%3A2407%2E13773%2E+%E2%80%94+%5BarXiv%3A2407%2E1)
13. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300.
14. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874.
15. Lei Li, Yuancheng Wei, Zhihui Xie, Xuqing Yang, Yifan Song, Peiyi Wang, Chenxin An, Tianyu Liu, Sujian Li, Bill Yuchen Lin, et al. 2024a. Vlre-wardbench: A challenging benchmark for vision-language generative reward models. arXiv preprint arXiv:2411.17451. — [arXiv:2411.17451](https://arxiv.org/abs/2411.17451) [find](https://scholar.google.com/scholar?q=Lei+Li%2C+Yuancheng+Wei%2C+Zhihui+Xie%2C+Xuqing+Yang%2C+Yifan+Song%2C+Peiyi+Wang%2C+Chenxin+An%2C+Tianyu+Liu%2C+Sujian+Li%2C+Bill+Yuchen+Lin%2C+et+al%2E+2024a%2E+Vlre%2Dwardbench%3A+A+challenging+benchmark+for+vision%2Dlanguage+ge)
16. Xiaoyuan Li, Wenjie Wang, Moxin Li, Junrong Guo, Yang Zhang, and Fuli Feng. 2024b. Evaluating mathematical reasoning of large language models: A focus on error identification and correction. arXiv preprint arXiv:2406.00755. [find](https://scholar.google.com/scholar?q=Xiaoyuan+Li%2C+Wenjie+Wang%2C+Moxin+Li%2C+Junrong+Guo%2C+Yang+Zhang%2C+and+Fuli+Feng%2E+2024b%2E+Evaluating+mathematical+reasoning+of+large+language+models%3A+A+focus+on+error+identification+and+correction%2E+arXiv+pre)
17. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). arXiv preprint arXiv:2305.20050.
18. Zicheng Lin, Zhibin Gou, Tian Liang, Ruilin Luo, Haowei Liu, and Yujiu Yang. 2024. Criticbench: Benchmarking llms for critique-correct reasoning. arXiv preprint arXiv:2402.14809. [find](https://scholar.google.com/scholar?q=Zicheng+Lin%2C+Zhibin+Gou%2C+Tian+Liang%2C+Ruilin+Luo%2C+Haowei+Liu%2C+and+Yujiu+Yang%2E+2024%2E+Criticbench%3A+Benchmarking+llms+for+critique%2Dcorrect+reasoning%2E+arXiv+preprint+arXiv%3A2402%2E14809%2E)
19. Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. 2024. [Rm-bench: Benchmarking reward models of language models with subtlety and style](https://md.research.elamur.ai/free/en/papers/liu-2024-rm-bench-benchmarking-reward-models/). arXiv preprint arXiv:2410.16184.
20. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jian-guang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. [Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/). arXiv preprint arXiv:2308.09583.
21. Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, Maja Trebacz, and Jan Leike. 2024. Llm critics help catch llm bugs. arXiv preprint arXiv:2407.00215. — [arXiv:2407.00215](https://arxiv.org/abs/2407.00215) [find](https://scholar.google.com/scholar?q=Nat+McAleese%2C+Rai+Michael+Pokorny%2C+Juan+Felipe+Ceron+Uribe%2C+Evgenia+Nitishinskaya%2C+Maja+Trebacz%2C+and+Jan+Leike%2E+2024%2E+Llm+critics+help+catch+llm+bugs%2E+arXiv+preprint+arXiv%3A2407%2E00215%2E+%E2%80%94+%5BarXiv%3A2407%2E00)
22. Skywork o1 Team. 2024. Skywork-o1 open series. [https://huggingface.co/Skywork](https://huggingface.co/Skywork). [↗](https://huggingface.co/Skywork)
23. OpenAI. 2024a. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). [https://cdn.openai.com/gpt-4o-system-card.pdf](https://cdn.openai.com/gpt-4o-system-card.pdf). Accessed: 2024-09-26. [↗](https://cdn.openai.com/gpt-4o-system-card.pdf)
24. OpenAI. 2024b. Learning to reason with llms. [https://openai.com/index/](https://openai.com/index/) learning-to-reason-with-llms/. [↗](https://openai.com/index/)
25. Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, et al. 2024. [O1 replication journey: A strategic progress report–part 1](https://md.research.elamur.ai/free/en/papers/qin-2024-o1-replication-journey-strategic-progress/). arXiv preprint arXiv:2410.18982.
26. Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, et al. 2025. [A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond](https://md.research.elamur.ai/free/en/papers/qu-2025-survey-efficient-reasoning-large-reasoning/). arXiv preprint arXiv:2503.21614.
27. Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. 2025. Open-thinkimg: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617. — [arXiv:2505.08617](https://arxiv.org/abs/2505.08617) [find](https://scholar.google.com/scholar?q=Zhaochen+Su%2C+Linjie+Li%2C+Mingyang+Song%2C+Yunzhuo+Hao%2C+Zhengyuan+Yang%2C+Jun+Zhang%2C+Guanjie+Chen%2C+Jiawei+Gu%2C+Juntao+Li%2C+Xiaoye+Qu%2C+et+al%2E+2025%2E+Open%2Dthinkimg%3A+Learning+to+think+with+images+via+visual+tool+)
28. Zhaochen Su, Jun Zhang, Xiaoye Qu, Tong Zhu, Yanshu Li, Jiashuo Sun, Juntao Li, Min Zhang, and Yu Cheng. 2024a. Conflictbank: A benchmark for evaluating the influence of knowledge conflicts in llm. arXiv preprint arXiv:2408.12076. — [arXiv:2408.12076](https://arxiv.org/abs/2408.12076) [find](https://scholar.google.com/scholar?q=Zhaochen+Su%2C+Jun+Zhang%2C+Xiaoye+Qu%2C+Tong+Zhu%2C+Yanshu+Li%2C+Jiashuo+Sun%2C+Juntao+Li%2C+Min+Zhang%2C+and+Yu+Cheng%2E+2024a%2E+Conflictbank%3A+A+benchmark+for+evaluating+the+influence+of+knowledge+conflicts+in+llm%2E+ar)
29. Zhaochen Su, Jun Zhang, Tong Zhu, Xiaoye Qu, Juntao Li, Min Zhang, and Yu Cheng. 2024b. [Timo: Towards better temporal reasoning for language models](https://md.research.elamur.ai/free/en/papers/su-2024-timo-towards-better-temporal-reasoning/). arXiv preprint arXiv:2406.14192.
30. Qwen Team. 2024a. Qwq: Reflect deeply on the boundaries of the unknown. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2024a%2E+Qwq%3A+Reflect+deeply+on+the+boundaries+of+the+unknown%2E)
31. ScalableMath Team. 2024b. Easy-to-hard generalization models. [https://huggingface.co/](https://huggingface.co/) ScalableMath. [↗](https://huggingface.co/)
32. Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. [Solving math word problems with process-and outcome-based feedback](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/). arXiv preprint arXiv:2211.14275.
33. Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Y Wu, and Zhifang Sui. 2023. [Math-shepherd: Verify and reinforce llms step-by-step without human annotations](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/). CoRR, abs/2312.08935.
34. Xueru Wen, Jie Lou, Yaojie Lu, Hongyu Lin, XingYu, Xinyu Lu, Ben He, Xianpei Han, Debing Zhang, and Le Sun. 2025. Rethinking reward model evaluation: Are we barking up the wrong tree? In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Xueru+Wen%2C+Jie+Lou%2C+Yaojie+Lu%2C+Hongyu+Lin%2C+XingYu%2C+Xinyu+Lu%2C+Ben+He%2C+Xianpei+Han%2C+Debing+Zhang%2C+and+Le+Sun%2E+2025%2E+Rethinking+reward+model+evaluation%3A+Are+we+barking+up+the+wrong+tree%3F+In+The+Thirteent)
35. Peng Xia, Kangyu Zhu, Haoran Li, Tianze Wang, Weijia Shi, Sheng Wang, Linjun Zhang, James Zou, and Huaxiu Yao. 2024a. Mmed-rag: Versatile multimodal rag system for medical vision language models. arXiv preprint arXiv:2410.13085. — [arXiv:2410.13085](https://arxiv.org/abs/2410.13085) [find](https://scholar.google.com/scholar?q=Peng+Xia%2C+Kangyu+Zhu%2C+Haoran+Li%2C+Tianze+Wang%2C+Weijia+Shi%2C+Sheng+Wang%2C+Linjun+Zhang%2C+James+Zou%2C+and+Huaxiu+Yao%2E+2024a%2E+Mmed%2Drag%3A+Versatile+multimodal+rag+system+for+medical+vision+language+models%2E+arXi)
36. Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu. 2024b. [Evaluating mathematical reasoning beyond accuracy](https://md.research.elamur.ai/free/en/papers/xia-2024-evaluating-mathematical-reasoning-accuracy/). arXiv preprint arXiv:2404.05692.
37. Wei Xiong, Hanning Zhang, Nan Jiang, and Tong Zhang. 2024. An implementation of generative prm. [https://github.com/RLHFlow/RLHF-Reward-Modeling](https://github.com/RLHFlow/RLHF-Reward-Modeling). [↗](https://github.com/RLHFlow/RLHF-Reward-Modeling)
38. Yibo Yan, Shen Wang, Jiahao Huo, Hang Li, Boyan Li, Jiamin Su, Xiong Gao, Yi-Fan Zhang, Tianlong Xu, Zhendong Chu, et al. 2024. [Errorradar: Benchmarking complex mathematical reasoning of multimodal large language models via error detection](https://md.research.elamur.ai/free/en/papers/yan-2026-errorradar-benchmarking-complex-mathematical-reasoning/). arXiv preprint arXiv:2410.04509.
39. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024a. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+et+al%2E+2024a%2E+Qwen2%2E+5+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E15115%2E)
40. An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. 2024b. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122. — [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://md.research.elamur.ai/free/en/papers/yang-2024-qwen2-5-math-technical-report/)
41. Di Zhang, Jingdi Lei, Junxian Li, Xunzhi Wang, Yujie Liu, Zonglin Yang, Jiatong Li, Weida Wang, Suorong Yang, Jianbo Wu, et al. 2024a. Critic-v: Vlm critics help catch vlm errors in multimodal reasoning. arXiv preprint arXiv:2411.18203. — [arXiv:2411.18203](https://arxiv.org/abs/2411.18203) [find](https://scholar.google.com/scholar?q=Di+Zhang%2C+Jingdi+Lei%2C+Junxian+Li%2C+Xunzhi+Wang%2C+Yujie+Liu%2C+Zonglin+Yang%2C+Jiatong+Li%2C+Weida+Wang%2C+Suorong+Yang%2C+Jianbo+Wu%2C+et+al%2E+2024a%2E+Critic%2Dv%3A+Vlm+critics+help+catch+vlm+errors+in+multimodal+reasoni)
42. Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, et al. 2024b. Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning. arXiv preprint arXiv:2410.02884. [find](https://scholar.google.com/scholar?q=Di+Zhang%2C+Jianbo+Wu%2C+Jingdi+Lei%2C+Tong+Che%2C+Jiatong+Li%2C+Tong+Xie%2C+Xiaoshui+Huang%2C+Shufei+Zhang%2C+Marco+Pavone%2C+Yuqiang+Li%2C+et+al%2E+2024b%2E+Llama%2Dberry%3A+Pairwise+optimization+for+o1%2Dlike+olympiad%2Dlevel+mat)
43. Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2025. [The lessons of developing process reward models in mathematical reasoning](https://md.research.elamur.ai/free/en/papers/zhang-2025-lessons-developing-process-reward-models/). arXiv preprint arXiv:2501.07301.
44. Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2023. [Multimodal chain-of-thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-multimodal-chain-thought-reasoning-language/). arXiv preprint arXiv:2302.00923.
45. Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2024. [Processbench: Identifying process errors in mathematical reasoning](https://md.research.elamur.ai/free/en/papers/zheng-2025-processbench-identifying-process-errors-mathematical/). arXiv preprint arXiv:2412.06559.
46. Zihao Zhou, Shudong Liu, Maizhen Ning, Wei Liu, Jindong Wang, Derek F Wong, Xiaowei Huang, Qiufeng Wang, and Kaizhu Huang. 2024. Is your model really a good math reasoner? evaluating mathematical reasoning with checklist. arXiv preprint arXiv:2407.08733. — [arXiv:2407.08733](https://arxiv.org/abs/2407.08733) [find](https://scholar.google.com/scholar?q=Zihao+Zhou%2C+Shudong+Liu%2C+Maizhen+Ning%2C+Wei+Liu%2C+Jindong+Wang%2C+Derek+F+Wong%2C+Xiaowei+Huang%2C+Qiufeng+Wang%2C+and+Kaizhu+Huang%2E+2024%2E+Is+your+model+really+a+good+math+reasoner%3F+evaluating+mathematical+reas)
47. Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. [Metamath: Bootstrap your own mathematical questions for large language models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/). arXiv preprint arXiv:2309.12284.
48. Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. 2024a. Evaluating large language models at evaluating instruction following. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zhiyuan+Zeng%2C+Jiatong+Yu%2C+Tianyu+Gao%2C+Yu+Meng%2C+Tanya+Goyal%2C+and+Danqi+Chen%2E+2024a%2E+Evaluating+large+language+models+at+evaluating+instruction+following%2E+In+The+Twelfth+International+Conference+on+Lear)
49. Zhongshen Zeng, Pengguang Chen, Shu Liu, Haiyun Jiang, and Jiaya Jia. 2023. Mr-gsm8k: A meta-reasoning benchmark for large language model evaluation. arXiv preprint arXiv:2312.17080. [find](https://scholar.google.com/scholar?q=Zhongshen+Zeng%2C+Pengguang+Chen%2C+Shu+Liu%2C+Haiyun+Jiang%2C+and+Jiaya+Jia%2E+2023%2E+Mr%2Dgsm8k%3A+A+meta%2Dreasoning+benchmark+for+large+language+model+evaluation%2E+arXiv+preprint+arXiv%3A2312%2E17080%2E)
50. Zhongshen Zeng, Yinhong Liu, Yingjia Wan, Jingyao Li, Pengguang Chen, Jianbo Dai, Yuxuan Yao, Rongwu Xu, Zehan Qi, Wanru Zhao, Linling Shen, Jianqiao Lu, Haochen Tan, Yukang Chen, Hao Zhang, Zhan Shi, Bailin Wang, Zhijiang Guo, and Jiaya Jia. 2024b. Mr-ben: A meta-reasoning benchmark for evaluating… — [arXiv:2406.13975](https://arxiv.org/abs/2406.13975) [find](https://scholar.google.com/scholar?q=Zhongshen+Zeng%2C+Yinhong+Liu%2C+Yingjia+Wan%2C+Jingyao+Li%2C+Pengguang+Chen%2C+Jianbo+Dai%2C+Yuxuan+Yao%2C+Rongwu+Xu%2C+Zehan+Qi%2C+Wanru+Zhao%2C+Linling+Shen%2C+Jianqiao+Lu%2C+Haochen+Tan%2C+Yukang+Chen%2C+Hao+Zhang%2C+Zhan+Shi%2C)

*Updated 2 November 2026*