---
title: "When To Solve, When To Verify — Singhi 2025"
description: "Large language models can spend extra test-time compute on reasoning in two main ways. 50 references, cited by 12 papers here."
authors: ["Nishad Singhi", "Hritik Bansal", "Arian Hosseini", "Aditya Grover", "Kai-Wei Chang", "Marcus Rohrbach", "Anna Rohrbach"]
year: 2025
venue: "arXiv:2504.01005"
source: "https://arxiv.org/abs/2504.01005"
doi: "10.48550/arxiv.2504.01005"
arxiv: "2504.01005"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/singhi-2025-solve-verify-compute-optimal-problem/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/singhi-2025-solve-verify-compute-optimal-problem/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/) / When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning

# When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning

**Authors:** Nishad Singhi, [Hritik Bansal](https://md.research.elamur.ai/free/en/authors/hritik-bansal/), [Arian Hosseini](https://md.research.elamur.ai/free/en/authors/arian-hosseini/), Aditya Grover, [Kai-Wei Chang](https://md.research.elamur.ai/free/en/authors/kai-wei-chang/), [Marcus Rohrbach](https://md.research.elamur.ai/free/en/authors/marcus-rohrbach/) and [Anna Rohrbach](https://md.research.elamur.ai/free/en/authors/anna-rohrbach/) · **Year:** 2025 · **Venue:** arXiv:2504.01005 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2504.01005) · [PDF](https://arxiv.org/pdf/2504.01005) · [DOI 10.48550/arxiv.2504.01005](https://doi.org/10.48550/arxiv.2504.01005) · [arXiv 2504.01005](https://arxiv.org/abs/2504.01005) · [OpenAlex](https://openalex.org/W4417231837)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 12 in our library, of the 140,884 papers whose bibliographies we read

## Summary
Large language models can spend extra test-time compute on reasoning in two main ways. The first is Self-Consistency (SC), which samples many candidate solutions and takes a majority vote. The second samples fewer solutions and ranks them with a Generative Reward Model (GenRM), which writes several verification chains-of-thought for each candidate. Earlier comparisons held only the number of solutions fixed and ignored the cost of verification, so GenRM looked more efficient than it is.

This paper repeats the comparison under equal inference FLOP budgets. A single LLM both generates and checks solutions. It verifies either through prompting (GenRM-Base) or after supervised fine-tuning (GenRM-FT) on verification rationales written by a GPT-4o teacher, which were filtered for correct verdicts and balanced between Yes and No.

At most realistic budgets, SC turns out to be the better choice. GenRM only pays off when compute is very large. The authors also fit scaling laws for dividing a budget between solutions and verifications, giving practitioners a concrete rule for allocating inference compute.

## Key points
- With Llama-3.1-8B-Instruct as solver and GenRM-FT as verifier on MATH, GenRM needs up to 8× more compute just to match SC. It needs about 128× more compute to beat SC by 3.8%.
- The conclusion holds across Llama and Qwen families, 7B and 70B sizes, the thinking model QwQ-32B, and several reasoning tasks. Fine-tuned verification (GenRM-FT) beats prompted verification at fixed compute.
- Compute-optimal settings follow power laws in the budget C: the optimal number of solutions grows as C^0.57, and the optimal number of verifications as C^0.39. Solutions should therefore be scaled roughly 1.5–2× faster than verifications.
- Limitations: the analysis uses FLOPs rather than latency, covers only decoder-only models, and studies only outcome-level Best-of-N. SC cannot be used for tasks without a single final answer, such as theorem proving.
- Code: https://github.com/nishadsinghi/sc-genrm-scaling

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Mathematics & statistics](https://md.research.elamur.ai/free/en/shelves/mathematics/)

Topics: [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/)

## Cite

Cite as: Nishad Singhi, Hritik Bansal, Arian Hosseini, Aditya Grover, Kai-Wei Chang, Marcus Rohrbach and Anna Rohrbach. 2025. When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning. arXiv:2504.01005.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{singhi2025when,
  title = {{When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning}},
  author = {Nishad Singhi, Hritik Bansal, Arian Hosseini, Aditya Grover, Kai-Wei Chang, Marcus Rohrbach and Anna Rohrbach},
  year = {2025},
  doi = {10.48550/arxiv.2504.01005},
  eprint = {2504.01005},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2504.01005}
}
```

- [← previous The Impact of Token Granularity on the Predictive Power of Language Model Surprisal](https://md.research.elamur.ai/free/en/papers/oh-2025-impact-token-granularity-predictive-power/)
- [next → AIN: The Arabic INclusive Large Multimodal Model](https://md.research.elamur.ai/free/en/papers/heakl-2025-ain-arabic-inclusive-large-multimodal/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/)

## References

50 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Zachary Ankner, Mansheej Paul, Brandon Cui, Jonathan D Chang, and Prithviraj Ammanabrolu. Critique-out-loud reward models. arXiv preprint arXiv:2408.11791, 2024. [find](https://scholar.google.com/scholar?q=Zachary+Ankner%2C+Mansheej+Paul%2C+Brandon+Cui%2C+Jonathan+D+Chang%2C+and+Prithviraj+Ammanabrolu%2E+Critique%2Dout%2Dloud+reward+models%2E+arXiv+preprint+arXiv%3A2408%2E11791%2C+2024%2E)
2. AoPS. Aime 2024 dataset, 2024. URL [https://artofproblemsolving.com/wiki/index.php/2024_AIME_I,II](https://artofproblemsolving.com/wiki/index.php/2024_AIME_I,II). [↗](https://artofproblemsolving.com/wiki/index.php/2024_AIME_I,II)
3. AoPS. Aime 2025 dataset, 2025. URL [https://artofproblemsolving.com/wiki/index.php/2025_AIME_I,II](https://artofproblemsolving.com/wiki/index.php/2025_AIME_I,II). [↗](https://artofproblemsolving.com/wiki/index.php/2025_AIME_I,II)
4. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022. [our card](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
5. Hritik Bansal, Arian Hosseini, Rishabh Agarwal, Vinh Q Tran, and Mehran Kazemi. Smaller, weaker, yet better: Training llm reasoners via compute-optimal sampling. arXiv preprint arXiv:2408.16737, 2024. [our card](https://md.research.elamur.ai/free/en/papers/bansal-2024-smaller-weaker-yet-better-training/)
6. Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024. [our card](https://md.research.elamur.ai/free/en/papers/brown-2024-large-language-monkeys-scaling-inference/)
7. Jiefeng Chen, Jie Ren, Xinyun Chen, Chengrun Yang, Ruoxi Sun, and Sercan Ö. Arik. SETS: leveraging self-verification and self-correction for improved test-time scaling. CoRR, abs/2501.19306, 2025. doi: 10.48550/ARXIV.2501.19306. URL https://doi.org/10.48550/arXiv.2501.19306. — [doi:10.48550/arxiv.2501.19306](https://doi.org/10.48550/arxiv.2501.19306) [↗](https://doi.org/10.48550/arXiv.2501.19306)
8. Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. Universal self-consistency for large language model generation. arXiv preprint arXiv:2311.17311, 2023. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Renat+Aksitov%2C+Uri+Alon%2C+Jie+Ren%2C+Kefan+Xiao%2C+Pengcheng+Yin%2C+Sushant+Prakash%2C+Charles+Sutton%2C+Xuezhi+Wang%2C+and+Denny+Zhou%2E+Universal+self%2Dconsistency+for+large+language+model+generation%2E+)
9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. [our card](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
10. Jonathan Cook, Tim Rocktäschel, Jakob N. Foerster, Dennis Aumiller, and Alex Wang. Ticking all the boxes: Generated checklists improve LLM evaluation and generation. CoRR, abs/2410.03608, 2024. doi: 10.48550/ARXIV.2410.03608. URL https://doi.org/10.48550/arXiv.2410.03608. [↗](https://doi.org/10.48550/arXiv.2410.03608)
11. DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948. [↗](https://arxiv.org/abs/2501.12948) [our card](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
12. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Ani… [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci)
13. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. CRITIC: large language models can self-correct with tool-interactive critiquing. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. [our card](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/)
14. OpenReview.net, 2024. URL [https://openreview.net/forum?id=Sx038qxjek](https://openreview.net/forum?id=Sx038qxjek). [↗](https://openreview.net/forum?id=Sx038qxjek)
15. Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. Think before you speak: Training language models with pause tokens. arXiv preprint arXiv:2310.02226, 2023. [find](https://scholar.google.com/scholar?q=Sachin+Goyal%2C+Ziwei+Ji%2C+Ankit+Singh+Rawat%2C+Aditya+Krishna+Menon%2C+Sanjiv+Kumar%2C+and+Vaishnavh+Nagarajan%2E+Think+before+you+speak%3A+Training+language+models+with+pause+tokens%2E+arXiv+preprint+arXiv%3A2310%2E02)
16. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. [find](https://scholar.google.com/scholar?q=Aaron+Grattafiori%2C+Abhimanyu+Dubey%2C+Abhinav+Jauhri%2C+Abhinav+Pandey%2C+Abhishek+Kadian%2C+Ahmad+Al%2DDahle%2C+Aiesha+Letman%2C+Akhil+Mathur%2C+Alan+Schelten%2C+Alex+Vaughan%2C+et+al%2E+The+llama+3+herd+of+models%2E+arXiv+)
17. Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. [find](https://scholar.google.com/scholar?q=Albert+Gu+and+Tri+Dao%2E+Mamba%3A+Linear%2Dtime+sequence+modeling+with+selective+state+spaces%2E+arXiv+preprint+arXiv%3A2312%2E00752%2C+2023%2E)
18. Etash Guha, Negin Raoof, Jean Mercat, Ryan Marten, Eric Frankel, Sedrick Keh, Sachin Grover, George Smyrnis, Trung Vu, Jon Saad-Falcon, Caroline Choi, Kushal Arora, Mike Merrill, Yichuan Deng, Ashima Suvarna, Hritik Bansal, Marianna Nezhurina, Yejin Choi, Reinhard Heckel, Seewong Oh, Tatsunori Hashi… [find](https://scholar.google.com/scholar?q=Etash+Guha%2C+Negin+Raoof%2C+Jean+Mercat%2C+Ryan+Marten%2C+Eric+Frankel%2C+Sedrick+Keh%2C+Sachin+Grover%2C+George+Smyrnis%2C+Trung+Vu%2C+Jon+Saad%2DFalcon%2C+Caroline+Choi%2C+Kushal+Arora%2C+Mike+Merrill%2C+Yichuan+Deng%2C+Ashima+)
19. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. [our card](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
20. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/)
21. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022. [our card](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/)
22. Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, and Rishabh Agarwal. V-star: Training verifiers for self-taught reasoners. arXiv preprint arXiv:2402.06457, 2024. [our card](https://md.research.elamur.ai/free/en/papers/hosseini-2024-v-star-training-verifiers-self/)
23. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. [our card](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/)
24. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. [our card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)
25. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024. [our card](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/)
26. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. [our card](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/)
27. Seungone Kim, Jamin Shin, Yejin Choi, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Seo. Prometheus: Inducing fine-grained evaluation capability in language models. In The Twelfth International Conference on Learning Representations, ICLR… [our card](https://md.research.elamur.ai/free/en/papers/kim-2023-prometheus-inducing-fine-grained-evaluation/)
28. OpenReview.net, 2024. URL [https://openreview.net/forum?id=8euJaTveKw](https://openreview.net/forum?id=8euJaTveKw). [↗](https://openreview.net/forum?id=8euJaTveKw)
29. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. [our card](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/)
30. Kuang-Huei Lee, Ian Fischer, Yueh-Hua Wu, Dave Marwood, Shumeet Baluja, Dale Schuurmans, and Xinyun Chen. Evolving deeper LLM thinking. CoRR, abs/2501.09891, 2025. doi: 10.48550/ARXIV.2501.09891. URL https://doi.org/10.48550/arXiv.2501.09891. [↗](https://doi.org/10.48550/arXiv.2501.09891)
31. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2023. [our card](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)
32. Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, and Abhinav Rastogi. Improve mathematical reasoning in language models by automated process supervision. CoRR, abs/2406.06592, 2024. doi: 10.48550/ARXIV.2406.06592. URL [https://doi.…](https://doi.…) [↗](https://doi.…) [our card](https://md.research.elamur.ai/free/en/papers/luo-2024-improve-mathematical-reasoning-language-models/)
33. Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. Generative reward models. arXiv preprint arXiv:2410.12832, 2024. [find](https://scholar.google.com/scholar?q=Dakota+Mahan%2C+Duy+Van+Phung%2C+Rafael+Rafailov%2C+Chase+Blagden%2C+Nathan+Lile%2C+Louis+Castricato%2C+Jan%2DPhilipp+Fr%C3%A4nken%2C+Chelsea+Finn%2C+and+Alon+Albalak%2E+Generative+reward+models%2E+arXiv+preprint+arXiv%3A2410%2E128)
34. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint arXiv:2502.09992, 2025. [our card](https://md.research.elamur.ai/free/en/papers/nie-2025-large-language-diffusion-models/)
35. OpenAI. Learning to reason with llms, 2024. learning-to-reason-with-llms/. [find](https://scholar.google.com/scholar?q=OpenAI%2E+Learning+to+reason+with+llms%2C+2024%2E+learning%2Dto%2Dreason%2Dwith%2Dllms%2F%2E)
36. Qwen. Qwq-32b: Embracing the power of reinforcement learning, 2025. URL [https://qwenlm.github.io/blog/qwq-32b/](https://qwenlm.github.io/blog/qwq-32b/). [↗](https://qwenlm.github.io/blog/qwq-32b/)
37. URL [https://openai.com/index/David](https://openai.com/index/David) Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. [↗](https://openai.com/index/David) [our card](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/)
38. Amrith Setlur, Nived Rajaraman, Sergey Levine, and Aviral Kumar. Scaling test-time compute without verification or rl is suboptimal, 2025. URL https://arxiv.org/abs/2502.12118. [↗](https://arxiv.org/abs/2502.12118)
39. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024. [our card](https://md.research.elamur.ai/free/en/papers/snell-2024-scaling-llm-test-time-compute/)
40. Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck, and Chuang Gan. Easy-to-hard generalization: Scalable alignment beyond human supervision. arXiv preprint arXiv:2403.09472, 2024. [find](https://scholar.google.com/scholar?q=Zhiqing+Sun%2C+Longhui+Yu%2C+Yikang+Shen%2C+Weiyang+Liu%2C+Yiming+Yang%2C+Sean+Welleck%2C+and+Chuang+Gan%2E+Easy%2Dto%2Dhard+generalization%3A+Scalable+alignment+beyond+human+supervision%2E+arXiv+preprint+arXiv%3A2403%2E09472%2C)
41. Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for… [our card](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/)
42. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022. [our card](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/)
43. Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models. arXiv preprint arXiv:2408.00724, 2024. [our card](https://md.research.elamur.ai/free/en/papers/wu-2024-inference-scaling-laws-empirical-analysis/)
44. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+et+al%2E+Qwen2%2E+5+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E15115%2C+2024%2E)
45. Fei Yu, Anningzhe Gao, and Benyou Wang. Ovm, outcome-supervised value models for planning in mathematical reasoning. In Kevin Duh, Helena Gómez-Adorno, and Steven Bethard (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pp. 858–875… [find](https://scholar.google.com/scholar?q=Fei+Yu%2C+Anningzhe+Gao%2C+and+Benyou+Wang%2E+Ovm%2C+outcome%2Dsupervised+value+models+for+planning+in+mathematical+reasoning%2E+In+Kevin+Duh%2C+Helena+G%C3%B3mez%2DAdorno%2C+and+Steven+Bethard+%28eds%2E%29%2C+Findings+of+the+Assoc)
46. Lifan Yuan, Wendi Li, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, and Hao Peng. Free process rewards without process labels, 2024. URL https://arxiv.org/abs/2412.01981. [↗](https://arxiv.org/abs/2412.01981)
47. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. arXiv preprint arXiv:2408.15240, 2024. [our card](https://md.research.elamur.ai/free/en/papers/zhang-2024-generative-verifiers-reward-modeling-as/)
48. Eric Zhao, Pranjal Awasthi, and Sreenivas Gollapudi. Sample, scrutinize and scale: Effective inference-time search by scaling verification, 2025. URL https://arxiv.org/abs/2502.01839. — [arXiv:2502.01839](https://arxiv.org/abs/2502.01839) [↗](https://arxiv.org/abs/2502.01839)
49. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595– 46623, 2023a. [find](https://scholar.google.com/scholar?q=Lianmin+Zheng%2C+Wei%2DLin+Chiang%2C+Ying+Sheng%2C+Siyuan+Zhuang%2C+Zhanghao+Wu%2C+Yonghao+Zhuang%2C+Zi+Lin%2C+Zhuohan+Li%2C+Dacheng+Li%2C+Eric+Xing%2C+et+al%2E+Judging+llm%2Das%2Da%2Djudge+with+mt%2Dbench+and+chatbot+arena%2E+Advance)
50. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Morit… [find](https://scholar.google.com/scholar?q=Lianmin+Zheng%2C+Wei%2DLin+Chiang%2C+Ying+Sheng%2C+Siyuan+Zhuang%2C+Zhanghao+Wu%2C+Yonghao+Zhuang%2C+Zi+Lin%2C+Zhuohan+Li%2C+Dacheng+Li%2C+Eric+P%2E+Xing%2C+Hao+Zhang%2C+Joseph+E%2E+Gonzalez%2C+and+Ion+Stoica%2E+Judging+llm%2Das%2Da%2Djud)