---
title: "Inference-Time Scaling for Generalist Reward Modeling (2025)"
description: "Reinforcement learning for LLMs depends on reliable reward signals. 88 references, cited by 62 papers here."
authors: ["Zijun Liu", "Peiyi Wang", "Runxin Xu", "Shirong Ma", "Chong Ruan", "Peng Li", "Yang Liu", "Yu Wu"]
year: 2025
venue: "arXiv:2504.02495"
source: "https://arxiv.org/abs/2504.02495"
doi: "10.48550/arxiv.2504.02495"
arxiv: "2504.02495"
licence: "CC BY-SA 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2025-inference-time-scaling-generalist/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2025-inference-time-scaling-generalist/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / Inference-Time Scaling for Generalist Reward Modeling

# Inference-Time Scaling for Generalist Reward Modeling

**Authors:** Zijun Liu, [Peiyi Wang](https://md.research.elamur.ai/free/en/authors/peiyi-wang/), [Runxin Xu](https://md.research.elamur.ai/free/en/authors/runxin-xu/), [Shirong Ma](https://md.research.elamur.ai/free/en/authors/shirong-ma/), [Chong Ruan](https://md.research.elamur.ai/free/en/authors/chong-ruan/), [Peng Li](https://md.research.elamur.ai/free/en/authors/peng-li/), [Yang Liu](https://md.research.elamur.ai/free/en/authors/yang-liu/) and [Yu Wu](https://md.research.elamur.ai/free/en/authors/yu-wu/) · **Year:** 2025 · **Venue:** arXiv:2504.02495 · **Licence:** [CC BY-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-sa/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2504.02495) · [PDF](https://arxiv.org/pdf/2504.02495) · [DOI 10.48550/arxiv.2504.02495](https://doi.org/10.48550/arxiv.2504.02495) · [arXiv 2504.02495](https://arxiv.org/abs/2504.02495) · [OpenAlex](https://openalex.org/W4416123545)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · 62 in our library, of the 141,840 papers whose bibliographies we read

## Summary
Reinforcement learning for LLMs depends on reliable reward signals. Outside verifiable areas like math or code, these are hard to obtain because the criteria vary and no ground truth exists. The authors ask whether a reward model (RM) can be trained so that giving it more inference compute yields better rewards on general queries. They argue for pointwise generative reward modeling (GRM), which scores one, two, or many responses through text. Unlike scalar RMs, a GRM can produce varied outputs when sampled repeatedly.

Their training method, Self-Principled Critique Tuning (SPCT), uses rule-based online RL. It teaches the model to write its own evaluation principles for each query, then critique and score the responses. Applying SPCT to Gemma-2-27B produces DeepSeek-GRM-27B. At inference, several samples are drawn in parallel and their scores are aggregated by voting. An additional meta RM filters out low-quality samples before voting.

Across several RM benchmarks, the model outperforms baseline methods and strong public RMs without severe domain bias. Its sampling-based scaling rivals much larger models, suggesting extra inference compute can substitute for model size.

## Key points
- Baselines include DeepSeek-BTRM-27B, DeepSeek-PairRM-27B, CLoud-Gemma-2-27B, LLM-as-a-Judge, Skywork-Reward-Gemma-2-27B, Nemotron-4-340B-Reward and GPT-4o.
- Sampling for DeepSeek-GRM-27B is scaled up to 32 samples, versus 8 for the other methods.
- On Reward Bench, plain voting over 32 samples roughly matches a 671B MoE model. Meta-RM-guided voting reaches the top result using only 8 samples.
- DeepSeek-R1-0120 was tested on a 300-example subset and scored below a 236B MoE RFT model. This indicates that long chain-of-thought reasoning does little for generalist reward modeling.
- The models are released on Hugging Face and ModelScope. Remaining issues include efficiency, certain task types, and potentially unfaithful self-generated principles.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 137 works · 83,262 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu and Yu Wu. 2025. Inference-Time Scaling for Generalist Reward Modeling. arXiv:2504.02495.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{liu2025inference,
  title = {{Inference-Time Scaling for Generalist Reward Modeling}},
  author = {Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu and Yu Wu},
  year = {2025},
  doi = {10.48550/arxiv.2504.02495},
  eprint = {2504.02495},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2504.02495}
}
```

- [← previous CoT-Valve: Length-Compressible Chain-of-Thought Tuning](https://md.research.elamur.ai/free/en/papers/ma-2025-cot-valve-length-compressible-chain/)
- [next → Seed1.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/guo-2025-seed1-5-vl-technical/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

88 references, 32 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Andrei Alexandru, Antonia Calvi, Henry Broomfield, Jackson Golden, Kyle Dai, Mathias Leys, Maurice Burger, Max Bartolo, Roman Engeler, Sashank Pisupati, Toby Drane, and Young Sun Park. Atla selene mini: A general purpose evaluation model. Computing Research Repository, arXiv:2501.17195, 2025. URL ht… — [arXiv:2501.17195](https://arxiv.org/abs/2501.17195) [find](https://scholar.google.com/scholar?q=Andrei+Alexandru%2C+Antonia+Calvi%2C+Henry+Broomfield%2C+Jackson+Golden%2C+Kyle+Dai%2C+Mathias+Leys%2C+Maurice+Burger%2C+Max+Bartolo%2C+Roman+Engeler%2C+Sashank+Pisupati%2C+Toby+Drane%2C+and+Young+Sun+Park%2E+Atla+selene+min)
2. Wei An, Xiao Bi, Guanting Chen, Shanhuang Chen, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Wenjun Gao, Kang Guan, Jianzhong Guo, Yongqiang Guo, Zhe Fu, Ying He, Panpan Huang, Jiashi Li, Wenfeng Liang, Xiaodong Liu, Xin Liu, Yiyuan Liu, Yuxuan Liu, Shanghao Lu, Xuan Lu, Xiaotao Nie, Tian Pei, J… [find](https://scholar.google.com/scholar?q=Wei+An%2C+Xiao+Bi%2C+Guanting+Chen%2C+Shanhuang+Chen%2C+Chengqi+Deng%2C+Honghui+Ding%2C+Kai+Dong%2C+Qiushi+Du%2C+Wenjun+Gao%2C+Kang+Guan%2C+Jianzhong+Guo%2C+Yongqiang+Guo%2C+Zhe+Fu%2C+Ying+He%2C+Panpan+Huang%2C+Jiashi+Li%2C+Wenfeng+)
3. Zachary Ankner, Mansheej Paul, Brandon Cui, Jonathan D. Chang, and Prithviraj Ammanabrolu. Critique-out-loud reward models. Computing Research Repository, arXiv:2408.11791, 2024. URL https://arxiv.org/abs/2408.11791. [↗](https://arxiv.org/abs/2408.11791)
4. Negar Arabzadeh, Siqing Huo, Nikhil Mehta, Qingyun Wu, Chi Wang, Ahmed Hassan Awadallah, Charles L. A. Clarke, and Julia Kiseleva. Assessing and verifying task utility in LLM-powered applications. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empi… [find](https://scholar.google.com/scholar?q=Negar+Arabzadeh%2C+Siqing+Huo%2C+Nikhil+Mehta%2C+Qingyun+Wu%2C+Chi+Wang%2C+Ahmed+Hassan+Awadallah%2C+Charles+L%2E+A%2E+Clarke%2C+and+Julia+Kiseleva%2E+Assessing+and+verifying+task+utility+in+LLM%2Dpowered+applications%2E+In+)
5. Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Jackson Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, a… [find](https://scholar.google.com/scholar?q=Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Andy+Jones%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Nova+DasSarma%2C+Nelson+Elhage%2C+Zac+Hatfield%2DDodds%2C+Danny+Hernandez%2C+Jackson+Kernion%2C)
6. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
7. [Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared…](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
8. Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y. Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. OpenAI Publication, 2025. URL [https://cdn.openai.com/pdf/34f2ada6-870f-4c26-9790-…](https://cdn.openai.com/pdf/34f2ada6-870f-4c26-9790-…) [↗](https://cdn.openai.com/pdf/34f2ada6-870f-4c26-9790-…)
9. Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, and Azalia Mirhoseini. [Large language monkeys: Scaling inference compute with repeated sampling](https://md.research.elamur.ai/free/en/papers/brown-2024-large-language-monkeys-scaling-inference/). Computing Research Repository, arXiv:2407.21787, 2024. URL https://arxiv.org/abs/2407.21787. [↗](https://arxiv.org/abs/2407.21787)
10. [Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjia…](https://md.research.elamur.ai/free/en/papers/cai-2024-internlm2-technical-report/)
11. Maosong Cao, Alexander Lam, Haodong Duan, Hongwei Liu, Songyang Zhang, and Kai Chen. Compassjudger-1: All-in-one judge model helps model evaluation and evolution. Computing Research Repository, arXiv:2410.16256, 2024. URL https://arxiv.org/abs/2410. 16256. — [arXiv:2410.16256](https://arxiv.org/abs/2410.16256) [↗](https://arxiv.org/abs/2410)
12. Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. Codet: Code generation with generated tests. In The Eleventh International Conference on Learning Representations, 2023. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) ktrw68Cmu9c. [↗](https://openreview.net/forum?id=)
13. Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang, Bryan Hooi, and Bingsheng He. Judgelrm: Large reasoning models as a judge. Computing Research Repository, arXiv:2504.00050, 2025. URL https://arxiv.org/abs/2504.00050. — [arXiv:2504.00050](https://arxiv.org/abs/2504.00050) [↗](https://arxiv.org/abs/2504.00050)
14. Yinlam Chow, Guy Tennenholtz, Izzeddin Gur, Vincent Zhuang, Bo Dai, Aviral Kumar, Rishabh Agarwal, Sridhar Thiagarajan, Craig Boutilier, and Aleksandra Faust. Inference-aware fine-tuning for best-of-n sampling in large language models. In The Thirteenth International Conference on Learning Represent… [find](https://scholar.google.com/scholar?q=Yinlam+Chow%2C+Guy+Tennenholtz%2C+Izzeddin+Gur%2C+Vincent+Zhuang%2C+Bo+Dai%2C+Aviral+Kumar%2C+Rishabh+Agarwal%2C+Sridhar+Thiagarajan%2C+Craig+Boutilier%2C+and+Aleksandra+Faust%2E+Inference%2Daware+fine%2Dtuning+for+best%2Dof%2Dn)
15. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). Computing Research Repository, arXiv:2110.14168, 2021. URL http…
16. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. ULTRAFEEDBACK: Boosting language models with scaled AI feedback. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jo… [find](https://scholar.google.com/scholar?q=Ganqu+Cui%2C+Lifan+Yuan%2C+Ning+Ding%2C+Guanming+Yao%2C+Bingxiang+He%2C+Wei+Zhu%2C+Yuan+Ni%2C+Guotong+Xie%2C+Ruobing+Xie%2C+Yankai+Lin%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+ULTRAFEEDBACK%3A+Boosting+language+models+with+scaled+)
17. DeepSeek-AI. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. Computing Research Repository, arXiv:2405.04434, 2024a. URL [https://arxiv.org/](https://arxiv.org/) abs/2405.04434. [↗](https://arxiv.org/)
18. DeepSeek-AI. Deepseek-v3 technical report. Computing Research Repository, arXiv:2412.19437, 2024b. URL https://arxiv.org/abs/2412.19437. [↗](https://arxiv.org/abs/2412.19437)
19. DeepSeek-AI. Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature, 645(7952):[633–638, September 2025](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). doi: 10.1038/s41586-025-09422-z.
20. Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding dataset difficulty with V -usable information. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 o… [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2C+Yejin+Choi%2C+and+Swabha+Swayamdipta%2E+Understanding+dataset+difficulty+with+V+%2Dusable+information%2E+In+Kamalika+Chaudhuri%2C+Stefanie+Jegelka%2C+Le+Song%2C+Csaba+Szepesvari%2C+Gang+Niu%2C+and+Siv)
21. Jan-Philipp Fränken, Eric Zelikman, Rafael Rafailov, Kanishk Gandhi, Tobias Gerstenberg, and Noah Goodman. Self-supervised alignment with mutual information: Learning to follow principles without preference labels. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024… [find](https://scholar.google.com/scholar?q=Jan%2DPhilipp+Fr%C3%A4nken%2C+Eric+Zelikman%2C+Rafael+Rafailov%2C+Kanishk+Gandhi%2C+Tobias+Gerstenberg%2C+and+Noah+Goodman%2E+Self%2Dsupervised+alignment+with+mutual+information%3A+Learning+to+follow+principles+without+pref)
22. Evan Frick, Tianle Li, Connor Chen, Wei-Lin Chiang, Anastasios Nikolas Angelopoulos, Jiantao Jiao, Banghua Zhu, Joseph E. Gonzalez, and Ion Stoica. How to evaluate reward models for RLHF. In The Thirteenth International Conference on Learning Representations, 2025. URL [https://openreview.net/forum?i…](https://openreview.net/forum?i…) [↗](https://openreview.net/forum?i…)
23. Leo Gao, John Schulman, and Jacob Hilton. [Scaling laws for reward model overoptimization](https://md.research.elamur.ai/free/en/papers/gao-2022-scaling-laws-reward-model-overoptimization/). In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings…
24. [Amelia Glaese, Nat McAleese, Maja Tr˛ebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug F…](https://md.research.elamur.ai/free/en/papers/glaese-2022-improving-alignment-dialogue-agents-targeted/)
25. Fang Guo, Wenyu Li, Honglei Zhuang, Yun Luo, Yafu Li, Le Yan, Qi Zhu, and Yue Zhang. Mcranker: Generating diverse criteria on-the-fly to improve pointwise llm rankers. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM ’25, pp. 944–953, New York, NY, US… — [doi:10.1145/3701551.3703583](https://doi.org/10.1145/3701551.3703583) [find](https://scholar.google.com/scholar?q=Fang+Guo%2C+Wenyu+Li%2C+Honglei+Zhuang%2C+Yun+Luo%2C+Yafu+Li%2C+Le+Yan%2C+Qi+Zhu%2C+and+Yue+Zhang%2E+Mcranker%3A+Generating+diverse+criteria+on%2Dthe%2Dfly+to+improve+pointwise+llm+rankers%2E+In+Proceedings+of+the+Eighteenth)
26. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. [Measuring mathematical problem solving with the MATH dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL ht…
27. Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. [LLM-blender: Ensembling large language models with pairwise ranking and generative fusion](https://md.research.elamur.ai/free/en/papers/jiang-2023-llm-blender-ensembling-large-language/). In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: L…
28. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. [SWE-bench: Can language models resolve real-world github issues?](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/) In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=VTF8yNQM66](https://openreview.net/forum?id=VTF8yNQM66). [↗](https://openreview.net/forum?id=VTF8yNQM66)
29. Ryo Kamoi, Sarkar Snigdha Sarathi Das, Renze Lou, Jihyun Janice Ahn, Yilun Zhao, Xiaoxin Lu, Nan Zhang, Yusen Zhang, Haoran Ranran Zhang, Sujeeth Reddy Vummanthala, Salika Dave, Shaobo Qin, Arman Cohan, Wenpeng Yin, and Rui Zhang. Evaluating LLMs at detecting errors in LLM responses. In First Confer… [find](https://scholar.google.com/scholar?q=Ryo+Kamoi%2C+Sarkar+Snigdha+Sarathi+Das%2C+Renze+Lou%2C+Jihyun+Janice+Ahn%2C+Yilun+Zhao%2C+Xiaoxin+Lu%2C+Nan+Zhang%2C+Yusen+Zhang%2C+Haoran+Ranran+Zhang%2C+Sujeeth+Reddy+Vummanthala%2C+Salika+Dave%2C+Shaobo+Qin%2C+Arman+Coha)
30. M. G. Kendall and B. Babington Smith. On the method of paired comparisons. Biometrika, 31(3/4):324–345, 1940. ISSN 00063444. URL [http://www.jstor.org/stable/2332613](http://www.jstor.org/stable/2332613). [↗](http://www.jstor.org/stable/2332613)
31. Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. [Prometheus 2: An open source language model specialized in evaluating other language models](https://md.research.elamur.ai/free/en/papers/kim-2024-prometheus-2-open-source-language/). In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), P…
32. Seungone Kim, Ian Wu, Jinu Lee, Xiang Yue, Seongyun Lee, Mingyeong Moon, Kiril Gashteovski, Carolin Lawrence, Julia Hockenmaier, Graham Neubig, and Sean Welleck. Scaling evaluation-time compute with reasoning models as process evaluators. Computing Research Repository, arXiv:2503.19877, 2025. URL ht… — [arXiv:2503.19877](https://arxiv.org/abs/2503.19877) [find](https://scholar.google.com/scholar?q=Seungone+Kim%2C+Ian+Wu%2C+Jinu+Lee%2C+Xiang+Yue%2C+Seongyun+Lee%2C+Mingyeong+Moon%2C+Kiril+Gashteovski%2C+Carolin+Lawrence%2C+Julia+Hockenmaier%2C+Graham+Neubig%2C+and+Sean+Welleck%2E+Scaling+evaluation%2Dtime+compute+with+r)
33. Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. Rewardbench: Evaluating reward models for language modeling. Computing Research Repository, arXiv:2403.13787, 2024.… [find](https://scholar.google.com/scholar?q=Nathan+Lambert%2C+Valentina+Pyatkin%2C+Jacob+Morrison%2C+LJ+Miranda%2C+Bill+Yuchen+Lin%2C+Khyathi+Chandu%2C+Nouha+Dziri%2C+Sachin+Kumar%2C+Tom+Zick%2C+Yejin+Choi%2C+Noah+A%2E+Smith%2C+and+Hannaneh+Hajishirzi%2E+Rewardbench%3A+Ev)
34. Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, hai zhao, and Pengfei Liu. [Generative judge for evaluating alignment](https://md.research.elamur.ai/free/en/papers/li-2023-generative-judge-evaluating-alignment/). In The Twelfth International Conference on Learning Representations, 2024a. URL [https://openreview.net/forum?id=gtkFw6sZGS](https://openreview.net/forum?id=gtkFw6sZGS). [↗](https://openreview.net/forum?id=gtkFw6sZGS)
35. Lei Li, Yekun Chai, Shuohuan Wang, Yu Sun, Hao Tian, Ningyu Zhang, and Hua Wu. Tool-augmented reward modeling. In The Twelfth International Conference on Learning Representations, 2024b. URL [https://openreview.net/forum?id=d94x0gWTUX](https://openreview.net/forum?id=d94x0gWTUX). [↗](https://openreview.net/forum?id=d94x0gWTUX)
36. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models. [https://github.com/tatsu-lab/alpaca_eval](https://github.com/tatsu-lab/alpaca_eval), 5 2023. [↗](https://github.com/tatsu-lab/alpaca_eval)
37. Shalev Lifshitz, Sheila A. McIlraith, and Yilun Du. Multi-agent verification: Scaling test-time compute with multiple verifiers. In Second Conference on Language Modeling, 2025. URL [https://openreview.net/forum?id=LriQ3NY9uL](https://openreview.net/forum?id=LriQ3NY9uL). [↗](https://openreview.net/forum?id=LriQ3NY9uL)
38. [Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2024. URL](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/) [https://openreview.net/forum?id=v8L0pN6EOi](https://openreview.net/forum?id=v8L0pN6EOi). [↗](https://openreview.net/forum?id=v8L0pN6EOi)
39. Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou. [Skywork-reward: Bag of tricks for reward modeling in llms](https://md.research.elamur.ai/free/en/papers/liu-2024-skywork-reward-bag-tricks/). Computing Research Repository, arXiv:2410.18451, 2024. URL https://arxiv.org/abs/2410. 18451. [↗](https://arxiv.org/abs/2410)
40. Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. Pairjudge rm: Perform best-of-n sampling with knockout tournament. Computing Research Repository, arXiv:2501.13007, 2025. URL https://arxiv.org/abs/2501.13007. — [arXiv:2501.13007](https://arxiv.org/abs/2501.13007) [↗](https://arxiv.org/abs/2501.13007)
41. Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. Generative reward models. Computing Research Repository, arXiv:2410.12832, 2024. URL https://arxiv.org/abs/2410. 12832. [↗](https://arxiv.org/abs/2410)
42. Tong Mu, Alec Helyar, Johannes Heidecke, Joshua Achiam, Andrea Vallone, Ian D Kivlichan, Molly Lin, Alex Beutel, John Schulman, and Lilian Weng. Rule based rewards for language model safety. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview… [find](https://scholar.google.com/scholar?q=Tong+Mu%2C+Alec+Helyar%2C+Johannes+Heidecke%2C+Joshua+Achiam%2C+Andrea+Vallone%2C+Ian+D+Kivlichan%2C+Molly+Lin%2C+Alex+Beutel%2C+John+Schulman%2C+and+Lilian+Weng%2E+Rule+based+rewards+for+language+model+safety%2E+In+The+Th)
43. Niklas Muennighoff, Qian Liu, Armel Randy Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Longpre. Octopack: Instruction tuning code large language models. In The Twelfth International Conference on Learning Representations, 2024. URL http… [find](https://scholar.google.com/scholar?q=Niklas+Muennighoff%2C+Qian+Liu%2C+Armel+Randy+Zebaze%2C+Qinkai+Zheng%2C+Binyuan+Hui%2C+Terry+Yue+Zhuo%2C+Swayam+Singh%2C+Xiangru+Tang%2C+Leandro+Von+Werra%2C+and+Shayne+Longpre%2E+Octopack%3A+Instruction+tuning+code+large+)
44. OpenAI. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). Computing Research Repository, arXiv:2412.16720, 2024. URL https://arxiv.org/abs/2412.16720. [↗](https://arxiv.org/abs/2412.16720)
45. OpenAI. Deep research system card. OpenAI Publication, 2025a. URL [https://cdn.openai.com/deep-research-system-card.pdf](https://cdn.openai.com/deep-research-system-card.pdf). [↗](https://cdn.openai.com/deep-research-system-card.pdf)
46. OpenAI. Openai gpt-4.5 system card. OpenAI Publication, 2025b. URL [https://cdn.openai.com/gpt-4-5-system-card-2272025.pdf](https://cdn.openai.com/gpt-4-5-system-card-2272025.pdf). [↗](https://cdn.openai.com/gpt-4-5-system-card-2272025.pdf)
47. OpenAI. Openai o3-mini system card. OpenAI Publication, 2025c. URL [https://cdn.openai.com/o3-mini-system-card-feb10.pdf](https://cdn.openai.com/o3-mini-system-card-feb10.pdf). [↗](https://cdn.openai.com/o3-mini-system-card-feb10.pdf)
48. [Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training la…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
49. Junsoo Park, Seungyeon Jwa, Ren Meiying, Daeyoung Kim, and Sanghyuk Choi. [OffsetBias: Leveraging debiased data for tuning evaluators](https://md.research.elamur.ai/free/en/papers/park-2024-offsetbias-leveraging-debiased-data-tuning/). In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 1043–1067, Miami, Florida, USA…
50. Hao Peng, Yunjia Qi, Xiaozhi Wang, Zijun Yao, Bin Xu, Lei Hou, and Juanzi Li. Agentic reward modeling: Integrating human preferences with verifiable correctness signals for reliable reward systems. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of… [find](https://scholar.google.com/scholar?q=Hao+Peng%2C+Yunjia+Qi%2C+Xiaozhi+Wang%2C+Zijun+Yao%2C+Bin+Xu%2C+Lei+Hou%2C+and+Juanzi+Li%2E+Agentic+reward+modeling%3A+Integrating+human+preferences+with+verifiable+correctness+signals+for+reliable+reward+systems%2E+In)
51. Paul Röttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. [XSTest: A test suite for identifying exaggerated safety behaviours in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/). In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024
52. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 5377–5400, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.naacl-long. 301. URL [https://aclanthology.or…](https://aclanthology.or…) — [doi:10.18653/v1/2024.naacl-long](https://doi.org/10.18653/v1/2024.naacl-long) [↗](https://aclanthology.or…)
53. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). Computing Research Repository, arXiv:2402.0330, 2024. URL https://arxiv.org/abs/240… [↗](https://arxiv.org/abs/240…)
54. Mrinank Sharma, Meg Tong, Jesse Mu, Jerry Wei, Jorrit Kruthoff, Scott Goodfriend, Euan Ong, Alwin Peng, Raj Agarwal, Cem Anil, Amanda Askell, Nathan Bailey, Joe Benton, Emma Bluemke, Samuel R. Bowman, Eric Christiansen, Hoagy Cunningham, Andy Dau, Anjali Gopal, Rob Gilson, Logan Graham, Logan Howard… [find](https://scholar.google.com/scholar?q=Mrinank+Sharma%2C+Meg+Tong%2C+Jesse+Mu%2C+Jerry+Wei%2C+Jorrit+Kruthoff%2C+Scott+Goodfriend%2C+Euan+Ong%2C+Alwin+Peng%2C+Raj+Agarwal%2C+Cem+Anil%2C+Amanda+Askell%2C+Nathan+Bailey%2C+Joe+Benton%2C+Emma+Bluemke%2C+Samuel+R%2E+Bowman%2C)
55. Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations, 2025. URL [https://openreview.net/forum?id=4FWAwZtd2n](https://openreview.net/forum?id=4FWAwZtd2n). [↗](https://openreview.net/forum?id=4FWAwZtd2n)
56. Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human feedback. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA, 202… [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeff+Wu%2C+Daniel+M%2E+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+Christiano%2E+Learning+to+summarize+from+human+feedback%2E+In+Proceedings+of+the+34th)
57. Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Daniel Cox, Yiming Yang, and Chuang Gan. [Principle-driven self-alignment of language models from scratch with minimal human supervision](https://md.research.elamur.ai/free/en/papers/sun-2023-principle-driven-self-alignment-language/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https:…
58. Zhiqing Sun, Yikang Shen, Hongxin Zhang, Qinhong Zhou, Zhenfang Chen, David Daniel Cox, Yiming Yang, and Chuang Gan. SALMON: Self-alignment with instructable reward models. In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=xJbsmB8UMx](https://openreview.net/forum?id=xJbsmB8UMx). [↗](https://openreview.net/forum?id=xJbsmB8UMx)
59. Gemma Team. Gemma 2: Improving open language models at a practical size. Computing Research Repository, arXiv:2408.0011, 2024. URL https://arxiv.org/abs/2408.00118. [↗](https://arxiv.org/abs/2408.00118)
60. Hemish Veeraboina. Aime problem set 1983-2024, 2023. URL [https://www.kaggle.com/](https://www.kaggle.com/) datasets/hemishveeraboina/aime-problem-set-1983-2024. [↗](https://www.kaggle.com/)
61. Tu Vu, Kalpesh Krishna, Salaheddin Alzubi, Chris Tar, Manaal Faruqui, and Yun-Hsuan Sung. Foundational autoraters: Taming large language models for better automatic evaluation. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Nat… [find](https://scholar.google.com/scholar?q=Tu+Vu%2C+Kalpesh+Krishna%2C+Salaheddin+Alzubi%2C+Chris+Tar%2C+Manaal+Faruqui%2C+and+Yun%2DHsuan+Sung%2E+Foundational+autoraters%3A+Taming+large+language+models+for+better+automatic+evaluation%2E+In+Yaser+Al%2DOnaizan%2C+Mo)
62. Chenglong Wang, Yang Gan, Yifu Huo, Yongyu Mu, Qiaozhi He, MuRun Yang, Bei Li, Tong Xiao, Chunliang Zhang, Tongran Liu, and JingBo Zhu. GRAM: A generative foundation reward model for reward generalization. In Forty-second International Conference on Machine Learning, 2025a. URL [https://openreview.ne…](https://openreview.ne…) [↗](https://openreview.ne…)
63. Haoxiang Wang, Wei Xiong, Tengyang Xie, Han Zhao, and Tong Zhang. [Interpretable preferences via multi-objective reward modeling and mixture-of-experts](https://md.research.elamur.ai/free/en/papers/wang-2024-interpretable-preferences-multi-objective-reward/). In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 10582–10592,…
64. Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. [Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/). In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for…
65. Tianlu Wang, Ilia Kulikov, Olga Golovneva, Ping Yu, Weizhe Yuan, Jane Dwivedi-Yu, Richard Yuanzhe Pang, Maryam Fazel-Zarandi, Jason Weston, and Xian Li. [Self-taught evaluators](https://md.research.elamur.ai/free/en/papers/wang-2024-self-taught-evaluators/). Computing Research Repository, arXiv:2408.02666, 2024c. URL https://arxiv.org/abs/2408.02666. [↗](https://arxiv.org/abs/2408.02666)
66. Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. [Do-not-answer: Evaluating safeguards in LLMs](https://md.research.elamur.ai/free/en/papers/wang-2024-do-not-answer-evaluating-safeguards/). In Yvette Graham and Matthew Purver (eds.), Findings of the Association for Computational Linguistics: EACL 2024, pp. 896–911, St.
67. Julian’s, Malta, March 2024d. [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/papers/wang-2024-do-not-answer-evaluating-safeguards/). URL https://aclanthology.org/2024.findings-eacl.61/. [↗](https://aclanthology.org/2024.findings-eacl.61/)
68. Zhilin Wang, Yi Dong, Olivier Delalleau, Jiaqi Zeng, Gerald Shen, Daniel Egert, Jimmy J. Zhang, Makesh Narsimhan Sreedhar, and Oleksii Kuchaiev. Helpsteer 2: Open-source dataset for training top-performing reward models. In The Thirty-eight Conference on Neural Information Processing Systems Dataset… [find](https://scholar.google.com/scholar?q=Zhilin+Wang%2C+Yi+Dong%2C+Olivier+Delalleau%2C+Jiaqi+Zeng%2C+Gerald+Shen%2C+Daniel+Egert%2C+Jimmy+J%2E+Zhang%2C+Makesh+Narsimhan+Sreedhar%2C+and+Oleksii+Kuchaiev%2E+Helpsteer+2%3A+Open%2Dsource+dataset+for+training+top%2Dperfo)
69. Zhilin Wang, Alexander Bukharin, Olivier Delalleau, Daniel Egert, Gerald Shen, Jiaqi Zeng, Oleksii Kuchaiev, and Yi Dong. Helpsteer2-preference: Complementing ratings with preferences. In The Thirteenth International Conference on Learning Representations, 2025b. URL [https://openreview.net/forum?id=…](https://openreview.net/forum?id=…) [↗](https://openreview.net/forum?id=…)
70. [Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Proce…](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
71. Genta Indra Winata, David Anugraha, Lucky Susanto, Garry Kuwanto, and Derry Tanti Wijaya. Metametrics: Calibrating metrics for generation tasks using human preferences. In The Thirteenth International Conference on Learning Representations, 2025. URL [https://openreview.net/forum?id=slO3xTt4CG](https://openreview.net/forum?id=slO3xTt4CG). [↗](https://openreview.net/forum?id=slO3xTt4CG)
72. Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason Weston, and Sainbayar Sukhbaatar. Meta-rewarding language models: Self-improving alignment with llm-as-a-meta-judge. Computing Research Repository, arXiv:2407.19594, 2024. URL https://arxiv.org/abs/2407.19594. [↗](https://arxiv.org/abs/2407.19594)
73. Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for LLM problem-solving. In The Thirteenth International Conference on Learning Representations, 2025. URL [https://openreview.net/forum?id=VNckp7JEHn](https://openreview.net/forum?id=VNckp7JEHn). [↗](https://openreview.net/forum?id=VNckp7JEHn)
74. [Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. OSWorld: Benchmarking multimodal agents for open-ended tasks in real…](https://md.research.elamur.ai/free/en/papers/xie-2024-osworld-benchmarking-multimodal-agents-open/)
75. Shunyu Yao, Howard Chen, John Yang, and Karthik R Narasimhan. [Webshop: Towards scalable real-world web interaction with grounded language agents](https://md.research.elamur.ai/free/en/papers/yao-2022-webshop-towards-scalable-real-world/). In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openrevie…
76. Zihuiwen Ye, Fraser David Greenlee, Max Bartolo, Phil Blunsom, Jon Ander Campos, and Matthias Gallé. Improving reward models with synthetic critiques. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Findings of the Association for Computational Linguistics: NAACL 2025, pp. 4506–4520, Albuquerque,… [find](https://scholar.google.com/scholar?q=Zihuiwen+Ye%2C+Fraser+David+Greenlee%2C+Max+Bartolo%2C+Phil+Blunsom%2C+Jon+Ander+Campos%2C+and+Matthias+Gall%C3%A9%2E+Improving+reward+models+with+synthetic+critiques%2E+In+Luis+Chiruzzo%2C+Alan+Ritter%2C+and+Lu+Wang+%28eds%2E%29)
77. Ziyi Ye, Xiangsheng Li, Qiuchi Li, Qingyao Ai, Yujia Zhou, Wei Shen, Dong Yan, and Yiqun LIU. Learning LLM-as-a-judge for preference alignment. In The Thirteenth International Conference on Learning Representations, 2025b. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) HZVIQE1MsJ. [↗](https://openreview.net/forum?id=)
78. Jiachen Yu, Shaoning Sun, Xiaohui Hu, Jiaxu Yan, Kaidong Yu, and Xuelong Li. Improve llm-as-a-judge ability as a general ability. Computing Research Repository, arXiv:2502.11689, 2025a. URL https://arxiv.org/abs/2502.11689. [↗](https://arxiv.org/abs/2502.11689)
79. Yue Yu, Zhengxing Chen, Aston Zhang, Liang Tan, Chenguang Zhu, Richard Yuanzhe Pang, Yundi Qian, Xuewei Wang, Suchin Gururangan, Chao Zhang, Melanie Kambadur, Dhruv Mahajan, and Rui Hou. Self-generated critiques boost reward modeling for language models. In Luis Chiruzzo, Alan Ritter, and Lu Wang (e… [find](https://scholar.google.com/scholar?q=Yue+Yu%2C+Zhengxing+Chen%2C+Aston+Zhang%2C+Liang+Tan%2C+Chenguang+Zhu%2C+Richard+Yuanzhe+Pang%2C+Yundi+Qian%2C+Xuewei+Wang%2C+Suchin+Gururangan%2C+Chao+Zhang%2C+Melanie+Kambadur%2C+Dhruv+Mahajan%2C+and+Rui+Hou%2E+Self%2Dgenerate)
80. Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason E Weston. Self-rewarding language models. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st I… [find](https://scholar.google.com/scholar?q=Weizhe+Yuan%2C+Richard+Yuanzhe+Pang%2C+Kyunghyun+Cho%2C+Xian+Li%2C+Sainbayar+Sukhbaatar%2C+Jing+Xu%2C+and+Jason+E+Weston%2E+Self%2Drewarding+language+models%2E+In+Ruslan+Salakhutdinov%2C+Zico+Kolter%2C+Katherine+Heller%2C+Ad)
81. Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. Evaluating large language models at evaluating instruction following. In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) tr0KidwPLc. [↗](https://openreview.net/forum?id=)
82. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. [Generative verifiers: Reward modeling as next-token prediction](https://md.research.elamur.ai/free/en/papers/zhang-2024-generative-verifiers-reward-modeling-as/). In The Thirteenth International Conference on Learning Representations, 2025a. URL [https://openreview.net/forum?id=Ccwp4tFEtE](https://openreview.net/forum?id=Ccwp4tFEtE). [↗](https://openreview.net/forum?id=Ccwp4tFEtE)
83. Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. [The lessons of developing process reward models in mathematical reasoning](https://md.research.elamur.ai/free/en/papers/zhang-2025-lessons-developing-process-reward-models/). In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the…
84. Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. [ProcessBench: Identifying process errors in mathematical reasoning](https://md.research.elamur.ai/free/en/papers/zheng-2025-processbench-identifying-process-errors-mathematical/). In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd A…
85. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In Proceedings of the 37th International Conference on Neural In… [find](https://scholar.google.com/scholar?q=Lianmin+Zheng%2C+Wei%2DLin+Chiang%2C+Ying+Sheng%2C+Siyuan+Zhuang%2C+Zhanghao+Wu%2C+Yonghao+Zhuang%2C+Zi+Lin%2C+Zhuohan+Li%2C+Dacheng+Li%2C+Eric+P%2E+Xing%2C+Hao+Zhang%2C+Joseph+E%2E+Gonzalez%2C+and+Ion+Stoica%2E+Judging+llm%2Das%2Da%2Djud)
86. Enyu Zhou, Guodong Zheng, Binghai Wang, Zhiheng Xi, Shihan Dou, Rong Bao, Wei Shen, Limao Xiong, Jessica Fan, Yurong Mou, Rui Zheng, Tao Gui, Qi Zhang, and Xuanjing Huang. RMB: Comprehensively benchmarking reward models in LLM alignment. In The Thirteenth International Conference on Learning Represe… [find](https://scholar.google.com/scholar?q=Enyu+Zhou%2C+Guodong+Zheng%2C+Binghai+Wang%2C+Zhiheng+Xi%2C+Shihan+Dou%2C+Rong+Bao%2C+Wei+Shen%2C+Limao+Xiong%2C+Jessica+Fan%2C+Yurong+Mou%2C+Rui+Zheng%2C+Tao+Gui%2C+Qi+Zhang%2C+and+Xuanjing+Huang%2E+RMB%3A+Comprehensively+benchma)
87. [Terry Yue Zhuo, Vu Minh Chien, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, Simon Brunner, Chen GONG, James Hoang, Armel Randy Zebaze, Xiaoheng Hong, Wen-Ding Li, Jean Kaddour, Ming Xu, Zhihan Zhang, Prateek Yadav, Naman Jain, Alex G…](https://md.research.elamur.ai/free/en/papers/zhuo-2024-bigcodebench-benchmarking-code-generation-diverse/)
88. Contents \#\# Introduction Preliminaries 2.1 Comparisons of Different RM approaches . . . . . . . . . . . . . . . . . . . . 2.2 Boosting Reward Quality with Principles . . . . . . . . . . . . . . . . . . . . Self-Principled Critique Tuning (SPCT) 3.1 Unpinning Principles from Understanding to Generati… [find](https://scholar.google.com/scholar?q=Contents+%5C%23%5C%23+Introduction+Preliminaries+2%2E1+Comparisons+of+Different+RM+approaches+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+2%2E2+Boosting+Reward+Quality+with+Principles+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+)

*Updated 24 October 2026*