---
title: "Outcome Accuracy is Not Enough — paper by Wang et al. (2026)"
description: "They propose Rationale Consistency, which measures how many human reasons a model recovers. 48 references, cited by 3 papers here."
authors: ["Binghai Wang", "Yantao Liu", "Yuxuan Liu", "Tianyi Tang", "Shenzhi Wang", "Chang Gao", "Chujie Zheng", "Yichang Zhang", "Le Yu", "Shixuan Liu", "Tao Gui", "Qi Zhang", "Xuanjing Huang", "Bowen Yu", "Fei Huang", "Junyang Lin"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1924/"
doi: "10.18653/v1/2026.acl-long.1924"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-outcome-accuracy-is-not-enough/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-outcome-accuracy-is-not-enough/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/) / Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models

# Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models

**Authors:** [Binghai Wang](https://md.research.elamur.ai/free/en/authors/binghai-wang/), [Yantao Liu](https://md.research.elamur.ai/free/en/authors/yantao-liu/), [Yuxuan Liu](https://md.research.elamur.ai/free/en/authors/yuxuan-liu/), [Tianyi Tang](https://md.research.elamur.ai/free/en/authors/tianyi-tang/), [Shenzhi Wang](https://md.research.elamur.ai/free/en/authors/shenzhi-wang/), [Chang Gao](https://md.research.elamur.ai/free/en/authors/chang-gao/), [Chujie Zheng](https://md.research.elamur.ai/free/en/authors/chujie-zheng/), [Yichang Zhang](https://md.research.elamur.ai/free/en/authors/yichang-zhang/), [Le Yu](https://md.research.elamur.ai/free/en/authors/le-yu/), [Shixuan Liu](https://md.research.elamur.ai/free/en/authors/shixuan-liu/), [Tao Gui](https://md.research.elamur.ai/free/en/authors/tao-gui/), [Qi Zhang](https://md.research.elamur.ai/free/en/authors/qi-zhang/), [Xuanjing Huang](https://md.research.elamur.ai/free/en/authors/xuanjing-huang/), [Bowen Yu](https://md.research.elamur.ai/free/en/authors/bowen-yu/), [Fei Huang](https://md.research.elamur.ai/free/en/authors/fei-huang/) and [Junyang Lin](https://md.research.elamur.ai/free/en/authors/junyang-lin/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1924/) · [PDF](https://aclanthology.org/2026.acl-long.1924.pdf) · [DOI 10.18653/v1/2026.acl-long.1924](https://doi.org/10.18653/v1/2026.acl-long.1924) · [OpenAlex](https://openalex.org/W7166808443)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 3 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Generative reward models (GenRMs) and LLM-as-a-Judge systems are usually trained and evaluated only on whether they pick the preferred response. The authors argue that this lets a judge reach the right verdict through shallow or wrong reasoning, a form of deceptive alignment that hurts generalization in RLHF. They propose Rationale Consistency, which measures how many human reasons a model recovers. It is computed by METAJUDGE: human rationales are split into atomic units, and an LLM matches them strictly one-to-one against the model's listed reasons.

Across 19 frontier LLMs, the metric separates top models and exposes shallow judges, while outcome accuracy nearly saturates and does neither. The authors then train GenRMs with GRPO on a hybrid reward that pays off only when both the verdict and the rationale are correct. This yields state-of-the-art reward-model results and better downstream RLHF. The work matters because it shows that reasoning quality, not just verdict accuracy, should be supervised.

## Key points
- **Benchmarks:** Two atomic-rationale benchmarks were built.
  - HelpSteer3-Atomic: 250 HelpSteer3 examples per domain, decomposed by GPT-5 and limited to 3–7 points each.
  - CW-Atomic: 350 creative-writing samples, each labeled by three annotators, reduced to 207 agreed cases.
- **Frontier model scores:** Even the strongest models recover under 40% of human rationales. In one case study, o3 and o3-mini both reach the correct verdict, but o3 matches 75% of the human reasons and o3-mini matches 0%.
- **Reward-model results:** The hybrid-reward GenRM scores 87.1% on RM-Bench and 82.0% on JudgeBench. These are gains of about 3 and 7 points over outcome-only training.
- **Downstream RLHF:** When this GenRM supplies the rewards during RLHF, creative writing on Arena Hard v2 improves by about 7%.
- **Rationale degeneration:**
  - Outcome-only training gradually replaces evidence-grounded reasoning with generic or style-based justifications.
  - The hybrid method raises rationale consistency from 25% to 37% and brings evidence-grounded reasoning up to 98.7%.
  - The main limitation is a dependence on costly expert annotation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/) — University, Shanghai, China · 167 works · 20,987 citations
- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 168 works · 15,386 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Binghai Wang, Yantao Liu, Yuxuan Liu, Tianyi Tang, Shenzhi Wang, Chang Gao, Chujie Zheng, Yichang Zhang, Le Yu, Shixuan Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Bowen Yu, Fei Huang and Junyang Lin. 2026. Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026outcome,
  title = {{Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models}},
  author = {Binghai Wang, Yantao Liu, Yuxuan Liu, Tianyi Tang, Shenzhi Wang, Chang Gao, Chujie Zheng, Yichang Zhang, Le Yu, Shixuan Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Bowen Yu, Fei Huang and Junyang Lin},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1924},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1924}
}
```

- [← previous Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation](https://md.research.elamur.ai/free/en/papers/xiao-2026-not-all-directions-matter-structured/)
- [next → P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs](https://md.research.elamur.ai/free/en/papers/zhao-2026-p2p-poison-poison-remedy-reliable/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/)

## References

48 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abhishek Anand, Negar Mokhberian, Prathyusha Kumar, Anweasha Saha, Zihao He, Ashwin Rao, Fred Morstatter, and Kristina Lerman. 2024. Don’t blame the data, blame the model: Understanding noise and bias when learning from subjective annotations. In Proceedings of the 1st Workshop on Uncertainty-Aware… [find](https://scholar.google.com/scholar?q=Abhishek+Anand%2C+Negar+Mokhberian%2C+Prathyusha+Kumar%2C+Anweasha+Saha%2C+Zihao+He%2C+Ashwin+Rao%2C+Fred+Morstatter%2C+and+Kristina+Lerman%2E+2024%2E+Don%E2%80%99t+blame+the+data%2C+blame+the+model%3A+Understanding+noise+and+bias)
2. Anthropic. 2024. Claude 3.5 sonnet. Accessed: 2026-01-06. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2024%2E+Claude+3%2E5+sonnet%2E+Accessed%3A+2026%2D01%2D06%2E)
3. Oliver Bentham, Nathan Stringham, and Ana Marasović. 2024. Chain-of-thought unfaithfulness as disguised accuracy. arXiv preprint arXiv:2402.14897. — [arXiv:2402.14897](https://arxiv.org/abs/2402.14897) [find](https://scholar.google.com/scholar?q=Oliver+Bentham%2C+Nathan+Stringham%2C+and+Ana+Marasovic%CC%81%2E+2024%2E+Chain%2Dof%2Dthought+unfaithfulness+as+disguised+accuracy%2E+arXiv+preprint+arXiv%3A2402%2E14897%2E+%E2%80%94+%5BarXiv%3A2402%2E14897%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E1489)
4. Joe Carlsmith. 2023. Scheming ais: Will ais fake alignment during training in order to get power? arXiv preprint arXiv:2311.08379. — [arXiv:2311.08379](https://arxiv.org/abs/2311.08379) [find](https://scholar.google.com/scholar?q=Joe+Carlsmith%2E+2023%2E+Scheming+ais%3A+Will+ais+fake+alignment+during+training+in+order+to+get+power%3F+arXiv+preprint+arXiv%3A2311%2E08379%2E+%E2%80%94+%5BarXiv%3A2311%2E08379%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2311%2E08379%29)
5. Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, Tony Wang, Samuel Marks, Charbel-Raphaël Segerie, Micah Carroll, Andi Peng, Phillip Christoffersen, Mehul Damani, Stewart Slocum, Usman Anwar… [find](https://scholar.google.com/scholar?q=Stephen+Casper%2C+Xander+Davies%2C+Claudia+Shi%2C+Thomas+Krendl+Gilbert%2C+J%C3%A9r%C3%A9my+Scheurer%2C+Javier+Rando%2C+Rachel+Freedman%2C+Tomasz+Korbak%2C+David+Lindner%2C+Pedro+Freire%2C+Tony+Wang%2C+Samuel+Marks%2C+Charbel%2DRapha%C3%ABl+)
6. Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, Hanghang Tong, and Heng Ji. 2025. [Rm-r1: Reward modeling as reasoning](https://md.research.elamur.ai/free/en/papers/chen-2025-rm-r1-reward-modeling-reasoning/). arXiv preprint arXiv:2505.02387.
7. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). Nature, 645:633–638.
8. Qingxiu Dong, Li Dong, Xingxing Zhang, Zhifang Sui, and Furu Wei. 2024. Self-boosting large language models with synthetic preference data. arXiv preprint arXiv:2410.06961. — [arXiv:2410.06961](https://arxiv.org/abs/2410.06961) [find](https://scholar.google.com/scholar?q=Qingxiu+Dong%2C+Li+Dong%2C+Xingxing+Zhang%2C+Zhifang+Sui%2C+and+Furu+Wei%2E+2024%2E+Self%2Dboosting+large+language+models+with+synthetic+preference+data%2E+arXiv+preprint+arXiv%3A2410%2E06961%2E+%E2%80%94+%5BarXiv%3A2410%2E06961%5D%28https%3A)
9. Leo Gao, John Schulman, and Jacob Hilton. 2023. [Scaling laws for reward model overoptimization](https://md.research.elamur.ai/free/en/papers/gao-2022-scaling-laws-reward-model-overoptimization/). In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 10835–10866. PMLR.
10. Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. 2020. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673. [find](https://scholar.google.com/scholar?q=Robert+Geirhos%2C+J%C3%B6rn%2DHenrik+Jacobsen%2C+Claudio+Michaelis%2C+Richard+Zemel%2C+Wieland+Brendel%2C+Matthias+Bethge%2C+and+Felix+A+Wichmann%2E+2020%2E+Shortcut+learning+in+deep+neural+networks%2E+Nature+Machine+Intellig)
11. Google DeepMind. 2025a. Gemini 3 flash model card. Technical report, Google DeepMind. [find](https://scholar.google.com/scholar?q=Google+DeepMind%2E+2025a%2E+Gemini+3+flash+model+card%2E+Technical+report%2C+Google+DeepMind%2E)
12. Google DeepMind. 2025b. Gemini 3 pro model card. Technical report, Google DeepMind. [find](https://scholar.google.com/scholar?q=Google+DeepMind%2E+2025b%2E+Gemini+3+pro+model+card%2E+Technical+report%2C+Google+DeepMind%2E)
13. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. 2024. A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594. [find](https://scholar.google.com/scholar?q=Jiawei+Gu%2C+Xuhui+Jiang%2C+Zhichao+Shi%2C+Hexiang+Tan%2C+Xuehao+Zhai%2C+Chengjin+Xu%2C+Wei+Li%2C+Yinghan+Shen%2C+Shengjie+Ma%2C+Honghao+Liu%2C+Saizhuo+Wang%2C+Kun+Zhang%2C+Yuanzhuo+Wang%2C+Wen+Gao%2C+Lionel+Ni%2C+and+Jian+Guo%2E+20)
14. Jiaxin Guo, Zewen Chi, Li Dong, Qingxiu Dong, Xun Wu, Shaohan Huang, and Furu Wei. 2025. Reward reasoning model. arXiv preprint arXiv:2505.14674. — [arXiv:2505.14674](https://arxiv.org/abs/2505.14674) [find](https://scholar.google.com/scholar?q=Jiaxin+Guo%2C+Zewen+Chi%2C+Li+Dong%2C+Qingxiu+Dong%2C+Xun+Wu%2C+Shaohan+Huang%2C+and+Furu+Wei%2E+2025%2E+Reward+reasoning+model%2E+arXiv+preprint+arXiv%3A2505%2E14674%2E+%E2%80%94+%5BarXiv%3A2505%2E14674%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2505%2E14674%29)
15. Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. 2019. Risks from learned optimization in advanced machine learning systems. arXiv preprint arXiv:1906.01820. — [arXiv:1906.01820](https://arxiv.org/abs/1906.01820) [find](https://scholar.google.com/scholar?q=Evan+Hubinger%2C+Chris+van+Merwijk%2C+Vladimir+Mikulik%2C+Joar+Skalse%2C+and+Scott+Garrabrant%2E+2019%2E+Risks+from+learned+optimization+in+advanced+machine+learning+systems%2E+arXiv+preprint+arXiv%3A1906%2E01820%2E+%E2%80%94+%5Ba)
16. Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Juntao Dai, Boren Zheng, Tianyi Qiu, Jiayi Zhou, Kaile Wang, Boxuan Li, Sirui Han, Yike Guo, and Yaodong Yang. 2024. [Pku-saferlhf: Towards multi-level safety alignment for llms with human preference](https://md.research.elamur.ai/free/en/papers/ji-2025-pku-saferlhf-multi-level-safety/). arXiv preprint arXiv:2406.15513.
17. Seungone Kim, Jamin Shin, Yejin Choi, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Seo. 2024. [Prometheus: Inducing fine-grained evaluation capability in language models](https://md.research.elamur.ai/free/en/papers/kim-2023-prometheus-inducing-fine-grained-evaluation/). In The Twelfth International Conference on Learning Representations,…
18. Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. 2024. Rewardbench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.13787. [find](https://scholar.google.com/scholar?q=Nathan+Lambert%2C+Valentina+Pyatkin%2C+Jacob+Morrison%2C+LJ+Miranda%2C+Bill+Yuchen+Lin%2C+Khyathi+Chandu%2C+Nouha+Dziri%2C+Sachin+Kumar%2C+Tom+Zick%2C+Yejin+Choi%2C+Noah+A%2E+Smith%2C+and+Hannaneh+Hajishirzi%2E+2024%2E+Rewardben)
19. Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu. 2024a. [Generative judge for evaluating alignment](https://md.research.elamur.ai/free/en/papers/li-2023-generative-judge-evaluating-alignment/). In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open-Review.net.
20. Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E. Gonzalez, and Ion Stoica. 2024b. [From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline](https://md.research.elamur.ai/free/en/papers/li-2024-crowdsourced-data-high-quality-benchmarks/). arXiv preprint arXiv:2406.11939.
21. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open-Rev…
22. Zicheng Lin, Zhibin Gou, Tian Liang, Ruilin Luo, Haowei Liu, and Yujiu Yang. 2024. Criticbench: Benchmarking llms for critique-correct reasoning. arXiv preprint arXiv:2402.14809. [find](https://scholar.google.com/scholar?q=Zicheng+Lin%2C+Zhibin+Gou%2C+Tian+Liang%2C+Ruilin+Luo%2C+Haowei+Liu%2C+and+Yujiu+Yang%2E+2024%2E+Criticbench%3A+Benchmarking+llms+for+critique%2Dcorrect+reasoning%2E+arXiv+preprint+arXiv%3A2402%2E14809%2E)
23. Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou. 2024a. [Skywork-reward: Bag of tricks for reward modeling in llms](https://md.research.elamur.ai/free/en/papers/liu-2024-skywork-reward-bag-tricks/). arXiv preprint arXiv:2410.18451.
24. Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. 2024b. [Rm-bench: Benchmarking reward models of language models with subtlety and style](https://md.research.elamur.ai/free/en/papers/liu-2024-rm-bench-benchmarking-reward-models/). arXiv preprint arXiv:2410.16184.
25. Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. 2025. [Inference-time scaling for generalist reward modeling](https://md.research.elamur.ai/free/en/papers/liu-2025-inference-time-scaling-generalist/). arXiv preprint arXiv:2504.02495.
26. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with… [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+Shashank+Gupta%2C+Bodhisattwa+Prasad+Majumder%2C+Katherine+Her)
27. OpenAI. 2025a. Gpt-4.1. Accessed: 2026-01-06. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025a%2E+Gpt%2D4%2E1%2E+Accessed%3A+2026%2D01%2D06%2E)
28. OpenAI. 2025b. Gpt-5 system card. Technical report, OpenAI. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025b%2E+Gpt%2D5+system+card%2E+Technical+report%2C+OpenAI%2E)
29. OpenAI. 2025c. Openai o3 and o4-mini system card. Technical report, OpenAI. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025c%2E+Openai+o3+and+o4%2Dmini+system+card%2E+Technical+report%2C+OpenAI%2E)
30. OpenAI. 2025d. Openai o3-mini system card. Technical report, OpenAI. Version Feb 10. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025d%2E+Openai+o3%2Dmini+system+card%2E+Technical+report%2C+OpenAI%2E+Version+Feb+10%2E)
31. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022.… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
32. Samuel J Paech. 2023. [Eq-bench: An emotional intelligence benchmark for large language models](https://md.research.elamur.ai/free/en/papers/paech-2023-eq-bench-emotional-intelligence-benchmark/). ArXiv preprint, abs/2312.06281.
33. Hinrich Schütze, Christopher D Manning, and Prabhakar Raghavan. 2008. Introduction to information retrieval, volume 39. Cambridge University Press Cambridge. [find](https://scholar.google.com/scholar?q=Hinrich+Sch%C3%BCtze%2C+Christopher+D+Manning%2C+and+Prabhakar+Raghavan%2E+2008%2E+Introduction+to+information+retrieval%2C+volume+39%2E+Cambridge+University+Press+Cambridge%2E)
34. Zhihong Shao, Yuxiang Luo, Chengda Lu, Z. Z. Ren, Jiewen Hu, Tian Ye, Zhibin Gou, Shirong Ma, and Xiaokang Zhang. 2025. [Deepseekmath-v2: Towards self-verifiable mathematical reasoning](https://md.research.elamur.ai/free/en/papers/shao-2025-deepseekmath-v2-self-verifiable/). arXiv preprint arXiv:2511.22570.
35. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
36. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
37. Shichao Sun, Junlong Li, Weizhe Yuan, Ruifeng Yuan, Wenjie Li, and Pengfei Liu. 2024. The critique of critique. ArXiv preprint, abs/2401.04518. [find](https://scholar.google.com/scholar?q=Shichao+Sun%2C+Junlong+Li%2C+Weizhe+Yuan%2C+Ruifeng+Yuan%2C+Wenjie+Li%2C+and+Pengfei+Liu%2E+2024%2E+The+critique+of+critique%2E+ArXiv+preprint%2C+abs%2F2401%2E04518%2E)
38. Zihuiwen Ye, Fraser Greenlee-Scott, Max Bartolo, Phil Blunsom, Jon Ander Campos, and Matthias Gallé. 2024. Improving reward models with synthetic critiques. arXiv preprint arXiv:2405.20850. [find](https://scholar.google.com/scholar?q=Zihuiwen+Ye%2C+Fraser+Greenlee%2DScott%2C+Max+Bartolo%2C+Phil+Blunsom%2C+Jon+Ander+Campos%2C+and+Matthias+Gall%C3%A9%2E+2024%2E+Improving+reward+models+with+synthetic+critiques%2E+arXiv+preprint+arXiv%3A2405%2E20850%2E)
39. Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Y. Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, and Ion Stoica. 2024. Judgebench: A benchmark for evaluating llm-based judges. arXiv preprint arXiv:2410.12784. [find](https://scholar.google.com/scholar?q=Sijun+Tan%2C+Siyuan+Zhuang%2C+Kyle+Montgomery%2C+William+Y%2E+Tang%2C+Alejandro+Cuadron%2C+Chenguang+Wang%2C+Raluca+Ada+Popa%2C+and+Ion+Stoica%2E+2024%2E+Judgebench%3A+A+benchmark+for+evaluating+llm%2Dbased+judges%2E+arXiv+pre)
40. Zhuohao Yu, Jiali Zeng, Weizheng Gu, Yidong Wang, Jindong Wang, Fandong Meng, Jie Zhou, Yue Zhang, Shikun Zhang, and Wei Ye. 2025. Rewardanything: Generalizable principle-following reward models. arXiv preprint arXiv:2506.03637. — [arXiv:2506.03637](https://arxiv.org/abs/2506.03637) [find](https://scholar.google.com/scholar?q=Zhuohao+Yu%2C+Jiali+Zeng%2C+Weizheng+Gu%2C+Yidong+Wang%2C+Jindong+Wang%2C+Fandong+Meng%2C+Jie+Zhou%2C+Yue+Zhang%2C+Shikun+Zhang%2C+and+Wei+Ye%2E+2025%2E+Rewardanything%3A+Generalizable+principle%2Dfollowing+reward+models%2E+arXi)
41. Zhengyang Tang, Ziniu Li, Zhenyang Xiao, Tian Ding, Ruoyu Sun, Benyou Wang, Dayiheng Liu, Fei Huang, Tianyu Liu, Bowen Yu, and Junyang Lin. 2025. Realcritic: Towards effectiveness-driven evaluation of language model critiques. arXiv preprint arXiv:2501.14492. — [arXiv:2501.14492](https://arxiv.org/abs/2501.14492) [find](https://scholar.google.com/scholar?q=Zhengyang+Tang%2C+Ziniu+Li%2C+Zhenyang+Xiao%2C+Tian+Ding%2C+Ruoyu+Sun%2C+Benyou+Wang%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Tianyu+Liu%2C+Bowen+Yu%2C+and+Junyang+Lin%2E+2025%2E+Realcritic%3A+Towards+effectiveness%2Ddriven+evaluation+of)
42. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In Advances in Neural Information Processing Systems 36: A…
43. Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. [Solving math word problems with process- and outcome-based feedback](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/). arXiv preprint arXiv:2211.14275.
44. Chenglong Wang, Yongyu Mu, Hang Zhou, Yifu Huo, Ziming Zhu, Jiali Zeng, Murun Yang, Bei Li, Xiaoyang Hao, Chunliang Zhang, Fandong Meng, Jingbo Zhu, and Tong Xiao. 2025a. Gram-r 2 : Self-training generative foundation reward models for reward reasoning. arXiv preprint arXiv:2509.02492. — [arXiv:2509.02492](https://arxiv.org/abs/2509.02492) [find](https://scholar.google.com/scholar?q=Chenglong+Wang%2C+Yongyu+Mu%2C+Hang+Zhou%2C+Yifu+Huo%2C+Ziming+Zhu%2C+Jiali+Zeng%2C+Murun+Yang%2C+Bei+Li%2C+Xiaoyang+Hao%2C+Chunliang+Zhang%2C+Fandong+Meng%2C+Jingbo+Zhu%2C+and+Tong+Xiao%2E+2025a%2E+Gram%2Dr+2+%3A+Self%2Dtraining+gene)
45. Zhilin Wang, Alexander Bukharin, Olivier Delalleau, Daniel Egert, Gerald Shen, Jiaqi Zeng, Oleksii Kuchaiev, and Yi Dong. 2024. Helpsteer2-preference: Complementing ratings with preferences. arXiv preprint arXiv:2410.01257. [find](https://scholar.google.com/scholar?q=Zhilin+Wang%2C+Alexander+Bukharin%2C+Olivier+Delalleau%2C+Daniel+Egert%2C+Gerald+Shen%2C+Jiaqi+Zeng%2C+Oleksii+Kuchaiev%2C+and+Yi+Dong%2E+2024%2E+Helpsteer2%2Dpreference%3A+Complementing+ratings+with+preferences%2E+arXiv+pre)
46. Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Ellie Evans, Daniel Egert, Hoo-Chang Shin, Felipe Soares, Yi Dong, and Oleksii Kuchaiev. 2025b. Rlbff: Binary flexible feedback to bridge between human feedback & verifiable rewards. arXiv preprint arXiv:2509.21319. — [arXiv:2509.21319](https://arxiv.org/abs/2509.21319) [find](https://scholar.google.com/scholar?q=Zhilin+Wang%2C+Jiaqi+Zeng%2C+Olivier+Delalleau%2C+Ellie+Evans%2C+Daniel+Egert%2C+Hoo%2DChang+Shin%2C+Felipe+Soares%2C+Yi+Dong%2C+and+Oleksii+Kuchaiev%2E+2025b%2E+Rlbff%3A+Binary+flexible+feedback+to+bridge+between+human+feed)
47. Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Hoo-Chang Shin, Felipe Soares, Alexander Bukharin, Ellie Evans, Yi Dong, and Oleksii Kuchaiev. 2025c. Helpsteer3-preference: Open human-annotated preference data across diverse tasks and languages. arXiv preprint arXiv:2505.11475. — [arXiv:2505.11475](https://arxiv.org/abs/2505.11475) [find](https://scholar.google.com/scholar?q=Zhilin+Wang%2C+Jiaqi+Zeng%2C+Olivier+Delalleau%2C+Hoo%2DChang+Shin%2C+Felipe+Soares%2C+Alexander+Bukharin%2C+Ellie+Evans%2C+Yi+Dong%2C+and+Oleksii+Kuchaiev%2E+2025c%2E+Helpsteer3%2Dpreference%3A+Open+human%2Dannotated+preference)
48. Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Jie Huang, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, and Quoc V. Le. 2024. [Long-form factuality in large language models](https://md.research.elamur.ai/free/en/papers/wei-2024-long-form-factuality-large-language/). In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Proce…

*Updated 31 October 2026*