---
title: "ReflectRM — paper by Qin et al. (2026) — summary and ideas"
description: "They build Reflection Data by sampling several outputs on standard preference examples and pairing analyses that reached… 48 references."
authors: ["Kai Qin", "Liangxin Liu", "Yu Liang", "Longzheng Wang", "Wangyan", "Zhang Yueyang", "Long Xia", "Zhiyuan Sun", "Houde Liu", "Daiting Shi"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1676/"
doi: "10.18653/v1/2026.acl-long.1676"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qin-2026-reflectrm-boosting-generative-reward-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qin-2026-reflectrm-boosting-generative-reward-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/41/) / ReflectRM: Boosting Generative Reward Models via Self-Reflection within a Unified Judgment Framework

# ReflectRM: Boosting Generative Reward Models via Self-Reflection within a Unified Judgment Framework

**Authors:** Kai Qin, Liangxin Liu, Yu Liang, Longzheng Wang, Wangyan, Zhang Yueyang, [Long Xia](https://md.research.elamur.ai/free/en/authors/long-xia/), Zhiyuan Sun, Houde Liu and [Daiting Shi](https://md.research.elamur.ai/free/en/authors/daiting-shi/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1676/) · [PDF](https://aclanthology.org/2026.acl-long.1676.pdf) · [DOI 10.18653/v1/2026.acl-long.1676](https://doi.org/10.18653/v1/2026.acl-long.1676) · [OpenAlex](https://openalex.org/W7166798745)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Generative reward models (GRMs) produce a written analysis along with a preference verdict. Most are trained only on whether the final verdict is correct, so the quality of the reasoning goes unsupervised. ReflectRM addresses this by treating both kinds of judgment as one conditional generation task. In the first, the model compares two responses to a query. In the second, it compares two of its own analyses ("critiques") of that response pair, which the authors call self-reflection. They build Reflection Data by sampling several outputs on standard preference examples and pairing analyses that reached correct verdicts with ones that reached wrong verdicts. This data is mixed with ordinary preference data for training.

At inference, the model:
1. samples multiple candidate outputs;
2. picks a high-confidence analysis as an anchor;
3. uses self-reflection to compare the remaining candidates against the anchor and keeps the most reliable ones for the final verdict.

The method improves accuracy on four benchmarks and three model sizes and noticeably reduces positional bias. It needs no auxiliary classifier or multi-stage pipeline.

## Key points
- On Qwen3-4B, average accuracy rises by +3.7 across RewardBench, RM-Bench, RMB and PPE-Preference. Gains hold at 4B, 8B and 14B scales.
- Positional consistency was tested on JudgeBench plus samples from the other benchmarks. It improves by 10.2 points over the base model, which is roughly twice the 5.4-point improvement achieved by the RFT baseline.
- The two capabilities reinforce each other:
  - With greedy decoding, mixing 13.7k Pref examples with Reflection Data beats an equal-sized 17.1k pure-Pref training set.
  - ReflectRM's self-reflection outperforms a separate Reflection-Only model.
- The main cost is inference: about twice the compute of majority voting, because it adds N − 1 pairwise comparisons. Code is published at github.com/yuliangCarmelo/ReflectRM.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 333 works · 87,019 citations
- [Baidu](https://md.research.elamur.ai/free/en/companies/baidu/) — Company, Beijing, China · 74 works · 15,263 citations
- [Tsinghua Shenzhen International Graduate School](https://md.research.elamur.ai/free/en/universities/tsinghua-shenzhen-international-graduate-school/) — University, Shenzhen, China · 28 works · 3,035 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Kai Qin, Liangxin Liu, Yu Liang, Longzheng Wang, Wangyan, Zhang Yueyang, Long Xia, Zhiyuan Sun, Houde Liu and Daiting Shi. 2026. ReflectRM: Boosting Generative Reward Models via Self-Reflection within a Unified Judgment Framework. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{qin2026reflectrm,
  title = {{ReflectRM: Boosting Generative Reward Models via Self-Reflection within a Unified Judgment Framework}},
  author = {Kai Qin, Liangxin Liu, Yu Liang, Longzheng Wang, Wangyan, Zhang Yueyang, Long Xia, Zhiyuan Sun, Houde Liu and Daiting Shi},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1676},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1676}
}
```

- [← previous Reflection in the Dark: Exposing and Escaping the Black Box in Reflective Prompt Optimization](https://md.research.elamur.ai/free/en/papers/liu-2026-reflection-dark-exposing-escaping-black/)
- [next → Reflexive Research with LLMs: Considering the Positionality of Users and Systems](https://md.research.elamur.ai/free/en/papers/smith-2026-reflexive-research-llms-considering-positionality/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/41/)

## References

48 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774. \#\# Limitations Paul F Christiano, Jan Leike, Tom Brown, Miljan Ma…
2. A primary limitation of ReflectRM is the increased computational overhead associated with its two-Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang, Bryan Hooi, and Bingsheng He. 2025a. Judgelrm: Large reasoning models as a judge. arXiv preprint arXiv:2504.00050. — [arXiv:2504.00050](https://arxiv.org/abs/2504.00050) [find](https://scholar.google.com/scholar?q=A+primary+limitation+of+ReflectRM+is+the+increased+computational+overhead+associated+with+its+two%2DNuo+Chen%2C+Zhiyuan+Hu%2C+Qingyun+Zou%2C+Jiaying+Wu%2C+Qian+Wang%2C+Bryan+Hooi%2C+and+Bingsheng+He%2E+2025a%2E+Judgelr)
3. Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, and 1 others. 2025b. [Rm-r1: Reward modeling as reasoning](https://md.research.elamur.ai/free/en/papers/chen-2025-rm-r1-reward-modeling-reasoning/). arXiv preprint arXiv:2505.02387.
4. Jacob Dineen, Aswin RRV, Qin Liu, Zhikun Xu, Xiao Ye, Ming Shen, Zhaonan Li, Shijie Lu, Chitta Baral, Muhao Chen, and Ben Zhou. 2025. Qa-lign: Aligning llms through constitutionally decomposed qa. arXiv preprint arXiv:2506.08123. [find](https://scholar.google.com/scholar?q=Jacob+Dineen%2C+Aswin+RRV%2C+Qin+Liu%2C+Zhikun+Xu%2C+Xiao+Ye%2C+Ming+Shen%2C+Zhaonan+Li%2C+Shijie+Lu%2C+Chitta+Baral%2C+Muhao+Chen%2C+and+Ben+Zhou%2E+2025%2E+Qa%2Dlign%3A+Aligning+llms+through+constitutionally+decomposed+qa%2E+arX)
5. Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, and Tong Zhang. 2024. Rlhf workflow: From reward modeling to online rlhf. arXiv preprint arXiv:2405.07863. [find](https://scholar.google.com/scholar?q=Hanze+Dong%2C+Wei+Xiong%2C+Bo+Pang%2C+Haoxiang+Wang%2C+Han+Zhao%2C+Yingbo+Zhou%2C+Nan+Jiang%2C+Doyen+Sahoo%2C+Caiming+Xiong%2C+and+Tong+Zhang%2E+2024%2E+Rlhf+workflow%3A+From+reward+modeling+to+online+rlhf%2E+arXiv+preprint+ar)
6. Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, and 1 others. 2025. Competitive programming with large reasoning models. arXiv preprint arXiv:2502.06807. [find](https://scholar.google.com/scholar?q=Ahmed+El%2DKishky%2C+Alexander+Wei%2C+Andre+Saraiva%2C+Borys+Minaiev%2C+Daniel+Selsam%2C+David+Dohan%2C+Francis+Song%2C+Hunter+Lightman%2C+Ignasi+Clavera%2C+Jakub+Pachocki%2C+and+1+others%2E+2025%2E+Competitive+programming+wit)
7. Evan Frick, Tianle Li, Connor Chen, Wei-Lin Chiang, Anastasios N Angelopoulos, Jiantao Jiao, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. 2024. How to evaluate reward models for rlhf. arXiv preprint arXiv:2410.14872. [find](https://scholar.google.com/scholar?q=Evan+Frick%2C+Tianle+Li%2C+Connor+Chen%2C+Wei%2DLin+Chiang%2C+Anastasios+N+Angelopoulos%2C+Jiantao+Jiao%2C+Banghua+Zhu%2C+Joseph+E+Gonzalez%2C+and+Ion+Stoica%2E+2024%2E+How+to+evaluate+reward+models+for+rlhf%2E+arXiv+preprin)
8. Yejin Choi, and 1 others. 2025. Rewardbench: Evaluating reward models for language modeling. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 1755–1797. [find](https://scholar.google.com/scholar?q=Yejin+Choi%2C+and+1+others%2E+2025%2E+Rewardbench%3A+Evaluating+reward+models+for+language+modeling%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+NAACL+2025%2C+pages+1755%E2%80%931797%2E)
9. Derek Li, Jiaming Zhou, Amirreza Kazemi, Qianyi Sun, Abbas Ghaddar, Liheng Ma, Yu Luo, Dong Li, Jianye HAO, and Yingxue Zhang. 2025. Omni-thinker: Scaling cross-domain generalization in llms via multi-task rl with hybrid rewards. In 2nd AI for Math Workshop@ ICML 2025. [find](https://scholar.google.com/scholar?q=Derek+Li%2C+Jiaming+Zhou%2C+Amirreza+Kazemi%2C+Qianyi+Sun%2C+Abbas+Ghaddar%2C+Liheng+Ma%2C+Yu+Luo%2C+Dong+Li%2C+Jianye+HAO%2C+and+Yingxue+Zhang%2E+2025%2E+Omni%2Dthinker%3A+Scaling+cross%2Ddomain+generalization+in+llms+via+multi)
10. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024a. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. [find](https://scholar.google.com/scholar?q=Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+and+1+others%2E+2024a%2E+Deepseek%2Dv3+technical+report%2E+arXiv+preprint+arXiv%3A241)
11. Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou. 2024b. [Skywork-reward: Bag of tricks for reward modeling in llms](https://md.research.elamur.ai/free/en/papers/liu-2024-skywork-reward-bag-tricks/). arXiv preprint arXiv:2410.18451.
12. Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. 2025. Deep think with confidence. arXiv preprint arXiv:2508.15260. [find](https://scholar.google.com/scholar?q=Yichao+Fu%2C+Xuewei+Wang%2C+Yuandong+Tian%2C+and+Jiawei+Zhao%2E+2025%2E+Deep+think+with+confidence%2E+arXiv+preprint+arXiv%3A2508%2E15260%2E)
13. Chris Yuhao Liu, Liang Zeng, Yuzhen Xiao, Jujie He, Jiacai Liu, Chaojie Wang, Rui Yan, Wei Shen, Fuxiang Zhang, Jiacheng Xu, and 1 others. 2025a. [Skywork-reward-v2: Scaling preference data curation via human-ai synergy](https://md.research.elamur.ai/free/en/papers/liu-2025-skywork-reward-v2-scaling/). arXiv preprint arXiv:2507.01352.
14. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
15. Fengyuan Liu, Nouar AlDahoul, Gregory Eady, Yasir Zaki, and Talal Rahwan. 2024c. Self-reflection makes large language models safer, less biased, and ideologically neutral. arXiv preprint arXiv:2406.10400. — [arXiv:2406.10400](https://arxiv.org/abs/2406.10400) [find](https://scholar.google.com/scholar?q=Fengyuan+Liu%2C+Nouar+AlDahoul%2C+Gregory+Eady%2C+Yasir+Zaki%2C+and+Talal+Rahwan%2E+2024c%2E+Self%2Dreflection+makes+large+language+models+safer%2C+less+biased%2C+and+ideologically+neutral%2E+arXiv+preprint+arXiv%3A2406%2E10)
16. Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, and Sean Hendryx. 2025. Rubrics as rewards: Reinforcement learning beyond verifiable domains. arXiv preprint arXiv:2507.17746. [find](https://scholar.google.com/scholar?q=Anisha+Gunjal%2C+Anthony+Wang%2C+Elaine+Lau%2C+Vaskar+Nath%2C+Yunzhong+He%2C+Bing+Liu%2C+and+Sean+Hendryx%2E+2025%2E+Rubrics+as+rewards%3A+Reinforcement+learning+beyond+verifiable+domains%2E+arXiv+preprint+arXiv%3A2507%2E177)
17. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025a. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
18. Jiaxin Guo, Zewen Chi, Li Dong, Qingxiu Dong, Xun Wu, Shaohan Huang, and Furu Wei. 2025b. Reward reasoning models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Jiaxin+Guo%2C+Zewen+Chi%2C+Li+Dong%2C+Qingxiu+Dong%2C+Xun+Wu%2C+Shaohan+Huang%2C+and+Furu+Wei%2E+2025b%2E+Reward+reasoning+models%2E+In+The+Thirty%2Dninth+Annual+Conference+on+Neural+Information+Processing+Systems%2E)
19. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). arXiv preprint arXiv:2412.16720.
20. Nathan Lambert. 2025. Reinforcement learning from human feedback. arXiv preprint arXiv:2504.12501. — [arXiv:2504.12501](https://arxiv.org/abs/2504.12501) [find](https://scholar.google.com/scholar?q=Nathan+Lambert%2E+2025%2E+Reinforcement+learning+from+human+feedback%2E+arXiv+preprint+arXiv%3A2504%2E12501%2E+%E2%80%94+%5BarXiv%3A2504%2E12501%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2504%2E12501%29)
21. Nathan Lambert, Valentina Pyatkin, Jacob Morrison, Lester James Validad Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Liangxin Liu, Xuebo Liu, Derek F Wong, Dongfang Li, Ziyi Wang, Baotian Hu, and Min Zhang. 2024d. Selectit: Selective instruction tuning for llms via… [find](https://scholar.google.com/scholar?q=Nathan+Lambert%2C+Valentina+Pyatkin%2C+Jacob+Morrison%2C+Lester+James+Validad+Miranda%2C+Bill+Yuchen+Lin%2C+Khyathi+Chandu%2C+Nouha+Dziri%2C+Sachin+Kumar%2C+Tom+Zick%2C+Liangxin+Liu%2C+Xuebo+Liu%2C+Derek+F+Wong%2C+Dongfang+L)
22. Shudong Liu, Hongwei Liu, Junnan Liu, Linchen Xiao, Songyang Gao, Chengqi Lyu, Yuzhe Gu, Wenwei Zhang, Derek F Wong, Songyang Zhang, and 1 others. 2025b. [Compassverifier: A unified and robust verifier for llms evaluation and outcome reward](https://md.research.elamur.ai/free/en/papers/liu-2025-compassverifier-unified-robust-verifier-llms/). In Proceedings of the 2025 Conference on Empirical Methods…
23. Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. 2024e. [Rm-bench: Benchmarking reward models of language models with subtlety and style](https://md.research.elamur.ai/free/en/papers/liu-2024-rm-bench-benchmarking-reward-models/). arXiv preprint arXiv:2410.16184.
24. Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. 2025c. [Inference-time scaling for generalist reward modeling](https://md.research.elamur.ai/free/en/papers/liu-2025-inference-time-scaling-generalist/). arXiv preprint arXiv:2504.02495.
25. Xueguang Ma, Qian Liu, Dongfu Jiang, Ge Zhang, Zejun Ma, and Wenhu Chen. 2025. [General-reasoner: Advancing llm reasoning across all domains](https://md.research.elamur.ai/free/en/papers/ma-2025-general-reasoner-advancing-llm-reasoning/). arXiv preprint arXiv:2505.14652.
26. Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. 2024. Generative reward models. arXiv preprint arXiv:2410.12832. [find](https://scholar.google.com/scholar?q=Dakota+Mahan%2C+Duy+Van+Phung%2C+Rafael+Rafailov%2C+Chase+Blagden%2C+Nathan+Lile%2C+Louis+Castricato%2C+Jan%2DPhilipp+Fr%C3%A4nken%2C+Chelsea+Finn%2C+and+Alon+Albalak%2E+2024%2E+Generative+reward+models%2E+arXiv+preprint+arXiv%3A24)
27. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
28. Chenglong Wang, Yongyu Mu, Hang Zhou, Yifu Huo, Ziming Zhu, Jiali Zeng, Murun Yang, Bei Li, Xiaoyang Hao, Chunliang Zhang, and 1 others. 2026. Gram-r 2 : Self-training generative foundation reward models for reward reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 4… [find](https://scholar.google.com/scholar?q=Chenglong+Wang%2C+Yongyu+Mu%2C+Hang+Zhou%2C+Yifu+Huo%2C+Ziming+Zhu%2C+Jiali+Zeng%2C+Murun+Yang%2C+Bei+Li%2C+Xiaoyang+Hao%2C+Chunliang+Zhang%2C+and+1+others%2E+2026%2E+Gram%2Dr+2+%3A+Self%2Dtraining+generative+foundation+reward+mod)
29. Haoxiang Wang, Wei Xiong, Tengyang Xie, Han Zhao, and Tong Zhang. 2024a. [Interpretable preferences via multi-objective reward modeling and mixture-of-experts](https://md.research.elamur.ai/free/en/papers/wang-2024-interpretable-preferences-multi-objective-reward/). arXiv preprint arXiv:2406.12845.
30. Junsoo Park, Seungyeon Jwa, Ren Meiying, Daeyoung Kim, and Sanghyuk Choi. 2024. [Offsetbias: Leveraging debiased data for tuning evaluators](https://md.research.elamur.ai/free/en/papers/park-2024-offsetbias-leveraging-debiased-data-tuning/). In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1043–1067.
31. Tianlu Wang, Ilia Kulikov, Olga Golovneva, Ping Yu, Weizhe Yuan, Jane Dwivedi-Yu, Richard Yuanzhe Pang, Maryam Fazel-Zarandi, Jason Weston, and Xian Li. 2024b. [Self-taught evaluators](https://md.research.elamur.ai/free/en/papers/wang-2024-self-taught-evaluators/). arXiv preprint arXiv:2408.02666.
32. Hoang Phan, Victor Li, and Qi Lei. 2025. Think twice, generate once: Safeguarding by progressive self-reflection. arXiv preprint arXiv:2510.01270. [find](https://scholar.google.com/scholar?q=Hoang+Phan%2C+Victor+Li%2C+and+Qi+Lei%2E+2025%2E+Think+twice%2C+generate+once%3A+Safeguarding+by+progressive+self%2Dreflection%2E+arXiv+preprint+arXiv%3A2510%2E01270%2E)
33. Yutong Wang, Jiali Zeng, Xuebo Liu, Fandong Meng, Jie Zhou, and Min Zhang. 2024c. Taste: Teaching large language models to translate through self-reflection. arXiv preprint arXiv:2406.08434. [find](https://scholar.google.com/scholar?q=Yutong+Wang%2C+Jiali+Zeng%2C+Xuebo+Liu%2C+Fandong+Meng%2C+Jie+Zhou%2C+and+Min+Zhang%2E+2024c%2E+Taste%3A+Teaching+large+language+models+to+translate+through+self%2Dreflection%2E+arXiv+preprint+arXiv%3A2406%2E08434%2E)
34. Matthew Renze and Erhan Guven. 2024. [Self-reflection in llm agents: Effects on problem-solving performance](https://md.research.elamur.ai/free/en/papers/renze-2024-self-reflection-llm-agents-effects/). arXiv preprint arXiv:2405.06682.
35. ByteDance Seed, Jiaze Chen, Tiantian Fan, Xin Liu, Lingjun Liu, Zhiqi Lin, Mingxuan Wang, Chengyi Wang, Xiangpeng Wei, Wenyuan Xu, and 1 others. 2025. Seed1. 5-thinking: Advancing superb reasoning models with reinforcement learning. arXiv preprint arXiv:2504.13914. — [Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/bytedance-seed-2025-seed1-5-thinking-advancing/)
36. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
37. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in neural information processing systems, 33:3008– 3021. [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeffrey+Wu%2C+Daniel+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+F+Christiano%2E+2020%2E+Learning+to+summarize+with+human+feedback%2E+Advances+in+neural)
38. Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Y Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, and Ion Stoica. 2024. Judgebench: A benchmark for evaluating llm-based judges. arXiv preprint arXiv:2410.12784. [find](https://scholar.google.com/scholar?q=Sijun+Tan%2C+Siyuan+Zhuang%2C+Kyle+Montgomery%2C+William+Y+Tang%2C+Alejandro+Cuadron%2C+Chenguang+Wang%2C+Raluca+Ada+Popa%2C+and+Ion+Stoica%2E+2024%2E+Judgebench%3A+A+benchmark+for+evaluating+llm%2Dbased+judges%2E+arXiv+prep)
39. Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, and 1 others. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599. — [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/)
40. Chenglong Wang, Yang Gan, Yifu Huo, Yongyu Mu, Qiaozhi He, Murun Yang, Bei Li, Tong Xiao, Chunliang Zhang, Tongran Liu, and 1 others. 2025a. Gram: A generative foundation reward model for reward generalization. arXiv preprint arXiv:2506.14175. — [arXiv:2506.14175](https://arxiv.org/abs/2506.14175) [find](https://scholar.google.com/scholar?q=Chenglong+Wang%2C+Yang+Gan%2C+Yifu+Huo%2C+Yongyu+Mu%2C+Qiaozhi+He%2C+Murun+Yang%2C+Bei+Li%2C+Tong+Xiao%2C+Chunliang+Zhang%2C+Tongran+Liu%2C+and+1+others%2E+2025a%2E+Gram%3A+A+generative+foundation+reward+model+for+reward+gener)
41. Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Hoo-Chang Shin, Felipe Soares, Alexander Bukharin, Ellie Evans, Yi Dong, and Oleksii Kuchaiev. 2025b. Helpsteer3-preference: Open human-annotated preference data across diverse tasks and languages. arXiv preprint arXiv:2505.11475. — [arXiv:2505.11475](https://arxiv.org/abs/2505.11475) [find](https://scholar.google.com/scholar?q=Zhilin+Wang%2C+Jiaqi+Zeng%2C+Olivier+Delalleau%2C+Hoo%2DChang+Shin%2C+Felipe+Soares%2C+Alexander+Bukharin%2C+Ellie+Evans%2C+Yi+Dong%2C+and+Oleksii+Kuchaiev%2E+2025b%2E+Helpsteer3%2Dpreference%3A+Open+human%2Dannotated+preference)
42. Chenxi Whitehouse, Tianlu Wang, Ping Yu, Xian Li, Jason Weston, Ilia Kulikov, and Swarnadeep Saha. 2025. J1: Incentivizing thinking in llm-asa-judge via reinforcement learning. arXiv preprint arXiv:2505.10320. — [arXiv:2505.10320](https://arxiv.org/abs/2505.10320) [find](https://scholar.google.com/scholar?q=Chenxi+Whitehouse%2C+Tianlu+Wang%2C+Ping+Yu%2C+Xian+Li%2C+Jason+Weston%2C+Ilia+Kulikov%2C+and+Swarnadeep+Saha%2E+2025%2E+J1%3A+Incentivizing+thinking+in+llm%2Dasa%2Djudge+via+reinforcement+learning%2E+arXiv+preprint+arXiv%3A25)
43. Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason E Weston, and Sainbayar Sukhbaatar. 2025. [Meta-rewarding language models: Self-improving alignment with llm-as-a-meta-judge](https://md.research.elamur.ai/free/en/papers/wu-2025-meta-rewarding-language-models-self/). In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,…
44. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
45. Ziyi Ye, Xiangsheng Li, Qiuchi Li, Qingyao Ai, Yujia Zhou, Wei Shen, Dong Yan, and Yiqun Liu. 2025. Learning llm-as-a-judge for preference alignment. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ziyi+Ye%2C+Xiangsheng+Li%2C+Qiuchi+Li%2C+Qingyao+Ai%2C+Yujia+Zhou%2C+Wei+Shen%2C+Dong+Yan%2C+and+Yiqun+Liu%2E+2025%2E+Learning+llm%2Das%2Da%2Djudge+for+preference+alignment%2E+In+The+Thirteenth+International+Conference+on+Lear)
46. Jiachen Yu, Shaoning Sun, Xiaohui Hu, Jiaxu Yan, Kaidong Yu, and Xuelong Li. 2025. Improve llm-asa-judge ability as a general ability. arXiv preprint arXiv:2502.11689. — [arXiv:2502.11689](https://arxiv.org/abs/2502.11689) [find](https://scholar.google.com/scholar?q=Jiachen+Yu%2C+Shaoning+Sun%2C+Xiaohui+Hu%2C+Jiaxu+Yan%2C+Kaidong+Yu%2C+and+Xuelong+Li%2E+2025%2E+Improve+llm%2Dasa%2Djudge+ability+as+a+general+ability%2E+arXiv+preprint+arXiv%3A2502%2E11689%2E+%E2%80%94+%5BarXiv%3A2502%2E11689%5D%28https%3A%2F%2Farx)
47. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. 2024. [Generative verifiers: Reward modeling as next-token prediction](https://md.research.elamur.ai/free/en/papers/zhang-2024-generative-verifiers-reward-modeling-as/). arXiv preprint arXiv:2408.15240.
48. Enyu Zhou, Guodong Zheng, Binghai Wang, Zhiheng Xi, Shihan Dou, Rong Bao, Wei Shen, Limao Xiong, Jessica Fan, Yurong Mou, and 1 others. 2024. [Rmb: Comprehensively benchmarking reward models in llm alignment](https://md.research.elamur.ai/free/en/papers/zhou-2024-rmb-comprehensively-benchmarking-reward-models/). arXiv preprint arXiv:2410.09893.

*Updated 1 November 2026*