---
title: "CoVerRL — paper by Pan et al. (2026) — summary and ideas"
description: "The authors show that this setup drifts into what they call the consensus trap. 51 references."
authors: ["Teng Pan", "Yuchen Yan", "Zixuan Wang", "Ruiqing Zhang", "Guiyang Hou", "Wenqi Zhang", "Weiming Lu", "Jun Xiao", "Yongliang Shen"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1376/"
doi: "10.18653/v1/2026.acl-long.1376"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pan-2026-coverrl-breaking-consensus-trap-label/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pan-2026-coverrl-breaking-consensus-trap-label/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/11/) / CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution

# CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution

**Authors:** Teng Pan, [Yuchen Yan](https://md.research.elamur.ai/free/en/authors/yuchen-yan/), [Zixuan Wang](https://md.research.elamur.ai/free/en/authors/zixuan-wang/), Ruiqing Zhang, [Guiyang Hou](https://md.research.elamur.ai/free/en/authors/guiyang-hou/), [Wenqi Zhang](https://md.research.elamur.ai/free/en/authors/wenqi-zhang/), [Weiming Lu](https://md.research.elamur.ai/free/en/authors/weiming-lu/), [Jun Xiao](https://md.research.elamur.ai/free/en/authors/jun-xiao/) and [Yongliang Shen](https://md.research.elamur.ai/free/en/authors/yongliang-shen/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1376/) · [PDF](https://aclanthology.org/2026.acl-long.1376.pdf) · [DOI 10.18653/v1/2026.acl-long.1376](https://doi.org/10.18653/v1/2026.acl-long.1376) · [OpenAlex](https://openalex.org/W7166830822)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Label-free reinforcement learning for LLM reasoning usually rewards a model for matching its own majority-voted answer. The authors show that this setup drifts into what they call the consensus trap. Optimizing for self-agreement shrinks output diversity, so the model becomes confident in recurring wrong answers, and reward accuracy erodes. Under TTRL it falls from about 95% to roughly 80%.

CoVerRL counters this by having one policy play two roles inside a multi-turn RL loop:
- **As generator**, the model samples solutions and drafts a pseudo-label by majority vote.
- **As verifier**, it inspects solutions that reach the majority answer and drops queries whose consensus it rejects.
- **In return**, the vote supplies balanced positive and negative examples for training the verifier.
- **Self-correction**: a negative judgment triggers a revision step. Everything is optimized with an Answer-Anchored GRPO variant.

Across Qwen and Llama models on mathematical reasoning benchmarks, CoVerRL beats label-free baselines by 4.7–5.9% on average. It also turns a near-chance self-verifier into a reliable one. This suggests that label-free self-improvement can escape stagnation without external supervision.

## Key points
- **Training dynamics on Qwen3-1.7B-Base:**
  - Under TTRL, label accuracy stalls near 70% while reward accuracy decays.
  - CoVerRL keeps reward accuracy above 90% and lifts label accuracy from 75% to almost 88%.
- **Gains over TTRL:** average improvements of 5.7%, 5.9% and 4.7% across three models (Qwen3-1.7B-Base, Qwen2.5-7B, Llama-3.2-3B-Instruct), evaluated on four reasoning benchmarks including GPQA.
- **Self-verification accuracy:** rises from 55.8% to 81.2% on Qwen3-1.7B and from 54.0% to 86.5% on Qwen2.5-7B.
- **Need for balanced sampling:**
  - A TTRL + PAG baseline (PAG's verify-then-revise workflow on pseudo-labels, trained with PPO) collapses on Llama-3.2-3B-Instruct. Its verifier rejects nearly everything, and Correct Recall drops close to zero.
  - CoVerRL avoids this by enforcing |V+| = |V−| and using GRPO.
- **Ablations on Qwen3-1.7B-Base:**
  - Removing verification filtering, verifier updates, self-correction, or AA-GRPO each lowers results.
  - Without self-correction training, GPQA Acc.@final (25.4%) falls below Acc.@first (31.0%).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Baidu](https://md.research.elamur.ai/free/en/companies/baidu/) — Company, Beijing, China · 74 works · 15,263 citations
- [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/) — University, Hangzhou, China · 172 works · 12,988 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Teng Pan, Yuchen Yan, Zixuan Wang, Ruiqing Zhang, Guiyang Hou, Wenqi Zhang, Weiming Lu, Jun Xiao and Yongliang Shen. 2026. CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pan2026coverrl,
  title = {{CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution}},
  author = {Teng Pan, Yuchen Yan, Zixuan Wang, Ruiqing Zhang, Guiyang Hou, Wenqi Zhang, Weiming Lu, Jun Xiao and Yongliang Shen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1376},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1376}
}
```

- [← previous Counterfactual Fairness Evaluation of LLM-Based Contact Center Agent Quality Assurance System](https://md.research.elamur.ai/free/en/papers/mayilvaghanan-2026-counterfactual-fairness-evaluation-llm-based/)
- [next → CPR-RAG: Clinical Prior-Regularized Retrieval for Anatomy-Aware 3D CT Report Generation](https://md.research.elamur.ai/free/en/papers/yang-2026-cpr-rag-clinical-prior-regularized/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/11/)

## References

51 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Anonymous. 2025. Label-free GUI grounding via confidence-guided negative reinforcement learning. [find](https://scholar.google.com/scholar?q=Anonymous%2E+2025%2E+Label%2Dfree+GUI+grounding+via+confidence%2Dguided+negative+reinforcement+learning%2E)
2. In Submitted to The Fourteenth International Conference on Learning Representations. Under review. [find](https://scholar.google.com/scholar?q=In+Submitted+to+The+Fourteenth+International+Conference+on+Learning+Representations%2E+Under+review%2E)
3. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2023. Teaching large language models to self-debug. Preprint, arXiv:2304.05128. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Maxwell+Lin%2C+Nathanael+Sch%C3%A4rli%2C+and+Denny+Zhou%2E+2023%2E+Teaching+large+language+models+to+self%2Ddebug%2E+Preprint%2C+arXiv%3A2304%2E05128%2E)
4. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, and 1 others. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). Preprint, arXiv:2110.14168.
5. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). Preprint, arXiv:2501.12948.
6. Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, and 1 others. 2025a. Retool: Reinforcement learning for strategic tool use in llms. Preprint, arXiv:2504.11536. [find](https://scholar.google.com/scholar?q=Jiazhan+Feng%2C+Shijue+Huang%2C+Xingwei+Qu%2C+Ge+Zhang%2C+Yujia+Qin%2C+and+1+others%2E+2025a%2E+Retool%3A+Reinforcement+learning+for+strategic+tool+use+in+llms%2E+Preprint%2C+arXiv%3A2504%2E11536%2E)
7. Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. 2025b. Group-in-group policy optimization for llm agent training. Preprint, arXiv:2505.10978. [find](https://scholar.google.com/scholar?q=Lang+Feng%2C+Zhenghai+Xue%2C+Tingcong+Liu%2C+and+Bo+An%2E+2025b%2E+Group%2Din%2Dgroup+policy+optimization+for+llm+agent+training%2E+Preprint%2C+arXiv%3A2505%2E10978%2E)
8. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). Preprint, arXiv:2407.21783.
9. Jiacheng Guo, Ling Yang, Peter Chen, Qixin Xiao, Yinjie Wang, and 1 others. 2025. Genenv: Difficulty-aligned co-evolution between llm agents and environment simulators. Preprint, arXiv:2512.19682. — [arXiv:2512.19682](https://arxiv.org/abs/2512.19682) [find](https://scholar.google.com/scholar?q=Jiacheng+Guo%2C+Ling+Yang%2C+Peter+Chen%2C+Qixin+Xiao%2C+Yinjie+Wang%2C+and+1+others%2E+2025%2E+Genenv%3A+Difficulty%2Daligned+co%2Devolution+between+llm+agents+and+environment+simulators%2E+Preprint%2C+arXiv%3A2512%2E19682%2E+%E2%80%94+%5B)
10. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, and 1 others. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). Preprint, arXiv:2103.03874.
11. Chengsong Huang, Wenhao Yu, Xiaoyang Wang, Hongming Zhang, Zongxia Li, and 1 others. 2025. R-zero: Self-evolving reasoning llm from zero data. Preprint, arXiv:2508.05004. — [arXiv:2508.05004](https://arxiv.org/abs/2508.05004) [find](https://scholar.google.com/scholar?q=Chengsong+Huang%2C+Wenhao+Yu%2C+Xiaoyang+Wang%2C+Hongming+Zhang%2C+Zongxia+Li%2C+and+1+others%2E+2025%2E+R%2Dzero%3A+Self%2Devolving+reasoning+llm+from+zero+data%2E+Preprint%2C+arXiv%3A2508%2E05004%2E+%E2%80%94+%5BarXiv%3A2508%2E05004%5D%28https%3A%2F%2F)
12. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. [Large language models cannot self-correct reasoning yet](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-cannot-self/). Preprint, arXiv:2310.01798.
13. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, and 1 others. 2024. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). Preprint, arXiv:2403.07974.
14. Yuhua Jiang, Yuwen Xiong, Yufeng Yuan, Chao Xin, Wenyuan Xu, and 1 others. 2025. [Pag: Multi-turn reinforced llm self-correction with policy as generative verifier](https://md.research.elamur.ai/free/en/papers/jiang-2025-pag-multi-turn-reinforced-llm/). Preprint, arXiv:2506.10406.
15. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, and 1 others. 2025. [Search-r1: Training llms to reason and leverage search engines with reinforcement learning](https://md.research.elamur.ai/free/en/papers/jin-2025-search-r1-training-llms-reason/). Preprint, arXiv:2503.09516.
16. Nathan Lambert, Valentina Pyatkin, Jacob Morrison, L. J. Miranda, Bill Yuchen Lin, and 1 others. 2024. Rewardbench: Evaluating reward models for language modeling. Preprint, arXiv:2403.13787. [find](https://scholar.google.com/scholar?q=Nathan+Lambert%2C+Valentina+Pyatkin%2C+Jacob+Morrison%2C+L%2E+J%2E+Miranda%2C+Bill+Yuchen+Lin%2C+and+1+others%2E+2024%2E+Rewardbench%3A+Evaluating+reward+models+for+language+modeling%2E+Preprint%2C+arXiv%3A2403%2E13787%2E)
17. Hyunseok Lee, Seunghyuk Oh, Jaehyung Kim, Jinwoo Shin, and Jihoon Tack. 2025. Revise: Learning to refine at test-time via intrinsic self-verification. Preprint, arXiv:2502.14565. — [arXiv:2502.14565](https://arxiv.org/abs/2502.14565) [find](https://scholar.google.com/scholar?q=Hyunseok+Lee%2C+Seunghyuk+Oh%2C+Jaehyung+Kim%2C+Jinwoo+Shin%2C+and+Jihoon+Tack%2E+2025%2E+Revise%3A+Learning+to+refine+at+test%2Dtime+via+intrinsic+self%2Dverification%2E+Preprint%2C+arXiv%3A2502%2E14565%2E+%E2%80%94+%5BarXiv%3A2502%2E14565%5D%28)
18. Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q. Jiang, Ziju Shen, and 1 others. 2024. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository. 13:9. [find](https://scholar.google.com/scholar?q=Jia+Li%2C+Edward+Beeching%2C+Lewis+Tunstall%2C+Ben+Lipkin%2C+Roman+Soletskyi%2C+Shengyi+Huang%2C+Kashif+Rasul%2C+Longhui+Yu%2C+Albert+Q%2E+Jiang%2C+Ziju+Shen%2C+and+1+others%2E+2024%2E+Numinamath%3A+The+largest+public+dataset+in)
19. Pengyi Li, Matvey Skripkin, Alexander Zubrey, Andrey Kuznetsov, and Ivan Oseledets. 2025. Confidence is all you need: Few-shot rl fine-tuning of language models. Preprint, arXiv:2506.06395. — [arXiv:2506.06395](https://arxiv.org/abs/2506.06395) [find](https://scholar.google.com/scholar?q=Pengyi+Li%2C+Matvey+Skripkin%2C+Alexander+Zubrey%2C+Andrey+Kuznetsov%2C+and+Ivan+Oseledets%2E+2025%2E+Confidence+is+all+you+need%3A+Few%2Dshot+rl+fine%2Dtuning+of+language+models%2E+Preprint%2C+arXiv%3A2506%2E06395%2E+%E2%80%94+%5BarXiv%3A2)
20. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, and 1 others. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). Preprint, arXiv:2305.20050.
21. Jia Liu, ChangYi He, YingQiao Lin, MingMin Yang, FeiYang Shen, and ShaoGuo Liu. 2025a. Ettrl: Balancing exploration and exploitation in llm test-time reinforcement learning via entropy mechanism. Preprint, arXiv:2508.11356. — [arXiv:2508.11356](https://arxiv.org/abs/2508.11356) [find](https://scholar.google.com/scholar?q=Jia+Liu%2C+ChangYi+He%2C+YingQiao+Lin%2C+MingMin+Yang%2C+FeiYang+Shen%2C+and+ShaoGuo+Liu%2E+2025a%2E+Ettrl%3A+Balancing+exploration+and+exploitation+in+llm+test%2Dtime+reinforcement+learning+via+entropy+mechanism%2E+Prep)
22. Licheng Liu, Zihan Wang, Linjie Li, Chenwei Xu, Yiping Lu, and 1 others. 2025b. A simple "try again" can elicit multi-turn llm reasoning. Preprint, arXiv:2507.14295. [find](https://scholar.google.com/scholar?q=Licheng+Liu%2C+Zihan+Wang%2C+Linjie+Li%2C+Chenwei+Xu%2C+Yiping+Lu%2C+and+1+others%2E+2025b%2E+A+simple+%22try+again%22+can+elicit+multi%2Dturn+llm+reasoning%2E+Preprint%2C+arXiv%3A2507%2E14295%2E)
23. Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, and 1 others. 2025c. [Agentbench: Evaluating llms as agents](https://md.research.elamur.ai/free/en/papers/liu-2023-agentbench-evaluating-llms-agents/). Preprint, arXiv:2308.03688.
24. Xiaoyuan Liu, Tian Liang, Zhiwei He, Jiahao Xu, Wenxuan Wang, and 1 others. 2025d. Trust, but verify: A self-verification approach to reinforcement learning with verifiable rewards. Preprint, arXiv:2505.13445. — [arXiv:2505.13445](https://arxiv.org/abs/2505.13445) [find](https://scholar.google.com/scholar?q=Xiaoyuan+Liu%2C+Tian+Liang%2C+Zhiwei+He%2C+Jiahao+Xu%2C+Wenxuan+Wang%2C+and+1+others%2E+2025d%2E+Trust%2C+but+verify%3A+A+self%2Dverification+approach+to+reinforcement+learning+with+verifiable+rewards%2E+Preprint%2C+arXiv%3A25)
25. Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, and 1 others. 2025. [S$^2$r: Teaching llms to self-verify and self-correct via reinforcement learning](https://md.research.elamur.ai/free/en/papers/ma-2025-s-2-r-teaching-llms/). Preprint, arXiv:2502.12853.
26. Ning Miao, Yee Whye Teh, and Tom Rainforth. 2023. Selfcheck: Using llms to zero-shot check their own step-by-step reasoning. Preprint, arXiv:2308.00436. [find](https://scholar.google.com/scholar?q=Ning+Miao%2C+Yee+Whye+Teh%2C+and+Tom+Rainforth%2E+2023%2E+Selfcheck%3A+Using+llms+to+zero%2Dshot+check+their+own+step%2Dby%2Dstep+reasoning%2E+Preprint%2C+arXiv%3A2308%2E00436%2E)
27. Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, and 1 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=Qwen%2C+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+and+1+others%2E+2025%2E+Qwen2%2E5+technical+report%2E+Preprint%2C+arXiv%3A2412%2E15115%2E)
28. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, and 1 others. 2023. [Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). Preprint, arXiv:2311.12022.
29. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). Preprint, arXiv:2402.03300.
30. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, and 1 others. 2025. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297.
31. Hexiang Tan, Fei Sun, Sha Liu, Du Su, Qi Cao, and 1 others. 2025. [Too consistent to detect: A study of self-consistent errors in llms](https://md.research.elamur.ai/free/en/papers/tan-2025-too-consistent-detect-study-self/). Preprint, arXiv:2505.17656.
32. Gladys Tyen, Hassan Mansoor, Victor Cărbune, Peter Chen, and Tony Mak. 2024. [Llms cannot find reasoning errors, but can correct them given the error location](https://md.research.elamur.ai/free/en/papers/tyen-2024-llms-cannot-find-reasoning-errors/). Preprint, arXiv:2311.08516.
33. Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, and 1 others. 2025. Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning. Preprint, arXiv:2504.20073. [find](https://scholar.google.com/scholar?q=Zihan+Wang%2C+Kangrui+Wang%2C+Qineng+Wang%2C+Pingyue+Zhang%2C+Linjie+Li%2C+and+1+others%2E+2025%2E+Ragen%3A+Understanding+self%2Devolution+in+llm+agents+via+multi%2Dturn+reinforcement+learning%2E+Preprint%2C+arXiv%3A2504%2E20073)
34. Yuxiang Wei, Zhiqing Sun, Emily McMilin, Jonas Gehring, David Zhang, and 1 others. 2025. Toward training superintelligent software agents through self-play swe-rl. Preprint, arXiv:2512.18552. — [arXiv:2512.18552](https://arxiv.org/abs/2512.18552) [find](https://scholar.google.com/scholar?q=Yuxiang+Wei%2C+Zhiqing+Sun%2C+Emily+McMilin%2C+Jonas+Gehring%2C+David+Zhang%2C+and+1+others%2E+2025%2E+Toward+training+superintelligent+software+agents+through+self%2Dplay+swe%2Drl%2E+Preprint%2C+arXiv%3A2512%2E18552%2E+%E2%80%94+%5BarXiv)
35. Peng Xia, Kaide Zeng, Jiaqi Liu, Can Qin, Fang Wu, and 1 others. 2025. Agent0: Unleashing self-evolving agents from zero data via tool-integrated reasoning. Preprint, arXiv:2511.16043. — [arXiv:2511.16043](https://arxiv.org/abs/2511.16043) [find](https://scholar.google.com/scholar?q=Peng+Xia%2C+Kaide+Zeng%2C+Jiaqi+Liu%2C+Can+Qin%2C+Fang+Wu%2C+and+1+others%2E+2025%2E+Agent0%3A+Unleashing+self%2Devolving+agents+from+zero+data+via+tool%2Dintegrated+reasoning%2E+Preprint%2C+arXiv%3A2511%2E16043%2E+%E2%80%94+%5BarXiv%3A2511%2E1)
36. Wei Xiong, Hanning Zhang, Chenlu Ye, Lichang Chen, Nan Jiang, and Tong Zhang. 2025. Self-rewarding correction for mathematical reasoning. Preprint, arXiv:2502.19613. — [arXiv:2502.19613](https://arxiv.org/abs/2502.19613) [find](https://scholar.google.com/scholar?q=Wei+Xiong%2C+Hanning+Zhang%2C+Chenlu+Ye%2C+Lichang+Chen%2C+Nan+Jiang%2C+and+Tong+Zhang%2E+2025%2E+Self%2Drewarding+correction+for+mathematical+reasoning%2E+Preprint%2C+arXiv%3A2502%2E19613%2E+%E2%80%94+%5BarXiv%3A2502%2E19613%5D%28https%3A%2F%2Farxiv)
37. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, and 1 others. 2025a. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
38. OpenAI, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, and 1 others. 2024. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). Preprint, arXiv:2412.16720.
39. Ziyi Yang, Weizhou Shen, Ruijun Chen, Chenliang Li, Fanqi Wan, and 1 others. 2025b. Spell: Self-play reinforcement learning for evolving long-context language models. Preprint, arXiv:2509.23863. — [arXiv:2509.23863](https://arxiv.org/abs/2509.23863) [find](https://scholar.google.com/scholar?q=Ziyi+Yang%2C+Weizhou+Shen%2C+Ruijun+Chen%2C+Chenliang+Li%2C+Fanqi+Wan%2C+and+1+others%2E+2025b%2E+Spell%3A+Self%2Dplay+reinforcement+learning+for+evolving+long%2Dcontext+language+models%2E+Preprint%2C+arXiv%3A2509%2E23863%2E+%E2%80%94+%5Bar)
40. Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. 2025. Maximizing confidence alone improves reasoning. Preprint, arXiv:2505.22660. — [arXiv:2505.22660](https://arxiv.org/abs/2505.22660) [find](https://scholar.google.com/scholar?q=Mihir+Prabhudesai%2C+Lili+Chen%2C+Alex+Ippoliti%2C+Katerina+Fragkiadaki%2C+Hao+Liu%2C+and+Deepak+Pathak%2E+2025%2E+Maximizing+confidence+alone+improves+reasoning%2E+Preprint%2C+arXiv%3A2505%2E22660%2E+%E2%80%94+%5BarXiv%3A2505%2E22660%5D%28ht)
41. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, and 1 others. 2025a. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). Preprint, arXiv:2503.14476.
42. Archiki Prasad, Weizhe Yuan, Richard Yuanzhe Pang, Jing Xu, Maryam Fazel-Zarandi, and 1 others. 2025. Self-consistency preference optimization. Preprint, arXiv:2411.04109. [find](https://scholar.google.com/scholar?q=Archiki+Prasad%2C+Weizhe+Yuan%2C+Richard+Yuanzhe+Pang%2C+Jing+Xu%2C+Maryam+Fazel%2DZarandi%2C+and+1+others%2E+2025%2E+Self%2Dconsistency+preference+optimization%2E+Preprint%2C+arXiv%3A2411%2E04109%2E)
43. Zhaoning Yu, Will Su, Leitian Tao, Haozhu Wang, Aashu Singh, and 1 others. 2025b. Restrain: From spurious votes to signals – self-driven rl with self-penalization. Preprint, arXiv:2510.02172. — [arXiv:2510.02172](https://arxiv.org/abs/2510.02172) [find](https://scholar.google.com/scholar?q=Zhaoning+Yu%2C+Will+Su%2C+Leitian+Tao%2C+Haozhu+Wang%2C+Aashu+Singh%2C+and+1+others%2E+2025b%2E+Restrain%3A+From+spurious+votes+to+signals+%E2%80%93+self%2Ddriven+rl+with+self%2Dpenalization%2E+Preprint%2C+arXiv%3A2510%2E02172%2E+%E2%80%94+%5BarXiv)
44. Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. 2025. Self-rewarding language models. Preprint, arXiv:2401.10020. [find](https://scholar.google.com/scholar?q=Weizhe+Yuan%2C+Richard+Yuanzhe+Pang%2C+Kyunghyun+Cho%2C+Xian+Li%2C+Sainbayar+Sukhbaatar%2C+Jing+Xu%2C+and+Jason+Weston%2E+2025%2E+Self%2Drewarding+language+models%2E+Preprint%2C+arXiv%3A2401%2E10020%2E)
45. Kaiwen Zha, Zhengqi Gao, Maohao Shen, Zhang-Wei Hong, Duane S. Boning, and Dina Katabi. 2025. Rl tango: Reinforcing generator and verifier together for language reasoning. Preprint, arXiv:2505.15034. — [arXiv:2505.15034](https://arxiv.org/abs/2505.15034) [find](https://scholar.google.com/scholar?q=Kaiwen+Zha%2C+Zhengqi+Gao%2C+Maohao+Shen%2C+Zhang%2DWei+Hong%2C+Duane+S%2E+Boning%2C+and+Dina+Katabi%2E+2025%2E+Rl+tango%3A+Reinforcing+generator+and+verifier+together+for+language+reasoning%2E+Preprint%2C+arXiv%3A2505%2E15034%2E+)
46. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. 2025a. [Generative verifiers: Reward modeling as next-token prediction](https://md.research.elamur.ai/free/en/papers/zhang-2024-generative-verifiers-reward-modeling-as/). Preprint, arXiv:2408.15240.
47. Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, and 1 others. 2025b. [The lessons of developing process reward models in mathematical reasoning](https://md.research.elamur.ai/free/en/papers/zhang-2025-lessons-developing-process-reward-models/). Preprint, arXiv:2501.07301.
48. Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, and 1 others. 2025a. Absolute zero: Reinforced self-play reasoning with zero data. Preprint, arXiv:2505.03335. [find](https://scholar.google.com/scholar?q=Andrew+Zhao%2C+Yiran+Wu%2C+Yang+Yue%2C+Tong+Wu%2C+Quentin+Xu%2C+and+1+others%2E+2025a%2E+Absolute+zero%3A+Reinforced+self%2Dplay+reasoning+with+zero+data%2E+Preprint%2C+arXiv%3A2505%2E03335%2E)
49. Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. 2025b. Learning to reason without external rewards. Preprint, arXiv:2505.19590. [find](https://scholar.google.com/scholar?q=Xuandong+Zhao%2C+Zhewei+Kang%2C+Aosong+Feng%2C+Sergey+Levine%2C+and+Dawn+Song%2E+2025b%2E+Learning+to+reason+without+external+rewards%2E+Preprint%2C+arXiv%3A2505%2E19590%2E)
50. Yujun Zhou, Zhenwen Liang, Haolin Liu, Wenhao Yu, Kishan Panaganti, and 1 others. 2025. Evolving language models without labels: Majority drives selection, novelty promotes variation. Preprint, arXiv:2509.15194. — [arXiv:2509.15194](https://arxiv.org/abs/2509.15194) [find](https://scholar.google.com/scholar?q=Yujun+Zhou%2C+Zhenwen+Liang%2C+Haolin+Liu%2C+Wenhao+Yu%2C+Kishan+Panaganti%2C+and+1+others%2E+2025%2E+Evolving+language+models+without+labels%3A+Majority+drives+selection%2C+novelty+promotes+variation%2E+Preprint%2C+arXiv%3A)
51. Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, and 1 others. 2025. Ttrl: Test-time reinforcement learning. Preprint, arXiv:2504.16084. [find](https://scholar.google.com/scholar?q=Yuxin+Zuo%2C+Kaiyan+Zhang%2C+Li+Sheng%2C+Shang+Qu%2C+Ganqu+Cui%2C+and+1+others%2E+2025%2E+Ttrl%3A+Test%2Dtime+reinforcement+learning%2E+Preprint%2C+arXiv%3A2504%2E16084%2E)

*Updated 31 October 2026*