---
title: "RL-PLUS — paper by Dong et al. (2026) — summary and ideas"
description: "RL-PLUS targets a known weakness of Reinforcement Learning with Verifiable Reward… 5.2 points. 45 references, cited by 6 papers here."
authors: ["Yihong Dong", "Xue Jiang", "Yongding Tao", "Huanyu Liu", "Kechi Zhang", "Lili Mou", "Rongyu Cao", "Yingwei MA", "Jue Chen", "Binhua Li", "Zhi Jin", "Fei Huang", "Yongbin Li", "Ge Li"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1994/"
doi: "10.18653/v1/2026.acl-long.1994"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/dong-2026-rl-plus-countering-capability-boundary/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dong-2026-rl-plus-countering-capability-boundary/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/) / RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization

# RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization

**Authors:** [Yihong Dong](https://md.research.elamur.ai/free/en/authors/yihong-dong/), [Xue Jiang](https://md.research.elamur.ai/free/en/authors/xue-jiang/), [Yongding Tao](https://md.research.elamur.ai/free/en/authors/yongding-tao/), Huanyu Liu, [Kechi Zhang](https://md.research.elamur.ai/free/en/authors/kechi-zhang/), [Lili Mou](https://md.research.elamur.ai/free/en/authors/lili-mou/), [Rongyu Cao](https://md.research.elamur.ai/free/en/authors/rongyu-cao/), [Yingwei MA](https://md.research.elamur.ai/free/en/authors/yingwei-ma/), Jue Chen, [Binhua Li](https://md.research.elamur.ai/free/en/authors/binhua-li/), [Zhi Jin](https://md.research.elamur.ai/free/en/authors/zhi-jin/), [Fei Huang](https://md.research.elamur.ai/free/en/authors/fei-huang/), [Yongbin Li](https://md.research.elamur.ai/free/en/authors/yongbin-li/) and [Ge Li](https://md.research.elamur.ai/free/en/authors/ge-li/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1994/) · [PDF](https://aclanthology.org/2026.acl-long.1994.pdf) · [DOI 10.18653/v1/2026.acl-long.1994](https://doi.org/10.18653/v1/2026.acl-long.1994) · [OpenAlex](https://openalex.org/W4417132896)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [6](https://md.research.elamur.ai/free/en/papers/dong-2026-rl-plus-countering-capability-boundary/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
RL-PLUS targets a known weakness of Reinforcement Learning with Verifiable Reward (RLVR). Standard on-policy training such as GRPO raises pass@1 but often lowers pass@k at large k. In other words, the tuned model solves fewer distinct problems than its base model, which the authors call capability boundary collapse. They attribute this to sparse binary rewards over a huge token-level action space, which push the policy toward refining paths it already knows.

RL-PLUS is a hybrid-policy method that mixes on-policy rollouts with external solution data. It has two parts:
- **Multiple Importance Sampling** corrects the distribution gap between the current policy and the external data, with lower bias and variance than naive ratios.
- **An Exploration-Based Advantage Function** gives more weight to correct reasoning steps that the current policy considers unlikely.

The authors support the method with theoretical analysis and experiments on six math benchmarks, six out-of-distribution tasks and several model families. RL-PLUS reaches state-of-the-art math results, and its pass@k curves stay above the base model's. This suggests RL can widen, not merely sharpen, an LLM's reasoning abilities.

## Key points
- **Math results:** evaluated on AIME 24, AIME25, AMC, MATH-500, Minerva and Olympiad. RL-PLUS averages 5.2 points higher than SFT+GRPO.
- **Qwen2.5-Math-7B:** RL-PLUS averages 53.4, compared with 19.0 for the base model, 44.1 for SFT and 45.5 for GRPO.
- **Other model families:** tested on LLaMA-3.1-8B, Deepseek-Math-7B and Qwen2.5-Math-1.5B/7B. Relative improvement over GRPO reaches up to 69.2%, including an 11.9-point absolute gain on LLaMA-3.1-8B, where GRPO performs poorly.
- **Ablation:**
  - Removing the Exploration-Based Advantage Function lowers the average from 53.4 to 50.9.
  - Removing Multiple Importance Sampling lowers it to 45.5.
  - Simpler ways of plugging in external data, including a LUFFY-like oracle assumption, all perform worse than RL-PLUS.
- **Training stability and limitations:** training Qwen2.5-Math-1.5B for over 10× the usual number of steps remains stable, and entropy stays non-zero. Experiments mostly use models of about 7B parameters. Code is available at github.com/YihongDong/RL-PLUS.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) — University, Beijing, China · 244 works · 39,359 citations
- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 168 works · 15,386 citations
- [University of Alberta](https://md.research.elamur.ai/free/en/universities/university-of-alberta/) — University, Edmonton, Canada · 44 works · 3,236 citations
- [Canadian Institute for Advanced Research](https://md.research.elamur.ai/free/en/labs/canadian-institute-for-advanced-research/) — Research institute, Toronto, Canada · 28 works · 16,725 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Yihong Dong, Xue Jiang, Yongding Tao, Huanyu Liu, Kechi Zhang, Lili Mou, Rongyu Cao, Yingwei MA, Jue Chen, Binhua Li, Zhi Jin, Fei Huang, Yongbin Li and Ge Li. 2026. RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{dong2026rl,
  title = {{RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization}},
  author = {Yihong Dong, Xue Jiang, Yongding Tao, Huanyu Liu, Kechi Zhang, Lili Mou, Rongyu Cao, Yingwei MA, Jue Chen, Binhua Li, Zhi Jin, Fei Huang, Yongbin Li and Ge Li},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1994},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1994}
}
```

- [← previous Revisiting Model Interpolation for Efficient Reasoning](https://md.research.elamur.ai/free/en/papers/wu-2026-revisiting-model-interpolation-efficient-reasoning/)
- [next → Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/tan-2026-scaling-behaviors-llm-reinforcement-learning/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/)

## References

45 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. language models trained on code. arXiv preprint arXiv:2107.03374. — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
2. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457.
3. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). CoRR, abs/2110.14168.
4. Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. 2025a. [Process reinforcement through implicit rewards](https://md.research.elamur.ai/free/en/papers/cui-2025-process-reinforcement-through-implicit-rewards/). arXiv preprint arXiv:2502.01456.
5. Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al. 2025b. The entropy mechanism of reinforcement learning for reasoning language models. arXiv preprint arXiv:2505.22617. [find](https://scholar.google.com/scholar?q=Ganqu+Cui%2C+Yuchen+Zhang%2C+Jiacheng+Chen%2C+Lifan+Yuan%2C+Zhi+Wang%2C+Yuxin+Zuo%2C+Haozhan+Li%2C+Yuchen+Fan%2C+Huayu+Chen%2C+Weize+Chen%2C+et+al%2E+2025b%2E+The+entropy+mechanism+of+reinforcement+learning+for+reasoning+lan)
6. Yihong Dong, Jiazheng Ding, Xue Jiang, Ge Li, Zhuo Li, and Zhi Jin. 2025a. Codescore: Evaluating code generation by learning code execution. ACM Trans. Softw. Eng. Methodol., 34(3):77:1–77:22. [find](https://scholar.google.com/scholar?q=Yihong+Dong%2C+Jiazheng+Ding%2C+Xue+Jiang%2C+Ge+Li%2C+Zhuo+Li%2C+and+Zhi+Jin%2E+2025a%2E+Codescore%3A+Evaluating+code+generation+by+learning+code+execution%2E+ACM+Trans%2E+Softw%2E+Eng%2E+Methodol%2E%2C+34%283%29%3A77%3A1%E2%80%9377%3A22%2E)
7. Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. 2024. Self-collaboration code generation via chatgpt. ACM Trans. Softw. Eng. Methodol., 33(7):189:1–189:38. [find](https://scholar.google.com/scholar?q=Yihong+Dong%2C+Xue+Jiang%2C+Zhi+Jin%2C+and+Ge+Li%2E+2024%2E+Self%2Dcollaboration+code+generation+via+chatgpt%2E+ACM+Trans%2E+Softw%2E+Eng%2E+Methodol%2E%2C+33%287%29%3A189%3A1%E2%80%93189%3A38%2E)
8. Yihong Dong, Xue Jiang, Jiaru Qian, Tian Wang, Kechi Zhang, Zhi Jin, and Ge Li. 2025b. A survey on code generation with llm-based agents. CoRR, abs/2508.00083. — [arXiv:2508.00083](https://arxiv.org/abs/2508.00083) [find](https://scholar.google.com/scholar?q=Yihong+Dong%2C+Xue+Jiang%2C+Jiaru+Qian%2C+Tian+Wang%2C+Kechi+Zhang%2C+Zhi+Jin%2C+and+Ge+Li%2E+2025b%2E+A+survey+on+code+generation+with+llm%2Dbased+agents%2E+CoRR%2C+abs%2F2508%2E00083%2E+%E2%80%94+%5BarXiv%3A2508%2E00083%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
9. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, et al. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
10. Hongyi James Cai, Junlin Wang, Xiaoyin Chen, and Bhuwan Dhingra. 2025. How much backtracking is enough? exploring the interplay of sft and rl in enhancing llm reasoning. arXiv preprint arXiv:2505.24273. [find](https://scholar.google.com/scholar?q=Hongyi+James+Cai%2C+Junlin+Wang%2C+Xiaoyin+Chen%2C+and+Bhuwan+Dhingra%2E+2025%2E+How+much+backtracking+is+enough%3F+exploring+the+interplay+of+sft+and+rl+in+enhancing+llm+reasoning%2E+arXiv+preprint+arXiv%3A2505%2E2427)
11. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. Deepseek-coder: When the large language model meets programming– the rise of code intelligence. arXiv preprint arXiv:2401.14196. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Qihao+Zhu%2C+Dejian+Yang%2C+Zhenda+Xie%2C+Kai+Dong%2C+Wentao+Zhang%2C+Guanting+Chen%2C+Xiao+Bi%2C+Yu+Wu%2C+YK+Li%2C+et+al%2E+2024%2E+Deepseek%2Dcoder%3A+When+the+large+language+model+meets+programming%E2%80%93+the+rise+of+co)
12. Jack Chen, Fazhong Liu, Naruto Liu, Yuhan Luo, Erqu Qin, Harry Zheng, Tian Dong, Haojin Zhu, Yan Meng, and Xiao Wang. 2025. Step-wise adaptive integration of supervised fine-tuning and reinforcement learning for task-specific llms. arXiv preprint arXiv:2505.13026. — [arXiv:2505.13026](https://arxiv.org/abs/2505.13026) [find](https://scholar.google.com/scholar?q=Jack+Chen%2C+Fazhong+Liu%2C+Naruto+Liu%2C+Yuhan+Luo%2C+Erqu+Qin%2C+Harry+Zheng%2C+Tian+Dong%2C+Haojin+Zhu%2C+Yan+Meng%2C+and+Xiao+Wang%2E+2025%2E+Step%2Dwise+adaptive+integration+of+supervised+fine%2Dtuning+and+reinforcement+l)
13. Alexander Havrilla, Yuqing Du, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi-Yu, et al. 2024. Teaching large language models to reason with reinforcement learning. arXiv preprint arXiv:2403.04642. [find](https://scholar.google.com/scholar?q=Alexander+Havrilla%2C+Yuqing+Du%2C+Sharath+Chandra+Raparthy%2C+Christoforos+Nalmpantis%2C+Jane+Dwivedi%2DYu%2C+et+al%2E+2024%2E+Teaching+large+language+models+to+reason+with+reinforcement+learning%2E+arXiv+preprint+arX)
14. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yux… [find](https://scholar.google.com/scholar?q=Mark+Chen%2C+Jerry+Tworek%2C+Heewoo+Jun%2C+Qiming+Yuan%2C+Henrique+Ponde+De+Oliveira+Pinto%2C+Jared+Kaplan%2C+Harri+Edwards%2C+Yuri+Burda%2C+Nicholas+Joseph%2C+Greg+Brockman%2C+et+al%2E+2021%2E+Evaluating+large+Chaoqun+He%2C+R)
15. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the MATH dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In NeurIPS Datasets and Benchmarks.
16. Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. 2025. [Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model](https://md.research.elamur.ai/free/en/papers/hu-2025-open-reasoner-zero-open-source/). arXiv preprint arXiv:2503.24290.
17. Yihao Liu, Shuocheng Li, Lang Cao, Yuhang Xie, Mengyu Zhou, Haoyu Dong, Xiaojun Ma, Shi Han, and Dongmei Zhang. 2025a. Superrl: Reinforcement learning with supervision to boost language model reasoning. arXiv preprint arXiv:2506.01096. — [arXiv:2506.01096](https://arxiv.org/abs/2506.01096) [find](https://scholar.google.com/scholar?q=Yihao+Liu%2C+Shuocheng+Li%2C+Lang+Cao%2C+Yuhang+Xie%2C+Mengyu+Zhou%2C+Haoyu+Dong%2C+Xiaojun+Ma%2C+Shi+Han%2C+and+Dongmei+Zhang%2E+2025a%2E+Superrl%3A+Reinforcement+learning+with+supervision+to+boost+language+model+reasonin)
18. Zichen Liu, Changyu Chen, Wenjun Li, Tianyu Pang, Chao Du, and Min Lin. 2025b. There may not be aha moment in r1-zero-like training — a pilot study. [https://oatllm.notion.site/oat-zero](https://oatllm.notion.site/oat-zero). Notion Blog. [↗](https://oatllm.notion.site/oat-zero)
19. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025c. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/). arXiv preprint arXiv:2503.20783.
20. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). arXiv preprint arXiv:2403.07974.
21. Lu Ma, Hao Liang, Meiyi Qiang, Lexiang Tang, Xiaochen Ma, Zhen Hao Wong, Junbo Niu, Chengyu Shen, Runming He, Bin Cui, et al. 2025. Learning what reinforcement learning can’t: Interleaved online fine-tuning for hardest questions. arXiv preprint arXiv:2506.07527. — [arXiv:2506.07527](https://arxiv.org/abs/2506.07527) [find](https://scholar.google.com/scholar?q=Lu+Ma%2C+Hao+Liang%2C+Meiyi+Qiang%2C+Lexiang+Tang%2C+Xiaochen+Ma%2C+Zhen+Hao+Wong%2C+Junbo+Niu%2C+Chengyu+Shen%2C+Runming+He%2C+Bin+Cui%2C+et+al%2E+2025%2E+Learning+what+reinforcement+learning+can%E2%80%99t%3A+Interleaved+online+fine%2D)
22. Xue Jiang, Yihong Dong, Mengyang Liu, Hongyi Deng, Tian Wang, Yongding Tao, Rongyu Cao, Binhua Li, Zhi Jin, Wenpin Jiao, Fei Huang, Yongbin Li, and Ge Li. 2025. [Coderl+: Improving code generation via reinforcement with execution semantics alignment](https://md.research.elamur.ai/free/en/papers/jiang-2026-coderl-improving-code-generation-reinforcement/). CoRR, abs/2510.18471.
23. OpenAI. 2024. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/) and model report. Technical report, OpenAI o1 series, available online.
24. Xue Jiang, Yihong Dong, Lecheng Wang, Zheng Fang, Qiwei Shang, Ge Li, Zhi Jin, and Wenpin Jiao. 2024. Self-planning code generation with large language models. ACM Trans. Softw. Eng. Methodol., 33(7):182:1–182:30. [find](https://scholar.google.com/scholar?q=Xue+Jiang%2C+Yihong+Dong%2C+Lecheng+Wang%2C+Zheng+Fang%2C+Qiwei+Shang%2C+Ge+Li%2C+Zhi+Jin%2C+and+Wenpin+Jiao%2E+2024%2E+Self%2Dplanning+code+generation+with+large+language+models%2E+ACM+Trans%2E+Softw%2E+Eng%2E+Methodol%2E%2C+33%287%29%3A)
25. Xue Jiang, Tianyu Zhang, Ge Li, Mengyang Liu, Taozhi Chen, Zhenhua Xu, Binhua Li, Wenpin Jiao, Zhi Jin, Yongbin Li, and Yihong Dong. 2026. Think anywhere in code generation. CoRR, abs/2603.29957. — [arXiv:2603.29957](https://arxiv.org/abs/2603.29957) [find](https://scholar.google.com/scholar?q=Xue+Jiang%2C+Tianyu+Zhang%2C+Ge+Li%2C+Mengyang+Liu%2C+Taozhi+Chen%2C+Zhenhua+Xu%2C+Binhua+Li%2C+Wenpin+Jiao%2C+Zhi+Jin%2C+Yongbin+Li%2C+and+Yihong+Dong%2E+2026%2E+Think+anywhere+in+code+generation%2E+CoRR%2C+abs%2F2603%2E29957%2E+%E2%80%94+%5Ba)
26. KimiTeam. 2025. [Kimi k1.5: Scaling reinforcement learning with llms](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/). arXiv preprint arXiv:2501.12599.
27. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay V. Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. 2022. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). In NeurIPS.
28. Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q. Jiang, Ziju Shen, et al. 2024. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. [https://huggingface.co/datasets/](https://huggingface.co/datasets/) Numi… [↗](https://huggingface.co/datasets/)
29. Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár. 2017. [Focal loss for dense object detection](https://md.research.elamur.ai/free/en/papers/lin-2017-focal-loss-dense-object-detection/). In ICCV, pages 2999–3007. IEEE Computer Society.
30. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
31. Martin L Puterman. 2014. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons. [find](https://scholar.google.com/scholar?q=Martin+L+Puterman%2E+2014%2E+Markov+decision+processes%3A+discrete+stochastic+dynamic+programming%2E+John+Wiley+%26+Sons%2E)
32. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. [Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). In First Conference on Language Modeling.
33. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
34. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). CoRR, abs/2402.03300.
35. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). arXiv preprint arXiv: 2409.19256.
36. Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, et al. 2025. Reinforcement learning for reasoning in large language models with one training example. arXiv preprint arXiv:2504.20571. [find](https://scholar.google.com/scholar?q=Yiping+Wang%2C+Qing+Yang%2C+Zhiyuan+Zeng%2C+Liliang+Ren%2C+Liyuan+Liu%2C+Baolin+Peng%2C+Hao+Cheng%2C+Xuehai+He%2C+Kuan+Wang%2C+Jianfeng+Gao%2C+et+al%2E+2025%2E+Reinforcement+learning+for+reasoning+in+large+language+models+wi)
37. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. 2024a. [Mmlu-pro: A more robust and challenging multi-task language understanding benchmark](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/). Advances in Neural Information Processing Systems, 37:95266– 95290.
38. Zhichao Wang, Bin Bi, Zixu Zhu, Xiangbo Mao, Jun Wang, and Shiyu Wang. 2024b. Uft: Unifying fine-tuning of sft and rlhf/dpo/una through a generalized implicit reward function. arXiv preprint arXiv:2410.21438. — [arXiv:2410.21438](https://arxiv.org/abs/2410.21438) [find](https://scholar.google.com/scholar?q=Zhichao+Wang%2C+Bin+Bi%2C+Zixu+Zhu%2C+Xiangbo+Mao%2C+Jun+Wang%2C+and+Shiyu+Wang%2E+2024b%2E+Uft%3A+Unifying+fine%2Dtuning+of+sft+and+rlhf%2Fdpo%2Funa+through+a+generalized+implicit+reward+function%2E+arXiv+preprint+arXiv%3A241)
39. Jinyang Wu, Chonghua Liao, Mingkuan Feng, Shuai Zhang, Zhengqi Wen, Pengpeng Shao, Huazhe Xu, and Jianhua Tao. 2025. Thought-augmented policy optimization: Bridging external guidance and internal capabilities. arXiv preprint arXiv:2505.15692. — [arXiv:2505.15692](https://arxiv.org/abs/2505.15692) [find](https://scholar.google.com/scholar?q=Jinyang+Wu%2C+Chonghua+Liao%2C+Mingkuan+Feng%2C+Shuai+Zhang%2C+Zhengqi+Wen%2C+Pengpeng+Shao%2C+Huazhe+Xu%2C+and+Jianhua+Tao%2E+2025%2E+Thought%2Daugmented+policy+optimization%3A+Bridging+external+guidance+and+internal+capa)
40. Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu, Yu Cheng, and Yue Zhang. 2025. Learning to reason under off-policy guidance. arXiv preprint arXiv:2504.14945. [find](https://scholar.google.com/scholar?q=Jianhao+Yan%2C+Yafu+Li%2C+Zican+Hu%2C+Zhi+Wang%2C+Ganqu+Cui%2C+Xiaoye+Qu%2C+Yu+Cheng%2C+and+Yue+Zhang%2E+2025%2E+Learning+to+reason+under+off%2Dpolicy+guidance%2E+arXiv+preprint+arXiv%3A2504%2E14945%2E)
41. An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. 2024. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122. — [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://md.research.elamur.ai/free/en/papers/yang-2024-qwen2-5-math-technical-report/)
42. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. 2025. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). arXiv preprint arXiv:2503.14476.
43. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. 2025a. [Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/) CoRR, abs/2504.13837.
44. Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, et al. 2025b. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks. arXiv preprint arXiv:2504.05118. [find](https://scholar.google.com/scholar?q=Yu+Yue%2C+Yufeng+Yuan%2C+Qiying+Yu%2C+Xiaochen+Zuo%2C+Ruofei+Zhu%2C+Wenyuan+Xu%2C+Jiaze+Chen%2C+Chengyi+Wang%2C+TianTian+Fan%2C+Zhengyin+Du%2C+et+al%2E+2025b%2E+Vapo%3A+Efficient+and+reliable+reinforcement+learning+for+advance)
45. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. 2025. [Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/). arXiv preprint arXiv:2503.18892.

*Updated 30 October 2026*