---
title: "Entropy Scheduling in Reinforcement Learning for Large"
description: "They then propose Entropy Scheduling (ES), which follows constant, cosine or stable-then-annealing targets. 34 references."
authors: ["Xingjin Wang", "Howe Tissue", "Lu Wang", "Linjing Li", "Daniel Dajun Zeng"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.206/"
doi: "10.18653/v1/2026.findings-acl.206"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-entropy-scheduling-reinforcement-learning-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-entropy-scheduling-reinforcement-learning-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/31/) / Entropy Scheduling in Reinforcement Learning for Large Language Models

# Entropy Scheduling in Reinforcement Learning for Large Language Models

**Authors:** Xingjin Wang, Howe Tissue, [Lu Wang](https://md.research.elamur.ai/free/en/authors/lu-wang/), [Linjing Li](https://md.research.elamur.ai/free/en/authors/linjing-li/) and [Daniel Dajun Zeng](https://md.research.elamur.ai/free/en/authors/daniel-dajun-zeng/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.206/) · [PDF](https://aclanthology.org/2026.findings-acl.206.pdf) · [DOI 10.18653/v1/2026.findings-acl.206](https://doi.org/10.18653/v1/2026.findings-acl.206) · [OpenAlex](https://openalex.org/W7166830543)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses how policy entropy should be handled during reinforcement learning with verifiable rewards (RLVR) for LLMs. Earlier work either keeps entropy high to preserve exploration, as in DAPO, or lets it collapse to gain quick improvements. The authors treat entropy as similar to the learning rate and argue that both behaviours can be combined in one run. In pilot runs with Qwen-2.5-7B-Base trained on DAPO-17k, they find what they call the Parallelogram Law of Entropy: holding entropy stable and lowering it near the end trails early decay at first but wins by the end. They then propose Entropy Scheduling (ES), which follows constant, cosine or stable-then-annealing targets. Because entropy cannot be set directly, a PID-based controller adjusts hyperparameters such as clip-higher to keep it on target. The annealing phase also suits curriculum-style additions such as higher-quality data. The work matters because it offers a simple, schedule-based way to tune RL training toward specific goals like Pass@k.

## Key points
- Annealing the entropy of DAPO-Qwen-32B with a lower clip value raised AIME2024 accuracy from 50.9 to 54.9 in only 40 training steps.
- In the pilot setup, entropy was kept stable using clip-higher (0.28), an entropy bonus (0.0015) or a KL penalty (0.005). All three methods produced similar effects, and evaluation used MATH500, AIME2024 and AIME2025.
- Suggested practice is either a cosine schedule or a constant entropy of about 0.3–0.5 followed by a 10% annealing phase. When optimizing Pass@k with large k, a smaller annealing ratio works better.
- In the annealing stage, adding AIME1983-2023 and BigMATH (difficulty levels 0–2) data improved results further. Further gains came from quadrupling the rollouts generated per prompt (8 up to 32) and extending the maximum response length fourfold (4k up to 16k).
- The experiments cover only reasoning-oriented RLVR; non-reasoning tasks such as dialogue and summarization were not tested.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) — Government body, Beijing, China · 246 works · 24,952 citations
- [Institute of Automation](https://md.research.elamur.ai/free/en/labs/institute-of-automation/) — Research institute, Beijing, China · 87 works · 14,883 citations
- [University of Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/universities/university-of-chinese-academy-of-sciences/) — University, Beijing, China · 188 works · 8,744 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Xingjin Wang, Howe Tissue, Lu Wang, Linjing Li and Daniel Dajun Zeng. 2026. Entropy Scheduling in Reinforcement Learning for Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026entropy,
  title = {{Entropy Scheduling in Reinforcement Learning for Large Language Models}},
  author = {Xingjin Wang, Howe Tissue, Lu Wang, Linjing Li and Daniel Dajun Zeng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.206},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.206}
}
```

- [← previous EntroBench: Evaluating LLM Watermarking Under Multi-Entropy Scenarios and Practical User Operations](https://md.research.elamur.ai/free/en/papers/qin-2026-entrobench-evaluating-llm-watermarking-under/)
- [next → Entropy-aware Masking for Masked Language Modeling](https://md.research.elamur.ai/free/en/papers/srinivasagan-2026-entropy-aware-masking-masked-language/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/31/)

## References

34 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abbas Abdolmaleki, Jost Tobias Springenberg, Yuval Tassa, Remi Munos, Nicolas Heess, and Martin Riedmiller. 2018. Maximum a posteriori policy optimisation. arXiv preprint arXiv:1806.06920. — [arXiv:1806.06920](https://arxiv.org/abs/1806.06920) [find](https://scholar.google.com/scholar?q=Abbas+Abdolmaleki%2C+Jost+Tobias+Springenberg%2C+Yuval+Tassa%2C+Remi+Munos%2C+Nicolas+Heess%2C+and+Martin+Riedmiller%2E+2018%2E+Maximum+a+posteriori+policy+optimisation%2E+arXiv+preprint+arXiv%3A1806%2E06920%2E+%E2%80%94+%5BarXiv%3A18)
2. Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. 2025. The unreasonable effectiveness of entropy minimization in llm reasoning. arXiv preprint arXiv: 2505.15134. [find](https://scholar.google.com/scholar?q=Shivam+Agarwal%2C+Zimin+Zhang%2C+Lifan+Yuan%2C+Jiawei+Han%2C+and+Hao+Peng%2E+2025%2E+The+unreasonable+effectiveness+of+entropy+minimization+in+llm+reasoning%2E+arXiv+preprint+arXiv%3A+2505%2E15134%2E)
3. Alon Albalak, Duy Phung, Nathan Lile, Rafael Rafailov, Kanishk Gandhi, Louis Castricato, Anikait Singh, Chase Blagden, Violet Xiang, Dakota Mahan, and Nick Haber. 2025. Big-math: A large-scale, high-quality math dataset for reinforcement learning in language models. Preprint, arXiv:2502.17387. [find](https://scholar.google.com/scholar?q=Alon+Albalak%2C+Duy+Phung%2C+Nathan+Lile%2C+Rafael+Rafailov%2C+Kanishk+Gandhi%2C+Louis+Castricato%2C+Anikait+Singh%2C+Chase+Blagden%2C+Violet+Xiang%2C+Dakota+Mahan%2C+and+Nick+Haber%2E+2025%2E+Big%2Dmath%3A+A+large%2Dscale%2C+high%2Dq)
4. Zhipeng Chen, Yingqian Min, Beichen Zhang, Jie Chen, Jinhao Jiang, Daixuan Cheng, Wayne Xin Zhao, Zheng Liu, Xu Miao, Yang Lu, and 1 others. 2025. An empirical study on eliciting and improving r1-like reasoning models. arXiv preprint arXiv:2503.04548. — [arXiv:2503.04548](https://arxiv.org/abs/2503.04548) [find](https://scholar.google.com/scholar?q=Zhipeng+Chen%2C+Yingqian+Min%2C+Beichen+Zhang%2C+Jie+Chen%2C+Jinhao+Jiang%2C+Daixuan+Cheng%2C+Wayne+Xin+Zhao%2C+Zheng+Liu%2C+Xu+Miao%2C+Yang+Lu%2C+and+1+others%2E+2025%2E+An+empirical+study+on+eliciting+and+improving+r1%2Dlike)
5. Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. 2025. [Reasoning with exploration: An entropy perspective](https://md.research.elamur.ai/free/en/papers/cheng-2025-reasoning-exploration-entropy-perspective/). arXiv preprint arXiv:2506.14758.
6. Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, Zhiyuan Liu, Hao Peng, Lei Bai, Wanli Ouyang, Yu Cheng, Bowen Zhou, and Ning Ding. 2025. [The entropy mechanism of reinforcement learning for reasoning language models](https://md.research.elamur.ai/free/en/papers/cui-2025-entropy-mechanism-reinforcement-learning-reasoning/). Preprint, a…
7. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in ll…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
8. Zitian Gao, Lynx Chen, Joey Zhou, and Bryan Dai. 2025. One-shot entropy minimization. arXiv preprint arXiv: 2505.20282. — [arXiv:2505.20282](https://arxiv.org/abs/2505.20282) [find](https://scholar.google.com/scholar?q=Zitian+Gao%2C+Lynx+Chen%2C+Joey+Zhou%2C+and+Bryan+Dai%2E+2025%2E+One%2Dshot+entropy+minimization%2E+arXiv+preprint+arXiv%3A+2505%2E20282%2E+%E2%80%94+%5BarXiv%3A2505%2E20282%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2505%2E20282%29)
9. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
10. Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, and 1 others. 2025. Skywork open reasoner 1 technical report. arXiv preprint arXiv:2505.22312. [find](https://scholar.google.com/scholar?q=Jujie+He%2C+Jiacai+Liu%2C+Chris+Yuhao+Liu%2C+Rui+Yan%2C+Chaojie+Wang%2C+Peng+Cheng%2C+Xiaoyu+Zhang%2C+Fuxiang+Zhang%2C+Jiacheng+Xu%2C+Wei+Shen%2C+and+1+others%2E+2025%2E+Skywork+open+reasoner+1+technical+report%2E+arXiv+prepri)
11. Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. 2025. [Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model](https://md.research.elamur.ai/free/en/papers/hu-2025-open-reasoner-zero-open-source/). arXiv preprint arXiv:2503.24290.
12. Kiam Heong Ang, Gregory Chong, and Yun Li. 2005. Pid control system analysis, design, and technology. IEEE transactions on control systems technology, 13(4):559–576. [find](https://scholar.google.com/scholar?q=Kiam+Heong+Ang%2C+Gregory+Chong%2C+and+Yun+Li%2E+2005%2E+Pid+control+system+analysis%2C+design%2C+and+technology%2E+IEEE+transactions+on+control+systems+technology%2C+13%284%29%3A559%E2%80%93576%2E)
13. Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zheng Leng Thai, Kaihuo Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, and 6 others. 2024. [Minicpm: Unveiling the potential of small lan…](https://md.research.elamur.ai/free/en/papers/hu-2024-minicpm-unveiling-potential-small-language/)
14. Johan Bjorck, Alon Benhaim, Vishrav Chaudhary, Furu Wei, and Xia Song. 2024. Scaling optimal lr across token horizons. arXiv preprint arXiv:2409.19913. — [arXiv:2409.19913](https://arxiv.org/abs/2409.19913) [find](https://scholar.google.com/scholar?q=Johan+Bjorck%2C+Alon+Benhaim%2C+Vishrav+Chaudhary%2C+Furu+Wei%2C+and+Xia+Song%2E+2024%2E+Scaling+optimal+lr+across+token+horizons%2E+arXiv+preprint+arXiv%3A2409%2E19913%2E+%E2%80%94+%5BarXiv%3A2409%2E19913%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2409%2E)
15. Natasha Jaques, Asma Ghandeharioun, Judy Hanwen Shen, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, and Rosalind Picard. 2019. Way off-policy batch deep reinforcement learning of implicit human preferences in dialog. arXiv preprint arXiv:1907.00456. — [arXiv:1907.00456](https://arxiv.org/abs/1907.00456) [find](https://scholar.google.com/scholar?q=Natasha+Jaques%2C+Asma+Ghandeharioun%2C+Judy+Hanwen+Shen%2C+Craig+Ferguson%2C+Agata+Lapedriza%2C+Noah+Jones%2C+Shixiang+Gu%2C+and+Rosalind+Picard%2E+2019%2E+Way+off%2Dpolicy+batch+deep+reinforcement+learning+of+implicit+)
16. Hilbert J Kappen, Vicenç Gómez, and Manfred Opper. 2012. Optimal control as a graphical model inference problem. Machine learning, 87:159–182. [find](https://scholar.google.com/scholar?q=Hilbert+J+Kappen%2C+Vicen%C3%A7+G%C3%B3mez%2C+and+Manfred+Opper%2E+2012%2E+Optimal+control+as+a+graphical+model+inference+problem%2E+Machine+learning%2C+87%3A159%E2%80%93182%2E)
17. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). In The Twelfth International Conference on Learning Representations.
18. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/), 2025. URL https://arxiv.org/abs/2503.20783. [↗](https://arxiv.org/abs/2503.20783)
19. Ilya Loshchilov and Frank Hutter. 2016. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2016%2E+Sgdr%3A+Stochastic+gradient+descent+with+warm+restarts%2E+arXiv+preprint+arXiv%3A1608%2E03983%2E)
20. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. 2025. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. Notion Blog. [find](https://scholar.google.com/scholar?q=Michael+Luo%2C+Sijun+Tan%2C+Justin+Wong%2C+Xiaoxiang+Shi%2C+William+Y%2E+Tang%2C+Manan+Roongta%2C+Colin+Cai%2C+Jeffrey+Luo%2C+Li+Erran+Li%2C+Raluca+Ada+Popa%2C+and+Ion+Stoica%2E+2025%2E+Deepscaler%3A+Surpassing+o1%2Dpreview+with+a)
21. OpenAI. 2024. Introducing openai o1. openai.com/o1/. https:// [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024%2E+Introducing+openai+o1%2E+openai%2Ecom%2Fo1%2F%2E+https%3A%2F%2F)
22. OpenAI. 2025. Introducing openai o3 and o4-mini. [https://openai.com/index/](https://openai.com/index/) introducing-o3-and-o4-mini/. [↗](https://openai.com/index/)
23. Jupinder Parmar, Sanjev Satheesh, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. 2024. [Reuse, don’t retrain: A recipe for continued pretraining of language models](https://md.research.elamur.ai/free/en/papers/parmar-2024-reuse-don-t-retrain-recipe/). Preprint, arXiv:2407.07263.
24. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=Qwen%2C+%3A%2C+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+)
25. Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, and 77 others. 2025. [Kimi k1.5: Scaling reinforcement l…](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/)
26. Howe Tissue, Venus Wang, and Lu Wang. 2024. Scaling law with learning rate annealing. Preprint, arXiv:2408.11029. — [arXiv:2408.11029](https://arxiv.org/abs/2408.11029) [find](https://scholar.google.com/scholar?q=Howe+Tissue%2C+Venus+Wang%2C+and+Lu+Wang%2E+2024%2E+Scaling+law+with+learning+rate+annealing%2E+Preprint%2C+arXiv%3A2408%2E11029%2E+%E2%80%94+%5BarXiv%3A2408%2E11029%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2408%2E11029%29)
27. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, and 1 others. 2025a. [Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/). arXiv preprint arXiv:2506.01939.
28. Xingjin Wang, Howe Tissue, Lu Wang, Linjing Li, and Daniel Dajun Zeng. 2025b. Learning dynamics in continual pre-training for large language models. Preprint, arXiv:2505.07796. — [arXiv:2505.07796](https://arxiv.org/abs/2505.07796) [find](https://scholar.google.com/scholar?q=Xingjin+Wang%2C+Howe+Tissue%2C+Lu+Wang%2C+Linjing+Li%2C+and+Daniel+Dajun+Zeng%2E+2025b%2E+Learning+dynamics+in+continual+pre%2Dtraining+for+large+language+models%2E+Preprint%2C+arXiv%3A2505%2E07796%2E+%E2%80%94+%5BarXiv%3A2505%2E07796%5D%28ht)
29. Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen. 2025c. [Reinforcement learning for reasoning in large language models with one training example](https://md.research.elamur.ai/free/en/papers/wang-2025-reinforcement-learning-reasoning-large-language/). arXiv preprint arX…
30. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, and 16 others. 2025. [Dapo: An open-source llm reinforcement learning s…](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/)
31. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, and 1 others. 2025. [Group sequence policy optimization](https://md.research.elamur.ai/free/en/papers/zheng-2025-group-sequence-policy-optimization/). arXiv preprint arXiv:2507.18071.
32. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. Preprint, arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+Preprint%2C+arXiv%3A1707%2E06347%2E)
33. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). Preprint, arXiv:2402.03300.
34. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). arXiv preprint arXiv: 2409.19256.

*Updated 2 November 2026*