---
title: "Revisiting Entropy Regularization — paper by Zhang et al. (2026)"
description: "Based on this, the authors propose Adaptive Entropy Regularization (AER), which has three parts: per-sample coefficients… 52 references."
authors: ["Xiaoyun Zhang", "Xiaojian Yuan", "Di Huang", "Wang You", "Chen Hu", "Jingqing Ruan", "Kejiang Chen", "Xing Hu"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.895/"
doi: "10.18653/v1/2026.findings-acl.895"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-revisiting-entropy-regularization-adaptive-coefficient/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-revisiting-entropy-regularization-adaptive-coefficient/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/47/) / Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning

# Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning

**Authors:** [Xiaoyun Zhang](https://md.research.elamur.ai/free/en/authors/xiaoyun-zhang/), Xiaojian Yuan, [Di Huang](https://md.research.elamur.ai/free/en/authors/di-huang/), Wang You, [Chen Hu](https://md.research.elamur.ai/free/en/authors/chen-hu/), Jingqing Ruan, Kejiang Chen and [Xing Hu](https://md.research.elamur.ai/free/en/authors/xing-hu/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.895/) · [PDF](https://aclanthology.org/2026.findings-acl.895.pdf) · [DOI 10.18653/v1/2026.findings-acl.895](https://doi.org/10.18653/v1/2026.findings-acl.895) · [OpenAlex](https://openalex.org/W7166869162)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses entropy collapse in Reinforcement Learning with Verifiable Rewards (RLVR) for LLMs. During RLVR, a policy can quickly become nearly deterministic, which stops exploration and limits reasoning gains. Entropy regularization is the standard fix, but recent pipelines often skip it because a fixed coefficient is fragile. If it is too small, entropy still collapses. If it is too large, entropy explodes. A value tuned for one setup can also hurt another.

Pilot runs with Qwen3-4B-Base and GRPO showed two things:
- Harder problems benefit from more exploration, while easy ones do not.
- Training works best when entropy stays moderately below its starting level.

Based on this, the authors propose Adaptive Entropy Regularization (AER), which has three parts:
- per-sample coefficients scaled by how difficult a question currently is for the policy;
- a target entropy defined as a fraction of the run's initial entropy;
- a global scale factor adjusted in a feedback loop to keep entropy near that target.

On several math reasoning benchmarks, AER beats GRPO and other baselines on both pass@1 and pass@k. This suggests that adaptive coefficients make entropy regularization a practical tool again.

## Key points
- Pilot setup: GSM8K served as the easy task and a mix of AIME and AMC as the hard one. A large fixed coefficient of 0.1 caused sudden accuracy drops and much longer outputs on both.
- In the ablation, adding all three components on top of GRPO raised AIME25 from 21.4 to 25.1.
- Pivot accuracy ρ: average pass@1 reached 55.8 at ρ=0 and fell to 53.5 at ρ=1.0, so a small ρ (≤0.2) is recommended.
- Reduction ratio τ: the best value was τ=0.4 (55.4 average pass@1), with recommended defaults of τ ∈ [0.4, 0.6].
- Transfer without retuning: with the same defaults (ρ=0.2, τ=0.4), swapping DeepScaleR for DAPO_MATH_17K still let AER outperform Clip-Higher and match or exceed GRPO on Qwen3-8B-Base and Qwen2.5-7B-Base.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 145 works · 114,409 citations
- [University of Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/universities/university-of-chinese-academy-of-sciences/) — University, Beijing, China · 188 works · 8,743 citations
- [Institute of Computing Technology](https://md.research.elamur.ai/free/en/labs/institute-of-computing-technology/) — Research institute, Beijing, China · 57 works · 3,782 citations
- [StepFun AI](https://md.research.elamur.ai/free/en/companies/stepfun-ai/) — Company, Shanghai, China · 5 works · 540 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Xiaoyun Zhang, Xiaojian Yuan, Di Huang, Wang You, Chen Hu, Jingqing Ruan, Kejiang Chen and Xing Hu. 2026. Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026revisiting,
  title = {{Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning}},
  author = {Xiaoyun Zhang, Xiaojian Yuan, Di Huang, Wang You, Chen Hu, Jingqing Ruan, Kejiang Chen and Xing Hu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.895},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.895}
}
```

- [← previous Revisiting a Pain in the Neck: A Semantic Reasoning Benchmark for Language Models](https://md.research.elamur.ai/free/en/papers/liu-2026-revisiting-pain-neck-semantic-reasoning/)
- [next → Revisiting Faithfulness Annotations for Long-form Summaries](https://md.research.elamur.ai/free/en/papers/zhong-2026-revisiting-faithfulness-annotations-long-form/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/47/)

## References

52 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. AI-MO. 2024. Amc 2023. [https://huggingface.co/datasets/AI-MO/aimo-validation-amc](https://huggingface.co/datasets/AI-MO/aimo-validation-amc). [↗](https://huggingface.co/datasets/AI-MO/aimo-validation-amc)
2. Karl Johan Åström and Richard Murray. 2021. Feedback systems: an introduction for scientists and engineers. Princeton university press. [find](https://scholar.google.com/scholar?q=Karl+Johan+%C3%85str%C3%B6m+and+Richard+Murray%2E+2021%2E+Feedback+systems%3A+an+introduction+for+scientists+and+engineers%2E+Princeton+university+press%2E)
3. Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. 2025. [Reasoning with exploration: An entropy perspective](https://md.research.elamur.ai/free/en/papers/cheng-2025-reasoning-exploration-entropy-perspective/). arXiv preprint arXiv:2506.14758.
4. Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al. 2025b. [The entropy mechanism of reinforcement learning for reasoning language models](https://md.research.elamur.ai/free/en/papers/cui-2025-entropy-mechanism-reinforcement-learning-reasoning/). arXiv preprint arXiv:2505.22617.
5. Runpeng Dai, Linfeng Song, Haolin Liu, Zhenwen Liang, Dian Yu, Haitao Mi, Zhaopeng Tu, Rui Liu, Tong Zheng, Hongtu Zhu, et al. 2025. Cde: Curiosity-driven exploration for efficient reinforcement learning in large language models. arXiv preprint arXiv:2509.09675. — [arXiv:2509.09675](https://arxiv.org/abs/2509.09675) [find](https://scholar.google.com/scholar?q=Runpeng+Dai%2C+Linfeng+Song%2C+Haolin+Liu%2C+Zhenwen+Liang%2C+Dian+Yu%2C+Haitao+Mi%2C+Zhaopeng+Tu%2C+Rui+Liu%2C+Tong+Zheng%2C+Hongtu+Zhu%2C+et+al%2E+2025%2E+Cde%3A+Curiosity%2Ddriven+exploration+for+efficient+reinforcement+learn)
6. Jingtong Gao, Ling Pan, Yejing Wang, Rui Zhong, Chi Lu, Qingpeng Cai, Peng Jiang, and Xiangyu Zhao. 2025. Navigate the unknown: Enhancing llm reasoning with intrinsic motivation guided exploration. arXiv preprint arXiv:2505.17621. — [arXiv:2505.17621](https://arxiv.org/abs/2505.17621) [find](https://scholar.google.com/scholar?q=Jingtong+Gao%2C+Ling+Pan%2C+Yejing+Wang%2C+Rui+Zhong%2C+Chi+Lu%2C+Qingpeng+Cai%2C+Peng+Jiang%2C+and+Xiangyu+Zhao%2E+2025%2E+Navigate+the+unknown%3A+Enhancing+llm+reasoning+with+intrinsic+motivation+guided+exploration%2E+ar)
7. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
8. Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, et al. 2025. Skywork open reasoner 1 technical report. arXiv preprint arXiv:2505.22312. [find](https://scholar.google.com/scholar?q=Jujie+He%2C+Jiacai+Liu%2C+Chris+Yuhao+Liu%2C+Rui+Yan%2C+Chaojie+Wang%2C+Peng+Cheng%2C+Xiaoyu+Zhang%2C+Fuxiang+Zhang%2C+Jiacheng+Xu%2C+Wei+Shen%2C+et+al%2E+2025%2E+Skywork+open+reasoner+1+technical+report%2E+arXiv+preprint+arXi)
9. Mikael Henaff, Roberta Raileanu, Minqi Jiang, and Tim Rocktäschel. 2022. Exploration via elliptical episodic bonuses. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Mikael+Henaff%2C+Roberta+Raileanu%2C+Minqi+Jiang%2C+and+Tim+Rockt%C3%A4schel%2E+2022%2E+Exploration+via+elliptical+episodic+bonuses%2E+In+Advances+in+Neural+Information+Processing+Systems%2E)
10. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874.
11. Jian Hu, Jason Klein Liu, Haotian Xu, and Wei Shen. 2025a. [Reinforce++: An efficient rlhf algorithm with robustness to both prompt and reward models](https://md.research.elamur.ai/free/en/papers/hu-2025-reinforce-stabilizing-critic-free-policy/). arXiv preprint arXiv:2501.03262.
12. Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. 2025b. [Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model](https://md.research.elamur.ai/free/en/papers/hu-2025-open-reasoner-zero-open-source/). arXiv preprint arXiv:2503.24290.
13. Haque Ishfaq, Qiwen Cui, Viet Nguyen, Alex Ayoub, Zhuoran Yang, Zhaoran Wang, Doina Precup, and Lin Yang. 2021. Randomized exploration in reinforcement learning with general value function approximation. In International Conference on Machine Learning, pages 4607–4616. PMLR. [find](https://scholar.google.com/scholar?q=Haque+Ishfaq%2C+Qiwen+Cui%2C+Viet+Nguyen%2C+Alex+Ayoub%2C+Zhuoran+Yang%2C+Zhaoran+Wang%2C+Doina+Precup%2C+and+Lin+Yang%2E+2021%2E+Randomized+exploration+in+reinforcement+learning+with+general+value+function+approximati)
14. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. 2024. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). arXiv preprint arXiv:2412.16720.
15. Ai Jian, Jingqing Ruan, Xing Ma, Dailin Li, Weipeng Zhang, Ke Zeng, and Xunliang Cai. 2026. Patarm: Bridging pairwise and pointwise signals via preference-aware task-adaptive reward modeling. Preprint, arXiv:2510.24235. from human feedback with ai feedback. In International Conference on Machine Lea… [find](https://scholar.google.com/scholar?q=Ai+Jian%2C+Jingqing+Ruan%2C+Xing+Ma%2C+Dailin+Li%2C+Weipeng+Zhang%2C+Ke+Zeng%2C+and+Xunliang+Cai%2E+2026%2E+Patarm%3A+Bridging+pairwise+and+pointwise+signals+via+preference%2Daware+task%2Dadaptive+reward+modeling%2E+Preprint)
16. Ang Li, Zhihang Yuan, Yang Zhang, Shouda Liu, and Yisen Wang. 2025a. Know when to explore: Difficulty-aware certainty as a guide for llm reinforcement learning. arXiv preprint arXiv:2509.00125. — [arXiv:2509.00125](https://arxiv.org/abs/2509.00125) [find](https://scholar.google.com/scholar?q=Ang+Li%2C+Zhihang+Yuan%2C+Yang+Zhang%2C+Shouda+Liu%2C+and+Yisen+Wang%2E+2025a%2E+Know+when+to+explore%3A+Difficulty%2Daware+certainty+as+a+guide+for+llm+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2509%2E00125%2E+%E2%80%94+%5BarX)
17. Jian Li, Yizhang Jin, Dongqi Liu, Hang Ding, Jiafu Wu, Dongsheng Chen, Yunhang Shen, Yulei Qin, Ying Tai, Chengjie Wang, Xiaotong Yuan, and Yabiao Wang. 2026. Se-search: Self-evolving search agent via memory and dense reward. Preprint, arXiv:2603.03293. — [arXiv:2603.03293](https://arxiv.org/abs/2603.03293) [find](https://scholar.google.com/scholar?q=Jian+Li%2C+Yizhang+Jin%2C+Dongqi+Liu%2C+Hang+Ding%2C+Jiafu+Wu%2C+Dongsheng+Chen%2C+Yunhang+Shen%2C+Yulei+Qin%2C+Ying+Tai%2C+Chengjie+Wang%2C+Xiaotong+Yuan%2C+and+Yabiao+Wang%2E+2026%2E+Se%2Dsearch%3A+Self%2Devolving+search+agent+via)
18. Jiazheng Li, Hong Lu, Kaiyue Wen, Zaiwen Yang, Jiaxuan Gao, Hongzhou Lin, Yi Wu, and Jingzhao Zhang. 2025b. Questa: Expanding reasoning capacity in llms via question augmentation. arXiv preprint arXiv:2507.13266. [find](https://scholar.google.com/scholar?q=Jiazheng+Li%2C+Hong+Lu%2C+Kaiyue+Wen%2C+Zaiwen+Yang%2C+Jiaxuan+Gao%2C+Hongzhou+Lin%2C+Yi+Wu%2C+and+Jingzhao+Zhang%2E+2025b%2E+Questa%3A+Expanding+reasoning+capacity+in+llms+via+question+augmentation%2E+arXiv+preprint+arXiv)
19. Qingbin Li, Rongkun Xue, Jie Wang, Ming Zhou, Zhi Li, Xiaofeng Ji, Yongqi Wang, Miao Liu, Zheming Yang, Minghui Qiu, et al. 2025c. Cure: Critical-token-guided re-concatenation for entropy-collapse prevention. arXiv preprint arXiv:2508.11016. — [arXiv:2508.11016](https://arxiv.org/abs/2508.11016) [find](https://scholar.google.com/scholar?q=Qingbin+Li%2C+Rongkun+Xue%2C+Jie+Wang%2C+Ming+Zhou%2C+Zhi+Li%2C+Xiaofeng+Ji%2C+Yongqi+Wang%2C+Miao+Liu%2C+Zheming+Yang%2C+Minghui+Qiu%2C+et+al%2E+2025c%2E+Cure%3A+Critical%2Dtoken%2Dguided+re%2Dconcatenation+for+entropy%2Dcollapse+pre)
20. Ziheng Li, Zexu Sun, Jinman Zhao, Erxue Min, Yongcheng Zeng, Hui Wu, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Xu Chen, et al. 2025d. Staying in the sweet spot: Responsive reasoning evolution via capability-adaptive hint scaffolding. arXiv preprint arXiv:2509.06923. — [arXiv:2509.06923](https://arxiv.org/abs/2509.06923) [find](https://scholar.google.com/scholar?q=Ziheng+Li%2C+Zexu+Sun%2C+Jinman+Zhao%2C+Erxue+Min%2C+Yongcheng+Zeng%2C+Hui+Wu%2C+Hengyi+Cai%2C+Shuaiqiang+Wang%2C+Dawei+Yin%2C+Xu+Chen%2C+et+al%2E+2025d%2E+Staying+in+the+sweet+spot%3A+Responsive+reasoning+evolution+via+capabi)
21. Ziniu Li, Tian Xu, Yushun Zhang, Zhihang Lin, Yang Yu, Ruoyu Sun, and Zhi-Quan Luo. 2024. Remax: a simple, effective, and efficient reinforcement learning method for aligning large language models. In Proceedings of the 41st International Conference on Machine Learning, pages 29128–29163. [find](https://scholar.google.com/scholar?q=Ziniu+Li%2C+Tian+Xu%2C+Yushun+Zhang%2C+Zhihang+Lin%2C+Yang+Yu%2C+Ruoyu+Sun%2C+and+Zhi%2DQuan+Luo%2E+2024%2E+Remax%3A+a+simple%2C+effective%2C+and+efficient+reinforcement+learning+method+for+aligning+large+language+models%2E+In)
22. Kehan Jiang, Haonan Dong, Zhaolu Kang, Zhengzhou Zhu, and Guojie Song. 2026. [Foe: Forest of errors makes the first solution the best in large reasoning models](https://md.research.elamur.ai/free/en/papers/jiang-2026-foe-forest-errors-makes-first/). Preprint, arXiv:2604.02967.
23. Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, and Yi Dong. 2025a. [Prorl: Prolonged reinforcement learning expands reasoning boundaries in large language models](https://md.research.elamur.ai/free/en/papers/liu-2025-prorl-prolonged-reinforcement-learning-expands/). arXiv preprint arXiv:2505.24864.
24. Yuhua Jiang, Yuwen Xiong, Yufeng Yuan, Chao Xin, Wenyuan Xu, Yu Yue, Qianchuan Zhao, and Lin Yan. 2025a. [Pag: Multi-turn reinforced llm self-correction with policy as generative verifier](https://md.research.elamur.ai/free/en/papers/jiang-2025-pag-multi-turn-reinforced-llm/). arXiv preprint arXiv:2506.10406.
25. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025b. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/). arXiv preprint arXiv:2503.20783.
26. Yuxian Jiang, Yafu Li, Guanxu Chen, Dongrui Liu, Yu Cheng, and Jing Shao. 2025b. Rethinking entropy regularization in large reasoning models. arXiv preprint arXiv:2509.25133. — [arXiv:2509.25133](https://arxiv.org/abs/2509.25133) [find](https://scholar.google.com/scholar?q=Yuxian+Jiang%2C+Yafu+Li%2C+Guanxu+Chen%2C+Dongrui+Liu%2C+Yu+Cheng%2C+and+Jing+Shao%2E+2025b%2E+Rethinking+entropy+regularization+in+large+reasoning+models%2E+arXiv+preprint+arXiv%3A2509%2E25133%2E+%E2%80%94+%5BarXiv%3A2509%2E25133%5D%28http)
27. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, et al. 2025. Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl. Notion Blog. [find](https://scholar.google.com/scholar?q=Michael+Luo%2C+Sijun+Tan%2C+Justin+Wong%2C+Xiaoxiang+Shi%2C+William+Y+Tang%2C+Manan+Roongta%2C+Colin+Cai%2C+Jeffrey+Luo%2C+Tianjun+Zhang%2C+Li+Erran+Li%2C+et+al%2E+2025%2E+Deepscaler%3A+Surpassing+o1%2Dpreview+with+a+1%2E5+b+model)
28. Hynek Kydlíček and Hugging Face. 2025. Math-verify. [https://github.com/huggingface/Math-Verif](https://github.com/huggingface/Math-Verif) y. [↗](https://github.com/huggingface/Math-Verif)
29. Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Ren Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, et al. 2024. [Rlaif vs](https://md.research.elamur.ai/free/en/papers/luo-2026-sparse-rl-breaking-memory-wall/). rlhf: Scaling reinforcement learning Sijia Luo, Xiaokang Zhang, Yuxuan Hu, Bohan Zhang, Ke Wang, Jinbo Su, Mengshu Sun, Lei Li…
30. MAA. 2024. American invitational mathematics examination - aime. [find](https://scholar.google.com/scholar?q=MAA%2E+2024%2E+American+invitational+mathematics+examination+%2D+aime%2E)
31. MAA. 2025. American invitational mathematics examination - aime. [find](https://scholar.google.com/scholar?q=MAA%2E+2025%2E+American+invitational+mathematics+examination+%2D+aime%2E)
32. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
33. Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. 2017. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pages 2778–2787. PMLR. [find](https://scholar.google.com/scholar?q=Deepak+Pathak%2C+Pulkit+Agrawal%2C+Alexei+A+Efros%2C+and+Trevor+Darrell%2E+2017%2E+Curiosity%2Ddriven+exploration+by+self%2Dsupervised+prediction%2E+In+International+conference+on+machine+learning%2C+pages+2778%E2%80%932787%2E+P)
34. Roberta Raileanu and Tim Rocktäschel. Ride: Rewarding impact-driven exploration for procedurally-generated environments. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Roberta+Raileanu+and+Tim+Rockt%C3%A4schel%2E+Ride%3A+Rewarding+impact%2Ddriven+exploration+for+procedurally%2Dgenerated+environments%2E+In+International+Conference+on+Learning+Representations%2E)
35. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
36. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, et al. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
37. Yuda Song, Julia Kempe, and Remi Munos. 2025. Outcome-based exploration for llm reasoning. arXiv preprint arXiv:2509.06941. [find](https://scholar.google.com/scholar?q=Yuda+Song%2C+Julia+Kempe%2C+and+Remi+Munos%2E+2025%2E+Outcome%2Dbased+exploration+for+llm+reasoning%2E+arXiv+preprint+arXiv%3A2509%2E06941%2E)
38. Richard S Sutton, Andrew G Barto, et al. 1998. Reinforcement learning: An introduction, volume 1. MIT press Cambridge. [find](https://scholar.google.com/scholar?q=Richard+S+Sutton%2C+Andrew+G+Barto%2C+et+al%2E+1998%2E+Reinforcement+learning%3A+An+introduction%2C+volume+1%2E+MIT+press+Cambridge%2E)
39. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805.
40. Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599. — [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/)
41. Marc Toussaint. 2009. Robot trajectory optimization using approximate inference. In Proceedings of the 26th annual international conference on machine learning, pages 1049–1056. [find](https://scholar.google.com/scholar?q=Marc+Toussaint%2E+2009%2E+Robot+trajectory+optimization+using+approximate+inference%2E+In+Proceedings+of+the+26th+annual+international+conference+on+machine+learning%2C+pages+1049%E2%80%931056%2E)
42. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
43. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. 2025. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). arXiv preprint arXiv:2503.14476.
44. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. 2025. [Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/) arXiv preprint arXiv:2504.13837.
45. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun MA, and Junxian He. 2025. [SimpleRL-zoo: Investigating and taming zero reinforcement learning for open base models in the wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/). In Second Conference on Language Modeling.
46. Kaiwen Zha, Zhengqi Gao, Maohao Shen, Zhang-Wei Hong, Duane S Boning, and Dina Katabi. 2025. Rl tango: Reinforcing generator and verifier together for language reasoning. arXiv preprint arXiv:2505.15034. — [arXiv:2505.15034](https://arxiv.org/abs/2505.15034) [find](https://scholar.google.com/scholar?q=Kaiwen+Zha%2C+Zhengqi+Gao%2C+Maohao+Shen%2C+Zhang%2DWei+Hong%2C+Duane+S+Boning%2C+and+Dina+Katabi%2E+2025%2E+Rl+tango%3A+Reinforcing+generator+and+verifier+together+for+language+reasoning%2E+arXiv+preprint+arXiv%3A2505%2E150)
47. Xiaoyun Zhang, Jingqing Ruan, Xing Ma, Yawen Zhu, Jiansong Chen, Ke Zeng, and Xunliang Cai. 2025a. A reasoner for real-world event detection: Scaling reinforcement learning via adaptive perplexity-aware sampling strategy. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language… [find](https://scholar.google.com/scholar?q=Xiaoyun+Zhang%2C+Jingqing+Ruan%2C+Xing+Ma%2C+Yawen+Zhu%2C+Jiansong+Chen%2C+Ke+Zeng%2C+and+Xunliang+Cai%2E+2025a%2E+A+reasoner+for+real%2Dworld+event+detection%3A+Scaling+reinforcement+learning+via+adaptive+perplexity%2Dawa)
48. Xiaoyun Zhang, Jingqing Ruan, Xing Ma, Yawen Zhu, Haodong Zhao, Hao Li, Jiansong Chen, Ke Zeng, and Xunliang Cai. 2025b. When to continue thinking: Adaptive thinking mode switching for efficient reasoning. arXiv preprint arXiv:2505.15400. [find](https://scholar.google.com/scholar?q=Xiaoyun+Zhang%2C+Jingqing+Ruan%2C+Xing+Ma%2C+Yawen+Zhu%2C+Haodong+Zhao%2C+Hao+Li%2C+Jiansong+Chen%2C+Ke+Zeng%2C+and+Xunliang+Cai%2E+2025b%2E+When+to+continue+thinking%3A+Adaptive+thinking+mode+switching+for+efficient+reaso)
49. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. 2025a. [Group sequence policy optimization](https://md.research.elamur.ai/free/en/papers/zheng-2025-group-sequence-policy-optimization/). arXiv preprint arXiv:2507.18071.
50. Tianyu Zheng, Tianshun Xing, Qingshui Gu, Taoran Liang, Xingwei Qu, Xin Zhou, Yizhi Li, Zhoufutu Wen, Chenghua Lin, Wenhao Huang, et al. 2025b. First return, entropy-eliciting explore. arXiv preprint arXiv:2507.07017. — [arXiv:2507.07017](https://arxiv.org/abs/2507.07017) [find](https://scholar.google.com/scholar?q=Tianyu+Zheng%2C+Tianshun+Xing%2C+Qingshui+Gu%2C+Taoran+Liang%2C+Xingwei+Qu%2C+Xin+Zhou%2C+Yizhi+Li%2C+Zhoufutu+Wen%2C+Chenghua+Lin%2C+Wenhao+Huang%2C+et+al%2E+2025b%2E+First+return%2C+entropy%2Deliciting+explore%2E+arXiv+preprint+)
51. Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey, et al. 2008. Maximum entropy inverse reinforcement learning. In Aaai, volume 8, pages 1433–1438. Chicago, IL, USA. [find](https://scholar.google.com/scholar?q=Brian+D+Ziebart%2C+Andrew+L+Maas%2C+J+Andrew+Bagnell%2C+Anind+K+Dey%2C+et+al%2E+2008%2E+Maximum+entropy+inverse+reinforcement+learning%2E+In+Aaai%2C+volume+8%2C+pages+1433%E2%80%931438%2E+Chicago%2C+IL%2C+USA%2E)
52. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, et al. 2025. [Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/). arXiv preprint arXiv:2506.01939.

*Updated 2 November 2026*