---
title: "Reinforced Efficient Reasoning via Semantically Diverse"
description: "RLVR methods such as GRPO assign one reward to a whole response. 36 references, cited by 9 papers here."
authors: ["Ziqi Zhao", "Zhaochun Ren", "Jiahong Zou", "Liu Yang", "Zhiwei Xu", "Xuri Ge", "Zhumin Chen", "Xinyu Ma", "Daiting Shi", "Shuaiqiang Wang", "Dawei Yin", "Xin Xin"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.2216/"
doi: "10.18653/v1/2026.acl-long.2216"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhao-2026-reinforced-efficient-reasoning-semantically-diverse/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhao-2026-reinforced-efficient-reasoning-semantically-diverse/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/) / Reinforced Efficient Reasoning via Semantically Diverse Exploration

# Reinforced Efficient Reasoning via Semantically Diverse Exploration

**Authors:** Ziqi Zhao, [Zhaochun Ren](https://md.research.elamur.ai/free/en/authors/zhaochun-ren/), Jiahong Zou, [Liu Yang](https://md.research.elamur.ai/free/en/authors/liu-yang/), Zhiwei Xu, Xuri Ge, [Zhumin Chen](https://md.research.elamur.ai/free/en/authors/zhumin-chen/), [Xinyu Ma](https://md.research.elamur.ai/free/en/authors/xinyu-ma/), [Daiting Shi](https://md.research.elamur.ai/free/en/authors/daiting-shi/), [Shuaiqiang Wang](https://md.research.elamur.ai/free/en/authors/shuaiqiang-wang/), [Dawei Yin](https://md.research.elamur.ai/free/en/authors/dawei-yin/) and Xin Xin · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.2216/) · [PDF](https://aclanthology.org/2026.acl-long.2216.pdf) · [DOI 10.18653/v1/2026.acl-long.2216](https://doi.org/10.18653/v1/2026.acl-long.2216) · [OpenAlex](https://openalex.org/W7166887741)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/zhao-2026-reinforced-efficient-reasoning-semantically-diverse/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
RLVR methods such as GRPO assign one reward to a whole response. MCTS-based variants refine this by building tree-structured rollouts, which allows credit to be assigned per segment. However, they usually choose branch points using token-level generation entropy. This often splits on near-synonyms (e.g., "can" vs. "need"), so the resulting paths stay semantically alike. These methods also do nothing to curb overthinking.

ROSE addresses both issues. It picks branching positions with a semantic-entropy score, which combines generation entropy with a probability-weighted divergence of token embeddings among the top candidates. An ε-exploration mechanism sometimes restarts a rollout from the root so the search does not stay too local. For efficiency, a length-aware segment-level advantage estimator gives more credit to short correct chains than to long ones from the same node. Only the rollout and advantage computation change, not the loss, so ROSE should work alongside other GRPO-style objectives.

On four math benchmarks with Qwen and Llama backbones, ROSE beats GRPO and Dr.GRPO in pass@8 while producing noticeably shorter reasoning.

## Key points
- **Semantic divergence:** computed over the top-20 most probable tokens at each position, using cosine similarity of the LLM's own token embeddings, weighted by token probabilities.
- **Setup:** benchmarks are AIME2024, AIME2025, AMC2023 and MATH500. Backbones are Llama-3.2-3B-Instruct, Qwen3-4B-Base and Qwen3-8B-Base, all up to 8B parameters.
- **ε-exploration ablation:** setting ε=1 turns ROSE into Dr.GRPO. ε=0.5 gave the best average pass@8 for every backbone:
  - Qwen3-8B-Base: 55.17 vs. 51.02 for Dr.GRPO
  - Qwen3-4B-Base: 51.87 vs. 47.15
  - Llama-3.2-3B-Instruct: 30.98 vs. 28.60
- **Length penalty α:** raising α shortens responses during both rollout and evaluation. α=1–2 gives the best accuracy/length trade-off, and even α=10 still beats the GRPO variants in pass@8.
- **Component ablations and availability:** removing semantic-entropy branching or the segment-level advantage both reduce performance. Code is published at github.com/ZiqiZhao1/ROSE-rl.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Baidu](https://md.research.elamur.ai/free/en/companies/baidu/) — Company, Beijing, China · 74 works · 15,263 citations
- [Leiden University](https://md.research.elamur.ai/free/en/universities/leiden-university/) — University, Leiden, Netherlands · 32 works · 1,074 citations
- [Shandong University](https://md.research.elamur.ai/free/en/universities/shandong-university/) — University, Jinan, China · 17 works · 789 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Ziqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang, Zhiwei Xu, Xuri Ge, Zhumin Chen, Xinyu Ma, Daiting Shi, Shuaiqiang Wang, Dawei Yin and Xin Xin. 2026. Reinforced Efficient Reasoning via Semantically Diverse Exploration. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhao2026reinforced,
  title = {{Reinforced Efficient Reasoning via Semantically Diverse Exploration}},
  author = {Ziqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang, Zhiwei Xu, Xuri Ge, Zhumin Chen, Xinyu Ma, Daiting Shi, Shuaiqiang Wang, Dawei Yin and Xin Xin},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2216},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2216}
}
```

- [← previous Overview of the MEDIQA-SYNUR 2026 Shared Task on Observation Extraction from Nurse Dictations](https://md.research.elamur.ai/free/en/papers/michalopoulos-2026-overview-mediqa-synur-2026-shared/)
- [next → RubricHub: A Comprehensive and Highly Discriminative Rubric Dataset via Automated Coarse-to-Fine Generation](https://md.research.elamur.ai/free/en/papers/li-2026-rubrichub-comprehensive-highly-discriminative-rubric/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/)

## References

36 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Daman Arora and Andrea Zanette. 2025. Training language models to reason efficiently. arXiv preprint arXiv:2502.04463. [find](https://scholar.google.com/scholar?q=Daman+Arora+and+Andrea+Zanette%2E+2025%2E+Training+language+models+to+reason+efficiently%2E+arXiv+preprint+arXiv%3A2502%2E04463%2E)
2. Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. 2013. The arcade learning environment: An evaluation platform for general agents. Journal of artificial intelligence research, 47:253– 279. [find](https://scholar.google.com/scholar?q=Marc+G+Bellemare%2C+Yavar+Naddaf%2C+Joel+Veness%2C+and+Michael+Bowling%2E+2013%2E+The+arcade+learning+environment%3A+An+evaluation+platform+for+general+agents%2E+Journal+of+artificial+intelligence+research%2C+47%3A253%E2%80%93)
3. Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, and 1 others. 2024. [Do not think that much for 2+ 3=?](https://md.research.elamur.ai/free/en/papers/chen-2024-think-that-much-2-3/) on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187.
4. Xiangxiang Chu, Hailang Huang, Xiao Zhang, Fei Wei, and Yong Wang. 2025. Gpg: A simple and strong reinforcement learning baseline for model reasoning. arXiv preprint arXiv:2504.02546. — [arXiv:2504.02546](https://arxiv.org/abs/2504.02546) [find](https://scholar.google.com/scholar?q=Xiangxiang+Chu%2C+Hailang+Huang%2C+Xiao+Zhang%2C+Fei+Wei%2C+and+Yong+Wang%2E+2025%2E+Gpg%3A+A+simple+and+strong+reinforcement+learning+baseline+for+model+reasoning%2E+arXiv+preprint+arXiv%3A2504%2E02546%2E+%E2%80%94+%5BarXiv%3A2504%2E02)
5. Muzhi Dai, Chenxu Yang, and Qingyi Si. 2025. S-grpo: Early exit via reinforcement learning in reasoning models. arXiv preprint arXiv:2505.07686. — [arXiv:2505.07686](https://arxiv.org/abs/2505.07686) [find](https://scholar.google.com/scholar?q=Muzhi+Dai%2C+Chenxu+Yang%2C+and+Qingyi+Si%2E+2025%2E+S%2Dgrpo%3A+Early+exit+via+reinforcement+learning+in+reasoning+models%2E+arXiv+preprint+arXiv%3A2505%2E07686%2E+%E2%80%94+%5BarXiv%3A2505%2E07686%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2505%2E07686%29)
6. Guanting Dong, Hangyu Mao, Kai Ma, Licheng Bao, Yifei Chen, Zhongyuan Wang, Zhongxia Chen, Jiazhen Du, Huiyang Wang, Fuzheng Zhang, and 1 others. 2025. Agentic reinforced policy optimization. arXiv preprint arXiv:2507.19849. — [arXiv:2507.19849](https://arxiv.org/abs/2507.19849) [find](https://scholar.google.com/scholar?q=Guanting+Dong%2C+Hangyu+Mao%2C+Kai+Ma%2C+Licheng+Bao%2C+Yifei+Chen%2C+Zhongyuan+Wang%2C+Zhongxia+Chen%2C+Jiazhen+Du%2C+Huiyang+Wang%2C+Fuzheng+Zhang%2C+and+1+others%2E+2025%2E+Agentic+reinforced+policy+optimization%2E+arXiv+pr)
7. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
8. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv:2501.12948. arXiv preprint Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel,…
9. Zhezheng Hao, Hong Wang, Haoyang Liu, Jian Luo, Jiarui Yu, Hande Dong, Qiang Lin, Can Wang, and Jiawei Chen. 2025. [Rethinking entropy interventions in rlvr: An entropy change perspective](https://md.research.elamur.ai/free/en/papers/hao-2026-rethinking-entropy-interventions-rlvr-entropy/). arXiv preprint arXiv:2510.10150.
10. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). NeurIPS.
11. Yuxiang Ji, Ziyu Ma, Yong Wang, Guanhua Chen, Xiangxiang Chu, and Liaoni Wu. 2025. Tree search for llm agent reinforcement learning. arXiv preprint arXiv:2509.21240. — [arXiv:2509.21240](https://arxiv.org/abs/2509.21240) [find](https://scholar.google.com/scholar?q=Yuxiang+Ji%2C+Ziyu+Ma%2C+Yong+Wang%2C+Guanhua+Chen%2C+Xiangxiang+Chu%2C+and+Liaoni+Wu%2E+2025%2E+Tree+search+for+llm+agent+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2509%2E21240%2E+%E2%80%94+%5BarXiv%3A2509%2E21240%5D%28https%3A%2F%2Farxiv)
12. Levente Kocsis and Csaba Szepesvári. 2006. Bandit based monte-carlo planning. In European conference on machine learning, pages 282–293. Springer. [find](https://scholar.google.com/scholar?q=Levente+Kocsis+and+Csaba+Szepesv%C3%A1ri%2E+2006%2E+Bandit+based+monte%2Dcarlo+planning%2E+In+European+conference+on+machine+learning%2C+pages+282%E2%80%93293%2E+Springer%2E)
13. Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Ren Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and 1 others. 2024. Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback. In International Conference on Machine Learn… [find](https://scholar.google.com/scholar?q=Harrison+Lee%2C+Samrat+Phatale%2C+Hassan+Mansoor%2C+Thomas+Mesnard%2C+Johan+Ferret%2C+Kellie+Ren+Lu%2C+Colton+Bishop%2C+Ethan+Hall%2C+Victor+Carbune%2C+Abhinav+Rastogi%2C+and+1+others%2E+2024%2E+Rlaif+vs%2E+rlhf%3A+Scaling+reinf)
14. Yizhi Li, Qingshui Gu, Zhoufutu Wen, Ziniu Li, Tianshun Xing, Shuyue Guo, Tianyu Zheng, Xin Zhou, Xingwei Qu, Wangchunshu Zhou, and 1 others. 2025. Treepo: Bridging the gap of policy optimization and efficacy and inference efficiency with heuristic tree-based modeling. arXiv preprint arXiv:2508.1744… [find](https://scholar.google.com/scholar?q=Yizhi+Li%2C+Qingshui+Gu%2C+Zhoufutu+Wen%2C+Ziniu+Li%2C+Tianshun+Xing%2C+Shuyue+Guo%2C+Tianyu+Zheng%2C+Xin+Zhou%2C+Xingwei+Qu%2C+Wangchunshu+Zhou%2C+and+1+others%2E+2025%2E+Treepo%3A+Bridging+the+gap+of+policy+optimization+and+)
15. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/). arXiv preprint arXiv:2503.20783.
16. Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, and 1 others. 2025. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. CoRR. [find](https://scholar.google.com/scholar?q=Fanqing+Meng%2C+Lingxiao+Du%2C+Zongkai+Liu%2C+Zhixiang+Zhou%2C+Quanfeng+Lu%2C+Daocheng+Fu%2C+Botian+Shi%2C+Wenhai+Wang%2C+Junjun+He%2C+Kaipeng+Zhang%2C+and+1+others%2E+2025%2E+Mm%2Deureka%3A+Exploring+visual+aha+moment+with+rule)
17. Shuaiyi Nie, Siyu Ding, Wenyuan Zhang, Linhao Yu, Tianmeng Yang, Yao Chen, Tingwen Liu, Weichong Yin, Yu Sun, and Hua Wu. 2026. [Attnpo: Attention-guided process supervision for efficient reasoning](https://md.research.elamur.ai/free/en/papers/nie-2026-attnpo-attention-guided-process-supervision/). arXiv preprint arXiv:2602.09953.
18. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
19. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
20. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
21. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025a. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
22. Zhicheng Yang, Zhijiang Guo, Yinya Huang, Xiaodan Liang, Yiwei Wang, and Jing Tang. 2025b. Treerpo: Tree relative policy optimization. arXiv preprint arXiv:2506.05183. — [arXiv:2506.05183](https://arxiv.org/abs/2506.05183) [find](https://scholar.google.com/scholar?q=Zhicheng+Yang%2C+Zhijiang+Guo%2C+Yinya+Huang%2C+Xiaodan+Liang%2C+Yiwei+Wang%2C+and+Jing+Tang%2E+2025b%2E+Treerpo%3A+Tree+relative+policy+optimization%2E+arXiv+preprint+arXiv%3A2506%2E05183%2E+%E2%80%94+%5BarXiv%3A2506%2E05183%5D%28https%3A%2F%2Farx)
23. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). arXiv preprint arXiv:2503.14476.
24. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297.
25. Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, and 1 others. 2025. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks. arXiv preprint arXiv:2504.05118. [find](https://scholar.google.com/scholar?q=Yu+Yue%2C+Yufeng+Yuan%2C+Qiying+Yu%2C+Xiaochen+Zuo%2C+Ruofei+Zhu%2C+Wenyuan+Xu%2C+Jiaze+Chen%2C+Chengyi+Wang%2C+TianTian+Fan%2C+Zhengyin+Du%2C+and+1+others%2E+2025%2E+Vapo%3A+Efficient+and+reliable+reinforcement+learning+for+a)
26. Richard S Sutton, Andrew G Barto, and 1 others. 1998. Reinforcement learning: An introduction, volume 1. MIT press Cambridge. [find](https://scholar.google.com/scholar?q=Richard+S+Sutton%2C+Andrew+G+Barto%2C+and+1+others%2E+1998%2E+Reinforcement+learning%3A+An+introduction%2C+volume+1%2E+MIT+press+Cambridge%2E)
27. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, and 1 others. 2025a. [Group sequence policy optimization](https://md.research.elamur.ai/free/en/papers/zheng-2025-group-sequence-policy-optimization/). arXiv preprint arXiv:2507.18071.
28. Maciej Świechowski, Konrad Godlewski, Bartosz Sawicki, and Jacek Mańdziuk. 2023. Monte carlo tree search: A review of recent modifications and applications. Artificial Intelligence Review, 56(3):2497– 2562. [find](https://scholar.google.com/scholar?q=Maciej+S%CC%81wiechowski%2C+Konrad+Godlewski%2C+Bartosz+Sawicki%2C+and+Jacek+Man%CC%81dziuk%2E+2023%2E+Monte+carlo+tree+search%3A+A+review+of+recent+modifications+and+applications%2E+Artificial+Intelligence+Review%2C+56%283%29%3A249)
29. Xu Wang, Sen Wang, Xingxing Liang, Dawei Zhao, Jincai Huang, Xin Xu, Bin Dai, and Qiguang Miao. 2022. Deep reinforcement learning: A survey. IEEE Transactions on Neural Networks and Learning Systems, 35(4):5064–5078. [find](https://scholar.google.com/scholar?q=Xu+Wang%2C+Sen+Wang%2C+Xingxing+Liang%2C+Dawei+Zhao%2C+Jincai+Huang%2C+Xin+Xu%2C+Bin+Dai%2C+and+Qiguang+Miao%2E+2022%2E+Deep+reinforcement+learning%3A+A+survey%2E+IEEE+Transactions+on+Neural+Networks+and+Learning+Systems%2C+)
30. Fei Wu, Zhenrong Zhang, Qikai Chang, Jianshu Zhang, Quan Liu, and Jun Du. 2026. [Step potential advantage estimation: Harnessing intermediate confidence and correctness for efficient mathematical reasoning](https://md.research.elamur.ai/free/en/papers/wu-2026-step-potential-advantage-estimation-harnessing/). arXiv preprint arXiv:2601.03823.
31. Tianyu Zheng, Tianshun Xing, Qingshui Gu, Taoran Liang, Xingwei Qu, Xin Zhou, Yizhi Li, Zhoufutu Wen, Chenghua Lin, Wenhao Huang, and 1 others. 2025b. First return, entropy-eliciting explore. arXiv preprint arXiv:2507.07017. — [arXiv:2507.07017](https://arxiv.org/abs/2507.07017) [find](https://scholar.google.com/scholar?q=Tianyu+Zheng%2C+Tianshun+Xing%2C+Qingshui+Gu%2C+Taoran+Liang%2C+Xingwei+Qu%2C+Xin+Zhou%2C+Yizhi+Li%2C+Zhoufutu+Wen%2C+Chenghua+Lin%2C+Wenhao+Huang%2C+and+1+others%2E+2025b%2E+First+return%2C+entropy%2Deliciting+explore%2E+arXiv+pr)
32. Yixiao Zhou, Yang Li, Dongzhou Cheng, Hehe Fan, and Yu Cheng. 2026. Look inward to explore outward: Learning temperature policy from llm internal states via hierarchical rl. arXiv preprint arXiv:2602.13035. — [arXiv:2602.13035](https://arxiv.org/abs/2602.13035) [find](https://scholar.google.com/scholar?q=Yixiao+Zhou%2C+Yang+Li%2C+Dongzhou+Cheng%2C+Hehe+Fan%2C+and+Yu+Cheng%2E+2026%2E+Look+inward+to+explore+outward%3A+Learning+temperature+policy+from+llm+internal+states+via+hierarchical+rl%2E+arXiv+preprint+arXiv%3A2602%2E)
33. Xinyu Zhu, Mengzhou Xia, Zhepei Wei, Wei-Lin Chen, Danqi Chen, and Yu Meng. 2025. The surprising effectiveness of negative reinforcement in llm reasoning. arXiv preprint arXiv:2506.01347. [find](https://scholar.google.com/scholar?q=Xinyu+Zhu%2C+Mengzhou+Xia%2C+Zhepei+Wei%2C+Wei%2DLin+Chen%2C+Danqi+Chen%2C+and+Yu+Meng%2E+2025%2E+The+surprising+effectiveness+of+negative+reinforcement+in+llm+reasoning%2E+arXiv+preprint+arXiv%3A2506%2E01347%2E)
34. Mingqi Wu, Zhihao Zhang, Qiaole Dong, Zhiheng Xi, Jun Zhao, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Huijie Lv, Ming Zhang, and 1 others. 2025a. Reasoning or memorization? unreliable results of reinforcement learning due to data contamination. arXiv preprint arXiv:2507.10532. — [arXiv:2507.10532](https://arxiv.org/abs/2507.10532) [find](https://scholar.google.com/scholar?q=Mingqi+Wu%2C+Zhihao+Zhang%2C+Qiaole+Dong%2C+Zhiheng+Xi%2C+Jun+Zhao%2C+Senjie+Jin%2C+Xiaoran+Fan%2C+Yuhao+Zhou%2C+Huijie+Lv%2C+Ming+Zhang%2C+and+1+others%2E+2025a%2E+Reasoning+or+memorization%3F+unreliable+results+of+reinforcem)
35. Zhenhe Wu, Jian Yang, Jiaheng Liu, Xianjie Wu, Changzai Pan, Jie Zhang, Yu Zhao, Shuangyong Song, Yongxiang Li, and Zhoujun Li. 2025b. Tabler1: Region-based reinforcement learning for table understanding. arXiv preprint arXiv:2505.12415. — [arXiv:2505.12415](https://arxiv.org/abs/2505.12415) [find](https://scholar.google.com/scholar?q=Zhenhe+Wu%2C+Jian+Yang%2C+Jiaheng+Liu%2C+Xianjie+Wu%2C+Changzai+Pan%2C+Jie+Zhang%2C+Yu+Zhao%2C+Shuangyong+Song%2C+Yongxiang+Li%2C+and+Zhoujun+Li%2E+2025b%2E+Tabler1%3A+Region%2Dbased+reinforcement+learning+for+table+understand)
36. Can Xie, Ruotong Pan, Xiangyu Wu, Yunfei Zhang, Jiayi Fu, Tingting Gao, and Guorui Zhou. 2025. [Unlocking exploration in rlvr: Uncertainty-aware advantage shaping for deeper reasoning](https://md.research.elamur.ai/free/en/papers/xie-2026-unlocking-exploration-rlvr-uncertainty-aware/). arXiv preprint arXiv:2510.10649.

*Updated 31 October 2026*