---
title: "Step Potential Advantage Estimation — paper by Wu et al. (2026)"
description: "SPAE builds on a DAPO plus Reinforce++-Baseline backbone. 43 references, cited by 11 papers here."
authors: ["Fei Wu", "Zhenrong Zhang", "Qikai Chang", "Jianshu Zhang", "Quan Liu", "Jun Du"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.317/"
doi: "10.18653/v1/2026.findings-acl.317"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2026-step-potential-advantage-estimation-harnessing/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2026-step-potential-advantage-estimation-harnessing/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/) / Step Potential Advantage Estimation: Harnessing Intermediate Confidence and Correctness for Efficient Mathematical Reasoning

# Step Potential Advantage Estimation: Harnessing Intermediate Confidence and Correctness for Efficient Mathematical Reasoning

**Authors:** [Fei Wu](https://md.research.elamur.ai/free/en/authors/fei-qing-wu/), Zhenrong Zhang, Qikai Chang, [Jianshu Zhang](https://md.research.elamur.ai/free/en/authors/jianshu-zhang/), Quan Liu and Jun Du · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.317/) · [PDF](https://aclanthology.org/2026.findings-acl.317.pdf) · [DOI 10.18653/v1/2026.findings-acl.317](https://doi.org/10.18653/v1/2026.findings-acl.317) · [OpenAlex](https://openalex.org/W7166812255)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [11](https://md.research.elamur.ai/free/en/papers/wu-2026-step-potential-advantage-estimation-harnessing/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Reinforcement learning with verifiable rewards (RLVR) trains long chain-of-thought models using only a final correct/incorrect signal. That signal says nothing about which reasoning steps actually helped. Models therefore keep verifying after they have already solved a problem ("Over-Checking"), and they sometimes revise a correct answer into a wrong one, which the authors call Right-to-Wrong (R2W) failure.

To make step-level progress visible, the authors split each trajectory into steps and add a probe after each one. The probe forces the model to state a provisional boxed answer. From several short probe samples they measure confidence and correctness, then merge the two into a "Step Potential" score.

SPAE builds on a DAPO plus Reinforce++-Baseline backbone. It rewards steps that raise potential, punishes steps that lower it, and penalizes tokens generated once potential has saturated.

On math benchmarks and GPQA, using Qwen and Llama models, SPAE improves accuracy while cutting response length by about a quarter. It beats RL, efficient-reasoning, and token-level advantage baselines, showing that dense step-level credit can come without any learned reward model.

## Key points
- **Probe design:** the trigger "**Final Answer** \n\\boxed{" is inserted at each ".\n\n" step boundary, and every probe is allowed to generate no more than 10 tokens. The probe adds no learnable parameters, and its extra cost is largely offset by shorter policy outputs.
- **Main result on DeepSeek-R1-Distill-Qwen-7B:**

  | Benchmark | Length reduction | Accuracy gain |
  |---|---|---|
  | AIME2024 | 25.1% | +6.7 |
  | AIME2025 | 25.3% | +3.3 |
  | GPQA | 24.4% | +1.1 |

- **Reasoning behavior on AIME 2024/2025:**
  - Post-solution checking drops from 1,511 tokens (Base) to 614.
  - The R2W rate falls from 8.10% to 2.65%.
  - Accuracy reaches 51.05%, the best among compared methods, including DAPO, RF-B, DAST and LC-R1.
- **Ablations:**
  - Full SPAE reaches 63.86% average accuracy.
  - Dropping confidence from the potential gives 62.43%.
  - Removing difference shaping (ξ=0) gives 62.47%.
  - Removing the saturation penalty (α=0) leaves accuracy about the same but lengthens responses from 6,825 to 7,517 tokens.
  - The best setting is ξ=α=0.5.
- **Limitations:** the method relies on structured long-CoT output and a delimiter heuristic, and it has only been tested on mathematical reasoning. Code is available at github.com/cii030/SPAE-RL.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 108 works · 112,698 citations
- [IFlyTek](https://md.research.elamur.ai/free/en/companies/iflytek/) — Company, Hefei, China · 10 works · 2,810 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Fei Wu, Zhenrong Zhang, Qikai Chang, Jianshu Zhang, Quan Liu and Jun Du. 2026. Step Potential Advantage Estimation: Harnessing Intermediate Confidence and Correctness for Efficient Mathematical Reasoning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wu2026step,
  title = {{Step Potential Advantage Estimation: Harnessing Intermediate Confidence and Correctness for Efficient Mathematical Reasoning}},
  author = {Fei Wu, Zhenrong Zhang, Qikai Chang, Jianshu Zhang, Quan Liu and Jun Du},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.317},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.317}
}
```

- [← previous SPARK: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learning](https://md.research.elamur.ai/free/en/papers/wu-2026-spark-strategic-policy-aware-exploration/)
- [next → Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/zhang-2026-think-natively-unlocking-multilingual-reasoning/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/)

## References

43 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Pranjal Aggarwal and Sean Welleck. 2025. [L1: Controlling how long a reasoning model thinks with reinforcement learning](https://md.research.elamur.ai/free/en/papers/aggarwal-2025-l1-controlling-long-reasoning-model/). In Second Conference on Language Modeling.
2. Ding Chen, Qingchen Yu, Pengyuan Wang, Wentao Zhang, Bo Tang, Feiyu Xiong, Xinchi Li, Minchuan Yang, and Zhiyu Li. 2025a. xverify: Efficient answer verifier for reasoning model evaluations. Preprint, arXiv:2504.10481. — [arXiv:2504.10481](https://arxiv.org/abs/2504.10481) [find](https://scholar.google.com/scholar?q=Ding+Chen%2C+Qingchen+Yu%2C+Pengyuan+Wang%2C+Wentao+Zhang%2C+Bo+Tang%2C+Feiyu+Xiong%2C+Xinchi+Li%2C+Minchuan+Yang%2C+and+Zhiyu+Li%2E+2025a%2E+xverify%3A+Efficient+answer+verifier+for+reasoning+model+evaluations%2E+Preprint%2C+)
3. Minghan Chen, Guikun Chen, Wenguan Wang, and Yi Yang. 2025b. Seed-grpo: Semantic entropy enhanced grpo for uncertainty-aware policy optimization. Preprint, arXiv:2505.12346. — [arXiv:2505.12346](https://arxiv.org/abs/2505.12346) [find](https://scholar.google.com/scholar?q=Minghan+Chen%2C+Guikun+Chen%2C+Wenguan+Wang%2C+and+Yi+Yang%2E+2025b%2E+Seed%2Dgrpo%3A+Semantic+entropy+enhanced+grpo+for+uncertainty%2Daware+policy+optimization%2E+Preprint%2C+arXiv%3A2505%2E12346%2E+%E2%80%94+%5BarXiv%3A2505%2E12346%5D%28https)
4. Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. 2025a. Reasoning with exploration: An entropy perspective. Preprint, arXiv:2506.14758. [find](https://scholar.google.com/scholar?q=Daixuan+Cheng%2C+Shaohan+Huang%2C+Xuekai+Zhu%2C+Bo+Dai%2C+Wayne+Xin+Zhao%2C+Zhenliang+Zhang%2C+and+Furu+Wei%2E+2025a%2E+Reasoning+with+exploration%3A+An+entropy+perspective%2E+Preprint%2C+arXiv%3A2506%2E14758%2E)
5. Zhengxiang Cheng, Dongping Chen, Mingyang Fu, and Tianyi Zhou. 2025b. [Optimizing length compression in large reasoning models](https://md.research.elamur.ai/free/en/papers/cheng-2026-optimizing-length-compression-large-reasoning/). Preprint, arXiv:2506.14755.
6. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseekr1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). Preprint, arXiv:2501.12948.
7. Yu Guo, Shenghao Ye, Shuangwu Chen, Zijian Wen, Tao Zhang, Qirui Bai, Dong Jin, Yunpeng Hou, Huasen He, Jian Yang, and Xiaobin Tan. 2026. Rethinking table pruning in tableqa: From sequential revisions to gold trajectory-supervised parallel search. Preprint, arXiv:2601.03851. [find](https://scholar.google.com/scholar?q=Yu+Guo%2C+Shenghao+Ye%2C+Shuangwu+Chen%2C+Zijian+Wen%2C+Tao+Zhang%2C+Qirui+Bai%2C+Dong+Jin%2C+Yunpeng+Hou%2C+Huasen+He%2C+Jian+Yang%2C+and+Xiaobin+Tan%2E+2026%2E+Rethinking+table+pruning+in+tableqa%3A+From+sequential+revisions)
8. Zhezheng Hao, Hong Wang, Haoyang Liu, Jian Luo, Jiarui Yu, Hande Dong, Qiang Lin, Can Wang, and Jiawei Chen. 2025. [Rethinking entropy interventions in rlvr: An entropy change perspective](https://md.research.elamur.ai/free/en/papers/hao-2026-rethinking-entropy-interventions-rlvr-entropy/). Preprint, arXiv:2510.10150.
9. Chenwei Lou, Zewei Sun, Xinnian Liang, Meng Qu, Wei Shen, Wenqi Wang, Yuntao Li, Qingping Yang, and Shuangzhi Wu. 2025. Adacot: Pareto-optimal adaptive chain-of-thought triggering via reinforcement learning. Preprint, arXiv:2505.11896. — [arXiv:2505.11896](https://arxiv.org/abs/2505.11896) [find](https://scholar.google.com/scholar?q=Chenwei+Lou%2C+Zewei+Sun%2C+Xinnian+Liang%2C+Meng+Qu%2C+Wei+Shen%2C+Wenqi+Wang%2C+Yuntao+Li%2C+Qingping+Yang%2C+and+Shuangzhi+Wu%2E+2025%2E+Adacot%3A+Pareto%2Doptimal+adaptive+chain%2Dof%2Dthought+triggering+via+reinforcement+le)
10. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. 2024. [OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). In Proce…
11. Shichao Ma, Zhiyuan Ma, Ming Yang, Xiaofan Li, Xing Wu, Jintao Du, Yu Cheng, Weiqiang Wang, Qiliang Liu, Zhengyang Zhou, and Yang Wang. 2026. [Tspo: Breaking the double homogenization dilemma in multi-turn search policy optimization](https://md.research.elamur.ai/free/en/papers/ma-2026-tspo-breaking-double-homogenization-dilemma/). Preprint, arXiv:2601.22776.
12. Jian Hu, Jason Klein Liu, Haotian Xu, and Wei Shen. 2025. Reinforce++: Stabilizing critic-free policy optimization with global advantage normalization. Preprint, arXiv:2501.03262. [find](https://scholar.google.com/scholar?q=Jian+Hu%2C+Jason+Klein+Liu%2C+Haotian+Xu%2C+and+Wei+Shen%2E+2025%2E+Reinforce%2B%2B%3A+Stabilizing+critic%2Dfree+policy+optimization+with+global+advantage+normalization%2E+Preprint%2C+arXiv%3A2501%2E03262%2E)
13. Thanh-Long V. Le, Myeongho Jeon, Kim Vu, Viet Lai, and Eunho Yang. 2025. No prompt left behind: Exploiting zero-variance prompts in llm reinforcement learning via entropy-guided advantage shaping. Preprint, arXiv:2509.21880. — [arXiv:2509.21880](https://arxiv.org/abs/2509.21880) [find](https://scholar.google.com/scholar?q=Thanh%2DLong+V%2E+Le%2C+Myeongho+Jeon%2C+Kim+Vu%2C+Viet+Lai%2C+and+Eunho+Yang%2E+2025%2E+No+prompt+left+behind%3A+Exploiting+zero%2Dvariance+prompts+in+llm+reinforcement+learning+via+entropy%2Dguided+advantage+shaping%2E+Pre)
14. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. 2022. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). In Advances in Ne…
15. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhiwei Li, Bao-Long Bi, Ling-Rui Mei, Junfeng Fang, Xiao Liang, Zhijiang Guo, and 2 others. 2025. [From system 1 to system 2:…](https://md.research.elamur.ai/free/en/papers/li-2025-system-1-system-2-survey/)
16. Ziniu Li, Tian Xu, Yushun Zhang, Zhihang Lin, Yang Yu, Ruoyu Sun, and Zhi-Quan Luo. 2024. Remax: A simple, effective, and efficient reinforcement learning method for aligning large language models. In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Ziniu+Li%2C+Tian+Xu%2C+Yushun+Zhang%2C+Zhihang+Lin%2C+Yang+Yu%2C+Ruoyu+Sun%2C+and+Zhi%2DQuan+Luo%2E+2024%2E+Remax%3A+A+simple%2C+effective%2C+and+efficient+reinforcement+learning+method+for+aligning+large+language+models%2E+In)
17. Zheng Liu, Mengjie Liu, Siwei Wen, Mengzhang Cai, Bin Cui, Conghui He, and Wentao Zhang. 2025a. From uniform to heterogeneous: Tailoring policy optimization to every token’s nature. Preprint, arXiv:2509.16591. — [arXiv:2509.16591](https://arxiv.org/abs/2509.16591) [find](https://scholar.google.com/scholar?q=Zheng+Liu%2C+Mengjie+Liu%2C+Siwei+Wen%2C+Mengzhang+Cai%2C+Bin+Cui%2C+Conghui+He%2C+and+Wentao+Zhang%2E+2025a%2E+From+uniform+to+heterogeneous%3A+Tailoring+policy+optimization+to+every+token%E2%80%99s+nature%2E+Preprint%2C+arXiv%3A25)
18. Zihe Liu, Jiashun Liu, Yancheng He, Weixun Wang, Jiaheng Liu, Ling Pan, Xinyu Hu, Shaopan Xiong, Ju Huang, Jian Hu, Shengyi Huang, Johan Obando-Ceron, Siran Yang, Jiamang Wang, Wenbo Su, and Bo Zheng. 2025b. Part i: Tricks or traps? a deep dive into rl for llm reasoning. Preprint, arXiv:2508.08221. [find](https://scholar.google.com/scholar?q=Zihe+Liu%2C+Jiashun+Liu%2C+Yancheng+He%2C+Weixun+Wang%2C+Jiaheng+Liu%2C+Ling+Pan%2C+Xinyu+Hu%2C+Shaopan+Xiong%2C+Ju+Huang%2C+Jian+Hu%2C+Shengyi+Huang%2C+Johan+Obando%2DCeron%2C+Siran+Yang%2C+Jiamang+Wang%2C+Wenbo+Su%2C+and+Bo+Zheng%2E)
19. Xinji Mai, Haotian Xu, Zhong-Zhi Li, Xing W, Weinong Wang, Jian Hu, Yingying Zhang, and Wenqiang Zhang. 2025. Agent rl scaling law: Agent rl with spontaneous code execution for mathematical problem solving. Preprint, arXiv:2505.07773. — [arXiv:2505.07773](https://arxiv.org/abs/2505.07773) [find](https://scholar.google.com/scholar?q=Xinji+Mai%2C+Haotian+Xu%2C+Zhong%2DZhi+Li%2C+Xing+W%2C+Weinong+Wang%2C+Jian+Hu%2C+Yingying+Zhang%2C+and+Wenqiang+Zhang%2E+2025%2E+Agent+rl+scaling+law%3A+Agent+rl+with+spontaneous+code+execution+for+mathematical+problem+so)
20. A. Ng, Daishi Harada, and Stuart J. Russell. 1999. Policy invariance under reward transformations: Theory and application to reward shaping. In International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=A%2E+Ng%2C+Daishi+Harada%2C+and+Stuart+J%2E+Russell%2E+1999%2E+Policy+invariance+under+reward+transformations%3A+Theory+and+application+to+reward+shaping%2E+In+International+Conference+on+Machine+Learning%2E)
21. Shuaiyi Nie, Siyu Ding, Wenyuan Zhang, Linhao Yu, Tianmeng Yang, Yao Chen, Tingwen Liu, Weichong Yin, Yu Sun, and Hua Wu. 2026. [Attnpo: Attention-guided process supervision for efficient reasoning](https://md.research.elamur.ai/free/en/papers/nie-2026-attnpo-attention-guided-process-supervision/). Preprint, arXiv:2602.09953.
22. OpenAI. 2024. Learning to reason with llms. [https://openai.com/index/](https://openai.com/index/) Ac-learning-to-reason-with-llms/. cessed [2025-12-25]. [↗](https://openai.com/index/)
23. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2024. [GPQA: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). In First Conference on Language Modeling.
24. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. Preprint, arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+Preprint%2C+arXiv%3A1707%2E06347%2E)
25. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). Preprint, arXiv:2402.03300.
26. Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu, and Shiguo Lian. 2025. DAST: Difficulty-adaptive slow-thinking for large reasoning models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Indust… [find](https://scholar.google.com/scholar?q=Yi+Shen%2C+Jian+Zhang%2C+Jieyun+Huang%2C+Shuming+Shi%2C+Wenjing+Zhang%2C+Jiangze+Yan%2C+Ning+Wang%2C+Kai+Wang%2C+Zhaoxiang+Liu%2C+and+Shiguo+Lian%2E+2025%2E+DAST%3A+Difficulty%2Dadaptive+slow%2Dthinking+for+large+reasoning+model)
27. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297.
28. Wei Sun, Wen Yang, Pu Jian, Qianlong Du, Fuwei Cui, Shuo Ren, and Jiajun Zhang. 2025. KTAE: A model-free algorithm to key-tokens advantage estimation in mathematical reasoning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Wei+Sun%2C+Wen+Yang%2C+Pu+Jian%2C+Qianlong+Du%2C+Fuwei+Cui%2C+Shuo+Ren%2C+and+Jiajun+Zhang%2E+2025%2E+KTAE%3A+A+model%2Dfree+algorithm+to+key%2Dtokens+advantage+estimation+in+mathematical+reasoning%2E+In+The+Thirty%2Dninth+Ann)
29. Haozhe Wang, Qixin Xu, Che Liu, Junhong Wu, Fangzhen Lin, and Wenhu Chen. 2025a. Emergent hierarchical reasoning in llms through reinforcement learning. Preprint, arXiv:2509.03646. — [arXiv:2509.03646](https://arxiv.org/abs/2509.03646) [find](https://scholar.google.com/scholar?q=Haozhe+Wang%2C+Qixin+Xu%2C+Che+Liu%2C+Junhong+Wu%2C+Fangzhen+Lin%2C+and+Wenhu+Chen%2E+2025a%2E+Emergent+hierarchical+reasoning+in+llms+through+reinforcement+learning%2E+Preprint%2C+arXiv%3A2509%2E03646%2E+%E2%80%94+%5BarXiv%3A2509%2E03646)
30. Jiawei Wang, Jiacai Liu, Yuqian Fu, Yingru Li, Xintao Wang, Yuan Lin, Yu Yue, Lin Zhang, Yang Wang, and Ke Wang. 2025b. Harnessing uncertainty: Entropy-modulated policy gradients for long-horizon llm agents. Preprint, arXiv:2509.09265. — [arXiv:2509.09265](https://arxiv.org/abs/2509.09265) [find](https://scholar.google.com/scholar?q=Jiawei+Wang%2C+Jiacai+Liu%2C+Yuqian+Fu%2C+Yingru+Li%2C+Xintao+Wang%2C+Yuan+Lin%2C+Yu+Yue%2C+Lin+Zhang%2C+Yang+Wang%2C+and+Ke+Wang%2E+2025b%2E+Harnessing+uncertainty%3A+Entropy%2Dmodulated+policy+gradients+for+long%2Dhorizon+llm+)
31. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. 2025c. [Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcem…](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/)
32. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, YuYue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, LingJun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, and 17 others. 2025. [DAPO: An open-source LLM reinforcement learning sy…](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/)
33. Jiajie Zhang, Nianyi Lin, Lei Hou, Ling Feng, and Juanzi Li. 2025. [AdaptThink: Reasoning models can learn when to think](https://md.research.elamur.ai/free/en/papers/zhang-2025-adaptthink-reasoning-models-can-learn/). In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 3716–3730, Suzhou, China.
34. Ziqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang, Zhiwei Xu, Xuri Ge, Zhumin Chen, Xinyu Ma, Daiting Shi, Shuaiqiang Wang, Dawei Yin, and Xin Xin. 2026. [Reinforced efficient reasoning via semantically diverse exploration](https://md.research.elamur.ai/free/en/papers/zhao-2026-reinforced-efficient-reasoning-semantically-diverse/). Preprint, arXiv:2601.05053.
35. Yixiao Zhou, Yang Li, Dongzhou Cheng, Hehe Fan, and Yu Cheng. 2026. Look inward to explore outward: Learning temperature policy from llm internal states via hierarchical rl. Preprint, arXiv:2602.13035. — [arXiv:2602.13035](https://arxiv.org/abs/2602.13035) [find](https://scholar.google.com/scholar?q=Yixiao+Zhou%2C+Yang+Li%2C+Dongzhou+Cheng%2C+Hehe+Fan%2C+and+Yu+Cheng%2E+2026%2E+Look+inward+to+explore+outward%3A+Learning+temperature+policy+from+llm+internal+states+via+hierarchical+rl%2E+Preprint%2C+arXiv%3A2602%2E13035)
36. Wei Zhu, Zhiwen Tang, and Kun Yue. 2026. Symphony: Synergistic multi-agent planning with heterogeneous language model assembly. Preprint, arXiv:2601.22623. — [arXiv:2601.22623](https://arxiv.org/abs/2601.22623) [find](https://scholar.google.com/scholar?q=Wei+Zhu%2C+Zhiwen+Tang%2C+and+Kun+Yue%2E+2026%2E+Symphony%3A+Synergistic+multi%2Dagent+planning+with+heterogeneous+language+model+assembly%2E+Preprint%2C+arXiv%3A2601%2E22623%2E+%E2%80%94+%5BarXiv%3A2601%2E22623%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
37. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824– 24837.
38. A Zhenhe Wu, Jian Yang, Jiaheng Liu, Xianjie Wu, Changzai Pan, Jie Zhang, Yu Zhao, Shuangyong Song, Yongxiang Li, and Zhoujun Li. 2025. Tabler1: Region-based reinforcement learning for table understanding. Preprint, arXiv:2505.12415. — [arXiv:2505.12415](https://arxiv.org/abs/2505.12415) [find](https://scholar.google.com/scholar?q=A+Zhenhe+Wu%2C+Jian+Yang%2C+Jiaheng+Liu%2C+Xianjie+Wu%2C+Changzai+Pan%2C+Jie+Zhang%2C+Yu+Zhao%2C+Shuangyong+Song%2C+Yongxiang+Li%2C+and+Zhoujun+Li%2E+2025%2E+Tabler1%3A+Region%2Dbased+reinforcement+learning+for+table+understan)
39. We used LLMs only to polish grammar and improve the clarity of the manuscript. All research ideas, experiments, and analyses were conducted by the authors. [find](https://scholar.google.com/scholar?q=We+used+LLMs+only+to+polish+grammar+and+improve+the+clarity+of+the+manuscript%2E+All+research+ideas%2C+experiments%2C+and+analyses+were+conducted+by+the+authors%2E)
40. Can Xie, Ruotong Pan, Xiangyu Wu, Yunfei Zhang, Jiayi Fu, Tingting Gao, and Guorui Zhou. 2025. [Unlocking exploration in rlvr: Uncertainty-aware advantage shaping for deeper reasoning](https://md.research.elamur.ai/free/en/papers/xie-2026-unlocking-exploration-rlvr-uncertainty-aware/). Preprint, arXiv:2510.10649.
41. B Tao Xiong, Xavier Hu, Wenyan Fan, and Shengyu Zhang. 2025. Mixture of reasonings: Teach large language models to reason with adaptive strategies. Preprint, arXiv:2507.00606. — [arXiv:2507.00606](https://arxiv.org/abs/2507.00606) [find](https://scholar.google.com/scholar?q=B+Tao+Xiong%2C+Xavier+Hu%2C+Wenyan+Fan%2C+and+Shengyu+Zhang%2E+2025%2E+Mixture+of+reasonings%3A+Teach+large+language+models+to+reason+with+adaptive+strategies%2E+Preprint%2C+arXiv%3A2507%2E00606%2E+%E2%80%94+%5BarXiv%3A2507%2E00606%5D%28htt)
42. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
43. Shenghao Ye, Yu Guo, Dong Jin, Yikai Shen, Yunpeng Hou, Shuangwu Chen, Jian Yang, and Xiaofeng Jiang. 2025. [When tableqa meets noise: A dual denoising framework for complex questions and large-scale tables](https://md.research.elamur.ai/free/en/papers/ye-2026-when-tableqa-meets-noise-dual/). Preprint, arXiv:2509.17680.

*Updated 31 October 2026*