---
title: "Distributional Clarity — paper by Sun et al. (2026)"
description: "They then propose Silhouette-Aware Reweighting, which gives more training weight to low-S samples. 42 references."
authors: ["Shaoning Sun", "Mingzhu Cai", "Huang He", "Bingjin Chen", "Siqi Bao", "Yujiu Yang", "Hua Wu", "Haifeng Wang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1004/"
doi: "10.18653/v1/2026.acl-long.1004"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sun-2026-distributional-clarity-hidden-driver-rl/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sun-2026-distributional-clarity-hidden-driver-rl/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/) / Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models

# Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models

**Authors:** [Shaoning Sun](https://md.research.elamur.ai/free/en/authors/shaoning-sun/), Mingzhu Cai, [Huang He](https://md.research.elamur.ai/free/en/authors/huang-he/), Bingjin Chen, [Siqi Bao](https://md.research.elamur.ai/free/en/authors/siqi-bao/), [Yujiu Yang](https://md.research.elamur.ai/free/en/authors/yujiu-yang/), [Hua Wu](https://md.research.elamur.ai/free/en/authors/hua-ren-wu/) and [Haifeng Wang](https://md.research.elamur.ai/free/en/authors/haifeng-wang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1004/) · [PDF](https://aclanthology.org/2026.acl-long.1004.pdf) · [DOI 10.18653/v1/2026.acl-long.1004](https://doi.org/10.18653/v1/2026.acl-long.1004) · [OpenAlex](https://openalex.org/W7166831962)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Under identical RL with verifiable rewards (RLVR) pipelines such as GRPO and DAPO, some model families (Qwen) improve markedly while others (Llama) gain little. The authors ask why, setting aside data composition and focusing on how models assign probability to their own outputs. They compare Qwen2.5-7B, OctoThinker-8B and Llama-3.1-8B-Instruct, which they treat as high, medium and low RL-Friendliness. Their analysis has three stages:
- **Pass rates on shared problems:** the models solve many of the same problems, but Qwen solves them more reliably.
- **Probability distributions:** in Qwen, length-normalized sequence probabilities of correct and incorrect answers form tight, well-separated groups. In the other models these groups overlap.
- **Reasoning behavior:** low separation coincides with serious logic errors and unstable reasoning.

They measure this "distributional clarity" per query with the Silhouette Coefficient (S). They then propose Silhouette-Aware Reweighting, which gives more training weight to low-S samples. This improves results on six math benchmarks for all model families. The main implication is that distributional clarity is a property that training can change, not only a diagnostic.

## Key points
- **Shared problems:** on AIME 2024 (K = 256 samples per query), the problems both models can solve make up about 71.4% of Qwen's solvable set and 90.9% of OctoThinker's. On most of these shared problems, Qwen achieves higher pass rates.
- **How S is computed:** a query's sampled responses are split into correct and incorrect clusters, and the Silhouette Coefficient is computed over their geometric-mean token probabilities. Higher S correlates strongly with pass rate.
- **Benchmarks and gains:** evaluation covers AIME24, AIME25, MATH500, AMC23, Minerva and Olympiad. On AIME24, the improvement varies by model family, from a low of 1.8 points to a high of 5.9.
- **OctoThinker-8B results:** DAPO-Silhouette (β = 0.2) raises the average from 27.8 to 30.1 and AIME24 from 4.9 to 8.2. Variants also beat the baseline: a Fisher Ratio version reaches 29.8 and β = 0.5 reaches 30.8. S improves under several confidence proxies.
- **Limitations:** the method relies on group rollouts (G = 16), which GRPO and DAPO already generate. Ultra-long reasoning traces (more than 10k tokens) are not studied.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 415 works · 94,028 citations
- [Baidu](https://md.research.elamur.ai/free/en/companies/baidu/) — Company, Beijing, China · 84 works · 17,681 citations
- [Tsinghua Shenzhen International Graduate School](https://md.research.elamur.ai/free/en/universities/tsinghua-shenzhen-international-graduate-school/) — University, Shenzhen, China · 29 works · 3,035 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Shaoning Sun, Mingzhu Cai, Huang He, Bingjin Chen, Siqi Bao, Yujiu Yang, Hua Wu and Haifeng Wang. 2026. Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sun2026distributional,
  title = {{Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models}},
  author = {Shaoning Sun, Mingzhu Cai, Huang He, Bingjin Chen, Siqi Bao, Yujiu Yang, Hua Wu and Haifeng Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1004},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1004}
}
```

- [← previous Distributional Alignment for Large Language Models under Domain Shift](https://md.research.elamur.ai/free/en/papers/pham-2026-distributional-alignment-large-language-models/)
- [next → Divergent Thinking: Escape the Homogeneity Trap in Generative Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/tao-2026-divergent-thinking-escape-homogeneity-trap/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/)

## References

42 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Syeda Nahida Akter, Shrimai Prabhumoye, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Yejin Choi, and Bryan Catanzaro. 2025. Front-loading reasoning: The synergy between pretraining and post-training data. arXiv preprint arXiv:2510.03264. [find](https://scholar.google.com/scholar?q=Syeda+Nahida+Akter%2C+Shrimai+Prabhumoye%2C+Eric+Nyberg%2C+Mostofa+Patwary%2C+Mohammad+Shoeybi%2C+Yejin+Choi%2C+and+Bryan+Catanzaro%2E+2025%2E+Front%2Dloading+reasoning%3A+The+synergy+between+pretraining+and+post%2Dtrainin)
2. Christopher M Bishop and Nasser M Nasrabadi. 2006. Pattern recognition and machine learning, volume 4. Springer. [find](https://scholar.google.com/scholar?q=Christopher+M+Bishop+and+Nasser+M+Nasrabadi%2E+2006%2E+Pattern+recognition+and+machine+learning%2C+volume+4%2E+Springer%2E)
3. Aili Chen, Aonian Li, Bangwei Gong, Binyang Jiang, Bo Fei, Bo Yang, Boji Shan, Changqing Yu, Chao Wang, Cheng Zhu, and 1 others. 2025a. [Minimax-m1: Scaling test-time compute efficiently with lightning attention](https://md.research.elamur.ai/free/en/papers/minimax-2025-minimax-m1-scaling-test/). arXiv preprint arXiv:2506.13585.
4. Minghan Chen, Guikun Chen, Wenguan Wang, and Yi Yang. 2025b. Seed-grpo: Semantic entropy enhanced grpo for uncertainty-aware policy optimization. arXiv preprint arXiv:2505.12346. — [arXiv:2505.12346](https://arxiv.org/abs/2505.12346) [find](https://scholar.google.com/scholar?q=Minghan+Chen%2C+Guikun+Chen%2C+Wenguan+Wang%2C+and+Yi+Yang%2E+2025b%2E+Seed%2Dgrpo%3A+Semantic+entropy+enhanced+grpo+for+uncertainty%2Daware+policy+optimization%2E+arXiv+preprint+arXiv%3A2505%2E12346%2E+%E2%80%94+%5BarXiv%3A2505%2E12346%5D%28)
5. Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. 2025. [Reasoning with exploration: An entropy perspective](https://md.research.elamur.ai/free/en/papers/cheng-2025-reasoning-exploration-entropy-perspective/). arXiv preprint arXiv:2506.14758.
6. Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, and 1 others. 2025. [The entropy mechanism of reinforcement learning for reasoning language models](https://md.research.elamur.ai/free/en/papers/cui-2025-entropy-mechanism-reinforcement-learning-reasoning/). arXiv preprint arXiv:2505.22617.
7. Runpeng Dai, Linfeng Song, Haolin Liu, Zhenwen Liang, Dian Yu, Haitao Mi, Zhaopeng Tu, Rui Liu, Tong Zheng, Hongtu Zhu, and 1 others. 2025. Cde: Curiosity-driven exploration for efficient reinforcement learning in large language models. arXiv preprint arXiv:2509.09675. — [arXiv:2509.09675](https://arxiv.org/abs/2509.09675) [find](https://scholar.google.com/scholar?q=Runpeng+Dai%2C+Linfeng+Song%2C+Haolin+Liu%2C+Zhenwen+Liang%2C+Dian+Yu%2C+Haitao+Mi%2C+Zhaopeng+Tu%2C+Rui+Liu%2C+Tong+Zheng%2C+Hongtu+Zhu%2C+and+1+others%2E+2025%2E+Cde%3A+Curiosity%2Ddriven+exploration+for+efficient+reinforcemen)
8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv e-prints, pages arXiv–2407.
9. Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. 2025. [Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars](https://md.research.elamur.ai/free/en/papers/gandhi-2025-cognitive-behaviors-that-enable-self/). arXiv preprint arXiv:2503.01307.
10. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
11. Yuto Harada, Yusuke Yamauchi, Yusuke Oda, Yohei Oseki, Yusuke Miyao, and Yu Takagi. 2025. Massive supervised fine-tuning experiments reveal how data, layer, and training factors shape llm alignment quality. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pa… [find](https://scholar.google.com/scholar?q=Yuto+Harada%2C+Yusuke+Yamauchi%2C+Yusuke+Oda%2C+Yohei+Oseki%2C+Yusuke+Miyao%2C+and+Yu+Takagi%2E+2025%2E+Massive+supervised+fine%2Dtuning+experiments+reveal+how+data%2C+layer%2C+and+training+factors+shape+llm+alignment+qu)
12. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874.
13. Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. 2025. [Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model](https://md.research.elamur.ai/free/en/papers/hu-2025-open-reasoner-zero-open-source/). arXiv preprint arXiv:2503.24290.
14. Zhen Huang, Zengzhi Wang, Shijie Xia, Xuefeng Li, Haoyang Zou, Ruijie Xu, Run-Ze Fan, Lyumanshan Ye, Ethan Chern, Yixin Ye, and 1 others. 2024. [Olympicarena: Benchmarking multi-discipline cognitive reasoning for superintelligent ai](https://md.research.elamur.ai/free/en/papers/huang-2024-olympicarena-benchmarking-multi-discipline-cognitive/). Advances in Neural Information Processing Systems, 37:19209– 19253.
15. Xisen Jin and Xiang Ren. 2024. Demystifying language model forgetting with low-rank example associations. arXiv preprint arXiv:2406.14026. — [arXiv:2406.14026](https://arxiv.org/abs/2406.14026) [find](https://scholar.google.com/scholar?q=Xisen+Jin+and+Xiang+Ren%2E+2024%2E+Demystifying+language+model+forgetting+with+low%2Drank+example+associations%2E+arXiv+preprint+arXiv%3A2406%2E14026%2E+%E2%80%94+%5BarXiv%3A2406%2E14026%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2406%2E14026%29)
16. Zhewei Kang, Xuandong Zhao, and Dawn Song. 2025. Scalable best-of-n selection for large language models via self-certainty. arXiv preprint arXiv:2502.18581. [find](https://scholar.google.com/scholar?q=Zhewei+Kang%2C+Xuandong+Zhao%2C+and+Dawn+Song%2E+2025%2E+Scalable+best%2Dof%2Dn+selection+for+large+language+models+via+self%2Dcertainty%2E+arXiv+preprint+arXiv%3A2502%2E18581%2E)
17. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, and 1 others. 2022. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). Advances in neural information processing systems, 35:3843–3857…
18. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297.
19. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and 1 others. 2025. [Kimi k2: Open agentic intelligence](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k2-open-agentic/). arXiv preprint arXiv:2507.20534.
20. Qwen Team. 2024. Qwen2.5: A party of foundation models. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2024%2E+Qwen2%2E5%3A+A+party+of+foundation+models%2E)
21. Yijun Tian, Shaoyu Chen, Zhichao Xu, Yawei Wang, Jinhe Bi, Peng Han, and Wei Wang. 2025. Reinforcement mid-training. arXiv preprint arXiv:2509.24375. — [arXiv:2509.24375](https://arxiv.org/abs/2509.24375) [find](https://scholar.google.com/scholar?q=Yijun+Tian%2C+Shaoyu+Chen%2C+Zhichao+Xu%2C+Yawei+Wang%2C+Jinhe+Bi%2C+Peng+Han%2C+and+Wei+Wang%2E+2025%2E+Reinforcement+mid%2Dtraining%2E+arXiv+preprint+arXiv%3A2509%2E24375%2E+%E2%80%94+%5BarXiv%3A2509%2E24375%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2509%2E24)
22. Chengying Tu, Xuemiao Zhang, Rongxiang Weng, Rumei Li, Chen Zhang, Yang Bai, Hongfei Yan, Jingang Wang, and Xunliang Cai. 2025. A survey on llm mid-training. arXiv preprint arXiv:2510.23081. — [arXiv:2510.23081](https://arxiv.org/abs/2510.23081) [find](https://scholar.google.com/scholar?q=Chengying+Tu%2C+Xuemiao+Zhang%2C+Rongxiang+Weng%2C+Rumei+Li%2C+Chen+Zhang%2C+Yang+Bai%2C+Hongfei+Yan%2C+Jingang+Wang%2C+and+Xunliang+Cai%2E+2025%2E+A+survey+on+llm+mid%2Dtraining%2E+arXiv+preprint+arXiv%3A2510%2E23081%2E+%E2%80%94+%5BarXiv%3A)
23. Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, and 1 others. 2025a. [Deepseek-v3](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-v3-2-pushing/). 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556.
24. Jinpeng Wang, Chao Li, Ting Ye, Mengyuan Zhang, Wei Liu, and Jian Luan. 2025a. Icpo: Intrinsic confidence-driven group relative preference optimization for efficient reinforcement learning. arXiv preprint arXiv:2511.21005. — [arXiv:2511.21005](https://arxiv.org/abs/2511.21005) [find](https://scholar.google.com/scholar?q=Jinpeng+Wang%2C+Chao+Li%2C+Ting+Ye%2C+Mengyuan+Zhang%2C+Wei+Liu%2C+and+Jian+Luan%2E+2025a%2E+Icpo%3A+Intrinsic+confidence%2Ddriven+group+relative+preference+optimization+for+efficient+reinforcement+learning%2E+arXiv+prep)
25. Emmy Liu, Graham Neubig, and Chenyan Xiong. 2025b. Midtraining bridges pretraining and posttraining distributions. arXiv preprint arXiv:2510.14865. — [arXiv:2510.14865](https://arxiv.org/abs/2510.14865) [find](https://scholar.google.com/scholar?q=Emmy+Liu%2C+Graham+Neubig%2C+and+Chenyan+Xiong%2E+2025b%2E+Midtraining+bridges+pretraining+and+posttraining+distributions%2E+arXiv+preprint+arXiv%3A2510%2E14865%2E+%E2%80%94+%5BarXiv%3A2510%2E14865%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2510%2E1486)
26. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, and 1 others. 2025b. [Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/). arXiv preprint arXiv:2506.01939.
27. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025c. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/). arXiv preprint arXiv:2503.20783.
28. Kaixiang Mo, Yuxin Shi, Weiwei Weng, Zhiqiang Zhou, Shuman Liu, Haibo Zhang, and Anxiang Zeng. 2025. Mid-training of [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). arXiv preprint arXiv:2510.06826. — [arXiv:2510.06826](https://arxiv.org/abs/2510.06826)
29. Kevin P Murphy. 2022. Probabilistic machine learning: an introduction. MIT press. [find](https://scholar.google.com/scholar?q=Kevin+P+Murphy%2E+2022%2E+Probabilistic+machine+learning%3A+an+introduction%2E+MIT+press%2E)
30. Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, and 1 others. 2025a. [Spurious rewards: Rethinking training signals in rlvr](https://md.research.elamur.ai/free/en/papers/shao-2025-spurious-rewards-rethinking-training-signals/). arXiv preprint arXiv:2506.10947.
31. Zhihong Shao, Yuxiang Luo, Chengda Lu, ZZ Ren, Jiewen Hu, Tian Ye, Zhibin Gou, Shirong Ma, and Xiaokang Zhang. 2025b. [Deepseekmath-v2: Towards self-verifiable mathematical reasoning](https://md.research.elamur.ai/free/en/papers/shao-2025-deepseekmath-v2-self-verifiable/). arXiv preprint arXiv:2511.22570.
32. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
33. Zengzhi Wang, Fan Zhou, Xuefeng Li, and Pengfei Liu. 2025c. Octothinker: Mid-training incentivizes reinforcement learning scaling. arXiv preprint arXiv:2506.20512. — [arXiv:2506.20512](https://arxiv.org/abs/2506.20512) [find](https://scholar.google.com/scholar?q=Zengzhi+Wang%2C+Fan+Zhou%2C+Xuefeng+Li%2C+and+Pengfei+Liu%2E+2025c%2E+Octothinker%3A+Mid%2Dtraining+incentivizes+reinforcement+learning+scaling%2E+arXiv+preprint+arXiv%3A2506%2E20512%2E+%E2%80%94+%5BarXiv%3A2506%2E20512%5D%28https%3A%2F%2Farxiv%2Eo)
34. Chao-Chung Wu, Zhi Rui Tam, Chieh-Yen Lin, Hung-yi Lee, and Yun-Nung Chen. 2025a. Clear minds think alike: What makes llm fine-tuning robust? a study of token perplexity. arXiv e-prints, pages arXiv–2501. [find](https://scholar.google.com/scholar?q=Chao%2DChung+Wu%2C+Zhi+Rui+Tam%2C+Chieh%2DYen+Lin%2C+Hung%2Dyi+Lee%2C+and+Yun%2DNung+Chen%2E+2025a%2E+Clear+minds+think+alike%3A+What+makes+llm+fine%2Dtuning+robust%3F+a+study+of+token+perplexity%2E+arXiv+e%2Dprints%2C+pages+arXiv%E2%80%932)
35. Yihong Wu, Liheng Ma, Lei Ding, Muzhi Li, Xinyu Wang, Kejia Chen, Zhan Su, Zhanguang Zhang, Chenyang Huang, Yingxue Zhang, Mark Coates, and Jian-Yun Nie. 2025b. It takes two: Your GRPO is secretly DPO. In NeurIPS 2025 Workshop on Efficient Reasoning. [find](https://scholar.google.com/scholar?q=Yihong+Wu%2C+Liheng+Ma%2C+Lei+Ding%2C+Muzhi+Li%2C+Xinyu+Wang%2C+Kejia+Chen%2C+Zhan+Su%2C+Zhanguang+Zhang%2C+Chenyang+Huang%2C+Yingxue+Zhang%2C+Mark+Coates%2C+and+Jian%2DYun+Nie%2E+2025b%2E+It+takes+two%3A+Your+GRPO+is+secretly+DPO)
36. Can Xie, Ruotong Pan, Xiangyu Wu, Yunfei Zhang, Jiayi Fu, Tingting Gao, and Guorui Zhou. 2025. [Unlocking exploration in rlvr: Uncertainty-aware advantage shaping for deeper reasoning](https://md.research.elamur.ai/free/en/papers/xie-2026-unlocking-exploration-rlvr-uncertainty-aware/). arXiv preprint arXiv:2510.10649.
37. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
38. Tianyu Yu, Bo Ji, Shouli Wang, Shu Yao, Zefan Wang, Ganqu Cui, Lifan Yuan, Ning Ding, Yuan Yao, Zhiyuan Liu, and 1 others. 2025b. Rlpr: Extrapolating rlvr to general domains without verifiers. arXiv preprint arXiv:2506.18254. — [arXiv:2506.18254](https://arxiv.org/abs/2506.18254) [find](https://scholar.google.com/scholar?q=Tianyu+Yu%2C+Bo+Ji%2C+Shouli+Wang%2C+Shu+Yao%2C+Zefan+Wang%2C+Ganqu+Cui%2C+Lifan+Yuan%2C+Ning+Ding%2C+Yuan+Yao%2C+Zhiyuan+Liu%2C+and+1+others%2E+2025b%2E+Rlpr%3A+Extrapolating+rlvr+to+general+domains+without+verifiers%2E+arXiv+p)
39. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. 2025. [Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/) arXiv preprint arXiv:2504.13837.
40. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. 2025. [Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/). arXiv preprint arXiv:2503.18892.
41. Charlie Zhang, Graham Neubig, and Xiang Yue. 2025. On the interplay of pre-training, mid-training, and rl on reasoning language models. arXiv preprint arXiv:2512.07783. — [arXiv:2512.07783](https://arxiv.org/abs/2512.07783) [find](https://scholar.google.com/scholar?q=Charlie+Zhang%2C+Graham+Neubig%2C+and+Xiang+Yue%2E+2025%2E+On+the+interplay+of+pre%2Dtraining%2C+mid%2Dtraining%2C+and+rl+on+reasoning+language+models%2E+arXiv+preprint+arXiv%3A2512%2E07783%2E+%E2%80%94+%5BarXiv%3A2512%2E07783%5D%28https%3A%2F%2Far)
42. Xinyu Zhu, Mengzhou Xia, Zhepei Wei, Wei-Lin Chen, Danqi Chen, and Yu Meng. 2025. The surprising effectiveness of negative reinforcement in llm reasoning. arXiv preprint arXiv:2506.01347. [find](https://scholar.google.com/scholar?q=Xinyu+Zhu%2C+Mengzhou+Xia%2C+Zhepei+Wei%2C+Wei%2DLin+Chen%2C+Danqi+Chen%2C+and+Yu+Meng%2E+2025%2E+The+surprising+effectiveness+of+negative+reinforcement+in+llm+reasoning%2E+arXiv+preprint+arXiv%3A2506%2E01347%2E)

*Updated 2 November 2026*