---
title: "Plug-and-Play Data Module for Code RL: Adaptive Ambiguity"
description: "Reinforcement learning helps LLMs generate and edit code, but good training data is scarce. 35 references."
authors: ["Jianqing Zhang", "Wei Xia", "Zhezheng Hao", "Hong Wang", "Hande Dong", "Qiang Lin", "Yang Liu", "Jian Cao", "Qiang Yang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.886/"
doi: "10.18653/v1/2026.findings-acl.886"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-plug-play-data-module-code/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-plug-play-data-module-code/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/) / Plug-and-Play Data Module for Code RL: Adaptive Ambiguity Replay

# Plug-and-Play Data Module for Code RL: Adaptive Ambiguity Replay

**Authors:** [Jianqing Zhang](https://md.research.elamur.ai/free/en/authors/jianqing-zhang/), [Wei Xia](https://md.research.elamur.ai/free/en/authors/wei-xia/), [Zhezheng Hao](https://md.research.elamur.ai/free/en/authors/zhezheng-hao/), [Hong Wang](https://md.research.elamur.ai/free/en/authors/hong-wang/), [Hande Dong](https://md.research.elamur.ai/free/en/authors/hande-dong/), [Qiang Lin](https://md.research.elamur.ai/free/en/authors/qiang-lin/), [Yang Liu](https://md.research.elamur.ai/free/en/authors/yang-liu/), [Jian Cao](https://md.research.elamur.ai/free/en/authors/jian-cao/) and [Qiang Yang](https://md.research.elamur.ai/free/en/authors/qiang-yang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.886/) · [PDF](https://aclanthology.org/2026.findings-acl.886.pdf) · [DOI 10.18653/v1/2026.findings-acl.886](https://doi.org/10.18653/v1/2026.findings-acl.886) · [OpenAlex](https://openalex.org/W7166861486)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Reinforcement learning helps LLMs generate and edit code, but good training data is scarce. Experience replay can stretch limited data. However, current LLM replay methods such as RePO and RLEP choose what to replay using static, single-epoch signals like reward or variance, and they can feed the model stale experience. The authors apply dataset cartography to RL training dynamics. They find that "ambiguous" samples, which matter most for generalization, quickly drift into the "easy-to-learn" region and stop being useful. Their remedy is Adaptive Ambiguity Replay (A2R), a plug-in data module that runs separately from the main training loop. A2R does three things:
- It keeps a FIFO buffer of per-prompt rewards across epochs, along with their rollouts.
- It estimates each prompt's cartography position from the mean and standard deviation of those rewards, then replays the top-K most ambiguous prompts.
- It down-weights old rollouts according to how far the current policy has moved from the policy that produced them.

Added to GRPO and DAPO, A2R improves exact match accuracy on real-world code editing, both in learned and unseen domains. The authors conclude that ambiguity tracked across epochs is a useful criterion for choosing what to replay.

## Key points
- Tested on nine general-purpose and code-specialized LLMs (3B–14B), including Qwen2.5-Coder and Qwen3. A2R plugs into GRPO and DAPO without changing their rollouts or advantage computation. Code is published as `verl-A2R`.
- On the learned-domain test set, DAPO + A2R improved over DAPO for every model reported:
  - Qwen2.5-Coder-3B: 38.74 → 42.18
  - Qwen2.5-Coder-7B: 41.64 → 43.36
  - Qwen3-4B: 37.99 → 40.31
  - Qwen3-8B: 39.80 → 42.91
- In the ablations, choosing replay samples by high reward, low reward, or within-epoch variance did worse than plain DAPO. Replaying hard (low-reward) samples hurt the most. The orthogonal-prompt filter and adaptive scaling each added further gains.
- The default hyperparameters are K = 128 replayed prompts, which balances accuracy against cost, and λ = 5 for importance-scaling strength, which sits at the peak of a unimodal curve.
- A variant that regenerates rollouts for the selected prompts, instead of reusing cached ones, adds up to +2.12 points, mostly for GRPO. This addresses the method's main limitation, outdated rollouts, at the price of extra compute.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 212 works · 19,544 citations
- [Tencent](https://md.research.elamur.ai/free/en/companies/tencent/) — Company, Shenzhen, China · 163 works · 21,032 citations
- [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/) — University, Hangzhou, China · 224 works · 13,587 citations
- [Hong Kong Polytechnic University](https://md.research.elamur.ai/free/en/universities/hong-kong-polytechnic-university/) — University, Hong Kong, China · 106 works · 5,190 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works · [🖥️ Computer Science Applications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-science-applications/) · 131 works

## Cite

Cite as: Jianqing Zhang, Wei Xia, Zhezheng Hao, Hong Wang, Hande Dong, Qiang Lin, Yang Liu, Jian Cao and Qiang Yang. 2026. Plug-and-Play Data Module for Code RL: Adaptive Ambiguity Replay. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026plug,
  title = {{Plug-and-Play Data Module for Code RL: Adaptive Ambiguity Replay}},
  author = {Jianqing Zhang, Wei Xia, Zhezheng Hao, Hong Wang, Hande Dong, Qiang Lin, Yang Liu, Jian Cao and Qiang Yang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.886},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.886}
}
```

- [← previous PlotGen-Bench: Evaluating VLMs on Generating Visualization Code from Diverse Plots across Multiple Libraries](https://md.research.elamur.ai/free/en/papers/zhao-2026-plotgen-bench-evaluating-vlms-generating/)
- [next → POaaS: Minimal-Edit Prompt Optimization as a Service to Lift Accuracy and Cut Hallucinations on On-Device sLLMs](https://md.research.elamur.ai/free/en/papers/shim-2026-poaas-minimal-edit-prompt-optimization/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/)

## References

35 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Natasha Butt, Blazej Manczak, Auke Wiggers, Corrado Rainone, David W Zhang, Michaël Defferrard, and Taco Cohen. 2024. Codeit: Self-improving language models with prioritized hindsight replay. In International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Natasha+Butt%2C+Blazej+Manczak%2C+Auke+Wiggers%2C+Corrado+Rainone%2C+David+W+Zhang%2C+Micha%C3%ABl+Defferrard%2C+and+Taco+Cohen%2E+2024%2E+Codeit%3A+Self%2Dimproving+language+models+with+prioritized+hindsight+replay%2E+In+Inter)
2. Federico Cassano, Luisa Li, Akul Sethi, Noah Shinn, Abby Brennan-Jones, Jacob Ginesin, Edward Berman, George Chakhnashvili, Anton Lozhkov, Carolyn Jane Anderson, and 1 others. 2024. Can it edit? evaluating the ability of large language models to follow code editing instructions. In First Conference… [find](https://scholar.google.com/scholar?q=Federico+Cassano%2C+Luisa+Li%2C+Akul+Sethi%2C+Noah+Shinn%2C+Abby+Brennan%2DJones%2C+Jacob+Ginesin%2C+Edward+Berman%2C+George+Chakhnashvili%2C+Anton+Lozhkov%2C+Carolyn+Jane+Anderson%2C+and+1+others%2E+2024%2E+Can+it+edit%3F+evalu)
3. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Victor Dibia, Adam Fourney, Gagan Bansal, Forough Poursabzi-Sangdeh, Han Liu, and Saleema Amershi. 2023. Aligning offline metrics and human judgments of value for code generation models. In Findings of… [find](https://scholar.google.com/scholar?q=Mark+Chen%2C+Jerry+Tworek%2C+Heewoo+Jun%2C+Qiming+Yuan%2C+Henrique+Ponde+De+Oliveira+Pinto%2C+Jared+Kaplan%2C+Victor+Dibia%2C+Adam+Fourney%2C+Gagan+Bansal%2C+Forough+Poursabzi%2DSangdeh%2C+Han+Liu%2C+and+Saleema+Amershi%2E+202)
4. Shihan Dou, Muling Wu, Jingwen Xu, Rui Zheng, Tao Gui, and Qi Zhang. 2025. Improving rl exploration for llm reasoning through retrospective replay. In CCF International Conference on Natural Language Processing and Chinese Computing, pages 594–606. Springer. [find](https://scholar.google.com/scholar?q=Shihan+Dou%2C+Muling+Wu%2C+Jingwen+Xu%2C+Rui+Zheng%2C+Tao+Gui%2C+and+Qi+Zhang%2E+2025%2E+Improving+rl+exploration+for+llm+reasoning+through+retrospective+replay%2E+In+CCF+International+Conference+on+Natural+Language+)
5. William Fedus, Prajit Ramachandran, Rishabh Agarwal, Yoshua Bengio, Hugo Larochelle, Mark Rowland, and Will Dabney. 2020. Revisiting fundamentals of experience replay. In International conference on machine learning. [find](https://scholar.google.com/scholar?q=William+Fedus%2C+Prajit+Ramachandran%2C+Rishabh+Agarwal%2C+Yoshua+Bengio%2C+Hugo+Larochelle%2C+Mark+Rowland%2C+and+Will+Dabney%2E+2020%2E+Revisiting+fundamentals+of+experience+replay%2E+In+International+conference+on+m)
6. Cuiyun Gao, Xing Hu, Shan Gao, Xin Xia, and Zhi Jin. 2025. The current challenges of software engineering in the era of large language models. ACM Transactions on Software Engineering and Methodology, 34(5):1–30. [find](https://scholar.google.com/scholar?q=Cuiyun+Gao%2C+Xing+Hu%2C+Shan+Gao%2C+Xin+Xia%2C+and+Zhi+Jin%2E+2025%2E+The+current+challenges+of+software+engineering+in+the+era+of+large+language+models%2E+ACM+Transactions+on+Software+Engineering+and+Methodology%2C)
7. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, and 1 others. 2024a. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Qihao+Zhu%2C+Dejian+Yang%2C+Zhenda+Xie%2C+Kai+Dong%2C+Wentao+Zhang%2C+Guanting+Chen%2C+Xiao+Bi%2C+Yu+Wu%2C+YK+Li%2C+and+1+others%2E+2024a%2E+Deepseek%2Dcoder%3A+When+the+large+language+model+meets+programming%E2%80%93the+ris)
8. Jiawei Guo, Ziming Li, Xueling Liu, Kaijing Ma, Tianyu Zheng, Zhouliang Yu, Ding Pan, Yizhi Li, Ruibo Liu, Yue Wang, and 1 others. 2024b. Codeeditorbench: Evaluating code editing capability of large language models. arXiv preprint arXiv:2404.03543. — [arXiv:2404.03543](https://arxiv.org/abs/2404.03543) [find](https://scholar.google.com/scholar?q=Jiawei+Guo%2C+Ziming+Li%2C+Xueling+Liu%2C+Kaijing+Ma%2C+Tianyu+Zheng%2C+Zhouliang+Yu%2C+Ding+Pan%2C+Yizhi+Li%2C+Ruibo+Liu%2C+Yue+Wang%2C+and+1+others%2E+2024b%2E+Codeeditorbench%3A+Evaluating+code+editing+capability+of+large+l)
9. Hugging Face. 2025. Hugging face — the ai community building the future. [https://huggingface.co/](https://huggingface.co/). Accessed: 2025-10-08. [↗](https://huggingface.co/)
10. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, and 1 others. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186. — [Qwen2.5-Coder Technical Report](https://md.research.elamur.ai/free/en/papers/hui-2024-qwen2-5-coder-technical-report/)
11. David Isele and Akansel Cosgun. 2018. Selective experience replay for lifelong learning. In Proceedings of the AAAI conference on artificial intelligence. [find](https://scholar.google.com/scholar?q=David+Isele+and+Akansel+Cosgun%2E+2018%2E+Selective+experience+replay+for+lifelong+learning%2E+In+Proceedings+of+the+AAAI+conference+on+artificial+intelligence%2E)
12. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
13. VI Lcvenshtcin. 1966. Binary coors capable or ‘correcting deletions, insertions, and reversals. In Soviet physics-doklady. [find](https://scholar.google.com/scholar?q=VI+Lcvenshtcin%2E+1966%2E+Binary+coors+capable+or+%E2%80%98correcting+deletions%2C+insertions%2C+and+reversals%2E+In+Soviet+physics%2Ddoklady%2E)
14. Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. 2022. [Coderl: Mastering code generation through pretrained models and deep reinforcement learning](https://md.research.elamur.ai/free/en/papers/le-2022-coderl-mastering-code-generation-through/). Advances in Neural Information Processing Systems.
15. Siheng Li, Zhanhui Zhou, Wai Lam, Chao Yang, and Chaochao Lu. 2025. Repo: Replay-enhanced policy optimization. arXiv preprint arXiv:2506.09340. — [arXiv:2506.09340](https://arxiv.org/abs/2506.09340) [find](https://scholar.google.com/scholar?q=Siheng+Li%2C+Zhanhui+Zhou%2C+Wai+Lam%2C+Chao+Yang%2C+and+Chaochao+Lu%2E+2025%2E+Repo%3A+Replay%2Denhanced+policy+optimization%2E+arXiv+preprint+arXiv%3A2506%2E09340%2E+%E2%80%94+%5BarXiv%3A2506%2E09340%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2506%2E09340%29)
16. Zichuan Liu, Jinyu Wang, Lei Song, and Jiang Bian. 2025. Sample-efficient llm optimization with reset replay. arXiv preprint arXiv:2508.06412. — [arXiv:2508.06412](https://arxiv.org/abs/2508.06412) [find](https://scholar.google.com/scholar?q=Zichuan+Liu%2C+Jinyu+Wang%2C+Lei+Song%2C+and+Jiang+Bian%2E+2025%2E+Sample%2Defficient+llm+optimization+with+reset+replay%2E+arXiv+preprint+arXiv%3A2508%2E06412%2E+%E2%80%94+%5BarXiv%3A2508%2E06412%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2508%2E06412%29)
17. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems.
18. Leonard S Pleiss, Tobias Sutter, and Maximilian Schiffer. 2025. Reliability-adjusted prioritized experience replay. arXiv preprint arXiv:2506.18482. — [arXiv:2506.18482](https://arxiv.org/abs/2506.18482) [find](https://scholar.google.com/scholar?q=Leonard+S+Pleiss%2C+Tobias+Sutter%2C+and+Maximilian+Schiffer%2E+2025%2E+Reliability%2Dadjusted+prioritized+experience+replay%2E+arXiv+preprint+arXiv%3A2506%2E18482%2E+%E2%80%94+%5BarXiv%3A2506%2E18482%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2506%2E184)
19. Qwen Team. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2024%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)
20. Santhosh Kumar Ravindran. 2025. Cosmocore affective dream-replay reinforcement learning for code generation. arXiv preprint arXiv:2510.18895. — [arXiv:2510.18895](https://arxiv.org/abs/2510.18895) [find](https://scholar.google.com/scholar?q=Santhosh+Kumar+Ravindran%2E+2025%2E+Cosmocore+affective+dream%2Dreplay+reinforcement+learning+for+code+generation%2E+arXiv+preprint+arXiv%3A2510%2E18895%2E+%E2%80%94+%5BarXiv%3A2510%2E18895%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2510%2E18895%29)
21. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
22. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
23. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). In Proceedings of the Twentieth European Conference on Computer Systems.
24. Yifan Sun, Jingyan Shen, Yibin Wang, Tianyu Chen, Zhendong Wang, Mingyuan Zhou, and Huan Zhang. 2025a. Improving data efficiency for llm reinforcement fine-tuning through difficulty-targeted online data selection and rollout replay. arXiv preprint arXiv:2506.05316. — [arXiv:2506.05316](https://arxiv.org/abs/2506.05316) [find](https://scholar.google.com/scholar?q=Yifan+Sun%2C+Jingyan+Shen%2C+Yibin+Wang%2C+Tianyu+Chen%2C+Zhendong+Wang%2C+Mingyuan+Zhou%2C+and+Huan+Zhang%2E+2025a%2E+Improving+data+efficiency+for+llm+reinforcement+fine%2Dtuning+through+difficulty%2Dtargeted+online+da)
25. Yiyou Sun, Yuhan Cao, Pohao Huang, Haoyue Bai, Hannaneh Hajishirzi, Nouha Dziri, and Dawn Song. 2025b. Delta: How does rl unlock and transfer new algorithms in llms? In The 5th Workshop on Mathematical Reasoning and AI at NeurIPS 2025. [find](https://scholar.google.com/scholar?q=Yiyou+Sun%2C+Yuhan+Cao%2C+Pohao+Huang%2C+Haoyue+Bai%2C+Hannaneh+Hajishirzi%2C+Nouha+Dziri%2C+and+Dawn+Song%2E+2025b%2E+Delta%3A+How+does+rl+unlock+and+transfer+new+algorithms+in+llms%3F+In+The+5th+Workshop+on+Mathematica)
26. Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A Smith, and Yejin Choi. 2020. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Swabha+Swayamdipta%2C+Roy+Schwartz%2C+Nicholas+Lourie%2C+Yizhong+Wang%2C+Hannaneh+Hajishirzi%2C+Noah+A+Smith%2C+and+Yejin+Choi%2E+2020%2E+Dataset+cartography%3A+Mapping+and+diagnosing+datasets+with+training+dynamics%2E+I)
27. Harm Vanseijen and Rich Sutton. 2015. A deeper look at planning as learning from replay. In International conference on machine learning. [find](https://scholar.google.com/scholar?q=Harm+Vanseijen+and+Rich+Sutton%2E+2015%2E+A+deeper+look+at+planning+as+learning+from+replay%2E+In+International+conference+on+machine+learning%2E)
28. Chao Wang, Tao Yang, Hongtao Tian, Yunsheng Shi, Qiyao Ma, Xiaotao Liu, Ting Yao, and Wenbo Ding. 2025a. Learning more with less: A dynamic dual-level down-sampling framework for efficient policy optimization. arXiv preprint arXiv:2509.22115. — [arXiv:2509.22115](https://arxiv.org/abs/2509.22115) [find](https://scholar.google.com/scholar?q=Chao+Wang%2C+Tao+Yang%2C+Hongtao+Tian%2C+Yunsheng+Shi%2C+Qiyao+Ma%2C+Xiaotao+Liu%2C+Ting+Yao%2C+and+Wenbo+Ding%2E+2025a%2E+Learning+more+with+less%3A+A+dynamic+dual%2Dlevel+down%2Dsampling+framework+for+efficient+policy+opti)
29. Chen Wang, Lai Wei, Yanzhi Zhang, Chenyang Shao, Zedong Dan, Weiran Huang, Yue Wang, and Yuzhi Zhang. 2025b. Eframe: Deeper reasoning via exploration-filtering-replay reinforcement learning framework. arXiv preprint arXiv:2506.22200. — [arXiv:2506.22200](https://arxiv.org/abs/2506.22200) [find](https://scholar.google.com/scholar?q=Chen+Wang%2C+Lai+Wei%2C+Yanzhi+Zhang%2C+Chenyang+Shao%2C+Zedong+Dan%2C+Weiran+Huang%2C+Yue+Wang%2C+and+Yuzhi+Zhang%2E+2025b%2E+Eframe%3A+Deeper+reasoning+via+exploration%2Dfiltering%2Dreplay+reinforcement+learning+framework%2E)
30. Hong Wang, Zhezheng Hao, Jian Luo, Chenxing Wei, Yao Shu, Lei Liu, Qiang Lin, Hande Dong, and Jiawei Chen. 2025c. Scheduling your llm reinforcement learning with reasoning trees. arXiv preprint arXiv:2510.24832. — [arXiv:2510.24832](https://arxiv.org/abs/2510.24832) [find](https://scholar.google.com/scholar?q=Hong+Wang%2C+Zhezheng+Hao%2C+Jian+Luo%2C+Chenxing+Wei%2C+Yao+Shu%2C+Lei+Liu%2C+Qiang+Lin%2C+Hande+Dong%2C+and+Jiawei+Chen%2E+2025c%2E+Scheduling+your+llm+reinforcement+learning+with+reasoning+trees%2E+arXiv+preprint+arXiv%3A)
31. Junqiao Wang, Zeng Zhang, Yangfan He, Zihao Zhang, Xinyuan Song, Yuyang Song, Tianyu Shi, Yuchen Li, Hengyuan Xu, Kunyu Wu, and 1 others. 2024. Enhancing code llms with reinforcement learning in code generation: A survey. arXiv preprint arXiv:2412.20367. — [arXiv:2412.20367](https://arxiv.org/abs/2412.20367) [find](https://scholar.google.com/scholar?q=Junqiao+Wang%2C+Zeng+Zhang%2C+Yangfan+He%2C+Zihao+Zhang%2C+Xinyuan+Song%2C+Yuyang+Song%2C+Tianyu+Shi%2C+Yuchen+Li%2C+Hengyuan+Xu%2C+Kunyu+Wu%2C+and+1+others%2E+2024%2E+Enhancing+code+llms+with+reinforcement+learning+in+code+)
32. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv:2505.09388. arXiv preprint John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir P…
33. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). arXiv preprint arXiv:2503.14476.
34. Zed Industries. 2025. Zeta: Repository-level code edit prediction dataset. [https://huggingface.co/](https://huggingface.co/) datasets/zed-industries/zeta. Dataset, Hugging Face; Apache-2.0 license. [↗](https://huggingface.co/)
35. Hongzhi Zhang, Jia Fu, Jingyuan Zhang, Kai Fu, Qi Wang, Fuzheng Zhang, and Guorui Zhou. 2025a. Rlep: Reinforcement learning with experience replay for llm reasoning. arXiv preprint arXiv:2507.07451. — [arXiv:2507.07451](https://arxiv.org/abs/2507.07451) [find](https://scholar.google.com/scholar?q=Hongzhi+Zhang%2C+Jia+Fu%2C+Jingyuan+Zhang%2C+Kai+Fu%2C+Qi+Wang%2C+Fuzheng+Zhang%2C+and+Guorui+Zhou%2E+2025a%2E+Rlep%3A+Reinforcement+learning+with+experience+replay+for+llm+reasoning%2E+arXiv+preprint+arXiv%3A2507%2E07451%2E+%E2%80%94)

*Updated 2 November 2026*