---
title: "Absolute Zero: Reinforced Self-play Reasoning with Zero Data"
description: "The authors propose Absolute Zero, a paradigm in which a single language model… 15.2 points. 122 references, cited by 48 papers here."
authors: ["Andrew Zhao", "Yiran Wu", "Yang Yue", "Tong Wu", "Quentin Xu", "Yang Yue", "Matthieu Lin", "Shenzhi Wang", "Qingyun Wu", "Zilong Zheng", "Gao Huang"]
year: 2025
venue: "arXiv:2505.03335"
source: "https://arxiv.org/abs/2505.03335"
doi: "10.48550/arxiv.2505.03335"
arxiv: "2505.03335"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhao-2025-absolute-zero-reinforced-self-play/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhao-2025-absolute-zero-reinforced-self-play/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / Absolute Zero: Reinforced Self-play Reasoning with Zero Data

# Absolute Zero: Reinforced Self-play Reasoning with Zero Data

**Authors:** [Andrew Zhao](https://md.research.elamur.ai/free/en/authors/andrew-zhao/), [Yiran Wu](https://md.research.elamur.ai/free/en/authors/yiran-wu/), [Yang Yue](https://md.research.elamur.ai/free/en/authors/yang-yue/), [Tong Wu](https://md.research.elamur.ai/free/en/authors/tong-wu/), Quentin Xu, [Yang Yue](https://md.research.elamur.ai/free/en/authors/yang-yue/), Matthieu Lin, [Shenzhi Wang](https://md.research.elamur.ai/free/en/authors/shenzhi-wang/), [Qingyun Wu](https://md.research.elamur.ai/free/en/authors/qingyun-wu/), [Zilong Zheng](https://md.research.elamur.ai/free/en/authors/zilong-zheng/) and [Gao Huang](https://md.research.elamur.ai/free/en/authors/gao-ming-huang/) · **Year:** 2025 · **Venue:** [arXiv:2505.03335](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) ([NeurIPS 2025](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2025/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2505.03335) · [PDF](https://arxiv.org/pdf/2505.03335) · [DOI 10.48550/arxiv.2505.03335](https://doi.org/10.48550/arxiv.2505.03335) · [arXiv 2505.03335](https://arxiv.org/abs/2505.03335) · [OpenAlex](https://openalex.org/W4416010183)

**Citations:** 9 citations (OpenAlex, 2026-10-14) · [48](https://md.research.elamur.ai/free/en/papers/zhao-2025-absolute-zero-reinforced-self-play/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Current "zero" RLVR methods skip human-written reasoning traces, but they still train on large human-curated sets of questions and answers. Such data is costly to produce, and it may limit models that eventually outgrow human-designed problems. The authors propose Absolute Zero, a paradigm in which a single language model invents its own training tasks, choosing ones expected to advance its learning the most, and then gets better by working through them. No external data is used. Their implementation, the Absolute Zero Reasoner (AZR), uses a Python code executor as the environment. The executor checks that generated tasks are valid and verifies the solutions, which gives a grounded reward that is hard to game. Tasks are built from program–input–output triplets and cover deduction, abduction and induction. Training uses a new advantage estimator designed for this multitask, two-role setup. Although AZR never sees benchmark-style data, it reaches state-of-the-art coding results and the best combined coding-plus-math average among comparable models. This suggests general reasoning can be developed without curated datasets.

## Key points
- AZR exceeds earlier "zero"-setting models trained on tens of thousands of in-domain examples by about 1.8 absolute points on the combined code and math average.
- Transfer across domains is strong. Expert code models gain only about 0.65 math points from RLVR, while AZR-Base-7B and AZR-Coder-7B gain 10.9 and 15.2 points.
- Code priors help. Qwen-Coder-7B starts 3.6 math points below Qwen-7B but ends 0.7 points above it after AZR training.
- Gains grow with model size. The 3B, 7B and 14B coder models improve by +5.7, +10.2 and +13.2 points respectively.
- Llama3.1-8B under AZR sometimes produced worrying reasoning chains (the "uh-oh moment"), which points to a need for safety oversight. The code, models and logs are released as open source.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 415 works · 94,028 citations
- [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) — Company, London, United Kingdom · 72 works · 84,326 citations
- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 201 works · 17,173 citations
- [Pennsylvania State University](https://md.research.elamur.ai/free/en/universities/pennsylvania-state-university/) — University, State College, United States · 67 works · 5,865 citations
- [Beijing Institute for General Artificial Intelligence](https://md.research.elamur.ai/free/en/government/beijing-institute-for-general-artificial-intelligence/) — Government body, Beijing, China · 11 works · 633 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Yang Yue, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng and Gao Huang. 2025. Absolute Zero: Reinforced Self-play Reasoning with Zero Data. arXiv:2505.03335.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zhao2025absolute,
  title = {{Absolute Zero: Reinforced Self-play Reasoning with Zero Data}},
  author = {Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Yang Yue, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng and Gao Huang},
  year = {2025},
  doi = {10.48550/arxiv.2505.03335},
  eprint = {2505.03335},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2505.03335}
}
```

- [← previous SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs](https://md.research.elamur.ai/free/en/papers/xu-2025-softcot-soft-chain-thought-efficient/)
- [next → Kimi-Audio Technical Report](https://md.research.elamur.ai/free/en/papers/kimiteam-2025-kimi-audio-technical-report/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

122 references, 46 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aryabumi, V., Su, Y., Ma, R., Morisot, A., Zhang, I., Locatelli, A., Fadaee, M., Üstün, A., and Hooker, S. To code, or not to code? exploring impact of code in pre-training. CoRR, abs/2408.10914, 2024. doi: 10.48550/ARXIV.2408.10914. URL https://doi.org/10.48550/arXiv.2408.10914. — [doi:10.48550/arxiv.2408.10914](https://doi.org/10.48550/arxiv.2408.10914) [↗](https://doi.org/10.48550/arXiv.2408.10914)
2. Austin, J., Odena, A., Nye, M. I., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C. J., Terry, M., Le, Q. V., and Sutton, C. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). CoRR, abs/2108.07732, 2021. URL https://arxiv.org/abs/2108.07732. [↗](https://arxiv.org/abs/2108.07732)
3. Burns, C., Izmailov, P., Kirchner, J. H., Baker, B., Gao, L., Aschenbrenner, L., Chen, Y., Ecoffet, A., Joglekar, M., Leike, J., Sutskever, I., and Wu, J. [Weak-to-strong generalization: Eliciting strong capabilities with weak supervision](https://md.research.elamur.ai/free/en/papers/burns-2023-weak-strong-generalization-eliciting-strong/). In Forty-first International Conference on Machine Learning,…
4. OpenReview.net, 2024. URL [https://openreview.net/forum?id=ghNRg2mEgN](https://openreview.net/forum?id=ghNRg2mEgN). [↗](https://openreview.net/forum?id=ghNRg2mEgN)
5. Canal, M. Radon: Python tool for code metrics. [https://github.com/rubik/radon](https://github.com/rubik/radon), 2023. Accessed: 2025-04-06. [↗](https://github.com/rubik/radon)
6. Chen, J., Zhang, B., Ma, R., Wang, P., Liang, X., Tu, Z., Li, X., and Wong, K.-Y. K. Spc: Evolving self-play critic via adversarial games for llm reasoning, 2025. URL https://arxiv.org/abs/2504.19162. — [arXiv:2504.19162](https://arxiv.org/abs/2504.19162) [↗](https://arxiv.org/abs/2504.19162)
7. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., et al. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). CoRR, abs/2107.03374, 2021. URL https://arxiv.org/abs/2107.03374. [↗](https://arxiv.org/abs/2107.03374)
8. Chen, Z., Deng, Y., Yuan, H., Ji, K., and Gu, Q. Self-play fine-tuning converts weak language models to strong language models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. [find](https://scholar.google.com/scholar?q=Chen%2C+Z%2E%2C+Deng%2C+Y%2E%2C+Yuan%2C+H%2E%2C+Ji%2C+K%2E%2C+and+Gu%2C+Q%2E+Self%2Dplay+fine%2Dtuning+converts+weak+language+models+to+strong+language+models%2E+In+Forty%2Dfirst+International+Conference+on+Machine+Learning%2C+ICML+2024%2C+)
9. OpenReview.net, 2024. URL [https://openreview.net/forum?id=O4cHTxW9BS](https://openreview.net/forum?id=O4cHTxW9BS). [↗](https://openreview.net/forum?id=O4cHTxW9BS)
10. Cheng, P., Hu, T., Xu, H., Zhang, Z., Dai, Y., Han, L., Du, N., and Li, X. Self-playing adversarial language game enhances LLM reasoning. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annu… [find](https://scholar.google.com/scholar?q=Cheng%2C+P%2E%2C+Hu%2C+T%2E%2C+Xu%2C+H%2E%2C+Zhang%2C+Z%2E%2C+Dai%2C+Y%2E%2C+Han%2C+L%2E%2C+Du%2C+N%2E%2C+and+Li%2C+X%2E+Self%2Dplaying+adversarial+language+game+enhances+LLM+reasoning%2E+In+Globersons%2C+A%2E%2C+Mackey%2C+L%2E%2C+Belgrave%2C+D%2E%2C+Fan%2C+A%2E%2C+Paquet%2C+)
11. Christiano, P. Approval-directed bootstrapping. [https://www.alignmentforum.org/posts/6x7oExXi32ot6HjJv/](https://www.alignmentforum.org/posts/6x7oExXi32ot6HjJv/) approval-directed-bootstrapping, 2018. AI Alignment Forum. [↗](https://www.alignmentforum.org/posts/6x7oExXi32ot6HjJv/)
12. Christiano, P. Capability amplification. [https://www.alignmentforum.org/posts/t3AJW5jP3sk36aGoC/](https://www.alignmentforum.org/posts/t3AJW5jP3sk36aGoC/) capability-amplification-1, 2019. AI Alignment Forum. [↗](https://www.alignmentforum.org/posts/t3AJW5jP3sk36aGoC/)
13. Colas, C., Karch, T., Moulin-Frier, C., and Oudeyer, P. Language and culture internalization for human-like autotelic AI. Nat. Mac. Intell., 4(12):1068–1076, 2022a. doi: 10.1038/S42256-022-00591-4. URL https://doi.org/10.1038/s42256-022-00591-4. — [doi:10.1038/s42256-022-00591-4](https://doi.org/10.1038/s42256-022-00591-4) [↗](https://doi.org/10.1038/s42256-022-00591-4)
14. Colas, C., Karch, T., Sigaud, O., and Oudeyer, P. Autotelic agents with intrinsically motivated goal-conditioned reinforcement learning: A short survey. J. Artif. Intell. Res., 74:1159–1199, 2022b. doi: 10.1613/JAIR.1.13554. URL https://doi.org/10.1613/jair. 1.13554. [↗](https://doi.org/10.1613/jair)
15. Cui, G., Yuan, L., Wang, Z., Wang, H., Li, W., He, B., Fan, Y., Yu, T., Xu, Q., Chen, W., Yuan, J., Chen, H., Zhang, K., Lv, X., Wang, S., Yao, Y., Han, X., Peng, H., Cheng, Y., Liu, Z., Sun, M., Zhou, B., and Ding, N. [Process reinforcement through implicit rewards](https://md.research.elamur.ai/free/en/papers/cui-2025-process-reinforcement-through-implicit-rewards/). CoRR, abs/2502.01456, 2025. doi:…
16. Demski, A. and Garrabrant, S. Embedded agency. CoRR, abs/1902.09469, 2019. URL http://arxiv.org/abs/1902.09469. — [arXiv:1902.09469](https://arxiv.org/abs/1902.09469) [↗](http://arxiv.org/abs/1902.09469)
17. Dennis, M., Jaques, N., Vinitsky, E., Bayen, A. M., Russell, S., Critch, A., and Levine, S. Emergent complexity and zero-shot transfer via unsupervised environment design. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems 3… [find](https://scholar.google.com/scholar?q=Dennis%2C+M%2E%2C+Jaques%2C+N%2E%2C+Vinitsky%2C+E%2E%2C+Bayen%2C+A%2E+M%2E%2C+Russell%2C+S%2E%2C+Critch%2C+A%2E%2C+and+Levine%2C+S%2E+Emergent+complexity+and+zero%2Dshot+transfer+via+unsupervised+environment+design%2E+In+Larochelle%2C+H%2E%2C+Ranzato%2C+)
18. Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A., Rozière, B., Biron, B., Tang… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
19. Ebert, C., Cain, J., Antoniol, G., Counsell, S., and Laplante, P. Cyclomatic complexity. IEEE software, 33(6):27–29, 2016. [find](https://scholar.google.com/scholar?q=Ebert%2C+C%2E%2C+Cain%2C+J%2E%2C+Antoniol%2C+G%2E%2C+Counsell%2C+S%2E%2C+and+Laplante%2C+P%2E+Cyclomatic+complexity%2E+IEEE+software%2C+33%286%29%3A27%E2%80%9329%2C+2016%2E)
20. Florensa, C., Held, D., Geng, X., and Abbeel, P. [Automatic goal generation for reinforcement learning agents](https://md.research.elamur.ai/free/en/papers/florensa-2017-automatic-goal-generation-reinforcement-learning/). In Dy, J. G. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proce…
21. PMLR, 2018. URL [http://proceedings.mlr.press/v80/florensa18a.html](http://proceedings.mlr.press/v80/florensa18a.html). [↗](http://proceedings.mlr.press/v80/florensa18a.html)
22. Gaven, L., Carta, T., Romac, C., Colas, C., Lamprier, S., Sigaud, O., and Oudeyer, P. MAGELLAN: metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces. CoRR, abs/2502.07709, 2025. doi: 10.48550/ARXIV.2502.07709. URL https://doi.org/10.48550/arXiv.2502.07709. [↗](https://doi.org/10.48550/arXiv.2502.07709)
23. Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. C., and Bengio, Y. Generative adversarial networks. Commun. ACM, 63(11):139–144, 2020. doi: 10.1145/3422622. URL https://doi.org/10.1145/3422622. — [doi:10.1145/3422622](https://doi.org/10.1145/3422622) [↗](https://doi.org/10.1145/3422622)
24. Gu, A., Rozière, B., Leather, H. J., Solar-Lezama, A., Synnaeve, G., and Wang, S. [Cruxeval: A benchmark for code reasoning, understanding and execution](https://md.research.elamur.ai/free/en/papers/gu-2024-cruxeval-benchmark-code-reasoning-understanding/). In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024.
25. OpenReview.net, 2024. URL [https://openreview.net/forum?id=Ffpg52swvg](https://openreview.net/forum?id=Ffpg52swvg). [↗](https://openreview.net/forum?id=Ffpg52swvg)
26. Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., Zhang, X., Yu, X., Wu, Y., Wu, Z. F., Gou, Z., Shao, Z., Li, Z., Gao, Z., Liu, A., Xue, B., Wang, B., Wu, B., Feng, B., Lu, C., Zhao, C., Deng, C., Ruan, C., Dai, D., Chen, D., Ji, D., Li, E., Lin, F., Dai,… — [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
27. Halstead, M. H. Elements of Software Science (Operating and programming systems series). Elsevier Science Inc., 1977. [find](https://scholar.google.com/scholar?q=Halstead%2C+M%2E+H%2E+Elements+of+Software+Science+%28Operating+and+programming+systems+series%29%2E+Elsevier+Science+Inc%2E%2C+1977%2E)
28. Haluptzok, P., Bowers, M., and Kalai, A. T. Language models can teach themselves to program better. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. [find](https://scholar.google.com/scholar?q=Haluptzok%2C+P%2E%2C+Bowers%2C+M%2E%2C+and+Kalai%2C+A%2E+T%2E+Language+models+can+teach+themselves+to+program+better%2E+In+The+Eleventh+International+Conference+on+Learning+Representations%2C+ICLR+2023%2C+Kigali%2C+Rwanda%2C+May)
29. OpenReview.net, 2023. URL [https://openreview.net/forum?id=SaRj2ka1XZ3](https://openreview.net/forum?id=SaRj2ka1XZ3). [↗](https://openreview.net/forum?id=SaRj2ka1XZ3)
30. He, C., Luo, R., Bai, Y., Hu, S., Thai, Z. L., Shen, J., Hu, J., Han, X., Huang, Y., Zhang, Y., Liu, J., Qi, L., Liu, Z., and Sun, M. [Olympiadbench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). In Ku, L., Martins, A., and Srikumar, V. (eds.)…
31. Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. [Measuring mathematical problem solving with the MATH dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In Vanschoren, J. and Yeung, S. (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, Neu…
32. Hinton, G. E., Vinyals, O., and Dean, J. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). CoRR, abs/1503.02531, 2015. URL http://arxiv.org/abs/1503.02531. [↗](http://arxiv.org/abs/1503.02531)
33. Hu, J. REINFORCE++: A simple and efficient approach for aligning large language models. CoRR, abs/2501.[03262, 2025](https://md.research.elamur.ai/free/en/papers/hu-2025-reinforce-stabilizing-critic-free-policy/). doi: 10.48550/ARXIV.2501.03262. URL https://doi.org/10.48550/arXiv.2501.03262. [↗](https://doi.org/10.48550/arXiv.2501.03262)
34. Hu, J., Zhang, Y., Han, Q., Jiang, D., Zhang, X., and Shum, H. [Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model](https://md.research.elamur.ai/free/en/papers/hu-2025-open-reasoner-zero-open-source/). CoRR, abs/2503.24290, 2025. doi: 10.48550/ARXIV.2503.24290. URL https://doi.org/10.48550/arXiv.2503.24290. [↗](https://doi.org/10.48550/arXiv.2503.24290)
35. Hubinger, E., van Merwijk, C., Mikulik, V., Skalse, J., and Garrabrant, S. Risks from learned optimization in advanced machine learning systems. CoRR, abs/1906.01820, 2019. URL http://arxiv.org/abs/1906.01820. — [arXiv:1906.01820](https://arxiv.org/abs/1906.01820) [↗](http://arxiv.org/abs/1906.01820)
36. Hughes, E., Dennis, M. D., Parker-Holder, J., Behbahani, F. M. P., Mavalankar, A., Shi, Y., Schaul, T., and Rocktäschel, T. Position: Open-endedness is essential for artificial superhuman intelligence. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-2… [find](https://scholar.google.com/scholar?q=Hughes%2C+E%2E%2C+Dennis%2C+M%2E+D%2E%2C+Parker%2DHolder%2C+J%2E%2C+Behbahani%2C+F%2E+M%2E+P%2E%2C+Mavalankar%2C+A%2E%2C+Shi%2C+Y%2E%2C+Schaul%2C+T%2E%2C+and+Rockt%C3%A4schel%2C+T%2E+Position%3A+Open%2Dendedness+is+essential+for+artificial+superhuman+intelligence)
37. OpenReview.net, 2024. URL [https://openreview.net/forum?id=Bc4vZ2CX7E](https://openreview.net/forum?id=Bc4vZ2CX7E). [↗](https://openreview.net/forum?id=Bc4vZ2CX7E)
38. Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Dang, K., Yang, A., Men, R., Huang, F., Ren, X., Ren, X., Zhou, J., and Lin, J. [Qwen2.5-coder technical report](https://md.research.elamur.ai/free/en/papers/hui-2024-qwen2-5-coder-technical-report/). CoRR, abs/2409.12186, 2024. doi: 10.48550/ARXIV.2409.12186. URL https://doi.org/10.48550/arXiv.2409.12… [↗](https://doi.org/10.48550/arXiv.2409.12…)
39. Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., Beutel, A., Carney, A., et al. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). arXiv preprint arXiv:2412.16720, 2024.
40. Jain, N., Han, K., Gu, A., Li, W., Yan, F., Zhang, T., Wang, S., Solar-Lezama, A., Sen, K., and Stoica, I. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). CoRR, abs/2403.07974, 2024. doi: 10.48550/ARXIV.2403.07974. URL https://doi.org/10.48550/arXiv.2403.… [↗](https://doi.org/10.48550/arXiv.2403.…)
41. Kirchner, J. H., Chen, Y., Edwards, H., Leike, J., McAleese, N., and Burda, Y. Prover-verifier games improve legibility of LLM outputs. CoRR, abs/2407.13692, 2024. doi: 10.48550/ARXIV.2407.13692. URL https://doi.org/10.48550/arXiv.2407.13692. — [doi:10.48550/arxiv.2407.13692](https://doi.org/10.48550/arxiv.2407.13692) [↗](https://doi.org/10.48550/arXiv.2407.13692)
42. Ladosz, P., Weng, L., Kim, M., and Oh, H. Exploration in deep reinforcement learning: A survey. Inf. Fusion, 85:1–22, 2022. doi: 10.1016/J.INFFUS.2022.03.003. URL https://doi.org/10.1016/j.inffus.2022.03.003. — [doi:10.1016/j.inffus.2022.03.003](https://doi.org/10.1016/j.inffus.2022.03.003) [↗](https://doi.org/10.1016/j.inffus.2022.03.003)
43. Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F., Miranda, L. J. V., Liu, A., Dziri, N., Lyu, S., Gu, Y., Malik, S., Graf, V., Hwang, J. D., Yang, J., Bras, R. L., Tafjord, O., Wilhelm, C., Soldaini, L., Smith, N. A., Wang, Y., Dasigi, P., and Hajishirzi, H. Tülu 3: Pushing… — [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
44. Laskin, M., Yarats, D., Liu, H., Lee, K., Zhan, A., Lu, K., Cang, C., Pinto, L., and Abbeel, P. URLB: unsupervised reinforcement learning benchmark. In Vanschoren, J. and Yeung, S. (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets a… [find](https://scholar.google.com/scholar?q=Laskin%2C+M%2E%2C+Yarats%2C+D%2E%2C+Liu%2C+H%2E%2C+Lee%2C+K%2E%2C+Zhan%2C+A%2E%2C+Lu%2C+K%2E%2C+Cang%2C+C%2E%2C+Pinto%2C+L%2E%2C+and+Abbeel%2C+P%2E+URLB%3A+unsupervised+reinforcement+learning+benchmark%2E+In+Vanschoren%2C+J%2E+and+Yeung%2C+S%2E+%28eds%2E%29%2C+Proceedings)
45. Leike, J. and Sutskever, I. Introducing superalignment. [https://openai.com/index/introducing-superalignment/](https://openai.com/index/introducing-superalignment/), 2023. OpenAI Blog. [↗](https://openai.com/index/introducing-superalignment/)
46. Lewkowycz, A., Andreassen, A., Dohan, D., Dyer, E., Michalewski, H., Ramasesh, V. V., Slone, A., Anil, C., Schlag, I., Gutman-Solo, T., Wu, Y., Neyshabur, B., Gur-Ari, G., and Misra, V. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D…
47. Li, J., Guo, D., Yang, D., Xu, R., Wu, Y., and He, J. [Codei/o: Condensing reasoning patterns via code input-output prediction](https://md.research.elamur.ai/free/en/papers/li-2025-codei-o-condensing-reasoning/). CoRR, abs/2502.07316, 2025. doi: 10.48550/ARXIV.2502.07316. URL https://doi.org/10.48550/arXiv.2502.07316. [↗](https://doi.org/10.48550/arXiv.2502.07316)
48. Li, R., Fu, J., Zhang, B., Huang, T., Sun, Z., Lyu, C., Liu, G., Jin, Z., and Li, G. [TACO: topics in algorithmic code generation dataset](https://md.research.elamur.ai/free/en/papers/li-2023-taco-topics-algorithmic-code-generation/). CoRR, abs/2312.14852, 2023. doi: 10.48550/ARXIV.2312.14852. URL https://doi.org/10.48550/arXiv.2312.14852. [↗](https://doi.org/10.48550/arXiv.2312.14852)
49. Liu, B., Guertler, L., Yu, S., Liu, Z., Qi, P., Balcells, D., Liu, M., Tan, C., Shi, W., Lin, M., Lee, W. S., and Jaques, N. SPIRAL: self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning. CoRR, abs/2506.24119, 2025a. doi: 10.48550/ARXIV.2506.24119. URL… — [doi:10.48550/arxiv.2506.24119](https://doi.org/10.48550/arxiv.2506.24119) [find](https://scholar.google.com/scholar?q=Liu%2C+B%2E%2C+Guertler%2C+L%2E%2C+Yu%2C+S%2E%2C+Liu%2C+Z%2E%2C+Qi%2C+P%2E%2C+Balcells%2C+D%2E%2C+Liu%2C+M%2E%2C+Tan%2C+C%2E%2C+Shi%2C+W%2E%2C+Lin%2C+M%2E%2C+Lee%2C+W%2E+S%2E%2C+and+Jaques%2C+N%2E+SPIRAL%3A+self%2Dplay+on+zero%2Dsum+games+incentivizes+reasoning+via+multi%2Dagent+)
50. Liu, J. and Zhang, L. Code-r1: Reproducing r1 for code with reliable rewards. GitHub, 2025. [find](https://scholar.google.com/scholar?q=Liu%2C+J%2E+and+Zhang%2C+L%2E+Code%2Dr1%3A+Reproducing+r1+for+code+with+reliable+rewards%2E+GitHub%2C+2025%2E)
51. Liu, J., Xia, C. S., Wang, Y., and Zhang, L. [Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL [https://openreview.net/forum?id=1qvx610Cu7](https://openreview.net/forum?id=1qvx610Cu7). [↗](https://openreview.net/forum?id=1qvx610Cu7)
52. Liu, M., Jiang, L., Liang, Y., Du, S. S., Choi, Y., Althoff, T., and Jaques, N. Chasing moving targets with online self-play reinforcement learning for safer language models. CoRR, abs/2506.07468, 2025b. doi: 10.48550/ARXIV.2506.07468. URL https://doi.org/10.48550/arXiv.2506.07468. — [doi:10.48550/arxiv.2506.07468](https://doi.org/10.48550/arxiv.2506.07468) [↗](https://doi.org/10.48550/arXiv.2506.07468)
53. Liu, Z., Chen, C., Li, W., Qi, P., Pang, T., Du, C., Lee, W. S., and Lin, M. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/). CoRR, abs/2503.20783, 2025c. doi: 10.48550/ARXIV.2503.20783. URL https://doi.org/10.48550/arXiv.2503.20783. [↗](https://doi.org/10.48550/arXiv.2503.20783)
54. Lopez, R. H. Q. Complexipy: An extremely fast python library to calculate the cognitive complexity of python files, written in rust, 2025. URL [https://github.com/rohaquinlop/complexipy](https://github.com/rohaquinlop/complexipy). Accessed: 2025-04-06. [↗](https://github.com/rohaquinlop/complexipy)
55. Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. [find](https://scholar.google.com/scholar?q=Loshchilov%2C+I%2E+and+Hutter%2C+F%2E+Decoupled+weight+decay+regularization%2E+In+7th+International+Conference+on+Learning+Representations%2C+ICLR+2019%2C+New+Orleans%2C+LA%2C+USA%2C+May+6%2D9%2C+2019%2E)
56. OpenReview.net, 2019. URL [https://openreview.net/forum?id=Bkg6RiCqY7](https://openreview.net/forum?id=Bkg6RiCqY7). [↗](https://openreview.net/forum?id=Bkg6RiCqY7)
57. Morris, J. There are no new ideas in ai. . . only new datasets. [https://blog.jxmo.io/p/there-are-no-new-ideas-in-ai-only](https://blog.jxmo.io/p/there-are-no-new-ideas-in-ai-only), 2025. [↗](https://blog.jxmo.io/p/there-are-no-new-ideas-in-ai-only)
58. OpenAI. Openai o3-mini, January 2025a. URL [https://openai.com/index/openai-o3-mini/](https://openai.com/index/openai-o3-mini/). Accessed: 2025-04-17. [↗](https://openai.com/index/openai-o3-mini/)
59. OpenAI. Introducing openai o3 and o4-mini, April 2025b. URL [https://openai.com/index/introducing-o3-and-o4-mini/](https://openai.com/index/introducing-o3-and-o4-mini/). Accessed: 2025-04-17. [↗](https://openai.com/index/introducing-o3-and-o4-mini/)
60. OpenAI, Plappert, M., Sampedro, R., Xu, T., Akkaya, I., Kosaraju, V., Welinder, P., D’Sa, R., Petron, A., de Oliveira Pinto, H. P., Paino, A., Noh, H., Weng, L., Yuan, Q., Chu, C., and Zaremba, W. Asymmetric self-play for automatic goal discovery in robotic manipulation. CoRR, abs/2101.04882, 2021.… [find](https://scholar.google.com/scholar?q=OpenAI%2C+Plappert%2C+M%2E%2C+Sampedro%2C+R%2E%2C+Xu%2C+T%2E%2C+Akkaya%2C+I%2E%2C+Kosaraju%2C+V%2E%2C+Welinder%2C+P%2E%2C+D%E2%80%99Sa%2C+R%2E%2C+Petron%2C+A%2E%2C+de+Oliveira+Pinto%2C+H%2E+P%2E%2C+Paino%2C+A%2E%2C+Noh%2C+H%2E%2C+Weng%2C+L%2E%2C+Yuan%2C+Q%2E%2C+Chu%2C+C%2E%2C+and+Zaremba%2C+W%2E+Asy)
61. Oudeyer, P.-Y., Gottlieb, J., and Lopes, M. Intrinsic motivation, curiosity, and learning: Theory and applications in educational technologies. Progress in brain research, 229:257–284, 2016. [find](https://scholar.google.com/scholar?q=Oudeyer%2C+P%2E%2DY%2E%2C+Gottlieb%2C+J%2E%2C+and+Lopes%2C+M%2E+Intrinsic+motivation%2C+curiosity%2C+and+learning%3A+Theory+and+applications+in+educational+technologies%2E+Progress+in+brain+research%2C+229%3A257%E2%80%93284%2C+2016%2E)
62. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744, 2022.
63. Poesia, G., Broman, D., Haber, N., and Goodman, N. D. Learning formal mathematics from intrinsic motivation. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Infor… [find](https://scholar.google.com/scholar?q=Poesia%2C+G%2E%2C+Broman%2C+D%2E%2C+Haber%2C+N%2E%2C+and+Goodman%2C+N%2E+D%2E+Learning+formal+mathematics+from+intrinsic+motivation%2E+In+Globersons%2C+A%2E%2C+Mackey%2C+L%2E%2C+Belgrave%2C+D%2E%2C+Fan%2C+A%2E%2C+Paquet%2C+U%2E%2C+Tomczak%2C+J%2E+M%2E%2C+and+Zhang)
64. Pourcel, J., Colas, C., Molinaro, G., Oudeyer, P., and Teodorescu, L. ACES: generating a diversity of challenging programming puzzles with autotelic generative models. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Informati… [find](https://scholar.google.com/scholar?q=Pourcel%2C+J%2E%2C+Colas%2C+C%2E%2C+Molinaro%2C+G%2E%2C+Oudeyer%2C+P%2E%2C+and+Teodorescu%2C+L%2E+ACES%3A+generating+a+diversity+of+challenging+programming+puzzles+with+autotelic+generative+models%2E+In+Globersons%2C+A%2E%2C+Mackey%2C+L%2E%2C+B)
65. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. [find](https://scholar.google.com/scholar?q=Radford%2C+A%2E%2C+Wu%2C+J%2E%2C+Child%2C+R%2E%2C+Luan%2C+D%2E%2C+Amodei%2C+D%2E%2C+Sutskever%2C+I%2E%2C+et+al%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2C+2019%2E)
66. Ren, Z. Z., Shao, Z., Song, J., Xin, H., Wang, H., Zhao, W., Zhang, L., Fu, Z., Zhu, Q., Yang, D., Wu, Z. F., Gou, Z., Ma, S., Tang, H., Liu, Y., Gao, W., Guo, D., and Ruan, C. [Deepseek-prover-v2: Advancing formal mathematical reasoning via reinforcement learning for subgoal decomposition](https://md.research.elamur.ai/free/en/papers/ren-2025-deepseek-prover-v2-advancing-formal/), 2025. URL…
67. Schaul, T. Boundless socratic learning with language games. arXiv preprint arXiv:2411.16905, 2024. — [arXiv:2411.16905](https://arxiv.org/abs/2411.16905) [find](https://scholar.google.com/scholar?q=Schaul%2C+T%2E+Boundless+socratic+learning+with+language+games%2E+arXiv+preprint+arXiv%3A2411%2E16905%2C+2024%2E+%E2%80%94+%5BarXiv%3A2411%2E16905%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2411%2E16905%29)
68. Schmidhuber, J. Exploring the predictable. In Advances in evolutionary computing: theory and applications, pp. 579–612. Springer, 2003. [find](https://scholar.google.com/scholar?q=Schmidhuber%2C+J%2E+Exploring+the+predictable%2E+In+Advances+in+evolutionary+computing%3A+theory+and+applications%2C+pp%2E+579%E2%80%93612%2E+Springer%2C+2003%2E)
69. Schmidhuber, J. POWERPLAY: training an increasingly general problem solver by continually searching for the simplest still unsolvable problem. CoRR, abs/1112.5309, 2011. URL http://arxiv.org/abs/1112.5309. — [arXiv:1112.5309](https://arxiv.org/abs/1112.5309) [↗](http://arxiv.org/abs/1112.5309)
70. Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). CoRR, abs/2402.03300, 2024. doi: 10.48550/ARXIV.2402.03300. URL https://doi.org/10.48550/arXiv.2402.03300. [↗](https://doi.org/10.48550/arXiv.2402.03300)
71. Sheng, G., Zhang, C., Ye, Z., Wu, X., Zhang, W., Zhang, R., Peng, Y., Lin, H., and Wu, C. [Hybridflow: A flexible and efficient RLHF framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). In Proceedings of the Twentieth European Conference on Computer Systems, EuroSys 2025, Rotterdam, The Netherlands, 30 March 2025 - 3 April 2025, pp. 1279–129…
72. ACM, 2025. [doi: 10.1145/3689031.3696075](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). URL https://doi.org/10.1145/3689031.3696075. [↗](https://doi.org/10.1145/3689031.3696075)
73. Silver, D. and Sutton, R. S. The era of experience. [https://storage.googleapis.com/deepmind-media/Era-of-Experience%](https://storage.googleapis.com/deepmind-media/Era-of-Experience%) 20/The%20Era%20of%20Experience%20Paper.pdf, 2025. [↗](https://storage.googleapis.com/deepmind-media/Era-of-Experience%)
74. Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T. P., Leach, M., Kavukcuoglu, K., Graepel, T., and Hassabis, D. Mast… [find](https://scholar.google.com/scholar?q=Silver%2C+D%2E%2C+Huang%2C+A%2E%2C+Maddison%2C+C%2E+J%2E%2C+Guez%2C+A%2E%2C+Sifre%2C+L%2E%2C+van+den+Driessche%2C+G%2E%2C+Schrittwieser%2C+J%2E%2C+Antonoglou%2C+I%2E%2C+Panneershelvam%2C+V%2E%2C+Lanctot%2C+M%2E%2C+Dieleman%2C+S%2E%2C+Grewe%2C+D%2E%2C+Nham%2C+J%2E%2C+Kalchbrenner%2C)
75. Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T. P., Simonyan, K., and Hassabis, D. [Mastering chess and shogi by self-play with a general reinforcement learning algorithm](https://md.research.elamur.ai/free/en/papers/al-2017-mastering-chess-shogi-self-play/). CoRR, abs/1712.01815, 2017. URL htt…
76. Snell, C., Lee, J., Xu, K., and Kumar, A. [Scaling LLM test-time compute optimally can be more effective than scaling model parameters](https://md.research.elamur.ai/free/en/papers/snell-2024-scaling-llm-test-time-compute/). CoRR, abs/2408.03314, 2024. doi: 10.48550/ARXIV.2408.03314. URL https://doi.org/10.48550/arXiv.2408.03314. [↗](https://doi.org/10.48550/arXiv.2408.03314)
77. Stuart, T. Understanding computation - from simple machines to impossible programs. O’Reilly, 2015. ISBN 978-1-449-32927-3. URL [http://www.oreilly.de/catalog/9781449329273/index.html](http://www.oreilly.de/catalog/9781449329273/index.html). [↗](http://www.oreilly.de/catalog/9781449329273/index.html)
78. Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., and Fergus, R. Intrinsic motivation and automatic curricula via asymmetric self-play. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 -May 3, 2018, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Sukhbaatar%2C+S%2E%2C+Lin%2C+Z%2E%2C+Kostrikov%2C+I%2E%2C+Synnaeve%2C+G%2E%2C+Szlam%2C+A%2E%2C+and+Fergus%2C+R%2E+Intrinsic+motivation+and+automatic+curricula+via+asymmetric+self%2Dplay%2E+In+6th+International+Conference+on+Learning+Repre)
79. OpenReview.net, 2018. URL [https://openreview.net/forum?id=SkT5Yg-RZ](https://openreview.net/forum?id=SkT5Yg-RZ). [↗](https://openreview.net/forum?id=SkT5Yg-RZ)
80. Suteu, M. and Guo, Y. Regularizing deep multi-task networks using orthogonal gradients. CoRR, abs/1912.06844, 2019. URL http://arxiv.org/abs/1912.06844. — [arXiv:1912.06844](https://arxiv.org/abs/1912.06844) [↗](http://arxiv.org/abs/1912.06844)
81. Sutskever, I., Vinyals, O., and Le, Q. V. Neurips 2024 test of time award session: Sequence to sequence learning with neural networks. Conference session, December 2024. URL [https://neurips.cc/virtual/2024/test-of-time/105032](https://neurips.cc/virtual/2024/test-of-time/105032). [↗](https://neurips.cc/virtual/2024/test-of-time/105032)
82. Sutton, R. S. Verification, the key to ai. [http://incompleteideas.net/IncIdeas/KeytoAI.html](http://incompleteideas.net/IncIdeas/KeytoAI.html), 2001. [↗](http://incompleteideas.net/IncIdeas/KeytoAI.html)
83. Team, K., Du, A., Gao, B., Xing, B., Jiang, C., Chen, C., Li, C., Xiao, C., Du, C., Liao, C., Tang, C., Wang, C., Zhang, D., Yuan, E., Lu, E., Tang, F., Sung, F., Wei, G., Lai, G., Guo, H., Zhu, H., Ding, H., Hu, H., Yang, H., Zhang, H., Yao, H., Zhao, H., Lu, H., Li, H., Yu, H., Gao, H., Zheng, H.,… — [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/)
84. Villalobos, P., Ho, A., Sevilla, J., Besiroglu, T., Heim, L., and Hobbhahn, M. Position: Will we run out of data? limits of LLM scaling based on human-generated data. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. [find](https://scholar.google.com/scholar?q=Villalobos%2C+P%2E%2C+Ho%2C+A%2E%2C+Sevilla%2C+J%2E%2C+Besiroglu%2C+T%2E%2C+Heim%2C+L%2E%2C+and+Hobbhahn%2C+M%2E+Position%3A+Will+we+run+out+of+data%3F+limits+of+LLM+scaling+based+on+human%2Dgenerated+data%2E+In+Forty%2Dfirst+International+Conf)
85. OpenReview.net, 2024. URL [https://openreview.net/forum?id=ViZcgDQjyG](https://openreview.net/forum?id=ViZcgDQjyG). [↗](https://openreview.net/forum?id=ViZcgDQjyG)
86. Wang, H., Yue, Y., Lu, R., Shi, J., Zhao, A., Wang, S., Song, S., and Huang, G. Model surgery: Modulating LLM‘s behavior via simple parameter editing. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics, pp. 6337–6357, 2025a. [find](https://scholar.google.com/scholar?q=Wang%2C+H%2E%2C+Yue%2C+Y%2E%2C+Lu%2C+R%2E%2C+Shi%2C+J%2E%2C+Zhao%2C+A%2E%2C+Wang%2C+S%2E%2C+Song%2C+S%2E%2C+and+Huang%2C+G%2E+Model+surgery%3A+Modulating+LLM%E2%80%98s+behavior+via+simple+parameter+editing%2E+In+Proceedings+of+the+2025+Conference+of+the+Nati)
87. Wang, R., Lehman, J., Clune, J., and Stanley, K. O. Paired open-ended trailblazer (POET): endlessly generating increasingly complex and diverse learning environments and their solutions. CoRR, abs/1901.01753, 2019. URL http://arxiv.org/abs/1901.01753. [↗](http://arxiv.org/abs/1901.01753)
88. Wang, S., Yang, Q., Gao, J., Lin, M. G., Chen, H., Wu, L., Jia, N., Song, S., and Huang, G. Train once, get a family: State-adaptive balances for offline-to-online reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL [https://openreview.net/forum?id…](https://openreview.net/forum?id…) [↗](https://openreview.net/forum?id…)
89. Wang, S., Liu, C., Zheng, Z., Qi, S., Chen, S., Yang, Q., Zhao, A., Wang, C., Song, S., and Huang, G. Boosting LLM agents with recursive contemplation for effective deception handling. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Findings of the Association for Computational Linguistics: ACL… [find](https://scholar.google.com/scholar?q=Wang%2C+S%2E%2C+Liu%2C+C%2E%2C+Zheng%2C+Z%2E%2C+Qi%2C+S%2E%2C+Chen%2C+S%2E%2C+Yang%2C+Q%2E%2C+Zhao%2C+A%2E%2C+Wang%2C+C%2E%2C+Song%2C+S%2E%2C+and+Huang%2C+G%2E+Boosting+LLM+agents+with+recursive+contemplation+for+effective+deception+handling%2E+In+Ku%2C+L%2E%2DW%2E%2C+M)
90. Wang, S., Yu, L., Gao, C., Zheng, C., Liu, S., Lu, R., Dang, K., Chen, X., Yang, J., Zhang, Z., Liu, Y., Yang, A., Zhao, A., Yue, Y., Song, S., Yu, B., Huang, G., and Lin, J. [Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for LLM reasoning](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/). CoRR, abs/2506.…
91. Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., Li, T., Ku, M., Wang, K., Zhuang, A., Fan, R., Yue, X., and Chen, W. [Mmlu-pro: A more robust and challenging multi-task language understanding benchmark](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/). In Globersons, A., Mackey, L., Belgrave, D., F…
92. Wang, Y., Yang, Q., Zeng, Z., Ren, L., Liu, L., Peng, B., Cheng, H., He, X., Wang, K., Gao, J., Chen, W., Wang, S., Du, S. S., and Shen, Y. [Reinforcement learning for reasoning in large language models with one training example](https://md.research.elamur.ai/free/en/papers/wang-2025-reinforcement-learning-reasoning-large-language/), 2025c. URL https://arxiv.org/abs/2504.20571. [↗](https://arxiv.org/abs/2504.20571)
93. Wu, Q., Bansal, G., Zhang, J., Wu, Y., Zhang, S., Zhu, E., Li, B., Jiang, L., Zhang, X., and Wang, C. [Autogen: Enabling next-gen LLM applications via multi-agent conversation](https://md.research.elamur.ai/free/en/papers/wu-2023-autogen-enabling-next-gen-llm/) framework. CoRR, abs/2308.08155, 2023. doi: 10.48550/ARXIV.2308.08155. URL https://doi.org/10.48550/arXiv.2308.08155. [↗](https://doi.org/10.48550/arXiv.2308.08155)
94. Wu, Y., Yue, T., Zhang, S., Wang, C., and Wu, Q. [Stateflow: Enhancing LLM task-solving through state-driven workflows](https://md.research.elamur.ai/free/en/papers/wu-2024-stateflow-enhancing-llm-task-solving/). CoRR, abs/2403.11322, 2024. doi: 10.48550/ARXIV.2403.11322. URL https://doi.org/10.48550/arXiv.2403.11322. [↗](https://doi.org/10.48550/arXiv.2403.11322)
95. Xie, T., Gao, Z., Ren, Q., Luo, H., Hong, Y., Dai, B., Zhou, J., Qiu, K., Wu, Z., and Luo, C. [Logic-rl: Unleashing LLM reasoning with rule-based reinforcement learning](https://md.research.elamur.ai/free/en/papers/xie-2025-logic-rl-unleashing-llm/). CoRR, abs/2502.14768, 2025. doi: 10.48550/ARXIV.2502.14768. URL https://doi.org/10.48550/arXiv.2502.14768. [↗](https://doi.org/10.48550/arXiv.2502.14768)
96. Xu, F., Yan, H., Ma, C., Zhao, H., Sun, Q., Cheng, K., He, J., Liu, J., and Wu, Z. [Genius: A generalizable and purely unsupervised self-training framework for advanced reasoning](https://md.research.elamur.ai/free/en/papers/xu-2025-genius-generalizable-purely-unsupervised-self/), 2025. URL https://arxiv.org/abs/2504.08672. [↗](https://arxiv.org/abs/2504.08672)
97. Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, T., Xia, T., Ren, X., R… [find](https://scholar.google.com/scholar?q=Yang%2C+A%2E%2C+Yang%2C+B%2E%2C+Zhang%2C+B%2E%2C+Hui%2C+B%2E%2C+Zheng%2C+B%2E%2C+Yu%2C+B%2E%2C+Li%2C+C%2E%2C+Liu%2C+D%2E%2C+Huang%2C+F%2E%2C+Wei%2C+H%2E%2C+Lin%2C+H%2E%2C+Yang%2C+J%2E%2C+Tu%2C+J%2E%2C+Zhang%2C+J%2E%2C+Yang%2C+J%2E%2C+Yang%2C+J%2E%2C+Zhou%2C+J%2E%2C+Lin%2C+J%2E%2C+Dang%2C+K%2E%2C+Lu%2C+K%2E%2C+Bao%2C+K%2E%2C+)
98. Yang, A., Zhang, B., Hui, B., Gao, B., Yu, B., Li, C., Liu, D., Tu, J., Zhou, J., Lin, J., Lu, K., Xue, M., Lin, R., Liu, T., Ren, X., and Zhang, Z. [Qwen2.5-math technical report: Toward mathematical expert model via self-improvement](https://md.research.elamur.ai/free/en/papers/yang-2024-qwen2-5-math-technical-report/). CoRR, abs/2409.12122, 2024b. doi: 10.48550/ARXIV.2409.12122. URL…
99. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. R., and Cao, Y. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023.
100. OpenReview.net, 2023. URL [https://openreview.net/forum?id=WE_vluYUL-X](https://openreview.net/forum?id=WE_vluYUL-X). [↗](https://openreview.net/forum?id=WE_vluYUL-X)
101. Ye, Z., Agarwal, R., Liu, T., Joshi, R., Velury, S., Le, Q. V., Tan, Q., and Liu, Y. Evolving alignment via asymmetric self-play. CoRR, abs/2411.00062, 2024. doi: 10.48550/ARXIV.2411.00062. URL https://doi.org/10.48550/arXiv.2411.00062. [↗](https://doi.org/10.48550/arXiv.2411.00062)
102. Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Fan, T., Liu, G., Liu, L., Liu, X., Lin, H., Lin, Z., Ma, B., Sheng, G., Tong, Y., Zhang, C., Zhang, M., Zhang, W., Zhu, H., Zhu, J., Chen, J., Chen, J., Wang, C., Yu, H., Dai, W., Song, Y., Wei, X., Zhou, H., Liu, J., Ma, W., Zhang, Y., Yan, L… — [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/)
103. Yuan, W., Pang, R. Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., and Weston, J. Self-rewarding language models. URL https://arxiv.org/abs/2401.10020, 2024. [↗](https://arxiv.org/abs/2401.10020)
104. Yuan, Y., Yu, Q., Zuo, X., Zhu, R., Xu, W., Chen, J., Wang, C., Fan, T., Du, Z., Wei, X., et al. [Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks](https://md.research.elamur.ai/free/en/papers/yue-2025-vapo-efficient-reliable-reinforcement-learning/). arXiv preprint arXiv:2504.05118, 2025.
105. Yue, Y., Lu, R., Kang, B., Song, S., and Huang, G. Understanding, predicting and better resolving q-value divergence in offline-rl. Advances in Neural Information Processing Systems, 36:60247–60277, 2023. [find](https://scholar.google.com/scholar?q=Yue%2C+Y%2E%2C+Lu%2C+R%2E%2C+Kang%2C+B%2E%2C+Song%2C+S%2E%2C+and+Huang%2C+G%2E+Understanding%2C+predicting+and+better+resolving+q%2Dvalue+divergence+in+offline%2Drl%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%3A60247%E2%80%9360277%2C+2)
106. Yue, Y., Wang, Y., Kang, B., Han, Y., Wang, S., Song, S., Feng, J., and Huang, G. Deer-vla: Dynamic inference of multimodal large language models for efficient robot execution. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural… [find](https://scholar.google.com/scholar?q=Yue%2C+Y%2E%2C+Wang%2C+Y%2E%2C+Kang%2C+B%2E%2C+Han%2C+Y%2E%2C+Wang%2C+S%2E%2C+Song%2C+S%2E%2C+Feng%2C+J%2E%2C+and+Huang%2C+G%2E+Deer%2Dvla%3A+Dynamic+inference+of+multimodal+large+language+models+for+efficient+robot+execution%2E+In+Globersons%2C+A%2E%2C+Mack)
107. Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Yue, Y., Song, S., and Huang, G. [Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/), 2025. URL https://arxiv.org/abs/2504.13837. [↗](https://arxiv.org/abs/2504.13837)
108. Zelikman, E., Wu, Y., Mu, J., and Goodman, N. [Star: Bootstrapping reasoning with reasoning](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/). Advances in Neural Information Processing Systems, 35:15476–15488, 2022.
109. Zeng, H., Jiang, D., Wang, H., Nie, P., Chen, X., and Chen, W. ACECODER: acing coder RL via automated test-case synthesis. CoRR, abs/2502.01718, 2025a. doi: 10.48550/ARXIV.2502.01718. URL https://doi.org/10.48550/arXiv.2502.01718. [↗](https://doi.org/10.48550/arXiv.2502.01718)
110. Zeng, W., Huang, Y., Liu, Q., Liu, W., He, K., Ma, Z., and He, J. [Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/). CoRR, abs/2503.18892, 2025b. doi: 10.48550/ARXIV.2503.18892. URL https://doi.org/10.48550/arXiv.2503.18892. [↗](https://doi.org/10.48550/arXiv.2503.18892)
111. Zhang, C., Deng, Y., Lin, X., Wang, B., Ng, D., Ye, H., Li, X., Xiao, Y., Mo, Z., Zhang, Q., et al. [100 days after deepseek-r1: A survey on replication studies and more directions for reasoning language models](https://md.research.elamur.ai/free/en/papers/zhang-2025-100-days-after-deepseek/). arXiv preprint arXiv:2505.00551, 2025a.
112. Zhang, Q., Wu, H., Zhang, C., Zhao, P., and Bian, Y. Right question is already half the answer: Fully unsupervised llm reasoning incentivization, 2025b. URL https://arxiv.org/abs/2504.05812. [↗](https://arxiv.org/abs/2504.05812)
113. Zhang, Y. and Yang, Q. A survey on multi-task learning. IEEE transactions on knowledge and data engineering, 34(12):5586–5609, 2021. [find](https://scholar.google.com/scholar?q=Zhang%2C+Y%2E+and+Yang%2C+Q%2E+A+survey+on+multi%2Dtask+learning%2E+IEEE+transactions+on+knowledge+and+data+engineering%2C+34%2812%29%3A5586%E2%80%935609%2C+2021%2E)
114. Zhao, A., Lin, M. G., Li, Y., Liu, Y., and Huang, G. A mixture of surprises for unsupervised reinforcement learning. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Proces… [find](https://scholar.google.com/scholar?q=Zhao%2C+A%2E%2C+Lin%2C+M%2E+G%2E%2C+Li%2C+Y%2E%2C+Liu%2C+Y%2E%2C+and+Huang%2C+G%2E+A+mixture+of+surprises+for+unsupervised+reinforcement+learning%2E+In+Koyejo%2C+S%2E%2C+Mohamed%2C+S%2E%2C+Agarwal%2C+A%2E%2C+Belgrave%2C+D%2E%2C+Cho%2C+K%2E%2C+and+Oh%2C+A%2E+%28eds%2E%29%2C+)
115. Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y., and Huang, G. Expel: LLM agents are experiential learners. In Wooldridge, M. J., Dy, J. G., and Natarajan, S. (eds.), Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial I… [find](https://scholar.google.com/scholar?q=Zhao%2C+A%2E%2C+Huang%2C+D%2E%2C+Xu%2C+Q%2E%2C+Lin%2C+M%2E%2C+Liu%2C+Y%2E%2C+and+Huang%2C+G%2E+Expel%3A+LLM+agents+are+experiential+learners%2E+In+Wooldridge%2C+M%2E+J%2E%2C+Dy%2C+J%2E+G%2E%2C+and+Natarajan%2C+S%2E+%28eds%2E%29%2C+Thirty%2DEighth+AAAI+Conference+on+Ar)
116. AAAI Press, 2024. doi: 10.1609/AAAI.V38I17.29936. URL https://doi.org/10.1609/aaai.v38i17.29936. — [doi:10.1609/aaai.v38i17.29936](https://doi.org/10.1609/aaai.v38i17.29936) [↗](https://doi.org/10.1609/aaai.v38i17.29936)
117. Zhao, A., Xu, Q., Lin, M., Wang, S., Liu, Y., Zheng, Z., and Huang, G. Diver-ct: Diversity-enhanced red teaming large language model assistants with relaxing constraints. In Walsh, T., Shah, J., and Kolter, Z. (eds.), AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligen… [find](https://scholar.google.com/scholar?q=Zhao%2C+A%2E%2C+Xu%2C+Q%2E%2C+Lin%2C+M%2E%2C+Wang%2C+S%2E%2C+Liu%2C+Y%2E%2C+Zheng%2C+Z%2E%2C+and+Huang%2C+G%2E+Diver%2Dct%3A+Diversity%2Denhanced+red+teaming+large+language+model+assistants+with+relaxing+constraints%2E+In+Walsh%2C+T%2E%2C+Shah%2C+J%2E%2C+and+K)
118. AAAI Press, 2025a. doi: 10.1609/AAAI.V39I24.34797. URL https://doi.org/10.1609/aaai.v39i24.34797. — [doi:10.1609/aaai.v39i24.34797](https://doi.org/10.1609/aaai.v39i24.34797) [↗](https://doi.org/10.1609/aaai.v39i24.34797)
119. Zhao, A., Zhu, E., Lu, R., Lin, M., Liu, Y., and Huang, G. Self-referencing agents for unsupervised reinforcement learning. Neural Networks, 188:107448, 2025b. doi: 10.1016/J.NEUNET.2025.107448. URL https://doi.org/10.1016/j.neunet.2025. 107448. — [doi:10.1016/j.neunet.2025.107448](https://doi.org/10.1016/j.neunet.2025.107448) [↗](https://doi.org/10.1016/j.neunet.2025)
120. Zitkovich, B., Yu, T., Xu, S., Xu, P., Xiao, T., Xia, F., Wu, J., Wohlhart, P., Welker, S., Wahid, A., Vuong, Q., Vanhoucke, V., Tran, H. T., Soricut, R., Singh, A., Singh, J., Sermanet, P., Sanketi, P. R., Salazar, G., Ryoo, M. S., Reymann, K., Rao, K., Pertsch, K., Mordatch, I., Michalewski, H., L… [find](https://scholar.google.com/scholar?q=Zitkovich%2C+B%2E%2C+Yu%2C+T%2E%2C+Xu%2C+S%2E%2C+Xu%2C+P%2E%2C+Xiao%2C+T%2E%2C+Xia%2C+F%2E%2C+Wu%2C+J%2E%2C+Wohlhart%2C+P%2E%2C+Welker%2C+S%2E%2C+Wahid%2C+A%2E%2C+Vuong%2C+Q%2E%2C+Vanhoucke%2C+V%2E%2C+Tran%2C+H%2E+T%2E%2C+Soricut%2C+R%2E%2C+Singh%2C+A%2E%2C+Singh%2C+J%2E%2C+Sermanet%2C+P%2E%2C+Sanketi%2C+)
121. PMLR, 2023. URL [https://proceedings.mlr.press/v229/zitkovich23a.html](https://proceedings.mlr.press/v229/zitkovich23a.html). [↗](https://proceedings.mlr.press/v229/zitkovich23a.html)
122. Zuo, Y., Zhang, K., Qu, S., Sheng, L., Zhu, X., Qi, B., Sun, Y., Cui, G., Ding, N., and Zhou, B. [Ttrl: Test-time reinforcement learning](https://md.research.elamur.ai/free/en/papers/zuo-2025-ttrl-test-time-reinforcement-learning/), 2025. URL https://arxiv.org/abs/2504.16084. [↗](https://arxiv.org/abs/2504.16084)

*Updated 2 November 2026*