---
title: "Curriculum Reinforcement Learning from Easy to Hard Tasks"
description: "This replaces a fixed switch after a set number of iterations. 54 references, cited by 21 papers here."
authors: ["Shubham Parashar", "Shurui Gui", "Xiner Li", "Hongyi Ling", "Sushil Vemuri", "Blake Olson", "Eric Li", "Yu Zhang", "James Caverlee", "Dileep Kalathil", "Shuiwang Ji"]
year: 2025
venue: "arXiv:2506.06632"
source: "https://arxiv.org/abs/2506.06632"
doi: "10.48550/arxiv.2506.06632"
arxiv: "2506.06632"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/parashar-2025-curriculum-reinforcement-learning-easy-hard/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/parashar-2025-curriculum-reinforcement-learning-easy-hard/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/5/) / Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning

# Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning

**Authors:** Shubham Parashar, Shurui Gui, [Xiner Li](https://md.research.elamur.ai/free/en/authors/xiner-li/), Hongyi Ling, Sushil Vemuri, Blake Olson, Eric Li, [Yu Zhang](https://md.research.elamur.ai/free/en/authors/yu-zhang/), [James Caverlee](https://md.research.elamur.ai/free/en/authors/james-caverlee/), Dileep Kalathil and Shuiwang Ji · **Year:** 2025 · **Venue:** arXiv:2506.06632 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2506.06632) · [PDF](https://arxiv.org/pdf/2506.06632) · [DOI 10.48550/arxiv.2506.06632](https://doi.org/10.48550/arxiv.2506.06632) · [arXiv 2506.06632](https://arxiv.org/abs/2506.06632) · [OpenAlex](https://openalex.org/W4417119453)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [21](https://md.research.elamur.ai/free/en/papers/parashar-2025-curriculum-reinforcement-learning-easy-hard/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
RL post-training, as used for DeepSeek-R1, helps LLMs reason, but it struggles on hard problems. When a base model rarely solves a task zero-shot, correctness-only rewards are too sparse to learn from. The authors frame reasoning as generalization: learning principles on simpler problems and transferring them to harder ones. Their method, E2H Reasoner, splits training data into difficulty tiers (trivial, easy, medium, hard), using either human annotations or model performance. A probabilistic scheduler then moves the sampling focus gradually from easy to hard. This replaces a fixed switch after a set number of iterations.

A key empirical finding is that easy tasks matter early but must be phased out to avoid overfitting. On the theory side, the authors analyse curriculum RL as approximate policy iteration. They derive convergence guarantees and finite-sample bounds, and show that a well-designed curriculum needs fewer total samples than training directly on the target task.

Experiments use small instruct models on planning and math benchmarks. They show gains on hard and out-of-distribution splits, and that models learn tasks they originally failed. This matters for making small, accessible models reason better.

## Key points
- **Schedulers:** two variants, E2H-G (Gaussian, with two tunable parameters) and E2H-C (cosine, parameter-free). Both are compared with balanced sampling and a traditional easy-then-hard curriculum.
- **Benchmarks:** Blocksworld, Countdown, MATH, GSM8K and AQuA.
- **Models:** Qwen 2.5 1.5B/3B Instruct and LLaMA 3.2 3B Instruct.
- **Results for Qwen 2.5 1.5B:** E2H-G (0.5, 0.5) averages 78.7 on GSM8K versus 77.1 for balanced sampling, and 66.1 on AQuA versus 63.3. Results with 3, 4 or 5 difficulty splits are nearly identical (78.6 / 78.7 / 78.5), and 4 splits is the default.
- **Combination with DAPO:** adding E2H reduces the share of zero-advantage batches, meaning sampled problems better match the model's current competence.
- **Code:** released at github.com/divelab/E2H-Reasoning.
- **Limitation:** the schedulers are fixed and do not adapt during training. Adaptive, advantage-based scheduling is left for future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Texas A&M University](https://md.research.elamur.ai/free/en/universities/texas-a-m-university/) — University, College Station, United States · 32 works · 2,823 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,948 works

## Cite

Cite as: Shubham Parashar, Shurui Gui, Xiner Li, Hongyi Ling, Sushil Vemuri, Blake Olson, Eric Li, Yu Zhang, James Caverlee, Dileep Kalathil and Shuiwang Ji. 2025. Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning. arXiv:2506.06632.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{parashar2025curriculum,
  title = {{Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning}},
  author = {Shubham Parashar, Shurui Gui, Xiner Li, Hongyi Ling, Sushil Vemuri, Blake Olson, Eric Li, Yu Zhang, James Caverlee, Dileep Kalathil and Shuiwang Ji},
  year = {2025},
  doi = {10.48550/arxiv.2506.06632},
  eprint = {2506.06632},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2506.06632}
}
```

- [← previous Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspective](https://md.research.elamur.ai/free/en/papers/yu-2025-chain-reasoning-unified-mathematical-reasoning/)
- [next → IMOL: Incomplete-Modality-Tolerant Learning for Multi-Domain Fake News Video Detection](https://md.research.elamur.ai/free/en/papers/zeng-2025-imol-incomplete-modality-tolerant-learning/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/5/)

## References

54 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alekh Agarwal, Sham M Kakade, Jason D Lee, and Gaurav Mahajan. On the theory of policy gradient methods: Optimality, approximation, and distribution shift. Journal of Machine Learning Research, 22(98):1–76, 2021. [find](https://scholar.google.com/scholar?q=Alekh+Agarwal%2C+Sham+M+Kakade%2C+Jason+D+Lee%2C+and+Gaurav+Mahajan%2E+On+the+theory+of+policy+gradient+methods%3A+Optimality%2C+approximation%2C+and+distribution+shift%2E+Journal+of+Machine+Learning+Research%2C+22%2898%29)
2. Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. [Invariant risk minimization](https://md.research.elamur.ai/free/en/papers/arjovsky-2019-invariant-risk-minimization/). arXiv preprint arXiv:1907.02893, 2019.
3. Sanghwan Bae, Jiwoo Hong, Min Young Lee, Hanbyul Kim, JeongYeon Nam, and Donghyun Kwak. [Online difficulty filtering for reasoning oriented reinforcement learning](https://md.research.elamur.ai/free/en/papers/bae-2026-online-difficulty-filtering-reasoning-oriented/). arXiv preprint arXiv:2504.03380, 2025.
4. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. [Curriculum learning](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/). In ICML, pp. 41–48, 2009.
5. Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, et al. [Llama-nemotron: Efficient reasoning models](https://md.research.elamur.ai/free/en/papers/bercovich-2025-llama-nemotron-efficient-reasoning-models/). arXiv preprint arXiv:2505.00949, 2025.
6. Xiaoyin Chen, Jiarui Lu, Minsu Kim, Dinghuai Zhang, Jian Tang, Alexandre Piché, Nicolas Gontier, Yoshua Bengio, and Ehsan Kamalloo. [Self-evolving curriculum for llm reasoning](https://md.research.elamur.ai/free/en/papers/chen-2025-self-evolving-curriculum-llm-reasoning/). arXiv preprint arXiv:2505.14970, 2025.
7. Zaiwei Chen and Siva Theja Maguluri. An approximate policy iteration viewpoint of actor–critic algorithms. Automatica, 179:112395, 2025. [find](https://scholar.google.com/scholar?q=Zaiwei+Chen+and+Siva+Theja+Maguluri%2E+An+approximate+policy+iteration+viewpoint+of+actor%E2%80%93critic+algorithms%2E+Automatica%2C+179%3A112395%2C+2025%2E)
8. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. [Sft memorizes, rl generalizes: A comparative study of foundation model post-training](https://md.research.elamur.ai/free/en/papers/chu-2025-sft-memorizes-rl-generalizes-comparative/). arXiv preprint arXiv:2501.17161, 2025.
9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021.
10. Thomas Foster, Anya Sims, Johannes Forkel, Mattie Fellows, and Jakob Foerster. Learning to reason at the frontier of learnability. arXiv preprint arXiv:2502.12272, 2025. — [arXiv:2502.12272](https://arxiv.org/abs/2502.12272) [find](https://scholar.google.com/scholar?q=Thomas+Foster%2C+Anya+Sims%2C+Johannes+Forkel%2C+Mattie+Fellows%2C+and+Jakob+Foerster%2E+Learning+to+reason+at+the+frontier+of+learnability%2E+arXiv+preprint+arXiv%3A2502%2E12272%2C+2025%2E+%E2%80%94+%5BarXiv%3A2502%2E12272%5D%28https%3A%2F%2Fa)
11. Kanishk Gandhi, Denise H J Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah Goodman. [Stream of search (sos): Learning to search in language](https://md.research.elamur.ai/free/en/papers/gandhi-2024-stream-search-sos-learning-search/). In CoLM, 2024.
12. Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. [Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars](https://md.research.elamur.ai/free/en/papers/gandhi-2025-cognitive-behaviors-that-enable-self/). arXiv preprint arXiv:2503.01307, 2025.
13. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783, 2024.
14. Alex Graves, Marc G Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu. Automated curriculum learning for neural networks. In ICML, pp. 1311–1320, 2017. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Marc+G+Bellemare%2C+Jacob+Menick%2C+Remi+Munos%2C+and+Koray+Kavukcuoglu%2E+Automated+curriculum+learning+for+neural+networks%2E+In+ICML%2C+pp%2E+1311%E2%80%931320%2C+2017%2E)
15. Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In ICLR, 2021. [find](https://scholar.google.com/scholar?q=Ishaan+Gulrajani+and+David+Lopez%2DPaz%2E+In+search+of+lost+domain+generalization%2E+In+ICLR%2C+2021%2E)
16. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. [Deepseek-r1: Incentivizing reasoning capability in LLMs via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948, 2025.
17. Shibo Hao, Yi Gu, Haotian Luo, Tianyang Liu, Xiyan Shao, Xinyuan Wang, Shuhua Xie, Haodi Ma, Adithya Samavedhi, Qiyue Gao, Zhen Wang, and Zhiting Hu. LLM reasoners: New evaluation, library, and analysis of step-by-step reasoning with large language models. In First Conference on Language Modeling, 2… [find](https://scholar.google.com/scholar?q=Shibo+Hao%2C+Yi+Gu%2C+Haotian+Luo%2C+Tianyang+Liu%2C+Xiyan+Shao%2C+Xinyuan+Wang%2C+Shuhua+Xie%2C+Haodi+Ma%2C+Adithya+Samavedhi%2C+Qiyue+Gao%2C+Zhen+Wang%2C+and+Zhiting+Hu%2E+LLM+reasoners%3A+New+evaluation%2C+library%2C+and+analys)
18. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. [OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). In Lun-Wei Ku,…
19. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In NeurIPS, 2021.
20. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. [Large language models cannot self-correct reasoning yet](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-cannot-self/). In ICLR, 2024.
21. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). arXiv preprint arXiv:2412.16720, 2024.
22. Cassidy Laidlaw, Shivam Singhal, and Anca Dragan. Correlated proxies: A new definition and improved mitigation for reward hacking. In ICLR, 2025. [find](https://scholar.google.com/scholar?q=Cassidy+Laidlaw%2C+Shivam+Singhal%2C+and+Anca+Dragan%2E+Correlated+proxies%3A+A+new+definition+and+improved+mitigation+for+reward+hacking%2E+In+ICLR%2C+2025%2E)
23. Hongyi Ling, Shubham Parashar, Sambhav Khurana, Blake Olson, Anwesha Basu, Gaurangi Sinha, Zhengzhong Tu, James Caverlee, and Shuiwang Ji. Complex LLM planning via automated heuristics discovery. arXiv preprint arXiv:2502.19295, 2025. — [arXiv:2502.19295](https://arxiv.org/abs/2502.19295) [find](https://scholar.google.com/scholar?q=Hongyi+Ling%2C+Shubham+Parashar%2C+Sambhav+Khurana%2C+Blake+Olson%2C+Anwesha+Basu%2C+Gaurangi+Sinha%2C+Zhengzhong+Tu%2C+James+Caverlee%2C+and+Shuiwang+Ji%2E+Complex+LLM+planning+via+automated+heuristics+discovery%2E+arXi)
24. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In ACL, pp. 158–167, 2017a. [find](https://scholar.google.com/scholar?q=Wang+Ling%2C+Dani+Yogatama%2C+Chris+Dyer%2C+and+Phil+Blunsom%2E+Program+induction+by+rationale+generation%3A+Learning+to+solve+and+explain+algebraic+word+problems%2E+In+ACL%2C+pp%2E+158%E2%80%93167%2C+2017a%2E)
25. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In ACL, pp. 158–167, 2017b. [find](https://scholar.google.com/scholar?q=Wang+Ling%2C+Dani+Yogatama%2C+Chris+Dyer%2C+and+Phil+Blunsom%2E+Program+induction+by+rationale+generation%3A+Learning+to+solve+and+explain+algebraic+word+problems%2E+In+ACL%2C+pp%2E+158%E2%80%93167%2C+2017b%2E)
26. Geoffrey J McLachlan and Thriyambakam Krishnan. The EM algorithm and extensions. John Wiley & Sons, 2008. [find](https://scholar.google.com/scholar?q=Geoffrey+J+McLachlan+and+Thriyambakam+Krishnan%2E+The+EM+algorithm+and+extensions%2E+John+Wiley+%26+Sons%2C+2008%2E)
27. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. [s1: Simple test-time scaling](https://md.research.elamur.ai/free/en/papers/muennighoff-2025-s1-simple-test-time-scaling/). arXiv preprint arXiv:2501.19393, 2025.
28. Rémi Munos. Error bounds for approximate policy iteration. In ICML, pp. 560–567, 2003. [find](https://scholar.google.com/scholar?q=R%C3%A9mi+Munos%2E+Error+bounds+for+approximate+policy+iteration%2E+In+ICML%2C+pp%2E+560%E2%80%93567%2C+2003%2E)
29. Sanmit Narvekar, Bei Peng, Matteo Leonetti, Jivko Sinapov, Matthew E Taylor, and Peter Stone. Curriculum learning for reinforcement learning domains: A framework and survey. Journal of Machine Learning Research, 21(181):1–50, 2020. [find](https://scholar.google.com/scholar?q=Sanmit+Narvekar%2C+Bei+Peng%2C+Matteo+Leonetti%2C+Jivko+Sinapov%2C+Matthew+E+Taylor%2C+and+Peter+Stone%2E+Curriculum+learning+for+reinforcement+learning+domains%3A+A+framework+and+survey%2E+Journal+of+Machine+Learnin)
30. Shubham Parashar, Blake Olson, Sambhav Khurana, Eric Li, Hongyi Ling, James Caverlee, and Shuiwang Ji. Inference-time computations for llm reasoning and planning: A benchmark and insights. arXiv preprint arXiv:2502.12521, 2025. — [arXiv:2502.12521](https://arxiv.org/abs/2502.12521) [find](https://scholar.google.com/scholar?q=Shubham+Parashar%2C+Blake+Olson%2C+Sambhav+Khurana%2C+Eric+Li%2C+Hongyi+Ling%2C+James+Caverlee%2C+and+Shuiwang+Ji%2E+Inference%2Dtime+computations+for+llm+reasoning+and+planning%3A+A+benchmark+and+insights%2E+arXiv+prepr)
31. Chenhao Qiu, Qianglong Chen, Jintang Li, Caiyu Wang, Runsen Hua, Minghui Li, Shengshan Hu, and Yechao Zhang. WISDOM: Progressive curriculum synthesis makes LLMs better mathematical reasoner, 2025. URL [https://openreview.net/forum?id=hFFAg5Dmw9](https://openreview.net/forum?id=hFFAg5Dmw9). [↗](https://openreview.net/forum?id=hFFAg5Dmw9)
32. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). NeurIPS, 36:53728–53741, 2023.
33. Douglas A Reynolds et al. Gaussian mixture models. Encyclopedia of biometrics, 741(659-663):3, 2009. [find](https://scholar.google.com/scholar?q=Douglas+A+Reynolds+et+al%2E+Gaussian+mixture+models%2E+Encyclopedia+of+biometrics%2C+741%28659%2D663%29%3A3%2C+2009%2E)
34. Abulhair Saparov and He He. [Language models are greedy reasoners: A systematic formal analysis of chain-of-thought](https://md.research.elamur.ai/free/en/papers/saparov-2022-language-models-greedy-reasoners-systematic/). In ICLR, 2023.
35. Bruno Scherrer. Approximate policy iteration schemes: A comparison. In ICML, pp. 1314–1322, 2014. [find](https://scholar.google.com/scholar?q=Bruno+Scherrer%2E+Approximate+policy+iteration+schemes%3A+A+comparison%2E+In+ICML%2C+pp%2E+1314%E2%80%931322%2C+2014%2E)
36. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2C+2017%2E)
37. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300, 2024.
38. Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In ICLR, 2025. [find](https://scholar.google.com/scholar?q=Charlie+Victor+Snell%2C+Jaehoon+Lee%2C+Kelvin+Xu%2C+and+Aviral+Kumar%2E+Scaling+LLM+test%2Dtime+compute+optimally+can+be+more+effective+than+scaling+parameters+for+reasoning%2E+In+ICLR%2C+2025%2E)
39. Kaya Stechly, Karthik Valmeekam, and Subbarao Kambhampati. Chain of thoughtlessness? an analysis of cot in planning. In NeurIPS, 2024. [find](https://scholar.google.com/scholar?q=Kaya+Stechly%2C+Karthik+Valmeekam%2C+and+Subbarao+Kambhampati%2E+Chain+of+thoughtlessness%3F+an+analysis+of+cot+in+planning%2E+In+NeurIPS%2C+2024%2E)
40. Fahim Tajwar, Yiding Jiang, Abitha Thankaraj, Sumaita Sadia Rahman, J Zico Kolter, Jeff Schneider, and Russ Salakhutdinov. Training a generally curious agent. In Forty-second International Conference on Machine Learning, 2025. URL [https://openreview.net/forum?id=UeB3Hdrhda](https://openreview.net/forum?id=UeB3Hdrhda). [↗](https://openreview.net/forum?id=UeB3Hdrhda)
41. Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with LLMs. arXiv preprint arXiv:2501.[12599, 2025](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/).
42. Karthik Valmeekam, Matthew Marquez, Sarath Sreedharan, and Subbarao Kambhampati. On the planning abilities of large language models-a critical investigation. NeurIPS, 36:75993–76005, 2023. — [On the Planning Abilities of Large Language Models : A Critical Investigation](https://md.research.elamur.ai/free/en/papers/valmeekam-2023-planning-abilities-large-language-models/)
43. Karthik Valmeekam, Kaya Stechly, and Subbarao Kambhampati. Llms still can’t plan; can lrms? a preliminary evaluation of openai’s o1 on planbench. arXiv preprint arXiv:2409.13373, 2024. — [arXiv:2409.13373](https://arxiv.org/abs/2409.13373) [find](https://scholar.google.com/scholar?q=Karthik+Valmeekam%2C+Kaya+Stechly%2C+and+Subbarao+Kambhampati%2E+Llms+still+can%E2%80%99t+plan%3B+can+lrms%3F+a+preliminary+evaluation+of+openai%E2%80%99s+o1+on+planbench%2E+arXiv+preprint+arXiv%3A2409%2E13373%2C+2024%2E+%E2%80%94+%5BarXiv%3A2409%2E1)
44. Hemish Veeraboina. Aime problem set 1983-2024, 2023. URL [https://www.kaggle.com/datasets/](https://www.kaggle.com/datasets/) hemishveeraboina/aime-problem-set-1983-2024. [↗](https://www.kaggle.com/datasets/)
45. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). In ICLR, 2023.
46. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). NeurIPS, 35: 24824–24837, 2022.
47. Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. [Logic-rl: Unleashing LLM reasoning with rule-based reinforcement learning](https://md.research.elamur.ai/free/en/papers/xie-2025-logic-rl-unleashing-llm/). arXiv preprint arXiv:2502.14768, 2025.
48. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+et+al%2E+Qwen2%2E+5+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E15115%2C+2024%2E)
49. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. NeurIPS, 36:11809– 11822, 2023. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+NeurIPS%2C+36%3A11809%E2%80%93+11822%2C+20)
50. Fei Yu, Hongbo Zhang, Prayag Tiwari, and Benyou Wang. [Natural language reasoning, a survey](https://md.research.elamur.ai/free/en/papers/yu-2023-natural-language-reasoning-survey/). ACM Computing Surveys, 56(12):1–39, 2024.
51. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). arXiv preprint arXiv:2503.14476, 2025.
52. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. [Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/) arXiv preprint arXiv:2504.13837, 2025.
53. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. [Star: Bootstrapping reasoning with reasoning](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/). NeurIPS, 35:15476–15488, 2022.
54. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. [Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/). arXiv preprint arXiv:2503.18892, 2025.

*Updated 3 November 2026*