---
title: "What Makes a Good Curriculum? Disentangling the Effects of"
description: "Earlier studies, however, differ in how they define difficulty, how they train, and which ordering direction… 60 references, cited by 4 papers here."
authors: ["Yaning Jia", "Chunhui Zhang", "Xingjian Diao", "Xiangchi Yuan", "Zhongyu Ouyang", "Chiyu Ma", "Soroush Vosoughi"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1591/"
doi: "10.18653/v1/2026.acl-long.1591"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jia-2026-what-makes-good-curriculum-disentangling/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jia-2026-what-makes-good-curriculum-disentangling/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) / What Makes a Good Curriculum? Disentangling the Effects of Data Ordering on LLM Mathematical Reasoning

# What Makes a Good Curriculum? Disentangling the Effects of Data Ordering on LLM Mathematical Reasoning

**Authors:** Yaning Jia, [Chunhui Zhang](https://md.research.elamur.ai/free/en/authors/chunhui-zhang/), [Xingjian Diao](https://md.research.elamur.ai/free/en/authors/xingjian-diao/), [Xiangchi Yuan](https://md.research.elamur.ai/free/en/authors/xiangchi-yuan/), [Zhongyu Ouyang](https://md.research.elamur.ai/free/en/authors/zhongyu-ouyang/), [Chiyu Ma](https://md.research.elamur.ai/free/en/authors/chiyu-ma/) and [Soroush Vosoughi](https://md.research.elamur.ai/free/en/authors/soroush-vosoughi/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1591/) · [PDF](https://aclanthology.org/2026.acl-long.1591.pdf) · [DOI 10.18653/v1/2026.acl-long.1591](https://doi.org/10.18653/v1/2026.acl-long.1591) · [OpenAlex](https://openalex.org/W7166812098)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/jia-2026-what-makes-good-curriculum-disentangling/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Curriculum learning, which sorts training examples from easy to hard, is often used to improve LLM reasoning. Earlier studies, however, differ in how they define difficulty, how they train, and which ordering direction they use, so their results are hard to compare. This paper tests data ordering in a controlled way. Training is offline supervised fine-tuning on MetaMathQA-40K: the data and budget stay fixed, and only the example order changes. Difficulty is measured along five dimensions. One is problem-side (Problem Difficulty), and four are model-side (Model Surprisal, Confidence Margin, Predictive Uncertainty, Decision Variability). Each yields forward, reverse, grouped and single-tier curricula, compared against a shuffled baseline. Experiments use Llama3.1-8B, Mistral-7B and Gemma3-4B on math benchmarks. No ordering wins everywhere. Stronger models on easier tasks tend to gain from easy-to-hard ordering, while weaker models or harder tasks often prefer the reverse. Curricula also change models' internal confidence and uncertainty, and they affect how training converges. The work gives practitioners guidance on choosing curricula for post-training rather than relying on a single default recipe.

## Key points
- **Difficulty metrics:**
  - Problem-side: Reasoning Step, Symbol Complexity and Comprehension Difficulty are rated by Qwen2.5-Math-72B-Instruct; Acc@K is computed with the target model.
  - Model-side: SLP/TLP, Logit Gap, SLE/TLE and VACC are averaged over 20 completions sampled at temperature 0.7, using top-5 token distributions.
- **Curriculum strategies:** FCL (easy→hard), RCL (hard→easy), SGC (one difficulty tier only), plus the tiered variants GFC and GRC, all compared with random shuffling (SHUF).
- **Effect on internal states:** On Llama3-8B, forward ordering yields lower TLP, SLP and LG than reverse ordering, giving more cautious models. Reverse ordering keeps models more decisive.
- **Effect on training dynamics:**
  - Inner-state metrics (TLP, TLE, LG) mainly change convergence speed.
  - Task-aligned metrics (ACC, VACC) mainly change the final loss and generalization.
- **Which samples help:** Harder tasks benefit most from high-difficulty, high-uncertainty samples, while easier tasks favor low-difficulty, high-confidence ones. Prioritizing decision-uncertain samples can give extra gains.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Georgia Institute of Technology](https://md.research.elamur.ai/free/en/universities/georgia-institute-of-technology/) — University, Atlanta, United States · 133 works · 24,497 citations
- [Dartmouth College](https://md.research.elamur.ai/free/en/universities/dartmouth-college/) — University, Hanover, United States · 37 works · 42,625 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 252 works · [🎓 Education](https://md.research.elamur.ai/free/en/shelves/social-sciences/education/) · 203 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Yaning Jia, Chunhui Zhang, Xingjian Diao, Xiangchi Yuan, Zhongyu Ouyang, Chiyu Ma and Soroush Vosoughi. 2026. What Makes a Good Curriculum? Disentangling the Effects of Data Ordering on LLM Mathematical Reasoning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{jia2026what,
  title = {{What Makes a Good Curriculum? Disentangling the Effects of Data Ordering on LLM Mathematical Reasoning}},
  author = {Yaning Jia, Chunhui Zhang, Xingjian Diao, Xiangchi Yuan, Zhongyu Ouyang, Chiyu Ma and Soroush Vosoughi},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1591},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1591}
}
```

- [← previous VulAgent: Hypothesis-Validation Driven Multi-Agent Architecture for Vulnerability Detection](https://md.research.elamur.ai/free/en/papers/wang-2026-vulagent-hypothesis-validation-driven-multi/)
- [next → When TableQA Meets Noise: A Dual Denoising Framework for Complex Questions and Large-scale Tables](https://md.research.elamur.ai/free/en/papers/ye-2026-when-tableqa-meets-noise-dual/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/)

## References

60 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shuang Ao, Stefan Rueger, and Advaith Siddharthan. 2023. Confidence-aware calibration and scoring functions for curriculum learning. In International Conference on Machine Vision. Available: [https://arxiv.org/pdf/2301.12589](https://arxiv.org/pdf/2301.12589). — [arXiv:2301.12589](https://arxiv.org/abs/2301.12589) [↗](https://arxiv.org/pdf/2301.12589)
2. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. [Curriculum learning](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/). In Annual International Conference on Machine Learning. Available: [https://dl.acm.org/doi/](https://dl.acm.org/doi/) pdf/10.1145/1553374.1553380. [↗](https://dl.acm.org/doi/)
3. Mayee Chen, Nicholas Roberts, Kush Bhatia, Jue Wang, Ce Zhang, Frederic Sala, and Christopher Ré. 2023. Skill-it! a data-driven skills framework for understanding and training language models. In Advances in Neural Information Processing Systems. Available: [https://arxiv.org/pdf/2307.14430](https://arxiv.org/pdf/2307.14430). — [arXiv:2307.14430](https://arxiv.org/abs/2307.14430) [↗](https://arxiv.org/pdf/2307.14430)
4. Xiaoyin Chen, Jiarui Lu, Minsu Kim, Dinghuai Zhang, Jian Tang, Alexandre Piché, Nicolas Gontier, Yoshua Bengio, and Ehsan Kamalloo. 2025. Self-evolving curriculum for llm reasoning. arXiv preprint arXiv:2505.14970. Available: [https://arxiv.org/pdf/2505.14970](https://arxiv.org/pdf/2505.14970). [↗](https://arxiv.org/pdf/2505.14970)
5. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168. Available: [https://arxiv.org/pdf/2110.14168](https://arxiv.org/pdf/2110.14168). [↗](https://arxiv.org/pdf/2110.14168)
6. Xingjian Diao, Zheyuan Liu, Chunhui Zhang, Weiyi Wu, Keyi Kong, Lin Shi, Kaize Ding, Soroush Vosoughi, and Jiang Gui. 2026. Addressing overthinking in large vision-language models via gated perception-reasoning optimization. arXiv preprint arXiv:2601.04442. Available: https://arxiv. org/pdf/2601.044… [find](https://scholar.google.com/scholar?q=Xingjian+Diao%2C+Zheyuan+Liu%2C+Chunhui+Zhang%2C+Weiyi+Wu%2C+Keyi+Kong%2C+Lin+Shi%2C+Kaize+Ding%2C+Soroush+Vosoughi%2C+and+Jiang+Gui%2E+2026%2E+Addressing+overthinking+in+large+vision%2Dlanguage+models+via+gated+perception)
7. Mucong Ding, Chenghao Deng, Jocelyn Choo, Zichu Wu, Aakriti Agrawal, Avi Schwarzschild, Tianyi Zhou, Tom Goldstein, John Langford, Animashree Anandkumar, and 1 others. 2024. Easy2hard-bench: Standardized difficulty labels for profiling llm performance and generalization. Advances in Neural Informati… — [arXiv:2409.18433](https://arxiv.org/abs/2409.18433) [find](https://scholar.google.com/scholar?q=Mucong+Ding%2C+Chenghao+Deng%2C+Jocelyn+Choo%2C+Zichu+Wu%2C+Aakriti+Agrawal%2C+Avi+Schwarzschild%2C+Tianyi+Zhou%2C+Tom+Goldstein%2C+John+Langford%2C+Animashree+Anandkumar%2C+and+1+others%2E+2024%2E+Easy2hard%2Dbench%3A+Standardi)
8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783. Available: [https://arxiv.org/pdf/2407.21783](https://arxiv.org/pdf/2407.21783). [↗](https://arxiv.org/pdf/2407.21783)
9. Jeffrey L Elman. 1993. Learning and development in neural networks: The importance of starting small. Cognition. Available: [https://www.sciencedirect.com/science/](https://www.sciencedirect.com/science/) article/pii/0010027793900584. [↗](https://www.sciencedirect.com/science/)
10. Tianye Fang, Xuanshu Luo, and Martin Werner. 2025. Entropy-driven curriculum for multi-task training in human mobility prediction. arXiv preprint arXiv:2509.01613. Available: [https://arxiv.org/pdf/2509.01613](https://arxiv.org/pdf/2509.01613). — [arXiv:2509.01613](https://arxiv.org/abs/2509.01613) [↗](https://arxiv.org/pdf/2509.01613)
11. Qi Feng, Yihong Liu, and Hinrich Schütze. 2025. Your pretrained model tells the difficulty itself: A self-adaptive curriculum learning paradigm for natural language understanding. arXiv preprint arXiv:2507.09758. Available: https://aclanthology.org/2025. acl-srw.15.pdf. [↗](https://aclanthology.org/2025)
12. Tao Feng, Zifeng Wang, and Jimeng Sun. 2023. Citing: Large language models create curriculum for instruction tuning. arXiv preprint arXiv:2310.02527. Available: [https://arxiv.org/pdf/2310](https://arxiv.org/pdf/2310). 02527. — [arXiv:2310.02527](https://arxiv.org/abs/2310.02527) [↗](https://arxiv.org/pdf/2310)
13. Zifa Gan, Hongfei Xu, and Hongying Zan. 2021. Self-supervised curriculum learning for spelling error correction. In Conference on Empirical Methods in Natural Language Processing. Available: https://aclanthology.org/2021. emnlp-main.281.pdf. [↗](https://aclanthology.org/2021)
14. Hasan Abed Al Kader Hammoud, Kumail Alhamoud, Abed Hammoud, Elie Bou-Zeid, Marzyeh Ghassemi, and Bernard Ghanem. 2025. Train long, think short: Curriculum learning for efficient reasoning. arXiv preprint arXiv:2508.08940. Available: [https://arxiv.org/pdf/2508.08940](https://arxiv.org/pdf/2508.08940). — [arXiv:2508.08940](https://arxiv.org/abs/2508.08940) [↗](https://arxiv.org/pdf/2508.08940)
15. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874. Available: [https://arxiv.org/pdf/2103.03874](https://arxiv.org/pdf/2103.03874). [↗](https://arxiv.org/pdf/2103.03874)
16. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations. Available: [https://arxiv.org/pdf/](https://arxiv.org/pdf/) 2106.09685v1/1000. [↗](https://arxiv.org/pdf/)
17. Yufei Huang and Deyi Xiong. 2024. It2acl learning easy-to-hard instructions via 2-phase automated curriculum learning for large language models. In Joint International Conference on Computational Linguistics, Language Resources and Evaluation. Available: https://aclanthology. org/2024.lrec-main.822.… [find](https://scholar.google.com/scholar?q=Yufei+Huang+and+Deyi+Xiong%2E+2024%2E+It2acl+learning+easy%2Dto%2Dhard+instructions+via+2%2Dphase+automated+curriculum+learning+for+large+language+models%2E+In+Joint+International+Conference+on+Computational+Ling)
18. Borna Jafarpour, Dawn Sepehr, and Nick Pogrebnyakov. 2021. Active curriculum learning. In First Workshop on Interactive Learning for Natural Language Processing in ACL. Available: https://aclanthology.org/2021. internlp-1.6.pdf. [↗](https://aclanthology.org/2021)
19. Albert Jiang, Timothée Balesni, Farima Hatami, Thomas Joseph, Thibaut Lavril, Gaël Leroux, Antoine Lheritier, Raphaël Lostis, Walid Moufarek, Alexandre Rochette, Lucile Saulnier, Roman Tang, Pablo Villegas, Pierre Wajsburt, and Guillaume Lample. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. Ava… [find](https://scholar.google.com/scholar?q=Albert+Jiang%2C+Timoth%C3%A9e+Balesni%2C+Farima+Hatami%2C+Thomas+Joseph%2C+Thibaut+Lavril%2C+Ga%C3%ABl+Leroux%2C+Antoine+Lheritier%2C+Rapha%C3%ABl+Lostis%2C+Walid+Moufarek%2C+Alexandre+Rochette%2C+Lucile+Saulnier%2C+Roman+Tang%2C+Pablo+Vil)
20. Lu Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, and Alexander Hauptmann. 2015. Self-paced curriculum learning. In AAAI Conference on Artificial Intelligence. Available: [https://ojs.aaai.org/index.php/](https://ojs.aaai.org/index.php/) AAAI/article/view/9608/9467. [↗](https://ojs.aaai.org/index.php/)
21. Jeesu Jung and Sangkeun Jung. 2025. Reasoning steps as curriculum: Using depth of thought as a difficulty signal for tuning llms. arXiv preprint arXiv:2508.18279. Available: [https://arxiv.org/pdf/2508.18279](https://arxiv.org/pdf/2508.18279). — [arXiv:2508.18279](https://arxiv.org/abs/2508.18279) [↗](https://arxiv.org/pdf/2508.18279)
22. Jisu Kim and Juhwan Lee. 2024. Strategic data ordering: Enhancing large language model performance through curriculum learning. arXiv preprint arXiv:2405.07490. Available: [https://arxiv.org/pdf/2405.07490](https://arxiv.org/pdf/2405.07490). — [arXiv:2405.07490](https://arxiv.org/abs/2405.07490) [↗](https://arxiv.org/pdf/2405.07490)
23. M Kumar, Benjamin Packer, and Daphne Koller. 2010. Self-paced learning for latent variable models. In Advances in Neural Information Processing Systems. Available: [https://ai.stanford.edu/~bpacker/selfPacedLVM.pdf](https://ai.stanford.edu/~bpacker/selfPacedLVM.pdf). [↗](https://ai.stanford.edu/~bpacker/selfPacedLVM.pdf)
24. Bruce W. Lee, Hyunsoo Cho, and Kang Min Yoo. 2024. Instruction tuning with human curriculum. In Findings of the Association for Computational Linguistics: NAACL 2024. Available: https://aclanthology.org/2024. findings-naacl.82.pdf. [↗](https://aclanthology.org/2024)
25. Mengdi Li, Jiaye Lin, Xufeng Zhao, Wenhao Lu, Peilin Zhao, Stefan Wermter, and Di Wang. 2025. Curriculum-rlaif: Curriculum alignment with reinforcement learning from ai feedback. arXiv preprint arXiv:2505.20075. Available: [https://arxiv.org/pdf/2505.20075](https://arxiv.org/pdf/2505.20075). [↗](https://arxiv.org/pdf/2505.20075)
26. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). In International Conference on Learning Representations. Available: [https://arxiv.org/pdf/2305.20050](https://arxiv.org/pdf/2305.20050). [↗](https://arxiv.org/pdf/2305.20050)
27. Haitao Liu. 2008. Dependency distance as a metric of language comprehension difficulty. Journal of Cognitive Science. Available: [https://web.archive.org/web/20170809174544id_/http://lingviko.net/jcs.pdf](https://web.archive.org/web/20170809174544id_/http://lingviko.net/jcs.pdf). [↗](https://web.archive.org/web/20170809174544id_/http://lingviko.net/jcs.pdf)
28. Hongwei Liu, Zilong Zheng, Yuxuan Qiao, Haodong Duan, Zhiwei Fei, Fengzhe Zhou, Wenwei Zhang, Songyang Zhang, Dahua Lin, and Kai Chen. 2024a. Mathbench: Evaluating the theory and application proficiency of llms with a hierarchical mathematics benchmark. In Findings of the Association for Computation… [find](https://scholar.google.com/scholar?q=Hongwei+Liu%2C+Zilong+Zheng%2C+Yuxuan+Qiao%2C+Haodong+Duan%2C+Zhiwei+Fei%2C+Fengzhe+Zhou%2C+Wenwei+Zhang%2C+Songyang+Zhang%2C+Dahua+Lin%2C+and+Kai+Chen%2E+2024a%2E+Mathbench%3A+Evaluating+the+theory+and+application+proficien)
29. Yinpeng Liu, Jiawei Liu, Xiang Shi, Qikai Cheng, Yong Huang, and Wei Lu. 2024b. Let’s learn step by step: Enhancing in-context learning ability with curriculum learning. arXiv preprint arXiv:2402.10738. Available: [https://arxiv.org/pdf/2402](https://arxiv.org/pdf/2402). 10738. — [arXiv:2402.10738](https://arxiv.org/abs/2402.10738) [↗](https://arxiv.org/pdf/2402)
30. Adyasha Maharana and Mohit Bansal. 2022. On curriculum learning for commonsense reasoning. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Available: https://aclanthology.org/2022. naacl-main.72.pdf. [↗](https://aclanthology.org/2022)
31. Jordan Meadows, Marco Valentino, Damien Teney, and Andre Freitas. 2024. A symbolic framework for evaluating mathematical reasoning and generalisation with transformers. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Availabl… [find](https://scholar.google.com/scholar?q=Jordan+Meadows%2C+Marco+Valentino%2C+Damien+Teney%2C+and+Andre+Freitas%2E+2024%2E+A+symbolic+framework+for+evaluating+mathematical+reasoning+and+generalisation+with+transformers%2E+In+Conference+of+the+North+Amer)
32. Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. 2020. A diverse corpus for evaluating and developing english math word problem solvers. In Annual Meeting of the Association for Computational Linguistics. Available: https://aclanthology.org/2020.acl-main.92.pdf. [↗](https://aclanthology.org/2020.acl-main.92.pdf)
33. Tasnim Mohiuddin, Philipp Koehn, Vishrav Chaudhary, James Cross, Shruti Bhosale, and Shafiq Joty. 2022. Data selection curriculum for neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2022. Available: https://aclanthology. org/2022.findings-emnlp.113.pdf… [find](https://scholar.google.com/scholar?q=Tasnim+Mohiuddin%2C+Philipp+Koehn%2C+Vishrav+Chaudhary%2C+James+Cross%2C+Shruti+Bhosale%2C+and+Shafiq+Joty%2E+2022%2E+Data+selection+curriculum+for+neural+machine+translation%2E+In+Findings+of+the+Association+for+Com)
34. Marwa Naïr, Kamel Yamani, Lynda Lhadj, and Riyadh Baghdadi. 2024. Curriculum learning for small code language models. In Student Research Workshop at Annual Meeting of the Association for Computational Linguistics. Available: https://aclanthology.org/2024. acl-srw.44.pdf. [↗](https://aclanthology.org/2024)
35. Tianyu Pang, Yujie Fang, Zihang Liu, Shenyang Deng, Lei Hsiung, Shuhua Yu, and Yaoqing Yang. 2026. [Htmuon: Improving muon via heavy-tailed spectral correction](https://md.research.elamur.ai/free/en/papers/pang-2026-htmuon-improving-muon-heavy-tailed/). arXiv preprint arXiv:2603.10067.
36. Wei Pang, Chuan Zhou, Xiao-Hua Zhou, and Xiaojie Wang. 2024. Phased instruction fine-tuning for large language models. In Findings of the Association for Computational Linguistics: ACL 2024. Available: https://aclanthology.org/2024.findings-acl.341.pdf. [↗](https://aclanthology.org/2024.findings-acl.341.pdf)
37. Shubham Parashar, Shurui Gui, Xiner Li, Hongyi Ling, Sushil Vemuri, Blake Olson, Eric Li, Yu Zhang, James Caverlee, Dileep Kalathil, and 1 others. 2025. Curriculum reinforcement learning from easy to hard tasks improves llm reasoning. arXiv preprint arXiv:2506.06632. Available: https://arxiv. org/pd… [find](https://scholar.google.com/scholar?q=Shubham+Parashar%2C+Shurui+Gui%2C+Xiner+Li%2C+Hongyi+Ling%2C+Sushil+Vemuri%2C+Blake+Olson%2C+Eric+Li%2C+Yu+Zhang%2C+James+Caverlee%2C+Dileep+Kalathil%2C+and+1+others%2E+2025%2E+Curriculum+reinforcement+learning+from+easy+to+)
38. Pulkit Pattnaik, Rishabh Maheshwary, Kelechi Ogueji, Vikas Yadav, and Sathwik Tejaswi Madhusudhan. 2024. Enhancing alignment using curriculum learning & ranked preferences. In Findings of the Association for Computational Linguistics: EMNLP 2024. Available: https://aclanthology. org/2024.findings-em… [find](https://scholar.google.com/scholar?q=Pulkit+Pattnaik%2C+Rishabh+Maheshwary%2C+Kelechi+Ogueji%2C+Vikas+Yadav%2C+and+Sathwik+Tejaswi+Madhusudhan%2E+2024%2E+Enhancing+alignment+using+curriculum+learning+%26+ranked+preferences%2E+In+Findings+of+the+Associat)
39. Keqin Peng, Liang Ding, Qihuang Zhong, Yuanxin Ouyang, Wenge Rong, Zhang Xiong, and Dacheng Tao. 2023. Token-level self-evolution training for sequence-to-sequence learning. In Annual Meeting of the Association for Computational Linguistics. Available: https://aclanthology. org/2023.acl-short.73.pdf… [find](https://scholar.google.com/scholar?q=Keqin+Peng%2C+Liang+Ding%2C+Qihuang+Zhong%2C+Yuanxin+Ouyang%2C+Wenge+Rong%2C+Zhang+Xiong%2C+and+Dacheng+Tao%2E+2023%2E+Token%2Dlevel+self%2Devolution+training+for+sequence%2Dto%2Dsequence+learning%2E+In+Annual+Meeting+of+the+A)
40. Stanislas Polu, Jesse Michael Han, Kunhao Zheng, Mantas Baksys, Igor Babuschkin, and Ilya Sutskever. 2022. [Formal mathematics statement curriculum learning](https://md.research.elamur.ai/free/en/papers/polu-2022-formal-mathematics-statement-curriculum-learning/). arXiv preprint arXiv:2202.01344. Available: [https://arxiv.org/pdf/2202](https://arxiv.org/pdf/2202). 01344. [↗](https://arxiv.org/pdf/2202)
41. Hadi Pouransari, Chun-Liang Li, Jen-Hao Chang, Pavan Kumar Anasosalu Vasu, Cem Koc, Vaishaal Shankar, and Oncel Tuzel. 2024. [Dataset decomposition: Faster llm training with variable sequence length curriculum](https://md.research.elamur.ai/free/en/papers/pouransari-2024-dataset-decomposition-faster-llm-training/). In Advances in Neural Information Processing Systems. Available: [https://arxiv.org/pdf/24…](https://arxiv.org/pdf/24…) [↗](https://arxiv.org/pdf/24…)
42. Jun Rao, Xuebo Liu, Lian Lian, Shengjun Cheng, Yunjie Liao, and Min Zhang. 2024. Commonit: Commonality-aware instruction tuning for large language models via data partitions. In Conference on Empirical Methods in Natural Language Processing. Available: https://aclanthology. org/2024.emnlp-main.561.p… [find](https://scholar.google.com/scholar?q=Jun+Rao%2C+Xuebo+Liu%2C+Lian+Lian%2C+Shengjun+Cheng%2C+Yunjie+Liao%2C+and+Min+Zhang%2E+2024%2E+Commonit%3A+Commonality%2Daware+instruction+tuning+for+large+language+models+via+data+partitions%2E+In+Conference+on+Empirica)
43. Linxin Song, Jieyu Zhang, Tianxiang Yang, and Masayuki Goto. 2022. Adaptive ranking-based sample selection for weakly supervised class-imbalanced text classification. arXiv preprint arXiv:2210.03092. Available: [https://arxiv.org/pdf/2210](https://arxiv.org/pdf/2210). 03092. [↗](https://arxiv.org/pdf/2210)
44. Kv Aditya Srivatsa and Ekaterina Kochmar. 2024. What makes math word problems challenging for LLMs? In Findings of the Association for Computational Linguistics: NAACL 2024. Available: https://aclanthology.org/2024. findings-naacl.72.pdf. [↗](https://aclanthology.org/2024)
45. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. Available: [https://arxiv.org/pdf/2503.19786](https://arxiv.org/pdf/2503.19786). [↗](https://arxiv.org/pdf/2503.19786)
46. Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. 2024. Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving. In Advances in Neural Information Processing Systems. Available: [https://arxiv.org/pdf/](https://arxiv.org/pdf/) 2407.13690. [↗](https://arxiv.org/pdf/)
47. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, and 1 others. 2025. [Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/). arXiv preprint arXiv:2506.01939. Available: https:…
48. Association for Computational Linguistics: ACL 2025. Available: https://aclanthology.org/2025.findings-acl.1091.pdf. [↗](https://aclanthology.org/2025.findings-acl.1091.pdf)
49. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171. Available: [https://arxiv.org/pdf/2203.11171](https://arxiv.org/pdf/2203.11171). [↗](https://arxiv.org/pdf/2203.11171)
50. Yang Zhang, Amr Mohamed, Hadi Abdine, Guokan Shang, and Michalis Vazirgiannis. 2025c. [Beyond random sampling: Efficient language model pre-training via curriculum learning](https://md.research.elamur.ai/free/en/papers/zhang-2026-beyond-random-sampling-efficient-language/). arXiv preprint arXiv:2506.11300. Available: [https://arxiv.org/pdf/2506.11300](https://arxiv.org/pdf/2506.11300). [↗](https://arxiv.org/pdf/2506.11300)
51. Zhiheng Xi, Wenxiang Chen, Boyang Hong, Senjie Jin, Rui Zheng, Wei He, Yiwen Ding, Shichun Liu, Xin Guo, Junzhe Wang, and 1 others. 2024. Training large language models for reasoning through reverse curriculum reinforcement learning. arXiv preprint arXiv:2402.05808. Available: https://arxiv. org/pdf… [find](https://scholar.google.com/scholar?q=Zhiheng+Xi%2C+Wenxiang+Chen%2C+Boyang+Hong%2C+Senjie+Jin%2C+Rui+Zheng%2C+Wei+He%2C+Yiwen+Ding%2C+Shichun+Liu%2C+Xin+Guo%2C+Junzhe+Wang%2C+and+1+others%2E+2024%2E+Training+large+language+models+for+reasoning+through+reverse+c)
52. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, and Ed H. Chi. 2023. [Least-to-most prompting enables complex reasoning in large language models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/). In International Conference on Learning Representations. Available:…
53. Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. [Metamath: Bootstrap your own mathematical questions for large language models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/). arXiv preprint arXiv:2309.12284. Available: [https://arxiv.org/pdf/2309.12284](https://arxiv.org/pdf/2309.12284). [↗](https://arxiv.org/pdf/2309.12284)
54. Yikai Zhou, Baosong Yang, Derek F. Wong, Yu Wan, and Lidia S. Chao. 2020. Uncertainty-aware curriculum learning for neural machine translation. In Annual Meeting of the Association for Computational Linguistics. Available: https://aclanthology.org/2020. acl-main.620.pdf. [↗](https://aclanthology.org/2020)
55. Xiangchi Yuan, Xiang Chen, Tong Yu, Dachuan Shi, Can Jin, Wenke Lee, and Saayan Mitra. 2025a. Mitigating forgetting between supervised and reinforcement learning yields stronger reasoners. arXiv preprint arXiv:2510.04454. Available: https://arxiv.org/abs/2510.04454. — [arXiv:2510.04454](https://arxiv.org/abs/2510.04454) [↗](https://arxiv.org/abs/2510.04454)
56. Xiangchi Yuan, Chunhui Zhang, Zheyuan Liu, Dachuan Shi, Leyan Pan, Soroush Vosoughi, and Wenke Lee. 2025b. [Superficial self-improved reasoners benefit from model merging](https://md.research.elamur.ai/free/en/papers/yuan-2025-superficial-self-improved-reasoners-benefit/). In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Available: https://arxiv. org/abs/2503…
57. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D Goodman. 2024. Star: Self-taught reasoner bootstrapping reasoning with reasoning. In International Conference on Neural Information Processing Systems. Available: [https://arxiv.org/pdf/](https://arxiv.org/pdf/) 2203.14465. [↗](https://arxiv.org/pdf/)
58. Feipeng Zhang, Wenyu Jiang, Jun Shu, Feng Zheng, Hongxin Wei, and 1 others. 2024. On the noise robustness of in-context learning for text generation. In Advances in Neural Information Processing Systems. Available: [https://arxiv.org/pdf/](https://arxiv.org/pdf/) 2405.17264. [↗](https://arxiv.org/pdf/)
59. Ruiqi Zhang, Daman Arora, Song Mei, and Andrea Zanette. 2025a. Speed-rl: Faster training of reasoning models via online curriculum learning. arXiv preprint arXiv:2506.09016. Available: [https://arxiv.org/pdf/2506.09016](https://arxiv.org/pdf/2506.09016). — [arXiv:2506.09016](https://arxiv.org/abs/2506.09016) [↗](https://arxiv.org/pdf/2506.09016)
60. Xuemiao Zhang, Xu Liangyu, Feiyu Duan, Yongwei Zhou, Sirui Wang, Rongxiang Weng, Jingang Wang, and Xunliang Cai. 2025b. Preference curriculum: LLMs should always be pre-trained on their preferred data. In Findings of the Contents of Appendix [find](https://scholar.google.com/scholar?q=Xuemiao+Zhang%2C+Xu+Liangyu%2C+Feiyu+Duan%2C+Yongwei+Zhou%2C+Sirui+Wang%2C+Rongxiang+Weng%2C+Jingang+Wang%2C+and+Xunliang+Cai%2E+2025b%2E+Preference+curriculum%3A+LLMs+should+always+be+pre%2Dtrained+on+their+preferred+data)

*Updated 31 October 2026*