---
title: "Demystifying Long Chain-of-Thought Reasoning in — Yeo 2025"
description: "The paper investigates how large language models acquire long chain-of-thought (CoT) reasoning, meaning extended traces that include branching…"
authors: ["Edward Yeo", "Yuxuan Tong", "Morry Niu", "Graham Neubig", "Xiang Yue"]
year: 2025
venue: "arXiv:2502.03373"
source: "https://arxiv.org/abs/2502.03373"
doi: "10.48550/arxiv.2502.03373"
arxiv: "2502.03373"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yeo-2025-demystifying-long-chain-thought-reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yeo-2025-demystifying-long-chain-thought-reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/) / Demystifying Long Chain-of-Thought Reasoning in LLMs

# Demystifying Long Chain-of-Thought Reasoning in LLMs

**Authors:** Edward Yeo, Yuxuan Tong, Morry Niu, [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/), [Xiang Yue](https://md.research.elamur.ai/free/en/authors/xiang-yue/) · **Year:** 2025 · **Venue:** arXiv:2502.03373 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2502.03373) · [PDF](https://arxiv.org/pdf/2502.03373) · [DOI 10.48550/arxiv.2502.03373](https://doi.org/10.48550/arxiv.2502.03373) · [arXiv 2502.03373](https://arxiv.org/abs/2502.03373) · [OpenAlex](https://openalex.org/W4407231747)

**Citations:** 3 citations (openalex-snapshot · 2026-10-14) · [110](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper investigates how large language models acquire long chain-of-thought (CoT) reasoning, meaning extended traces that include branching, backtracking and self-correction, and which training choices make this behaviour appear reliably. The authors run extensive supervised fine-tuning (SFT) and reinforcement learning (RL) ablations on Llama-3.1-8B and Qwen2.5-Math-7B. They train mainly on the MATH training prompts and use PPO with a rule-based answer verifier in OpenRLHF. They evaluate on MATH-500, AIME 2024, TheoremQA and MMLU-Pro-1k.

They find that SFT is optional but makes training easier and more efficient, and that long-CoT SFT data leaves more room for later RL gains than short-CoT data. Because CoT length does not grow stably under plain RL, they propose a cosine length-scaling reward combined with a repetition penalty. Noisy web-derived answers, such as WebInstruct, become useful once they are filtered. Abilities like error correction already exist in base models, but strengthening them through RL is costly. Together, these results give practitioners concrete recipes for building o1-style reasoners, and the authors release their code.

## Key points
- Long-CoT SFT data was distilled from QwQ-32B-Preview and short-CoT data from Qwen2.5-Math-72B-Instruct, both via rejection sampling. On MATH-500, Llama-3.1-8B with long-CoT SFT passed 70% and was still improving at about 3.5B tokens, while short-CoT SFT levelled off below 55%.
- RL added more than 3 points on MATH-500 to models initialised with long-CoT SFT. Models initialised with short-CoT SFT gained almost nothing from RL.
- On Qwen2.5-Math-7B, combining long-CoT SFT with RL reached a 49.7 average across the four benchmarks. This beats RL started directly from the base model (41.0) and SFT alone (47.3).
- Filtered, web-extracted "silver" supervision (WebInstruct) proved especially helpful for out-of-distribution STEM tasks.
- REINFORCE++ was less stable than PPO in their setup. Attempts to scale to 32B models were halted by GPU requirements, which highlights how immature current RL infrastructure is.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Mathematics & statistics](https://md.research.elamur.ai/free/en/shelves/mathematics/)

## Cited by

Cited by

110 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 103 not yet.

- [Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs](https://md.research.elamur.ai/free/en/papers/gandhi-2025-cognitive-behaviors-that-enable-self/)
- [Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model](https://md.research.elamur.ai/free/en/papers/hu-2025-open-reasoner-zero-open-source/)
- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2025-system-1-system-2-survey/)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/)
- [Beyond Simple Term Injection: Reasoning Models for Legal Translation in a Non-Dominant Language Variety](https://md.research.elamur.ai/free/en/papers/natale-2026-beyond-simple-term-injection-reasoning/)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://md.research.elamur.ai/free/en/papers/sui-2025-stop-overthinking-survey-efficient-reasoning/)
- [SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{yeo2025demystifying,
  title = {{Demystifying Long Chain-of-Thought Reasoning in LLMs}},
  author = {Yeo, Edward and Tong, Yuxuan and Niu, Morry and Neubig, Graham and Yue, Xiang},
  year = {2025},
  doi = {10.48550/arxiv.2502.03373},
  eprint = {2502.03373},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2502.03373}
}
```

- [← previous DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://md.research.elamur.ai/free/en/papers/deepseek-2025-ai-deepseek-v3-2-pushing/)
- [next → Dense Retrieval for Low Resource Languages -- the Case of Amharic Language](https://md.research.elamur.ai/free/en/papers/yeshambel-2025-dense-retrieval-low-resource-languages/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/)

## References

53 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Anthropic. Introducing claude, 2023. [https://www.anthropic.com/index/introducing-claude](https://www.anthropic.com/index/introducing-claude). [↗](https://www.anthropic.com/index/introducing-claude)
2. URL Broder, A. On the resemblance and containment of documents. In Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No.97TB100171), pp. 21–29, 1997. doi: 10.1109/SEQUEN.1997.666900. [find](https://scholar.google.com/scholar?q=URL+Broder%2C+A%2E+On+the+resemblance+and+containment+of+documents%2E+In+Proceedings%2E+Compression+and+Complexity+of+SEQUENCES+1997+%28Cat%2E+No%2E97TB100171%29%2C+pp%2E+21%E2%80%9329%2C+1997%2E+doi%3A+10%2E1109%2FSEQUEN%2E1997%2E666900%2E)
3. Broder, A. Z., Charikar, M., Frieze, A. M., and Mitzenmacher, M. Min-wise independent permutations. In Proceedings of the thirtieth annual ACM symposium on Theory of computing, pp. 327–336, 1998. [find](https://scholar.google.com/scholar?q=Broder%2C+A%2E+Z%2E%2C+Charikar%2C+M%2E%2C+Frieze%2C+A%2E+M%2E%2C+and+Mitzenmacher%2C+M%2E+Min%2Dwise+independent+permutations%2E+In+Proceedings+of+the+thirtieth+annual+ACM+symposium+on+Theory+of+computing%2C+pp%2E+327%E2%80%93336%2C+1998%2E)
4. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877–1901, 2020. [find](https://scholar.google.com/scholar?q=Brown%2C+T%2E%2C+Mann%2C+B%2E%2C+Ryder%2C+N%2E%2C+Subbiah%2C+M%2E%2C+Kaplan%2C+J%2E+D%2E%2C+Dhariwal%2C+P%2E%2C+Neelakantan%2C+A%2E%2C+Shyam%2C+P%2E%2C+Sastry%2C+G%2E%2C+Askell%2C+A%2E%2C+et+al%2E+Language+models+are+few%2Dshot+learners%2E+Advances+in+neural+informati)
5. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C.… [find](https://scholar.google.com/scholar?q=Chen%2C+M%2E%2C+Tworek%2C+J%2E%2C+Jun%2C+H%2E%2C+Yuan%2C+Q%2E%2C+de+Oliveira+Pinto%2C+H%2E+P%2E%2C+Kaplan%2C+J%2E%2C+Edwards%2C+H%2E%2C+Burda%2C+Y%2E%2C+Joseph%2C+N%2E%2C+Brockman%2C+G%2E%2C+Ray%2C+A%2E%2C+Puri%2C+R%2E%2C+Krueger%2C+G%2E%2C+Petrov%2C+M%2E%2C+Khlaaf%2C+H%2E%2C+Sastry%2C+G%2E%2C+Mis) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
6. Chen, W., Yin, M., Ku, M., Lu, P., Wan, Y., Ma, X., Xu, J., Wang, X., and Xia, T. TheoremQA: A theorem-driven question answering dataset. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. [find](https://scholar.google.com/scholar?q=Chen%2C+W%2E%2C+Yin%2C+M%2E%2C+Ku%2C+M%2E%2C+Lu%2C+P%2E%2C+Wan%2C+Y%2E%2C+Ma%2C+X%2E%2C+Xu%2C+J%2E%2C+Wang%2C+X%2E%2C+and+Xia%2C+T%2E+TheoremQA%3A+A+theorem%2Ddriven+question+answering+dataset%2E+In+The+2023+Conference+on+Empirical+Methods+in+Natural+Languag) [our card](https://md.research.elamur.ai/free/en/papers/chen-2023-theoremqa-theorem-driven-question-answering/)
7. Chung, H. W. Dont teach. incentivize. Presentation slides, 2024. URL [https://t.co/2sjhynKxzJ](https://t.co/2sjhynKxzJ). Slide 48. [↗](https://t.co/2sjhynKxzJ)
8. Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. [find](https://scholar.google.com/scholar?q=Cobbe%2C+K%2E%2C+Kosaraju%2C+V%2E%2C+Bavarian%2C+M%2E%2C+Chen%2C+M%2E%2C+Jun%2C+H%2E%2C+Kaiser%2C+L%2E%2C+Plappert%2C+M%2E%2C+Tworek%2C+J%2E%2C+Hilton%2C+J%2E%2C+Nakano%2C+R%2E%2C+et+al%2E+Training+verifiers+to+solve+math+word+problems%2E+arXiv+preprint+arXiv%3A2110) [our card](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
9. Dao, T. FlashAttention-2: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations (ICLR), 2024. [find](https://scholar.google.com/scholar?q=Dao%2C+T%2E+FlashAttention%2D2%3A+Faster+attention+with+better+parallelism+and+work+partitioning%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2C+2024%2E)
10. DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2E+Deepseek%2Dr1%3A+Incentivizing+reasoning+capability+in+llms+via+reinforcement+learning%2C+2025%2E) [our card](https://md.research.elamur.ai/free/en/papers/deepseek-2025-ai-deepseek-r1-incentivizing-reasoning-capability/)
11. Dong, H., Xiong, W., Goyal, D., Zhang, Y., Chow, W., Pan, R., Diao, S., Zhang, J., KaShun, S., and Zhang, T. Raft: Reward ranked finetuning for generative foundation model alignment. Transactions on Machine Learning Research, 2023. [find](https://scholar.google.com/scholar?q=Dong%2C+H%2E%2C+Xiong%2C+W%2E%2C+Goyal%2C+D%2E%2C+Zhang%2C+Y%2E%2C+Chow%2C+W%2E%2C+Pan%2C+R%2E%2C+Diao%2C+S%2E%2C+Zhang%2C+J%2E%2C+KaShun%2C+S%2E%2C+and+Zhang%2C+T%2E+Raft%3A+Reward+ranked+finetuning+for+generative+foundation+model+alignment%2E+Transactions+on+M) [our card](https://md.research.elamur.ai/free/en/papers/dong-2023-raft-reward-ranked-finetuning-generative/)
12. Feng, X., Wan, Z., Wen, M., McAleer, S. M., Wen, Y., Zhang, W., and Wang, J. Alphazero-like tree-search can guide large language model decoding and training, 2023. [find](https://scholar.google.com/scholar?q=Feng%2C+X%2E%2C+Wan%2C+Z%2E%2C+Wen%2C+M%2E%2C+McAleer%2C+S%2E+M%2E%2C+Wen%2C+Y%2E%2C+Zhang%2C+W%2E%2C+and+Wang%2C+J%2E+Alphazero%2Dlike+tree%2Dsearch+can+guide+large+language+model+decoding+and+training%2C+2023%2E)
13. Gao, Z., Wang, H., Lu, C., Lu, T., Froudist-Walsh, S., Chen, M., Wang, X.-J., Hu, J., and Sun, W. The neural basis of delayed gratification. Science Advances, 7(49):eabg6611, 2021. doi: 10.1126/sciadv.abg6611. — [doi:10.1126/sciadv.abg6611](https://doi.org/10.1126/sciadv.abg6611) [find](https://scholar.google.com/scholar?q=Gao%2C+Z%2E%2C+Wang%2C+H%2E%2C+Lu%2C+C%2E%2C+Lu%2C+T%2E%2C+Froudist%2DWalsh%2C+S%2E%2C+Chen%2C+M%2E%2C+Wang%2C+X%2E%2DJ%2E%2C+Hu%2C+J%2E%2C+and+Sun%2C+W%2E+The+neural+basis+of+delayed+gratification%2E+Science+Advances%2C+7%2849%29%3Aeabg6611%2C+2021%2E+doi%3A+10%2E1126%2Fsciadv)
14. Gulcehre, C., Paine, T. L., Srinivasan, S., Konyushkova, K., Weerts, L., Sharma, A., Siddhant, A., Ahern, A., Wang, M., Gu, C., et al. Reinforced self-training (rest) for language modeling. arXiv preprint arXiv:2308.08998, 2023. [find](https://scholar.google.com/scholar?q=Gulcehre%2C+C%2E%2C+Paine%2C+T%2E+L%2E%2C+Srinivasan%2C+S%2E%2C+Konyushkova%2C+K%2E%2C+Weerts%2C+L%2E%2C+Sharma%2C+A%2E%2C+Siddhant%2C+A%2E%2C+Ahern%2C+A%2E%2C+Wang%2C+M%2E%2C+Gu%2C+C%2E%2C+et+al%2E+Reinforced+self%2Dtraining+%28rest%29+for+language+modeling%2E+arXiv+prep) [our card](https://md.research.elamur.ai/free/en/papers/gulcehre-2023-reinforced-self-training-rest-language/)
15. Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring mathematical problem solving with the math dataset. In Vanschoren, J. and Yeung, S. (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volum… [find](https://scholar.google.com/scholar?q=Hendrycks%2C+D%2E%2C+Burns%2C+C%2E%2C+Kadavath%2C+S%2E%2C+Arora%2C+A%2E%2C+Basart%2C+S%2E%2C+Tang%2C+E%2E%2C+Song%2C+D%2E%2C+and+Steinhardt%2C+J%2E+Measuring+mathematical+problem+solving+with+the+math+dataset%2E+In+Vanschoren%2C+J%2E+and+Yeung%2C+S%2E+%28eds) [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/)
16. Hou, Z., Lv, X., Lu, R., Zhang, J., Li, Y., Yao, Z., Li, J., Tang, J., and Dong, Y. Advancing language model reasoning through reinforcement learning and inference scaling, 2025. [find](https://scholar.google.com/scholar?q=Hou%2C+Z%2E%2C+Lv%2C+X%2E%2C+Lu%2C+R%2E%2C+Zhang%2C+J%2E%2C+Li%2C+Y%2E%2C+Yao%2C+Z%2E%2C+Li%2C+J%2E%2C+Tang%2C+J%2E%2C+and+Dong%2C+Y%2E+Advancing+language+model+reasoning+through+reinforcement+learning+and+inference+scaling%2C+2025%2E)
17. Hu, J. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03262, 2025. [find](https://scholar.google.com/scholar?q=Hu%2C+J%2E+Reinforce%2B%2B%3A+A+simple+and+efficient+approach+for+aligning+large+language+models%2E+arXiv+preprint+arXiv%3A2501%2E03262%2C+2025%2E)
18. Hu, J., Wu, X., Zhu, Z., Xianyu, Wang, W., Zhang, D., and Cao, Y. Openrlhf: An easy-to-use, scalable and high-performance rlhf framework, 2024. [find](https://scholar.google.com/scholar?q=Hu%2C+J%2E%2C+Wu%2C+X%2E%2C+Zhu%2C+Z%2E%2C+Xianyu%2C+Wang%2C+W%2E%2C+Zhang%2C+D%2E%2C+and+Cao%2C+Y%2E+Openrlhf%3A+An+easy%2Dto%2Duse%2C+scalable+and+high%2Dperformance+rlhf+framework%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/hu-2024-openrlhf-easy-use-scalable/)
19. Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., and Narasimhan, K. R. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=Jimenez%2C+C%2E+E%2E%2C+Yang%2C+J%2E%2C+Wettig%2C+A%2E%2C+Yao%2C+S%2E%2C+Pei%2C+K%2E%2C+Press%2C+O%2E%2C+and+Narasimhan%2C+K%2E+R%2E+SWE%2Dbench%3A+Can+language+models+resolve+real%2Dworld+github+issues%3F+In+The+Twelfth+International+Conference+on+Lea) [our card](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/)
20. Kimi Team. Kimi k1.5: Scaling reinforcement learning with llms, 2025. [find](https://scholar.google.com/scholar?q=Kimi+Team%2E+Kimi+k1%2E5%3A+Scaling+reinforcement+learning+with+llms%2C+2025%2E) [our card](https://md.research.elamur.ai/free/en/papers/kimi-2025-team-kimi-k1-5-scaling-reinforcement/)
21. Lamb, A. M., ALIAS PARTH GOYAL, A. G., Zhang, Y., Zhang, S., Courville, A. C., and Bengio, Y. Professor forcing: A new algorithm for training recurrent networks. In Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 29. Cu… [find](https://scholar.google.com/scholar?q=Lamb%2C+A%2E+M%2E%2C+ALIAS+PARTH+GOYAL%2C+A%2E+G%2E%2C+Zhang%2C+Y%2E%2C+Zhang%2C+S%2E%2C+Courville%2C+A%2E+C%2E%2C+and+Bengio%2C+Y%2E+Professor+forcing%3A+A+new+algorithm+for+training+recurrent+networks%2E+In+Lee%2C+D%2E%2C+Sugiyama%2C+M%2E%2C+Luxburg%2C+U%2E%2C)
22. Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F., Miranda, L. J. V., Liu, A., Dziri, N., Lyu, S., Gu, Y., Malik, S., Graf, V., Hwang, J. D., Yang, J., Bras, R. L., Tafjord, O., Wilhelm, C., Soldaini, L., Smith, N. A., Wang, Y., Dasigi, P., and Hajishirzi, H. Tulu 3: Pushing… [find](https://scholar.google.com/scholar?q=Lambert%2C+N%2E%2C+Morrison%2C+J%2E%2C+Pyatkin%2C+V%2E%2C+Huang%2C+S%2E%2C+Ivison%2C+H%2E%2C+Brahman%2C+F%2E%2C+Miranda%2C+L%2E+J%2E+V%2E%2C+Liu%2C+A%2E%2C+Dziri%2C+N%2E%2C+Lyu%2C+S%2E%2C+Gu%2C+Y%2E%2C+Malik%2C+S%2E%2C+Graf%2C+V%2E%2C+Hwang%2C+J%2E+D%2E%2C+Yang%2C+J%2E%2C+Bras%2C+R%2E+L%2E%2C+Tafjord%2C+O) [our card](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
23. Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=Lightman%2C+H%2E%2C+Kosaraju%2C+V%2E%2C+Burda%2C+Y%2E%2C+Edwards%2C+H%2E%2C+Baker%2C+B%2E%2C+Lee%2C+T%2E%2C+Leike%2C+J%2E%2C+Schulman%2C+J%2E%2C+Sutskever%2C+I%2E%2C+and+Cobbe%2C+K%2E+Let%E2%80%99s+verify+step+by+step%2E+In+The+Twelfth+International+Conference+on+Lear) [our card](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)
24. Longpre, S., Hou, L., Vu, T., Webson, A., Chung, H. W., Tay, Y., Zhou, D., Le, Q. V., Zoph, B., Wei, J., and Roberts, A. The flan collection: Designing data and methods for effective instruction tuning. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proce… [find](https://scholar.google.com/scholar?q=Longpre%2C+S%2E%2C+Hou%2C+L%2E%2C+Vu%2C+T%2E%2C+Webson%2C+A%2E%2C+Chung%2C+H%2E+W%2E%2C+Tay%2C+Y%2E%2C+Zhou%2C+D%2E%2C+Le%2C+Q%2E+V%2E%2C+Zoph%2C+B%2E%2C+Wei%2C+J%2E%2C+and+Roberts%2C+A%2E+The+flan+collection%3A+Designing+data+and+methods+for+effective+instruction+tunin) [our card](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/)
25. Loshchilov, I. and Hutter, F. Sgdr: Stochastic gradient descent with warm restarts, 2017. [find](https://scholar.google.com/scholar?q=Loshchilov%2C+I%2E+and+Hutter%2C+F%2E+Sgdr%3A+Stochastic+gradient+descent+with+warm+restarts%2C+2017%2E)
26. Meta. Introducing meta llama 3: The most capable openly available llm to date., 2024. URL [https://ai.meta.com/blog/meta-llama-3](https://ai.meta.com/blog/meta-llama-3). [↗](https://ai.meta.com/blog/meta-llama-3)
27. OpenAI. Gpt-4 technical report. arXiv:2303.08774, 2023. arXiv preprint OpenAI. Learning to reason with llms, 2024. URL [https://openai.com/index/learning-to-reason-with-llms/](https://openai.com/index/learning-to-reason-with-llms/). [↗](https://openai.com/index/learning-to-reason-with-llms/) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
28. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. Training language models to follow instructions with… [find](https://scholar.google.com/scholar?q=Ouyang%2C+L%2E%2C+Wu%2C+J%2E%2C+Jiang%2C+X%2E%2C+Almeida%2C+D%2E%2C+Wainwright%2C+C%2E+L%2E%2C+Mishkin%2C+P%2E%2C+Zhang%2C+C%2E%2C+Agarwal%2C+S%2E%2C+Slama%2C+K%2E%2C+Ray%2C+A%2E%2C+Schulman%2C+J%2E%2C+Hilton%2C+J%2E%2C+Kelton%2C+F%2E%2C+Miller%2C+L%2E%2C+Simens%2C+M%2E%2C+Askell%2C+A%2E%2C+Welind)
29. Pan, J., Zhang, J., Wang, X., and Yuan, L. Tinyzero, 2025. URL [https://github.com/Jiayi-Pan/TinyZero](https://github.com/Jiayi-Pan/TinyZero). Accessed: 2025-01-24. [↗](https://github.com/Jiayi-Pan/TinyZero)
30. Paster, K., Santos, M. D., Azerbayev, Z., and Ba, J. Openwebmath: An open dataset of high-quality mathematical web text, 2023. [find](https://scholar.google.com/scholar?q=Paster%2C+K%2E%2C+Santos%2C+M%2E+D%2E%2C+Azerbayev%2C+Z%2E%2C+and+Ba%2C+J%2E+Openwebmath%3A+An+open+dataset+of+high%2Dquality+mathematical+web+text%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/paster-2023-openwebmath-open-dataset-high-quality/)
31. Qwen Team. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement, 2024a. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+Qwen2%2E5%2Dmath+technical+report%3A+Toward+mathematical+expert+model+via+self%2Dimprovement%2C+2024a%2E) [our card](https://md.research.elamur.ai/free/en/papers/yang-2024-qwen2-5-math-technical-report/)
32. Qwen Team. Qwq: Reflect deeply on the boundaries of the unknown, 2024b. URL [https://qwenlm.github.io/blog/qwq-32b-preview/](https://qwenlm.github.io/blog/qwq-32b-preview/). [↗](https://qwenlm.github.io/blog/qwq-32b-preview/)
33. Rajbhandari, S., Rasley, J., Ruwase, O., and He, Y. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–16. IEEE, 2020. [find](https://scholar.google.com/scholar?q=Rajbhandari%2C+S%2E%2C+Rasley%2C+J%2E%2C+Ruwase%2C+O%2E%2C+and+He%2C+Y%2E+Zero%3A+Memory+optimizations+toward+training+trillion+parameter+models%2E+In+SC20%3A+International+Conference+for+High+Performance+Computing%2C+Networking%2C+)
34. Rana, A. Common crawl building an open web-scale crawl using hadoop, 2010. URL [https://www.slideshare.net/hadoopusergroup/common-crawlpresentation](https://www.slideshare.net/hadoopusergroup/common-crawlpresentation). [↗](https://www.slideshare.net/hadoopusergroup/common-crawlpresentation)
35. Rasley, J., Rajbhandari, S., Ruwase, O., and He, Y. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3505–3506, 2020. [find](https://scholar.google.com/scholar?q=Rasley%2C+J%2E%2C+Rajbhandari%2C+S%2E%2C+Ruwase%2C+O%2E%2C+and+He%2C+Y%2E+Deepspeed%3A+System+optimizations+enable+training+deep+learning+models+with+over+100+billion+parameters%2E+In+Proceedings+of+the+26th+ACM+SIGKDD+Interna)
36. Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y., Dirani, J., Michael, J., and Bowman, S. R. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. [find](https://scholar.google.com/scholar?q=Rein%2C+D%2E%2C+Hou%2C+B%2E+L%2E%2C+Stickland%2C+A%2E+C%2E%2C+Petty%2C+J%2E%2C+Pang%2C+R%2E+Y%2E%2C+Dirani%2C+J%2E%2C+Michael%2C+J%2E%2C+and+Bowman%2C+S%2E+R%2E+GPQA%3A+A+graduate%2Dlevel+google%2Dproof+q%26a+benchmark%2E+In+First+Conference+on+Language+Modeling%2C+) [our card](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/)
37. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms, 2017. [find](https://scholar.google.com/scholar?q=Schulman%2C+J%2E%2C+Wolski%2C+F%2E%2C+Dhariwal%2C+P%2E%2C+Radford%2C+A%2E%2C+and+Klimov%2C+O%2E+Proximal+policy+optimization+algorithms%2C+2017%2E)
38. Singh, A., Co-Reyes, J. D., Agarwal, R., Anand, A., Patil, P., Liu, P. J., Harrison, J., Lee, J., Xu, K., Parisi, A., et al. Beyond human data: Scaling self-training for problem-solving with language models. arXiv preprint arXiv:2312.06585, 2023. [find](https://scholar.google.com/scholar?q=Singh%2C+A%2E%2C+Co%2DReyes%2C+J%2E+D%2E%2C+Agarwal%2C+R%2E%2C+Anand%2C+A%2E%2C+Patil%2C+P%2E%2C+Liu%2C+P%2E+J%2E%2C+Harrison%2C+J%2E%2C+Lee%2C+J%2E%2C+Xu%2C+K%2E%2C+Parisi%2C+A%2E%2C+et+al%2E+Beyond+human+data%3A+Scaling+self%2Dtraining+for+problem%2Dsolving+with+language+) [our card](https://md.research.elamur.ai/free/en/papers/singh-2023-human-data-scaling-self-training/)
39. Snell, C., Lee, J., Xu, K., and Kumar, A. Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024. [find](https://scholar.google.com/scholar?q=Snell%2C+C%2E%2C+Lee%2C+J%2E%2C+Xu%2C+K%2E%2C+and+Kumar%2C+A%2E+Scaling+llm+test%2Dtime+compute+optimally+can+be+more+effective+than+scaling+model+parameters%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/snell-2024-scaling-llm-test-time-compute/)
40. Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., and Liu, Y. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024. [find](https://scholar.google.com/scholar?q=Su%2C+J%2E%2C+Ahmed%2C+M%2E%2C+Lu%2C+Y%2E%2C+Pan%2C+S%2E%2C+Bo%2C+W%2E%2C+and+Liu%2C+Y%2E+Roformer%3A+Enhanced+transformer+with+rotary+position+embedding%2E+Neurocomputing%2C+568%3A127063%2C+2024%2E)
41. Sutton, R. S. and Barto, A. G. Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA, 2018. ISBN 0262039249. [find](https://scholar.google.com/scholar?q=Sutton%2C+R%2E+S%2E+and+Barto%2C+A%2E+G%2E+Reinforcement+Learning%3A+An+Introduction%2E+A+Bradford+Book%2C+Cambridge%2C+MA%2C+USA%2C+2018%2E+ISBN+0262039249%2E)
42. Tong, Y., Zhang, X., Wang, R., Wu, R., and He, J. DART-math: Difficulty-aware rejection tuning for mathematical problem-solving. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [find](https://scholar.google.com/scholar?q=Tong%2C+Y%2E%2C+Zhang%2C+X%2E%2C+Wang%2C+R%2E%2C+Wu%2C+R%2E%2C+and+He%2C+J%2E+DART%2Dmath%3A+Difficulty%2Daware+rejection+tuning+for+mathematical+problem%2Dsolving%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Neural+Information+Processing)
43. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozire, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lample, G. Llama: Open and efficient foundation language models, 2023. [find](https://scholar.google.com/scholar?q=Touvron%2C+H%2E%2C+Lavril%2C+T%2E%2C+Izacard%2C+G%2E%2C+Martinet%2C+X%2E%2C+Lachaux%2C+M%2E%2DA%2E%2C+Lacroix%2C+T%2E%2C+Rozire%2C+B%2E%2C+Goyal%2C+N%2E%2C+Hambro%2C+E%2E%2C+Azhar%2C+F%2E%2C+Rodriguez%2C+A%2E%2C+Joulin%2C+A%2E%2C+Grave%2C+E%2E%2C+and+Lample%2C+G%2E+Llama%3A+Open+and+effi)
44. Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., Li, T., Ku, M., Wang, K., Zhuang, A., Fan, R., Yue, X., and Chen, W. MMLU-pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-eight Conference on Neural Infor… [find](https://scholar.google.com/scholar?q=Wang%2C+Y%2E%2C+Ma%2C+X%2E%2C+Zhang%2C+G%2E%2C+Ni%2C+Y%2E%2C+Chandra%2C+A%2E%2C+Guo%2C+S%2E%2C+Ren%2C+W%2E%2C+Arulraj%2C+A%2E%2C+He%2C+X%2E%2C+Jiang%2C+Z%2E%2C+Li%2C+T%2E%2C+Ku%2C+M%2E%2C+Wang%2C+K%2E%2C+Zhuang%2C+A%2E%2C+Fan%2C+R%2E%2C+Yue%2C+X%2E%2C+and+Chen%2C+W%2E+MMLU%2Dpro%3A+A+more+robust+and+cha) [our card](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/)
45. Wang, Z., Li, Y., Wu, Y., Luo, L., Hou, L., Yu, H., and Shang, J. Multi-step problem solving through a verifier: An empirical analysis on model-induced process supervision, 2024b. [find](https://scholar.google.com/scholar?q=Wang%2C+Z%2E%2C+Li%2C+Y%2E%2C+Wu%2C+Y%2E%2C+Luo%2C+L%2E%2C+Hou%2C+L%2E%2C+Yu%2C+H%2E%2C+and+Shang%2C+J%2E+Multi%2Dstep+problem+solving+through+a+verifier%3A+An+empirical+analysis+on+model%2Dinduced+process+supervision%2C+2024b%2E)
46. Wei, J., Wang, X., Schuurmans, D., Bosma, M., ichter, b., Xia, F., Chi, E., Le, Q. V., and Zhou, D. Chain-of-thought prompting elicits reasoning in large language models. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Syst… [find](https://scholar.google.com/scholar?q=Wei%2C+J%2E%2C+Wang%2C+X%2E%2C+Schuurmans%2C+D%2E%2C+Bosma%2C+M%2E%2C+ichter%2C+b%2E%2C+Xia%2C+F%2E%2C+Chi%2C+E%2E%2C+Le%2C+Q%2E+V%2E%2C+and+Zhou%2C+D%2E+Chain%2Dof%2Dthought+prompting+elicits+reasoning+in+large+language+models%2E+In+Koyejo%2C+S%2E%2C+Mohamed%2C+S%2E%2C+A) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
47. Yu, L., Jiang, W., Shi, H., YU, J., Liu, Z., Zhang, Y., Kwok, J., Li, Z., Weller, A., and Liu, W. Metamath: Bootstrap your own mathematical questions for large language models. In The Twelfth International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=Yu%2C+L%2E%2C+Jiang%2C+W%2E%2C+Shi%2C+H%2E%2C+YU%2C+J%2E%2C+Liu%2C+Z%2E%2C+Zhang%2C+Y%2E%2C+Kwok%2C+J%2E%2C+Li%2C+Z%2E%2C+Weller%2C+A%2E%2C+and+Liu%2C+W%2E+Metamath%3A+Bootstrap+your+own+mathematical+questions+for+large+language+models%2E+In+The+Twelfth+Internat) [our card](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/)
48. Yuan, Z., Yuan, H., Li, C., Dong, G., Tan, C., and Zhou, C. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825, 2023. [find](https://scholar.google.com/scholar?q=Yuan%2C+Z%2E%2C+Yuan%2C+H%2E%2C+Li%2C+C%2E%2C+Dong%2C+G%2E%2C+Tan%2C+C%2E%2C+and+Zhou%2C+C%2E+Scaling+relationship+on+learning+mathematical+reasoning+with+large+language+models%2E+arXiv+preprint+arXiv%3A2308%2E01825%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/yuan-2023-scaling-relationship-learning-mathematical-reasoning/)
49. Yue, X., Qu, X., Zhang, G., Fu, Y., Huang, W., Sun, H., Su, Y., and Chen, W. MAmmoTH: Building math generalist models through hybrid instruction tuning. In The Twelfth International Conference on Learning Representations, 2024a. [find](https://scholar.google.com/scholar?q=Yue%2C+X%2E%2C+Qu%2C+X%2E%2C+Zhang%2C+G%2E%2C+Fu%2C+Y%2E%2C+Huang%2C+W%2E%2C+Sun%2C+H%2E%2C+Su%2C+Y%2E%2C+and+Chen%2C+W%2E+MAmmoTH%3A+Building+math+generalist+models+through+hybrid+instruction+tuning%2E+In+The+Twelfth+International+Conference+on+Lear) [our card](https://md.research.elamur.ai/free/en/papers/yue-2023-mammoth-building-math-generalist-models/)
50. Yue, X., Zheng, T., Zhang, G., and Chen, W. Mammoth2: Scaling instructions from the web. NeurIPS 2024, 2024b. [find](https://scholar.google.com/scholar?q=Yue%2C+X%2E%2C+Zheng%2C+T%2E%2C+Zhang%2C+G%2E%2C+and+Chen%2C+W%2E+Mammoth2%3A+Scaling+instructions+from+the+web%2E+NeurIPS+2024%2C+2024b%2E) [our card](https://md.research.elamur.ai/free/en/papers/yue-2024-mammoth2-scaling-instructions-web/)
51. Zelikman, E., Wu, Y., Mu, J., and Goodman, N. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476–15488, 2022. [find](https://scholar.google.com/scholar?q=Zelikman%2C+E%2E%2C+Wu%2C+Y%2E%2C+Mu%2C+J%2E%2C+and+Goodman%2C+N%2E+Star%3A+Bootstrapping+reasoning+with+reasoning%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A15476%E2%80%9315488%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/)
52. Zeng, W., Huang, Y., Liu, W., He, K., Liu, Q., Ma, Z., and He, J. 7b model and 8k examples: Emerging reasoning with reinforcement learning is both effective and efficient. [https://hkust-nlp.notion.site/simplerl-reason](https://hkust-nlp.notion.site/simplerl-reason), 2025. Notion Blog. [↗](https://hkust-nlp.notion.site/simplerl-reason)
53. Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., Barrett, C., and Sheng, Y. SGLang: Efficient execution of structured language model programs. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [find](https://scholar.google.com/scholar?q=Zheng%2C+L%2E%2C+Yin%2C+L%2E%2C+Xie%2C+Z%2E%2C+Sun%2C+C%2E%2C+Huang%2C+J%2E%2C+Yu%2C+C%2E+H%2E%2C+Cao%2C+S%2E%2C+Kozyrakis%2C+C%2E%2C+Stoica%2C+I%2E%2C+Gonzalez%2C+J%2E+E%2E%2C+Barrett%2C+C%2E%2C+and+Sheng%2C+Y%2E+SGLang%3A+Efficient+execution+of+structured+language+model+pro) [our card](https://md.research.elamur.ai/free/en/papers/zheng-2023-sglang-efficient-execution-structured-language/)