---
title: "Light-R1 — paper by Wen et al. (2025) — summary and ideas"
description: "Strong long chain-of-thought reasoning models are hard to reproduce because the data behind… 48 references, cited by 74 papers here."
authors: ["Liang Wen", "Yunke Cai", "Fenrui Xiao", "Xin He", "Qi An", "Zhenyu Duan", "Yimin Du", "Junchen Liu", "Lifu Tang", "Xiaowei Lv", "Haosheng Zou", "Yongchao Deng", "Shousheng Jia", "Xiangzheng Zhang"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)"
source: "https://aclanthology.org/2025.acl-industry.24/"
doi: "10.18653/v1/2025.acl-industry.24"
arxiv: "2503.10460"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wen-2025-light-r1-curriculum-sft-dpo/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wen-2025-light-r1-curriculum-sft-dpo/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond

# Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond

**Authors:** Liang Wen, Yunke Cai, Fenrui Xiao, [Xin He](https://md.research.elamur.ai/free/en/authors/xin-he/), Qi An, [Zhenyu Duan](https://md.research.elamur.ai/free/en/authors/zhenyu-duan/), Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia and [Xiangzheng Zhang](https://md.research.elamur.ai/free/en/authors/xiangzheng-zhang/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-industry.24/) · [PDF](https://aclanthology.org/2025.acl-industry.24.pdf) · [DOI 10.18653/v1/2025.acl-industry.24](https://doi.org/10.18653/v1/2025.acl-industry.24) · [arXiv 2503.10460](https://arxiv.org/abs/2503.10460) · [OpenAlex](https://openalex.org/W4412886857)

**Citations:** 10 citations (OpenAlex, 2026-10-14) · [74](https://md.research.elamur.ai/free/en/papers/wen-2025-light-r1-curriculum-sft-dpo/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Strong long chain-of-thought reasoning models are hard to reproduce because the data behind DeepSeek-R1 is proprietary. Light-R1 is an open, low-cost training recipe that uses only public data and models. The authors collect about 1M math questions with ground-truth answers and remove any that overlap with AIME24/25, MATH-500 and GPQA. They filter by difficulty twice, first with DeepScaleR-1.5B-Preview and then with DeepSeek-R1, and keep only DeepSeek-R1 answers verified as correct. Training follows a curriculum: SFT on about 76k problems, a second SFT on the roughly 3k hardest, then DPO with NCA loss and model merging. Starting from Qwen2.5-32B-Instruct, the resulting Light-R1-32B beats DeepSeek-R1-Distill-Qwen-32B in math. The 3k set alone also improves the DeepSeek-R1-Distill models at 7B, 14B and 32B. Finally, GRPO on the 14B model produced Light-R1-14B-DS. During this RL training, reward and response length grew together. The work shows that small models can reach R1-level math reasoning cheaply and reproducibly. All models, data and code are released.

## Key points
- Light-R1-14B-DS scores 74.0 on AIME24 and 60.2 on AIME25, the best among 14B models in math. It beats several 32B models and DeepSeek-R1-Distill-Llama-70B, and it also improves on GPQA even though training was math-only.
- Training Light-R1-32B from scratch cost about $1000 (around 6 hours on 12×H800 GPUs). Light-R1-32B-DS performs on par with QwQ-32B and DeepSeek-R1.
- The evaluation samples 64 responses per question, because using 16 or fewer made scores vary by more than 3 points between runs. With this setup the authors reproduced the reported DeepSeek-R1-Distill and QwQ scores within about 1 point.
- Before RL, the training prompts were filtered offline to keep those that Light-R1-7B-DS solves 25–62.5% of the time. GRPO (implemented in verl) was then run on deduplicated Big-Math, with two stabilizers: a clipped length reward and two-sided importance-sampling clipping. RL took about 220 steps on 128 A100 GPUs.
- The authors found that MATH-500 contains dozens of contaminated questions, while AIME24/25 are clean. Release: github.com/Qihoo360/Light-R1.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Renmin University of China](https://md.research.elamur.ai/free/en/universities/renmin-university-of-china/) — University, Beijing, China · 139 works · 17,752 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia and Xiangzheng Zhang. 2025. Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wen2025light,
  title = {{Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond}},
  author = {Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia and Xiangzheng Zhang},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-industry.24},
  eprint = {2503.10460},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2025.acl-industry.24}
}
```

- [← previous Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Proving](https://md.research.elamur.ai/free/en/papers/lin-2025-goedel-prover-frontier-model-open/)
- [next → Simulating Classroom Education with LLM-Empowered Agents](https://md.research.elamur.ai/free/en/papers/zhang-2025-simulating-classroom-education-llm-empowered/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

48 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alon Albalak, Duy Phung, Nathan Lile, Rafael Rafailov, Kanishk Gandhi, Louis Castricato, Anikait Singh, Chase Blagden, Violet Xiang, Dakota Mahan, and Nick Haber. 2025. Big-math: A large-scale, high-quality math dataset for reinforcement learning in language models. Preprint, arXiv:2502.17387. [find](https://scholar.google.com/scholar?q=Alon+Albalak%2C+Duy+Phung%2C+Nathan+Lile%2C+Rafael+Rafailov%2C+Kanishk+Gandhi%2C+Louis+Castricato%2C+Anikait+Singh%2C+Chase+Blagden%2C+Violet+Xiang%2C+Dakota+Mahan%2C+and+Nick+Haber%2E+2025%2E+Big%2Dmath%3A+A+large%2Dscale%2C+high%2Dq)
2. Huayu Chen, Guande He, Hang Su, and Jun Zhu. 2024. Noise contrastive alignment of language models with explicit rewards. arXiv preprint arXiv:2402.05369. — [arXiv:2402.05369](https://arxiv.org/abs/2402.05369) [find](https://scholar.google.com/scholar?q=Huayu+Chen%2C+Guande+He%2C+Hang+Su%2C+and+Jun+Zhu%2E+2024%2E+Noise+contrastive+alignment+of+language+models+with+explicit+rewards%2E+arXiv+preprint+arXiv%3A2402%2E05369%2E+%E2%80%94+%5BarXiv%3A2402%2E05369%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F240)
3. Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, Jiarui Yuan, Huayu Chen, Kaiyan Zhang, Xingtai Lv, Shuo Wang, Yuan Yao, Xu Han, Hao Peng, Yu Cheng, Zhiyuan Liu, Maosong Sun, Bowen Zhou, and Ning Ding. 2025. [Process Reinforcement th…](https://md.research.elamur.ai/free/en/papers/cui-2025-process-reinforcement-through-implicit-rewards/)
4. DeepSeek-AI. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). Preprint, arXiv:2501.12948.
5. Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. 2024. [Omni-math: A universal olympiad level ma…](https://md.research.elamur.ai/free/en/papers/gao-2024-omni-math-universal-olympiad-level/)
6. Leo Gao, John Schulman, and Jacob Hilton. 2023. [Scaling laws for reward model overoptimization](https://md.research.elamur.ai/free/en/papers/gao-2022-scaling-laws-reward-model-overoptimization/). In International Conference on Machine Learning, pages 10835–10866. PMLR.
7. Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. [Arcee’s MergeKit: A toolkit for merging large language models](https://md.research.elamur.ai/free/en/papers/goddard-2024-arcee-s-mergekit-toolkit-merging/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing:…
8. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021a. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). Preprint, arXiv:2103.03874.
9. Kimi. 2025. [Kimi k1.5: Scaling reinforcement learning with llms](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/). Preprint, arXiv:2501.12599.
10. Bespoke Labs. 2025. Bespoke-stratos: The unreasonable effectiveness of reasoning distillation. Accessed: 2025-01-22. [find](https://scholar.google.com/scholar?q=Bespoke+Labs%2E+2025%2E+Bespoke%2Dstratos%3A+The+unreasonable+effectiveness+of+reasoning+distillation%2E+Accessed%3A+2025%2D01%2D22%2E)
11. Xuefeng Li, Haoyang Zou, and Pengfei Liu. 2025. Limr: Less is more for rl scaling. Preprint, arXiv:2502.11886. [find](https://scholar.google.com/scholar?q=Xuefeng+Li%2C+Haoyang+Zou%2C+and+Pengfei+Liu%2E+2025%2E+Limr%3A+Less+is+more+for+rl+scaling%2E+Preprint%2C+arXiv%3A2502%2E11886%2E)
12. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). Preprint, arXiv:2305.20050.
13. Zichen Liu, Changyu Chen, Wenjun Li, Tianyu Pang, Chao Du, and Min Lin. 2025. There may not be aha moment in r1-zero-like training — a pilot study. [https://oatllm.notion.site/oat-zero](https://oatllm.notion.site/oat-zero). Notion Blog. [↗](https://oatllm.notion.site/oat-zero)
14. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jian-guang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, Yansong Tang, and Dongmei Zhang. 2025a. [Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/). Preprint, arXiv:2308.09583.
15. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. 2025b. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. Notion Blog. [find](https://scholar.google.com/scholar?q=Michael+Luo%2C+Sijun+Tan%2C+Justin+Wong%2C+Xiaoxiang+Shi%2C+William+Y%2E+Tang%2C+Manan+Roongta%2C+Colin+Cai%2C+Jeffrey+Luo%2C+Tianjun+Zhang%2C+Li+Erran+Li%2C+Raluca+Ada+Popa%2C+and+Ion+Stoica%2E+2025b%2E+Deepscaler%3A+Surpassing+o)
16. MAA. 2024. American invitational mathematics examination - aime. In American Invitational Mathematics Examination - AIME 2024. [find](https://scholar.google.com/scholar?q=MAA%2E+2024%2E+American+invitational+mathematics+examination+%2D+aime%2E+In+American+Invitational+Mathematics+Examination+%2D+AIME+2024%2E)
17. Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, Wayne Xin Zhao, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. 2024. [Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems](https://md.research.elamur.ai/free/en/papers/min-2024-imitate-explore-self-improve-reproduction/). Preprint, arXi…
18. MiniMax. 2025. [MiniMax-01: Scaling Foundation Models with Lightning Attention](https://md.research.elamur.ai/free/en/papers/minimax-2025-minimax-01-scaling-foundation/). Preprint, arXiv:2501.08313.
19. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. [s1: Simple test-time scaling](https://md.research.elamur.ai/free/en/papers/muennighoff-2025-s1-simple-test-time-scaling/). arXiv preprint arXiv:2501.19393.
20. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021b. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). NeurIPS.
21. Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, and Heung-Yeung Shum Xiangyu Zhang. 2025. Open-reasoner-zero: An open source approach to scaling reinforcement learning on the base model. [https://github.com/Open-Reasoner-Zero/](https://github.com/Open-Reasoner-Zero/) Open-Reasoner-Zero. [↗](https://github.com/Open-Reasoner-Zero/)
22. OpenAI. 2024. Learning to reason with llms. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024%2E+Learning+to+reason+with+llms%2E)
23. OpenR1. 2025. Open r1: A fully open reproduction of deepseek-r1. [find](https://scholar.google.com/scholar?q=OpenR1%2E+2025%2E+Open+r1%3A+A+fully+open+reproduction+of+deepseek%2Dr1%2E)
24. OpenThoughts. 2025. Open Thoughts. [https://openthoughts.ai.324](https://openthoughts.ai.324) 7 [↗](https://openthoughts.ai.324)
25. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
26. Qwen. 2024. Qwen2.5: A party of foundation models. [find](https://scholar.google.com/scholar?q=Qwen%2E+2024%2E+Qwen2%2E5%3A+A+party+of+foundation+models%2E)
27. Qwen. 2025. Qwq-32b: Embracing the power of reinforcement learning. [find](https://scholar.google.com/scholar?q=Qwen%2E+2025%2E+Qwq%2D32b%3A+Embracing+the+power+of+reinforcement+learning%2E)
28. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Preprint, arXiv:2305.18290.
29. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023. [Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). Preprint, arXiv:2311.12022.
30. John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. 2015. High-dimensional continuous control using generalized advantage estimation. Preprint, arXiv:1506.02438. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Philipp+Moritz%2C+Sergey+Levine%2C+Michael+Jordan%2C+and+Pieter+Abbeel%2E+2015%2E+High%2Ddimensional+continuous+control+using+generalized+advantage+estimation%2E+Preprint%2C+arXiv%3A1506%2E02438%2E)
31. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. Preprint, arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+Preprint%2C+arXiv%3A1707%2E06347%2E)
32. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). Preprint, arXiv:2402.03300.
33. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). arXiv preprint arXiv: 2409.19256.
34. Yifan Song, Guoyin Wang, Sujian Li, and Bill Yuchen Lin. 2024. [The good, the bad, and the greedy: Evaluation of llms should not ignore non-determinism](https://md.research.elamur.ai/free/en/papers/song-2025-good-bad-greedy-evaluation-llms/). arXiv preprint arXiv:2407.10457.
35. Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. 2024. [Openmathinstruct-2: Accelerating ai for math with massive open-source instruction data](https://md.research.elamur.ai/free/en/papers/toshniwal-2024-openmathinstruct-2-accelerating-ai-math/). arXiv preprint arXiv:2410.01560.
36. Peiyi Wang, Lei Li, Zhihong Shao, R. X. Xu, Damai Dai, Yifei Li, Deli Chen, Y. Wu, and Zhifang Sui. 2024. [Math-shepherd: Verify and reinforce llms step-by-step without human annotations](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/). Preprint, arXiv:2312.08935.
37. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In NeurIPS.
38. Zhiheng Xi, Wenxiang Chen, Boyang Hong, Senjie Jin, Rui Zheng, Wei He, Yiwen Ding, Shichun Liu, Xin Guo, Junzhe Wang, Honglin Guo, Wei Shen, Xiaoran Fan, Yuhao Zhou, Shihan Dou, Xiao Wang, Xinbo Zhang, Peng Sun, Tao Gui, Qi Zhang, and Xuanjing Huang. 2024. Training large language models for reasonin… [find](https://scholar.google.com/scholar?q=Zhiheng+Xi%2C+Wenxiang+Chen%2C+Boyang+Hong%2C+Senjie+Jin%2C+Rui+Zheng%2C+Wei+He%2C+Yiwen+Ding%2C+Shichun+Liu%2C+Xin+Guo%2C+Junzhe+Wang%2C+Honglin+Guo%2C+Wei+Shen%2C+Xiaoran+Fan%2C+Yuhao+Zhou%2C+Shihan+Dou%2C+Xiao+Wang%2C+Xinbo+Zhang)
39. Haotian Xu, Xing Wu, Weinong Wang, Zhongzhi Li, Da Zheng, Boyuan Chen, Yi Hu, Shijia Kang, Jiaming Ji, Yingying Zhang, Zhijiang Guo, Yaodong Yang, Muhan Zhang, and Debing Zhang. 2025. Red-star: Does scaling long-cot data unlock better slow-reasoning systems? Preprint, arXiv:2501.11284. [find](https://scholar.google.com/scholar?q=Haotian+Xu%2C+Xing+Wu%2C+Weinong+Wang%2C+Zhongzhi+Li%2C+Da+Zheng%2C+Boyuan+Chen%2C+Yi+Hu%2C+Shijia+Kang%2C+Jiaming+Ji%2C+Yingying+Zhang%2C+Zhijiang+Guo%2C+Yaodong+Yang%2C+Muhan+Zhang%2C+and+Debing+Zhang%2E+2025%2E+Red%2Dstar%3A+Does+s)
40. Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023. [Ties-merging: Resolving interference when merging models](https://md.research.elamur.ai/free/en/papers/yadav-2023-ties-merging-resolving-interference-merging/). Preprint, arXiv:2306.01708.
41. Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. 2025. [Limo: Less is more for reasoning](https://md.research.elamur.ai/free/en/papers/ye-2025-limo-less-more-reasoning/). Preprint, arXiv:2502.03387.
42. Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. 2025. [Demystifying Long Chain-of-Thought Reasoning in LLMs](https://md.research.elamur.ai/free/en/papers/yeo-2025-demystifying-long-chain-thought-reasoning/). Preprint, arXiv:2502.03373.
43. Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024. [Metamath: Bootstrap your own mathematical questions for large language models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/). Preprint, arXiv:2309.12284.
44. Siyu Yuan, Zehui Chen, Zhiheng Xi, Junjie Ye, Zhengyin Du, and Jiecao Chen. 2025a. Agent-r: Training language model agents to reflect via iterative self-training. Preprint, arXiv:2501.11425. — [arXiv:2501.11425](https://arxiv.org/abs/2501.11425) [find](https://scholar.google.com/scholar?q=Siyu+Yuan%2C+Zehui+Chen%2C+Zhiheng+Xi%2C+Junjie+Ye%2C+Zhengyin+Du%2C+and+Jiecao+Chen%2E+2025a%2E+Agent%2Dr%3A+Training+language+model+agents+to+reflect+via+iterative+self%2Dtraining%2E+Preprint%2C+arXiv%3A2501%2E11425%2E+%E2%80%94+%5BarXiv%3A)
45. Yufeng Yuan, Yu Yue, Ruofei Zhu, Tiantian Fan, and Lin Yan. 2025b. What’s Behind PPO’s Collapse in Long-CoT? Value Optimization Holds the Secret. Preprint, arXiv:2503.01491. — [arXiv:2503.01491](https://arxiv.org/abs/2503.01491) [find](https://scholar.google.com/scholar?q=Yufeng+Yuan%2C+Yu+Yue%2C+Ruofei+Zhu%2C+Tiantian+Fan%2C+and+Lin+Yan%2E+2025b%2E+What%E2%80%99s+Behind+PPO%E2%80%99s+Collapse+in+Long%2DCoT%3F+Value+Optimization+Holds+the+Secret%2E+Preprint%2C+arXiv%3A2503%2E01491%2E+%E2%80%94+%5BarXiv%3A2503%2E01491%5D%28https)
46. Weihao Zeng, Yuzhen Huang, Wei Liu, Keqing He, Qian Liu, Zejun Ma, and Junxian He. 2025. 7b model and 8k examples: Emerging reasoning with reinforcement learning is both effective and efficient. [https://hkust-nlp.notion.site/](https://hkust-nlp.notion.site/) simplerl-reason. Notion Blog. [↗](https://hkust-nlp.notion.site/)
47. Hanning Zhang, Jiarui Yao, Chenlu Ye, Wei Xiong, and Tong Zhang. 2025. Online-dpo-r1: Unlocking effective reasoning without the ppo overhead. Notion Blog. [find](https://scholar.google.com/scholar?q=Hanning+Zhang%2C+Jiarui+Yao%2C+Chenlu+Ye%2C+Wei+Xiong%2C+and+Tong+Zhang%2E+2025%2E+Online%2Ddpo%2Dr1%3A+Unlocking+effective+reasoning+without+the+ppo+overhead%2E+Notion+Blog%2E)
48. Haosheng Zou, Xiaowei Lv, Shousheng Jia, and Xiangzheng Zhang. 2024. 360-llama-factory. 325 8 [find](https://scholar.google.com/scholar?q=Haosheng+Zou%2C+Xiaowei+Lv%2C+Shousheng+Jia%2C+and+Xiangzheng+Zhang%2E+2024%2E+360%2Dllama%2Dfactory%2E+325+8)

*Updated 2 November 2026*