---
title: "Harnessing Negative Signals — paper by Xu et al. (2026)"
description: "This paper studies how those incorrect traces can be used in a purely offline setting. 34 references, cited by 2 papers here."
authors: ["Shuyao Xu", "Cheng Peng", "Jiangxuan Long", "Weidi Xu", "Wei Chu", "Yuan Qi"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.74/"
doi: "10.18653/v1/2026.acl-long.74"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xu-2026-harnessing-negative-signals-reinforcement-distillation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xu-2026-harnessing-negative-signals-reinforcement-distillation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) / Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning

# Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning

**Authors:** Shuyao Xu, [Cheng Peng](https://md.research.elamur.ai/free/en/authors/cheng-peng/), Jiangxuan Long, [Weidi Xu](https://md.research.elamur.ai/free/en/authors/weidi-xu/), [Wei Chu](https://md.research.elamur.ai/free/en/authors/wei-chu/) and [Yuan Qi](https://md.research.elamur.ai/free/en/authors/yuan-qi/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.74/) · [PDF](https://aclanthology.org/2026.acl-long.74.pdf) · [DOI 10.18653/v1/2026.acl-long.74](https://doi.org/10.18653/v1/2026.acl-long.74) · [OpenAlex](https://openalex.org/W4414859134)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Distilling reasoning skills from a large teacher into a small student usually relies on rejection sampling. Only teacher traces with verified correct answers are kept, and the failed attempts are discarded. This paper studies how those incorrect traces can be used in a purely offline setting. The authors use the Open-R1 data, which contains DeepSeek-R1 traces, and train in two stages. First comes standard SFT on correct traces. Then the model is refined on pairs that match a correct and an incorrect trace for the same problem.

They show that DPO and SimPO face a trade-off controlled by β: stronger regularization stabilizes training but caps peak accuracy. Letting β approach zero yields a simple reference-free, REINFORCE-style loss. Their REDI objective adds asymmetric weighting that scales down gradients from negative traces, which keeps training stable. The resulting Qwen-REDI-1.5B was trained on 131k open traces. It roughly matches DeepSeek-R1-Distill-Qwen-1.5B, which used 800k proprietary samples. This shows that negative traces can make distillation considerably more data-efficient.

## Key points
- **Headline result:** Qwen-REDI-1.5B scores 83.1% on MATH-500, close to DeepSeek-R1-Distill-Qwen-1.5B's 83.2%.
  - Its average is 49.5 across MATH-500, AIME24, AMC23, Minerva and OlympiadBench (pass@1 over 16 samples).
  - The comparison averages are 48.6 for DeepSeek-R1-Distill-Qwen-1.5B and 45.8 for the SFT-only baseline Qwen-SFT-1.5B-5ep.
- **Controlled comparison:** all methods start from Qwen-SFT-1.5B-3ep (average 41.9).
  - Best-checkpoint averages are 47.2 for DPO, 47.2 for SimPO, 47.7 for the symmetric β→0 loss and 48.3 for REDI with α = 0.8.
  - Lowering α to 0.5 or 0.2 weakened the negative signal and reduced peak performance.
- **Failure mode addressed:** training collapse appears as a sharp drop in the likelihood of both correct and incorrect traces. Down-weighting the negative-trace gradient allows larger learning rates without this collapse.
- **Generalization:**
  - REDI also improves over SFT baselines on Llama-3.2-3B and Qwen2.5-Math-7B.
  - Although training used only math data, it brings out-of-domain gains on GPQA and HumanEval.
  - pass@16 is maintained or improved, so the models remain suitable for later online RL.
- **Limitation and code:** all traces come from a single teacher model. Code is released at github.com/Tim-Siu/reinforcement-distillation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 180 works · 34,304 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

## Cite

Cite as: Shuyao Xu, Cheng Peng, Jiangxuan Long, Weidi Xu, Wei Chu and Yuan Qi. 2026. Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xu2026harnessing,
  title = {{Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning}},
  author = {Shuyao Xu, Cheng Peng, Jiangxuan Long, Weidi Xu, Wei Chu and Yuan Qi},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.74},
  url = {https://doi.org/10.18653/v1/2026.acl-long.74}
}
```

- [← previous HalluTruthQA: A Fine-Grained Benchmark for Hallucination Detection, Localization, and Explanation in Arabic Question Answering](https://md.research.elamur.ai/free/en/papers/bouchekif-2026-hallutruthqa-fine-grained-benchmark-hallucination/)
- [next → HAUNTATTACK: When Attack Follows Reasoning as a Shadow](https://md.research.elamur.ai/free/en/papers/ma-2026-hauntattack-when-attack-follows-reasoning/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/)

## References

34 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 202… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
2. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in ll…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
3. Hugging Face. 2025. Open r1: A fully open reproduction of deepseek-r1. [find](https://scholar.google.com/scholar?q=Hugging+Face%2E+2025%2E+Open+r1%3A+A+fully+open+reproduction+of+deepseek%2Dr1%2E)
4. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, and 5 others.… [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci)
5. Nathan Habib, Clémentine Fourrier, Hynek Kydlíček, Thomas Wolf, and Lewis Tunstall. 2023. Lighteval: A lightweight framework for llm evaluation. [find](https://scholar.google.com/scholar?q=Nathan+Habib%2C+Cl%C3%A9mentine+Fourrier%2C+Hynek+Kydl%C3%AD%C4%8Dek%2C+Thomas+Wolf%2C+and+Lewis+Tunstall%2E+2023%2E+Lighteval%3A+A+lightweight+framework+for+llm+evaluation%2E)
6. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. 2024. [OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). In Proce…
7. Hynek Kydlíček. Math-Verify: Math Verification Library. [find](https://scholar.google.com/scholar?q=Hynek+Kydl%C3%AD%C4%8Dek%2E+Math%2DVerify%3A+Math+Verification+Library%2E)
8. Bespoke Labs. 2025. Bespoke-stratos: The unreasonable effectiveness of reasoning distillation. [https://www.bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning-distillation](https://www.bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning-distillation). Accessed: 2025-01-22. [↗](https://www.bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning-distillation)
9. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. 2022. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). Preprint, arXiv:2…
10. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). arXiv preprint arXiv:2305.20050.
11. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. Preprint, arXiv:1711.05101. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2019%2E+Decoupled+weight+decay+regularization%2E+Preprint%2C+arXiv%3A1711%2E05101%2E)
12. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. 2025. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. Notion Blog. [find](https://scholar.google.com/scholar?q=Michael+Luo%2C+Sijun+Tan%2C+Justin+Wong%2C+Xiaoxiang+Shi%2C+William+Y%2E+Tang%2C+Manan+Roongta%2C+Colin+Cai%2C+Jeffrey+Luo%2C+Li+Erran+Li%2C+Raluca+Ada+Popa%2C+and+Ion+Stoica%2E+2025%2E+Deepscaler%3A+Surpassing+o1%2Dpreview+with+a)
13. Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. SimPO: Simple preference optimization with a reference-free reward. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Yu+Meng%2C+Mengzhou+Xia%2C+and+Danqi+Chen%2E+2024%2E+SimPO%3A+Simple+preference+optimization+with+a+reference%2Dfree+reward%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Neural+Information+Processing+Systems%2E)
14. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. [s1: Simple test-time scaling](https://md.research.elamur.ai/free/en/papers/muennighoff-2025-s1-simple-test-time-scaling/). Preprint, arXiv:2501.19393.
15. OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, and 244 others. 2024. [Openai o1 sy…](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/)
16. Jiayi Pan, Junjie Zhang, Xingyao Wang, Lifan Yuan, Hao Peng, and Alane Suhr. 2025. Tinyzero. [https://github.com/Jiayi-Pan/TinyZero](https://github.com/Jiayi-Pan/TinyZero). Accessed: 2025-01-24. [↗](https://github.com/Jiayi-Pan/TinyZero)
17. Richard Yuanzhe Pang, Weizhe Yuan, He He, Kyunghyun Cho, Sainbayar Sukhbaatar, and Jason E Weston. 2024. [Iterative reasoning preference optimization](https://md.research.elamur.ai/free/en/papers/pang-2024-iterative-reasoning-preference-optimization/). In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
18. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Preprint, arXiv:2305.18290.
19. Noam Razin, Sadhika Malladi, Adithya Bhaskar, Danqi Chen, Sanjeev Arora, and Boris Hanin. 2025. Unintentional unalignment: Likelihood displacement in direct preference optimization. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Noam+Razin%2C+Sadhika+Malladi%2C+Adithya+Bhaskar%2C+Danqi+Chen%2C+Sanjeev+Arora%2C+and+Boris+Hanin%2E+2025%2E+Unintentional+unalignment%3A+Likelihood+displacement+in+direct+preference+optimization%2E+In+The+Thirteenth+)
20. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2024. [GPQA: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). In First Conference on Language Modeling.
21. Yi Ren and Danica J. Sutherland. 2025. Learning dynamics of LLM finetuning. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yi+Ren+and+Danica+J%2E+Sutherland%2E+2025%2E+Learning+dynamics+of+LLM+finetuning%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations%2E)
22. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). Preprint, arXiv:2402.03300.
23. OpenThoughts Team. 2025. [https://open-thoughts.ai](https://open-thoughts.ai). [↗](https://open-thoughts.ai)
24. Hanning Zhang, Jiarui Yao, Chenlu Ye, Wei Xiong, and Tong Zhang. Online-dpo-r1: Unlocking effective reasoning without the ppo overhead, 2025. Notion Blog. [find](https://scholar.google.com/scholar?q=Hanning+Zhang%2C+Jiarui+Yao%2C+Chenlu+Ye%2C+Wei+Xiong%2C+and+Tong+Zhang%2E+Online%2Ddpo%2Dr1%3A+Unlocking+effective+reasoning+without+the+ppo+overhead%2C+2025%2E+Notion+Blog%2E)
25. Wenhao Zhang, Yuexiang Xie, Yuchang Sun, Yanxi Chen, Guoyin Wang, Yaliang Li, Bolin Ding, and Jingren Zhou. 2025. On-policy rl meets off-policy experts: Harmonizing supervised fine-tuning and reinforcement learning via dynamic weighting. arXiv preprint arXiv:2508.11408. — [arXiv:2508.11408](https://arxiv.org/abs/2508.11408) [find](https://scholar.google.com/scholar?q=Wenhao+Zhang%2C+Yuexiang+Xie%2C+Yuchang+Sun%2C+Yanxi+Chen%2C+Guoyin+Wang%2C+Yaliang+Li%2C+Bolin+Ding%2C+and+Jingren+Zhou%2E+2025%2E+On%2Dpolicy+rl+meets+off%2Dpolicy+experts%3A+Harmonizing+supervised+fine%2Dtuning+and+reinforc)
26. Xinyu Zhu, Mengzhou Xia, Zhepei Wei, Wei-Lin Chen, Danqi Chen, and Yu Meng. 2025. The surprising effectiveness of negative reinforcement in llm reasoning. arXiv preprint arXiv:2506.01347. [find](https://scholar.google.com/scholar?q=Xinyu+Zhu%2C+Mengzhou+Xia%2C+Zhepei+Wei%2C+Wei%2DLin+Chen%2C+Danqi+Chen%2C+and+Yu+Meng%2E+2025%2E+The+surprising+effectiveness+of+negative+reinforcement+in+llm+reasoning%2E+arXiv+preprint+arXiv%3A2506%2E01347%2E)
27. Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, and Xiangzheng Zhang. 2025. Lightr1: Curriculum sft, dpo and rl for long cot from scratch and beyond. Preprint, arXiv:2503.10460. [find](https://scholar.google.com/scholar?q=Liang+Wen%2C+Yunke+Cai%2C+Fenrui+Xiao%2C+Xin+He%2C+Qi+An%2C+Zhenyu+Duan%2C+Yimin+Du%2C+Junchen+Liu%2C+Lifu+Tang%2C+Xiaowei+Lv%2C+Haosheng+Zou%2C+Yongchao+Deng%2C+Shousheng+Jia%2C+and+Xiangzheng+Zhang%2E+2025%2E+Lightr1%3A+Curriculum)
28. Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229–256. [find](https://scholar.google.com/scholar?q=Ronald+J+Williams%2E+1992%2E+Simple+statistical+gradient%2Dfollowing+algorithms+for+connectionist+reinforcement+learning%2E+Machine+learning%2C+8%283%29%3A229%E2%80%93256%2E)
29. Jinyang Wu, Chonghua Liao, Mingkuan Feng, Shuai Zhang, Zhengqi Wen, Pengpeng Shao, Huazhe Xu, and Jianhua Tao. 2025. Thought-augmented policy optimization: Bridging external guidance and internal capabilities. arXiv preprint arXiv:2505.15692. — [arXiv:2505.15692](https://arxiv.org/abs/2505.15692) [find](https://scholar.google.com/scholar?q=Jinyang+Wu%2C+Chonghua+Liao%2C+Mingkuan+Feng%2C+Shuai+Zhang%2C+Zhengqi+Wen%2C+Pengpeng+Shao%2C+Huazhe+Xu%2C+and+Jianhua+Tao%2E+2025%2E+Thought%2Daugmented+policy+optimization%3A+Bridging+external+guidance+and+internal+capa)
30. Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu, Yu Cheng, and Yue Zhang. 2025a. Learning to reason under off-policy guidance. arXiv preprint arXiv:2504.14945. [find](https://scholar.google.com/scholar?q=Jianhao+Yan%2C+Yafu+Li%2C+Zican+Hu%2C+Zhi+Wang%2C+Ganqu+Cui%2C+Xiaoye+Qu%2C+Yu+Cheng%2C+and+Yue+Zhang%2E+2025a%2E+Learning+to+reason+under+off%2Dpolicy+guidance%2E+arXiv+preprint+arXiv%3A2504%2E14945%2E)
31. Yuzi Yan, Yibo Miao, Jialian Li, Yipin Zhang, Jian Xie, Zhijie Deng, and Dong Yan. 2025b. 3d-properties: Identifying challenges in dpo and charting a path forward. Preprint, arXiv:2406.07327. — [arXiv:2406.07327](https://arxiv.org/abs/2406.07327) [find](https://scholar.google.com/scholar?q=Yuzi+Yan%2C+Yibo+Miao%2C+Jialian+Li%2C+Yipin+Zhang%2C+Jian+Xie%2C+Zhijie+Deng%2C+and+Dong+Yan%2E+2025b%2E+3d%2Dproperties%3A+Identifying+challenges+in+dpo+and+charting+a+path+forward%2E+Preprint%2C+arXiv%3A2406%2E07327%2E+%E2%80%94+%5BarXiv)
32. Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. 2025. [Limo: Less is more for reasoning](https://md.research.elamur.ai/free/en/papers/ye-2025-limo-less-more-reasoning/). Preprint, arXiv:2502.03387.
33. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. 2025. [Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/) arXiv preprint arXiv:2504.13837.
34. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. 2025. [Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/). Preprint, arXiv:2503.18892.

*Updated 31 October 2026*