---
title: "Spurious Rewards: Rethinking Training Signals in RLVR (2025)"
description: "They replace the usual ground-truth reward with progressively weaker signals… 10.0 points. 39 references, cited by 52 papers here."
authors: ["Rulin Shao", "Shuyue Stella Li", "Rui Xin", "Scott Geng", "Yiping Wang", "Sewoong Oh", "Simon Shaolei Du", "Nathan Lambert", "Sewon Min", "Ranjay Krishna", "Yulia Tsvetkov", "Hannaneh Hajishirzi", "Pang Wei Koh", "Luke Zettlemoyer"]
year: 2025
venue: "arXiv:2506.10947"
source: "https://arxiv.org/abs/2506.10947"
doi: "10.48550/arxiv.2506.10947"
arxiv: "2506.10947"
licence: "CC BY-NC-SA 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-nc-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shao-2025-spurious-rewards-rethinking-training-signals/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shao-2025-spurious-rewards-rethinking-training-signals/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / Spurious Rewards: Rethinking Training Signals in RLVR

# Spurious Rewards: Rethinking Training Signals in RLVR

**Authors:** [Rulin Shao](https://md.research.elamur.ai/free/en/authors/rulin-shao/), [Shuyue Stella Li](https://md.research.elamur.ai/free/en/authors/shuyue-stella-li/), [Rui Xin](https://md.research.elamur.ai/free/en/authors/rui-xin/), Scott Geng, [Yiping Wang](https://md.research.elamur.ai/free/en/authors/yiping-wang/), Sewoong Oh, Simon Shaolei Du, [Nathan Lambert](https://md.research.elamur.ai/free/en/authors/nathan-lambert/), [Sewon Min](https://md.research.elamur.ai/free/en/authors/sewon-min/), [Ranjay Krishna](https://md.research.elamur.ai/free/en/authors/ranjay-krishna/), [Yulia Tsvetkov](https://md.research.elamur.ai/free/en/authors/yulia-tsvetkov/), [Hannaneh Hajishirzi](https://md.research.elamur.ai/free/en/authors/hannaneh-hajishirzi/), [Pang Wei Koh](https://md.research.elamur.ai/free/en/authors/pang-wei-koh/) and [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/) · **Year:** 2025 · **Venue:** arXiv:2506.10947 · **Licence:** [CC BY-NC-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-nc-sa/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2506.10947) · [PDF](https://arxiv.org/pdf/2506.10947) · [DOI 10.48550/arxiv.2506.10947](https://doi.org/10.48550/arxiv.2506.10947) · [arXiv 2506.10947](https://arxiv.org/abs/2506.10947) · [OpenAlex](https://openalex.org/W4417357116)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [52](https://md.research.elamur.ai/free/en/papers/shao-2025-spurious-rewards-rethinking-training-signals/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Reinforcement learning with verifiable rewards (RLVR) is credited with improving math reasoning in language models, but it is unclear what in the training signal actually causes the gains. The authors test this by fine-tuning models with GRPO on DeepScaleR questions. They replace the usual ground-truth reward with progressively weaker signals: majority-vote pseudo-labels, a format check for a non-empty \boxed{} answer, rewards for deliberately wrong labels, and coin-flip rewards.

On Qwen2.5-Math, nearly every one of these signals raises MATH-500 and AMC accuracy almost as much as correct supervision does. Llama3, OLMo2 and, to a lesser degree, Qwen2.5 gain little or get worse. The authors attribute this to the clipping term in GRPO, which tends to reinforce behaviours the base model already prefers. For Qwen2.5-Math, the main such behaviour is "code reasoning": writing solutions in Python without running them.

The study warns that RLVR conclusions drawn from one model family can be misleading. It recommends checking results on several models and using random or format rewards as sanity baselines.

## Key points
- After RLVR on Qwen2.5-Math-7B, MATH-500 gains were +29.1 points with ground-truth rewards, +24.1 with incorrect-label rewards and +21.4 with random rewards (reward of 1 with probability γ = 0.5).
- Spurious rewards give minimal gains or degrade performance on Llama3.1-8B-Instruct and OLMo2-7B. This shows that pretraining priors shape RLVR outcomes.
- In Qwen2.5-Math-7B, code reasoning appears in about 65% of answers before training and in over 90% after spurious-reward RLVR. Answers that use it are 60.9% accurate, compared with 28.0% for answers that do not.
- Forcing responses to begin with a Python cue raises MATH-500 accuracy by 24.2, 15.0 and 10.0 points for Qwen2.5-Math-1.5B, Qwen2.5-Math-7B and Qwen2.5-1.5B. The same prompt hurts Llama and OLMo2.
- A reward given only for containing the string "python" pushes code usage in Qwen2.5-Math-7B above 99% within 20 steps. The authors stress that spurious rewards are analysis tools, not a training recipe.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, Yulia Tsvetkov, Hannaneh Hajishirzi, Pang Wei Koh and Luke Zettlemoyer. 2025. Spurious Rewards: Rethinking Training Signals in RLVR. arXiv:2506.10947.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{shao2025spurious,
  title = {{Spurious Rewards: Rethinking Training Signals in RLVR}},
  author = {Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, Yulia Tsvetkov, Hannaneh Hajishirzi, Pang Wei Koh and Luke Zettlemoyer},
  year = {2025},
  doi = {10.48550/arxiv.2506.10947},
  eprint = {2506.10947},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2506.10947}
}
```

- [← previous AEGIS2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails](https://md.research.elamur.ai/free/en/papers/ghosh-2025-aegis2-0-diverse-ai-safety/)
- [next → Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors](https://md.research.elamur.ai/free/en/papers/maurya-2025-unifying-ai-tutor-evaluation-evaluation/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/)

## References

39 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. AI, E., :, Shah, D. J., Rushton, P., Singla, S., Parmar, M., Smith, K., Vanjani, Y., Vaswani, A., Chaluvaraju, A., Hojel, A., Ma, A., Thomas, A., Polloreno, A., Tanwer, A., Sibai, B. D., Mansingka, D. S., Shivaprasad, D., Shah, I., Stratos, K., Nguyen, K., Callahan, M., Pust, M., Iyer, M., Monk, P.,… — [Rethinking Reflection in Pre-Training](https://md.research.elamur.ai/free/en/papers/shah-2025-rethinking-reflection-pre-training/)
2. Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., Chen, C., Olsson, C., Olah, C., Hernandez, D., Drain, D., Ganguli, D., Li, D., Tran-Johnson, E., Perez, E., Kerr, J., Mueller, J., Ladish, J., Landau, J., Ndousse, K., Lukosuite,… — [Constitutional AI: Harmlessness from AI Feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
3. Choshen, L., Fox, L., Aizenbud, Z., and Abend, O. On the weaknesses of reinforcement learning for neural machine translation, 2020. URL https://arxiv.org/abs/1907.01752. — [arXiv:1907.01752](https://arxiv.org/abs/1907.01752) [↗](https://arxiv.org/abs/1907.01752)
4. Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A. S., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A., Rozière, B., Biron, B., T… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
5. Gandhi, K., Chakravarthy, A., Singh, A., Lile, N., and Goodman, N. D. [Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars](https://md.research.elamur.ai/free/en/papers/gandhi-2025-cognitive-behaviors-that-enable-self/). arXiv preprint arXiv:2503.01307, 2025.
6. Gao, J., Xu, S., Ye, W., Liu, W., He, C., Fu, W., Mei, Z., Wang, G., and Wu, Y. On designing effective rl reward at training time for llm reasoning. arXiv preprint arXiv:2410.15115, 2024. — [arXiv:2410.15115](https://arxiv.org/abs/2410.15115) [find](https://scholar.google.com/scholar?q=Gao%2C+J%2E%2C+Xu%2C+S%2E%2C+Ye%2C+W%2E%2C+Liu%2C+W%2E%2C+He%2C+C%2E%2C+Fu%2C+W%2E%2C+Mei%2C+Z%2E%2C+Wang%2C+G%2E%2C+and+Wu%2C+Y%2E+On+designing+effective+rl+reward+at+training+time+for+llm+reasoning%2E+arXiv+preprint+arXiv%3A2410%2E15115%2C+2024%2E+%E2%80%94+%5BarXiv%3A241)
7. Gao, Z., Chen, L., Zhou, J., and Dai, B. One-shot entropy minimization, 2025. URL https://arxiv.org/abs/2505.20282. — [arXiv:2505.20282](https://arxiv.org/abs/2505.20282) [↗](https://arxiv.org/abs/2505.20282)
8. Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., Zhang, X., Yu, X., Wu, Y., Wu, Z. F., Gou, Z., Shao, Z., Li, Z., Gao, Z., Liu, A., Xue, B., Wang, B., Wu, B., Feng, B., Lu, C., Zhao, C., Deng, C., Ruan, C., Dai, D., Chen, D., Ji, D., Li, E., Lin, F., Dai,… — [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
9. Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874, 2021.
10. Hochlehnert, A., Bhatnagar, H., Udandarao, V., Albanie, S., Prabhu, A., and Bethge, M. A sober look at progress in language model reasoning: Pitfalls and paths to reproducibility, 2025. URL https://arxiv.org/abs/2504.07086. [↗](https://arxiv.org/abs/2504.07086)
11. Hu, J., Wu, X., Shen, W., Liu, J. K., Zhu, Z., Wang, W., Jiang, S., Wang, H., Chen, H., Chen, B., Fang, W., Xianyu, Cao, Y., Xu, H., and Liu, Y. [Openrlhf: An easy-to-use, scalable and high-performance rlhf framework](https://md.research.elamur.ai/free/en/papers/hu-2024-openrlhf-easy-use-scalable/), 2025a. URL https://arxiv.org/abs/2405.11143. [↗](https://arxiv.org/abs/2405.11143)
12. Hu, J., Zhang, Y., Han, Q., Jiang, D., Zhang, X., and Shum, H.-Y. [Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model](https://md.research.elamur.ai/free/en/papers/hu-2025-open-reasoner-zero-open-source/). arXiv preprint arXiv:2503.24290, 2025b.
13. Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F., Miranda, L. J. V., Liu, A., Dziri, N., Lyu, S., Gu, Y., Malik, S., Graf, V., Hwang, J. D., Yang, J., Bras, R. L., Tafjord, O., Wilhelm, C., Soldaini, L., Smith, N. A., Wang, Y., Dasigi, P., and Hajishirzi, H. Tülu 3: Pushing… — [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
14. Li, J., Beeching, E., Tunstall, L., Lipkin, B., Soletskyi, R., Huang, S. C., Rasul, K., Yu, L., Jiang, A., Shen, Z., Qin, Z., Dong, B., Zhou, L., Fleureau, Y., Lample, G., and Polu, S. Numinamath. [https://huggingface.co/AI-MO/NuminaMath-CoT](https://github.com/project-numina/aimo-progress-prize/blo… [find](https://scholar.google.com/scholar?q=Li%2C+J%2E%2C+Beeching%2C+E%2E%2C+Tunstall%2C+L%2E%2C+Lipkin%2C+B%2E%2C+Soletskyi%2C+R%2E%2C+Huang%2C+S%2E+C%2E%2C+Rasul%2C+K%2E%2C+Yu%2C+L%2E%2C+Jiang%2C+A%2E%2C+Shen%2C+Z%2E%2C+Qin%2C+Z%2E%2C+Dong%2C+B%2E%2C+Zhou%2C+L%2E%2C+Fleureau%2C+Y%2E%2C+Lample%2C+G%2E%2C+and+Polu%2C+S%2E+Numinamath%2E+%5Bht)
15. Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2023. — [Let's Verify Step by Step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)
16. Liu, Z., Chen, C., Li, W., Qi, P., Pang, T., Du, C., Lee, W. S., and Lin, M. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/). arXiv preprint arXiv:2503.20783, 2025.
17. Luo, M., Tan, S., Wong, J., Shi, X., Tang, W. Y., Roongta, M., Cai, C., Luo, J., Li, L. E., Popa, R. A., and Stoica, I. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. [https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Mode](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Mode) 2025. Notion Blog. [↗](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Mode)
18. OLMo, T., Walsh, P., Soldaini, L., Groeneveld, D., Lo, K., Arora, S., Bhagia, A., Gu, Y., Huang, S., Jordan, M., Lambert, N., Schwenk, D., Tafjord, O., Anderson, T., Atkinson, D., Brahman, F., Clark, C., Dasigi, P., Dziri, N., Guerquin, M., Ivison, H., Koh, P. W., Liu, J., Malik, S., Merrill, W., Mi… [find](https://scholar.google.com/scholar?q=OLMo%2C+T%2E%2C+Walsh%2C+P%2E%2C+Soldaini%2C+L%2E%2C+Groeneveld%2C+D%2E%2C+Lo%2C+K%2E%2C+Arora%2C+S%2E%2C+Bhagia%2C+A%2E%2C+Gu%2C+Y%2E%2C+Huang%2C+S%2E%2C+Jordan%2C+M%2E%2C+Lambert%2C+N%2E%2C+Schwenk%2C+D%2E%2C+Tafjord%2C+O%2E%2C+Anderson%2C+T%2E%2C+Atkinson%2C+D%2E%2C+Brahman%2C+F%2E%2C+Clark%2C+)
19. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. Training language models to follow instructions with… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
20. Prabhudesai, M., Chen, L., Ippoliti, A., Fragkiadaki, K., Liu, H., and Pathak, D. Maximizing confidence alone improves reasoning, 2025. URL https://arxiv.org/abs/2505.22660. — [arXiv:2505.22660](https://arxiv.org/abs/2505.22660) [↗](https://arxiv.org/abs/2505.22660)
21. Prasad, A., Yuan, W., Pang, R. Y., Xu, J., Fazel-Zarandi, M., Bansal, M., Sukhbaatar, S., Weston, J., and Yu, J. Self-consistency preference optimization. arXiv preprint arXiv:2411.04109, 2024. [find](https://scholar.google.com/scholar?q=Prasad%2C+A%2E%2C+Yuan%2C+W%2E%2C+Pang%2C+R%2E+Y%2E%2C+Xu%2C+J%2E%2C+Fazel%2DZarandi%2C+M%2E%2C+Bansal%2C+M%2E%2C+Sukhbaatar%2C+S%2E%2C+Weston%2C+J%2E%2C+and+Yu%2C+J%2E+Self%2Dconsistency+preference+optimization%2E+arXiv+preprint+arXiv%3A2411%2E04109%2C+2024%2E)
22. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms, 2017. URL https://arxiv.org/abs/1707.06347. [↗](https://arxiv.org/abs/1707.06347)
23. Sclar, M., Choi, Y., Tsvetkov, Y., and Suhr, A. Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting, 2024. URL https://arxiv.org/abs/2310.11324. [↗](https://arxiv.org/abs/2310.11324)
24. Shafayat, S., Tajwar, F., Salakhutdinov, R., Schneider, J., and Zanette, A. Can large reasoning models self-train?, 2025. URL https://arxiv.org/abs/2505.21444. [↗](https://arxiv.org/abs/2505.21444)
25. Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/), 2024. URL https://arxiv.org/abs/2402.03300. [↗](https://arxiv.org/abs/2402.03300)
26. Wang, J., Fang, M., Wan, Z., Wen, M., Zhu, J., Liu, A., Gong, Z., Song, Y., Chen, L., Ni, L. M., Yang, L., Wen, Y., and Zhang, W. Openr: An open source framework for advanced reasoning with large language models, 2024. URL https://arxiv.org/abs/2410.09671. [↗](https://arxiv.org/abs/2410.09671)
27. Wang, S., Yu, L., Gao, C., Zheng, C., Liu, S., Lu, R., Dang, K., Chen, X., Yang, J., Zhang, Z., Liu, Y., Yang, A., Zhao, A., Yue, Y., Song, S., Yu, B., Huang, G., and Lin, J. [Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/), 2025a. URL http…
28. Wang, Y., Yang, Q., Zeng, Z., Ren, L., Liu, L., Peng, B., Cheng, H., He, X., Wang, K., Gao, J., Chen, W., Wang, S., Du, S. S., and Shen, Y. [Reinforcement learning for reasoning in large language models with one training example](https://md.research.elamur.ai/free/en/papers/wang-2025-reinforcement-learning-reasoning-large-language/), 2025b. URL https://arxiv.org/abs/2504.20571. [↗](https://arxiv.org/abs/2504.20571)
29. Wen, L., Cai, Y., Xiao, F., He, X., An, Q., Duan, Z., Du, Y., Liu, J., Tang, L., Lv, X., Zou, H., Deng, Y., Jia, S., and Zhang, X. [Light-r1: Curriculum SFT, DPO and RL for long COT from scratch and beyond](https://md.research.elamur.ai/free/en/papers/wen-2025-light-r1-curriculum-sft-dpo/). In Rehm, G. and Li, Y. (eds.), Proceedings of the 63rd Annual Meeting of the Association for C…
30. Xie, T., Gao, Z., Ren, Q., Luo, H., Hong, Y., Dai, B., Zhou, J., Qiu, K., Wu, Z., and Luo, C. [Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning](https://md.research.elamur.ai/free/en/papers/xie-2025-logic-rl-unleashing-llm/). arXiv preprint arXiv:2502.14768, 2025.
31. Yang, A., Zhang, B., Hui, B., Gao, B., Yu, B., Li, C., Liu, D., Tu, J., Zhou, J., Lin, J., Lu, K., Xue, M., Lin, R., Liu, T., Ren, X., and Zhang, Z. [Qwen2.5-math technical report: Toward mathematical expert model via self-improvement](https://md.research.elamur.ai/free/en/papers/yang-2024-qwen2-5-math-technical-report/), 2024a. URL https://arxiv.org/abs/2409.12122. [↗](https://arxiv.org/abs/2409.12122)
32. Yang, Q. A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, T., Xia, T… [find](https://scholar.google.com/scholar?q=Yang%2C+Q%2E+A%2E%2C+Yang%2C+B%2E%2C+Zhang%2C+B%2E%2C+Hui%2C+B%2E%2C+Zheng%2C+B%2E%2C+Yu%2C+B%2E%2C+Li%2C+C%2E%2C+Liu%2C+D%2E%2C+Huang%2C+F%2E%2C+Dong%2C+G%2E%2C+Wei%2C+H%2E%2C+Lin%2C+H%2E%2C+Yang%2C+J%2E%2C+Tu%2C+J%2E%2C+Zhang%2C+J%2E%2C+Yang%2C+J%2E%2C+Yang%2C+J%2E%2C+Zhou%2C+J%2E%2C+Lin%2C+J%2E%2C+Dang%2C+K%2E%2C+Lu%2C+)
33. Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., YuYue, Dai, W., Fan, T., Liu, G., Liu, J., Liu, L., Liu, X., Lin, H., Lin, Z., Ma, B., Sheng, G., Tong, Y., Zhang, C., Zhang, M., Zhang, R., Zhang, W., Zhu, H., Zhu, J., Chen, J., Chen, J., Wang, C., Yu, H., Song, Y., Wei, X., Zhou, H., Liu, J., Ma, W.-… — [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/)
34. Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Song, S., and Huang, G. [Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/) arXiv preprint arXiv:2504.13837, 2025.
35. Zeng, W., Huang, Y., Liu, Q., Liu, W., He, K., Ma, Z., and He, J. [Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/). arXiv preprint arXiv:2503.18892, 2025.
36. Zhang, Q., Wu, H., Zhang, C., Zhao, P., and Bian, Y. Right question is already half the answer: Fully unsupervised llm reasoning incentivization. arXiv preprint arXiv:2504.05812, 2025. [find](https://scholar.google.com/scholar?q=Zhang%2C+Q%2E%2C+Wu%2C+H%2E%2C+Zhang%2C+C%2E%2C+Zhao%2C+P%2E%2C+and+Bian%2C+Y%2E+Right+question+is+already+half+the+answer%3A+Fully+unsupervised+llm+reasoning+incentivization%2E+arXiv+preprint+arXiv%3A2504%2E05812%2C+2025%2E)
37. Zhao, R., Meterez, A., Kakade, S., Pehlevan, C., Jelassi, S., and Malach, E. Echo chamber: Rl post-training amplifies behaviors learned in pretraining. arXiv preprint arXiv:2504.07912, 2025a. [find](https://scholar.google.com/scholar?q=Zhao%2C+R%2E%2C+Meterez%2C+A%2E%2C+Kakade%2C+S%2E%2C+Pehlevan%2C+C%2E%2C+Jelassi%2C+S%2E%2C+and+Malach%2C+E%2E+Echo+chamber%3A+Rl+post%2Dtraining+amplifies+behaviors+learned+in+pretraining%2E+arXiv+preprint+arXiv%3A2504%2E07912%2C+2025a%2E)
38. Zhao, X., Kang, Z., Feng, A., Levine, S., and Song, D. Learning to reason without external rewards. arXiv preprint arXiv:2505.19590, 2025b. [find](https://scholar.google.com/scholar?q=Zhao%2C+X%2E%2C+Kang%2C+Z%2E%2C+Feng%2C+A%2E%2C+Levine%2C+S%2E%2C+and+Song%2C+D%2E+Learning+to+reason+without+external+rewards%2E+arXiv+preprint+arXiv%3A2505%2E19590%2C+2025b%2E)
39. Zuo, Y., Zhang, K., Qu, S., Sheng, L., Zhu, X., Qi, B., Sun, Y., Cui, G., Ding, N., and Zhou, B. [Ttrl: Test-time reinforcement learning](https://md.research.elamur.ai/free/en/papers/zuo-2025-ttrl-test-time-reinforcement-learning/). arXiv preprint arXiv:2504.16084, 2025.

*Updated 2 November 2026*