---
title: "AReaL — paper by Fu et al. (2025) — summary and ideas"
description: "In these designs, generation and training alternate in lockstep, so every batch waits for its longest… 63 references, cited by 76 papers here."
authors: ["Wei Fu", "Jiaxuan Gao", "Xujie Shen", "Chen Zhu", "Zhiyu Mei", "Chuyi He", "Shusheng Xu", "Guo Wei", "Jun Mei", "Jiashu Wang", "Tongkai Yang", "Binhang Yuan", "Yi Wu"]
year: 2025
venue: "arXiv:2505.24298"
source: "https://arxiv.org/abs/2505.24298"
doi: "10.48550/arxiv.2505.24298"
arxiv: "2505.24298"
licence: "CC BY-NC-ND 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-nc-nd/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/fu-2025-areal-large-scale-asynchronous-reinforcement/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/fu-2025-areal-large-scale-asynchronous-reinforcement/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning

# AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning

**Authors:** Wei Fu, Jiaxuan Gao, Xujie Shen, [Chen Zhu](https://md.research.elamur.ai/free/en/authors/chen-zhu/), Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, Tongkai Yang, [Binhang Yuan](https://md.research.elamur.ai/free/en/authors/binhang-yuan/) and Yi Wu · **Year:** 2025 · **Venue:** [arXiv:2505.24298](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) ([NeurIPS 2025](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2025/)) · **Licence:** [CC BY-NC-ND 4.0 (arXiv)](http://creativecommons.org/licenses/by-nc-nd/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2505.24298) · [PDF](https://arxiv.org/pdf/2505.24298) · [DOI 10.48550/arxiv.2505.24298](https://doi.org/10.48550/arxiv.2505.24298) · [arXiv 2505.24298](https://arxiv.org/abs/2505.24298) · [OpenAlex](https://openalex.org/W4414856864)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [76](https://md.research.elamur.ai/free/en/papers/fu-2025-areal-large-scale-asynchronous-reinforcement/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
RL training of large reasoning models is slowed down by synchronous system designs. In these designs, generation and training alternate in lockstep, so every batch waits for its longest response, and GPUs sit idle. AReaL is a system that fully separates rollout generation from model updates and runs them on separate GPU clusters. Rollout workers produce trajectories in a continuous stream. Trainer workers update the model as soon as enough samples have accumulated, and new weights are then pushed back to the generators.

Because one training batch can mix samples from several policy versions, the authors add two safeguards:
- a cap on how stale a sample may be;
- a modified, decoupled PPO objective that tolerates off-policy data.

They also add system optimizations such as interruptible rollouts, token-budget micro-batching and a parallel reward service. On math and code reasoning with models up to 32B parameters, AReaL trains up to 2.77× faster than synchronous systems on equal hardware, with equal or better accuracy. This shows that asynchrony can make RL for LLMs much cheaper.

## Key points
- AReaL was developed by Tsinghua IIIS, Ant Group and HKUST, and presented at NeurIPS 2025. The code is open-source at github.com/inclusionAI/AReaL.
- The abstract reports up to a 2.77× training speedup over synchronous baselines; the introduction cites up to 2.57× higher throughput. Scaling is near-linear up to 512 GPUs. The baseline verl ran out of memory at 32k context with the 32B model.
- In the staleness ablation on a 1.5B math model, naive PPO lost accuracy even with small staleness. With the decoupled objective, a maximum staleness of about 8 steps or less was close to the synchronous oracle, while unbounded staleness still hurt.
- Dynamic micro-batching (a budget of 32,768 tokens per micro-batch, versus 32 fixed micro-batches) gave about 30% higher training throughput on average.
- Interruptible generation increased generation throughput by 12% for 1.5B models and 17% for 7B models on 4 nodes.

## Interesting ideas
- **Bounded staleness as a tunable knob.** Instead of insisting on strictly on-policy data, set a maximum allowed version lag. This turns the trade-off between throughput and accuracy into one explicit parameter, which could be useful in any pipeline where data producers and model consumers run at different speeds.
- **Separating the data-generating policy from the clipping anchor in PPO.** Old rollouts become usable once the trust region is centred on a recent policy rather than on whichever policy generated the sample. The same idea could carry over to other off-policy or replay-based fine-tuning setups.
- **Interrupting generation for weight updates.** In-flight rollouts are stopped when new weights arrive, and a buffer keeps the batch size constant. This avoids waiting on long-tail outputs and suits any workload where output lengths vary widely, such as long-form generation for annotation.
- **Packing micro-batches by token budget.** Batches are filled up to a fixed number of tokens rather than a fixed number of sequences. This is a cheap trick for variable-length training data, such as corpora with very uneven document lengths, that both avoids OOM errors and increases GPU utilization.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 415 works · 94,028 citations
- [Hong Kong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/hong-kong-university-of-science-and-technology/) — University, Hong Kong, China · 170 works · 26,341 citations
- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 201 works · 17,173 citations
- [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/) — University, Hangzhou, China · 224 works · 13,587 citations
- [Ant Group](https://md.research.elamur.ai/free/en/companies/ant-group/) — Company, Hangzhou, China · 51 works · 1,211 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Wei Fu, Jiaxuan Gao, Xujie Shen, Chen Zhu, Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, Tongkai Yang, Binhang Yuan and Yi Wu. 2025. AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning. arXiv:2505.24298.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{fu2025areal,
  title = {{AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning}},
  author = {Wei Fu, Jiaxuan Gao, Xujie Shen, Chen Zhu, Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, Tongkai Yang, Binhang Yuan and Yi Wu},
  year = {2025},
  doi = {10.48550/arxiv.2505.24298},
  eprint = {2505.24298},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2505.24298}
}
```

- [← previous Style Over Substance: Evaluation Biases for Large Language Models](https://md.research.elamur.ai/free/en/papers/wu-2025-style-over-substance-evaluation-biases/)
- [next → MoBA: Mixture of Block Attention for Long-Context LLMs](https://md.research.elamur.ai/free/en/papers/lu-2025-moba-mixture-block-attention/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

63 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. B. R. Bartoldson, S. Venkatraman, J. Diffenderfer, M. Jain, T. Ben-Nun, S. Lee, M. Kim, J. Obando-Ceron, Y. Bengio, and B. Kailkhura. Trajectory balance with asynchrony: Decoupling exploration and learning for fast, scalable LLM post-training. CoRR, abs/2503.18929, 2025. doi: 10.48550/ARXIV.2503.189… — [doi:10.48550/arxiv.2503.18929](https://doi.org/10.48550/arxiv.2503.18929) [find](https://scholar.google.com/scholar?q=B%2E+R%2E+Bartoldson%2C+S%2E+Venkatraman%2C+J%2E+Diffenderfer%2C+M%2E+Jain%2C+T%2E+Ben%2DNun%2C+S%2E+Lee%2C+M%2E+Kim%2C+J%2E+Obando%2DCeron%2C+Y%2E+Bengio%2C+and+B%2E+Kailkhura%2E+Trajectory+balance+with+asynchrony%3A+Decoupling+exploration+and+lea)
2. C. Berner, G. Brockman, B. Chan, V. Cheung, P. Debiak, C. Dennison, D. Farhi, Q. Fischer, S. Hashme, C. Hesse, R. Józefowicz, S. Gray, C. Olsson, J. Pachocki, M. Petrov, H. P. de Oliveira Pinto, J. Raiman, T. Salimans, J. Schlatter, J. Schneider, S. Sidor, I. Sutskever, J. Tang, F. Wolski, and S. Zh… [find](https://scholar.google.com/scholar?q=C%2E+Berner%2C+G%2E+Brockman%2C+B%2E+Chan%2C+V%2E+Cheung%2C+P%2E+Debiak%2C+C%2E+Dennison%2C+D%2E+Farhi%2C+Q%2E+Fischer%2C+S%2E+Hashme%2C+C%2E+Hesse%2C+R%2E+J%C3%B3zefowicz%2C+S%2E+Gray%2C+C%2E+Olsson%2C+J%2E+Pachocki%2C+M%2E+Petrov%2C+H%2E+P%2E+de+Oliveira+Pinto%2C+J%2E+Ra)
3. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such,… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
4. Z. Chen, A. May, R. Svirschevski, Y. Huang, M. Ryabinin, Z. Jia, and B. Chen. Sequoia: Scalable and robust speculative decoding. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 129531–1… [find](https://scholar.google.com/scholar?q=Z%2E+Chen%2C+A%2E+May%2C+R%2E+Svirschevski%2C+Y%2E+Huang%2C+M%2E+Ryabinin%2C+Z%2E+Jia%2C+and+B%2E+Chen%2E+Sequoia%3A+Scalable+and+robust+speculative+decoding%2E+In+A%2E+Globerson%2C+L%2E+Mackey%2C+D%2E+Belgrave%2C+A%2E+Fan%2C+U%2E+Paquet%2C+J%2E+Tomczak%2C)
5. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). CoRR, abs/2110.14168, 2021. URL https://arxiv.org/abs/2110.14168. [↗](https://arxiv.org/abs/2110.14168)
6. DeepSeek-AI, D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, X. Zhang, X. Yu, Y. Wu, Z. F. Wu, Z. Gou, Z. Shao, Z. Li, Z. Gao, A. Liu, B. Xue, B. Wang, B. Wu, B. Feng, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, D. Dai, D. Chen, D. Ji, E. Li, F. Lin, F. Dai, F. Lu… — [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
7. L. Espeholt, H. Soyer, R. Munos, K. Simonyan, V. Mnih, T. Ward, Y. Doron, V. Firoiu, T. Harley, I. Dunning, S. Legg, and K. Kavukcuoglu. IMPALA: scalable distributed deep-rl with importance weighted actor-learner architectures. In J. G. Dy and A. Krause, editors, Proceedings of the 35th Internationa… [find](https://scholar.google.com/scholar?q=L%2E+Espeholt%2C+H%2E+Soyer%2C+R%2E+Munos%2C+K%2E+Simonyan%2C+V%2E+Mnih%2C+T%2E+Ward%2C+Y%2E+Doron%2C+V%2E+Firoiu%2C+T%2E+Harley%2C+I%2E+Dunning%2C+S%2E+Legg%2C+and+K%2E+Kavukcuoglu%2E+IMPALA%3A+scalable+distributed+deep%2Drl+with+importance+weighted+a)
8. L. Espeholt, R. Marinier, P. Stanczyk, K. Wang, and M. Michalski. SEED RL: scalable and efficient deep-rl with accelerated central inference. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https://openreview… [find](https://scholar.google.com/scholar?q=L%2E+Espeholt%2C+R%2E+Marinier%2C+P%2E+Stanczyk%2C+K%2E+Wang%2C+and+M%2E+Michalski%2E+SEED+RL%3A+scalable+and+efficient+deep%2Drl+with+accelerated+central+inference%2E+In+8th+International+Conference+on+Learning+Representation)
9. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. [Measuring mathematical problem solving with the MATH dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In J. Vanschoren and S. Yeung, editors, Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Da…
10. J. Hilton, K. Cobbe, and J. Schulman. Batch size-invariance for policy optimization. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New… [find](https://scholar.google.com/scholar?q=J%2E+Hilton%2C+K%2E+Cobbe%2C+and+J%2E+Schulman%2E+Batch+size%2Dinvariance+for+policy+optimization%2E+In+S%2E+Koyejo%2C+S%2E+Mohamed%2C+A%2E+Agarwal%2C+D%2E+Belgrave%2C+K%2E+Cho%2C+and+A%2E+Oh%2C+editors%2C+Advances+in+Neural+Information+Proce)
11. J. Hu, X. Wu, W. Wang, Xianyu, D. Zhang, and Y. Cao. [Openrlhf: An easy-to-use, scalable and high-performance RLHF framework](https://md.research.elamur.ai/free/en/papers/hu-2024-openrlhf-easy-use-scalable/). CoRR, abs/2405.11143, 2024. doi: 10.48550/ARXIV.2405.11143. URL https://doi.org/10.48550/arXiv.2405.11143. [↗](https://doi.org/10.48550/arXiv.2405.11143)
12. J. Hu, Y. Zhang, Q. Han, D. Jiang, X. Zhang, and H. Shum. [Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model](https://md.research.elamur.ai/free/en/papers/hu-2025-open-reasoner-zero-open-source/). CoRR, abs/2503.24290, 2025. doi: 10.48550/ARXIV.2503.24290. URL https://doi.org/10.48550/arXiv.2503.24290. [↗](https://doi.org/10.48550/arXiv.2503.24290)
13. B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Dang, A. Yang, R. Men, F. Huang, X. Ren, X. Ren, J. Zhou, and J. Lin. [Qwen2.5-coder technical report](https://md.research.elamur.ai/free/en/papers/hui-2024-qwen2-5-coder-technical-report/). CoRR, abs/2409.12186, 2024. doi: 10.48550/ARXIV.2409.12186. URL https://doi.org/10.48550/arXiv.2409.12186. [↗](https://doi.org/10.48550/arXiv.2409.12186)
14. N. Jain, K. Han, A. Gu, W. Li, F. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 20…
15. C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. R. Narasimhan. [Swe-bench: Can language models resolve real-world github issues?](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/) In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https://openr…
16. S. Kapturowski, G. Ostrovski, J. Quan, R. Munos, and W. Dabney. Recurrent experience replay in distributed reinforcement learning. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL [https://openreview.net/forum?id=r1…](https://openreview.net/forum?id=r1…) [↗](https://openreview.net/forum?id=r1…)
17. KimiTeam, A. Du, B. Gao, B. Xing, C. Jiang, C. Chen, C. Li, C. Xiao, C. Du, C. Liao, C. Tang, C. Wang, D. Zhang, E. Yuan, E. Lu, F. Tang, F. Sung, G. Wei, G. Lai, H. Guo, H. Zhu, H. Ding, H. Hu, H. Yang, H. Zhang, H. Yao, H. Zhao, H. Lu, H. Li, H. Yu, H. Gao, H. Zheng, H. Yuan, J. Chen, J. Guo, J. S… — [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/)
18. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In J. Flinn, M. I. Seltzer, P. Druschel, A. Kaufmann, and J. Mace, editors, Proceedings of the 29th Symposium on Operating…
19. K. Lei, Y. Jin, M. Zhai, K. Huang, H. Ye, and J. Zhai. PUZZLE: efficiently aligning large language models through light-weight context switch. In S. Bagchi and Y. Zhang, editors, Proceedings of the 2024 USENIX Annual Technical Conference, USENIX ATC 2024, Santa Clara, CA, USA, July 10-12, 2024, page… [find](https://scholar.google.com/scholar?q=K%2E+Lei%2C+Y%2E+Jin%2C+M%2E+Zhai%2C+K%2E+Huang%2C+H%2E+Ye%2C+and+J%2E+Zhai%2E+PUZZLE%3A+efficiently+aligning+large+language+models+through+light%2Dweight+context+switch%2E+In+S%2E+Bagchi+and+Y%2E+Zhang%2C+editors%2C+Proceedings+of+the+20)
20. J. LI, E. Beeching, L. Tunstall, B. Lipkin, R. Soletskyi, S. C. Huang, K. Rasul, L. Yu, A. Jiang, Z. Shen, Z. Qin, B. Dong, L. Zhou, Y. Fleureau, G. Lample, and S. Polu. Numinamath. [https://huggingface.co/AI-MO/NuminaMath-CoT](https://github.com/project-numina/aimo-progress-prize/blob/main/report/n… [find](https://scholar.google.com/scholar?q=J%2E+LI%2C+E%2E+Beeching%2C+L%2E+Tunstall%2C+B%2E+Lipkin%2C+R%2E+Soletskyi%2C+S%2E+C%2E+Huang%2C+K%2E+Rasul%2C+L%2E+Yu%2C+A%2E+Jiang%2C+Z%2E+Shen%2C+Z%2E+Qin%2C+B%2E+Dong%2C+L%2E+Zhou%2C+Y%2E+Fleureau%2C+G%2E+Lample%2C+and+S%2E+Polu%2E+Numinamath%2E+%5Bhttps%3A%2F%2Fhuggingfa)
21. E. Liang, R. Liaw, R. Nishihara, P. Moritz, R. Fox, K. Goldberg, J. Gonzalez, M. I. Jordan, and I. Stoica. Rllib: Abstractions for distributed reinforcement learning. In J. G. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan… [find](https://scholar.google.com/scholar?q=E%2E+Liang%2C+R%2E+Liaw%2C+R%2E+Nishihara%2C+P%2E+Moritz%2C+R%2E+Fox%2C+K%2E+Goldberg%2C+J%2E+Gonzalez%2C+M%2E+I%2E+Jordan%2C+and+I%2E+Stoica%2E+Rllib%3A+Abstractions+for+distributed+reinforcement+learning%2E+In+J%2E+G%2E+Dy+and+A%2E+Krause%2C+editor)
22. B. Y. Lin, R. L. Bras, K. Richardson, A. Sabharwal, R. Poovendran, P. Clark, and Y. Choi. [Zebralogic: On the scaling limits of llms for logical reasoning](https://md.research.elamur.ai/free/en/papers/lin-2025-zebralogic-scaling-limits-llms-logical/), 2025. URL https://arxiv.org/abs/2502.01100. [↗](https://arxiv.org/abs/2502.01100)
23. X. Liu, H. Yu, H. Zhang, Y. Xu, X. Lei, H. Lai, Y. Gu, H. Ding, K. Men, K. Yang, S. Zhang, X. Deng, A. Zeng, Z. Du, C. Zhang, S. Shen, T. Zhang, Y. Su, H. Sun, M. Huang, Y. Dong, and J. Tang. [Agentbench: Evaluating llms as agents](https://md.research.elamur.ai/free/en/papers/liu-2023-agentbench-evaluating-llms-agents/). In The Twelfth International Conference on Learning Representations,…
24. M. Luo, S. Tan, R. Huang, A. Patel, A. Ariyak, Q. Wu, X. Shi, R. Xin, C. Cai, M. Weber, et al. Deepcoder: A fully open-source 14b coder at o3-mini level, 2025. [find](https://scholar.google.com/scholar?q=M%2E+Luo%2C+S%2E+Tan%2C+R%2E+Huang%2C+A%2E+Patel%2C+A%2E+Ariyak%2C+Q%2E+Wu%2C+X%2E+Shi%2C+R%2E+Xin%2C+C%2E+Cai%2C+M%2E+Weber%2C+et+al%2E+Deepcoder%3A+A+fully+open%2Dsource+14b+coder+at+o3%2Dmini+level%2C+2025%2E)
25. M. Luo, S. Tan, J. Wong, X. Shi, W. Y. Tang, M. Roongta, C. Cai, J. Luo, L. E. Li, R. A. Popa, and I. Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. [https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8c…](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8c…) [↗](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8c…)
26. Z. Mei, W. Fu, J. Gao, G. Wang, H. Zhang, and Y. Wu. SRL: scaling distributed reinforcement learning to over ten thousand cores. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL [https://openreview.net/forum?id=…](https://openreview.net/forum?id=…) [↗](https://openreview.net/forum?id=…)
27. Z. Mei, W. Fu, K. Li, G. Wang, H. Zhang, and Y. Wu. Realhf: Optimized RLHF training for large language models through parameter reallocation. CoRR, abs/2406.14088, 2024. doi: 10.48550/ARXIV.2406.14088. URL https://doi.org/10.48550/arXiv.2406.14088. [↗](https://doi.org/10.48550/arXiv.2406.14088)
28. X. Miao, G. Oliaro, Z. Zhang, X. Cheng, Z. Wang, Z. Zhang, R. Y. Y. Wong, A. Zhu, L. Yang, X. Shi, C. Shi, Z. Chen, D. Arfeen, R. Abhyankar, and Z. Jia. Specinfer: Accelerating large language model serving with tree-based speculative inference and verification. In Proceedings of the 29th ACM Interna… — [SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification](https://md.research.elamur.ai/free/en/papers/miao-2023-specinfer-accelerating-generative-large-language/)
29. N. Muennighoff, Z. Yang, W. Shi, X. L. Li, L. Fei-Fei, H. Hajishirzi, L. Zettlemoyer, P. Liang, E. J. Candès, and T. Hashimoto. [s1: Simple test-time scaling](https://md.research.elamur.ai/free/en/papers/muennighoff-2025-s1-simple-test-time-scaling/). CoRR, abs/2501.19393, 2025. doi: 10.48550/ARXIV.2501.19393. URL https://doi.org/10.48550/arXiv.2501.19393. [↗](https://doi.org/10.48550/arXiv.2501.19393)
30. M. Noukhovitch, S. Huang, S. Xhonneux, A. Hosseini, R. Agarwal, and A. C. Courville. Asynchronous RLHF: faster and more efficient off-policy RL for language models. CoRR, abs/2410.18252, 2024. doi: 10.48550/ARXIV.2410.18252. URL https://doi.org/10.48550/arXiv.2410.18252. — [doi:10.48550/arxiv.2410.18252](https://doi.org/10.48550/arxiv.2410.18252) [↗](https://doi.org/10.48550/arXiv.2410.18252)
31. OpenAI, Sep 2024. learning-to-reason-with-llms/. Apr 2025. [find](https://scholar.google.com/scholar?q=OpenAI%2C+Sep+2024%2E+learning%2Dto%2Dreason%2Dwith%2Dllms%2F%2E+Apr+2025%2E)
32. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. F. Christiano, J. Leike, and R. Lowe. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/).…
33. J. Pan, J. Zhang, X. Wang, L. Yuan, H. Peng, and A. Suhr. Tinyzero. [https://github.com/Jiayi-Pan/TinyZero](https://github.com/Jiayi-Pan/TinyZero), 2025. Accessed: 2025-01-24. [↗](https://github.com/Jiayi-Pan/TinyZero)
34. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Z. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, high-performance deep learni… — [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/)
35. L. Phan, A. Gatti, Z. Han, N. Li, J. Hu, H. Zhang, S. Shi, M. Choi, A. Agrawal, A. Chopra, A. Khoja, R. Kim, J. Hausenloy, O. Zhang, M. Mazeika, D. Anderson, T. Nguyen, M. Mahmood, F. Feng, S. Y. Feng, H. Zhao, M. Yu, V. Gangal, C. Zou, Z. Wang, J. P. Wang, P. Kumar, O. Pokutnyi, R. Gerbicz, S. Popo… — [Humanity's Last Exam](https://md.research.elamur.ai/free/en/papers/phan-2025-humanity-s-last-exam/)
36. M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley Series in Probability and Statistics. Wiley, 1994. ISBN 978-0-47161977-2. doi: 10.1002/9780470316887. URL https://doi.org/10.1002/9780470316887. — [doi:10.1002/9780470316887](https://doi.org/10.1002/9780470316887) [↗](https://doi.org/10.1002/9780470316887)
37. S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He. Zero: memory optimizations toward training trillion parameter models. In C. Cuicchi, I. Qualters, and W. T. Kramer, editors, Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020, Virtual… [find](https://scholar.google.com/scholar?q=S%2E+Rajbhandari%2C+J%2E+Rasley%2C+O%2E+Ruwase%2C+and+Y%2E+He%2E+Zero%3A+memory+optimizations+toward+training+trillion+parameter+models%2E+In+C%2E+Cuicchi%2C+I%2E+Qualters%2C+and+W%2E+T%2E+Kramer%2C+editors%2C+Proceedings+of+the+Interna)
38. D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman. [GPQA: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). CoRR, abs/2311.12022, 2023. doi: 10.48550/ARXIV.2311.12022. URL https://doi.org/10.48550/arXiv.2311.12022. [↗](https://doi.org/10.48550/arXiv.2311.12022)
39. N. L. Roux, M. G. Bellemare, J. Lebensold, A. Bergeron, J. Greaves, A. Fréchette, C. Pelletier, E. Thibodeau-Laufer, S. Tóth, and S. Work. Tapered off-policy REINFORCE: stable and efficient reinforcement learning for llms. CoRR, abs/2503.14286, 2025. doi: 10.48550/ARXIV.2503.14286. URL [https://doi.o…](https://doi.o…) — [doi:10.48550/arxiv.2503.14286](https://doi.org/10.48550/arxiv.2503.14286) [↗](https://doi.o…)
40. J. Schulman, P. Moritz, S. Levine, M. I. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. In Y. Bengio and Y. LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track… [find](https://scholar.google.com/scholar?q=J%2E+Schulman%2C+P%2E+Moritz%2C+S%2E+Levine%2C+M%2E+I%2E+Jordan%2C+and+P%2E+Abbeel%2E+High%2Ddimensional+continuous+control+using+generalized+advantage+estimation%2E+In+Y%2E+Bengio+and+Y%2E+LeCun%2C+editors%2C+4th+International+Confer)
41. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017. URL http://arxiv.org/abs/1707.06347. [↗](http://arxiv.org/abs/1707.06347)
42. Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, M. Zhang, Y. K. Li, Y. Wu, and D. Guo. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). CoRR, abs/2402.03300, 2024. doi: 10.48550/ARXIV.2402.03300. URL https://doi.org/10.48550/arXiv.2402.03300. [↗](https://doi.org/10.48550/arXiv.2402.03300)
43. G. Shen, Z. Wang, O. Delalleau, J. Zeng, Y. Dong, D. Egert, S. Sun, J. J. Zhang, S. Jain, A. Taghibakhshi, M. S. Ausin, A. Aithal, and O. Kuchaiev. Nemo-aligner: Scalable toolkit for efficient model alignment. CoRR, abs/2405.01481, 2024. doi: 10.48550/ARXIV.2405.01481. URL https://doi.org/10.48550/a… [find](https://scholar.google.com/scholar?q=G%2E+Shen%2C+Z%2E+Wang%2C+O%2E+Delalleau%2C+J%2E+Zeng%2C+Y%2E+Dong%2C+D%2E+Egert%2C+S%2E+Sun%2C+J%2E+J%2E+Zhang%2C+S%2E+Jain%2C+A%2E+Taghibakhshi%2C+M%2E+S%2E+Ausin%2C+A%2E+Aithal%2C+and+O%2E+Kuchaiev%2E+Nemo%2Daligner%3A+Scalable+toolkit+for+efficient+model+a)
44. G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu. [Hybridflow: A flexible and efficient RLHF framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). In Proceedings of the Twentieth European Conference on Computer Systems, EuroSys 2025, Rotterdam, The Netherlands, 30 March 2025 - 3 April 2025, pages 1279–1297. ACM…
45. M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism. CoRR, abs/1909.08053, 2019. URL http://arxiv.org/abs/1909.08053. [↗](http://arxiv.org/abs/1909.08053)
46. C. V. Snell, J. Lee, K. Xu, and A. Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL https://openreview… [find](https://scholar.google.com/scholar?q=C%2E+V%2E+Snell%2C+J%2E+Lee%2C+K%2E+Xu%2C+and+A%2E+Kumar%2E+Scaling+LLM+test%2Dtime+compute+optimally+can+be+more+effective+than+scaling+parameters+for+reasoning%2E+In+The+Thirteenth+International+Conference+on+Learning+Re)
47. P. I. Team, S. Jaghouar, J. Mattern, J. M. Ong, J. Straube, M. Basra, A. Pazdera, K. Thaman, M. D. Ferrante, F. Gabriel, F. Obeid, K. Erdem, M. Keiblinger, and J. Hagemann. Intellect-2: A reasoning model trained through globally decentralized reinforcement learning, 2025. URL https://arxiv.org/abs/2… — [arXiv:2505.07291](https://arxiv.org/abs/2505.07291) [↗](https://arxiv.org/abs/2…)
48. P. I. Team, S. Jaghouar, J. Mattern, J. M. Ong, J. Straube, M. Basra, A. Pazdera, K. Thaman, M. D. Ferrante, F. Gabriel, F. Obeid, K. Erdem, M. Keiblinger, and J. Hagemann. Intellect-2: A reasoning model trained through globally decentralized reinforcement learning, 2025. URL https://arxiv.org/abs/2… — [arXiv:2505.07291](https://arxiv.org/abs/2505.07291) [↗](https://arxiv.org/abs/2…)
49. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30: Ann…
50. O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, J. Oh, D. Horgan, M. Kroiss, I. Danihelka, A. Huang, L. Sifre, T. Cai, J. P. Agapiou, M. Jaderberg, A. S. Vezhnevets, R. Leblond, T. Pohlen, V. Dalibard, D. Budden, Y. Sulsky,… [find](https://scholar.google.com/scholar?q=O%2E+Vinyals%2C+I%2E+Babuschkin%2C+W%2E+M%2E+Czarnecki%2C+M%2E+Mathieu%2C+A%2E+Dudzik%2C+J%2E+Chung%2C+D%2E+H%2E+Choi%2C+R%2E+Powell%2C+T%2E+Ewalds%2C+P%2E+Georgiev%2C+J%2E+Oh%2C+D%2E+Horgan%2C+M%2E+Kroiss%2C+I%2E+Danihelka%2C+A%2E+Huang%2C+L%2E+Sifre%2C+T%2E+Cai%2C+J%2E+P%2E)
51. J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: A…
52. H. Xin, D. Guo, Z. Shao, Z. Ren, Q. Zhu, B. Liu, C. Ruan, W. Li, and X. Liang. [Deepseek-prover: Advancing theorem proving in llms through large-scale synthetic data](https://md.research.elamur.ai/free/en/papers/xin-2024-deepseek-prover-advancing-theorem-proving/). CoRR, abs/2405.14333, 2024. doi: 10.48550/ARXIV.2405.14333. URL https://doi.org/10.48550/arXiv.2405.14333. [↗](https://doi.org/10.48550/arXiv.2405.14333)
53. A. Yang, B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huang, G. Dong, H. Wei, H. Lin, J. Tang, J. Wang, J. Yang, J. Tu, J. Zhang, J. Ma, J. Yang, J. Xu, J. Zhou, J. Bai, J. He, J. Lin, K. Dang, K. Lu, K. Chen, K. Yang, M. Li, M. Xue, N. Ni, P. Zhang, P. Wang, R. Peng, R. Men,… [find](https://scholar.google.com/scholar?q=A%2E+Yang%2C+B%2E+Yang%2C+B%2E+Hui%2C+B%2E+Zheng%2C+B%2E+Yu%2C+C%2E+Zhou%2C+C%2E+Li%2C+C%2E+Li%2C+D%2E+Liu%2C+F%2E+Huang%2C+G%2E+Dong%2C+H%2E+Wei%2C+H%2E+Lin%2C+J%2E+Tang%2C+J%2E+Wang%2C+J%2E+Yang%2C+J%2E+Tu%2C+J%2E+Zhang%2C+J%2E+Ma%2C+J%2E+Yang%2C+J%2E+Xu%2C+J%2E+Zhou%2C+J%2E+Bai%2C+J%2E+He%2C+)
54. A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang,… [find](https://scholar.google.com/scholar?q=A%2E+Yang%2C+B%2E+Yang%2C+B%2E+Zhang%2C+B%2E+Hui%2C+B%2E+Zheng%2C+B%2E+Yu%2C+C%2E+Li%2C+D%2E+Liu%2C+F%2E+Huang%2C+H%2E+Wei%2C+H%2E+Lin%2C+J%2E+Yang%2C+J%2E+Tu%2C+J%2E+Zhang%2C+J%2E+Yang%2C+J%2E+Yang%2C+J%2E+Zhou%2C+J%2E+Lin%2C+K%2E+Dang%2C+K%2E+Lu%2C+K%2E+Bao%2C+K%2E+Yang%2C+L%2E+Yu%2C+M%2E+Li)
55. A. Yang, B. Zhang, B. Hui, B. Gao, B. Yu, C. Li, D. Liu, J. Tu, J. Zhou, J. Lin, K. Lu, M. Xue, R. Lin, T. Liu, X. Ren, and Z. Zhang. [Qwen2.5-math technical report: Toward mathematical expert model via self-improvement](https://md.research.elamur.ai/free/en/papers/yang-2024-qwen2-5-math-technical-report/). CoRR, abs/2409.12122, 2024. doi: 10.48550/ARXIV.2409.12122. URL https://doi.org/…
56. S. Yao, N. Shinn, P. Razavi, and K. Narasimhan. τ -bench: A benchmark for tool-agent-user interaction in real-world domains. CoRR, abs/2406.[12045, 2024](https://md.research.elamur.ai/free/en/papers/yao-2024-tau-bench-benchmark-tool/). doi: 10.48550/ARXIV.2406.12045. URL https://doi.org/10.48550/arXiv.2406.12045. [↗](https://doi.org/10.48550/arXiv.2406.12045)
57. Z. Ye, L. Chen, R. Lai, W. Lin, Y. Zhang, S. Wang, T. Chen, B. Kasikci, V. Grover, A. Krishnamurthy, and L. Ceze. [Flashinfer: Efficient and customizable attention engine for LLM inference serving](https://md.research.elamur.ai/free/en/papers/ye-2025-flashinfer-efficient-customizable-attention-engine/). CoRR, abs/2501.01005, 2025. doi: 10.48550/ARXIV.2501.01005. URL https://doi.org/10.48550/arXiv.2501.010… [↗](https://doi.org/10.48550/arXiv.2501.010…)
58. A. B. Yoo, M. A. Jette, and M. Grondona. SLURM: simple linux utility for resource management. In D. G. Feitelson, L. Rudolph, and U. Schwiegelshohn, editors, Job Scheduling Strategies for Parallel Processing, 9th International Workshop, JSSPP 2003, Seattle, WA, USA, June 24, 2003, Revised Papers, vo… — [doi:10.1007/10968987\\_3](https://doi.org/10.1007/10968987_3) [find](https://scholar.google.com/scholar?q=A%2E+B%2E+Yoo%2C+M%2E+A%2E+Jette%2C+and+M%2E+Grondona%2E+SLURM%3A+simple+linux+utility+for+resource+management%2E+In+D%2E+G%2E+Feitelson%2C+L%2E+Rudolph%2C+and+U%2E+Schwiegelshohn%2C+editors%2C+Job+Scheduling+Strategies+for+Parallel+Pro)
59. Q. Yu, Z. Zhang, R. Zhu, Y. Yuan, X. Zuo, Y. Yue, T. Fan, G. Liu, L. Liu, X. Liu, H. Lin, Z. Lin, B. Ma, G. Sheng, Y. Tong, C. Zhang, M. Zhang, W. Zhang, H. Zhu, J. Zhu, J. Chen, J. Chen, C. Wang, H. Yu, W. Dai, Y. Song, X. Wei, H. Zhou, J. Liu, W. Ma, Y. Zhang, L. Yan, M. Qiao, Y. Wu, and M. Wang.… — [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/)
60. Y. Yue, Y. Yuan, Q. Yu, X. Zuo, R. Zhu, W. Xu, J. Chen, C. Wang, T. Fan, Z. Du, X. Wei, X. Yu, G. Liu, J. Liu, L. Liu, H. Lin, Z. Lin, B. Ma, C. Zhang, M. Zhang, W. Zhang, H. Zhu, R. Zhang, X. Liu, M. Wang, Y. Wu, and L. Yan. Vapo: Efficient and reliable reinforcement learning for advanced reasoning… — [VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks](https://md.research.elamur.ai/free/en/papers/yue-2025-vapo-efficient-reliable-reinforcement-learning/)
61. Y. Zhao, A. Gu, R. Varma, L. Luo, C. Huang, M. Xu, L. Wright, H. Shojanazeri, M. Ott, S. Shleifer, A. Desmaison, C. Balioglu, P. Damania, B. Nguyen, G. Chauhan, Y. Hao, A. Mathews, and S. Li. Pytorch FSDP: experiences on scaling fully sharded data parallel. Proc. VLDB Endow., 16(12):3848–3860, 2023.… [find](https://scholar.google.com/scholar?q=Y%2E+Zhao%2C+A%2E+Gu%2C+R%2E+Varma%2C+L%2E+Luo%2C+C%2E+Huang%2C+M%2E+Xu%2C+L%2E+Wright%2C+H%2E+Shojanazeri%2C+M%2E+Ott%2C+S%2E+Shleifer%2C+A%2E+Desmaison%2C+C%2E+Balioglu%2C+P%2E+Damania%2C+B%2E+Nguyen%2C+G%2E+Chauhan%2C+Y%2E+Hao%2C+A%2E+Mathews%2C+and+S%2E+Li%2E+Pytorch+)
62. L. Zheng, L. Yin, Z. Xie, C. Sun, J. Huang, C. H. Yu, S. Cao, C. Kozyrakis, I. Stoica, J. E. Gonzalez, C. W. Barrett, and Y. Sheng. [Sglang: Efficient execution of structured language model programs](https://md.research.elamur.ai/free/en/papers/zheng-2023-sglang-efficient-execution-structured-language/). In A. Globersons, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. M. Tomczak, and C. Zhang, editors, Ad…
63. Y. Zhong, Z. Zhang, X. Song, H. Hu, C. Jin, B. Wu, N. Chen, Y. Chen, Y. Zhou, C. Wan, H. Zhou, Y. Jiang, Y. Zhu, and D. Jiang. Streamrl: Scalable, heterogeneous, and elastic rl for llms with disaggregated stream generation, 2025. URL https://arxiv.org/abs/2504.15930. — [arXiv:2504.15930](https://arxiv.org/abs/2504.15930) [↗](https://arxiv.org/abs/2504.15930)

*Updated 2 November 2026*