---
title: "When Are Teacher Tokens Reliable? Position-Weighted"
description: "To tell these apart, they build a branch-viability diagnostic. 41 references, cited by 8 papers here."
authors: ["Xiaogeng Liu", "Xinyan Wang", "Yingzi Ma", "Yechao Zhang", "Chaowei Xiao"]
year: 2026
venue: "arXiv:2605.21606"
source: "https://arxiv.org/abs/2605.21606"
doi: "10.48550/arxiv.2605.21606"
arxiv: "2605.21606"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2026-teacher-tokens-reliable-position-weighted/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2026-teacher-tokens-reliable-position-weighted/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/) / When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning

# When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning

**Authors:** [Xiaogeng Liu](https://md.research.elamur.ai/free/en/authors/xiaogeng-liu/), [Xinyan Wang](https://md.research.elamur.ai/free/en/authors/xinyan-wang/), Yingzi Ma, Yechao Zhang and [Chaowei Xiao](https://md.research.elamur.ai/free/en/authors/chaowei-xiao/) · **Year:** 2026 · **Venue:** arXiv:2605.21606 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2605.21606) · [PDF](https://arxiv.org/pdf/2605.21606) · [DOI 10.48550/arxiv.2605.21606](https://doi.org/10.48550/arxiv.2605.21606) · [arXiv 2605.21606](https://arxiv.org/abs/2605.21606) · [OpenAlex](https://openalex.org/W7162093714)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/liu-2026-teacher-tokens-reliable-position-weighted/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
On-policy self-distillation (OPSD) trains a reasoning model on its own sampled solutions. The teacher is the same model, prompted with the reference answer. Standard OPSD gives every token identical forward-KL supervision. Earlier adaptive variants reweight tokens using teacher entropy or similar local signals. The authors argue that such local ambiguity is uninformative: it can signal either a risky branch or harmless alternative solution routes.

To tell these apart, they build a branch-viability diagnostic. Next-token alternatives proposed under the answer-aware teacher prompt are inserted into a correct student rollout. Generation then continues under the ordinary student prompt. If these continuations fail to reach the right answer, the teacher is judged unreliable at that point. On Qwen3-4B, a token's position in the trace predicts reliability much better than entropy-style scores: early branch points are fragile, while late ones rarely fail.

PW-OPSD therefore scales token losses by a rising sigmoid of normalized position, with a minimum weight. It needs no extra teacher computation. Gains are small but consistent across three model families, which suggests teacher reliability follows the structure of the trajectory.

## Key points
- **Diagnostic result (Qwen3-4B):** the oriented position score separates unreliable from benign branch points with AUROC 0.83 (95% cluster-bootstrap CI [0.66, 0.95]). Local uncertainty scores reach at most 0.57.
- **Main Qwen3-4B results (max generation length 38,912 tokens, three seeds, Avg@12):**
  - AIME 2024 rises from 75.19 to 76.20 (+1.0).
  - AIME 2025 rises from 66.67 to 67.78 (+1.1).
  - MATH-500 is essentially unchanged (95.33 vs. 95.34).
  - HMMT 2025 dips slightly with the default Moderate schedule. The Aggressive schedule instead gains +1.48 Avg@12 and +1.11 Maj@12 on HMMT 2025.
- **Schedule sweep:** four schedules (Mild, Moderate, Sharp, Aggressive) are defined by the parameters (w_min, τ, s). All four beat the OPSD four-benchmark mean of 70.27 by +0.30 to +0.65 pp. Moderate, (0.25, 0.30, 0.10), was chosen in advance from the diagnostic curves.
- **Cross-model transfer (no retuning, four-benchmark mean Avg@12):**
  - DeepSeek-R1-Distill-Llama-8B: 45.56 → 45.91
  - Olmo-3-7B-Think: 70.07 → 70.57
  - Qwen3-4B: 70.27 → 70.66
- **Ablation and code:** a 2×2 factorial ablation indicates that both components are needed for the AIME 2024 gain: position weighting and per-sequence loss averaging. Code is available at github.com/SaFo-Lab/PW-OPSD.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Johns Hopkins University](https://md.research.elamur.ai/free/en/universities/johns-hopkins-university/) — University, Baltimore, United States · 258 works · 108,081 citations
- [Nanyang Technological University](https://md.research.elamur.ai/free/en/universities/nanyang-technological-university/) — University, Singapore, Singapore · 193 works · 27,739 citations
- [University of Wisconsin–Madison](https://md.research.elamur.ai/free/en/universities/university-of-wisconsin-madison/) — University, Madison, United States · 60 works · 49,213 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,948 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 328 works

## Cite

Cite as: Xiaogeng Liu, Xinyan Wang, Yingzi Ma, Yechao Zhang and Chaowei Xiao. 2026. When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning. arXiv:2605.21606.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{liu2026when,
  title = {{When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning}},
  author = {Xiaogeng Liu, Xinyan Wang, Yingzi Ma, Yechao Zhang and Chaowei Xiao},
  year = {2026},
  doi = {10.48550/arxiv.2605.21606},
  eprint = {2605.21606},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2605.21606}
}
```

- [← previous TurkBench: A Benchmark for Evaluating Turkish Large Language Models](https://md.research.elamur.ai/free/en/papers/toraman-2026-turkbench-benchmark-evaluating-turkish-large/)
- [next → A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages](https://md.research.elamur.ai/free/en/papers/zhao-2026-comprehensive-evaluation-multilingual-chain-thought/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/)

## References

41 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://md.research.elamur.ai/free/en/papers/agarwal-2023-policy-distillation-language-models-learning/). arXiv preprint arXiv:2306.13649, 2023. URL https://arxiv.org/abs/2306.13649. Accepted at ICLR 2024. Firs… [↗](https://arxiv.org/abs/2306.13649)
2. Allen Institute for AI. Olmo-3-7B-Think. [https://huggingface.co/allenai/](https://huggingface.co/allenai/) Olmo-3-7B-Think, 2026. Hugging Face model card. [↗](https://huggingface.co/allenai/)
3. Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher R’e, and Azalia Mirhoseini. [Large language monkeys: Scaling inference compute with repeated sampling](https://md.research.elamur.ai/free/en/papers/brown-2024-large-language-monkeys-scaling-inference/). ArXiv, abs/2407.21787, 2024.
4. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé, Jared Kaplan, Harrison Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, G. Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea P… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
5. DeepSeek-AI. DeepSeek-R1-Distill-Llama-8B. [https://huggingface.co/deepseek-ai/](https://huggingface.co/deepseek-ai/) DeepSeek-R1-Distill-Llama-8B, 2025. Hugging Face model card. [↗](https://huggingface.co/deepseek-ai/)
6. Jasper Dekoninck, Nikola Jovanović, Tim Gehrunger, Kári Rögnvalddson, Ivo Petrov, Chenhao Sun, and Martin Vechev. Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs. arXiv preprint arXiv:2605.00674, 2026. URL https://arxiv.org/abs/2605. 00674. — [arXiv:2605.00674](https://arxiv.org/abs/2605.00674) [↗](https://arxiv.org/abs/2605)
7. Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. arXiv preprint arXiv:1506.02142, 2015. URL https://arxiv.org/abs/1506.02142. 12 pages, 6 figures; fixed a mistake with standard error and added a new table with updated results (ma… [↗](https://arxiv.org/abs/1506.02142)
8. Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. [MiniLLM: On-Policy Distillation of Large Language Models](https://md.research.elamur.ai/free/en/papers/gu-2023-minillm-policy-distillation-large-language/). arXiv preprint arXiv:2306.08543, 2023. URL https://arxiv.org/abs/ 2306.08543. Published as a conference paper in ICLR 2024. [↗](https://arxiv.org/abs/)
9. Arnav Gudibande, Eric Wallace, Charles Burton Snell, Xinyang Geng, Hao Liu, P. Abbeel, S. Levine, and Dawn Song. [The false promise of imitating proprietary llms](https://md.research.elamur.ai/free/en/papers/gudibande-2023-false-promise-imitating-proprietary-llms/). ArXiv, abs/2305.15717, 2023.
10. E. Guha, Ryan Marten, Sedrick Scott Keh, Negin Raoof, G. Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean-Pierre Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Ben Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang, Shre… — [OpenThoughts: Data Recipes for Reasoning Models](https://md.research.elamur.ai/free/en/papers/guha-2025-openthoughts-data-recipes-reasoning-models/)
11. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Che… — [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
12. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, D. Song, and J. Steinhardt. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). ArXiv, abs/2103.03874, 2021.
13. Geoffrey E. Hinton, O. Vinyals, and J. Dean. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). ArXiv, abs/1503.02531, 2015.
14. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/), 2021. URL https://arxiv.org/abs/2106.09685. [↗](https://arxiv.org/abs/2106.09685)
15. HuggingFaceH4. MATH-500. [https://huggingface.co/datasets/HuggingFaceH4/](https://huggingface.co/datasets/HuggingFaceH4/) MATH-500, 2024. Hugging Face dataset; MATH-500 subset from PRM800K/MATH provenance. [↗](https://huggingface.co/datasets/HuggingFaceH4/)
16. HuggingFaceH4. AIME 2024. [https://huggingface.co/datasets/HuggingFaceH4/aime_](https://huggingface.co/datasets/HuggingFaceH4/aime_) 2024, 2025. Hugging Face dataset for the 2024 competition; derived from AI-MO/aimo-validation-aime. [↗](https://huggingface.co/datasets/HuggingFaceH4/aime_)
17. Woogyeol Jin, Taywon Min, Yongjin Yang, Swanand Ravindra Kadhe, Yi Zhou, Dennis Wei, Nathalie Baracaldo, and Kimin Lee. [Entropy-Aware On-Policy Distillation of Language Models](https://md.research.elamur.ai/free/en/papers/jin-2026-entropy-aware-policy-distillation-language/). arXiv preprint arXiv:2603.07079, 2026. URL https://arxiv.org/abs/2603.07079. 16 pages, 11 figures, preprint. [↗](https://arxiv.org/abs/2603.07079)
18. Seongryong Jung, Suwan Yoon, DongGeon Kim, and Hwanhee Lee. [ToDi: Token-wise Distillation via Fine-Grained Divergence Control](https://md.research.elamur.ai/free/en/papers/jung-2025-todi-token-wise-distillation-fine/). arXiv preprint arXiv:2505.16297, 2025. URL https://arxiv.org/abs/2505.16297. [↗](https://arxiv.org/abs/2505.16297)
19. Alex Kendall and Y. Gal. [What uncertainties do we need in bayesian deep learning for computer vision?](https://md.research.elamur.ai/free/en/papers/kendall-2017-uncertainties-need-bayesian-deep-learning/) ArXiv, abs/1703.04977, 2017.
20. Yoon Kim and Alexander M. Rush. [Sequence-level knowledge distillation](https://md.research.elamur.ai/free/en/papers/kim-2016-sequence-level-knowledge-distillation/). ArXiv, abs/1606.07947, 2016.
21. Jongwoo Ko, Sungnyun Kim, Tianyi Chen, and SeYoung Yun. [Distillm: Towards streamlined distillation for large language models](https://md.research.elamur.ai/free/en/papers/ko-2024-distillm-towards-streamlined-distillation-large/). ArXiv, abs/2402.03898, 2024.
22. Jongwoo Ko, Sara Abdali, Young Jin Kim, Tianyi Chen, and Pashmina Cameron. [Scaling Reasoning Efficiently via Relaxed On-Policy Distillation](https://md.research.elamur.ai/free/en/papers/ko-2026-scaling-reasoning-efficiently-relaxed-policy/). arXiv preprint arXiv:2603.11137, 2026. URL https://arxiv.org/abs/2603.11137. [↗](https://arxiv.org/abs/2603.11137)
23. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). arXiv preprint arXiv:2309.06180, 2023. URL https://arxiv.org/abs/2309.06180. [↗](https://arxiv.org/abs/2309.06180)
24. Hynek Kydlíček. Math-Verify: Math verification library. [https://github.com/huggingface/](https://github.com/huggingface/) math-verify, 2025. Version 0.6.1. [↗](https://github.com/huggingface/)
25. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step, 2023. [URL https://arxiv.org/abs/2305.20050](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). [↗](https://arxiv.org/abs/2305.20050)
26. Andrey Malinin and Mark Gales. Predictive Uncertainty Estimation via Prior Networks. arXiv preprint arXiv:1802.10501, 2018. URL https://arxiv.org/abs/1802.10501. — [arXiv:1802.10501](https://arxiv.org/abs/1802.10501) [↗](https://arxiv.org/abs/1802.10501)
27. Aaron Meurer, Christopher P Smith, Mateusz Paprocki, Ondřej Čertík, Sergey B Kirpichev, Matthew Rocklin, AMiT Kumar, Sergiu Ivanov, Jason K Moore, Sartaj Singh, et al. Sympy: symbolic computing in python. PeerJ Computer Science, 3:e103, 2017. [find](https://scholar.google.com/scholar?q=Aaron+Meurer%2C+Christopher+P+Smith%2C+Mateusz+Paprocki%2C+Ondr%CC%8Cej+C%CC%8Cert%C3%ADk%2C+Sergey+B+Kirpichev%2C+Matthew+Rocklin%2C+AMiT+Kumar%2C+Sergiu+Ivanov%2C+Jason+K+Moore%2C+Sartaj+Singh%2C+et+al%2E+Sympy%3A+symbolic+computing+in+p)
28. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, S. Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, M. Simens, Amanda Askell, Peter Welinder, P. Christiano, Jan Leike, and Ryan J. Lowe. Training language… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
29. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, N. Gimelshein, L. Antiga, Alban Desmaison, Andreas Köpf, E. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chin… — [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/)
30. Qwen Team. Qwen3-4B. [https://huggingface.co/Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), 2025. Hugging Face model card. [↗](https://huggingface.co/Qwen/Qwen3-4B)
31. Stephane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. arXiv preprint arXiv:1011.0686, 2010. URL https://arxiv.org/abs/1011.0686. Appearing in the 14th International Conference on Artificial Intelligence and… [↗](https://arxiv.org/abs/1011.0686)
32. M. Sensoy, M. Kandemir, and Lance M. Kaplan. Evidential deep learning to quantify classification uncertainty. ArXiv, abs/1806.01768, 2018. siyanzhao. OpenThoughts-Math-30k OPSD. [https://huggingface.co/datasets/siyanzhao/](https://huggingface.co/datasets/siyanzhao/) Openthoughts_math_30k_opsd, 2026. Hugging Face dataset; train split with 29,434… — [arXiv:1806.01768](https://arxiv.org/abs/1806.01768) [↗](https://huggingface.co/datasets/siyanzhao/)
33. Mingyang Song and Mao Zheng. [A Survey of On-Policy Distillation for Large Language Models](https://md.research.elamur.ai/free/en/papers/song-2026-survey-policy-distillation-large-language/). arXiv preprint arXiv:2604.00626, 2026. URL https://arxiv.org/abs/2604.00626. [↗](https://arxiv.org/abs/2604.00626)
34. Alex Stein, Furong Huang, and Tom Goldstein. Gates: Self-distillation under privileged context with consensus gating. ArXiv, abs/2602.20574, 2026. — [arXiv:2602.20574](https://arxiv.org/abs/2602.20574) [find](https://scholar.google.com/scholar?q=Alex+Stein%2C+Furong+Huang%2C+and+Tom+Goldstein%2E+Gates%3A+Self%2Ddistillation+under+privileged+context+with+consensus+gating%2E+ArXiv%2C+abs%2F2602%2E20574%2C+2026%2E+%E2%80%94+%5BarXiv%3A2602%2E20574%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2602%2E20574)
35. Xuezhi Wang, Jason Wei, D. Schuurmans, Quoc Le, Ed H. Chi, and Denny Zhou. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). ArXiv, abs/2203.11171, 2022.
36. Yuqiao Wen, Zichao Li, Wenyu Du, and Lili Mou. [f-divergence minimization for sequence-level knowledge distillation](https://md.research.elamur.ai/free/en/papers/wen-2023-f-divergence-minimization-sequence-level/). ArXiv, abs/2307.15190, 2023.
37. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoe… — [Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
38. Taiqiang Wu, Chaofan Tao, Jiahao Wang, Runming Yang, Zhe Zhao, and Ngai Wong. [Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models](https://md.research.elamur.ai/free/en/papers/wu-2025-rethinking-kullback-leibler-divergence-knowledge/). arXiv preprint arXiv:2404.02657, 2024. URL https://arxiv.org/abs/2404.02657. [↗](https://arxiv.org/abs/2404.02657)
39. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jin… — [Qwen3 Technical Report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/)
40. AIME 2025. [https://huggingface.co/datasets/yentinglin/aime_2025](https://huggingface.co/datasets/yentinglin/aime_2025), 2025. Hugging Face dataset. [↗](https://huggingface.co/datasets/yentinglin/aime_2025)
41. Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. [Self-distilled reasoner: On-policy self-distillation for large language models](https://md.research.elamur.ai/free/en/papers/zhao-2026-self-distilled-reasoner-policy-self/). ArXiv, abs/2601.18734, 2026.

*Updated 3 November 2026*