---
title: "Student Guides Teacher — paper by Wang et al. (2026)"
description: "A weaker Student model proposes short candidate continuations. 51 references. Among them: 0."
authors: ["Dayu Wang", "Jiaye Yang", "Weikang Li", "Jiahui Liang", "Yang Li", "Deguo Xia", "Jizhou Huang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.761/"
doi: "10.18653/v1/2026.acl-long.761"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-student-guides-teacher-weak-strong/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-student-guides-teacher-weak-strong/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/51/) / Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration

# Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration

**Authors:** Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, [Yang Li](https://md.research.elamur.ai/free/en/authors/yang-li/), Deguo Xia and [Jizhou Huang](https://md.research.elamur.ai/free/en/authors/jizhou-huang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.761/) · [PDF](https://aclanthology.org/2026.acl-long.761.pdf) · [DOI 10.18653/v1/2026.acl-long.761](https://doi.org/10.18653/v1/2026.acl-long.761) · [OpenAlex](https://openalex.org/W7166869090)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses "Reasoning Collapse" in LLMs on hard math problems. In this failure mode, repeated sampling yields reworded versions of the same flawed argument instead of genuinely new approaches. The authors link this to geometry: the effective rank of the local hidden-state covariance shrinks as long, incorrect chains unfold, which confines the model to a low-rank "Bias Manifold."

Their remedy, Spectral Orthogonal Exploration (SOE), works at inference time in three steps:
- It takes the Teacher's wrong greedy traces and cuts them at reasoning milestones.
- At each cut, it estimates the Teacher's dominant subspace with Monte Carlo look-ahead and Micro-SVD.
- A weaker Student model proposes short candidate continuations. The one whose hidden representation lies furthest outside that subspace is inserted, and the Teacher then resumes sampling.

The Student is useful because its representations differ from the Teacher's, not because it is more accurate. On math benchmarks, SOE substantially raises accuracy and sampling efficiency over baselines. Early tests on logic and code show the same direction. The work suggests that targeted interventions in latent space can outperform simply adding sampling noise.

## Key points
- The pilot study ran Qwen3-4B-Instruct-2507 with temperature T = 0.7 across four benchmarks (Omni-Math, Olympiad Bench, AIME 25, and AIME 24). It examined wrong chains longer than 4,000 tokens, with a sliding window of k = 64. Effective Rank (Roy & Vetterli) decays steadily, despite a full dimension of d = 2560.
- Student probes are fixed at L = 8 tokens. The selected probe is the one with the largest orthogonal projection residual relative to the Teacher's local bias manifold, a step the paper calls Orthogonal Latent Stitching.
- Averaged over baselines, SOE reports gains of +62.4% in accuracy and +113.7% in sampling efficiency on math benchmarks.
- Preliminary results outside math:
  - ZebraLogic: 56.23% → 58.72%
  - HumanEvalPlus: 10.00% → 16.67%
- Limitations:
  - Extra wall-clock latency from look-ahead and scoring.
  - Requires access to hidden states, so it only works with open-weights models.
  - Evaluation is mostly limited to mathematics.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li, Deguo Xia and Jizhou Huang. 2026. Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026student,
  title = {{Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration}},
  author = {Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li, Deguo Xia and Jizhou Huang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.761},
  url = {https://doi.org/10.18653/v1/2026.acl-long.761}
}
```

- [← previous STRUDEL: Unrolling a Benchmark for Evaluating Vision-Language Models on Structured Diagram Understanding across Domains](https://md.research.elamur.ai/free/en/papers/steinigen-2026-strudel-unrolling-benchmark-evaluating-vision/)
- [next → Students’ awareness of the importance of using proper English grammar in the job market: a critical examination of perceptions and practices](https://md.research.elamur.ai/free/en/papers/allehyani-2026-students-awareness-importance-using-proper/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/51/)

## References

51 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alexandra Bazarova, Aleksandr Yugay, Andrey Shulga, Alina Ermilova, Andrei Volodichev, Konstantin Polev, Julia Belikova, Rauf Parchiev, Dmitry Simakov, Maxim Savchenko, Andrey Savchenko, Serguei Barannikov, and Alexey Zaytsev. 2025. Hallucination detection in llms with topological divergence on atte… [find](https://scholar.google.com/scholar?q=Alexandra+Bazarova%2C+Aleksandr+Yugay%2C+Andrey+Shulga%2C+Alina+Ermilova%2C+Andrei+Volodichev%2C+Konstantin+Polev%2C+Julia+Belikova%2C+Rauf+Parchiev%2C+Dmitry+Simakov%2C+Maxim+Savchenko%2C+Andrey+Savchenko%2C+Serguei+Baran)
2. Sébastien Bubeck, Christian Coester, Ronen Eldan, Timothy Gowers, Yin Tat Lee, Alexandru Lupsasca, Mehtaab Sawhney, Robert Scherrer, Mark Sellke, Brian K. Spears, Derya Unutmaz, Kevin Weil, Steven Yin, and Nikita Zhivotovskiy. 2025. Early science acceleration experiments with gpt-5. Preprint, arXiv:… — [arXiv:2511.16072](https://arxiv.org/abs/2511.16072) [find](https://scholar.google.com/scholar?q=S%C3%A9bastien+Bubeck%2C+Christian+Coester%2C+Ronen+Eldan%2C+Timothy+Gowers%2C+Yin+Tat+Lee%2C+Alexandru+Lupsasca%2C+Mehtaab+Sawhney%2C+Robert+Scherrer%2C+Mark+Sellke%2C+Brian+K%2E+Spears%2C+Derya+Unutmaz%2C+Kevin+Weil%2C+Steven+Yin)
3. Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, and 1 others. 2023. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307.15217. [find](https://scholar.google.com/scholar?q=Stephen+Casper%2C+Xander+Davies%2C+Claudia+Shi%2C+Thomas+Krendl+Gilbert%2C+J%C3%A9r%C3%A9my+Scheurer%2C+Javier+Rando%2C+Rachel+Freedman%2C+Tomasz+Korbak%2C+David+Lindner%2C+Pedro+Freire%2C+and+1+others%2E+2023%2E+Open+problems+and+fun)
4. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025a. [Deepseek-r1: Incentivizing reasoning capability in l…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
5. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, and 181 others. 2025b. Deepseek-v3 technical report. Preprint, arXiv:… [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2C+Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+Damai+Dai%2C+Daya+Guo%2C+Dejian+Yang%2C+Deli+Chen%2C+Dongjie+Ji%2C+Erhan)
6. DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenhao Xu, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, and 245 others. 2025c. [Deepseek-v3.2: Pushing the frontier of ope…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-v3-2-pushing/)
7. Elvis Dohmatob, Yunzhen Feng, and Julia Kempe. 2024. Model collapse demystified: The case of regression. Preprint, arXiv:2402.07712. — [arXiv:2402.07712](https://arxiv.org/abs/2402.07712) [find](https://scholar.google.com/scholar?q=Elvis+Dohmatob%2C+Yunzhen+Feng%2C+and+Julia+Kempe%2E+2024%2E+Model+collapse+demystified%3A+The+case+of+regression%2E+Preprint%2C+arXiv%3A2402%2E07712%2E+%E2%80%94+%5BarXiv%3A2402%2E07712%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E07712%29)
8. Yihe Dong, Jean-Baptiste Cordonnier, and Andreas Loukas. 2021. Attention is not all you need: Pure attention loses rank doubly exponentially with depth. In International conference on machine learning, pages 2793–2803. PMLR. [find](https://scholar.google.com/scholar?q=Yihe+Dong%2C+Jean%2DBaptiste+Cordonnier%2C+and+Andreas+Loukas%2E+2021%2E+Attention+is+not+all+you+need%3A+Pure+attention+loses+rank+doubly+exponentially+with+depth%2E+In+International+conference+on+machine+learning)
9. Kawin Ethayarajh. 2019. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. arXiv preprint arXiv:1909.00512. [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2E+2019%2E+How+contextual+are+contextualized+word+representations%3F+comparing+the+geometry+of+bert%2C+elmo%2C+and+gpt%2D2+embeddings%2E+arXiv+preprint+arXiv%3A1909%2E00512%2E)
10. Ziqing Fan, Siyuan Du, Shengchao Hu, Pingjie Wang, Li Shen, Ya Zhang, Dacheng Tao, and Yanfeng Wang. 2025. Combatting dimensional collapse in llm pre-training data via diversified file selection. Preprint, arXiv:2504.20644. — [arXiv:2504.20644](https://arxiv.org/abs/2504.20644) [find](https://scholar.google.com/scholar?q=Ziqing+Fan%2C+Siyuan+Du%2C+Shengchao+Hu%2C+Pingjie+Wang%2C+Li+Shen%2C+Ya+Zhang%2C+Dacheng+Tao%2C+and+Yanfeng+Wang%2E+2025%2E+Combatting+dimensional+collapse+in+llm+pre%2Dtraining+data+via+diversified+file+selection%2E+Prep)
11. Tyler Farghly, Peter Potaptchik, Samuel Howard, George Deligiannidis, and Jakiw Pidstrigach. 2025. Diffusion models and the manifold hypothesis: Log-domain smoothing is geometry adaptive. Preprint, arXiv:2510.02305. — [arXiv:2510.02305](https://arxiv.org/abs/2510.02305) [find](https://scholar.google.com/scholar?q=Tyler+Farghly%2C+Peter+Potaptchik%2C+Samuel+Howard%2C+George+Deligiannidis%2C+and+Jakiw+Pidstrigach%2E+2025%2E+Diffusion+models+and+the+manifold+hypothesis%3A+Log%2Ddomain+smoothing+is+geometry+adaptive%2E+Preprint%2C+ar)
12. Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. 2024. [Omni-math: A universal olympiad level ma…](https://md.research.elamur.ai/free/en/papers/gao-2024-omni-math-universal-olympiad-level/)
13. Jun Gao, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tie-Yan Liu. 2019. [Representation degeneration problem in training natural language generation models](https://md.research.elamur.ai/free/en/papers/gao-2019-representation-degeneration-problem-training-natural/). arXiv preprint arXiv:1907.12009.
14. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495. [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Roei+Schuster%2C+Jonathan+Berant%2C+and+Omer+Levy%2E+2021%2E+Transformer+feed%2Dforward+layers+are+key%2Dvalue+memories%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
15. Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. [The false promise of imitating proprietary llms](https://md.research.elamur.ai/free/en/papers/gudibande-2023-false-promise-imitating-proprietary-llms/). arXiv preprint arXiv:2305.15717.
16. Dadi Guo, Jiayu Liu, Zhiyuan Fan, Zhitao He, Haoran Li, Yuxin Li, Yumeng Wang, and Yi R. Fung. 2025. [Mathematical proof as a litmus test: Revealing failure modes of advanced large reasoning models](https://md.research.elamur.ai/free/en/papers/guo-2026-mathematical-proof-as-litmus-test/). Preprint, arXiv:2506.17114.
17. Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, and Han Xiao. 2023. [Jina embeddings 2: 8192-token general-purpose text embeddings for long documents](https://md.research.elamur.ai/free/en/papers/gunther-2023-jina-embeddings-2-8192-token/). Preprint, ar…
18. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, and 1 others. 2024. [Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). In Proceedings of the 62nd Annual Meeting…
19. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874.
20. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, E… — [Training Compute-Optimal Large Language Models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/)
21. Gouki Minegishi, Hiroki Furuta, Takeshi Kojima, Yusuke Iwasawa, and Yutaka Matsuo. 2025. Topology of reasoning: Understanding large reasoning models through reasoning graph properties. Preprint, arXiv:2506.05744. — [arXiv:2506.05744](https://arxiv.org/abs/2506.05744) [find](https://scholar.google.com/scholar?q=Gouki+Minegishi%2C+Hiroki+Furuta%2C+Takeshi+Kojima%2C+Yusuke+Iwasawa%2C+and+Yutaka+Matsuo%2E+2025%2E+Topology+of+reasoning%3A+Understanding+large+reasoning+models+through+reasoning+graph+properties%2E+Preprint%2C+arXiv)
22. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). Preprint, arXiv:1904.09751.
23. Bhrij Patel, Souradip Chakraborty, Wesley A. Suttle, Mengdi Wang, Amrit Singh Bedi, and Dinesh Manocha. 2024. Aime: Ai system optimization via multiple llm evaluators. Preprint, arXiv:2410.03131. — [arXiv:2410.03131](https://arxiv.org/abs/2410.03131) [find](https://scholar.google.com/scholar?q=Bhrij+Patel%2C+Souradip+Chakraborty%2C+Wesley+A%2E+Suttle%2C+Mengdi+Wang%2C+Amrit+Singh+Bedi%2C+and+Dinesh+Manocha%2E+2024%2E+Aime%3A+Ai+system+optimization+via+multiple+llm+evaluators%2E+Preprint%2C+arXiv%3A2410%2E03131%2E+%E2%80%94+%5Ba)
24. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
25. Federico Arangath Joseph, Jerome Sieber, Melanie N. Zeilinger, and Carmen Amo Alonso. 2025. Lambda-skip connections: the architectural component that prevents rank collapse. Preprint, arXiv:2410.10609. — [arXiv:2410.10609](https://arxiv.org/abs/2410.10609) [find](https://scholar.google.com/scholar?q=Federico+Arangath+Joseph%2C+Jerome+Sieber%2C+Melanie+N%2E+Zeilinger%2C+and+Carmen+Amo+Alonso%2E+2025%2E+Lambda%2Dskip+connections%3A+the+architectural+component+that+prevents+rank+collapse%2E+Preprint%2C+arXiv%3A2410%2E10609)
26. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361.
27. Thomas Laurent, James H. von Brecht, and Xavier Bresson. 2023. Feature collapse. Preprint, arXiv:2305.16162. — [arXiv:2305.16162](https://arxiv.org/abs/2305.16162) [find](https://scholar.google.com/scholar?q=Thomas+Laurent%2C+James+H%2E+von+Brecht%2C+and+Xavier+Bresson%2E+2023%2E+Feature+collapse%2E+Preprint%2C+arXiv%3A2305%2E16162%2E+%E2%80%94+%5BarXiv%3A2305%2E16162%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2305%2E16162%29)
28. Melody Zixuan Li, Kumar Krishna Agrawal, Arna Ghosh, Komal Kumar Teru, Adam Santoro, Guillaume Lajoie, and Blake A. Richards. 2025. Tracing the representation geometry of language models from pretraining to post-training. Preprint, arXiv:2509.23024. — [arXiv:2509.23024](https://arxiv.org/abs/2509.23024) [find](https://scholar.google.com/scholar?q=Melody+Zixuan+Li%2C+Kumar+Krishna+Agrawal%2C+Arna+Ghosh%2C+Komal+Kumar+Teru%2C+Adam+Santoro%2C+Guillaume+Lajoie%2C+and+Blake+A%2E+Richards%2E+2025%2E+Tracing+the+representation+geometry+of+language+models+from+pretrain)
29. Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi. 2025. [Zebralogic: On the scaling limits of llms for logical reasoning](https://md.research.elamur.ai/free/en/papers/lin-2025-zebralogic-scaling-limits-llms-logical/). arXiv preprint arXiv:2502.01100.
30. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. [Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/). Advances in neural information processing systems, 36:21558–21572.
31. Pan Lu, Liang Qiu, Wenhao Yu, Sean Welleck, and Kai-Wei Chang. 2023. A survey of deep learning for mathematical reasoning. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pages 14605– 14631. [find](https://scholar.google.com/scholar?q=Pan+Lu%2C+Liang+Qiu%2C+Wenhao+Yu%2C+Sean+Welleck%2C+and+Kai%2DWei+Chang%2E+2023%2E+A+survey+of+deep+learning+for+mathematical+reasoning%2E+In+Proceedings+of+the+61st+annual+meeting+of+the+association+for+computationa)
32. Seongheon Park, Xuefeng Du, Min-Hsuan Yeh, Haobo Wang, and Yixuan Li. 2025. Steer llm latents for hallucination detection. Preprint, arXiv:2503.01917. — [arXiv:2503.01917](https://arxiv.org/abs/2503.01917) [find](https://scholar.google.com/scholar?q=Seongheon+Park%2C+Xuefeng+Du%2C+Min%2DHsuan+Yeh%2C+Haobo+Wang%2C+and+Yixuan+Li%2E+2025%2E+Steer+llm+latents+for+hallucination+detection%2E+Preprint%2C+arXiv%3A2503%2E01917%2E+%E2%80%94+%5BarXiv%3A2503%2E01917%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2503%2E0)
33. Ivo Petrov, Jasper Dekoninck, Lyuben Baltadzhiev, Maria Drencheva, Kristian Minchev, Mislav Balunović, Nikola Jovanović, and Martin Vechev. 2025. Proof or bluff? evaluating llms on 2025 usa math olympiad. Preprint, arXiv:2503.21934. [find](https://scholar.google.com/scholar?q=Ivo+Petrov%2C+Jasper+Dekoninck%2C+Lyuben+Baltadzhiev%2C+Maria+Drencheva%2C+Kristian+Minchev%2C+Mislav+Balunovic%CC%81%2C+Nikola+Jovanovic%CC%81%2C+and+Martin+Vechev%2E+2025%2E+Proof+or+bluff%3F+evaluating+llms+on+2025+usa+math+oly)
34. Edward Phillips, Sean Wu, Soheila Molaei, Danielle Belgrave, Anshul Thakur, and David Clifton. 2025. Geometric uncertainty for detecting and correcting hallucinations in llms. Preprint, arXiv:2509.13813. — [arXiv:2509.13813](https://arxiv.org/abs/2509.13813) [find](https://scholar.google.com/scholar?q=Edward+Phillips%2C+Sean+Wu%2C+Soheila+Molaei%2C+Danielle+Belgrave%2C+Anshul+Thakur%2C+and+David+Clifton%2E+2025%2E+Geometric+uncertainty+for+detecting+and+correcting+hallucinations+in+llms%2E+Preprint%2C+arXiv%3A2509%2E138)
35. Olivier Roy and Martin Vetterli. 2007. The effective rank: A measure of effective dimensionality. In 2007 15th European signal processing conference, pages 606–610. IEEE. [find](https://scholar.google.com/scholar?q=Olivier+Roy+and+Martin+Vetterli%2E+2007%2E+The+effective+rank%3A+A+measure+of+effective+dimensionality%2E+In+2007+15th+European+signal+processing+conference%2C+pages+606%E2%80%93610%2E+IEEE%2E)
36. Mohamed El Amine Seddik, Suei-Wen Chen, Soufiane Hayou, Pierre Youssef, and Merouane Debbah. 2024. How bad is training on synthetic data? a statistical analysis of language model collapse. Preprint, arXiv:2404.05090. — [arXiv:2404.05090](https://arxiv.org/abs/2404.05090) [find](https://scholar.google.com/scholar?q=Mohamed+El+Amine+Seddik%2C+Suei%2DWen+Chen%2C+Soufiane+Hayou%2C+Pierre+Youssef%2C+and+Merouane+Debbah%2E+2024%2E+How+bad+is+training+on+synthetic+data%3F+a+statistical+analysis+of+language+model+collapse%2E+Preprint%2C+a)
37. Weijie Shi, Yue Cui, Yaguang Wu, Jingzhi Fang, Shibo Zhang, Mengze Li, Sirui Han, Jia Zhu, Jiajie Xu, and Xiaofang Zhou. 2025. Semantic-guided diverse decoding for large language model. Preprint, arXiv:2506.23601. — [arXiv:2506.23601](https://arxiv.org/abs/2506.23601) [find](https://scholar.google.com/scholar?q=Weijie+Shi%2C+Yue+Cui%2C+Yaguang+Wu%2C+Jingzhi+Fang%2C+Shibo+Zhang%2C+Mengze+Li%2C+Sirui+Han%2C+Jia+Zhu%2C+Jiajie+Xu%2C+and+Xiaofang+Zhou%2E+2025%2E+Semantic%2Dguided+diverse+decoding+for+large+language+model%2E+Preprint%2C+arXi)
38. Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar. 2025. [The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity](https://md.research.elamur.ai/free/en/papers/shojaee-2025-illusion-thinking-understanding-strengths-limitations/). Preprint, arXiv:2506.06941.
39. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. 2023. [The curse of recursion: Training on generated data makes models forget](https://md.research.elamur.ai/free/en/papers/shumailov-2023-curse-recursion-training-generated-data/). arXiv preprint arXiv:2305.17493.
40. Alexander Shypula, Shuo Li, Botong Zhang, Vishakh Padmakumar, Kayo Yin, and Osbert Bastani. 2025. Evaluating the diversity and quality of llm generated content. Preprint, arXiv:2504.12522. — [arXiv:2504.12522](https://arxiv.org/abs/2504.12522) [find](https://scholar.google.com/scholar?q=Alexander+Shypula%2C+Shuo+Li%2C+Botong+Zhang%2C+Vishakh+Padmakumar%2C+Kayo+Yin%2C+and+Osbert+Bastani%2E+2025%2E+Evaluating+the+diversity+and+quality+of+llm+generated+content%2E+Preprint%2C+arXiv%3A2504%2E12522%2E+%E2%80%94+%5BarXiv%3A25)
41. Gemma Team. 2025a. Gemma 3. [find](https://scholar.google.com/scholar?q=Gemma+Team%2E+2025a%2E+Gemma+3%2E)
42. Qwen Team. 2025b. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
43. Qwen Team. 2025c. Qwq-32b: Embracing the power of reinforcement learning. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2025c%2E+Qwq%2D32b%3A+Embracing+the+power+of+reinforcement+learning%2E)
44. A.1 Input Prompt System Prompt: Please reason step by step, and put your final answer within \\boxed{}. User Problem: In triangleABC points D and E lie on overlineAB so that AD \\< AE \\< AB, while points F and G lie on overlineAC so that AF \\< AG \\< AC. Suppose AD = 4, DE = 16, EB = 8, AF = 13, F G = 52,… [find](https://scholar.google.com/scholar?q=A%2E1+Input+Prompt+System+Prompt%3A+Please+reason+step+by+step%2C+and+put+your+final+answer+within+%5C%5Cboxed%7B%7D%2E+User+Problem%3A+In+triangleABC+points+D+and+E+lie+on+overlineAB+so+that+AD+%5C%3C+AE+%5C%3C+AB%2C+while+poin)
45. Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma. 2020. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768. [find](https://scholar.google.com/scholar?q=Sinong+Wang%2C+Belinda+Z+Li%2C+Madian+Khabsa%2C+Han+Fang%2C+and+Hao+Ma%2E+2020%2E+Linformer%3A+Self%2Dattention+with+linear+complexity%2E+arXiv+preprint+arXiv%3A2006%2E04768%2E)
46. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/). Preprint, arXiv:22…
47. Khurram Yamin, Shantanu Gupta, Gaurav R. Ghosal, Zachary C. Lipton, and Bryan Wilder. 2025. Failure modes of llms for causal reasoning on narratives. Preprint, arXiv:2410.23884. [find](https://scholar.google.com/scholar?q=Khurram+Yamin%2C+Shantanu+Gupta%2C+Gaurav+R%2E+Ghosal%2C+Zachary+C%2E+Lipton%2C+and+Bryan+Wilder%2E+2025%2E+Failure+modes+of+llms+for+causal+reasoning+on+narratives%2E+Preprint%2C+arXiv%3A2410%2E23884%2E)
48. Problem Input A.2 Baseline: Reasoning Collapse (Greedy Decoding) The Teacher model (π θ ) initially fails to solve the problem. Following is the generation. We are given: Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W Cohen. 2017. Breaking the softmax bottleneck: A high-rank rnn langua… [find](https://scholar.google.com/scholar?q=Problem+Input+A%2E2+Baseline%3A+Reasoning+Collapse+%28Greedy+Decoding%29+The+Teacher+model+%28%CF%80+%CE%B8+%29+initially+fails+to+solve+the+problem%2E+Following+is+the+generation%2E+We+are+given%3A+Zhilin+Yang%2C+Zihang+Dai%2C+Rusl)
49. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, and 1 others. 2025. [Siren’s song in the ai ocean: A survey on hallucination in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/). Computational Linguistics, pages 1–46.
50. Chengshuai Zhao, Zhen Tan, Pingchuan Ma, Dawei Li, Bohan Jiang, Yancheng Wang, Yingzhen Yang, and Huan Liu. 2025. [Is chain-of-thought reasoning of llms a mirage? a data distribution lens](https://md.research.elamur.ai/free/en/papers/zhao-2026-is-chain-thought-reasoning-llms/). Preprint, arXiv:2508.01191.
51. Points D, E on AB such that AD \\< AE \\< AB, • Points F, G on AC such that AF \\< AG \\< AC, • Lengths: – AD = 4 – DE = 16 – EB = 8 – AF = 13 – F G = 52 – GC = 26 • M is the reflection of D through F , • N is the reflection of G through E, [find](https://scholar.google.com/scholar?q=Points+D%2C+E+on+AB+such+that+AD+%5C%3C+AE+%5C%3C+AB%2C+%E2%80%A2+Points+F%2C+G+on+AC+such+that+AF+%5C%3C+AG+%5C%3C+AC%2C+%E2%80%A2+Lengths%3A+%E2%80%93+AD+%3D+4+%E2%80%93+DE+%3D+16+%E2%80%93+EB+%3D+8+%E2%80%93+AF+%3D+13+%E2%80%93+F+G+%3D+52+%E2%80%93+GC+%3D+26+%E2%80%A2+M+is+the+reflection+of+D+through+F+%2C+%E2%80%A2)

*Updated 2 November 2026*