---
title: "KisMATH — paper by Saha et al. (2026) — summary and ideas"
description: "The resource matters because it replaces random perturbations with targeted, structure-aware interventions, and it is openly… 46 references."
authors: ["Soumadeep Saha", "Akshay Chaturvedi", "Saptarshi Saha", "Utpal Garain", "Nicholas Asher"]
year: 2026
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2026.tacl-1.59/"
doi: "10.1162/tacl.a.729"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/saha-2026-kismath-do-llms-have-knowledge/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/saha-2026-kismath-do-llms-have-knowledge/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) / KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?

# KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?

**Authors:** Soumadeep Saha, Akshay Chaturvedi, Saptarshi Saha, Utpal Garain and [Nicholas Asher](https://md.research.elamur.ai/free/en/authors/nicholas-asher/) · **Year:** 2026 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2026](https://md.research.elamur.ai/free/en/venues/tacl/2026/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.tacl-1.59/) · [PDF](https://aclanthology.org/2026.tacl-1.59.pdf) · [DOI 10.1162/tacl.a.729](https://doi.org/10.1162/tacl.a.729) · [OpenAlex](https://openalex.org/W4414946172)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Why chain-of-thought (CoT) prompting helps LLMs on math is still disputed. One view holds that models genuinely split problems into sub-steps. Another holds that CoT only supports a kind of approximate recall from stored knowledge. The authors test this by making the hidden dependency structure of a solution explicit. They had OpenAI o3 solve problems from GSM8K, MATH500 and AIME and kept only the correctly answered ones. They then parsed every mathematical expression with SymPy and linked expressions whenever one reuses another, tracing backwards from the answer to the question. Each problem thus gets a directed acyclic Causal CoT Graph (CCGraph), and these graphs form the KisMATH dataset. Tests on 15 open-weight LLMs show two things. Intermediate expressions causally mediate the final answer, and models give graph-aligned reasoning paths more probability than random ones, which suggests they internally rely on similar structures. The resource matters because it replaces random perturbations with targeted, structure-aware interventions, and it is openly released.

## Key points
- KisMATH contains 1671 problems (983 GSM8K, 384 MATH500, 304 AIME). Each has an o3 solution and a CCGraph with 9–40 reasoning nodes and 6–10 hops from question to answer. Earlier manually annotated collections covered only 10–30 traces.
- Two expressions are linked if their strings are identical or their SymPy parse trees share a node. Nodes without a path to the question are pruned. About 10% of graphs collapsed to the answer node alone and were repaired by hand. Attention suppression was used to validate the graphs.
- Across 15 models (1B–70B parameters), removing reasoning nodes clearly raised answer entropy. CCGraph-aligned paths consistently received more probability mass than random paths.
- Model behaviour falls into two regimes. In an "exponential" regime, nearly all transitions are high-probability. In a "bell-shaped" regime, a few low-probability "fork" tokens enable more exploration.
- With pass@k, DeepSeek R1 32B (bell-shaped, distilled) outperforms Qwen3 32B (RLVR-trained): 71.6% vs 68.6% at k=1 and 90% vs 87% at k=10. The authors take this as evidence that RLVR may reduce exploration.
- The method does not readily extend to geometry, abstract algebra or commonsense reasoning. Data, code and outputs are released under the MIT license at espressovi.github.io/KisMATH.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Institut de Recherche en Informatique de Toulouse](https://md.research.elamur.ai/free/en/labs/institut-de-recherche-en-informatique-de-toulouse/) — Research institute, Toulouse, France · 35 works · 1,872 citations
- [Linagora](https://md.research.elamur.ai/free/en/companies/linagora/) — Company, Toulouse, France · 6 works · 133 citations

Also on the byline: [Indian Statistical Institute](https://ror.org/00q2w1j53).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [➗ Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/) · 177 works · [➗ Applied Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/applied-mathematics/) · 46 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Soumadeep Saha, Akshay Chaturvedi, Saptarshi Saha, Utpal Garain and Nicholas Asher. 2026. KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{saha2026kismath,
  title = {{KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?}},
  author = {Soumadeep Saha, Akshay Chaturvedi, Saptarshi Saha, Utpal Garain and Nicholas Asher},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.729},
  url = {https://doi.org/10.1162/tacl.a.729}
}
```

- [← previous KinyCOMET: Automatic Evaluation of Machine Translation Systems for Kinyarwanda–English](https://md.research.elamur.ai/free/en/papers/mazimpaka-2026-kinycomet-automatic-evaluation-machine-translation/)
- [next → KliniskVestBERT: BERT Model Specialised to Norwegian Clinical Texts](https://md.research.elamur.ai/free/en/papers/autenried-2026-kliniskvestbert-bert-model-specialised-norwegian/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/)

## References

46 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Nicholas Asher. 1993. Reference to Abstract Objects in Discourse. Dordrecht, Boston, and London. [find](https://scholar.google.com/scholar?q=Nicholas+Asher%2E+1993%2E+Reference+to+Abstract+Objects+in+Discourse%2E+Dordrecht%2C+Boston%2C+and+London%2E)
2. Nicholas Asher and Alex Lascarides. 2003. Logics of Conversation. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Nicholas+Asher+and+Alex+Lascarides%2E+2003%2E+Logics+of+Conversation%2E+Cambridge+University+Press%2E)
3. Fazl Barez, Tung-Yu Wu, Iván Arcuschin, Michael Lan, Vincent Wang, Noah Siegel, Nicolas Col-lignon, Clement Neo, Isabelle Lee, Alasdair Paren, Adel Bibi, Robert Trager, Damiano Fornasiere, John Yan, Yanai Elazar, and Yoshua Bengio. 2020. Chain-of-Thought Is Not Explainability. preprint, abs/2025.02. [find](https://scholar.google.com/scholar?q=Fazl+Barez%2C+Tung%2DYu+Wu%2C+Iv%C3%A1n+Arcuschin%2C+Michael+Lan%2C+Vincent+Wang%2C+Noah+Siegel%2C+Nicolas+Col%2Dlignon%2C+Clement+Neo%2C+Isabelle+Lee%2C+Alasdair+Paren%2C+Adel+Bibi%2C+Robert+Trager%2C+Damiano+Fornasiere%2C+John+Yan%2C+Y)
4. Paul C. Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy. 2025. Thought Anchors: Which LLM Reasoning Steps Matter? ArXiv preprint, abs/2506.19143. [find](https://scholar.google.com/scholar?q=Paul+C%2E+Bogdan%2C+Uzay+Macar%2C+Neel+Nanda%2C+and+Arthur+Conmy%2E+2025%2E+Thought+Anchors%3A+Which+LLM+Reasoning+Steps+Matter%3F+ArXiv+preprint%2C+abs%2F2506%2E19143%2E)
5. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training Verifiers to Solve Math Word Problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). Preprint, arXiv:2110.14168.
6. DeepSeek-AI. 2025. DeepSeek-R1-0528 Release. [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2E+2025%2E+DeepSeek%2DR1%2D0528+Release%2E)
7. Erwan Fagnou, Paul Caillon, Blaise Delattre, and Alexandre Allauzen. 2024. Chain and Causal Attention for Efficient Entity Tracking. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13174–13188, Miami, Florida, USA. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Erwan+Fagnou%2C+Paul+Caillon%2C+Blaise+Delattre%2C+and+Alexandre+Allauzen%2E+2024%2E+Chain+and+Causal+Attention+for+Efficient+Entity+Tracking%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natur)
8. Olga Golovneva, Moya Peng Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. 2023. [ROSCOE: A suite of metrics for scoring step-by-step reasoning](https://md.research.elamur.ai/free/en/papers/golovneva-2022-roscoe-suite-metrics-scoring-step/). In The Eleventh International Conference on Learning Representations.
9. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
10. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and 175 others. 2025. Deepseek-r1 incentivizes reasoning in llms through rein… [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Dejian+Yang%2C+Haowei+Zhang%2C+Junxiao+Song%2C+Peiyi+Wang%2C+Qihao+Zhu%2C+Runxin+Xu%2C+Ruoyu+Zhang%2C+Shirong+Ma%2C+Xiao+Bi%2C+Xiaokang+Zhang%2C+Xingkai+Yu%2C+Yu+Wu%2C+Z%2E+F%2E+Wu%2C+Zhibin+Gou%2C+Zhihong+Shao%2C+Zhuoshu+Li)
11. Alon Jacovi, Yonatan Bitton, Bernd Bohnet, Jonathan Herzig, Or Honovich, Michael Tseng, Michael Collins, Roee Aharoni, and Mor Geva. 2024. [A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains](https://md.research.elamur.ai/free/en/papers/jacovi-2024-chain-thought-is-as-strong/). In Proceedings of the 62nd Annual Meeting of the Association…
12. Subbarao Kambhampati. 2024. Can large language models reason and plan? Annals of the New York Academy of Sciences, 1534(1):15–18. [find](https://scholar.google.com/scholar?q=Subbarao+Kambhampati%2E+2024%2E+Can+large+language+models+reason+and+plan%3F+Annals+of+the+New+York+Academy+of+Sciences%2C+1534%281%29%3A15%E2%80%9318%2E)
13. Subbarao Kambhampati, Kaya Stechly, and Karthik Valmeekam. 2025. (How) Do reasoning models reason? Annals of the New York Academy of Sciences, 1547(1):33–40. [find](https://scholar.google.com/scholar?q=Subbarao+Kambhampati%2C+Kaya+Stechly%2C+and+Karthik+Valmeekam%2E+2025%2E+%28How%29+Do+reasoning+models+reason%3F+Annals+of+the+New+York+Academy+of+Sciences%2C+1547%281%29%3A33%E2%80%9340%2E)
14. Alistair Knott, Jon Oberlander, Michael O’Donnell, and Chris Mellish. 2000. Beyond elaboration: The interaction of relations and focus in coherent text. In Text Representation: Linguistic and Psycholinguistic Aspects, chapter 7, Human Cognitive Processing, pages 181–196. John Benjamins Publishing Co… [find](https://scholar.google.com/scholar?q=Alistair+Knott%2C+Jon+Oberlander%2C+Michael+O%E2%80%99Donnell%2C+and+Chris+Mellish%2E+2000%2E+Beyond+elaboration%3A+The+interaction+of+relations+and+focus+in+coherent+text%2E+In+Text+Representation%3A+Linguistic+and+Psycholi)
15. Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, an… — [Measuring Faithfulness in Chain-of-Thought Reasoning](https://md.research.elamur.ai/free/en/papers/lanham-2023-measuring-faithfulness-chain-thought-reasoning/)
16. Jinu Lee, Sagnik Mukherjee, Dilek Hakkani-Tur, and Julia Hockenmaier. 2025. Reason-ingFlow: Semantic Structure of Complex Reasoning Traces. ArXiv preprint, abs/2506.02532. — [arXiv:2506.02532](https://arxiv.org/abs/2506.02532) [find](https://scholar.google.com/scholar?q=Jinu+Lee%2C+Sagnik+Mukherjee%2C+Dilek+Hakkani%2DTur%2C+and+Julia+Hockenmaier%2E+2025%2E+Reason%2DingFlow%3A+Semantic+Structure+of+Complex+Reasoning+Traces%2E+ArXiv+preprint%2C+abs%2F2506%2E02532%2E+%E2%80%94+%5BarXiv%3A2506%2E02532%5D%28https%3A%2F)
17. Belinda Z. Li, Maxwell Nye, and Jacob Andreas. 2021. Implicit representations of meaning in neural language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pa… [find](https://scholar.google.com/scholar?q=Belinda+Z%2E+Li%2C+Maxwell+Nye%2C+and+Jacob+Andreas%2E+2021%2E+Implicit+representations+of+meaning+in+neural+language+models%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Lingu)
18. Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G Patil, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. 2025. Language models can easily learn to reason from demonstrations. In Findings of the Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Dacheng+Li%2C+Shiyi+Cao%2C+Tyler+Griggs%2C+Shu+Liu%2C+Xiangxi+Mo%2C+Eric+Tang%2C+Sumanth+Hegde%2C+Kourosh+Hakhamaneshi%2C+Shishir+G+Patil%2C+Matei+Zaharia%2C+Joseph+E%2E+Gonzalez%2C+and+Ion+Stoica%2E+2025%2E+Language+models+can+)
19. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s Verify Step by Step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). Preprint, arXiv:2305.20050.
20. Yanming Liu, Xinyue Peng, Tianyu Du, Jianwei Yin, Weihao Liu, and Xuhong Zhang. 2024. ERA-CoT: Improving chain-of-thought through entity relationship analysis. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8780–8794, Bangkok… [find](https://scholar.google.com/scholar?q=Yanming+Liu%2C+Xinyue+Peng%2C+Tianyu+Du%2C+Jianwei+Yin%2C+Weihao+Liu%2C+and+Xuhong+Zhang%2E+2024%2E+ERA%2DCoT%3A+Improving+chain%2Dof%2Dthought+through+entity+relationship+analysis%2E+In+Proceedings+of+the+62nd+Annual+Meetin)
21. Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. 2023. [Faithful Chain-of-Thought Reasoning](https://md.research.elamur.ai/free/en/papers/lyu-2023-faithful-chain-thought-reasoning/). In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter…
22. William C Mann and Sandra A Thompson. 1987. Rhetorical structure theory: Description and construction of text structures. In Natural language generation: New results in artificial intelligence, psychology and linguistics, pages 85–95. Springer. [find](https://scholar.google.com/scholar?q=William+C+Mann+and+Sandra+A+Thompson%2E+1987%2E+Rhetorical+structure+theory%3A+Description+and+construction+of+text+structures%2E+In+Natural+language+generation%3A+New+results+in+artificial+intelligence%2C+psycho)
23. Jordan Meadows and André Freitas. 2023. Introduction to mathematical language processing: Informal proofs, word problems, and supporting tasks. Transactions of the Association for Computational Linguistics, 11:1162–1184. [find](https://scholar.google.com/scholar?q=Jordan+Meadows+and+Andr%C3%A9+Freitas%2E+2023%2E+Introduction+to+mathematical+language+processing%3A+Informal+proofs%2C+word+problems%2C+and+supporting+tasks%2E+Transactions+of+the+Association+for+Computational+Lingui)
24. Meta. 2024a. Introducing Llama 3.1: Our most capable models to date. [find](https://scholar.google.com/scholar?q=Meta%2E+2024a%2E+Introducing+Llama+3%2E1%3A+Our+most+capable+models+to+date%2E)
25. Meta. 2024b. Llama 3.3 Model Cards and Prompt formats. [find](https://scholar.google.com/scholar?q=Meta%2E+2024b%2E+Llama+3%2E3+Model+Cards+and+Prompt+formats%2E)
26. Sagnik Mukherjee, Abhinav Chinta, Takyoung Kim, Tarun Anoop Sharma, and Dilek Hakkani Tur. 2025. Premise-augmented reasoning chains improve error identification in math reasoning with LLMs. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Sagnik+Mukherjee%2C+Abhinav+Chinta%2C+Takyoung+Kim%2C+Tarun+Anoop+Sharma%2C+and+Dilek+Hakkani+Tur%2E+2025%2E+Premise%2Daugmented+reasoning+chains+improve+error+identification+in+math+reasoning+with+LLMs%2E+In+Forty%2Ds)
27. Ansong Ni, Pengcheng Yin, Yilun Zhao, Martin Riddell, Troy Feng, Rui Shen, Stephen Yin, Ye Liu, Semih Yavuz, Caiming Xiong, Shafiq Joty, Yingbo Zhou, Dragomir Radev, Arman Cohan, and Arman Cohan. 2024. [L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models](https://md.research.elamur.ai/free/en/papers/ni-2024-l2ceval-evaluating-language-code-generation/). Transaction…
28. OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, and 244 others. 2024. [OpenAI o1 Sy…](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/)
29. OpenAI. 2025. Introducing OpenAI o3 and o4-mini. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025%2E+Introducing+OpenAI+o3+and+o4%2Dmini%2E)
30. Debjit Paul, Robert West, Antoine Bosselut, and Boi Faltings. 2024. [Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning](https://md.research.elamur.ai/free/en/papers/paul-2024-making-reasoning-matter-measuring-improving/). In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15012–15032, Miami, Florida, USA. Association for Computati…
31. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse Tree-Bank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Asso… [find](https://scholar.google.com/scholar?q=Rashmi+Prasad%2C+Nikhil+Dinesh%2C+Alan+Lee%2C+Eleni+Miltsakaki%2C+Livio+Robaldo%2C+Aravind+Joshi%2C+and+Bonnie+Webber%2E+2008%2E+The+Penn+Discourse+Tree%2DBank+2%2E0%2E+In+Proceedings+of+the+Sixth+International+Conference+)
32. Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, and 24 others. 2024. Qwen2.5 Technical Report. ArXiv preprint, abs/241… [find](https://scholar.google.com/scholar?q=Qwen%2C+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+Yan)
33. Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, Yulia Tsvetkov, Hannaneh Hajishirzi, Pang Wei Koh, and Luke Zettlemoyer. 2025. Spurious Rewards: Rethinking Training Signals in RLVR. ArXiv preprint, abs/2506.1094… [find](https://scholar.google.com/scholar?q=Rulin+Shao%2C+Shuyue+Stella+Li%2C+Rui+Xin%2C+Scott+Geng%2C+Yiping+Wang%2C+Sewoong+Oh%2C+Simon+Shaolei+Du%2C+Nathan+Lambert%2C+Sewon+Min%2C+Ranjay+Krishna%2C+Yulia+Tsvetkov%2C+Hannaneh+Hajishirzi%2C+Pang+Wei+Koh%2C+and+Luke+Zet)
34. Krish Sharma, Niyar R. Barman, Akshay Chaturvedi, and Nicholas Asher. 2025. DIM-SUM: Discourse in Mathematical Reasoning as a Supervision Module. In Proceedings of the 26th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 294–310, Avignon, France. Association for Computa… [find](https://scholar.google.com/scholar?q=Krish+Sharma%2C+Niyar+R%2E+Barman%2C+Akshay+Chaturvedi%2C+and+Nicholas+Asher%2E+2025%2E+DIM%2DSUM%3A+Discourse+in+Mathematical+Reasoning+as+a+Supervision+Module%2E+In+Proceedings+of+the+26th+Annual+Meeting+of+the+Speci)
35. Kaya Stechly, Karthik Valmeekam, and Subbarao Kambhampati. 2024. Chain of Thoughtlessness? An Analysis of CoT in Planning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Kaya+Stechly%2C+Karthik+Valmeekam%2C+and+Subbarao+Kambhampati%2E+2024%2E+Chain+of+Thoughtlessness%3F+An+Analysis+of+CoT+in+Planning%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Neural+Information+Processing+Syste)
36. Kaya Stechly, Karthik Valmeekam, Vardhan Palod, Atharva Gundawar, and Subbarao Kambhampati. 2025. Beyond semantics: The unreasonable effectiveness of reasonless intermediate tokens. In First Workshop on Foundations of Reasoning in Language Models. [find](https://scholar.google.com/scholar?q=Kaya+Stechly%2C+Karthik+Valmeekam%2C+Vardhan+Palod%2C+Atharva+Gundawar%2C+and+Subbarao+Kambhampati%2E+2025%2E+Beyond+semantics%3A+The+unreasonable+effectiveness+of+reasonless+intermediate+tokens%2E+In+First+Workshop+)
37. Juanhe (TJ) Tan. 2023. Causal Abstraction for Chain-of-Thought Reasoning in Arithmetic Word Problems. In Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 155–168, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Juanhe+%28TJ%29+Tan%2E+2023%2E+Causal+Abstraction+for+Chain%2Dof%2DThought+Reasoning+in+Arithmetic+Word+Problems%2E+In+Proceedings+of+the+6th+BlackboxNLP+Workshop%3A+Analyzing+and+Interpreting+Neural+Networks+for+NLP)
38. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+Louis+Rouillard%2C+Thomas+Mesnard%2C+Geoffrey)
39. Hemish Veeraboina. 2023. AIME Problem Set 1983-2024. [find](https://scholar.google.com/scholar?q=Hemish+Veeraboina%2E+2023%2E+AIME+Problem+Set+1983%2D2024%2E)
40. Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2023a. [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://md.research.elamur.ai/free/en/papers/wang-2023-understanding-chain-thought-prompting-empirical/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa…
41. Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023b. Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo… [find](https://scholar.google.com/scholar?q=Lei+Wang%2C+Wanyu+Xu%2C+Yihuai+Lan%2C+Zhiqiang+Hu%2C+Yunshi+Lan%2C+Roy+Ka%2DWei+Lee%2C+and+Ee%2DPeng+Lim%2E+2023b%2E+Plan%2Dand%2DSolve+Prompting%3A+Improving+Zero%2DShot+Chain%2Dof%2DThought+Reasoning+by+Large+Language+Models%2E+In+P)
42. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xiong-Hui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. 2025. [Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcem…](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/)
43. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022. [Chain of Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems.
44. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 Technical Report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
45. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. 2025. [Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/) In The Thirty-ninth Annual Conference on Neural Information Processing Systems.
46. Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2023. [Automatic Chain of Thought Prompting in Large Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2022-automatic-chain-thought-prompting-large/). In The Eleventh International Conference on Learning Representations.

*Updated 1 November 2026*