---
title: "What Happened in LLMs Layers when Trained for Fast vs. Slow"
description: "Instead of probing a finished model, the authors examine gradients during instruction-tuning-style… 56 references, cited by 14 papers here."
authors: ["Ming Li", "Yanhong Li", "Tianyi Zhou"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.1545/"
doi: "10.18653/v1/2025.acl-long.1545"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/li-2025-what-happened-llms-layers-when/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/li-2025-what-happened-llms-layers-when/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/) / What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective

# What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective

**Authors:** [Ming Li](https://md.research.elamur.ai/free/en/authors/ming-li/), Yanhong Li and [Tianyi Zhou](https://md.research.elamur.ai/free/en/authors/tianyi-zhou/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.1545/) · [PDF](https://aclanthology.org/2025.acl-long.1545.pdf) · [DOI 10.18653/v1/2025.acl-long.1545](https://doi.org/10.18653/v1/2025.acl-long.1545) · [OpenAlex](https://openalex.org/W4412889579)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [14](https://md.research.elamur.ai/free/en/papers/li-2025-what-happened-llms-layers-when/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks how training an LLM on fast-thinking responses (no or brief reasoning) differs from training on slow-thinking responses (detailed chain-of-thought) at the level of individual layers. Instead of probing a finished model, the authors examine gradients during instruction-tuning-style backpropagation. For each attention projection matrix (Query, Key, Value, Output), they compute the nuclear norm via SVD. They then compare how these norms vary across layers using Mean Absolute Difference (MAD) and pairwise Relative Difference (RD).

The experiments cover 5 base and 5 instruction-finetuned LLMs and three task families: math, commonsense reasoning and Wiki knowledge. They contrast correct with irrelevant responses and three levels of reasoning detail.

Detailed CoT produces smaller gradients that are more uniform across layers. Fast thinking produces large early-layer gradients and sharp jumps between layers. Only with detailed reasoning do gradients separate correct from irrelevant answers. In knowledge tasks, making responses longer does not reproduce this effect.

These results suggest gradient behaviour could help judge data quality and point to critical layers for more efficient fine-tuning.

## Key points
- **Datasets:**
  - Math: AQuA, GSM8K, and three MATH subsets (Algebra, Counting and Geometry).
  - Commonsense: four sets, namely Sensemaking, CREAK, ECQA and StrategyQA.
  - Wiki knowledge: popular facts with response lengths of 100, 500 and 1000, plus unpopular facts.
- **Reasoning conditions:** None CoT, Simplified CoT, and Detailed CoT, with the detailed paths written by GPT-4o.
- **Correctness detection:** Instruction-finetuned models were no better than base models at signalling incorrect reasoning paths through their gradients.
- **Unpopular knowledge:**
  - Learning unpopular facts sharply raises gradient scale and fluctuation, especially for instructed models.
  - Nonsense knowledge responses yield nearly the same curves as correct ones.
- **Model families and released data:**
  - Peak patterns differ by family: gemma2 shows two peaks, Qwen2 one, and Llama3 none.
  - The layer-level gradient statistics, which took thousands of GPU hours, are published in the Layer_Gradient GitHub repository.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

💡 Concepts (1): [💡 Latent semantic indexing](https://md.research.elamur.ai/free/en/concepts/latent-semantic-indexing/) · 29 works

## Cite

Cite as: Ming Li, Yanhong Li and Tianyi Zhou. 2025. What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{li2025what,
  title = {{What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective}},
  author = {Ming Li, Yanhong Li and Tianyi Zhou},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1545},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1545}
}
```

- [← previous What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practices](https://md.research.elamur.ai/free/en/papers/chen-2025-what-are-essential-factors-crafting/)
- [next → A Survey on Efficient Large Language Model Training: From Data-centric Perspectives](https://md.research.elamur.ai/free/en/papers/luo-2025-survey-efficient-large-language-model/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/)

## References

56 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, and Dinesh Garg. 2021. [Explanations for CommonsenseQA: New Dataset and Models](https://md.research.elamur.ai/free/en/papers/aggarwal-2021-explanations-commonsenseqa-new-dataset-models/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confe…
2. Guillaume Alain and Yoshua Bengio. 2017. Understanding intermediate layers using linear classifier probes. [find](https://scholar.google.com/scholar?q=Guillaume+Alain+and+Yoshua+Bengio%2E+2017%2E+Understanding+intermediate+layers+using+linear+classifier+probes%2E)
3. Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John Patrick Cunningham. 2024. LoRA learns less and forgets less. Transactions on Machine Learning Research. Feature… [find](https://scholar.google.com/scholar?q=Dan+Biderman%2C+Jacob+Portes%2C+Jose+Javier+Gonzalez+Ortiz%2C+Mansheej+Paul%2C+Philip+Greengard%2C+Connor+Jennings%2C+Daniel+King%2C+Sam+Havens%2C+Vitaliy+Chiley%2C+Jonathan+Frankle%2C+Cody+Blakeney%2C+and+John+Patrick+Cun)
4. Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan Hayase, A Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, et al. 2024. Stealing part of a production language model. arXiv preprint arXiv:2403.06634. — [arXiv:2403.06634](https://arxiv.org/abs/2403.06634) [find](https://scholar.google.com/scholar?q=Nicholas+Carlini%2C+Daniel+Paleka%2C+Krishnamurthy+Dj+Dvijotham%2C+Thomas+Steinke%2C+Jonathan+Hayase%2C+A+Feder+Cooper%2C+Katherine+Lee%2C+Matthew+Jagielski%2C+Milad+Nasr%2C+Arthur+Conmy%2C+et+al%2E+2024%2E+Stealing+part+of+)
5. Xiaodong Chen, Yuxuan Hu, Jing Zhang, Yanling Wang, Cuiping Li, and Hong Chen. 2024. Streamlining redundant layers to compress large language models. Preprint, arXiv:2403.19135. — [arXiv:2403.19135](https://arxiv.org/abs/2403.19135) [find](https://scholar.google.com/scholar?q=Xiaodong+Chen%2C+Yuxuan+Hu%2C+Jing+Zhang%2C+Yanling+Wang%2C+Cuiping+Li%2C+and+Hong+Chen%2E+2024%2E+Streamlining+redundant+layers+to+compress+large+language+models%2E+Preprint%2C+arXiv%3A2403%2E19135%2E+%E2%80%94+%5BarXiv%3A2403%2E19135%5D%28h)
6. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). Preprint, arXiv:2110.14168.
7. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez,… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
8. Siqi Fan, Xin Jiang, Xiang Li, Xuying Meng, Peng Han, Shuo Shang, Aixin Sun, Yequan Wang, and Zhongyuan Wang. 2024. Not all layers of llms are necessary during inference. Preprint, arXiv:2403.02181. — [arXiv:2403.02181](https://arxiv.org/abs/2403.02181) [find](https://scholar.google.com/scholar?q=Siqi+Fan%2C+Xin+Jiang%2C+Xiang+Li%2C+Xuying+Meng%2C+Peng+Han%2C+Shuo+Shang%2C+Aixin+Sun%2C+Yequan+Wang%2C+and+Zhongyuan+Wang%2E+2024%2E+Not+all+layers+of+llms+are+necessary+during+inference%2E+Preprint%2C+arXiv%3A2403%2E02181%2E+%E2%80%94)
9. Chongyang Gao, Kezhen Chen, Jinmeng Rao, Baochen Sun, Ruibo Liu, Daiyi Peng, Yawen Zhang, Xiaoyuan Guo, Jie Yang, and VS Subrahmanian. 2024. Higher layers need more lora experts. Preprint, arXiv:2402.08562. — [arXiv:2402.08562](https://arxiv.org/abs/2402.08562) [find](https://scholar.google.com/scholar?q=Chongyang+Gao%2C+Kezhen+Chen%2C+Jinmeng+Rao%2C+Baochen+Sun%2C+Ruibo+Liu%2C+Daiyi+Peng%2C+Yawen+Zhang%2C+Xiaoyuan+Guo%2C+Jie+Yang%2C+and+VS+Subrahmanian%2E+2024%2E+Higher+layers+need+more+lora+experts%2E+Preprint%2C+arXiv%3A2402%2E)
10. Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346– 361. [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Daniel+Khashabi%2C+Elad+Segal%2C+Tushar+Khot%2C+Dan+Roth%2C+and+Jonathan+Berant%2E+2021%2E+Did+aristotle+use+a+laptop%3F+a+question+answering+benchmark+with+implicit+reasoning+strategies%2E+Transactions+of+)
11. Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Ramaneswaran S, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, and Dinesh Manocha. 2024. A closer look at the limitations of instruction tuning. Preprint, arXiv:2402.05119. — [arXiv:2402.05119](https://arxiv.org/abs/2402.05119) [find](https://scholar.google.com/scholar?q=Sreyan+Ghosh%2C+Chandra+Kiran+Reddy+Evuru%2C+Sonal+Kumar%2C+Ramaneswaran+S%2C+Deepali+Aneja%2C+Zeyu+Jin%2C+Ramani+Duraiswami%2C+and+Dinesh+Manocha%2E+2024%2E+A+closer+look+at+the+limitations+of+instruction+tuning%2E+Prep)
12. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings. [find](https://scholar.google.com/scholar?q=Xavier+Glorot+and+Yoshua+Bengio%2E+2010%2E+Understanding+the+difficulty+of+training+deep+feedforward+neural+networks%2E+In+Proceedings+of+the+thirteenth+international+conference+on+artificial+intelligence+a)
13. Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. 2023. Finding neurons in a haystack: Case studies with sparse probing. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Wes+Gurnee%2C+Neel+Nanda%2C+Matthew+Pauly%2C+Katherine+Harvey%2C+Dmitrii+Troitskii%2C+and+Dimitris+Bertsimas%2E+2023%2E+Finding+neurons+in+a+haystack%3A+Case+studies+with+sparse+probing%2E+Transactions+on+Machine+Learn)
14. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations.
15. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023. [A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). Preprint, arXiv:2311.05232.
16. Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao Liu, Heng Ji,… — [TrustLLM: Trustworthiness in Large Language Models](https://md.research.elamur.ai/free/en/papers/huang-2024-trustllm-trustworthiness-large-language-models/)
17. Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, and Yongfeng Zhang. 2024. Exploring concept depth: How large language models acquire knowledge at different layers? Preprint, arXiv:2404.07066. — [arXiv:2404.07066](https://arxiv.org/abs/2404.07066) [find](https://scholar.google.com/scholar?q=Mingyu+Jin%2C+Qinkai+Yu%2C+Jingyuan+Huang%2C+Qingcheng+Zeng%2C+Zhenting+Wang%2C+Wenyue+Hua%2C+Haiyan+Zhao%2C+Kai+Mei%2C+Yanda+Meng%2C+Kaize+Ding%2C+Fan+Yang%2C+Mengnan+Du%2C+and+Yongfeng+Zhang%2E+2024%2E+Exploring+concept+depth%3A)
18. Tianjie Ju, Weiwei Sun, Wei Du, Xinwei Yuan, Zhaochun Ren, and Gongshen Liu. 2024. How large language models encode context knowledge? a layer-wise probing study. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 20… [find](https://scholar.google.com/scholar?q=Tianjie+Ju%2C+Weiwei+Sun%2C+Wei+Du%2C+Xinwei+Yuan%2C+Zhaochun+Ren%2C+and+Gongshen+Liu%2E+2024%2E+How+large+language+models+encode+context+knowledge%3F+a+layer%2Dwise+probing+study%2E+In+Proceedings+of+the+2024+Joint+Inte)
19. Daniel Kahneman. 2011. Thinking, fast and slow. Farrar, Straus and Giroux. [find](https://scholar.google.com/scholar?q=Daniel+Kahneman%2E+2011%2E+Thinking%2C+fast+and+slow%2E+Farrar%2C+Straus+and+Giroux%2E)
20. Satyapriya Krishna, Jiaqi Ma, Dylan Slack, Asma Ghandeharioun, Sameer Singh, and Himabindu Lakkaraju. 2024. Post hoc explanations of language models can improve language models. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Satyapriya+Krishna%2C+Jiaqi+Ma%2C+Dylan+Slack%2C+Asma+Ghandeharioun%2C+Sameer+Singh%2C+and+Himabindu+Lakkaraju%2E+2024%2E+Post+hoc+explanations+of+language+models+can+improve+language+models%2E+Advances+in+Neural+Inf)
21. Ming Li, Jiuhai Chen, Lichang Chen, and Tianyi Zhou. 2024a. Can LLMs speak for diverse people? tuning LLMs via debate to generate controllable controversial statements. In Findings of the Association for Computational Linguistics ACL 2024, pages 16160– 16176, Bangkok, Thailand and virtual meeting. A… [find](https://scholar.google.com/scholar?q=Ming+Li%2C+Jiuhai+Chen%2C+Lichang+Chen%2C+and+Tianyi+Zhou%2E+2024a%2E+Can+LLMs+speak+for+diverse+people%3F+tuning+LLMs+via+debate+to+generate+controllable+controversial+statements%2E+In+Findings+of+the+Association+)
22. Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, Jiuxiang Gu, and Tianyi Zhou. 2024b. Selective reflection-tuning: Student-selected data recycling for LLM instruction-tuning. In Findings of the Association for Computational Linguistics ACL 2024, pages 16189– 16211, Bangkok, Thailand and virtual meeting… [find](https://scholar.google.com/scholar?q=Ming+Li%2C+Lichang+Chen%2C+Jiuhai+Chen%2C+Shwai+He%2C+Jiuxiang+Gu%2C+and+Tianyi+Zhou%2E+2024b%2E+Selective+reflection%2Dtuning%3A+Student%2Dselected+data+recycling+for+LLM+instruction%2Dtuning%2E+In+Findings+of+the+Associati)
23. Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, and Tianyi Zhou. 2023. Reflection-tuning: Recycling data for better instruction-tuning. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following. [find](https://scholar.google.com/scholar?q=Ming+Li%2C+Lichang+Chen%2C+Jiuhai+Chen%2C+Shwai+He%2C+and+Tianyi+Zhou%2E+2023%2E+Reflection%2Dtuning%3A+Recycling+data+for+better+instruction%2Dtuning%2E+In+NeurIPS+2023+Workshop+on+Instruction+Tuning+and+Instruction+Fol)
24. Ming Li, Yanhong Li, Ziyue Li, and Tianyi Zhou. 2025a. How instruction and reasoning data shape post-training: Data quality through the lens of layer-wise gradients. arXiv preprint arXiv:2504.10766. [find](https://scholar.google.com/scholar?q=Ming+Li%2C+Yanhong+Li%2C+Ziyue+Li%2C+and+Tianyi+Zhou%2E+2025a%2E+How+instruction+and+reasoning+data+shape+post%2Dtraining%3A+Data+quality+through+the+lens+of+layer%2Dwise+gradients%2E+arXiv+preprint+arXiv%3A2504%2E10766%2E)
25. Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. 2024c. [Superfiltering: Weak-to-strong data filtering for fast instruction-tuning](https://md.research.elamur.ai/free/en/papers/li-2024-superfiltering-weak-strong-data-filtering/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag…
26. Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. 2024d. [From quantity to quality: Boosting LLM performance with self-guided data selection for instruction tuning](https://md.research.elamur.ai/free/en/papers/li-2023-quantity-quality-boosting-llm-performance/). In Proceedings of the 2024 Conference of the North American Chapter of t…
27. Shen Li, Liuyi Yao, Lan Zhang, and Yaliang Li. 2024e. Safety layers in aligned large language models: The key to llm security. Preprint, arXiv:2408.17003. — [arXiv:2408.17003](https://arxiv.org/abs/2408.17003) [find](https://scholar.google.com/scholar?q=Shen+Li%2C+Liuyi+Yao%2C+Lan+Zhang%2C+and+Yaliang+Li%2E+2024e%2E+Safety+layers+in+aligned+large+language+models%3A+The+key+to+llm+security%2E+Preprint%2C+arXiv%3A2408%2E17003%2E+%E2%80%94+%5BarXiv%3A2408%2E17003%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F24)
28. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhiwei Li, Bao-Long Bi, Ling-Rui Mei, Junfeng Fang, Zhijiang Guo, Le Song, and Cheng-Lin Liu. 2025b. [From system 1 to system…](https://md.research.elamur.ai/free/en/papers/li-2025-system-1-system-2-survey/)
29. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 158–167, Vancouver… [find](https://scholar.google.com/scholar?q=Wang+Ling%2C+Dani+Yogatama%2C+Chris+Dyer%2C+and+Phil+Blunsom%2E+2017%2E+Program+induction+by+rationale+generation%3A+Learning+to+solve+and+explain+algebraic+word+problems%2E+In+Proceedings+of+the+55th+Annual+Meetin)
30. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. [The flan collection: Designing data and methods for effective instruction tuning](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/). Preprint, arXiv:2301.13688.
31. Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. 2024. [Shortgpt: Layers in large language models are more redundant than you expect](https://md.research.elamur.ai/free/en/papers/men-2024-shortgpt-layers-large-language-models/). Preprint, arXiv:2403.03853.
32. Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan I… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
33. Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021. Cross-task generalization via natural language crowdsourcing instructions. arXiv preprint arXiv:2104.08773. [find](https://scholar.google.com/scholar?q=Swaroop+Mishra%2C+Daniel+Khashabi%2C+Chitta+Baral%2C+and+Hannaneh+Hajishirzi%2E+2021%2E+Cross%2Dtask+generalization+via+natural+language+crowdsourcing+instructions%2E+arXiv+preprint+arXiv%3A2104%2E08773%2E)
34. Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agrawal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, Hamid Palangi, Guoqing Zheng, Corby Rosset, Hamed Khanpour, and Ahmed Awadallah. 2023. [Orca 2: Teaching small language models how to reason](https://md.research.elamur.ai/free/en/papers/mitra-2023-orca-2-teaching-small-language/).…
35. Yasumasa Onoe, Michael J. Q. Zhang, Eunsol Choi, and Greg Durrett. 2021. Creak: A dataset for commonsense reasoning over entity knowledge. Preprint, arXiv:2109.01653. [find](https://scholar.google.com/scholar?q=Yasumasa+Onoe%2C+Michael+J%2E+Q%2E+Zhang%2C+Eunsol+Choi%2C+and+Greg+Durrett%2E+2021%2E+Creak%3A+A+dataset+for+commonsense+reasoning+over+entity+knowledge%2E+Preprint%2C+arXiv%3A2109%2E01653%2E)
36. Ben Prystawski, Michael Y. Li, and Noah Goodman. 2023. Why think step by step? reasoning emerges from the locality of experience. In Thirty-seventh Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Ben+Prystawski%2C+Michael+Y%2E+Li%2C+and+Noah+Goodman%2E+2023%2E+Why+think+step+by+step%3F+reasoning+emerges+from+the+locality+of+experience%2E+In+Thirty%2Dseventh+Conference+on+Neural+Information+Processing+Systems%2E)
37. Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong. 2022. [Interpretable machine learning: Fundamental principles and 10 grand challenges](https://md.research.elamur.ai/free/en/papers/rudin-2021-interpretable-machine-learning-fundamental/). Statistic Surveys, 16:1–85.
38. Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. 2024. [Rethinking interpretability in the era of large language models](https://md.research.elamur.ai/free/en/papers/singh-2024-rethinking-interpretability-era-large-language/). arXiv preprint arXiv:2402.01761.
39. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
40. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton T… [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+Cassidy+Hardin%2C+Surya+Bhupatiraju%2C+L%C3%A9onard+Hussenot%2C+Thomas+Mesnard%2C+Bobak+Shahriari%2C+Alexandre+Ram%C3%A9%2C+Johan+Ferret%2C+Peter+Liu%2C+Pouya+Ta)
41. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing…
42. Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. 2023a. [Label words are anchors: An information flow perspective for understanding in-context learning](https://md.research.elamur.ai/free/en/papers/wang-2023-label-words-anchors-information-flow/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9840–9855, Si…
43. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023b. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023b%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+In+Pro)
44. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations.
45. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Preprint, arXiv:2201.11903.
46. Zhengxuan Wu, Atticus Geiger, Thomas Icard, Christopher Potts, and Noah Goodman. 2024. [Interpretability at scale: Identifying causal mechanisms in alpaca](https://md.research.elamur.ai/free/en/papers/wu-2023-interpretability-scale-identifying-causal-mechanisms/). Advances in Neural Information Processing Systems, 36.
47. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). Preprint, arXiv:2304.12244.
48. Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. 2024. A survey on knowledge distillation of large language models. ArXiv, abs/2402.13116. — [arXiv:2402.13116](https://arxiv.org/abs/2402.13116) [find](https://scholar.google.com/scholar?q=Xiaohan+Xu%2C+Ming+Li%2C+Chongyang+Tao%2C+Tao+Shen%2C+Reynold+Cheng%2C+Jinyang+Li%2C+Can+Xu%2C+Dacheng+Tao%2C+and+Tianyi+Zhou%2E+2024%2E+A+survey+on+knowledge+distillation+of+large+language+models%2E+ArXiv%2C+abs%2F2402%2E13116%2E)
49. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Juny… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Guanting+Dong%2C+Haoran+Wei%2C+Huan+Lin%2C+Jialong+Tang%2C+Jialin+Wang%2C+Jian+Yang%2C+Jian)
50. Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen-Zhu. 2024. Physics of language models: Part 2.1, grade-school math and the hidden reasoning process. Preprint, arXiv:2407.20311. [find](https://scholar.google.com/scholar?q=Tian+Ye%2C+Zicheng+Xu%2C+Yuanzhi+Li%2C+and+Zeyuan+Allen%2DZhu%2E+2024%2E+Physics+of+language+models%3A+Part+2%2E1%2C+grade%2Dschool+math+and+the+hidden+reasoning+process%2E+Preprint%2C+arXiv%3A2407%2E20311%2E)
51. Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. 2023. [Instruction tuning for large language models: A survey](https://md.research.elamur.ai/free/en/papers/zhang-2023-instruction-tuning-large-language-models/). Preprint, arXiv:2308.10792.
52. Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2023a. [Explainability for large language models: A survey](https://md.research.elamur.ai/free/en/papers/zhao-2023-explainability-large-language-models-survey/). Preprint, arXiv:2309.01029.
53. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023b. [A survey of…](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/)
54. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023. Lima: Less is more for alignment. Preprint, arXiv:2305.11206. [find](https://scholar.google.com/scholar?q=Chunting+Zhou%2C+Pengfei+Liu%2C+Puxin+Xu%2C+Srini+Iyer%2C+Jiao+Sun%2C+Yuning+Mao%2C+Xuezhe+Ma%2C+Avia+Efrat%2C+Ping+Yu%2C+Lili+Yu%2C+Susan+Zhang%2C+Gargi+Ghosh%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Omer+Levy%2E+2023%2E+Lima%3A+Less)
55. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendry… — [Representation Engineering: A Top-Down Approach to AI Transparency](https://md.research.elamur.ai/free/en/papers/zou-2023-representation-engineering-top-down-approach/)
56. Table of Contents A Dataset Description A.1 Math Reasoning . . . . . . . . . . A.2 Commonsense Reasoning . . . . . A.3 Wiki Knowledge Learning . . . . 14 14 14 14 [find](https://scholar.google.com/scholar?q=Table+of+Contents+A+Dataset+Description+A%2E1+Math+Reasoning+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+%2E+A%2E2+Commonsense+Reasoning+%2E+%2E+%2E+%2E+%2E+A%2E3+Wiki+Knowledge+Learning+%2E+%2E+%2E+%2E+14+14+14+14)

*Updated 30 October 2026*