---
title: "Mechanistic Interpretability Meets Cognitive — Proietti 2026"
description: "Mechanistic Interpretability (MI) has mostly been applied to AI-safety questions and seldom to hypotheses from linguistic theory."
authors: ["Mattia Proietti", "Afra Alishahi", "Grzegorz Chrupała", "Alessandro Lenci"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.885/"
doi: "10.63317/4iwnnycar4sc"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/proietti-2026-mechanistic-interpretability-meets-cognitive-linguistics/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/proietti-2026-mechanistic-interpretability-meets-cognitive-linguistics/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/) / Mechanistic Interpretability Meets Cognitive Linguistics: Modelling Locative Image Schemas in the Circuit Framework

# Mechanistic Interpretability Meets Cognitive Linguistics: Modelling Locative Image Schemas in the Circuit Framework

**Authors:** Mattia Proietti, [Afra Alishahi](https://md.research.elamur.ai/free/en/authors/afra-alishahi/), [Grzegorz Chrupała](https://md.research.elamur.ai/free/en/authors/grzegorz-chrupala/), [Alessandro Lenci](https://md.research.elamur.ai/free/en/authors/alessandro-lenci/) · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.885/) · [PDF](https://aclanthology.org/2026.lrec-1.885.pdf) · [DOI 10.63317/4iwnnycar4sc](https://doi.org/10.63317/4iwnnycar4sc) · [OpenAlex](https://openalex.org/W7160061604)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Mechanistic Interpretability (MI) has mostly been applied to AI-safety questions and seldom to hypotheses from linguistic theory. This paper connects the two. It asks whether LlaMA-3.2-1b has dedicated internal mechanisms for choosing between the locative prepositions *in* and *on* in simple copular sentences. In Cognitive Linguistics, these prepositions encode two Image Schemas, Containment and Surface, and the choice depends largely on the Landmark noun (a box contains, a mat supports).

In normal English word order the Landmark comes after the preposition, which makes the choice hard to test in a left-to-right model. The authors therefore use preposition stranding, as in a question asking which mat the cat was on, so that the preposition becomes the next token to predict. They build a small, manually curated stimulus set for this task. They then apply several MI techniques to measure the Landmark token's influence and to look for a compact circuit behind the prediction.

The extracted text omits the experiments and results, so no findings are reported here. The work offers a template for testing linguistic hypotheses with MI tools.

## Key points
- The model studied is LlaMA-3.2-1b. The new task is named Stranded Locative Preposition Selection in the abstract and Locative Stranded Preposition Prediction in the body.
- There are two research questions: (RQ1) what role the Landmark token plays in preposition choice, and (RQ2) whether a small subgraph (circuit) responsible for the behaviour can be localised.
- The paper claims three contributions: an evaluation framework pairing MI with linguistic theory, the stranded-preposition task, and a curated benchmark dataset. Code and data are at github.com/aittam9/stranded_prep.
- The theoretical basis is the Figure/Landmark distinction: Containment implies inclusion (*in*), while Surface implies contact and support (*on*). The authors acknowledge exceptions such as *on a bus/train*.
- The paper was published at LREC 2026, pp. 11320–11331, by authors from Tilburg University and the University of Pisa.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{proietti2026mechanistic,
  title = {{Mechanistic Interpretability Meets Cognitive Linguistics: Modelling Locative Image Schemas in the Circuit Framework}},
  author = {Proietti, Mattia and Alishahi, Afra and Chrupała, Grzegorz and Lenci, Alessandro},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/4iwnnycar4sc},
  url = {https://doi.org/10.63317/4iwnnycar4sc}
}
```

- [← previous Measuring Idiomaticity in Text Embedding Models with epsilon-compositionality](https://md.research.elamur.ai/free/en/papers/wold-2026-measuring-idiomaticity-text-embedding-models/)
- [next → MedAraBench: Large-Scale Arabic Medical Question Answering Dataset and Benchmark](https://md.research.elamur.ai/free/en/papers/abu-2026-daoud-medarabench-large-scale-arabic-medical/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/)

## References

65 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aryaman Arora, Dan Jurafsky, and Christopher Potts. 2024. CausalGym: Benchmarking causal interpretability methods on linguistic tasks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14638–14663, Bangkok, Thailand. Association… [find](https://scholar.google.com/scholar?q=Aryaman+Arora%2C+Dan+Jurafsky%2C+and+Christopher+Potts%2E+2024%2E+CausalGym%3A+Benchmarking+causal+interpretability+methods+on+linguistic+tasks%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+)
2. Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219. [find](https://scholar.google.com/scholar?q=Yonatan+Belinkov%2E+2022%2E+Probing+classifiers%3A+Promises%2C+shortcomings%2C+and+advances%2E+Computational+Linguistics%2C+48%281%29%3A207%E2%80%93219%2E) [our card](https://md.research.elamur.ai/free/en/papers/belinkov-2022-probing-classifiers-promises-shortcomings-advances/)
3. Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185–5198, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender+and+Alexander+Koller%2E+2020%2E+Climbing+towards+NLU%3A+On+meaning%2C+form%2C+and+understanding+in+the+age+of+data%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational) [our card](https://md.research.elamur.ai/free/en/papers/bender-2020-climbing-nlu-meaning-form-understanding/)
4. Leonard Bereska and Efstratios Gavves. 2024. Mechanistic interpretability for ai safety–a review. arXiv preprint arXiv:2404.14082. [find](https://scholar.google.com/scholar?q=Leonard+Bereska+and+Efstratios+Gavves%2E+2024%2E+Mechanistic+interpretability+for+ai+safety%E2%80%93a+review%2E+arXiv+preprint+arXiv%3A2404%2E14082%2E)
5. Sasha Boguraev, Christopher Potts, and Kyle Mahowald. 2025. Causal interventions reveal shared structure across english filler-gap constructions. arXiv preprint arXiv:2505.16002. [find](https://scholar.google.com/scholar?q=Sasha+Boguraev%2C+Christopher+Potts%2C+and+Kyle+Mahowald%2E+2025%2E+Causal+interventions+reveal+shared+structure+across+english+filler%2Dgap+constructions%2E+arXiv+preprint+arXiv%3A2505%2E16002%2E) [our card](https://md.research.elamur.ai/free/en/papers/boguraev-2025-causal-interventions-reveal-shared-structure/)
6. Timothy C Clausner and William Croft. 1997. Productivity and schematicity in metaphors. Cognitive science, 21(3):247–282. [find](https://scholar.google.com/scholar?q=Timothy+C+Clausner+and+William+Croft%2E+1997%2E+Productivity+and+schematicity+in+metaphors%2E+Cognitive+science%2C+21%283%29%3A247%E2%80%93282%2E)
7. Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. 2023. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36:16318–16352. [find](https://scholar.google.com/scholar?q=Arthur+Conmy%2C+Augustine+Mavor%2DParker%2C+Aengus+Lynch%2C+Stefan+Heimersheim%2C+and+Adri%C3%A0+Garriga%2DAlonso%2E+2023%2E+Towards+automated+circuit+discovery+for+mechanistic+interpretability%2E+Advances+in+Neural+Informa)
8. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Ja… [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Neel+Nanda%2C+Catherine+Olsson%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Tom+Conerly%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Zac+Hatfield%2DDodds%2C+Da)
9. Javier Ferrando and Marta R. Costa-jussà. 2024. On the similarity of circuits across languages: a case study on the subject-verb agreement task. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10115–10125, Miami, Florida, USA. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Javier+Ferrando+and+Marta+R%2E+Costa%2Djuss%C3%A0%2E+2024%2E+On+the+similarity+of+circuits+across+languages%3A+a+case+study+on+the+subject%2Dverb+agreement+task%2E+In+Findings+of+the+Association+for+Computational+Lingui)
10. Javier Ferrando, Gabriele Sarti, Arianna Bisazza, and Marta R Costa-Jussà. 2024. A primer on the inner workings of transformer-based language models. arXiv preprint arXiv:2405.00208. [find](https://scholar.google.com/scholar?q=Javier+Ferrando%2C+Gabriele+Sarti%2C+Arianna+Bisazza%2C+and+Marta+R+Costa%2DJuss%C3%A0%2E+2024%2E+A+primer+on+the+inner+workings+of+transformer%2Dbased+language+models%2E+arXiv+preprint+arXiv%3A2405%2E00208%2E)
11. Javier Ferrando and Elena Voita. 2024. Information flow routes: Automatically interpreting language models at scale. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17432–17445, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Javier+Ferrando+and+Elena+Voita%2E+2024%2E+Information+flow+routes%3A+Automatically+interpreting+language+models+at+scale%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Proc)
12. Jon Gauthier, Jennifer Hu, Ethan Wilcox, Peng Qian, and Roger Levy. 2020. SyntaxGym: An online platform for targeted evaluation of language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 70–76, Online. Association for… [find](https://scholar.google.com/scholar?q=Jon+Gauthier%2C+Jennifer+Hu%2C+Ethan+Wilcox%2C+Peng+Qian%2C+and+Roger+Levy%2E+2020%2E+SyntaxGym%3A+An+online+platform+for+targeted+evaluation+of+language+models%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Ass) [our card](https://md.research.elamur.ai/free/en/papers/gauthier-2020-syntaxgym-online-platform-targeted-evaluation/)
13. Atticus Geiger, Duligur Ibeling, Amir Zur, Maheep Chaudhary, Sonakshi Chauhan, Jing Huang, Aryaman Arora, Zhengxuan Wu, Noah Goodman, Christopher Potts, et al. 2025. Causal abstraction: A theoretical foundation for mechanistic interpretability. Journal of Machine Learning Research, 26(83):1–64. [find](https://scholar.google.com/scholar?q=Atticus+Geiger%2C+Duligur+Ibeling%2C+Amir+Zur%2C+Maheep+Chaudhary%2C+Sonakshi+Chauhan%2C+Jing+Huang%2C+Aryaman+Arora%2C+Zhengxuan+Wu%2C+Noah+Goodman%2C+Christopher+Potts%2C+et+al%2E+2025%2E+Causal+abstraction%3A+A+theoretical+)
14. Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. 2021. Causal abstractions of neural networks. Advances in Neural Information Processing Systems, 34:9574–9586. [find](https://scholar.google.com/scholar?q=Atticus+Geiger%2C+Hanson+Lu%2C+Thomas+Icard%2C+and+Christopher+Potts%2E+2021%2E+Causal+abstractions+of+neural+networks%2E+Advances+in+Neural+Information+Processing+Systems%2C+34%3A9574%E2%80%939586%2E)
15. Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. 2022. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 30–45, Abu Dhabi, United Arab Emirates. Asso… [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Avi+Caciularu%2C+Kevin+Wang%2C+and+Yoav+Goldberg%2E+2022%2E+Transformer+feed%2Dforward+layers+build+predictions+by+promoting+concepts+in+the+vocabulary+space%2E+In+Proceedings+of+the+2022+Conference+on+) [our card](https://md.research.elamur.ai/free/en/papers/geva-2022-transformer-feed-forward-layers-build/)
16. Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato, and Aryaman Arora. 2023. Localizing model behavior with path patching. arXiv preprint arXiv:2304.05969. [find](https://scholar.google.com/scholar?q=Nicholas+Goldowsky%2DDill%2C+Chris+MacLeod%2C+Lucas+Sato%2C+and+Aryaman+Arora%2E+2023%2E+Localizing+model+behavior+with+path+patching%2E+arXiv+preprint+arXiv%3A2304%2E05969%2E)
17. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. [find](https://scholar.google.com/scholar?q=Aaron+Grattafiori%2C+Abhimanyu+Dubey%2C+Abhinav+Jauhri%2C+Abhinav+Pandey%2C+Abhishek+Kadian%2C+Ahmad+Al%2DDahle%2C+Aiesha+Letman%2C+Akhil+Mathur%2C+Alan+Schelten%2C+Alex+Vaughan%2C+et+al%2E+2024%2E+The+llama+3+herd+of+models%2E+)
18. Abhijeet Gupta, Gemma Boleda, Marco Baroni, and Sebastian Padó. 2015. Distributional vectors encode referential attributes. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 12–21, Lisbon, Portugal. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Abhijeet+Gupta%2C+Gemma+Boleda%2C+Marco+Baroni%2C+and+Sebastian+Pad%C3%B3%2E+2015%2E+Distributional+vectors+encode+referential+attributes%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Langua)
19. Michael Hanna, Yonatan Belinkov, and Sandro Pezzelle. 2025. Are formal and functional linguistic mechanisms dissociated in language models? Computational Linguistics, pages 1–40. [find](https://scholar.google.com/scholar?q=Michael+Hanna%2C+Yonatan+Belinkov%2C+and+Sandro+Pezzelle%2E+2025%2E+Are+formal+and+functional+linguistic+mechanisms+dissociated+in+language+models%3F+Computational+Linguistics%2C+pages+1%E2%80%9340%2E)
20. Michael Hanna, Ollie Liu, and Alexandre Variengien. 2023. How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. Advances in Neural Information Processing Systems, 36:76033–76060. [find](https://scholar.google.com/scholar?q=Michael+Hanna%2C+Ollie+Liu%2C+and+Alexandre+Variengien%2E+2023%2E+How+does+gpt%2D2+compute+greater%2Dthan%3F%3A+Interpreting+mathematical+abilities+in+a+pre%2Dtrained+language+model%2E+Advances+in+Neural+Information+Proc) [our card](https://md.research.elamur.ai/free/en/papers/hanna-2023-gpt-2-compute-greater-than/)
21. Michael Hanna and Aaron Mueller. 2025. Incremental sentence processing mechanisms in autoregressive transformer language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long… [find](https://scholar.google.com/scholar?q=Michael+Hanna+and+Aaron+Mueller%2E+2025%2E+Incremental+sentence+processing+mechanisms+in+autoregressive+transformer+language+models%2E+In+Proceedings+of+the+2025+Conference+of+the+Nations+of+the+Americas+Ch)
22. Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov. 2024. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms. In ICML 2024 Workshop on Mechanistic Interpretability. [find](https://scholar.google.com/scholar?q=Michael+Hanna%2C+Sandro+Pezzelle%2C+and+Yonatan+Belinkov%2E+2024%2E+Have+faith+in+faithfulness%3A+Going+beyond+circuit+overlap+when+finding+model+mechanisms%2E+In+ICML+2024+Workshop+on+Mechanistic+Interpretabilit)
23. Stefan Heimersheim and Neel Nanda. 2024. How to use and interpret activation patching. arXiv preprint arXiv:2404.15255. — [arXiv:2404.15255](https://arxiv.org/abs/2404.15255) [find](https://scholar.google.com/scholar?q=Stefan+Heimersheim+and+Neel+Nanda%2E+2024%2E+How+to+use+and+interpret+activation+patching%2E+arXiv+preprint+arXiv%3A2404%2E15255%2E+%E2%80%94+%5BarXiv%3A2404%2E15255%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2404%2E15255%29)
24. Annette Herskovits. 1985. Semantics and pragmatics of locative expressions. Cognitive Science, 9(3):341–378. [find](https://scholar.google.com/scholar?q=Annette+Herskovits%2E+1985%2E+Semantics+and+pragmatics+of+locative+expressions%2E+Cognitive+Science%2C+9%283%29%3A341%E2%80%93378%2E)
25. Annette Herskovits. 2009. Language and spatial cognition: an interdisciplinary study of the prepositions in English, paperback re-issue edition. Studies in natural language processing. Cambridge Univ. Press, Cambridge. [find](https://scholar.google.com/scholar?q=Annette+Herskovits%2E+2009%2E+Language+and+spatial+cognition%3A+an+interdisciplinary+study+of+the+prepositions+in+English%2C+paperback+re%2Dissue+edition%2E+Studies+in+natural+language+processing%2E+Cambridge+Univ%2E)
26. Thomas Hoffmann. 2011. Preposition Placement in English: A Usage-based Approach, 1 edition. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Thomas+Hoffmann%2E+2011%2E+Preposition+Placement+in+English%3A+A+Usage%2Dbased+Approach%2C+1+edition%2E+Cambridge+University+Press%2E)
27. Mark Johnson. 1987. The body in the mind: The bodily basis of meaning, imagination, and reason. The Personalist Forum, 5(1):58–60. [find](https://scholar.google.com/scholar?q=Mark+Johnson%2E+1987%2E+The+body+in+the+mind%3A+The+bodily+basis+of+meaning%2C+imagination%2C+and+reason%2E+The+Personalist+Forum%2C+5%281%29%3A58%E2%80%9360%2E)
28. Roni Katzir. 2023. Why large language models are poor theories of human linguistic cognition: A reply to piantadosi. Biolinguistics, 17:1–12. [find](https://scholar.google.com/scholar?q=Roni+Katzir%2E+2023%2E+Why+large+language+models+are+poor+theories+of+human+linguistic+cognition%3A+A+reply+to+piantadosi%2E+Biolinguistics%2C+17%3A1%E2%80%9312%2E)
29. Geonhee Kim, Marco Valentino, and Andre Freitas. 2025. Reasoning circuits in language models: A mechanistic interpretation of syllogistic inference. In Findings of the Association for Computational Linguistics: ACL 2025, pages 10074–10095, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Geonhee+Kim%2C+Marco+Valentino%2C+and+Andre+Freitas%2E+2025%2E+Reasoning+circuits+in+language+models%3A+A+mechanistic+interpretation+of+syllogistic+inference%2E+In+Findings+of+the+Association+for+Computational+Li)
30. Arne Köhn. 2015. What’s in an embedding? analyzing word embeddings through multilingual evaluation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2067–2073, Lisbon, Portugal. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Arne+K%C3%B6hn%2E+2015%2E+What%E2%80%99s+in+an+embedding%3F+analyzing+word+embeddings+through+multilingual+evaluation%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+206)
31. George Lakoff. 1987. Women, fire, and dangerous things: what categories reveal about the mind, paperback ed., [nachdr.] edition. The Univ. of Chicago Press, Chicago. [find](https://scholar.google.com/scholar?q=George+Lakoff%2E+1987%2E+Women%2C+fire%2C+and+dangerous+things%3A+what+categories+reveal+about+the+mind%2C+paperback+ed%2E%2C+%5Bnachdr%2E%5D+edition%2E+The+Univ%2E+of+Chicago+Press%2C+Chicago%2E)
32. Alessandro Lenci. 2023. Understanding natural language understanding systems. Sistemi intelligenti, 35(2):277–302. [find](https://scholar.google.com/scholar?q=Alessandro+Lenci%2E+2023%2E+Understanding+natural+language+understanding+systems%2E+Sistemi+intelligenti%2C+35%282%29%3A277%E2%80%93302%2E)
33. Stephen C. Levinson. 2003. Space in Language and Cognition: Explorations in Cognitive Diversity. Language Culture and Cognition. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Stephen+C%2E+Levinson%2E+2003%2E+Space+in+Language+and+Cognition%3A+Explorations+in+Cognitive+Diversity%2E+Language+Culture+and+Cognition%2E+Cambridge+University+Press%2E)
34. Kyle Mahowald, Anna A Ivanova, Idan A Blank, Nancy Kanwisher, Joshua B Tenenbaum, and Evelina Fedorenko. 2024. Dissociating language and thought in large language models. Trends in cognitive sciences, 28(6):517–540. [find](https://scholar.google.com/scholar?q=Kyle+Mahowald%2C+Anna+A+Ivanova%2C+Idan+A+Blank%2C+Nancy+Kanwisher%2C+Joshua+B+Tenenbaum%2C+and+Evelina+Fedorenko%2E+2024%2E+Dissociating+language+and+thought+in+large+language+models%2E+Trends+in+cognitive+sciences%2C)
35. Jean M. Mandler. 1994. Precursors of linguistic knowledge. Philosophical Transactions: Biological Sciences, 346(1315):63–69. [find](https://scholar.google.com/scholar?q=Jean+M%2E+Mandler%2E+1994%2E+Precursors+of+linguistic+knowledge%2E+Philosophical+Transactions%3A+Biological+Sciences%2C+346%281315%29%3A63%E2%80%9369%2E)
36. Jean M. Mandler and Cristóbal Pagán Cánovas. 2014. On defining image schemas. Language and Cognition, 6(4):510–532. [find](https://scholar.google.com/scholar?q=Jean+M%2E+Mandler+and+Crist%C3%B3bal+Pag%C3%A1n+C%C3%A1novas%2E+2014%2E+On+defining+image+schemas%2E+Language+and+Cognition%2C+6%284%29%3A510%E2%80%93532%2E)
37. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359–17372. [find](https://scholar.google.com/scholar?q=Kevin+Meng%2C+David+Bau%2C+Alex+Andonian%2C+and+Yonatan+Belinkov%2E+2022%2E+Locating+and+editing+factual+associations+in+gpt%2E+Advances+in+neural+information+processing+systems%2C+35%3A17359%E2%80%9317372%2E)
38. Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. 2023. Circuit component reuse across tasks in transformer language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jack+Merullo%2C+Carsten+Eickhoff%2C+and+Ellie+Pavlick%2E+2023%2E+Circuit+component+reuse+across+tasks+in+transformer+language+models%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
39. Raphaël Millière. 2024. Language models as models of language. arXiv preprint arXiv:2408.07144. [find](https://scholar.google.com/scholar?q=Rapha%C3%ABl+Milli%C3%A8re%2E+2024%2E+Language+models+as+models+of+language%2E+arXiv+preprint+arXiv%3A2408%2E07144%2E)
40. Hosein Mohebbi, Willem Zuidema, Grzegorz Chrupała, and Afra Alishahi. 2023. Quantifying context mixing in transformers. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3378–3400, Dubrovnik, Croatia. Association for Computational L… [find](https://scholar.google.com/scholar?q=Hosein+Mohebbi%2C+Willem+Zuidema%2C+Grzegorz+Chrupa%C5%82a%2C+and+Afra+Alishahi%2E+2023%2E+Quantifying+context+mixing+in+transformers%2E+In+Proceedings+of+the+17th+Conference+of+the+European+Chapter+of+the+Association)
41. Aaron Mueller, Jannik Brinkmann, Millicent Li, Samuel Marks, Koyena Pal, Nikhil Prakash, Can Rager, Aruna Sankaranarayanan, Arnab Sen Sharma, Jiuding Sun, et al. 2024. The quest for the right mediator: A history, survey, and theoretical grounding of causal interpretability. arXiv preprint arXiv:2408… — [arXiv:2408.01416](https://arxiv.org/abs/2408.01416) [find](https://scholar.google.com/scholar?q=Aaron+Mueller%2C+Jannik+Brinkmann%2C+Millicent+Li%2C+Samuel+Marks%2C+Koyena+Pal%2C+Nikhil+Prakash%2C+Can+Rager%2C+Aruna+Sankaranarayanan%2C+Arnab+Sen+Sharma%2C+Jiuding+Sun%2C+et+al%2E+2024%2E+The+quest+for+the+right+mediator)
42. Victoria Mussemann. 2024. A topic which i want to know more about – preposition placement in finite wh-relative clauses in world englishes. English Language and Linguistics, 28(2):341–370. nostalgebraist. 2020. interpreting gpt: the logit lens. LessWrong. [find](https://scholar.google.com/scholar?q=Victoria+Mussemann%2E+2024%2E+A+topic+which+i+want+to+know+more+about+%E2%80%93+preposition+placement+in+finite+wh%2Drelative+clauses+in+world+englishes%2E+English+Language+and+Linguistics%2C+28%282%29%3A341%E2%80%93370%2E+nostalgebra)
43. Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. 2020. Zoom in: An introduction to circuits. Distill. Https://distill.pub/2020/circuits/zoom-in. [find](https://scholar.google.com/scholar?q=Chris+Olah%2C+Nick+Cammarata%2C+Ludwig+Schubert%2C+Gabriel+Goh%2C+Michael+Petrov%2C+and+Shan+Carter%2E+2020%2E+Zoom+in%3A+An+introduction+to+circuits%2E+Distill%2E+Https%3A%2F%2Fdistill%2Epub%2F2020%2Fcircuits%2Fzoom%2Din%2E)
44. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. 2022. In-context learning and induction heads. arXiv preprint arXiv:2209.11895. [find](https://scholar.google.com/scholar?q=Catherine+Olsson%2C+Nelson+Elhage%2C+Neel+Nanda%2C+Nicholas+Joseph%2C+Nova+DasSarma%2C+Tom+Henighan%2C+Ben+Mann%2C+Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+et+al%2E+2022%2E+In%2Dcontext+learning+and+induction+heads%2E+arXiv+p)
45. Ellie Pavlick. 2023. Symbols and grounding in large language models. Philosophical Transactions of the Royal Society A, 381(2251):20220041. [find](https://scholar.google.com/scholar?q=Ellie+Pavlick%2E+2023%2E+Symbols+and+grounding+in+large+language+models%2E+Philosophical+Transactions+of+the+Royal+Society+A%2C+381%282251%29%3A20220041%2E)
46. Judea Pearl. 2000. Models, reasoning and inference. Cambridge University Press, Cambridge. [find](https://scholar.google.com/scholar?q=Judea+Pearl%2E+2000%2E+Models%2C+reasoning+and+inference%2E+Cambridge+University+Press%2C+Cambridge%2E)
47. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825– 2830. [find](https://scholar.google.com/scholar?q=Fabian+Pedregosa%2C+Ga%C3%ABl+Varoquaux%2C+Alexandre+Gramfort%2C+Vincent+Michel%2C+Bertrand+Thirion%2C+Olivier+Grisel%2C+Mathieu+Blondel%2C+Peter+Prettenhofer%2C+Ron+Weiss%2C+Vincent+Dubourg%2C+et+al%2E+2011%2E+Scikit%2Dlearn%3A+Mach)
48. Steven Piantadosi and Felix Hill. 2022. Meaning without reference in large language models. In NeurIPS 2022 Workshop on Neuro Causal and Symbolic AI (nCSI). [find](https://scholar.google.com/scholar?q=Steven+Piantadosi+and+Felix+Hill%2E+2022%2E+Meaning+without+reference+in+large+language+models%2E+In+NeurIPS+2022+Workshop+on+Neuro+Causal+and+Symbolic+AI+%28nCSI%29%2E)
49. Steven T Piantadosi. 2023. Modern language models refute chomsky’s approach to language. From fieldwork to linguistic theory: A tribute to Dan Everett, 15:353–414. [find](https://scholar.google.com/scholar?q=Steven+T+Piantadosi%2E+2023%2E+Modern+language+models+refute+chomsky%E2%80%99s+approach+to+language%2E+From+fieldwork+to+linguistic+theory%3A+A+tribute+to+Dan+Everett%2C+15%3A353%E2%80%93414%2E)
50. Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. 2024. A practical review of mechanistic interpretability for transformer-based language models. arXiv preprint arXiv:2407.02646. [find](https://scholar.google.com/scholar?q=Daking+Rai%2C+Yilun+Zhou%2C+Shi+Feng%2C+Abulhair+Saparov%2C+and+Ziyu+Yao%2E+2024%2E+A+practical+review+of+mechanistic+interpretability+for+transformer%2Dbased+language+models%2E+arXiv+preprint+arXiv%3A2407%2E02646%2E)
51. Ivan A. Sag. 2010. English filler-gap constructions. Language, 86(3):486–545. [find](https://scholar.google.com/scholar?q=Ivan+A%2E+Sag%2E+2010%2E+English+filler%2Dgap+constructions%2E+Language%2C+86%283%29%3A486%E2%80%93545%2E)
52. Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, et al. 2025. Open problems in mechanistic interpretability. arXiv preprint arXiv:2501.16496. — [arXiv:2501.16496](https://arxiv.org/abs/2501.16496) [find](https://scholar.google.com/scholar?q=Lee+Sharkey%2C+Bilal+Chughtai%2C+Joshua+Batson%2C+Jack+Lindsey%2C+Jeff+Wu%2C+Lucius+Bushnaq%2C+Nicholas+Goldowsky%2DDill%2C+Stefan+Heimersheim%2C+Alejandro+Ortega%2C+Joseph+Bloom%2C+et+al%2E+2025%2E+Open+problems+in+mechanisti)
53. Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning important features through propagating activation differences. In International conference on machine learning, pages 3145–3153. PMlR. [find](https://scholar.google.com/scholar?q=Avanti+Shrikumar%2C+Peyton+Greenside%2C+and+Anshul+Kundaje%2E+2017%2E+Learning+important+features+through+propagating+activation+differences%2E+In+International+conference+on+machine+learning%2C+pages+3145%E2%80%933153%2E+) [our card](https://md.research.elamur.ai/free/en/papers/shrikumar-2017-learning-important-features-through-propagating/)
54. Shriyank Somvanshi, Md Monzurul Islam, Amir Rafe, Anannya Ghosh Tusti, Arka Chakraborty, Anika Baitullah, Tausif Islam Chowdhury, Nawaf Alnawmasi, Anandi Dutta, and Subasish Das. 2025. Bridging the black box: A survey on mechanistic interpretability in ai. Available at SSRN 5345552. [find](https://scholar.google.com/scholar?q=Shriyank+Somvanshi%2C+Md+Monzurul+Islam%2C+Amir+Rafe%2C+Anannya+Ghosh+Tusti%2C+Arka+Chakraborty%2C+Anika+Baitullah%2C+Tausif+Islam+Chowdhury%2C+Nawaf+Alnawmasi%2C+Anandi+Dutta%2C+and+Subasish+Das%2E+2025%2E+Bridging+the+bl)
55. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319–3328. PMLR. [find](https://scholar.google.com/scholar?q=Mukund+Sundararajan%2C+Ankur+Taly%2C+and+Qiqi+Yan%2E+2017%2E+Axiomatic+attribution+for+deep+networks%2E+In+International+conference+on+machine+learning%2C+pages+3319%E2%80%933328%2E+PMLR%2E)
56. Aaquib Syed, Can Rager, and Arthur Conmy. 2024. Attribution patching outperforms automated circuit discovery. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 407–416, Miami, Florida, US. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aaquib+Syed%2C+Can+Rager%2C+and+Arthur+Conmy%2E+2024%2E+Attribution+patching+outperforms+automated+circuit+discovery%2E+In+Proceedings+of+the+7th+BlackboxNLP+Workshop%3A+Analyzing+and+Interpreting+Neural+Networks)
57. Leonard Talmy. 2000. Toward a cognitive semantics: Concept structuring systems, volume 1. MIT press. [find](https://scholar.google.com/scholar?q=Leonard+Talmy%2E+2000%2E+Toward+a+cognitive+semantics%3A+Concept+structuring+systems%2C+volume+1%2E+MIT+press%2E)
58. Andrea Tyler and Vyvyan Evans. 2003. The Semantics of English Prepositions: Spatial Scenes, Embodied Meaning, and Cognition. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Andrea+Tyler+and+Vyvyan+Evans%2E+2003%2E+The+Semantics+of+English+Prepositions%3A+Spatial+Scenes%2C+Embodied+Meaning%2C+and+Cognition%2E+Cambridge+University+Press%2E)
59. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+Advances+in+neural+information+processing+)
60. Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, and Anna Korhonen. 2020. Probing pretrained language models for lexical semantics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7222–7240, Online. Association for Computational… [find](https://scholar.google.com/scholar?q=Ivan+Vuli%C4%87%2C+Edoardo+Maria+Ponti%2C+Robert+Litschko%2C+Goran+Glava%C5%A1%2C+and+Anna+Korhonen%2E+2020%2E+Probing+pretrained+language+models+for+lexical+semantics%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Me) [our card](https://md.research.elamur.ai/free/en/papers/vulic-2020-probing-pretrained-language-models-lexical/)
61. Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Kevin+Ro+Wang%2C+Alexandre+Variengien%2C+Arthur+Conmy%2C+Buck+Shlegeris%2C+and+Jacob+Steinhardt%2E+2023%2E+Interpretability+in+the+wild%3A+a+circuit+for+indirect+object+identification+in+gpt%2D2+small%2E+In+The+Elevent)
62. Yongjie Wang, Tong Zhang, Xu Guo, and Zhiqi Shen. 2024. Gradient based feature attribution in explainable ai: A technical review. arXiv preprint arXiv:2403.10415. — [arXiv:2403.10415](https://arxiv.org/abs/2403.10415) [find](https://scholar.google.com/scholar?q=Yongjie+Wang%2C+Tong+Zhang%2C+Xu+Guo%2C+and+Zhiqi+Shen%2E+2024%2E+Gradient+based+feature+attribution+in+explainable+ai%3A+A+technical+review%2E+arXiv+preprint+arXiv%3A2403%2E10415%2E+%E2%80%94+%5BarXiv%3A2403%2E10415%5D%28https%3A%2F%2Farxiv%2Eor)
63. Qinan Yu, Jack Merullo, and Ellie Pavlick. 2023. Characterizing mechanisms for factual recall in language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9924–9959. [find](https://scholar.google.com/scholar?q=Qinan+Yu%2C+Jack+Merullo%2C+and+Ellie+Pavlick%2E+2023%2E+Characterizing+mechanisms+for+factual+recall+in+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Process)
64. Fred Zhang and Neel Nanda. 2023. Towards best practices of activation patching in language models: Metrics and methods. arXiv preprint arXiv:2309.16042. [find](https://scholar.google.com/scholar?q=Fred+Zhang+and+Neel+Nanda%2E+2023%2E+Towards+best+practices+of+activation+patching+in+language+models%3A+Metrics+and+methods%2E+arXiv+preprint+arXiv%3A2309%2E16042%2E)
65. Joost Zwarts. 2017. Spatial semantics: Modeling the meaning of prepositions. Language and Linguistics Compass, 11(5):e12241. [find](https://scholar.google.com/scholar?q=Joost+Zwarts%2E+2017%2E+Spatial+semantics%3A+Modeling+the+meaning+of+prepositions%2E+Language+and+Linguistics+Compass%2C+11%285%29%3Ae12241%2E)