---
title: "Construction Grammar Evidence for How LLMs Use Context-Directed Extrapolation — Madabushi 2025"
description: "They develop their earlier framework. 61 references, cited by 1 paper here."
authors: ["Harish Tayyar Madabushi", "Claire Bonial"]
year: 2025
venue: "Proceedings of the Second International Workshop on Construction Grammars and NLP"
source: "https://aclanthology.org/2025.cxgsnlp-1.20/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/madabushi-2025-construction-grammar-evidence-how-llms/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/madabushi-2025-construction-grammar-evidence-how-llms/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / Construction Grammar Evidence for How LLMs Use Context-Directed Extrapolation to Solve Tasks

# Construction Grammar Evidence for How LLMs Use Context-Directed Extrapolation to Solve Tasks

**Authors:** [Harish Tayyar Madabushi](https://md.research.elamur.ai/free/en/authors/harish-tayyar-madabushi/) and [Claire Bonial](https://md.research.elamur.ai/free/en/authors/claire-bonial/) · **Year:** 2025 · **Venue:** Proceedings of the Second International Workshop on Construction Grammars and NLP · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.cxgsnlp-1.20/) · [PDF](https://aclanthology.org/2025.cxgsnlp-1.20.pdf)

**Citations:** 1 in our library, of the 140,884 papers whose bibliographies we read

## Summary
This is a position paper on how large language models (LLMs) solve tasks. The authors seek an explanation between two extremes: models as "stochastic parrots" and models as "sparks of AGI". They develop their earlier framework. According to it, both the successful and the erroneous answers of LLMs arise from extrapolation from patterns learned during pretraining. The direction of this extrapolation is set by the context: in base models, by the ICL examples, and in instruction-tuned models, by the task description itself. The theoretical foundation is Construction Grammar (CxG). In it, a construction is viewed as a class, and its concrete realizations as members of that class. Therefore, genuine generalization means that knowledge transfers to new, previously unseen members. A review of CxG research shows that models can do more than mechanically repeat. For example, they apply the definition of an invented verb in new contexts. At the same time, they make errors where humans find it easy, in particular when several multiword expressions have to be combined. For evaluation, this leads to the following conclusion: LLM results should be compared against how many similar examples were present in the training data. CxG provides tools for building tests outside the distribution of these data.

## Key points
- The paper is a position paper: it proposes no new corpora or models. It builds on the framework from Tayyar Madabushi et al. (2025b) and on a review of work at the intersection of CxG and NLP.
- Examples of the mechanism. From a few examples of "shifted" addition, a base model infers the rule a + b + 1. Base models do not solve logic tasks from Big-Bench without examples, whereas instruction-tuned models solve them from the description alone.
- Mechanistic arguments. The ability to continue the pattern [A][B]…[A] → [B] (Olsson et al., 2022) weakens when the tokens are rare (Niu et al., 2025). ICL works even when the labels are permuted or replaced with meaningless words (Wei et al., 2023). Hence, everything depends on the patterns learned during pretraining.
- Limits of capabilities. When two MWEs interact in a novel usage, human annotators consistently give the correct interpretation, whereas GPT-o1 and GPT-4o make errors. LLMs also systematically fail counterfactual tests.
- Recommendations:
  - test models on examples that are grammatically correct but unlikely in the training data, such as constructions with the same form and different meanings;
  - view LLMs as an intermediary between natural language and formal resources such as AMR and frame semantics;
  - improve models through prompts, fine-tuning and external memory, not only through scaling.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/)

## Cite

Cite as: Harish Tayyar Madabushi and Claire Bonial. 2025. Construction Grammar Evidence for How LLMs Use Context-Directed Extrapolation to Solve Tasks. Proceedings of the Second International Workshop on Construction Grammars and NLP.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{madabushi2025construction,
  title = {{Construction Grammar Evidence for How LLMs Use Context-Directed Extrapolation to Solve Tasks}},
  author = {Harish Tayyar Madabushi and Claire Bonial},
  booktitle = {Proceedings of the Second International Workshop on Construction Grammars and NLP},
  year = {2025},
  url = {https://aclanthology.org/2025.cxgsnlp-1.20/}
}
```

- [← previous Comparative constructions in Bisha Colloquial Arabic: a case study](https://md.research.elamur.ai/free/en/papers/alharthy-2025-comparative-constructions-bisha-colloquial-arabic/)
- [next → Constructions All the Way Up: From Sensory Experiences to Construction Grammars](https://md.research.elamur.ai/free/en/papers/ekila-2025-constructions-all-way-up-sensory/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/)

## References

61 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. 2023. What learning algorithm is in-context learning? investigations with linear models. [our card](https://md.research.elamur.ai/free/en/papers/akyurek-2022-learning-algorithm-context-learning-investigations/)
2. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract meaning representation for sembanking. In Proceedings of the 7th linguistic annotation workshop and interoperability with discou… [our card](https://md.research.elamur.ai/free/en/papers/banarescu-2013-abstract-meaning-representation-sembanking/)
3. Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020a. Language models are few-shot learners. In Advances in Neu… [our card](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
4. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 610–623, New York, NY, USA. Associat… [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+2021%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+Conference)
5. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… [our card](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
6. Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185–5198, Online. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/bender-2020-climbing-nlu-meaning-form-understanding/)
7. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. [our card](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/)
8. Irina Bigoulaeva, Harish Tayyar Madabushi, and Iryna Gurevych. 2025. The inherent limits of pretrained llms: The unexpected convergence of instruction tuning and in-context learning capabilities. [find](https://scholar.google.com/scholar?q=Irina+Bigoulaeva%2C+Harish+Tayyar+Madabushi%2C+and+Iryna+Gurevych%2E+2025%2E+The+inherent+limits+of+pretrained+llms%3A+The+unexpected+convergence+of+instruction+tuning+and+in%2Dcontext+learning+capabilities%2E)
9. Stephanie Chan, Adam Santoro, Andrew Lampinen, Jane Wang, Aaditya Singh, Pierre Richemond, James McClelland, and Felix Hill. 2022. Data distributional properties drive emergent in-context learning in transformers. Advances in neural information processing systems, 35:18878–18891. [find](https://scholar.google.com/scholar?q=Stephanie+Chan%2C+Adam+Santoro%2C+Andrew+Lampinen%2C+Jane+Wang%2C+Aaditya+Singh%2C+Pierre+Richemond%2C+James+McClelland%2C+and+Felix+Hill%2E+2022%2E+Data+distributional+properties+drive+emergent+in%2Dcontext+learning+in+)
10. Benjamin S Bloom et al. 1956. Taxonomy of. Educational Objectives. [find](https://scholar.google.com/scholar?q=Benjamin+S+Bloom+et+al%2E+1956%2E+Taxonomy+of%2E+Educational+Objectives%2E)
11. Claire Bonial, Bianca Badarau, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Tim O’Gorman, Martha Palmer, and Nathan Schneider. 2018. Abstract meaning representation of constructions: The more we include, the better the representation. In Proceedings of the eleventh international conference on languag… [find](https://scholar.google.com/scholar?q=Claire+Bonial%2C+Bianca+Badarau%2C+Kira+Griffitt%2C+Ulf+Hermjakob%2C+Kevin+Knight%2C+Tim+O%E2%80%99Gorman%2C+Martha+Palmer%2C+and+Nathan+Schneider%2E+2018%2E+Abstract+meaning+representation+of+constructions%3A+The+more+we+includ)
12. Claire Bonial, Julia Bonn, and Harish Tayyar Madabushi. 2025. Dancing with deer: A constructional perspective on mwes in the era of llms. [find](https://scholar.google.com/scholar?q=Claire+Bonial%2C+Julia+Bonn%2C+and+Harish+Tayyar+Madabushi%2E+2025%2E+Dancing+with+deer%3A+A+constructional+perspective+on+mwes+in+the+era+of+llms%2E)
13. Claire Bonial and Harish Tayyar Madabushi. 2024a. Constructing understanding: on the constructional information encoded in large language models. Language Resources and Evaluation, pages 1–40. [find](https://scholar.google.com/scholar?q=Claire+Bonial+and+Harish+Tayyar+Madabushi%2E+2024a%2E+Constructing+understanding%3A+on+the+constructional+information+encoded+in+large+language+models%2E+Language+Resources+and+Evaluation%2C+pages+1%E2%80%9340%2E)
14. Claire Bonial and Harish Tayyar Madabushi. 2024b. A construction grammar corpus of varying schematicity: A dataset for the evaluation of abstractions in language models. In Proceedings of the Language Resources and Evaluation Conference (LREC). [our card](https://md.research.elamur.ai/free/en/papers/bonial-2024-construction-grammar-corpus-varying-schematicity/)
15. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, François Chollet. 2019. On the measure of intelligence. arXiv p… [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretchen)
16. Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. 2023. Why can GPT learn in-context? language models secretly perform gradient descent as meta-optimizers. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4005–4019, Toronto, Canada. Associat… [our card](https://md.research.elamur.ai/free/en/papers/dai-2023-why-can-gpt-learn-context/)
17. Jonathan Dunn. 2017. Computational learning of construction grammars. Language and cognition, 9(2):254–292. [find](https://scholar.google.com/scholar?q=Jonathan+Dunn%2E+2017%2E+Computational+learning+of+construction+grammars%2E+Language+and+cognition%2C+9%282%29%3A254%E2%80%93292%2E)
18. Julian Martin Eisenschlos, Jeremy R. Cole, Fangyu Liu, and William W. Cohen. 2023. WinoDict: Probing language models for in-context word acquisition. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 94–102, Dubrovnik, Croatia. Asso… [find](https://scholar.google.com/scholar?q=Julian+Martin+Eisenschlos%2C+Jeremy+R%2E+Cole%2C+Fangyu+Liu%2C+and+William+W%2E+Cohen%2E+2023%2E+WinoDict%3A+Probing+language+models+for+in%2Dcontext+word+acquisition%2E+In+Proceedings+of+the+17th+Conference+of+the+Europ)
19. Charles J. Fillmore, Russell Lee-Goldman, and Russell Rhomieux. 2012. The framenet constructicon. In Hans C. Boas and Ivan A. Sag, editors, Sign-Based Construction Grammar, number 193 in CSLI Lecture Notes, pages 309–372. CSLI Publications, Stanford, CA. [find](https://scholar.google.com/scholar?q=Charles+J%2E+Fillmore%2C+Russell+Lee%2DGoldman%2C+and+Russell+Rhomieux%2E+2012%2E+The+framenet+constructicon%2E+In+Hans+C%2E+Boas+and+Ivan+A%2E+Sag%2C+editors%2C+Sign%2DBased+Construction+Grammar%2C+number+193+in+CSLI+Lecture+)
20. Adele E Goldberg. 2003. Constructions: A new theoretical approach to language. Trends in cognitive sciences, 7(5):219–224. [find](https://scholar.google.com/scholar?q=Adele+E+Goldberg%2E+2003%2E+Constructions%3A+A+new+theoretical+approach+to+language%2E+Trends+in+cognitive+sciences%2C+7%285%29%3A219%E2%80%93224%2E)
21. Adele Eva Goldberg. 1992. Argument structure constructions. University of California, Berkeley. [find](https://scholar.google.com/scholar?q=Adele+Eva+Goldberg%2E+1992%2E+Argument+structure+constructions%2E+University+of+California%2C+Berkeley%2E)
22. Michael Hahn and Navin Goyal. 2023. A theory of emergent in-context learning as implicit structure induction. arXiv preprint arXiv:2303.07971. — [arXiv:2303.07971](https://arxiv.org/abs/2303.07971) [find](https://scholar.google.com/scholar?q=Michael+Hahn+and+Navin+Goyal%2E+2023%2E+A+theory+of+emergent+in%2Dcontext+learning+as+implicit+structure+induction%2E+arXiv+preprint+arXiv%3A2303%2E07971%2E+%E2%80%94+%5BarXiv%3A2303%2E07971%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2303%2E07971%29)
23. Steve Hanneke, Adam Tauman Kalai, Gautam Kamath, and Christos Tzamos. 2018. Actively avoiding nonsense in generative models. In Proceedings of the 31st Conference On Learning Theory, volume 75 of Proceedings of Machine Learning Research, pages 209–227. PMLR. [find](https://scholar.google.com/scholar?q=Steve+Hanneke%2C+Adam+Tauman+Kalai%2C+Gautam+Kamath%2C+and+Christos+Tzamos%2E+2018%2E+Actively+avoiding+nonsense+in+generative+models%2E+In+Proceedings+of+the+31st+Conference+On+Learning+Theory%2C+volume+75+of+Proc)
24. Thomas Hoffmann and Graeme Trousdale. 2013. The Oxford Handbook of Construction Grammar. Oxford University Press, Oxford. [find](https://scholar.google.com/scholar?q=Thomas+Hoffmann+and+Graeme+Trousdale%2E+2013%2E+The+Oxford+Handbook+of+Construction+Grammar%2E+Oxford+University+Press%2C+Oxford%2E)
25. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2):… [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/)
26. Michal Kosinski. 2024. Evaluating large language models in theory of mind tasks. Proceedings of the National Academy of Sciences, 121(45):e2405460121. [our card](https://md.research.elamur.ai/free/en/papers/kosinski-2023-evaluating-large-language-models-theory/)
27. David R Krathwohl. 2002. A revision of bloom’s taxonomy: An overview. Theory into practice, 41(4):212– 218. [find](https://scholar.google.com/scholar?q=David+R+Krathwohl%2E+2002%2E+A+revision+of+bloom%E2%80%99s+taxonomy%3A+An+overview%2E+Theory+into+practice%2C+41%284%29%3A212%E2%80%93+218%2E)
28. Bai Li, Zining Zhu, Guillaume Thomas, Frank Rudzicz, and Yang Xu. 2022. Neural reality of argument structure constructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7410–7423, Dublin, Ireland. Association for Computatio… [find](https://scholar.google.com/scholar?q=Bai+Li%2C+Zining+Zhu%2C+Guillaume+Thomas%2C+Frank+Rudzicz%2C+and+Yang+Xu%2E+2022%2E+Neural+reality+of+argument+structure+constructions%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computation)
29. Yingcong Li, M. Emrullah Ildiz, Dimitris Papail-iopoulos, and Samet Oymak. 2023a. Transformers as algorithms: generalization and stability in in-context learning. In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org. [find](https://scholar.google.com/scholar?q=Yingcong+Li%2C+M%2E+Emrullah+Ildiz%2C+Dimitris+Papail%2Diopoulos%2C+and+Samet+Oymak%2E+2023a%2E+Transformers+as+algorithms%3A+generalization+and+stability+in+in%2Dcontext+learning%2E+In+Proceedings+of+the+40th+Internatio)
30. Yingcong Li, Muhammed Emrullah Ildiz, Dimitris Pa-pailiopoulos, and Samet Oymak. 2023b. Transformers as algorithms: Generalization and stability in in-context learning. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pa… [find](https://scholar.google.com/scholar?q=Yingcong+Li%2C+Muhammed+Emrullah+Ildiz%2C+Dimitris+Pa%2Dpailiopoulos%2C+and+Samet+Oymak%2E+2023b%2E+Transformers+as+algorithms%3A+Generalization+and+stability+in+in%2Dcontext+learning%2E+In+Proceedings+of+the+40th+Inte)
31. Zhuang Liu, Wayne Lin, Ya Shi, and Jun Zhao. 2021. A robustly optimized bert pre-training approach with post-training. In China National Conference on Chinese Computational Linguistics, pages 471–484. Springer. [our card](https://md.research.elamur.ai/free/en/papers/zhuang-2021-robustly-optimized-bert-pre-training/)
32. Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, and Iryna Gurevych. 2023. Are emergent abilities in large language models just in-context learning? [find](https://scholar.google.com/scholar?q=Sheng+Lu%2C+Irina+Bigoulaeva%2C+Rachneet+Sachdeva%2C+Harish+Tayyar+Madabushi%2C+and+Iryna+Gurevych%2E+2023%2E+Are+emergent+abilities+in+large+language+models+just+in%2Dcontext+learning%3F)
33. Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, and Iryna Gurevych. 2024. Are emergent abilities in large language models just in-context learning? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5098–5… [find](https://scholar.google.com/scholar?q=Sheng+Lu%2C+Irina+Bigoulaeva%2C+Rachneet+Sachdeva%2C+Harish+Tayyar+Madabushi%2C+and+Iryna+Gurevych%2E+2024%2E+Are+emergent+abilities+in+large+language+models+just+in%2Dcontext+learning%3F+In+Proceedings+of+the+62nd+A)
34. Melanie Mitchell and David C. Krakauer. 2023. The debate over understanding in ai’s large language models. Proceedings of the National Academy of Sciences, 120(13):e2215907120. [find](https://scholar.google.com/scholar?q=Melanie+Mitchell+and+David+C%2E+Krakauer%2E+2023%2E+The+debate+over+understanding+in+ai%E2%80%99s+large+language+models%2E+Proceedings+of+the+National+Academy+of+Sciences%2C+120%2813%29%3Ae2215907120%2E)
35. Marianna Nezhurina, Lucia Cipolina-Kun, Mehdi Cherti, and Jenia Jitsev. 2025. Alice in wonderland: Simple tasks showing complete reasoning breakdown in state-of-the-art large language models. [find](https://scholar.google.com/scholar?q=Marianna+Nezhurina%2C+Lucia+Cipolina%2DKun%2C+Mehdi+Cherti%2C+and+Jenia+Jitsev%2E+2025%2E+Alice+in+wonderland%3A+Simple+tasks+showing+complete+reasoning+breakdown+in+state%2Dof%2Dthe%2Dart+large+language+models%2E)
36. Jingcheng Niu, Subhabrata Dutta, Ahmed Elshabrawy, Harish Tayyar Madabushi, and Iryna Gurevych. 2025. Illusion or algorithm? investigating memorization, emergence, and symbolic processing in in-context learning. [find](https://scholar.google.com/scholar?q=Jingcheng+Niu%2C+Subhabrata+Dutta%2C+Ahmed+Elshabrawy%2C+Harish+Tayyar+Madabushi%2C+and+Iryna+Gurevych%2E+2025%2E+Illusion+or+algorithm%3F+investigating+memorization%2C+emergence%2C+and+symbolic+processing+in+in%2Dcontex)
37. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amode… [find](https://scholar.google.com/scholar?q=Catherine+Olsson%2C+Nelson+Elhage%2C+Neel+Nanda%2C+Nicholas+Joseph%2C+Nova+DasSarma%2C+Tom+Henighan%2C+Ben+Mann%2C+Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Tom+Conerly%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Zac+Hatfield%2DDodds%2C+Da)
38. Wesley Scivetti, Tatsuya Aoyama, Ethan Wilcox, and Nathan Schneider. 2025a. Unpacking let alone: Human-scale models generalize to a rare construction in form but not meaning. arXiv preprint arXiv:2506.04408. — [arXiv:2506.04408](https://arxiv.org/abs/2506.04408) [find](https://scholar.google.com/scholar?q=Wesley+Scivetti%2C+Tatsuya+Aoyama%2C+Ethan+Wilcox%2C+and+Nathan+Schneider%2E+2025a%2E+Unpacking+let+alone%3A+Human%2Dscale+models+generalize+to+a+rare+construction+in+form+but+not+meaning%2E+arXiv+preprint+arXiv%3A2506)
39. Wesley Scivetti, Melissa Torgbi, Austin Blodgett, Mollie Shichman, Taylor Hudson, Claire Bonial, and Harish Tayyar Madabushi. 2025b. Assessing language comprehension in large language models using construction grammar. [find](https://scholar.google.com/scholar?q=Wesley+Scivetti%2C+Melissa+Torgbi%2C+Austin+Blodgett%2C+Mollie+Shichman%2C+Taylor+Hudson%2C+Claire+Bonial%2C+and+Harish+Tayyar+Madabushi%2E+2025b%2E+Assessing+language+comprehension+in+large+language+models+using+con)
40. Natalie Shapira, Guy Zwirn, and Yoav Goldberg. 2023. How well do large language models perform on faux pas tests? In Findings of the Association for Computational Linguistics: ACL 2023, pages 10438–10451, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Natalie+Shapira%2C+Guy+Zwirn%2C+and+Yoav+Goldberg%2E+2023%2E+How+well+do+large+language+models+perform+on+faux+pas+tests%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2023%2C+pages+10438%E2%80%9310)
41. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2023a. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning… [our card](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
42. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, et al. 2023b. Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language mode… [our card](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
43. Harish Tayyar Madabushi, Taylor Pellegrin, and Claire Bonial. 2025a. Generative framenet: Scalable and adaptive frames for interpretable knowledge storage and retrieval for llms powered by llms. Bridging Neurons and Symbols for Natural Language Processing and Knowledge Graphs Reasoning@ COLING 2025,… [find](https://scholar.google.com/scholar?q=Harish+Tayyar+Madabushi%2C+Taylor+Pellegrin%2C+and+Claire+Bonial%2E+2025a%2E+Generative+framenet%3A+Scalable+and+adaptive+frames+for+interpretable+knowledge+storage+and+retrieval+for+llms+powered+by+llms%2E+Bridg)
44. Harish Tayyar Madabushi, Laurence Romain, Dagmar Divjak, and Petar Milin. 2020. Cxgbert: Bert meets construction grammar. arXiv preprint arXiv:2011.04134. [our card](https://md.research.elamur.ai/free/en/papers/madabushi-2020-cxgbert-bert-meets-construction-grammar/)
45. Harish Tayyar Madabushi, Melissa Torgbi, and Claire Bonial. 2025b. Neither stochastic parroting nor agi: Llms solve tasks through context-directed extrapolation from training data priors. [find](https://scholar.google.com/scholar?q=Harish+Tayyar+Madabushi%2C+Melissa+Torgbi%2C+and+Claire+Bonial%2E+2025b%2E+Neither+stochastic+parroting+nor+agi%3A+Llms+solve+tasks+through+context%2Ddirected+extrapolation+from+training+data+priors%2E)
46. Michael Tomasello. 2009. The usage-based theory of language acquisition. In The Cambridge handbook of child language, pages 69–87. Cambridge Univ. Press. [find](https://scholar.google.com/scholar?q=Michael+Tomasello%2E+2009%2E+The+usage%2Dbased+theory+of+language+acquisition%2E+In+The+Cambridge+handbook+of+child+language%2C+pages+69%E2%80%9387%2E+Cambridge+Univ%2E+Press%2E)
47. Karthik Valmeekam, Matthew Marquez, Sarath Sreed-haran, and Subbarao Kambhampati. 2023. On the planning abilities of large language models-a critical investigation. Advances in Neural Information Processing Systems, 36:75993–76005. [find](https://scholar.google.com/scholar?q=Karthik+Valmeekam%2C+Matthew+Marquez%2C+Sarath+Sreed%2Dharan%2C+and+Subbarao+Kambhampati%2E+2023%2E+On+the+planning+abilities+of+large+language+models%2Da+critical+investigation%2E+Advances+in+Neural+Information+Proc)
48. Johannes Von Oswald, Eyvind Niklasson, Ettore Ran-dazzo, Joao Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. 2023. Transformers learn in-context by gradient descent. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machin… [our card](https://md.research.elamur.ai/free/en/papers/oswald-2022-transformers-learn-context-gradient-descent/)
49. Alex Warstadt, Yian Zhang, Haau-Sing Li, Haokun Liu, and Samuel R Bowman. 2020. Learning which features matter: Roberta acquires a preference for linguistic generalizations (eventually). arXiv preprint arXiv:2010.05358. [find](https://scholar.google.com/scholar?q=Alex+Warstadt%2C+Yian+Zhang%2C+Haau%2DSing+Li%2C+Haokun+Liu%2C+and+Samuel+R+Bowman%2E+2020%2E+Learning+which+features+matter%3A+Roberta+acquires+a+preference+for+linguistic+generalizations+%28eventually%29%2E+arXiv+preprin)
50. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022a. Finetuned language models are zero-shot learners. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Maarten+Bosma%2C+Vincent+Zhao%2C+Kelvin+Guu%2C+Adams+Wei+Yu%2C+Brian+Lester%2C+Nan+Du%2C+Andrew+M%2E+Dai%2C+and+Quoc+V+Le%2E+2022a%2E+Finetuned+language+models+are+zero%2Dshot+learners%2E+In+International+Conferen)
51. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022b. Emergent abilities of large language models. Transactions on M… [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/)
52. Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, and Tengyu Ma. 2023. Larger language models do in-context learning differently. [our card](https://md.research.elamur.ai/free/en/papers/wei-2023-larger-language-models-context-learning/)
53. Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, and Tengyu Ma. 2024. Larger language models do in-context learning differently. [our card](https://md.research.elamur.ai/free/en/papers/wei-2023-larger-language-models-context-learning/)
54. Leonie Weissweiler, Valentin Hofmann, Abdullatif Köksal, and Hinrich Schütze. 2022a. The better your syntax, the better your semantics? probing pretrained language models for the English comparative correlative. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processin… [find](https://scholar.google.com/scholar?q=Leonie+Weissweiler%2C+Valentin+Hofmann%2C+Abdullatif+K%C3%B6ksal%2C+and+Hinrich+Sch%C3%BCtze%2E+2022a%2E+The+better+your+syntax%2C+the+better+your+semantics%3F+probing+pretrained+language+models+for+the+English+comparative+c)
55. Leonie Weissweiler, Valentin Hofmann, Abdullatif Köksal, and Hinrich Schütze. 2022b. The better your syntax, the better your semantics? probing pretrained language models for the english comparative correlative. arXiv preprint arXiv:2210.13181. [find](https://scholar.google.com/scholar?q=Leonie+Weissweiler%2C+Valentin+Hofmann%2C+Abdullatif+K%C3%B6ksal%2C+and+Hinrich+Sch%C3%BCtze%2E+2022b%2E+The+better+your+syntax%2C+the+better+your+semantics%3F+probing+pretrained+language+models+for+the+english+comparative+c)
56. Heinz Wimmer and Josef Perner. 1983. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception. Cognition, 13(1):103–128. [find](https://scholar.google.com/scholar?q=Heinz+Wimmer+and+Josef+Perner%2E+1983%2E+Beliefs+about+beliefs%3A+Representation+and+constraining+function+of+wrong+beliefs+in+young+children%E2%80%99s+understanding+of+deception%2E+Cognition%2C+13%281%29%3A103%E2%80%93128%2E)
57. Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. 2024. Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks. In Proceedings of the 2024 Conference of the North American… [find](https://scholar.google.com/scholar?q=Zhaofeng+Wu%2C+Linlu+Qiu%2C+Alexis+Ross%2C+Ekin+Aky%C3%BCrek%2C+Boyuan+Chen%2C+Bailin+Wang%2C+Najoung+Kim%2C+Jacob+Andreas%2C+and+Yoon+Kim%2E+2024%2E+Reasoning+or+reciting%3F+exploring+the+capabilities+and+limitations+of+langua)
58. Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2021. An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080. [our card](https://md.research.elamur.ai/free/en/papers/xie-2021-explanation-context-learning-implicit-bayesian/)
59. Ruiqi Zhang, Spencer Frei, and Peter L. Bartlett. 2024. Trained transformers learn linear models in-context. Journal of Machine Learning Research, 25(49):1–55. [find](https://scholar.google.com/scholar?q=Ruiqi+Zhang%2C+Spencer+Frei%2C+and+Peter+L%2E+Bartlett%2E+2024%2E+Trained+transformers+learn+linear+models+in%2Dcontext%2E+Journal+of+Machine+Learning+Research%2C+25%2849%29%3A1%E2%80%9355%2E)
60. Yufeng Zhang, Fengzhuo Zhang, Zhuoran Yang, and Zhaoran Wang. 2023. What and how does in-context learning learn? bayesian model averaging, parameterization, and generalization. [find](https://scholar.google.com/scholar?q=Yufeng+Zhang%2C+Fengzhuo+Zhang%2C+Zhuoran+Yang%2C+and+Zhaoran+Wang%2E+2023%2E+What+and+how+does+in%2Dcontext+learning+learn%3F+bayesian+model+averaging%2C+parameterization%2C+and+generalization%2E)
61. Shijia Zhou, Leonie Weissweiler, Taiqi He, Hinrich Schütze, David R. Mortensen, and Lori Levin. 2024. Constructions Are So Difficult That Even Large Language Models Get Them Right for the Wrong Reasons. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language… [find](https://scholar.google.com/scholar?q=Shijia+Zhou%2C+Leonie+Weissweiler%2C+Taiqi+He%2C+Hinrich+Sch%C3%BCtze%2C+David+R%2E+Mortensen%2C+and+Lori+Levin%2E+2024%2E+Constructions+Are+So+Difficult+That+Even+Large+Language+Models+Get+Them+Right+for+the+Wrong+Reason)