---
title: "Are Frequent Phrases Directly Retrieved like — Rambelli 2023"
description: "The paper asks whether frequent but fully compositional word sequences are handled like idioms in comprehension, meaning retrieved as stored units rather…"
authors: ["Giulia Rambelli", "Emmanuele Chersoni", "Marco S. G. Senaldi", "Philippe Blache", "Alessandro Lenci"]
year: 2023
venue: "Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023)"
source: "https://aclanthology.org/2023.mwe-1.13/"
doi: "10.18653/v1/2023.mwe-1.13"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/rambelli-2023-are-frequent-phrases-directly-retrieved/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/rambelli-2023-are-frequent-phrases-directly-retrieved/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2023](https://md.research.elamur.ai/free/papers/2023/) / Are Frequent Phrases Directly Retrieved like Idioms? An Investigation with Self-Paced Reading and Language Models

# Are Frequent Phrases Directly Retrieved like Idioms? An Investigation with Self-Paced Reading and Language Models

**Authors:** Giulia Rambelli, [Emmanuele Chersoni](https://md.research.elamur.ai/free/authors/emmanuele-chersoni/), Marco S. G. Senaldi, Philippe Blache, [Alessandro Lenci](https://md.research.elamur.ai/free/authors/alessandro-lenci/) · **Year:** 2023 · **Venue:** Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.mwe-1.13/) · [PDF](https://aclanthology.org/2023.mwe-1.13.pdf) · [DOI 10.18653/v1/2023.mwe-1.13](https://doi.org/10.18653/v1/2023.mwe-1.13) · [OpenAlex](https://openalex.org/W4376652178)

**Citations:** 5 citations (openalex-snapshot · 2026-10-14) · [6](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper asks whether frequent but fully compositional word sequences are handled like idioms in comprehension, meaning retrieved as stored units rather than assembled word by word. The authors built 48 English verb-determiner-noun items. Each item appears in three versions sharing the same verb: idiomatic (ID, *spill the beans*), high-frequency compositional (HF, *spill the milk*) and low-frequency compositional (LF, *spill the rice*). HF and LF variants were chosen with frequency and LogDice association queries on enTenTen18. Nouns were balanced for frequency and length. Each stimulus followed a context sentence.

Human reading times were collected in a self-paced reading study. The same materials were then scored with Surprisal from several RNN and Transformer language models. Humans read ID and HF phrases at similar speeds, and both faster than LF phrases. The models mirrored this by giving LF items higher Surprisal. The closest match to human behaviour came from the smallest models, consistent with reports of inverse scaling.

The results support usage-based accounts in which frequent multiword sequences are processed more holistically than compositionally.

## Key points
- Stimuli: 48 triplets (ID / HF / LF). The idioms were sourced from Kyriacou et al. (2020) and from two normative collections, those of Bulkes and Tanner (2017) and of Libben and Titone (2008). They were then screened using an Idiom Familiarity survey and a Typical Objects Production study.
- Matched compositional phrases were drawn from enTenTen18 (about 21.9 billion words) using SketchEngine. The LogDice score controlled verb-noun association.
- Models tested:
  - GPT2 in four sizes (base, medium, large, xl)
  - BERT-base-uncased
  - T5-base
  - tinyLSTM
  - GRNN
- The small GPT2 and a 2-layer LSTM reproduced the human pattern exactly: no significant ID–HF difference, and significantly higher Surprisal for LF.
- Removing the preceding context sentence raised GPT2 Surprisal overall, but the ordering of conditions stayed the same. A Context × Condition interaction appeared only for GPT2-xl (F = 106.49). Data and code are shared on OSF.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 5 not yet.

- [Semantics of Multiword Expressions in Transformer-Based Models: A Survey](https://md.research.elamur.ai/free/papers/miletic-2024-semantics-multiword-expressions-transformer-based/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rambelli2023are,
  title = {{Are Frequent Phrases Directly Retrieved like Idioms? An Investigation with Self-Paced Reading and Language Models}},
  author = {Rambelli, Giulia and Chersoni, Emmanuele and Senaldi, Marco S. G. and Blache, Philippe and Lenci, Alessandro},
  booktitle = {Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023)},
  year = {2023},
  doi = {10.18653/v1/2023.mwe-1.13},
  url = {https://doi.org/10.18653/v1/2023.mwe-1.13}
}
```

- [← previous Are Emergent Abilities of Large Language Models a Mirage?](https://md.research.elamur.ai/free/papers/schaeffer-2023-emergent-abilities-large-language-models/)
- [next → Are idioms surprising?](https://md.research.elamur.ai/free/papers/philipp-2023-are-idioms-surprising/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/papers/2023/)

## References

72 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Verna Dankers, Christopher G Lucas, and Ivan Titov. 2022. Can Transformer be too Compositional? Analysing Idiom Processing in Neural Machine Translation. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Verna+Dankers%2C+Christopher+G+Lucas%2C+and+Ivan+Titov%2E+2022%2E+Can+Transformer+be+too+Compositional%3F+Analysing+Idiom+Processing+in+Neural+Machine+Translation%2E+In+Proceedings+of+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/dankers-2022-can-transformer-be-too-compositional/)
2. Ben Ambridge. 2020. Against Stored Abstractions: A Radical Exemplar Model of Language Acquisition. First Language, 40(5-6):509–559. [find](https://scholar.google.com/scholar?q=Ben+Ambridge%2E+2020%2E+Against+Stored+Abstractions%3A+A+Radical+Exemplar+Model+of+Language+Acquisition%2E+First+Language%2C+40%285%2D6%29%3A509%E2%80%93559%2E)
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of NAACL. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+Deep+Bidirectional+Transformers+for+Language+Understanding%2E+In+Proceedings+of+NAACL%2E)
4. Inbal Arnon and Neal Snider. 2010. More than Words: Frequency Effects for Multi-word Phrases. Journal of Memory and Language, 62(1):67–82. [find](https://scholar.google.com/scholar?q=Inbal+Arnon+and+Neal+Snider%2E+2010%2E+More+than+Words%3A+Frequency+Effects+for+Multi%2Dword+Phrases%2E+Journal+of+Memory+and+Language%2C+62%281%29%3A67%E2%80%9382%2E)
5. Jeffrey L Elman. 1990. Finding structure in time. Cognitive science, 14(2):179–211. [find](https://scholar.google.com/scholar?q=Jeffrey+L+Elman%2E+1990%2E+Finding+structure+in+time%2E+Cognitive+science%2C+14%282%29%3A179%E2%80%93211%2E)
6. Colin Bannard and Danielle Matthews. 2008. Stored Word Sequences in Language Learning: The Effect of Familiarity on Children’s Repetition of Four-word Combinations. Psychological Science, 19(3). [find](https://scholar.google.com/scholar?q=Colin+Bannard+and+Danielle+Matthews%2E+2008%2E+Stored+Word+Sequences+in+Language+Learning%3A+The+Effect+of+Familiarity+on+Children%E2%80%99s+Repetition+of+Four%2Dword+Combinations%2E+Psychological+Science%2C+19%283%29%2E)
7. Richard Futrell, Ethan Wilcox, Takashi Morita, and Roger Levy. 2018. RNNs as Psycholinguistic Subjects: Syntactic State and Grammatical Dependency. arXiv preprint arXiv:1809.01329. [find](https://scholar.google.com/scholar?q=Richard+Futrell%2C+Ethan+Wilcox%2C+Takashi+Morita%2C+and+Roger+Levy%2E+2018%2E+RNNs+as+Psycholinguistic+Subjects%3A+Syntactic+State+and+Grammatical+Dependency%2E+arXiv+preprint+arXiv%3A1809%2E01329%2E)
8. Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1):1– 48. [find](https://scholar.google.com/scholar?q=Douglas+Bates%2C+Martin+M%C3%A4chler%2C+Ben+Bolker%2C+and+Steve+Walker%2E+2015%2E+Fitting+Linear+Mixed%2DEffects+Models+Using+lme4%2E+Journal+of+Statistical+Software%2C+67%281%29%3A1%E2%80%93+48%2E) [our card](https://md.research.elamur.ai/free/papers/bates-2015-fitting-linear-mixed-effects-models/)
9. Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2021a. Assessing the Representations of Idiomaticity in Vector Models with a Noun Compound Dataset Labeled at Type and Token Levels. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Marcos+Garcia%2C+Tiago+Kramer+Vieira%2C+Carolina+Scarton%2C+Marco+Idiart%2C+and+Aline+Villavicencio%2E+2021a%2E+Assessing+the+Representations+of+Idiomaticity+in+Vector+Models+with+a+Noun+Compound+Dataset+Labeled+)
10. Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. 2023. Eliciting Latent Predictions from Transformers with the Tuned Lens. arXiv preprint arXiv:2303.08112. [find](https://scholar.google.com/scholar?q=Nora+Belrose%2C+Zach+Furman%2C+Logan+Smith%2C+Danny+Halawi%2C+Igor+Ostrovsky%2C+Lev+McKinney%2C+Stella+Biderman%2C+and+Jacob+Steinhardt%2E+2023%2E+Eliciting+Latent+Predictions+from+Transformers+with+the+Tuned+Lens%2E+arX) [our card](https://md.research.elamur.ai/free/papers/belrose-2023-eliciting-latent-predictions-transformers-tuned/)
11. Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2021b. Probing for Idiomaticity in Vector Space Models. In Proceedings of EACL. [find](https://scholar.google.com/scholar?q=Marcos+Garcia%2C+Tiago+Kramer+Vieira%2C+Carolina+Scarton%2C+Marco+Idiart%2C+and+Aline+Villavicencio%2E+2021b%2E+Probing+for+Idiomaticity+in+Vector+Space+Models%2E+In+Proceedings+of+EACL%2E) [our card](https://md.research.elamur.ai/free/papers/garcia-2021-probing-idiomaticity-vector-space-models/)
12. Douglas Biber, Stig Johansson, Geoffrey Leech, Susan Conrad, and Edward Finegan. 2000. Longman Grammar of Spoken and Written English. Longman London. [find](https://scholar.google.com/scholar?q=Douglas+Biber%2C+Stig+Johansson%2C+Geoffrey+Leech%2C+Susan+Conrad%2C+and+Edward+Finegan%2E+2000%2E+Longman+Grammar+of+Spoken+and+Written+English%2E+Longman+London%2E)
13. Jon Gauthier, Jennifer Hu, Ethan Wilcox, Peng Qian, and Roger Levy. 2020. SyntaxGym: An Online Platform for Targeted Evaluation of Language Models. In Proceedings of ACL: Demo. [find](https://scholar.google.com/scholar?q=Jon+Gauthier%2C+Jennifer+Hu%2C+Ethan+Wilcox%2C+Peng+Qian%2C+and+Roger+Levy%2E+2020%2E+SyntaxGym%3A+An+Online+Platform+for+Targeted+Evaluation+of+Language+Models%2E+In+Proceedings+of+ACL%3A+Demo%2E)
14. Nyssa Z. Bulkes and Darren Tanner. 2017. “Going to town”: Large-scale Norming and Statistical Analysis of 870 American English Idioms. Behavior Research Methods, 49(2):772–783. [find](https://scholar.google.com/scholar?q=Nyssa+Z%2E+Bulkes+and+Darren+Tanner%2E+2017%2E+%E2%80%9CGoing+to+town%E2%80%9D%3A+Large%2Dscale+Norming+and+Statistical+Analysis+of+870+American+English+Idioms%2E+Behavior+Research+Methods%2C+49%282%29%3A772%E2%80%93783%2E)
15. Adele E Goldberg. 2003. Constructions: A New Theoretical Approach to Language. Trends in Cognitive Sciences, 7(5):219–224. [find](https://scholar.google.com/scholar?q=Adele+E+Goldberg%2E+2003%2E+Constructions%3A+A+New+Theoretical+Approach+to+Language%2E+Trends+in+Cognitive+Sciences%2C+7%285%29%3A219%E2%80%93224%2E)
16. Joan Bybee. 2006. From Usage to Grammar: The Mind’s Response to Repetition. Language, pages 711–733. [find](https://scholar.google.com/scholar?q=Joan+Bybee%2E+2006%2E+From+Usage+to+Grammar%3A+The+Mind%E2%80%99s+Response+to+Repetition%2E+Language%2C+pages+711%E2%80%93733%2E)
17. Adele E. Goldberg. 2006. Constructions at Work: The Nature of Generalization in Language. Oxford University Press on Demand. [find](https://scholar.google.com/scholar?q=Adele+E%2E+Goldberg%2E+2006%2E+Constructions+at+Work%3A+The+Nature+of+Generalization+in+Language%2E+Oxford+University+Press+on+Demand%2E)
18. Joan L. Bybee. 2010. Language, Usage and Cognition. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Joan+L%2E+Bybee%2E+2010%2E+Language%2C+Usage+and+Cognition%2E+Cambridge+University+Press%2E)
19. Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. Colorless Green Recurrent Networks Dream Hierarchically. In Proceedings of NAACL. [find](https://scholar.google.com/scholar?q=Kristina+Gulordava%2C+Piotr+Bojanowski%2C+Edouard+Grave%2C+Tal+Linzen%2C+and+Marco+Baroni%2E+2018%2E+Colorless+Green+Recurrent+Networks+Dream+Hierarchically%2E+In+Proceedings+of+NAACL%2E) [our card](https://md.research.elamur.ai/free/papers/gulordava-2018-colorless-green-recurrent-networks-dream/)
20. Cristina Cacciari and Patrizia Tabossi. 1988. The Comprehension of Idioms. Journal of Memory and Language, 27(6):668–683. [find](https://scholar.google.com/scholar?q=Cristina+Cacciari+and+Patrizia+Tabossi%2E+1988%2E+The+Comprehension+of+Idioms%2E+Journal+of+Memory+and+Language%2C+27%286%29%3A668%E2%80%93683%2E)
21. John Hale. 2001. A Probabilistic Earley Parser as a Psycholinguistic Model. In Proceedings of NAACL. [find](https://scholar.google.com/scholar?q=John+Hale%2E+2001%2E+A+Probabilistic+Earley+Parser+as+a+Psycholinguistic+Model%2E+In+Proceedings+of+NAACL%2E) [our card](https://md.research.elamur.ai/free/papers/hale-2001-probabilistic-earley-parser-as-psycholinguistic/)
22. Won Ik Cho, Emmanuele Chersoni, Yu-Yin Hsu, and Chu-Ren Huang. 2021. Modeling the Influence of Verb Aspect on the Activation of Typical Event Locations with BERT. In Findings of ACL-IJCNLP. [find](https://scholar.google.com/scholar?q=Won+Ik+Cho%2C+Emmanuele+Chersoni%2C+Yu%2DYin+Hsu%2C+and+Chu%2DRen+Huang%2E+2021%2E+Modeling+the+Influence+of+Verb+Aspect+on+the+Activation+of+Typical+Event+Locations+with+BERT%2E+In+Findings+of+ACL%2DIJCNLP%2E)
23. Ray Jackendoff. 2002. Foundations of Language. Oxford University Press. [find](https://scholar.google.com/scholar?q=Ray+Jackendoff%2E+2002%2E+Foundations+of+Language%2E+Oxford+University+Press%2E)
24. Miloš Jakubíček, Adam Kilgarriff, Vojtěch Kovář, Pavel Rychlý, and Vít Suchomel. 2013. The tenten corpus family. In 7th International Corpus Linguistics Conference CL, pages 125–127. Lancaster University. [find](https://scholar.google.com/scholar?q=Milo%C5%A1+Jakub%C3%ADc%CC%8Cek%2C+Adam+Kilgarriff%2C+Vojte%CC%8Cch+Kov%C3%A1r%CC%8C%2C+Pavel+Rychl%C3%BD%2C+and+V%C3%ADt+Suchomel%2E+2013%2E+The+tenten+corpus+family%2E+In+7th+International+Corpus+Linguistics+Conference+CL%2C+pages+125%E2%80%93127%2E+Lancaster+Univ)
25. Noam Chomsky. 1993. A Minimalist Program for Linguistic Theory. The View from Building 20: Essays in Linguistics in Honor of Sylvain Bromberger. [find](https://scholar.google.com/scholar?q=Noam+Chomsky%2E+1993%2E+A+Minimalist+Program+for+Linguistic+Theory%2E+The+View+from+Building+20%3A+Essays+in+Linguistics+in+Honor+of+Sylvain+Bromberger%2E)
26. Kathy Conklin and Norbert Schmitt. 2008. Formulaic Sequences: Are They Processed More Quickly than Nonformulaic Language by Native and Nonnative Speakers? Applied Linguistics, 29(1):72–89. [find](https://scholar.google.com/scholar?q=Kathy+Conklin+and+Norbert+Schmitt%2E+2008%2E+Formulaic+Sequences%3A+Are+They+Processed+More+Quickly+than+Nonformulaic+Language+by+Native+and+Nonnative+Speakers%3F+Applied+Linguistics%2C+29%281%29%3A72%E2%80%9389%2E)
27. Joel Jang, Seonghyeon Ye, and Minjoon Seo. 2023. Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts. In Transfer Learning for Natural Language Processing Workshop, pages 52–62. PMLR. [find](https://scholar.google.com/scholar?q=Joel+Jang%2C+Seonghyeon+Ye%2C+and+Minjoon+Seo%2E+2023%2E+Can+Large+Language+Models+Truly+Understand+Prompts%3F+A+Case+Study+with+Negated+Prompts%2E+In+Transfer+Learning+for+Natural+Language+Processing+Workshop%2C+p)
28. Pablo Contreras Kallens and Morten H Christiansen. 2022. Models of Language and Multiword Expressions. Frontiers in Artificial Intelligence, 5:24. [find](https://scholar.google.com/scholar?q=Pablo+Contreras+Kallens+and+Morten+H+Christiansen%2E+2022%2E+Models+of+Language+and+Multiword+Expressions%2E+Frontiers+in+Artificial+Intelligence%2C+5%3A24%2E)
29. Jill Jegerski. 2013. Self-paced Reading. In Research Methods in Second Language Psycholinguistics, pages 36–65. Routledge. [find](https://scholar.google.com/scholar?q=Jill+Jegerski%2E+2013%2E+Self%2Dpaced+Reading%2E+In+Research+Methods+in+Second+Language+Psycholinguistics%2C+pages+36%E2%80%9365%2E+Routledge%2E)
30. Hajnal Jolsvai, Stewart M McCauley, and Morten H Christiansen. 2020. Meaningfulness Beats Frequency in Multiword Chunk Processing. Cognitive Science, 44(10):e12885. [find](https://scholar.google.com/scholar?q=Hajnal+Jolsvai%2C+Stewart+M+McCauley%2C+and+Morten+H+Christiansen%2E+2020%2E+Meaningfulness+Beats+Frequency+in+Multiword+Chunk+Processing%2E+Cognitive+Science%2C+44%2810%29%3Ae12885%2E)
31. Byung-Doh Oh and William Schuler. 2022. Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times? In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Byung%2DDoh+Oh+and+William+Schuler%2E+2022%2E+Why+Does+Surprisal+From+Larger+Transformer%2DBased+Language+Models+Provide+a+Poorer+Fit+to+Human+Reading+Times%3F+In+Proceedings+of+EMNLP%2E)
32. Adam Kilgarriff, Vít Baisa, Jan Bušta, Miloš Jakubíček, Vojtěch Kovář, Jan Michelfeit, Pavel Rychlý, and Vít Suchomel. 2014. The Sketch Engine: Ten Years On. Lexicography, 1(1):7–36. [find](https://scholar.google.com/scholar?q=Adam+Kilgarriff%2C+V%C3%ADt+Baisa%2C+Jan+Bu%C5%A1ta%2C+Milo%C5%A1+Jakub%C3%ADc%CC%8Cek%2C+Vojte%CC%8Cch+Kov%C3%A1r%CC%8C%2C+Jan+Michelfeit%2C+Pavel+Rychl%C3%BD%2C+and+V%C3%ADt+Suchomel%2E+2014%2E+The+Sketch+Engine%3A+Ten+Years+On%2E+Lexicography%2C+1%281%29%3A7%E2%80%9336%2E)
33. Ludovica Pannitto and Aurélie Herbelot. 2023. CALaMo: A Constructionist Assessment of Language Models. arXiv preprint arXiv:2302.03589. [find](https://scholar.google.com/scholar?q=Ludovica+Pannitto+and+Aur%C3%A9lie+Herbelot%2E+2023%2E+CALaMo%3A+A+Constructionist+Assessment+of+Language+Models%2E+arXiv+preprint+arXiv%3A2302%2E03589%2E)
34. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language Models Are Unsupervised Multitask Learners. OpenAI Blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+2019%2E+Language+Models+Are+Unsupervised+Multitask+Learners%2E+OpenAI+Blog%2C+1%288%29%3A9%2E)
35. Alexandra Kuznetsova, Per B. Brockhoff, and Rune HB Christensen. 2017. lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software, 82(1):1–26. [find](https://scholar.google.com/scholar?q=Alexandra+Kuznetsova%2C+Per+B%2E+Brockhoff%2C+and+Rune+HB+Christensen%2E+2017%2E+lmerTest+Package%3A+Tests+in+Linear+Mixed+Effects+Models%2E+Journal+of+Statistical+Software%2C+82%281%29%3A1%E2%80%9326%2E)
36. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-text Transformer. Journal of Machine Learning Research, 21(1):5485–5551. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+and+Peter+J+Liu%2E+2020%2E+Exploring+the+Limits+of+Transfer+Learning+with+a+Unified+Text%2Dto%2Dtext)
37. Marianna Kyriacou, Kathy Conklin, and Dominic Thompson. 2020. Passivizability of Idioms: Has the Wrong Tree Been Barked Up? Language and speech, 63(2):404–435. [find](https://scholar.google.com/scholar?q=Marianna+Kyriacou%2C+Kathy+Conklin%2C+and+Dominic+Thompson%2E+2020%2E+Passivizability+of+Idioms%3A+Has+the+Wrong+Tree+Been+Barked+Up%3F+Language+and+speech%2C+63%282%29%3A404%E2%80%93435%2E)
38. Roger Levy. 2008. Expectation-based Syntactic Comprehension. Cognition, 106(3):1126–1177. [find](https://scholar.google.com/scholar?q=Roger+Levy%2E+2008%2E+Expectation%2Dbased+Syntactic+Comprehension%2E+Cognition%2C+106%283%29%3A1126%E2%80%931177%2E)
39. Giulia Rambelli, Emmanuele Chersoni, Philippe Blache, Chu-Ren Huang, and Alessandro Lenci. 2019. Distributional Semantics Meets Construction Grammar. Towards a Unified Usage-based Model of Grammar and Meaning. In Proceedings of the ACL Workshop on Designing Meaning Representations. [find](https://scholar.google.com/scholar?q=Giulia+Rambelli%2C+Emmanuele+Chersoni%2C+Philippe+Blache%2C+Chu%2DRen+Huang%2C+and+Alessandro+Lenci%2E+2019%2E+Distributional+Semantics+Meets+Construction+Grammar%2E+Towards+a+Unified+Usage%2Dbased+Model+of+Grammar+and)
40. Bai Li, Zining Zhu, Guillaume Thomas, Frank Rudzicz, and Yang Xu. 2022. Neural Reality of Argument Structure Constructions. In Proceedngs of ACL. [find](https://scholar.google.com/scholar?q=Bai+Li%2C+Zining+Zhu%2C+Guillaume+Thomas%2C+Frank+Rudzicz%2C+and+Yang+Xu%2E+2022%2E+Neural+Reality+of+Argument+Structure+Constructions%2E+In+Proceedngs+of+ACL%2E)
41. Maya R. Libben and Debra A. Titone. 2008. The Multi-determined Nature of Idiom Processing. Memory & Cognition, 36(6):1103–1121. [find](https://scholar.google.com/scholar?q=Maya+R%2E+Libben+and+Debra+A%2E+Titone%2E+2008%2E+The+Multi%2Ddetermined+Nature+of+Idiom+Processing%2E+Memory+%26+Cognition%2C+36%286%29%3A1103%E2%80%931121%2E)
42. Giulia Rambelli, Emmanuele Chersoni, Philippe Blache, and Alessandro Lenci. 2022. Compositionality as an Analogical Process: Introducing ANNE. In Proceedings of the AACL-IJCNLP Workshop on Cognitive Aspects of the Lexicon. [find](https://scholar.google.com/scholar?q=Giulia+Rambelli%2C+Emmanuele+Chersoni%2C+Philippe+Blache%2C+and+Alessandro+Lenci%2E+2022%2E+Compositionality+as+an+Analogical+Process%3A+Introducing+ANNE%2E+In+Proceedings+of+the+AACL%2DIJCNLP+Workshop+on+Cognitive+A)
43. Harish Tayyar Madabushi, Laurence Romain, Dagmar Divjak, and Petar Milin. 2020. CxGBERT: BERT Meets Construction Grammar. In Proceedings of COLING. [find](https://scholar.google.com/scholar?q=Harish+Tayyar+Madabushi%2C+Laurence+Romain%2C+Dagmar+Divjak%2C+and+Petar+Milin%2E+2020%2E+CxGBERT%3A+BERT+Meets+Construction+Grammar%2E+In+Proceedings+of+COLING%2E)
44. Giulia Rambelli, Emmanuele Chersoni, Alessandro Lenci, Philippe Blache, and Chu-Ren Huang. 2020. Comparing Probabilistic, Distributional and Transformer-based Models on Logical Metonymy Interpretation. In Proceedings of AACL-IJCNLP. [find](https://scholar.google.com/scholar?q=Giulia+Rambelli%2C+Emmanuele+Chersoni%2C+Alessandro+Lenci%2C+Philippe+Blache%2C+and+Chu%2DRen+Huang%2E+2020%2E+Comparing+Probabilistic%2C+Distributional+and+Transformer%2Dbased+Models+on+Logical+Metonymy+Interpretation)
45. Alec Marantz. 1995. The Minimalist Program. In The Principles and Parameters Approach to Linguistic Theory, pages 351–382. Blackwell. [find](https://scholar.google.com/scholar?q=Alec+Marantz%2E+1995%2E+The+Minimalist+Program%2E+In+The+Principles+and+Parameters+Approach+to+Linguistic+Theory%2C+pages+351%E2%80%93382%2E+Blackwell%2E)
46. Danny Merkx and Stefan L Frank. 2021. Human Sentence Processing: Recurrence or Attention? In Proceedings of the NAACL Workshop on Cognitive Modeling and Computational Linguistics. [find](https://scholar.google.com/scholar?q=Danny+Merkx+and+Stefan+L+Frank%2E+2021%2E+Human+Sentence+Processing%3A+Recurrence+or+Attention%3F+In+Proceedings+of+the+NAACL+Workshop+on+Cognitive+Modeling+and+Computational+Linguistics%2E)
47. Keith Rayner and Susan A Duffy. 1986. Lexical Complexity and Fixation Times in Reading: Effects of Word Frequency, Verb Complexity, and Lexical Ambiguity. Memory & Cognition, 14(3):191–201. [find](https://scholar.google.com/scholar?q=Keith+Rayner+and+Susan+A+Duffy%2E+1986%2E+Lexical+Complexity+and+Fixation+Times+in+Reading%3A+Effects+of+Word+Frequency%2C+Verb+Complexity%2C+and+Lexical+Ambiguity%2E+Memory+%26+Cognition%2C+14%283%29%3A191%E2%80%93201%2E)
48. James A Michaelov and Benjamin K Bergen. 2020. How Well Does Surprisal Explain N400 Amplitude under Different Experimental Conditions? In Proceedings of CONLL. [find](https://scholar.google.com/scholar?q=James+A+Michaelov+and+Benjamin+K+Bergen%2E+2020%2E+How+Well+Does+Surprisal+Explain+N400+Amplitude+under+Different+Experimental+Conditions%3F+In+Proceedings+of+CONLL%2E)
49. Erik D. Reichle, Keith Rayner, and Alexander Pollatsek. 2003. The EZ Reader Model of Eye-Movement Control in Reading: Comparisons to Other Models. Behavioral and Brain Sciences, 26(4):445–476. [find](https://scholar.google.com/scholar?q=Erik+D%2E+Reichle%2C+Keith+Rayner%2C+and+Alexander+Pollatsek%2E+2003%2E+The+EZ+Reader+Model+of+Eye%2DMovement+Control+in+Reading%3A+Comparisons+to+Other+Models%2E+Behavioral+and+Brain+Sciences%2C+26%284%29%3A445%E2%80%93476%2E)
50. James A Michaelov and Benjamin K Bergen. 2022a. Collateral Facilitation in Humans and Language Models. In Proceedings of CONLL. [find](https://scholar.google.com/scholar?q=James+A+Michaelov+and+Benjamin+K+Bergen%2E+2022a%2E+Collateral+Facilitation+in+Humans+and+Language+Models%2E+In+Proceedings+of+CONLL%2E)
51. Pavel Rychlý. 2008. A Lexicographer-Friendly Association Score. In RASLAN, pages 6–9. [find](https://scholar.google.com/scholar?q=Pavel+Rychl%C3%BD%2E+2008%2E+A+Lexicographer%2DFriendly+Association+Score%2E+In+RASLAN%2C+pages+6%E2%80%939%2E) [our card](https://md.research.elamur.ai/free/papers/lex-2008-rychly-logdice/)
52. James A Michaelov and Benjamin K Bergen. 2022b. ’Rarely’ a Problem? Language Models Exhibit Inverse Scaling in their Predictions Following ’Few’-type Quantifiers. arXiv preprint arXiv:2212.08700. [find](https://scholar.google.com/scholar?q=James+A+Michaelov+and+Benjamin+K+Bergen%2E+2022b%2E+%E2%80%99Rarely%E2%80%99+a+Problem%3F+Language+Models+Exhibit+Inverse+Scaling+in+their+Predictions+Following+%E2%80%99Few%E2%80%99%2Dtype+Quantifiers%2E+arXiv+preprint+arXiv%3A2212%2E08700%2E)
53. Soo Hyun Ryu and Richard L Lewis. 2021. Accounting for Agreement Phenomena in Sentence Comprehension with Transformer Language Models: Effects of Similarity-based Interference on Surprisal and Attention. In Proceedings of the NAACL Workshop on Cognitive Modeling and Computational Linguistics. [find](https://scholar.google.com/scholar?q=Soo+Hyun+Ryu+and+Richard+L+Lewis%2E+2021%2E+Accounting+for+Agreement+Phenomena+in+Sentence+Comprehension+with+Transformer+Language+Models%3A+Effects+of+Similarity%2Dbased+Interference+on+Surprisal+and+Attenti)
54. James A Michaelov, Seana Coulson, and Benjamin K Bergen. 2023. Can Peanuts Fall in Love with Distributional Semantics? arXiv preprint arXiv:2301.08731. — [arXiv:2301.08731](https://arxiv.org/abs/2301.08731) [find](https://scholar.google.com/scholar?q=James+A+Michaelov%2C+Seana+Coulson%2C+and+Benjamin+K+Bergen%2E+2023%2E+Can+Peanuts+Fall+in+Love+with+Distributional+Semantics%3F+arXiv+preprint+arXiv%3A2301%2E08731%2E+%E2%80%94+%5BarXiv%3A2301%2E08731%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2301%2E)
55. Marco S. G. Senaldi, Junyan Wei, Jason W Gullifer, and Debra Titone. 2022. Scratching your Tête over Language-switched Idioms: Evidence from Eye-movement Measures of Reading. Memory & Cognition, 50(6):1230–1256. [find](https://scholar.google.com/scholar?q=Marco+S%2E+G%2E+Senaldi%2C+Junyan+Wei%2C+Jason+W+Gullifer%2C+and+Debra+Titone%2E+2022%2E+Scratching+your+T%C3%AAte+over+Language%2Dswitched+Idioms%3A+Evidence+from+Eye%2Dmovement+Measures+of+Reading%2E+Memory+%26+Cognition%2C+50%286%29)
56. Kanishka Misra. 2022. minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models. arXiv preprint arXiv:2203.13112. [find](https://scholar.google.com/scholar?q=Kanishka+Misra%2E+2022%2E+minicons%3A+Enabling+Flexible+Behavioral+and+Representational+Analyses+of+Transformer+Language+Models%2E+arXiv+preprint+arXiv%3A2203%2E13112%2E)
57. Marco S.G. Senaldi and Debra Titone. 2022. Less Direct, More Analytical: Eye-Movement Measures of L2 Idiom Reading. Languages, 7(2):91. [find](https://scholar.google.com/scholar?q=Marco+S%2EG%2E+Senaldi+and+Debra+Titone%2E+2022%2E+Less+Direct%2C+More+Analytical%3A+Eye%2DMovement+Measures+of+L2+Idiom+Reading%2E+Languages%2C+7%282%29%3A91%2E)
58. Francesco Vespignani, Paolo Canal, Nicola Molinaro, Sergio Fonda, and Cristina Cacciari. 2010. Predictive Mechanisms in Idiom Comprehension. Journal of Cognitive Neuroscience, 22(8):1682–1700. [find](https://scholar.google.com/scholar?q=Francesco+Vespignani%2C+Paolo+Canal%2C+Nicola+Molinaro%2C+Sergio+Fonda%2C+and+Cristina+Cacciari%2E+2010%2E+Predictive+Mechanisms+in+Idiom+Comprehension%2E+Journal+of+Cognitive+Neuroscience%2C+22%288%29%3A1682%E2%80%931700%2E)
59. Vered Shwartz and Ido Dagan. 2019. Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition. Transactions of the Association for Computational Linguistics, 7:403–419. [find](https://scholar.google.com/scholar?q=Vered+Shwartz+and+Ido+Dagan%2E+2019%2E+Still+a+Pain+in+the+Neck%3A+Evaluating+Text+Representations+on+Lexical+Composition%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+7%3A403%E2%80%93419%2E) [our card](https://md.research.elamur.ai/free/papers/shwartz-2019-still-pain-neck-evaluating-text/)
60. Jason Wei, Yi Tay, and Quoc V Le. 2022. Inverse Scaling Can Become U-shaped. arXiv preprint arXiv:2211.02011. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Yi+Tay%2C+and+Quoc+V+Le%2E+2022%2E+Inverse+Scaling+Can+Become+U%2Dshaped%2E+arXiv+preprint+arXiv%3A2211%2E02011%2E)
61. Leonie Weissweiler, Taiqi He, Naoki Otani, David R Mortensen, Lori Levin, and Hinrich Schütze. 2023. Construction Grammar Provides Unique Insight into Neural Language Models. arXiv preprint arXiv:2302.02178. [find](https://scholar.google.com/scholar?q=Leonie+Weissweiler%2C+Taiqi+He%2C+Naoki+Otani%2C+David+R+Mortensen%2C+Lori+Levin%2C+and+Hinrich+Sch%C3%BCtze%2E+2023%2E+Construction+Grammar+Provides+Unique+Insight+into+Neural+Language+Models%2E+arXiv+preprint+arXiv%3A2302)
62. Anna Siyanova-Chanturia, Kathy Conklin, and Norbert Schmitt. 2011. Adding More Fuel to the Fire: An Eye-tracking Study of Idiom Processing by Native and Non-native Speakers. Second Language Research, 27(2):251–272. [find](https://scholar.google.com/scholar?q=Anna+Siyanova%2DChanturia%2C+Kathy+Conklin%2C+and+Norbert+Schmitt%2E+2011%2E+Adding+More+Fuel+to+the+Fire%3A+An+Eye%2Dtracking+Study+of+Idiom+Processing+by+Native+and+Non%2Dnative+Speakers%2E+Second+Language+Research%2C+)
63. Leonie Weissweiler, Valentin Hofmann, Abdullatif Köksal, and Hinrich Schütze. 2022. The Better Your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlative. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Leonie+Weissweiler%2C+Valentin+Hofmann%2C+Abdullatif+K%C3%B6ksal%2C+and+Hinrich+Sch%C3%BCtze%2E+2022%2E+The+Better+Your+Syntax%2C+the+Better+Your+Semantics%3F+Probing+Pretrained+Language+Models+for+the+English+Comparative+Co)
64. Nathaniel J Smith and Roger Levy. 2013. The Effect of Word Predictability on Reading Time Is Logarithmic. Cognition, 128(3):302–319. [find](https://scholar.google.com/scholar?q=Nathaniel+J+Smith+and+Roger+Levy%2E+2013%2E+The+Effect+of+Word+Predictability+on+Reading+Time+Is+Logarithmic%2E+Cognition%2C+128%283%29%3A302%E2%80%93319%2E)
65. Merity Stephen, Xiong Caiming, Bradbury James, and Richard Socher. 2017. Pointer Sentinel Mixture Models. Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Merity+Stephen%2C+Xiong+Caiming%2C+Bradbury+James%2C+and+Richard+Socher%2E+2017%2E+Pointer+Sentinel+Mixture+Models%2E+Proceedings+of+ICLR%2E) [our card](https://md.research.elamur.ai/free/papers/merity-2016-pointer-sentinel-mixture-models/)
66. Ethan Wilcox, Roger Levy, Takashi Morita, and Richard Futrell. 2018. What Do RNN Language Models Learn about Filler-gap Dependencies? arXiv preprint arXiv:1809.00042. [find](https://scholar.google.com/scholar?q=Ethan+Wilcox%2C+Roger+Levy%2C+Takashi+Morita%2C+and+Richard+Futrell%2E+2018%2E+What+Do+RNN+Language+Models+Learn+about+Filler%2Dgap+Dependencies%3F+arXiv+preprint+arXiv%3A1809%2E00042%2E) [our card](https://md.research.elamur.ai/free/papers/wilcox-2018-rnn-language-models-learn-about/)
67. David A Swinney and Anne Cutler. 1979. The Access and Processing of Idiomatic Expressions. Journal of Verbal Learning and Verbal Behavior, 18(5):523– 534. [find](https://scholar.google.com/scholar?q=David+A+Swinney+and+Anne+Cutler%2E+1979%2E+The+Access+and+Processing+of+Idiomatic+Expressions%2E+Journal+of+Verbal+Learning+and+Verbal+Behavior%2C+18%285%29%3A523%E2%80%93+534%2E)
68. Ethan Gotlieb Wilcox, Jon Gauthier, Jennifer Hu, Peng Qian, and Roger Levy. 2020. On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior. arXiv preprint arXiv:2006.01912. [find](https://scholar.google.com/scholar?q=Ethan+Gotlieb+Wilcox%2C+Jon+Gauthier%2C+Jennifer+Hu%2C+Peng+Qian%2C+and+Roger+Levy%2E+2020%2E+On+the+Predictive+Power+of+Neural+Language+Models+for+Human+Real%2DTime+Comprehension+Behavior%2E+arXiv+preprint+arXiv%3A200)
69. Zoltán Gendler Szabó. 2004. Compositionality. Stanford Encyclopedia of Philosophy. [find](https://scholar.google.com/scholar?q=Zolt%C3%A1n+Gendler+Szab%C3%B3%2E+2004%2E+Compositionality%2E+Stanford+Encyclopedia+of+Philosophy%2E)
70. Stefanie Wulff. 2008. Rethinking Idiomaticity: A Usage-based Approach. A&C Black. [find](https://scholar.google.com/scholar?q=Stefanie+Wulff%2E+2008%2E+Rethinking+Idiomaticity%3A+A+Usage%2Dbased+Approach%2E+A%26C+Black%2E)
71. Debra Titone, Kyle Lovseth, Kristina Kasparian, and Mehrgol Tiv. 2019. Are Figurative Interpretations of Idioms Directly Retrieved, Compositionally Built, or Both? Evidence from Eye Movement Measures of Reading. Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Expérimental… [find](https://scholar.google.com/scholar?q=Debra+Titone%2C+Kyle+Lovseth%2C+Kristina+Kasparian%2C+and+Mehrgol+Tiv%2E+2019%2E+Are+Figurative+Interpretations+of+Idioms+Directly+Retrieved%2C+Compositionally+Built%2C+or+Both%3F+Evidence+from+Eye+Movement+Measures+)
72. Kayo Yin and Graham Neubig. 2022. Interpreting Language Models with Contrastive Explanations. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Kayo+Yin+and+Graham+Neubig%2E+2022%2E+Interpreting+Language+Models+with+Contrastive+Explanations%2E+In+Proceedings+of+EMNLP%2E)