---
title: "Semantics of Multiword Expressions in — Miletić 2024"
description: "Multiword expressions (MWEs) such as noun compounds, particle verbs and idioms spread their meaning across several words."
authors: ["Filip Miletić", "Sabine Schulte im Walde"]
year: 2024
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2024.tacl-1.33/"
doi: "10.1162/tacl_a_00657"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/miletic-2024-semantics-multiword-expressions-transformer-based/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/miletic-2024-semantics-multiword-expressions-transformer-based/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2024](https://md.research.elamur.ai/free/papers/2024/page/4/) / Semantics of Multiword Expressions in Transformer-Based Models: A Survey

# Semantics of Multiword Expressions in Transformer-Based Models: A Survey

**Authors:** [Filip Miletić](https://md.research.elamur.ai/free/authors/filip-miletic/), [Sabine Schulte im Walde](https://md.research.elamur.ai/free/authors/sabine-schulte-im-walde/) · **Year:** 2024 · **Venue:** Transactions of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.tacl-1.33/) · [PDF](https://aclanthology.org/2024.tacl-1.33.pdf) · [DOI 10.1162/tacl\_a\_00657](https://doi.org/10.1162/tacl_a_00657) · [OpenAlex](https://openalex.org/W4395680635)

**Citations:** 6 citations (openalex-snapshot · 2026-10-14) · [14](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Multiword expressions (MWEs) such as noun compounds, particle verbs and idioms spread their meaning across several words. They range from transparent to opaque, and their reading can shift with context. This TACL 2024 survey asks how well transformer language models handle such meanings. The authors review a fast-growing but fragmented literature, mostly intrinsic evaluations of pretrained and optimized models. These are largely encoder-only (BERT, mBERT, RoBERTa, XLM-R, SBERT), with some BART, T5, XLNet and GPT. Tasks include phrase similarity, compound compositionality and paraphrase, and idiomaticity classification of potentially idiomatic expressions (PIEs). Findings are organized around three questions: overall ability and generalization, where MWE information is located in the model, and which linguistic properties matter. The verdict is that models capture MWE semantics only partially and unevenly. They rely on surface overlap and memorization, generalize poorly to unseen expressions, and encode MWE meaning mostly in lower layers. Representations are better for expressions that are less idiosyncratic, less ambiguous and more concrete. The survey matters because it challenges assumptions about transformer semantics and calls for comparable evaluation setups.

## Key points
- On phrase similarity, model–human correlation falls from about 0.6 on the full dataset to about 0.2 on pairs with swapped constituents (e.g., law school vs. school law, 12% of pairs). This points to reliance on word overlap.
- For noun compound compositionality, Nandakumar et al. (2019) obtain Pearson r between 0.15 and 0.60, with BERT roughly 0.2 below static embeddings. Garcia et al. (2021a) report ρ = 0.37 for English and 0.26 for Portuguese, versus 0.73 and 0.60 for the static-embedding SOTA (Cordeiro et al., 2019).
- Fine-tuning and knowledge enhancement can improve MWE representations, but the gains vary widely across models, tasks and expression types.
- There is broad agreement that lower layers work best for compound compositionality, PIE idiomaticity and paraphrase similarity. Modeling the target expression within a wider context also gives the most adequate representations.
- Cross-lingual evidence is thin, since most comparisons rest on single studies (e.g., English vs. Russian PIE transfer). Future work should cover more MWE types and non-English languages and use harder tasks that test generalization.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

14 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 12 not yet.

- [Modeling the Evolution of English Noun Compounds with Feature-Rich Diachronic Compositionality Prediction](https://md.research.elamur.ai/free/papers/miletic-2025-modeling-evolution-english-noun-compounds/)
- [PARSEME 2.0 Multilingual Corpus of Multiword Expressions](https://md.research.elamur.ai/free/papers/savary-2026-parseme-2-0-multilingual-corpus/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{miletic2024semantics,
  title = {{Semantics of Multiword Expressions in Transformer-Based Models: A Survey}},
  author = {Miletić, Filip and Walde, Sabine Schulte im},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.1162/tacl\_a\_00657},
  url = {https://doi.org/10.1162/tacl_a_00657}
}
```

- [← previous SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://md.research.elamur.ai/free/papers/cheng-2024-seeclick-harnessing-gui-grounding-advanced/)
- [next → ShortGPT: Layers in Large Language Models are More Redundant Than You Expect](https://md.research.elamur.ai/free/papers/men-2024-shortgpt-layers-large-language-models/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/papers/2024/page/4/)

## References

79 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Katsiaryna Aharodnik, Anna Feldman, and Jing Peng. 2018. Designing a Russian idiom-annotated corpus. In Proceedings of the Eleventh International Conference on Mojca Brglez. 2023. Dispersing the clouds of doubt: Can cosine similarity of word embeddings help identify relation-level metaphors in the g… [find](https://scholar.google.com/scholar?q=Katsiaryna+Aharodnik%2C+Anna+Feldman%2C+and+Jing+Peng%2E+2018%2E+Designing+a+Russian+idiom%2Dannotated+corpus%2E+In+Proceedings+of+the+Eleventh+International+Conference+on+Mojca+Brglez%2E+2023%2E+Dispersing+the+cloud)
2. Slovene? In Proceedings of the 9th Workshop on Slavic Natural Language Processing 2023 (SlavicNLP 2023), pages 61–69, Dubrovnik, Croatia. Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.bsnlp-1.8 [↗](https://doi.org/10.18653/v1/2023.bsnlp-1.8)
3. Amir Cohen, Hila Gonen, Ori Shapira, Ran Levy, and Yoav Goldberg. 2022. McPhraSy: Multi-context phrase similarity and clustering. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3538–3550, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. htt… [find](https://scholar.google.com/scholar?q=Amir+Cohen%2C+Hila+Gonen%2C+Ori+Shapira%2C+Ran+Levy%2C+and+Yoav+Goldberg%2E+2022%2E+McPhraSy%3A+Multi%2Dcontext+phrase+similarity+and+clustering%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+20)
4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chr… [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D%2E+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretche)
5. Albert Coil and Vered Shwartz. 2023. From chocolate bunny to chocolate crocodile: Do language models understand noun compounds? In Findings of the Association for Computational Linguistics: ACL 2023, pages 2698–2710, Toronto, Canada. Association for Computational Linguistics. https://doi.org/10 .186… [find](https://scholar.google.com/scholar?q=Albert+Coil+and+Vered+Shwartz%2E+2023%2E+From+chocolate+bunny+to+chocolate+crocodile%3A+Do+language+models+understand+noun+compounds%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2023%2C+)
6. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association f… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzma%CC%81n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dling)
7. Lars Buijtelaar and Sandro Pezzelle. 2023. A psycholinguistic analysis of BERT’s representations of compounds. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2230–2241, Dubrovnik, Croatia. Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Lars+Buijtelaar+and+Sandro+Pezzelle%2E+2023%2E+A+psycholinguistic+analysis+of+BERT%E2%80%99s+representations+of+compounds%2E+In+Proceedings+of+the+17th+Conference+of+the+European+Chapter+of+the+Association+for+Comp)
8. Laura Burdick, Jonathan K. Kummerfeld, and Rada Mihalcea. 2022. Using paraphrases to study properties of contextual embeddings. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4558–4568, Seattle,… [find](https://scholar.google.com/scholar?q=Laura+Burdick%2C+Jonathan+K%2E+Kummerfeld%2C+and+Rada+Mihalcea%2E+2022%2E+Using+paraphrases+to+study+properties+of+contextual+embeddings%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Chapter+of+t)
9. Mathieu Constant, Gülşen Eryiǧit, Johanna Monti, Lonneke van der Plas, Carlos Ramisch, Michael Rosner, and Amalia Todirascu. 2017. Multiword expression processing: A survey. Computational Linguistics, 43(4):837–892. https://doi.org/10.1162/COLI a 00302 — [doi:10.1162/coli](https://doi.org/10.1162/coli) [↗](https://doi.org/10.1162/COLI)
10. Paul Cook, Afsaneh Fazly, and Suzanne Stevenson. 2008. The VNC-tokens dataset. In Proceedings of the LREC Workshop Towards a Shared Task for Multiword Expressions (MWE 2008), pages 19–22, Marrakech, Morocco. [find](https://scholar.google.com/scholar?q=Paul+Cook%2C+Afsaneh+Fazly%2C+and+Suzanne+Stevenson%2E+2008%2E+The+VNC%2Dtokens+dataset%2E+In+Proceedings+of+the+LREC+Workshop+Towards+a+Shared+Task+for+Multiword+Expressions+%28MWE+2008%29%2C+pages+19%E2%80%9322%2C+Marrakech%2C+M)
11. Tuhin Chakrabarty, Yejin Choi, and Vered Shwartz. 2022. It’s not rocket science: Interpreting figurative language in narratives. Transactions of the Association for Computational Linguistics, 10:589–606. https://doi.org/10.1162/tacl a 00478 [↗](https://doi.org/10.1162/tacl) [our card](https://md.research.elamur.ai/free/papers/chakrabarty-2022-it-s-not-rocket-science/)
12. Silvio Cordeiro, Aline Villavicencio, Marco Idiart, and Carlos Ramisch. 2019. Unsupervised compositionality prediction of nominal compounds. Computational Linguistics, 45(1):1–57. https://doi.org/10.1162/coli a 00341 [↗](https://doi.org/10.1162/coli)
13. Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, and Martin Wattenberg. 2019. Visualizing and measuring Verna Dankers, Christopher Lucas, and Ivan Titov. 2022. Can transformer be too compositional? Analysing idiom processing in neural machine translation. In Proceedings of… [find](https://scholar.google.com/scholar?q=Andy+Coenen%2C+Emily+Reif%2C+Ann+Yuan%2C+Been+Kim%2C+Adam+Pearce%2C+Fernanda+Vie%CC%81gas%2C+and+Martin+Wattenberg%2E+2019%2E+Visualizing+and+measuring+Verna+Dankers%2C+Christopher+Lucas%2C+and+Ivan+Titov%2E+2022%2E+Can+transform) [our card](https://md.research.elamur.ai/free/papers/dankers-2022-can-transformer-be-too-compositional/)
14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional Transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+Transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
15. Neele Falk, Yana Strakatova, Eva Huber, and Erhard Hinrichs. 2021. Automatic classification of attributes in German adjective-noun phrases. In Proceedings of the 14th International Conference on Computational Semantics (IWCS), pages 239–249, Groningen, The Netherlands (online). Association for Compu… [find](https://scholar.google.com/scholar?q=Neele+Falk%2C+Yana+Strakatova%2C+Eva+Huber%2C+and+Erhard+Hinrichs%2E+2021%2E+Automatic+classification+of+attributes+in+German+adjective%2Dnoun+phrases%2E+In+Proceedings+of+the+14th+International+Conference+on+Compu)
16. Pablo Gamallo, Manuel Corral, and Marcos Garcia. 2021. Comparing dependency-based compositional models with contextualized word embeddings. In Proceedings of the 13th International Conference on Agents and Artificial Intelligence, pages 1258–1265. SCITEPRESS - Science and Technology Publications. ht… [find](https://scholar.google.com/scholar?q=Pablo+Gamallo%2C+Manuel+Corral%2C+and+Marcos+Garcia%2E+2021%2E+Comparing+dependency%2Dbased+compositional+models+with+contextualized+word+embeddings%2E+In+Proceedings+of+the+13th+International+Conference+on+Agent)
17. Luis Espinosa Anke, Joan Codina-Filba, and Leo Wanner. 2021. Evaluating language models for the retrieval and categorization of lexical collocations. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1406–1417, Online.… [find](https://scholar.google.com/scholar?q=Luis+Espinosa+Anke%2C+Joan+Codina%2DFilba%2C+and+Leo+Wanner%2E+2021%2E+Evaluating+language+models+for+the+retrieval+and+categorization+of+lexical+collocations%2E+In+Proceedings+of+the+16th+Conference+of+the+Europ)
18. Pablo Gamallo, Marcos Garcia, and Iria de-Dios-Flores. 2022. Evaluating contextualized vectors from both large language models and compositional strategies. Procesamiento del Lenguaje Natural, 153–164. https://doi.org/10.26342/2022-69-13 — [doi:10.26342/2022-69-13](https://doi.org/10.26342/2022-69-13) [↗](https://doi.org/10.26342/2022-69-13)
19. Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2021a. Assessing the representations of idiomaticity in vector models with a noun compound dataset labeled at type and token levels. In Proceedings of the 59th Annual Meeting of the Association for Computati… [find](https://scholar.google.com/scholar?q=Marcos+Garcia%2C+Tiago+Kramer+Vieira%2C+Carolina+Scarton%2C+Marco+Idiart%2C+and+Aline+Villavicencio%2E+2021a%2E+Assessing+the+representations+of+idiomaticity+in+vector+models+with+a+noun+compound+dataset+labeled+)
20. Kawin Ethayarajh. 2019. How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process… [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2E+2019%2E+How+contextual+are+contextualized+word+representations%3F+Comparing+the+geometry+of+BERT%2C+ELMo%2C+and+GPT%2D2+embeddings%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+i)
21. Allyson Ettinger. 2020. What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8:34–48. https://doi.org/10.1162/tacl a 00298 [↗](https://doi.org/10.1162/tacl) [our card](https://md.research.elamur.ai/free/papers/ettinger-2020-what-bert-is-not-lessons/)
22. Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2021b. Probing for idiomaticity in vector space models. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3551–3564, Online. A… [find](https://scholar.google.com/scholar?q=Marcos+Garcia%2C+Tiago+Kramer+Vieira%2C+Carolina+Scarton%2C+Marco+Idiart%2C+and+Aline+Villavicencio%2E+2021b%2E+Probing+for+idiomaticity+in+vector+space+models%2E+In+Proceedings+of+the+16th+Conference+of+the+Europe) [our card](https://md.research.elamur.ai/free/papers/garcia-2021-probing-idiomaticity-vector-space-models/)
23. Hessel Haagsma, Johan Bos, and Malvina Nissim. 2020. MAGPIE: A large corpus of potentially idiomatic expressions. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 279–287, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Hessel+Haagsma%2C+Johan+Bos%2C+and+Malvina+Nissim%2E+2020%2E+MAGPIE%3A+A+large+corpus+of+potentially+idiomatic+expressions%2E+In+Proceedings+of+the+Twelfth+Language+Resources+and+Evaluation+Conference%2C+pages+279%E2%80%93) [our card](https://md.research.elamur.ai/free/papers/haagsma-2020-magpie-large-corpus-potentially-idiomatic/)
24. Christopher Jenkins, Filip Miletić, and Sabine Schulte im Walde. 2023. To split or not to split: Composing compounds in contextual vector spaces. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16131–16136, Singapore. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Christopher+Jenkins%2C+Filip+Miletic%CC%81%2C+and+Sabine+Schulte+im+Walde%2E+2023%2E+To+split+or+not+to+split%3A+Composing+compounds+in+contextual+vector+spaces%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Me)
25. Matthias Hartung. 2015. Distributional Semantic Models of Attribute Meaning in Adjectives and Nouns. Ph.D. thesis, Heidelberg University. [find](https://scholar.google.com/scholar?q=Matthias+Hartung%2E+2015%2E+Distributional+Semantic+Models+of+Attribute+Meaning+in+Adjectives+and+Nouns%2E+Ph%2ED%2E+thesis%2C+Heidelberg+University%2E)
26. Chikara Hashimoto and Daisuke Kawahara. 2008. Construction of an idiom corpus and its application to idiom identification based on WSD incorporating idiom-specific features. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 992–1001, Honolulu, Hawaii. A… [find](https://scholar.google.com/scholar?q=Chikara+Hashimoto+and+Daisuke+Kawahara%2E+2008%2E+Construction+of+an+idiom+corpus+and+its+application+to+idiom+identification+based+on+WSD+incorporating+idiom%2Dspecific+features%2E+In+Proceedings+of+the+2008)
27. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. SpanBERT: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77. https://doi.org/10 .1162/tacl a 00300 [find](https://scholar.google.com/scholar?q=Mandar+Joshi%2C+Danqi+Chen%2C+Yinhan+Liu%2C+Daniel+S%2E+Weld%2C+Luke+Zettlemoyer%2C+and+Omer+Levy%2E+2020%2E+SpanBERT%3A+Improving+pre%2Dtraining+by+representing+and+predicting+spans%2E+Transactions+of+the+Association+for+) [our card](https://md.research.elamur.ai/free/papers/joshi-2019-spanbert-improving-pre-training-representing/)
28. Adi Haviv, Ido Cohen, Jacob Gidron, Roei Schuster, Yoav Goldberg, and Mor Geva. 2023. Understanding transformer memorization recall through idioms. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 248–264, Dubrovnik, Croatia. Assoc… [find](https://scholar.google.com/scholar?q=Adi+Haviv%2C+Ido+Cohen%2C+Jacob+Gidron%2C+Roei+Schuster%2C+Yoav+Goldberg%2C+and+Mor+Geva%2E+2023%2E+Understanding+transformer+memorization+recall+through+idioms%2E+In+Proceedings+of+the+17th+Conference+of+the+Europea) [our card](https://md.research.elamur.ai/free/papers/haviv-2023-understanding-transformer-memorization-recall-through/)
29. Barbara J. Juhasz, Yun-Hsuan Lai, and Michelle L. Woodcock. 2015. A database of 629 English compound words: Ratings of familiarity, lexeme meaning dominance, semantic transparency, age of acquisition, imageability, and sensory experience. Behavior Research Methods, 47(4):1004–1019. https://doi.org/… — [doi:10.3758/s13428-014-0523-6](https://doi.org/10.3758/s13428-014-0523-6) [find](https://scholar.google.com/scholar?q=Barbara+J%2E+Juhasz%2C+Yun%2DHsuan+Lai%2C+and+Michelle+L%2E+Woodcock%2E+2015%2E+A+database+of+629+English+compound+words%3A+Ratings+of+familiarity%2C+lexeme+meaning+dominance%2C+semantic+transparency%2C+age+of+acquisition%2C)
30. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. DeBERTa: Decoding-enhanced BERT with disentangled attention. In Proceedings of the Ninth International Conference on Learning Representations (ICLR 2021). [find](https://scholar.google.com/scholar?q=Pengcheng+He%2C+Xiaodong+Liu%2C+Jianfeng+Gao%2C+and+Weizhu+Chen%2E+2021%2E+DeBERTa%3A+Decoding%2Denhanced+BERT+with+disentangled+attention%2E+In+Proceedings+of+the+Ninth+International+Conference+on+Learning+Represent) [our card](https://md.research.elamur.ai/free/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/)
31. Filip Klubička, Vasudevan Nedumpozhimana, and John Kelleher. 2023. Idioms, probing and dangerous things: Towards structural probing for idiomaticity in vector space. In Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), pages 45–57, Dubrovnik, Croatia. Association for Computation… [find](https://scholar.google.com/scholar?q=Filip+Klubic%CC%8Cka%2C+Vasudevan+Nedumpozhimana%2C+and+John+Kelleher%2E+2023%2E+Idioms%2C+probing+and+dangerous+things%3A+Towards+structural+probing+for+idiomaticity+in+vector+space%2E+In+Proceedings+of+the+19th+Worksh)
32. Iris Hendrickx, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Stan Szpakowicz, and Tony Veale. 2013. SemEval-2013 task 4: Free paraphrases of noun compounds. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Worksh… [find](https://scholar.google.com/scholar?q=Iris+Hendrickx%2C+Zornitsa+Kozareva%2C+Preslav+Nakov%2C+Diarmuid+O%CC%81+Se%CC%81aghdha%2C+Stan+Szpakowicz%2C+and+Tony+Veale%2E+2013%2E+SemEval%2D2013+task+4%3A+Free+paraphrases+of+noun+compounds%2E+In+Second+Joint+Conference+on+L)
33. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In Proceedings of the Eighth International Conference on Learning Representations (ICLR 2020). [find](https://scholar.google.com/scholar?q=Zhenzhong+Lan%2C+Mingda+Chen%2C+Sebastian+Goodman%2C+Kevin+Gimpel%2C+Piyush+Sharma%2C+and+Radu+Soricut%2E+2020%2E+ALBERT%3A+A+lite+BERT+for+self%2Dsupervised+learning+of+language+representations%2E+In+Proceedings+of+the+)
34. Beth Levin, Lelia Glass, and Dan Jurafsky. 2019. Systematicity in the semantics of noun Ganesh Jawahar, Benoı̂t Sagot, and Djamé Seddah. 2019. What does BERT learn about compounds: The role of artifacts vs. natural kinds. Linguistics, 57(3):429–471. https://doi.org/10.1515/ling-2019-0013 do you me… — [doi:10.1515/ling-2019-0013](https://doi.org/10.1515/ling-2019-0013) [↗](https://doi.org/10.1515/ling-2019-0013)
35. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the… [find](https://scholar.google.com/scholar?q=Mike+Lewis%2C+Yinhan+Liu%2C+Naman+Goyal%2C+Marjan+Ghazvininejad%2C+Abdelrahman+Mohamed%2C+Omer+Levy%2C+Veselin+Stoyanov%2C+and+Luke+Zettlemoyer%2E+2020%2E+BART%3A+Denoising+sequence%2Dto%2Dsequence+pre%2Dtraining+for+natural+l) [our card](https://md.research.elamur.ai/free/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
36. Filip Miletić and Sabine Schulte im Walde. 2023. A systematic search for compound semantics in pretrained BERT architectures. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1499–1512, Dubrovnik, Croatia. Association for Computat… [find](https://scholar.google.com/scholar?q=Filip+Miletic%CC%81+and+Sabine+Schulte+im+Walde%2E+2023%2E+A+systematic+search+for+compound+semantics+in+pretrained+BERT+architectures%2E+In+Proceedings+of+the+17th+Conference+of+the+European+Chapter+of+the+Asso)
37. Siyan Li, Riley Carlson, and Christopher Potts. 2022. Systematicity in GPT-3’s interpretation of novel English noun compounds. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 717–728, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. https://… [find](https://scholar.google.com/scholar?q=Siyan+Li%2C+Riley+Carlson%2C+and+Christopher+Potts%2E+2022%2E+Systematicity+in+GPT%2D3%E2%80%99s+interpretation+of+novel+English+noun+compounds%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2022%2C)
38. Jeff Mitchell and Mirella Lapata. 2008. Vector-based models of semantic composition. In Proceedings of ACL-08: HLT, pages 236–244, Columbus, Ohio. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jeff+Mitchell+and+Mirella+Lapata%2E+2008%2E+Vector%2Dbased+models+of+semantic+composition%2E+In+Proceedings+of+ACL%2D08%3A+HLT%2C+pages+236%E2%80%93244%2C+Columbus%2C+Ohio%2E+Association+for+Computational+Linguistics%2E)
39. Diego Moussallem, Mohamed Ahmed Sherif, Diego Esteves, Marcos Zampieri, and Axel-Cyrille Ngonga Ngomo. 2018. LIdioms: A multilingual linked idioms data set. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Re… [find](https://scholar.google.com/scholar?q=Diego+Moussallem%2C+Mohamed+Ahmed+Sherif%2C+Diego+Esteves%2C+Marcos+Zampieri%2C+and+Axel%2DCyrille+Ngonga+Ngomo%2E+2018%2E+LIdioms%3A+A+multilingual+linked+idioms+data+set%2E+In+Proceedings+of+the+Eleventh+Internationa)
40. Emmy Liu, Chenxuan Cui, Kenneth Zheng, and Graham Neubig. 2022. Testing the ability of language models to interpret figurative language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4437–4452,… [find](https://scholar.google.com/scholar?q=Emmy+Liu%2C+Chenxuan+Cui%2C+Kenneth+Zheng%2C+and+Graham+Neubig%2E+2022%2E+Testing+the+ability+of+language+models+to+interpret+figurative+language%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Cha)
41. Navnita Nandakumar, Timothy Baldwin, and Bahar Salehi. 2019. How well do embedding models capture non-compositionality? A view from multiword expressions. In Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP, pages 27–34, Minneapolis, USA. Association for Computation… [find](https://scholar.google.com/scholar?q=Navnita+Nandakumar%2C+Timothy+Baldwin%2C+and+Bahar+Salehi%2E+2019%2E+How+well+do+embedding+models+capture+non%2Dcompositionality%3F+A+view+from+multiword+expressions%2E+In+Proceedings+of+the+3rd+Workshop+on+Evaluat)
42. Emmy Liu and Graham Neubig. 2022. Are representations built from the ground up? An empirical examination of local composition in language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9053–9073, Abu Dhabi, United Arab Emirates. Association f… [find](https://scholar.google.com/scholar?q=Emmy+Liu+and+Graham+Neubig%2E+2022%2E+Are+representations+built+from+the+ground+up%3F+An+empirical+examination+of+local+composition+in+language+models%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Met)
43. Vasudevan Nedumpozhimana and John Kelleher. 2021. Finding BERT’s idiomatic key. In Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021), pages 57–62, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.mwe-1.7 [↗](https://doi.org/10.18653/v1/2021.mwe-1.7)
44. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv: 1907.11692. https://doi.org/10.48550/arXiv.1907.11692 [↗](https://doi.org/10.48550/arXiv.1907.11692)
45. Vasudevan Nedumpozhimana, Filip Klubička, and John D. Kelleher. 2022. Shapley idioms: Analysing BERT sentence embeddings for general idiom token identification. Frontiers in Artificial Intelligence, 5:813967. https://doi.org/10.3389/frai.2022.813967, PubMed: 35360661 — [doi:10.3389/frai.2022.813967](https://doi.org/10.3389/frai.2022.813967) [↗](https://doi.org/10.3389/frai.2022.813967)
46. Timothee Mickus, Denis Paperno, Mathieu Constant, and Kees van Deemter. 2020. What Diarmuid Ó. Séaghdha and Ann Copestake. 2007. Co-occurrence contexts for noun compound interpretation. In Proceedings of the Workshop on A Broader Perspective on Multiword Expressions, pages 57–64, Prague, Czech Rep… [find](https://scholar.google.com/scholar?q=Timothee+Mickus%2C+Denis+Paperno%2C+Mathieu+Constant%2C+and+Kees+van+Deemter%2E+2020%2E+What+Diarmuid+O%CC%81%2E+Se%CC%81aghdha+and+Ann+Copestake%2E+2007%2E+Co%2Doccurrence+contexts+for+noun+compound+interpretation%2E+In+Proceedin)
47. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(1):5485–5551. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+and+Peter+J%2E+Liu%2E+2020%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dtex)
48. Giulia Rambelli, Emmanuele Chersoni, Marco S. G. Senaldi, Philippe Blache, and Alessandro Lenci. 2023. Are frequent phrases directly retrieved like idioms? An investigation with self-paced reading and language models. In Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), pages 87–… [find](https://scholar.google.com/scholar?q=Giulia+Rambelli%2C+Emmanuele+Chersoni%2C+Marco+S%2E+G%2E+Senaldi%2C+Philippe+Blache%2C+and+Alessandro+Lenci%2E+2023%2E+Are+frequent+phrases+directly+retrieved+like+idioms%3F+An+investigation+with+self%2Dpaced+reading+and) [our card](https://md.research.elamur.ai/free/papers/rambelli-2023-are-frequent-phrases-directly-retrieved/)
49. Ellie Pavlick, Pushpendre Rastogi, Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2015. PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational… [find](https://scholar.google.com/scholar?q=Ellie+Pavlick%2C+Pushpendre+Rastogi%2C+Juri+Ganitkevitch%2C+Benjamin+Van+Durme%2C+and+Chris+Callison%2DBurch%2E+2015%2E+PPDB+2%2E0%3A+Better+paraphrase+ranking%2C+fine%2Dgrained+entailment+relations%2C+word+embeddings%2C+and+s) [our card](https://md.research.elamur.ai/free/papers/pavlick-2015-ppdb-2-0-better-paraphrase/)
50. Siva Reddy, Diana McCarthy, and Suresh Manandhar. 2011. An empirical study on compositionality in compound nouns. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 210–218, Chiang Mai, Thailand. Asian Federation of Natural Language Processing. [find](https://scholar.google.com/scholar?q=Siva+Reddy%2C+Diana+McCarthy%2C+and+Suresh+Manandhar%2E+2011%2E+An+empirical+study+on+compositionality+in+compound+nouns%2E+In+Proceedings+of+5th+International+Joint+Conference+on+Natural+Language+Processing%2C+p) [our card](https://md.research.elamur.ai/free/papers/reddy-2011-empirical-study-compositionality-compound-nouns/)
51. Thang Pham, Seunghyun Yoon, Trung Bui, and Anh Nguyen. 2023. PiC: A phrase-in-context dataset for phrase understanding and semantic search. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1–26, Dubrovnik, Croatia. Association for… [find](https://scholar.google.com/scholar?q=Thang+Pham%2C+Seunghyun+Yoon%2C+Trung+Bui%2C+and+Anh+Nguyen%2E+2023%2E+PiC%3A+A+phrase%2Din%2Dcontext+dataset+for+phrase+understanding+and+semantic+search%2E+In+Proceedings+of+the+17th+Conference+of+the+European+Chapte)
52. Yuval Pinter, Cassandra L. Jacobs, and Max Bittker. 2020a. NYTWIT: A dataset of novel words in the New York Times. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6509–6515, Barcelona, Spain (Online). International Committee on Computational Linguistics. https… [find](https://scholar.google.com/scholar?q=Yuval+Pinter%2C+Cassandra+L%2E+Jacobs%2C+and+Max+Bittker%2E+2020a%2E+NYTWIT%3A+A+dataset+of+novel+words+in+the+New+York+Times%2E+In+Proceedings+of+the+28th+International+Conference+on+Computational+Linguistics%2C+pag)
53. Yuval Pinter, Cassandra L. Jacobs, and Jacob Eisenstein. 2020b. Will it unblend? In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1525–1535, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020 .findings-emnlp.138 [↗](https://doi.org/10.18653/v1/2020)
54. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E)
55. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2019%2E+Sentence%2DBERT%3A+Sentence+embeddings+using+Siamese+BERT%2DNetworks%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the)
56. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in BERTology: What we know about how BERT works. Transactions of the Association for Computational Linguistics, 8:842–866. https://doi.org/10.1162/tacl_a_00349 [↗](https://doi.org/10.1162/tacl_a_00349) [our card](https://md.research.elamur.ai/free/papers/rogers-2020-primer-bertology-what-we-know/)
57. Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multiword expressions: A pain in the neck for NLP. In Computational Linguistics and Intelligent Text Processing, pages 1–15, Berlin. Springer. https://doi.org/10.1007/3 -540-45715-1_1 [find](https://scholar.google.com/scholar?q=Ivan+A%2E+Sag%2C+Timothy+Baldwin%2C+Francis+Bond%2C+Ann+Copestake%2C+and+Dan+Flickinger%2E+2002%2E+Multiword+expressions%3A+A+pain+in+the+neck+for+NLP%2E+In+Computational+Linguistics+and+Intelligent+Text+Processing%2C+pa) [our card](https://md.research.elamur.ai/free/papers/idioms-2002-sag-multiword-expressions-pain-neck-nlp/)
58. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. In 5th Workshop on Energy Efficient Machine Learning and Cognitive Computing – NeurIPS 2019. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+2019%2E+DistilBERT%2C+a+distilled+version+of+BERT%3A+Smaller%2C+faster%2C+cheaper+and+lighter%2E+In+5th+Workshop+on+Energy+Efficient+Machine+Learning)
59. Dominican Republic. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2021.emnlp-main.230 [find](https://scholar.google.com/scholar?q=Dominican+Republic%2E+Association+for+Computational+Linguistics%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E18653%2Fv1%2F2021%2Eemnlp%2Dmain%2E230) [our card](https://md.research.elamur.ai/free/papers/sinha-2021-masked-language-modeling-distributional-hypothesis/)
60. Agata Savary, Carlos Ramisch, Silvio Cordeiro, Federico Sangati, Veronika Vincze, Behrang QasemiZadeh, Marie Candito, Fabienne Cap, Voula Giouli, Ivelina Stoyanova, and Antoine Doucet. 2017. The PARSEME shared task on automatic identification of verbal multiword expressions. In Proceedings of the 13… [find](https://scholar.google.com/scholar?q=Agata+Savary%2C+Carlos+Ramisch%2C+Silvio+Cordeiro%2C+Federico+Sangati%2C+Veronika+Vincze%2C+Behrang+QasemiZadeh%2C+Marie+Candito%2C+Fabienne+Cap%2C+Voula+Giouli%2C+Ivelina+Stoyanova%2C+and+Antoine+Doucet%2E+2017%2E+The+PARSE) [our card](https://md.research.elamur.ai/free/papers/savary-2017-parseme-shared-task-automatic-identification/)
61. Tadej Škvorc, Polona Gantar, and Marko Robnik-Šikonja. 2022. MICE: Mining idioms with contextual embeddings. Knowledge-Based Systems, 235:107606. https://doi.org/10 .1016/j.knosys.2021.107606 [find](https://scholar.google.com/scholar?q=Tadej+S%CC%8Ckvorc%2C+Polona+Gantar%2C+and+Marko+Robnik%2DS%CC%8Cikonja%2E+2022%2E+MICE%3A+Mining+idioms+with+contextual+embeddings%2E+Knowledge%2DBased+Systems%2C+235%3A107606%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E1016%2Fj%2Eknosys%2E2021%2E107606)
62. Yana Strakatova, Neele Falk, Isabel Fuhrmann, Erhard Hinrichs, and Daniela Rossmann. 2020. All that glitters is not gold: A gold standard of adjective-noun collocations for German. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4368–4378, Marseille, France. Europea… [find](https://scholar.google.com/scholar?q=Yana+Strakatova%2C+Neele+Falk%2C+Isabel+Fuhrmann%2C+Erhard+Hinrichs%2C+and+Daniela+Rossmann%2E+2020%2E+All+that+glitters+is+not+gold%3A+A+gold+standard+of+adjective%2Dnoun+collocations+for+German%2E+In+Proceedings+of+t)
63. Prateek Saxena and Soma Paul. 2020. EPIE dataset: A corpus for possible idiomatic expressions. In Petr Sojka, Ivan Kopeček, Karel Pala, and Aleš Horák, editors, Text, Speech, and Dialogue, volume 12284, pages 87–94, Cham. Springer. https://doi.org/10 .1007/978-3-030-58323-1_9 [find](https://scholar.google.com/scholar?q=Prateek+Saxena+and+Soma+Paul%2E+2020%2E+EPIE+dataset%3A+A+corpus+for+possible+idiomatic+expressions%2E+In+Petr+Sojka%2C+Ivan+Kopec%CC%8Cek%2C+Karel+Pala%2C+and+Ales%CC%8C+Hora%CC%81k%2C+editors%2C+Text%2C+Speech%2C+and+Dialogue%2C+volume+1)
64. Ryosuke Takahashi, Ryohei Sasano, and Koichi Takeda. 2022. Leveraging three types of embeddings from masked language models in idiom token classification. In Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, pages 234–239, Seattle, Washington. Association for Computati… [find](https://scholar.google.com/scholar?q=Ryosuke+Takahashi%2C+Ryohei+Sasano%2C+and+Koichi+Takeda%2E+2022%2E+Leveraging+three+types+of+embeddings+from+masked+language+models+in+idiom+token+classification%2E+In+Proceedings+of+the+11th+Joint+Conference+o)
65. Sabine Schulte im Walde. forthc. Collecting and investigating features of compositionality ratings. In Voula Giouli and Verginica Barbu Mititelu, editors, Multiword Expressions in Lexical Resources. Linguistic, Lexicographic and Computational Perspectives, Phraseology and Multiword Expressions, Lang… [find](https://scholar.google.com/scholar?q=Sabine+Schulte+im+Walde%2E+forthc%2E+Collecting+and+investigating+features+of+compositionality+ratings%2E+In+Voula+Giouli+and+Verginica+Barbu+Mititelu%2C+editors%2C+Multiword+Expressions+in+Lexical+Resources%2E+L)
66. Minghuan Tan and Jing Jiang. 2020. A BERT-based dual embedding model for Chinese idiom prediction. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1312–1322, Barcelona, Spain (Online). International Committee on Computational Linguistics. https://doi.org/10 .1… [find](https://scholar.google.com/scholar?q=Minghuan+Tan+and+Jing+Jiang%2E+2020%2E+A+BERT%2Dbased+dual+embedding+model+for+Chinese+idiom+prediction%2E+In+Proceedings+of+the+28th+International+Conference+on+Computational+Linguistics%2C+pages+1312%E2%80%931322%2C+Ba)
67. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In Proceedings of the Fifth International Conference on Learning Representations (ICLR 2017). [find](https://scholar.google.com/scholar?q=Minjoon+Seo%2C+Aniruddha+Kembhavi%2C+Ali+Farhadi%2C+and+Hannaneh+Hajishirzi%2E+2017%2E+Bidirectional+attention+flow+for+machine+comprehension%2E+In+Proceedings+of+the+Fifth+International+Conference+on+Learning+Re) [our card](https://md.research.elamur.ai/free/papers/seo-2016-bidirectional-attention-flow-machine-comprehension/)
68. Minghuan Tan and Jing Jiang. 2021. Does BERT understand idioms? A probing-based empirical study of BERT encodings of idioms. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1397–1407, Held Online. INCOMA Ltd. [https://doi.org](https://doi.org) /10.26… [↗](https://doi.org)
69. Vered Shwartz and Ido Dagan. 2019. Still a pain in the neck: Evaluating text representations on lexical composition. Transactions of the Association for Computational Linguistics, 7:403–419. https://doi.org/10.1162/tacl_a_00277 [↗](https://doi.org/10.1162/tacl_a_00277) [our card](https://md.research.elamur.ai/free/papers/shwartz-2019-still-pain-neck-evaluating-text/)
70. Minghuan Tan, Jing Jiang, and Bing Tian Dai. 2021. A BERT-based two-stage model for Chinese chengyu recommendation. ACM Transactions on Asian and Low-Resource Language Information Processing, 20(6):1–18. https://doi.org/10.1145/3453185 — [doi:10.1145/3453185](https://doi.org/10.1145/3453185) [↗](https://doi.org/10.1145/3453185)
71. Koustuv Sinha, Robin Jia, Dieuwke Hupkes, Joelle Pineau, Adina Williams, and Douwe Kiela. 2021. Masked language modeling and the distributional hypothesis: Order word matters pre-training for little. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 288… [find](https://scholar.google.com/scholar?q=Koustuv+Sinha%2C+Robin+Jia%2C+Dieuwke+Hupkes%2C+Joelle+Pineau%2C+Adina+Williams%2C+and+Douwe+Kiela%2E+2021%2E+Masked+language+modeling+and+the+distributional+hypothesis%3A+Order+word+matters+pre%2Dtraining+for+little%2E+) [our card](https://md.research.elamur.ai/free/papers/sinha-2021-masked-language-modeling-distributional-hypothesis/)
72. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R. Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems, volume 32. Curran Associates. [find](https://scholar.google.com/scholar?q=Zhilin+Yang%2C+Zihang+Dai%2C+Yiming+Yang%2C+Jaime+Carbonell%2C+Russ+R%2E+Salakhutdinov%2C+and+Quoc+V%2E+Le%2E+2019%2E+XLNet%3A+Generalized+autoregressive+pretraining+for+language+understanding%2E+In+Advances+in+Neural+Info) [our card](https://md.research.elamur.ai/free/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/)
73. Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, and Anna Korhonen. 2020. Probing pretrained language models for lexical semantics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7222–7240, Online. Association for Computation… [find](https://scholar.google.com/scholar?q=Ivan+Vulic%CC%81%2C+Edoardo+Maria+Ponti%2C+Robert+Litschko%2C+Goran+Glavas%CC%8C%2C+and+Anna+Korhonen%2E+2020%2E+Probing+pretrained+language+models+for+lexical+semantics%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+) [our card](https://md.research.elamur.ai/free/papers/vulic-2020-probing-pretrained-language-models-lexical/)
74. Shufan Wang, Laure Thompson, and Mohit Iyyer. 2021. Phrase-BERT: Improved phrase embeddings from BERT with an application to corpus exploration. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10837–10851, Online and Punta Cana, Dominican Republic. As… [find](https://scholar.google.com/scholar?q=Shufan+Wang%2C+Laure+Thompson%2C+and+Mohit+Iyyer%2E+2021%2E+Phrase%2DBERT%3A+Improved+phrase+embeddings+from+BERT+with+an+application+to+corpus+exploration%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Meth)
75. Gregor Wiedemann, Steffen Remus, Avi Chawla, and Chris Biemann. 2019. Does BERT make any sense? Interpretable word sense disambiguation with contextualized embeddings. In Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019): Long Papers, pages 161–170, Erlangen, Germany.… [find](https://scholar.google.com/scholar?q=Gregor+Wiedemann%2C+Steffen+Remus%2C+Avi+Chawla%2C+and+Chris+Biemann%2E+2019%2E+Does+BERT+make+any+sense%3F+Interpretable+word+sense+disambiguation+with+contextualized+embeddings%2E+In+Proceedings+of+the+15th+Confe)
76. Lang Yu and Allyson Ettinger. 2020. Assessing phrasal representation and composition in Transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4896–4907, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1 /20… [find](https://scholar.google.com/scholar?q=Lang+Yu+and+Allyson+Ettinger%2E+2020%2E+Assessing+phrasal+representation+and+composition+in+Transformers%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C)
77. Lang Yu and Allyson Ettinger. 2021. On the interplay between fine-tuning and composition in Transformers. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2279–2293, Online. Association for Computational Linguistics. [https://doi.org](https://doi.org) /10.18653/v1/2021.findings-acl.… [↗](https://doi.org)
78. Ziheng Zeng and Suma Bhat. 2021. Idiomatic expression identification using semantic compatibility. Transactions of the Association for Computational Linguistics, 9:1546–1562. https://doi.org/10.1162/tacl a 00442 [↗](https://doi.org/10.1162/tacl) [our card](https://md.research.elamur.ai/free/papers/zeng-2021-idiomatic-expression-identification-semantic-compatibility/)
79. Ziheng Zeng and Suma Bhat. 2022. Getting BART to ride the idiomatic train: Learning to represent idiomatic expressions. Transactions of the Association for Computational Linguistics, 10:1120–1137. https://doi.org/10.1162/tacl a 00510 [↗](https://doi.org/10.1162/tacl)