---
title: "Getting BART to Ride the Idiomatic Train: Learning to Represent"
description: "Pre-trained language models, however, build meaning compositionally through… 50.7% accuracy. 67 references, cited by 18 papers here."
authors: ["Ziheng Zeng", "Suma Bhat"]
year: 2022
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2022.tacl-1.65/"
doi: "10.1162/tacl_a_00510"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zeng-2022-getting-bart-ride-idiomatic-train/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zeng-2022-getting-bart-ride-idiomatic-train/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/) / Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions

# Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions

**Authors:** [Ziheng Zeng](https://md.research.elamur.ai/free/en/authors/ziheng-zeng/) and [Suma Bhat](https://md.research.elamur.ai/free/en/authors/suma-bhat/) · **Year:** 2022 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2022](https://md.research.elamur.ai/free/en/venues/tacl/2022/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.tacl-1.65/) · [PDF](https://aclanthology.org/2022.tacl-1.65.pdf) · [DOI 10.1162/tacl\_a\_00510](https://doi.org/10.1162/tacl_a_00510) · [OpenAlex](https://openalex.org/W4312558730)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · [18](https://md.research.elamur.ai/free/en/papers/zeng-2022-getting-bart-ride-idiomatic-train/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Idioms such as "break a leg" mean something their individual words do not. Pre-trained language models, however, build meaning compositionally through subword tokenization and self-attention. The authors first show this weakness in BART. When they average its embeddings of idioms from the MAGPIE dataset, the nearest neighbours of an idiom tend to share words or syntax rather than meaning. They also hand-sorted 129 idioms into 20 meaning classes and clustered them. BART embeddings reached a homogeneity score of only 0.45, compared with 0.68 for MPNet embeddings of dictionary definitions. To fix this, they propose GIEA (Generation of Idiom Embedding with Adapter). It adds a compact Pfeiffer adapter to BART, trained on idiomatic sentences with denoising auto-encoder objectives that rebuild masked sentence parts, plus a similarity-forcing objective. Idioms are not turned into new tokens, and training needs only about 60MB of text. GIEA raises homogeneity by 0.19 and holds its own against other methods in IE sense disambiguation. On IE span detection, it improves sequence accuracy by as much as 25%. This offers a cheap, task-agnostic way to make language models idiom-aware.

## Key points
- A probe of GPT-3 with simple questions about 75 idioms of varying BNC frequency produced only 50.7% accuracy.
- The BART base idiom embeddings were built by mean pooling over 1,480 MAGPIE idioms, with about 22 sentences per idiom on average.
- The intrinsic evaluation used agglomerative clustering (complete linkage, cosine distance) on 129 idioms in 20 meaning groups, checked by two annotators.
- GIEA's adapter-based training data (~60MB) is tiny compared with BART's pre-training corpus (~160GB). The best variant is labelled ITI+SF+SI.
- The extrinsic tasks were IE sense disambiguation and IE span detection. Code is available at github.com/zzeng13/GIEA.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Illinois Urbana-Champaign](https://md.research.elamur.ai/free/en/universities/university-of-illinois-urbana-champaign/) — University, Urbana, United States · 184 works · 69,372 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧷 Idioms & multiword expressions](https://md.research.elamur.ai/free/en/topics/multiword-expressions/) · 217 works

## Cite

Cite as: Ziheng Zeng and Suma Bhat. 2022. Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{zeng2022getting,
  title = {{Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions}},
  author = {Ziheng Zeng and Suma Bhat},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.1162/tacl\_a\_00510},
  url = {https://doi.org/10.1162/tacl_a_00510}
}
```

- [← previous Encoding and Fusing Semantic Connection and Linguistic Evidence for Implicit Discourse Relation Recognition](https://md.research.elamur.ai/free/en/papers/xiang-2022-encoding-fusing-semantic-connection-linguistic/)
- [next → Low-Resource Authorship Style Transfer: Can Non-Famous Authors Be Imitated?](https://md.research.elamur.ai/free/en/papers/patel-2022-low-resource-authorship-style-transfer/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/)

## References

67 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Tosin P. Adewumi, Saleha Javed, Roshanak Vadoodi, Aparajita Tripathy, Konstantina Nikolaidou, Foteini Simistira Liwicki, and Marcus Liwicki. 2021. [Potential idiomatic expression (PIE)-english: Corpus for classes of idioms](https://md.research.elamur.ai/free/en/papers/adewumi-2022-potential-idiomatic-expression-pie-english/). ArXiv, abs/2105.03280.
2. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2016. [Fine-grained analysis of sentence embeddings using auxiliary prediction tasks](https://md.research.elamur.ai/free/en/papers/adi-2016-fine-grained-analysis-sentence-embeddings/). CoRR, abs/1608.04207.
3. Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020. Probing linguistic features of sentence-level representations in neural relation extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1534–1545, Online. Association for Computatio… [find](https://scholar.google.com/scholar?q=Christoph+Alt%2C+Aleksandra+Gabryszak%2C+and+Leonhard+Hennig%2E+2020%2E+Probing+linguistic+features+of+sentence%2Dlevel+representations+in+neural+relation+extraction%2E+In+Proceedings+of+the+58th+Annual+Meeting+o)
4. Alan Ansell, Edoardo Maria Ponti, Jonas Pfeiffer, Sebastian Ruder, Goran Glavaš, Ivan Vulić, and Anna Korhonen. 2021. [MAD-G: Multilingual adapter generation for efficient cross-lingual transfer](https://md.research.elamur.ai/free/en/papers/ansell-2021-mad-g-multilingual-adapter-generation/). In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4762–4781, Punta Cana,…
5. John Ayto and Oxford University Press. 2009. Oxford Dictionary of English Idioms / [edited] by John Ayto, 3rd edition. Oxford University Press [Oxford]. [find](https://scholar.google.com/scholar?q=John+Ayto+and+Oxford+University+Press%2E+2009%2E+Oxford+Dictionary+of+English+Idioms+%2F+%5Bedited%5D+by+John+Ayto%2C+3rd+edition%2E+Oxford+University+Press+%5BOxford%5D%2E)
6. Timothy Baldwin and Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya and Fred J. Damerau, editors, Handbook of Natural Language Processing, Second Edition, pages 267–292. Chapman and Hall/CRC. [find](https://scholar.google.com/scholar?q=Timothy+Baldwin+and+Su+Nam+Kim%2E+2010%2E+Multiword+expressions%2E+In+Nitin+Indurkhya+and+Fred+J%2E+Damerau%2C+editors%2C+Handbook+of+Natural+Language+Processing%2C+Second+Edition%2C+pages+267%E2%80%93292%2E+Chapman+and+Hall%2FC)
7. Ankur Bapna and Orhan Firat. 2019. [Simple, scalable adaptation for neural machine translation](https://md.research.elamur.ai/free/en/papers/bapna-2019-simple-scalable-adaptation-neural-machine/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1538–1548, Hong Kon…
8. Rhys Biddle, Aditya Joshi, Shaowu Liu, Cecile Paris, and Guandong Xu. 2020. Leveraging sentiment distributions to distinguish figurative from literal health reports on Twitter. In Proceedings of The Web Conference 2020, pages 1217–1227. https://doi.org/10 .1145/3366423.3380198 [find](https://scholar.google.com/scholar?q=Rhys+Biddle%2C+Aditya+Joshi%2C+Shaowu+Liu%2C+Cecile+Paris%2C+and+Guandong+Xu%2E+2020%2E+Leveraging+sentiment+distributions+to+distinguish+figurative+from+literal+health+reports+on+Twitter%2E+In+Proceedings+of+The+W)
9. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, SandhiniAgarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
10. Tuhin Chakrabarty, Debanjan Ghosh, Adam Poliak, and Smaranda Muresan. 2021. [Figurative language in recognizing textual entailment](https://md.research.elamur.ai/free/en/papers/chakrabarty-2021-figurative-language-recognizing-textual-entailment/). In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3354–3361, Online. Association for Computational Linguistics. https://doi .org/10.18…
11. Paul Cook, Afsaneh Fazly, and Suzanne Stevenson. 2008. The VNC-tokens dataset. In Proceedings of the LREC Workshop Towards a Shared Task for Multiword Expressions (MWE 2008), pages 19–22. [find](https://scholar.google.com/scholar?q=Paul+Cook%2C+Afsaneh+Fazly%2C+and+Suzanne+Stevenson%2E+2008%2E+The+VNC%2Dtokens+dataset%2E+In+Proceedings+of+the+LREC+Workshop+Towards+a+Shared+Task+for+Multiword+Expressions+%28MWE+2008%29%2C+pages+19%E2%80%9322%2E)
12. Silvio Cordeiro, Carlos Ramisch, Marco Idiart, and Aline Villavicencio. 2016. Predicting the compositionality of nominal compounds: Giving word embeddings a hard time. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1986–1997.… [find](https://scholar.google.com/scholar?q=Silvio+Cordeiro%2C+Carlos+Ramisch%2C+Marco+Idiart%2C+and+Aline+Villavicencio%2E+2016%2E+Predicting+the+compositionality+of+nominal+compounds%3A+Giving+word+embeddings+a+hard+time%2E+In+Proceedings+of+the+54th+Annua)
13. Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2018. Examining the tip of the iceberg: A data set for idiom translation. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Marzieh+Fadaee%2C+Arianna+Bisazza%2C+and+Christof+Monz%2E+2018%2E+Examining+the+tip+of+the+iceberg%3A+A+data+set+for+idiom+translation%2E+In+Proceedings+of+the+Eleventh+International+Conference+on+Language+Resour)
14. Afsaneh Fazly, Paul Cook, and Suzanne Stevenson. 2009. [Unsupervised type and token identification of idiomatic expressions](https://md.research.elamur.ai/free/en/papers/fazly-2009-unsupervised-type-token-identification-idiomatic/). Computational Linguistics, 35(1):61–103. https://doi.org/10.1162/coli.08 -010-R1-07-048 [↗](https://doi.org/10.1162/coli.08)
15. Afsaneh Fazly and Suzanne Stevenson. 2006. [Automatically constructing a lexicon of verb phrase idiomatic combinations](https://md.research.elamur.ai/free/en/papers/fazly-2006-automatically-constructing-lexicon-verb-phrase/). In 11th Conference of the European Chapter of the Association for Computational Linguistics.
16. Anna Feldman and Jing Peng. 2013. Automatic detection of idiomatic clauses. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 435–446. Springer. https://doi.org/10.1007/978-3-642-37247-6_35 [↗](https://doi.org/10.1007/978-3-642-37247-6_35)
17. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. [PPDB: The paraphrase database](https://md.research.elamur.ai/free/en/papers/ganitkevitch-2013-ppdb-paraphrase-database/). In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 758–764.
18. Sahar Ghannay, Benoit Favre, Yannick Estève, and Nathalie Camelin. 2016. Word embedding evaluation and combination. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 300–305, Portorož, Slovenia. European Language Resources Association (ELRA)… [find](https://scholar.google.com/scholar?q=Sahar+Ghannay%2C+Benoit+Favre%2C+Yannick+Este%CC%80ve%2C+and+Nathalie+Camelin%2E+2016%2E+Word+embedding+evaluation+and+combination%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resources+and+Eval)
19. Hessel Haagsma, Johan Bos, and Malvina Nissim. 2020. [MAGPIE: A large corpus of potentially idiomatic expressions](https://md.research.elamur.ai/free/en/papers/haagsma-2020-magpie-large-corpus-potentially-idiomatic/). In Proceedings of the 12th Language Resources and Evaluation Conference, pages 279–287.
20. Reyhaneh Hashempour and Aline Villavicencio. 2020. Leveraging contextual embeddings and idiom principle for detecting idiomaticity in potentially idiomatic expressions. In Proceedings of the Workshop on the Cognitive Aspects of the Lexicon, pages 472–80, Online. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Reyhaneh+Hashempour+and+Aline+Villavicencio%2E+2020%2E+Leveraging+contextual+embeddings+and+idiom+principle+for+detecting+idiomaticity+in+potentially+idiomatic+expressions%2E+In+Proceedings+of+the+Workshop+)
21. Dominican Republic. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2021.emnlp-main.592 [find](https://scholar.google.com/scholar?q=Dominican+Republic%2E+Association+for+Computational+Linguistics%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E18653%2Fv1%2F2021%2Eemnlp%2Dmain%2E592)
22. Kazuma Hashimoto and Yoshimasa Tsuruoka. 2016. Adaptive joint learning of compositional and non-compositional phrase embeddings. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 205–215, Berlin, Germany. Association for Computa… [find](https://scholar.google.com/scholar?q=Kazuma+Hashimoto+and+Yoshimasa+Tsuruoka%2E+2016%2E+Adaptive+joint+learning+of+compositional+and+non%2Dcompositional+phrase+embeddings%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Comput)
23. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. [BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/). In Proceedings of the 58th Annual Meeting of the…
24. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Ma… [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+De+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+NLP%2E+In+)
25. Bing Li, Xiaochun Yang, Bin Wang, Wei Wang, Wei Cui, and Xianchao Zhang. 2018a. An adaptive hierarchical compositional model for phrase embedding. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 4144–4151. International Joint Conference… [find](https://scholar.google.com/scholar?q=Bing+Li%2C+Xiaochun+Yang%2C+Bin+Wang%2C+Wei+Wang%2C+Wei+Cui%2C+and+Xianchao+Zhang%2E+2018a%2E+An+adaptive+hierarchical+compositional+model+for+phrase+embedding%2E+In+Proceedings+of+the+Twenty%2DSeventh+International+Jo)
26. Dieuwke Hupkes and Willem Zuidema. 2018. Visualisation and ‘diagnostic classifiers’ reveal how recurrent and recursive neural networks process hierarchical structure (extended abstract). In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 5… — [doi:10.24963/ijcai.2018/796](https://doi.org/10.24963/ijcai.2018/796) [find](https://scholar.google.com/scholar?q=Dieuwke+Hupkes+and+Willem+Zuidema%2E+2018%2E+Visualisation+and+%E2%80%98diagnostic+classifiers%E2%80%99+reveal+how+recurrent+and+recursive+neural+networks+process+hierarchical+structure+%28extended+abstract%29%2E+In+Proceeding)
27. Abhik Jana, Dima Puzyrev, Alexander Panchenko, Pawan Goyal, Chris Biemann, and Animesh Mukherjee. 2019. On the compositionality prediction of noun phrases using poincaré embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3263–3274, Florence… [find](https://scholar.google.com/scholar?q=Abhik+Jana%2C+Dima+Puzyrev%2C+Alexander+Panchenko%2C+Pawan+Goyal%2C+Chris+Biemann%2C+and+Animesh+Mukherjee%2E+2019%2E+On+the+compositionality+prediction+of+noun+phrases+using+poincare%CC%81+embeddings%2E+In+Proceedings+of)
28. Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Taylor Berg-Kirkpatrick. 2021. Investigating robustness of dialog models to popular figurative language constructs. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7476–7485, Online and Punta Cana, Mingle… [find](https://scholar.google.com/scholar?q=Harsh+Jhamtani%2C+Varun+Gangal%2C+Eduard+Hovy%2C+and+Taylor+Berg%2DKirkpatrick%2E+2021%2E+Investigating+robustness+of+dialog+models+to+popular+figurative+language+constructs%2E+In+Proceedings+of+the+2021+Conference)
29. Zhenxi Lin, Qianli Ma, Jiangyue Yan, and Jieyu Chen. 2021. CATE: A contrastive pre-trained model for metaphor detection with semi-supervised learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3888–3898, Online and Punta Cana, Dominican Republic… [find](https://scholar.google.com/scholar?q=Zhenxi+Lin%2C+Qianli+Ma%2C+Jiangyue+Yan%2C+and+Jieyu+Chen%2E+2021%2E+CATE%3A+A+contrastive+pre%2Dtrained+model+for+metaphor+detection+with+semi%2Dsupervised+learning%2E+In+Proceedings+of+the+2021+Conference+on+Empirica)
30. Changsheng Liu. 2019. Toward Robust and Efficient Interpretations of Idiomatic Expressions in Context. Ph.D. thesis, University of Pittsburgh. [find](https://scholar.google.com/scholar?q=Changsheng+Liu%2E+2019%2E+Toward+Robust+and+Efficient+Interpretations+of+Idiomatic+Expressions+in+Context%2E+Ph%2ED%2E+thesis%2C+University+of+Pittsburgh%2E)
31. Changsheng Liu and Rebecca Hwa. 2017. Representations of context in recognizing the figurative and literal usages of idioms. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31. https://doi.org/10.1609/aaai.v31i1.10998 — [doi:10.1609/aaai.v31i1.10998](https://doi.org/10.1609/aaai.v31i1.10998) [↗](https://doi.org/10.1609/aaai.v31i1.10998)
32. Changsheng Liu and Rebecca Hwa. 2019. A generalized idiom usage recognition model based on semantic compatibility. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6738–6745. https://doi.org/10.1609/aaai.v33i01 .33016738 in Natural Language Processing (EMNLP), page… [↗](https://doi.org/10.1609/aaai.v33i01)
33. Pengfei Liu, Kaiyu Qian, Xipeng Qiu, and Xuan-Jing Huang. 2017. Idiom-aware compositional distributed semantics. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1204–1213. [find](https://scholar.google.com/scholar?q=Pengfei+Liu%2C+Kaiyu+Qian%2C+Xipeng+Qiu%2C+and+Xuan%2DJing+Huang%2E+2017%2E+Idiom%2Daware+compositional+distributed+semantics%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processi)
34. Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020a. [Adapterhub: A framework for adapting transformers](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterhub-framework-adapting-transformers/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020): Systems…
35. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. [Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/). ArXiv, abs/2107.13586.
36. Rosamund Moon. 1998. Fixed Expressions and Idioms in English: A Corpus-Based Approach, Oxford University Press. [find](https://scholar.google.com/scholar?q=Rosamund+Moon%2E+1998%2E+Fixed+Expressions+and+Idioms+in+English%3A+A+Corpus%2DBased+Approach%2C+Oxford+University+Press%2E)
37. Vasudevan Nedumpozhimana, Filip Klubička, and John D. Kelleher. 2022. Shapley idioms: Analysing BERT sentence embeddings for general idiom token identification. Frontiers in Artificial Intelligence, 5. https://doi.org/10.3389/frai.2022.813967, PubMed: 35360661 — [doi:10.3389/frai.2022.813967](https://doi.org/10.3389/frai.2022.813967) [↗](https://doi.org/10.3389/frai.2022.813967)
38. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:282… [find](https://scholar.google.com/scholar?q=F%2E+Pedregosa%2C+G%2E+Varoquaux%2C+A%2E+Gramfort%2C+V%2E+Michel%2C+B%2E+Thirion%2C+O%2E+Grisel%2C+M%2E+Blondel%2C+P%2E+Prettenhofer%2C+R%2E+Weiss%2C+V%2E+Dubourg%2C+J%2E+Vanderplas%2C+A%2E+Passos%2C+D%2E+Cournapeau%2C+M%2E+Brucher%2C+M%2E+Perrot%2C+and+E%2E+Duc)
39. Jing Peng and Anna Feldman. 2016. Automatic idiom recognition with word embeddings. In Information Management and Big Data - Second Annual International Symposium, SIMBig 2015, Cusco, Peru, September 2-4, 2015, and Third Annual International Symposium, SIMBig 2016, Cusco, Peru, September 1-3, 2016,… — [doi:10.1007/978-3-319-55209-5_2](https://doi.org/10.1007/978-3-319-55209-5_2) [find](https://scholar.google.com/scholar?q=Jing+Peng+and+Anna+Feldman%2E+2016%2E+Automatic+idiom+recognition+with+word+embeddings%2E+In+Information+Management+and+Big+Data+%2D+Second+Annual+International+Symposium%2C+SIMBig+2015%2C+Cusco%2C+Peru%2C+September+)
40. Jing Peng, Anna Feldman, and Ekaterina Vylomova. 2014. [Classifying idiomatic and literal expressions using topic models and intensity of emotions](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-mad-x-adapter-based-framework/). In Proceedings of the 2014 Conference on Empirical Methods Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020b. MAD-X: An adapter-bas…
41. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In ICML.
42. Abhilasha Ravichander, Yonatan Belinkov, and Eduard Hovy. 2021. [Probing the probing paradigm: Does probing accuracy entail task relevance?](https://md.research.elamur.ai/free/en/papers/ravichander-2021-probing-probing-paradigm-does-probing/) In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3363–3377, Online. Association…
43. Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. [Learning multiple visual domains with residual adapters](https://md.research.elamur.ai/free/en/papers/rebuffi-2017-learning-multiple-visual-domains-residual/). In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
44. Sylvestre-Alvise Rebuffi, Andrea Vedaldi, and Hakan Bilen. 2018. Efficient parametrization of multi-domain deep neural networks. In 2018 [find](https://scholar.google.com/scholar?q=Sylvestre%2DAlvise+Rebuffi%2C+Andrea+Vedaldi%2C+and+Hakan+Bilen%2E+2018%2E+Efficient+parametrization+of+multi%2Ddomain+deep+neural+networks%2E+In+2018)
45. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8119–8127. [find](https://scholar.google.com/scholar?q=IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pages+8119%E2%80%938127%2E)
46. Siva Reddy, Diana McCarthy, and Suresh Manandhar. 2011. [An empirical study on compositionality in compound nouns](https://md.research.elamur.ai/free/en/papers/reddy-2011-empirical-study-compositionality-compound-nouns/). In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 210–218. Asian Federation of Natural Language Processing.
47. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-1410 [↗](https://doi.org/10.18653/v1/D19-1410)
48. Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. [Multiword expressions: A pain in the neck for NLP](https://md.research.elamur.ai/free/en/papers/sag-2002-multiword-expressions-pain-neck-nlp/). In International Conference on Intelligent Text Processing and Computational Linguistics, pages 1–15. Springer. https://doi.org/10.1007/3-540-45715-1_1 [↗](https://doi.org/10.1007/3-540-45715-1_1)
49. Giancarlo Salton, Robert Ross, and John Kelleher. 2014. An empirical study of the impact of idioms on phrase based statistical machine translation of English to Brazilian-Portuguese. In Proceedings of the 3rd Workshop on Hybrid Approaches to Machine Translation (HyTra), pages 36–41. Association for… [find](https://scholar.google.com/scholar?q=Giancarlo+Salton%2C+Robert+Ross%2C+and+John+Kelleher%2E+2014%2E+An+empirical+study+of+the+impact+of+idioms+on+phrase+based+statistical+machine+translation+of+English+to+Brazilian%2DPortuguese%2E+In+Proceedings+of)
50. Giancarlo Salton, Robert Ross, and John Kelleher. 2016. [Idiom token classification using sentential distributed semantics](https://md.research.elamur.ai/free/en/papers/salton-2016-idiom-token-classification-sentential-distributed/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 194–204. https://doi.org/10.18653/v1/P16-1019 pression… [↗](https://doi.org/10.18653/v1/P16-1019)
51. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Lin…
52. Ekaterina Shutova, Lin Sun, and Anna Korhonen. 2010. [Metaphor identification using verb and noun clustering](https://md.research.elamur.ai/free/en/papers/shutova-2010-metaphor-identification-verb-noun-clustering/). In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1002–1010.
53. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. [Mpnet: Masked and permuted pre-training for language understanding](https://md.research.elamur.ai/free/en/papers/song-2020-mpnet-masked-permuted-pre-training/). In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
54. Patrizia Tabossi, Rachele Fanari, and Kinou Wolf. 2008. Processing idiomatic expressions: Effects of semantic compositionality. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(2):313. https://doi.org/10.1037/0278-7393.34.2.313, PubMed: 18315408 — [doi:10.1037/0278-7393.34.2.313](https://doi.org/10.1037/0278-7393.34.2.313) [↗](https://doi.org/10.1037/0278-7393.34.2.313)
55. Patrizia Tabossi, Rachele Fanari, and Kinou Wolf. 2009. Why are idioms recognized fast? Memory & Cognition, 37(4):529–540. https://doi.org/10.3758/MC.37.4.529, PubMed: 19460959 — [doi:10.3758/mc.37.4.529](https://doi.org/10.3758/mc.37.4.529) [↗](https://doi.org/10.3758/MC.37.4.529)
56. Tobias Schnabel, Igor Labutov, David Mimno, and Thorsten Joachims. 2015. [Evaluation methods for unsupervised word embeddings](https://md.research.elamur.ai/free/en/papers/schnabel-2015-evaluation-methods-unsupervised-word-embeddings/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 298–307, Lisbon, Portugal. Association for Computational Linguistics. https:/…
57. Shiva Taslimipoor, Omid Rohanian, Ruslan Mitkov, and Afsaneh Fazly. 2018. Identification of multiword expressions: A fresh look at modelling and evaluation. In Multiword Expressions at Length and in Depth: Extended Papers from the MWE 2017 Workshop, volume 2, page 299. Language Science Press. [find](https://scholar.google.com/scholar?q=Shiva+Taslimipoor%2C+Omid+Rohanian%2C+Ruslan+Mitkov%2C+and+Afsaneh+Fazly%2E+2018%2E+Identification+of+multiword+expressions%3A+A+fresh+look+at+modelling+and+evaluation%2E+In+Multiword+Expressions+at+Length+and+in+D)
58. Nathan Schneider and Noah A. Smith. 2015. A corpus and model integrating multiword ex-Harish Tayyar Madabushi, Edward Gow-Smith, Carolina Scarton, and Aline Villavicencio. 2021. [AStitchInLanguageModels: Dataset and methods for the exploration of idiomaticity in pre-trained language models](https://md.research.elamur.ai/free/en/papers/madabushi-2021-astitchinlanguagemodels-dataset-methods-exploration-idiomaticity/). In Findin…
59. Bin Wang, Angela Wang, Fenxiao Chen, Yuncheng Wang, and C.-C. Jay Kuo. 2019. Evaluating word embedding models: methods and experimental results. APSIPA Transactions on Signal and Information Processing, 8:e19. https://doi.org/10.1017/ATSIP.2019.12 [↗](https://doi.org/10.1017/ATSIP.2019.12)
60. Alex Warstadt, Yu Cao, Ioana Grosu, Wei Peng, Hagen Blix, Yining Nie, Anna Alsop, Shikha Bordia, Haokun Liu, Alicia Parrish, Sheng-Fu Wang, Jason Phang, Anhad Mohananey, Phu Mon Htut, Paloma Jeretic, and Samuel R. Bowman. 2019. [Investigating BERT’s knowledge of language: Five analysis methods with N…](https://md.research.elamur.ai/free/en/papers/warstadt-2019-investigating-bert-s-knowledge-language/)
61. Dag Westerståhl. 2002. On the compositionality of idioms. In Proceedings of LLC8. CSLI Publications. [find](https://scholar.google.com/scholar?q=Dag+Westersta%CC%8Ahl%2E+2002%2E+On+the+compositionality+of+idioms%2E+In+Proceedings+of+LLC8%2E+CSLI+Publications%2E)
62. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho… — [Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
63. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, N… [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V%2E+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+Jeff+Klingner%2C+Apurva+Shah%2C+Melvin+Johnson%2C+Xiaobing+Liu%2C+Luk)
64. W. Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, and Meng Jiang. 2021. Dict-BERT: Enhancing language model pre-training with dictionary. ArXiv, abs/2110.06490. https://doi.org/10.18653/v1/2022.findings -acl.150 [↗](https://doi.org/10.18653/v1/2022.findings)
65. Ziheng Zeng and Suma Bhat. 2021. [Idiomatic expression identification using semantic compatibility](https://md.research.elamur.ai/free/en/papers/zeng-2021-idiomatic-expression-identification-semantic-compatibility/). Transactions of the Association for Computational Linguistics, 9:1546–1562. https://doi.org/10.1162/tacl a 00442 [↗](https://doi.org/10.1162/tacl)
66. Jianing Zhou, Ziheng Zeng, Hongyu Gong, and Suma Bhat. 2021. Idiomatic expression paraphrasing without strong supervision. https://doi.org/10.1609/aaai.v36i10.21433 [↗](https://doi.org/10.1609/aaai.v36i10.21433)
67. Tadej Škvorc, Polona Gantar, and Marko Robnik-Šikonja. 2022. Mice: Mining idioms with contextual embeddings. Knowledge-Based Systems, 235:107606. https://doi.org/10.1016/j.knosys.2021.107606 — [doi:10.1016/j.knosys.2021.107606](https://doi.org/10.1016/j.knosys.2021.107606) [↗](https://doi.org/10.1016/j.knosys.2021.107606)

*Updated 30 October 2026*