---
title: "No Context Needed — Cheng 2024"
description: "Idiomatic expressions (IEs) are hard for NLP systems because their figurative sense usually cannot be derived from the individual words."
authors: ["Kellen Tan Cheng", "Suma Bhat"]
year: 2024
venue: "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)"
source: "https://aclanthology.org/2024.naacl-long.272/"
doi: "10.18653/v1/2024.naacl-long.272"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/cheng-2024-no-context-needed-contextual-quandary/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/cheng-2024-no-context-needed-contextual-quandary/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2024](https://md.research.elamur.ai/free/papers/2024/page/3/) / No Context Needed: Contextual Quandary In Idiomatic Reasoning With Pre-Trained Language Models

# No Context Needed: Contextual Quandary In Idiomatic Reasoning With Pre-Trained Language Models

**Authors:** [Kellen Tan Cheng](https://md.research.elamur.ai/free/authors/kellen-tan-cheng/), [Suma Bhat](https://md.research.elamur.ai/free/authors/suma-bhat/) · **Year:** 2024 · **Venue:** Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.naacl-long.272/) · [PDF](https://aclanthology.org/2024.naacl-long.272.pdf) · [DOI 10.18653/v1/2024.naacl-long.272](https://doi.org/10.18653/v1/2024.naacl-long.272) · [OpenAlex](https://openalex.org/W4401042548)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14) · 3 in our library, of the 140673 papers whose bibliographies we read

## Summary
Idiomatic expressions (IEs) are hard for NLP systems because their figurative sense usually cannot be derived from the individual words. One would therefore expect pre-trained language models (PTLMs) to rely on the surrounding text, as human readers do. The authors test this assumption with a data-centric design: the models stay fixed and only the inputs change.

They use two English classification tasks:
- idiomatic NLI on IMPLI;
- idiomatic continuation acceptability, built from the FigurativeNarrativeBenchmark.

The inputs are perturbed in several ways: deleting context, deleting both sentential and extra-sentential context, shuffling non-idiom words, and trimming narrative words starting from the front. The tested models range from 140M to 7B parameters and include both general-purpose and IE-aware variants.

Contrary to expectations, general PTLMs usually score higher once context is removed, by up to 3.89%. The authors argue that good benchmark numbers on idiom tasks therefore do not show human-like figurative reasoning. They recommend IE-aware models for such tasks and call for more careful construction of idiom datasets.

## Key points
- **IMPLI (idiomatic NLI):** 13,650 balanced training samples come from the silver split. Evaluation uses the gold split of 1,157 hand-crafted pairs: 528 entailment, 254 ordinary non-entailment and 375 antonym non-entailment.
- **FigurativeNarrativeBenchmark (continuation acceptability):** each sample is split into a correct and an incorrect narrative–continuation pair, giving 6,408 training and 3,084 test examples, both class-balanced. FLUTE results appear in an appendix.
- **Models:** BART-large, BART-IEKG, BART-MNLI, BART-MNLI-IEKG (IEKG-injected, from Zeng et al., 2023), Mistral-7B, Mistral-FT (only the classification head fine-tuned) and the IE-aware PIER+ (BART-base, 140M).
- **Main finding:** dropping context almost always helps general PTLMs, with a maximum gain of 3.89%. IE-aware models behave more in line with human-like use of context.
- **Limitations:** the study covers only English classification tasks, and compute limits prevented full fine-tuning of Mistral-7B. Code is available at github.com/kellentan/No_Contexts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{cheng2024no,
  title = {{No Context Needed: Contextual Quandary In Idiomatic Reasoning With Pre-Trained Language Models}},
  author = {Cheng, Kellen Tan and Bhat, Suma},
  booktitle = {Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.272},
  url = {https://doi.org/10.18653/v1/2024.naacl-long.272}
}
```

- [← previous Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://md.research.elamur.ai/free/papers/zhang-2024-negative-preference-optimization-catastrophic-collapse/)
- [next → NoLoR: An ASR-Based Framework for Expedited Endangered Language Documentation with Neo-Aramaic as a Case Study](https://md.research.elamur.ai/free/papers/langstream-2024-nazari-nolor-asr-based-framework-expedited/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/papers/2024/page/3/)

## References

59 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Tosin Adewumi, Foteini Liwicki, and Marcus Liwicki. 2022. Vector representations of idioms in conversational systems. Sci, 4(4). [find](https://scholar.google.com/scholar?q=Tosin+Adewumi%2C+Foteini+Liwicki%2C+and+Marcus+Liwicki%2E+2022%2E+Vector+representations+of+idioms+in+conversational+systems%2E+Sci%2C+4%284%29%2E)
2. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. The falcon series of open language mo… [find](https://scholar.google.com/scholar?q=Ebtesam+Almazrouei%2C+Hamza+Alobeidli%2C+Abdulaziz+Alshamsi%2C+Alessandro+Cappelli%2C+Ruxandra+Cojocaru%2C+M%C3%A9rouane+Debbah%2C+%C3%89tienne+Goffinet%2C+Daniel+Hesslow%2C+Julien+Launay%2C+Quentin+Malartic%2C+Daniele+Mazzotta%2C+B) [our card](https://md.research.elamur.ai/free/papers/almazrouei-2023-falcon-series-open-language-models/)
3. Adrien Bibal, Rémi Cardon, David Alfter, Rodrigo Wilkens, Xiaoou Wang, Thomas François, and Patrick Watrin. 2022. Is attention explanation? an introduction to the debate. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3889–39… [find](https://scholar.google.com/scholar?q=Adrien+Bibal%2C+R%C3%A9mi+Cardon%2C+David+Alfter%2C+Rodrigo+Wilkens%2C+Xiaoou+Wang%2C+Thomas+Fran%C3%A7ois%2C+and+Patrick+Watrin%2E+2022%2E+Is+attention+explanation%3F+an+introduction+to+the+debate%2E+In+Proceedings+of+the+60th+An)
4. C. Cacciari and P. Tabossi. 2014. Idioms: Processing, Structure, and Interpretation. Taylor & Francis. [find](https://scholar.google.com/scholar?q=C%2E+Cacciari+and+P%2E+Tabossi%2E+2014%2E+Idioms%3A+Processing%2C+Structure%2C+and+Interpretation%2E+Taylor+%26+Francis%2E)
5. Kate Cain, Andrea S. Towse, and Rachael S. Knight. 2009. The development of idiom comprehension: An investigation of semantic and contextual processing skills. Journal of Experimental Child Psychology, 102(3):280–298. [find](https://scholar.google.com/scholar?q=Kate+Cain%2C+Andrea+S%2E+Towse%2C+and+Rachael+S%2E+Knight%2E+2009%2E+The+development+of+idiom+comprehension%3A+An+investigation+of+semantic+and+contextual+processing+skills%2E+Journal+of+Experimental+Child+Psychology)
6. Tuhin Chakrabarty, Yejin Choi, and Vered Shwartz. 2022a. It’s not rocket science: Interpreting figurative language in narratives. Transactions of the Association for Computational Linguistics, 10:589–606. [find](https://scholar.google.com/scholar?q=Tuhin+Chakrabarty%2C+Yejin+Choi%2C+and+Vered+Shwartz%2E+2022a%2E+It%E2%80%99s+not+rocket+science%3A+Interpreting+figurative+language+in+narratives%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+10%3A589%E2%80%93) [our card](https://md.research.elamur.ai/free/papers/chakrabarty-2022-it-s-not-rocket-science/)
7. Tuhin Chakrabarty, Arkadiy Saakyan, Debanjan Ghosh, and Smaranda Muresan. 2022b. FLUTE: Figurative language understanding through textual explanations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7139–7159, Abu Dhabi, United Arab Emirates. Associa… [find](https://scholar.google.com/scholar?q=Tuhin+Chakrabarty%2C+Arkadiy+Saakyan%2C+Debanjan+Ghosh%2C+and+Smaranda+Muresan%2E+2022b%2E+FLUTE%3A+Figurative+language+understanding+through+textual+explanations%2E+In+Proceedings+of+the+2022+Conference+on+Empiric)
8. Maria Chiara Levorato, Barbara Nesi, and Cristina Cacciari. 2004. Reading comprehension and understanding idiomatic expressions: A developmental study. Brain and Language, 91(3):303–314. [find](https://scholar.google.com/scholar?q=Maria+Chiara+Levorato%2C+Barbara+Nesi%2C+and+Cristina+Cacciari%2E+2004%2E+Reading+comprehension+and+understanding+idiomatic+expressions%3A+A+developmental+study%2E+Brain+and+Language%2C+91%283%29%3A303%E2%80%93314%2E)
9. Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan, Hannah R. Sheahan, Antonia Creswell, Dharshan Kumaran, James L. McClelland, and Felix Hill. 2023. Language models show human-like content effects on reasoning tasks. [find](https://scholar.google.com/scholar?q=Ishita+Dasgupta%2C+Andrew+K%2E+Lampinen%2C+Stephanie+C%2E+Y%2E+Chan%2C+Hannah+R%2E+Sheahan%2C+Antonia+Creswell%2C+Dharshan+Kumaran%2C+James+L%2E+McClelland%2C+and+Felix+Hill%2E+2023%2E+Language+models+show+human%2Dlike+content+eff)
10. Mark Davies. 2010. The Corpus of Contemporary American English as the first reliable monitor corpus of English. Literary and Linguistic Computing, 25(4):447–464. [find](https://scholar.google.com/scholar?q=Mark+Davies%2E+2010%2E+The+Corpus+of+Contemporary+American+English+as+the+first+reliable+monitor+corpus+of+English%2E+Literary+and+Linguistic+Computing%2C+25%284%29%3A447%E2%80%93464%2E)
11. Hessel Haagsma, Johan Bos, and Malvina Nissim. 2020. MAGPIE: A large corpus of potentially idiomatic expressions. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 279–287, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Hessel+Haagsma%2C+Johan+Bos%2C+and+Malvina+Nissim%2E+2020%2E+MAGPIE%3A+A+large+corpus+of+potentially+idiomatic+expressions%2E+In+Proceedings+of+the+Twelfth+Language+Resources+and+Evaluation+Conference%2C+pages+279%E2%80%93) [our card](https://md.research.elamur.ai/free/papers/haagsma-2020-magpie-large-corpus-potentially-idiomatic/)
12. Zellig S. Harris. 1954. Distributional structure. WORD, 10(2-3):146–162. [find](https://scholar.google.com/scholar?q=Zellig+S%2E+Harris%2E+1954%2E+Distributional+structure%2E+WORD%2C+10%282%2D3%29%3A146%E2%80%93162%2E)
13. Hrayr Harutyunyan, Alessandro Achille, Giovanni Paolini, Orchid Majumder, Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. 2021. Estimating informativeness of samples with smooth unique information. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hrayr+Harutyunyan%2C+Alessandro+Achille%2C+Giovanni+Paolini%2C+Orchid+Majumder%2C+Avinash+Ravichandran%2C+Rahul+Bhotika%2C+and+Stefano+Soatto%2E+2021%2E+Estimating+informativeness+of+samples+with+smooth+unique+inform)
14. Kazuma Hashimoto and Yoshimasa Tsuruoka. 2016. Adaptive joint learning of compositional and non-compositional phrase embeddings. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 205–215, Berlin, Germany. Association for Computa… [find](https://scholar.google.com/scholar?q=Kazuma+Hashimoto+and+Yoshimasa+Tsuruoka%2E+2016%2E+Adaptive+joint+learning+of+compositional+and+non%2Dcompositional+phrase+embeddings%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Comput)
15. Sarthak Jain and Byron C. Wallace. 2019. Attention is not Explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3543–3556, Minneapolis, Minnesota. Associa… [find](https://scholar.google.com/scholar?q=Sarthak+Jain+and+Byron+C%2E+Wallace%2E+2019%2E+Attention+is+not+Explanation%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Langu) [our card](https://md.research.elamur.ai/free/papers/jain-2019-attention-is-not-explanation/)
16. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
17. Ioannis Korkontzelos, Torsten Zesch, Fabio Massimo Zanzotto, and Chris Biemann. 2013. SemEval-2013 task 5: Evaluating phrasal semantics. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEv… [find](https://scholar.google.com/scholar?q=Ioannis+Korkontzelos%2C+Torsten+Zesch%2C+Fabio+Massimo+Zanzotto%2C+and+Chris+Biemann%2E+2013%2E+SemEval%2D2013+task+5%3A+Evaluating+phrasal+semantics%2E+In+Second+Joint+Conference+on+Lexical+and+Computational+Semanti)
18. Maria Chiara Levorato and Cristina Cacciari. 1992. Children’s comprehension and production of idioms: the role of context and familiarity. Journal of Child Language, 19(2):415–433. [find](https://scholar.google.com/scholar?q=Maria+Chiara+Levorato+and+Cristina+Cacciari%2E+1992%2E+Children%E2%80%99s+comprehension+and+production+of+idioms%3A+the+role+of+context+and+familiarity%2E+Journal+of+Child+Language%2C+19%282%29%3A415%E2%80%93433%2E)
19. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the… [find](https://scholar.google.com/scholar?q=Mike+Lewis%2C+Yinhan+Liu%2C+Naman+Goyal%2C+Marjan+Ghazvininejad%2C+Abdelrahman+Mohamed%2C+Omer+Levy%2C+Veselin+Stoyanov%2C+and+Luke+Zettlemoyer%2E+2020%2E+BART%3A+Denoising+sequence%2Dto%2Dsequence+pre%2Dtraining+for+natural+l) [our card](https://md.research.elamur.ai/free/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
20. Changsheng Liu. 2019. Toward robust and efficient interpretations of idiomatic expressions in context. [find](https://scholar.google.com/scholar?q=Changsheng+Liu%2E+2019%2E+Toward+robust+and+efficient+interpretations+of+idiomatic+expressions+in+context%2E)
21. Changsheng Liu and Rebecca Hwa. 2017. Representations of context in recognizing the figurative and literal usages of idioms. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). [find](https://scholar.google.com/scholar?q=Changsheng+Liu+and+Rebecca+Hwa%2E+2017%2E+Representations+of+context+in+recognizing+the+figurative+and+literal+usages+of+idioms%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+31%281%29%2E)
22. Emmy Liu, Chenxuan Cui, Kenneth Zheng, and Graham Neubig. 2022. Testing the ability of language models to interpret figurative language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4437–4452,… [find](https://scholar.google.com/scholar?q=Emmy+Liu%2C+Chenxuan+Cui%2C+Kenneth+Zheng%2C+and+Graham+Neubig%2E+2022%2E+Testing+the+ability+of+language+models+to+interpret+figurative+language%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Cha)
23. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the middle: How language models use long contexts. [find](https://scholar.google.com/scholar?q=Nelson+F%2E+Liu%2C+Kevin+Lin%2C+John+Hewitt%2C+Ashwin+Paranjape%2C+Michele+Bevilacqua%2C+Fabio+Petroni%2C+and+Percy+Liang%2E+2023%2E+Lost+in+the+middle%3A+How+language+models+use+long+contexts%2E) [our card](https://md.research.elamur.ai/free/papers/liu-2023-lost-middle-language-models-use/)
24. Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7052–7063, Online and Punta Cana, Dominican Re… [find](https://scholar.google.com/scholar?q=Shayne+Longpre%2C+Kartik+Perisetla%2C+Anthony+Chen%2C+Nikhil+Ramesh%2C+Chris+DuBois%2C+and+Sameer+Singh%2E+2021%2E+Entity%2Dbased+knowledge+conflicts+in+question+answering%2E+In+Proceedings+of+the+2021+Conference+on+Em) [our card](https://md.research.elamur.ai/free/papers/longpre-2021-entity-based-knowledge-conflicts-question/)
25. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy. Association for Computational Ling… [find](https://scholar.google.com/scholar?q=Tom+McCoy%2C+Ellie+Pavlick%2C+and+Tal+Linzen%2E+2019%2E+Right+for+the+wrong+reasons%3A+Diagnosing+syntactic+heuristics+in+natural+language+inference%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association) [our card](https://md.research.elamur.ai/free/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/)
26. Syrielle Montariol, Aina Garí Soler, and Alexandre Allauzen. 2019. Exploring sentence informativeness. In Actes de la Conférence sur le Traitement Automatique des Langues Naturelles (TALN) PFIA 2019. Volume II : Articles courts, pages 303–312, Toulouse, France. ATALA. [find](https://scholar.google.com/scholar?q=Syrielle+Montariol%2C+Aina+Gar%C3%AD+Soler%2C+and+Alexandre+Allauzen%2E+2019%2E+Exploring+sentence+informativeness%2E+In+Actes+de+la+Conf%C3%A9rence+sur+le+Traitement+Automatique+des+Langues+Naturelles+%28TALN%29+PFIA+2019%2E+)
27. R. Moon. 1998. Fixed Expressions and Idioms in English: A Corpus-based Approach. Oxford Studies in Lexicography. Clarendon Press. [find](https://scholar.google.com/scholar?q=R%2E+Moon%2E+1998%2E+Fixed+Expressions+and+Idioms+in+English%3A+A+Corpus%2Dbased+Approach%2E+Oxford+Studies+in+Lexicography%2E+Clarendon+Press%2E)
28. Joe O’Connor and Jacob Andreas. 2021. What context features can transformer language models use? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 851–86… [find](https://scholar.google.com/scholar?q=Joe+O%E2%80%99Connor+and+Jacob+Andreas%2E+2021%2E+What+context+features+can+transformer+language+models+use%3F+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11th)
29. OpenAI. 2023. Gpt-4 technical report. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2023%2E+Gpt%2D4+technical+report%2E) [our card](https://md.research.elamur.ai/free/papers/openai-2023-gpt-4-technical-report/)
30. Andrew Ortony, Diane L. Schallert, Ralph E. Reynolds, and Stephen J. Antos. 1978. Interpreting metaphors and idioms: Some effects of context on comprehension. Journal of Verbal Learning and Verbal Behavior, 17(4):465–477. [find](https://scholar.google.com/scholar?q=Andrew+Ortony%2C+Diane+L%2E+Schallert%2C+Ralph+E%2E+Reynolds%2C+and+Stephen+J%2E+Antos%2E+1978%2E+Interpreting+metaphors+and+idioms%3A+Some+effects+of+context+on+comprehension%2E+Journal+of+Verbal+Learning+and+Verbal+Beh)
31. Isabel Papadimitriou, Richard Futrell, and Kyle Mahowald. 2022. When classifying grammatical role, BERT doesn’t care about word order... except when it matters. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 636–643, Dublin,… [find](https://scholar.google.com/scholar?q=Isabel+Papadimitriou%2C+Richard+Futrell%2C+and+Kyle+Mahowald%2E+2022%2E+When+classifying+grammatical+role%2C+BERT+doesn%E2%80%99t+care+about+word+order%2E%2E%2E+except+when+it+matters%2E+In+Proceedings+of+the+60th+Annual+Meeti)
32. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018. Hypothesis only baselines in natural language inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 180–191, New Orleans, Louisiana. Association for Compu… [find](https://scholar.google.com/scholar?q=Adam+Poliak%2C+Jason+Naradowsky%2C+Aparajita+Haldar%2C+Rachel+Rudinger%2C+and+Benjamin+Van+Durme%2E+2018%2E+Hypothesis+only+baselines+in+natural+language+inference%2E+In+Proceedings+of+the+Seventh+Joint+Conference+)
33. Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multiword expressions: A pain in the neck for nlp. In Computational Linguistics and Intelligent Text Processing, pages 1–15, Berlin, Heidelberg. Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Ivan+A%2E+Sag%2C+Timothy+Baldwin%2C+Francis+Bond%2C+Ann+Copestake%2C+and+Dan+Flickinger%2E+2002%2E+Multiword+expressions%3A+A+pain+in+the+neck+for+nlp%2E+In+Computational+Linguistics+and+Intelligent+Text+Processing%2C+pa) [our card](https://md.research.elamur.ai/free/papers/idioms-2002-sag-multiword-expressions-pain-neck-nlp/)
34. Timo Schick and Hinrich Schütze. 2019. Attentive mimicking: Better word embeddings by attending to informative contexts. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)… [find](https://scholar.google.com/scholar?q=Timo+Schick+and+Hinrich+Sch%C3%BCtze%2E+2019%2E+Attentive+mimicking%3A+Better+word+embeddings+by+attending+to+informative+contexts%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Asso)
35. Patrick Schramowski, Cigdem Turan-Schwiewager, Nico Andersen, Constantin Rothkopf, and Kristian Kersting. 2022. Large pre-trained language models contain human-like biases of what is right and wrong to do. Nature Machine Intelligence, 4:258–268. [find](https://scholar.google.com/scholar?q=Patrick+Schramowski%2C+Cigdem+Turan%2DSchwiewager%2C+Nico+Andersen%2C+Constantin+Rothkopf%2C+and+Kristian+Kersting%2E+2022%2E+Large+pre%2Dtrained+language+models+contain+human%2Dlike+biases+of+what+is+right+and+wrong+t)
36. Sofia Serrano and Noah A. Smith. 2019. Is attention interpretable? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2931–2951, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sofia+Serrano+and+Noah+A%2E+Smith%2E+2019%2E+Is+attention+interpretable%3F+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+2931%E2%80%932951%2C+Florence%2C+Italy%2E+Associa) [our card](https://md.research.elamur.ai/free/papers/serrano-2019-is-attention-interpretable/)
37. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2023. Language models are multilingual chain-of-thought reasoners. In The Eleventh International Conference on Learning Representa… [find](https://scholar.google.com/scholar?q=Freda+Shi%2C+Mirac+Suzgun%2C+Markus+Freitag%2C+Xuezhi+Wang%2C+Suraj+Srivats%2C+Soroush+Vosoughi%2C+Hyung+Won+Chung%2C+Yi+Tay%2C+Sebastian+Ruder%2C+Denny+Zhou%2C+Dipanjan+Das%2C+and+Jason+Wei%2E+2023%2E+Language+models+are+mult) [our card](https://md.research.elamur.ai/free/papers/shi-2022-language-models-multilingual-chain-thought/)
38. Koustuv Sinha, Jon Gauthier, Aaron Mueller, Kanishka Misra, Keren Fuentes, Roger Levy, and Adina Williams. 2023. Language model acceptability judgements are not always robust to context. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers… [find](https://scholar.google.com/scholar?q=Koustuv+Sinha%2C+Jon+Gauthier%2C+Aaron+Mueller%2C+Kanishka+Misra%2C+Keren+Fuentes%2C+Roger+Levy%2C+and+Adina+Williams%2E+2023%2E+Language+model+acceptability+judgements+are+not+always+robust+to+context%2E+In+Proceeding)
39. Kevin Stowe, Prasetya Utama, and Iryna Gurevych. 2022. IMPLI: Investigating NLI models’ performance on figurative language. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5375–5388, Dublin, Ireland. Association for Computatio… [find](https://scholar.google.com/scholar?q=Kevin+Stowe%2C+Prasetya+Utama%2C+and+Iryna+Gurevych%2E+2022%2E+IMPLI%3A+Investigating+NLI+models%E2%80%99+performance+on+figurative+language%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computation) [our card](https://md.research.elamur.ai/free/papers/stowe-2022-impli-investigating-nli-models-performance/)
40. Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, and Mohit Iyyer. 2021. Do long-range language models actually use long-range context? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 807– 822, Online and Punta Cana, Dominican Republic. Associatio… [find](https://scholar.google.com/scholar?q=Simeng+Sun%2C+Kalpesh+Krishna%2C+Andrew+Mattarella%2DMicke%2C+and+Mohit+Iyyer%2E+2021%2E+Do+long%2Drange+language+models+actually+use+long%2Drange+context%3F+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+i)
41. Minghuan Tan and Jing Jiang. 2021. Does BERT understand idioms? a probing-based empirical study of BERT encodings of idioms. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1397–1407, Held Online. INCOMA Ltd. [find](https://scholar.google.com/scholar?q=Minghuan+Tan+and+Jing+Jiang%2E+2021%2E+Does+BERT+understand+idioms%3F+a+probing%2Dbased+empirical+study+of+BERT+encodings+of+idioms%2E+In+Proceedings+of+the+International+Conference+on+Recent+Advances+in+Natura)
42. Harish Tayyar Madabushi, Edward Gow-Smith, Carolina Scarton, and Aline Villavicencio. 2021. AStitchInLanguageModels: Dataset and methods for the exploration of idiomaticity in pre-trained language models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3464–3477, Punt… [find](https://scholar.google.com/scholar?q=Harish+Tayyar+Madabushi%2C+Edward+Gow%2DSmith%2C+Carolina+Scarton%2C+and+Aline+Villavicencio%2E+2021%2E+AStitchInLanguageModels%3A+Dataset+and+methods+for+the+exploration+of+idiomaticity+in+pre%2Dtrained+language+mod) [our card](https://md.research.elamur.ai/free/papers/madabushi-2021-astitchinlanguagemodels-dataset-methods-exploration-idiomaticity/)
43. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+Dan+Bikel%2C+Lukas+Blecher%2C+Cristian+Canton+Fe)
44. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81+ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+Neural+Information+Process)
45. Meiling Wang, Min Xiao, Changliang Li, Yu Guo, Zhixin Zhao, and Xiaonan Liu. 2019. STAC: Science toolkit based on Chinese idiom knowledge graph. In Proceedings of the Workshop on Extracting Structured Knowledge from Scientific Publications, pages 57–61, Minneapolis, Minnesota. Association for Comput… [find](https://scholar.google.com/scholar?q=Meiling+Wang%2C+Min+Xiao%2C+Changliang+Li%2C+Yu+Guo%2C+Zhixin+Zhao%2C+and+Xiaonan+Liu%2E+2019%2E+STAC%3A+Science+toolkit+based+on+Chinese+idiom+knowledge+graph%2E+In+Proceedings+of+the+Workshop+on+Extracting+Structured)
46. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Xuezhi+Wang%2C+Jason+Wei%2C+Dale+Schuurmans%2C+Quoc+V+Le%2C+Ed+H%2E+Chi%2C+Sharan+Narang%2C+Aakanksha+Chowdhery%2C+and+Denny+Zhou%2E+2023%2E+Self%2Dconsistency+improves+chain+of+thought+reasoning+in+language+models%2E+In+The) [our card](https://md.research.elamur.ai/free/papers/wang-2022-self-consistency-improves-chain-thought/)
47. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Maarten+Bosma%2C+brian+ichter%2C+Fei+Xia%2C+Ed+H%2E+Chi%2C+Quoc+V+Le%2C+and+Denny+Zhou%2E+2022%2E+Chain+of+thought+prompting+elicits+reasoning+in+large+language+models%2E+In+Adv) [our card](https://md.research.elamur.ai/free/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
48. Sarah Wiegreffe and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 11–20, Hong Kong, China. Association for… [find](https://scholar.google.com/scholar?q=Sarah+Wiegreffe+and+Yuval+Pinter%2E+2019%2E+Attention+is+not+not+explanation%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the+9th+International+Joint+Conf) [our card](https://md.research.elamur.ai/free/papers/wiegreffe-2019-attention-is-not-not-explanation/)
49. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa… [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chap)
50. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pier%2Dric+Cistac%2C+Tim+Rault%2C+Remi+Louf%2C+Morgan+Funtow%2Dicz%2C+Joe+Davison%2C+Sam+Shleifer%2C+Patrick+von+Platen%2C+Clara) [our card](https://md.research.elamur.ai/free/papers/wolf-2020-transformers-state-art-natural-language/)
51. Jianing Zhou, Hongyu Gong, and Suma Bhat. 2021. PIE: A parallel idiomatic expression corpus for idiomatic sentence generation and paraphrasing. In Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021), pages 33–48, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jianing+Zhou%2C+Hongyu+Gong%2C+and+Suma+Bhat%2E+2021%2E+PIE%3A+A+parallel+idiomatic+expression+corpus+for+idiomatic+sentence+generation+and+paraphrasing%2E+In+Proceedings+of+the+17th+Workshop+on+Multiword+Express)
52. Jianing Zhou, Ziheng Zeng, and Suma Bhat. 2023a. CLCL: Non-compositional expression detection with contrastive learning and curriculum learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 730–743, Toronto, Canada. Associa… [find](https://scholar.google.com/scholar?q=Jianing+Zhou%2C+Ziheng+Zeng%2C+and+Suma+Bhat%2E+2023a%2E+CLCL%3A+Non%2Dcompositional+expression+detection+with+contrastive+learning+and+curriculum+learning%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Associ)
53. Jianing Zhou, Ziheng Zeng, Hongyu Gong, and Suma Bhat. 2023b. Non-compositional expression generation based on curriculum learning and continual learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4320–4335, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jianing+Zhou%2C+Ziheng+Zeng%2C+Hongyu+Gong%2C+and+Suma+Bhat%2E+2023b%2E+Non%2Dcompositional+expression+generation+based+on+curriculum+learning+and+continual+learning%2E+In+Findings+of+the+Association+for+Computatio)
54. Tadej Škvorc, Polona Gantar, and Marko Robnik-Šikonja. 2022. Mice: Mining idioms with contextual embeddings. Knowledge-Based Systems, 235:107606. [find](https://scholar.google.com/scholar?q=Tadej+%C5%A0kvorc%2C+Polona+Gantar%2C+and+Marko+Robnik%2D%C5%A0ikonja%2E+2022%2E+Mice%3A+Mining+idioms+with+contextual+embeddings%2E+Knowledge%2DBased+Systems%2C+235%3A107606%2E)
55. Fei Yu, Hongbo Zhang, Prayag Tiwari, and Benyou Wang. 2023. Natural language reasoning, a survey. [find](https://scholar.google.com/scholar?q=Fei+Yu%2C+Hongbo+Zhang%2C+Prayag+Tiwari%2C+and+Benyou+Wang%2E+2023%2E+Natural+language+reasoning%2C+a+survey%2E)
56. Ziheng Zeng and Suma Bhat. 2021. Idiomatic Expression Identification using Semantic Compatibility. Transactions of the Association for Computational Linguistics, 9:1546–1562. [find](https://scholar.google.com/scholar?q=Ziheng+Zeng+and+Suma+Bhat%2E+2021%2E+Idiomatic+Expression+Identification+using+Semantic+Compatibility%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+9%3A1546%E2%80%931562%2E) [our card](https://md.research.elamur.ai/free/papers/zeng-2021-idiomatic-expression-identification-semantic-compatibility/)
57. Ziheng Zeng and Suma Bhat. 2022. Getting BART to ride the idiomatic train: Learning to represent idiomatic expressions. Transactions of the Association for Computational Linguistics, 10:1120–1137. [find](https://scholar.google.com/scholar?q=Ziheng+Zeng+and+Suma+Bhat%2E+2022%2E+Getting+BART+to+ride+the+idiomatic+train%3A+Learning+to+represent+idiomatic+expressions%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+10%3A1120%E2%80%931137%2E)
58. Ziheng Zeng and Suma Bhat. 2023. Unified representation for non-compositional and compositional expressions. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11696– 11710, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ziheng+Zeng+and+Suma+Bhat%2E+2023%2E+Unified+representation+for+non%2Dcompositional+and+compositional+expressions%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2023%2C+pages+11696%E2%80%93+1171)
59. Ziheng Zeng, Kellen Cheng, Srihari Nanniyur, Jianing Zhou, and Suma Bhat. 2023. IEKG: A commonsense knowledge graph for idiomatic expressions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14243–14264, Singapore. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Ziheng+Zeng%2C+Kellen+Cheng%2C+Srihari+Nanniyur%2C+Jianing+Zhou%2C+and+Suma+Bhat%2E+2023%2E+IEKG%3A+A+commonsense+knowledge+graph+for+idiomatic+expressions%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Method) [our card](https://md.research.elamur.ai/free/papers/zeng-2023-iekg-commonsense-knowledge-graph-idiomatic/)