---
title: "Semeval-2022 Task 1 — paper by Mickus et al. (2022)"
description: "The paper describes the CODWOE shared task at SemEval-2022. 65 references, cited by 38 papers here."
authors: ["Timothee Mickus", "Kees Van Deemter", "Mathieu Constant", "Denis Paperno"]
year: 2022
venue: "SemEval-2022 (Task 1)"
source: "https://aclanthology.org/2022.semeval-1.1/"
doi: "10.18653/v1/2022.semeval-1.1"
arxiv: "2205.13858"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mickus-2022-codwoe-dictionaries-embeddings/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mickus-2022-codwoe-dictionaries-embeddings/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) / Semeval-2022 Task 1: CODWOE – Comparing Dictionaries and Word Embeddings

# Semeval-2022 Task 1: CODWOE – Comparing Dictionaries and Word Embeddings

**Authors:** Timothee Mickus, [Kees Van Deemter](https://md.research.elamur.ai/free/en/authors/kees-van-deemter/), [Mathieu Constant](https://md.research.elamur.ai/free/en/authors/mathieu-constant/) and [Denis Paperno](https://md.research.elamur.ai/free/en/authors/denis-paperno/) · **Year:** 2022 · **Venue:** [SemEval-2022 (Task 1)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2022](https://md.research.elamur.ai/free/en/venues/semeval/2022/)) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://doi.org/10.18653/v1/2022.semeval-1.1) · [PDF](https://aclanthology.org/2022.semeval-1.1.pdf) · [DOI 10.18653/v1/2022.semeval-1.1](https://doi.org/10.18653/v1/2022.semeval-1.1) · [arXiv 2205.13858](https://arxiv.org/abs/2205.13858) · [OpenAlex](https://openalex.org/W4287889472)

**Citations:** 21 citations (OpenAlex, 2026-10-14) · [38](https://md.research.elamur.ai/free/en/papers/mickus-2022-codwoe-dictionaries-embeddings/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper describes the CODWOE shared task at SemEval-2022. It tests whether word embeddings contain the same information as dictionary definitions. The task is divided into two tracks. In the Definition Modeling track, a definition must be generated from a word's vector. In the Reverse Dictionary track, the components of the vector must be reconstructed from the text of the definition. The authors collected parallel data for English, Spanish, French, Italian and Russian. The definitions were taken from DBnary, i.e., from Wiktionary, and the embeddings were trained using the same scheme on corpora of comparable size. To ensure a fair comparison, participants were not allowed to use external resources or pretrained models. Overall, the results are weak. Standard text generation metrics yield low and unreliable scores, and a substantial proportion of the generated definitions are irrelevant. Character-based embeddings rarely lagged noticeably behind distributional ones. The authors consider the question of whether embeddings and dictionaries are equivalent to remain open. The suitability of Definition Modeling for evaluating embeddings also remains doubtful. For low-resource languages, this is an important signal: building lexical resources from scratch in this way is not yet very effective.

## Key points
- Data: definitions only for nouns, adjectives, verbs and adverbs from DBnary. The authors parsed the English portion themselves from Wiktionary dumps in order to obtain usage examples. The datasets are published at https://codwoe.atilf.fr/.
- Three types of embeddings with dimensionality 256:
  - word2vec (gensim);
  - contextual ELECTRA embeddings, only for en, fr and ru, since the Spanish and Italian Wiktionary contain too few usage examples;
  - character-based embeddings from an LSTM autoencoder as a non-semantic baseline.
- The corpus for each language contains approximately 1 billion tokens: 2.5 GB of Wikipedia, 2.2 GB of OpenSubtitles (OPUS) and 0.3 GB of books (LiberLiber, Wikisource or Gutenberg). Its size is comparable to the 1B Words benchmark.
- The metrics are unreliable. BLEU scores are considerably lower than those typically reported in the literature. A meaningless "definition" such as ", or ." ranks among the best by S-BLEU for some languages.
- The BLCU-ICALL team manually evaluated 200 generated definitions: 52% were irrelevant and 23.5% were overly general. According to an observation by the Edinburgh team, ELECTRA vectors cluster by word frequency, while character-based vectors separate by word length.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Helsinki](https://md.research.elamur.ai/free/en/universities/university-of-helsinki/) — University, Helsinki, Finland · 33 works · 4,705 citations
- [Utrecht University](https://md.research.elamur.ai/free/en/universities/utrecht-university/) — University, Utrecht, Netherlands · 21 works · 1,281 citations
- [Université de Lorraine](https://md.research.elamur.ai/free/en/universities/universite-de-lorraine/) — University, Nancy, France · 14 works · 831 citations
- [Analyse et Traitement Informatique de la Langue Française](https://md.research.elamur.ai/free/en/labs/analyse-et-traitement-informatique-de-la-langue-francaise/) — Research institute, Nancy, France · 3 works · 359 citations

📚 Shelves (2): [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

💡 Concepts (1): [📒 Lexicography](https://md.research.elamur.ai/free/en/concepts/lexicography/) · 30 works

## Cite

Cite as: Timothee Mickus, Kees Van Deemter, Mathieu Constant and Denis Paperno. 2022. Semeval-2022 Task 1: CODWOE – Comparing Dictionaries and Word Embeddings. SemEval-2022 (Task 1).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mickus2022semeval,
  title = {{Semeval-2022 Task 1: CODWOE – Comparing Dictionaries and Word Embeddings}},
  author = {Timothee Mickus, Kees {Van Deemter}, Mathieu Constant and Denis Paperno},
  booktitle = {SemEval-2022 (Task 1)},
  year = {2022},
  doi = {10.18653/v1/2022.semeval-1.1},
  eprint = {2205.13858},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2022.semeval-1.1}
}
```

- [← previous Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction](https://md.research.elamur.ai/free/en/papers/ma-2022-linguistic-rules-based-corpus-generation/)
- [next → AlephBERT: Language Model Pre-training and Evaluation from Sub-Word to Sentence Level](https://md.research.elamur.ai/free/en/papers/seker-2022-alephbert-language-model-pre-training/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/)

## References

65 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. reverse dictionaries: Can foreign entries enhance an english reverse dictionary? [find](https://scholar.google.com/scholar?q=reverse+dictionaries%3A+Can+foreign+entries+enhance+an+english+reverse+dictionary%3F)
2. Alfonso Ardoiz, Miguel Ortega-Martín, Óscar García-Sierra, Jorge Álvarez, Ignacio Arranz, and Adrián Alonso. 2022. MMG at SemEval-2022 Task 1: A reverse dictionary approach based on a review of the dataset from a lexicographic perspective. [find](https://scholar.google.com/scholar?q=Alfonso+Ardoiz%2C+Miguel+Ortega%2DMart%C3%ADn%2C+%C3%93scar+Garc%C3%ADa%2DSierra%2C+Jorge+%C3%81lvarez%2C+Ignacio+Arranz%2C+and+Adri%C3%A1n+Alonso%2E+2022%2E+MMG+at+SemEval%2D2022+Task+1%3A+A+reverse+dictionary+approach+based+on+a+review+of+the+da)
3. Ciprian Chelba, Tomás Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, and Phillipp Koehn. 2013. [One billion word benchmark for measuring progress in statistical language modeling](https://md.research.elamur.ai/free/en/papers/chelba-2013-one-billion-word-benchmark-measuring/). CoRR, abs/1312.3005.
4. Satanjeev Banerjee and Alon Lavie. 2005. [METEOR: An automatic metric for MT evaluation with improved correlation with human judgments](https://md.research.elamur.ai/free/en/papers/banerjee-2005-meteor-automatic-metric-mt-evaluation/). In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan. Ass…
5. Pinzhen Chen and Zheng Zhao. 2022. Edinburgh at SemEval-2022 Task 1: Jointly fishing for word embeddings and definitions. [find](https://scholar.google.com/scholar?q=Pinzhen+Chen+and+Zheng+Zhao%2E+2022%2E+Edinburgh+at+SemEval%2D2022+Task+1%3A+Jointly+fishing+for+word+embeddings+and+definitions%2E)
6. Kevin Clark, Minh-Thang Luong, Quoc Le, and Christopher D. Manning. 2020. Pre-training transformers as energy-based cloze models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 285–294, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kevin+Clark%2C+Minh%2DThang+Luong%2C+Quoc+Le%2C+and+Christopher+D%2E+Manning%2E+2020%2E+Pre%2Dtraining+transformers+as+energy%2Dbased+cloze+models%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+)
7. Diego Bear and Paul Cook. 2021. Cross-lingual wolastoqey-English definition modelling. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 138–146, Held Online. INCOMA Ltd. [find](https://scholar.google.com/scholar?q=Diego+Bear+and+Paul+Cook%2E+2021%2E+Cross%2Dlingual+wolastoqey%2DEnglish+definition+modelling%2E+In+Proceedings+of+the+International+Conference+on+Recent+Advances+in+Natural+Language+Processing+%28RANLP+2021%29%2C+pa)
8. Isabelle Dautriche, Kyle Mahowald, Edward Gibson, and Steven T. Piantadosi. 2017. Wordform similarity increases with semantic similarity: An analysis of 100 languages. Cognitive Science, 41(8):2149–2169. [find](https://scholar.google.com/scholar?q=Isabelle+Dautriche%2C+Kyle+Mahowald%2C+Edward+Gibson%2C+and+Steven+T%2E+Piantadosi%2E+2017%2E+Wordform+similarity+increases+with+semantic+similarity%3A+An+analysis+of+100+languages%2E+Cognitive+Science%2C+41%288%29%3A2149%E2%80%9321)
9. Nihed Bendahman, Julien Breton, Lina Nicolaieff, Mokhtar Boumedyen Billami, Christophe Bortolaso, and Youssef Miloudi. 2022. BL.Research at SemEval-2022 Task 1: Deep networks for reverse dictionary using embeddings and lstm autoencoders. [find](https://scholar.google.com/scholar?q=Nihed+Bendahman%2C+Julien+Breton%2C+Lina+Nicolaieff%2C+Mokhtar+Boumedyen+Billami%2C+Christophe+Bortolaso%2C+and+Youssef+Miloudi%2E+2022%2E+BL%2EResearch+at+SemEval%2D2022+Task+1%3A+Deep+networks+for+reverse+dictionary+us)
10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
11. Emily M. Bender and Alexander Koller. 2020. [Climbing towards NLU: On meaning, form, and understanding in the age of data](https://md.research.elamur.ai/free/en/papers/bender-2020-climbing-nlu-meaning-form-understanding/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185–5198, Online. Association for Computational Linguistics.
12. Dominique Dutoit and Pierre Nugues. 2002. A lexical database and an algorithm to find words from definitions. In Proceedings of the 15th European Conference on Artificial Intelligence, ECAI’02, page 450–454, NLD. IOS Press. [find](https://scholar.google.com/scholar?q=Dominique+Dutoit+and+Pierre+Nugues%2E+2002%2E+A+lexical+database+and+an+algorithm+to+find+words+from+definitions%2E+In+Proceedings+of+the+15th+European+Conference+on+Artificial+Intelligence%2C+ECAI%E2%80%9902%2C+page+4)
13. Slaven Bila, Wataru Watanabe, Taiichi Hashimoto, Takenobu Tokunaga, and Hozumi Tanaka. 2004. Dictionary search based on the target word description. In Proceedings of the 10th Annual Meeting of the Association for Natural Language Processing (ANLP 2004). [find](https://scholar.google.com/scholar?q=Slaven+Bila%2C+Wataru+Watanabe%2C+Taiichi+Hashimoto%2C+Takenobu+Tokunaga%2C+and+Hozumi+Tanaka%2E+2004%2E+Dictionary+search+based+on+the+target+word+description%2E+In+Proceedings+of+the+10th+Annual+Meeting+of+the+As)
14. Ilknur Durgar El Khalout and Kemal Oflazer. 2004. Use of wordnet for retrieving words from their meanings. In Proceedings of the Second Global Wordnet Conference (GWC 2004), pages 118–123. [find](https://scholar.google.com/scholar?q=Ilknur+Durgar+El+Khalout+and+Kemal+Oflazer%2E+2004%2E+Use+of+wordnet+for+retrieving+words+from+their+meanings%2E+In+Proceedings+of+the+Second+Global+Wordnet+Conference+%28GWC+2004%29%2C+pages+118%E2%80%93123%2E)
15. Gemma Boleda. 2020. Distributional semantics and linguistic theory. Annual Review of Linguistics, 6(1):213–234. [find](https://scholar.google.com/scholar?q=Gemma+Boleda%2E+2020%2E+Distributional+semantics+and+linguistic+theory%2E+Annual+Review+of+Linguistics%2C+6%281%29%3A213%E2%80%93234%2E)
16. Kawin Ethayarajh. 2019. How contextual are contextualized word representations? comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process… [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2E+2019%2E+How+contextual+are+contextualized+word+representations%3F+comparing+the+geometry+of+BERT%2C+ELMo%2C+and+GPT%2D2+embeddings%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+i)
17. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
18. Artyom Gadetsky, Ilya Yakubovskiy, and Dmitry Vetrov. 2018. Conditional generators of words definitions. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 266–271, Melbourne, Australia. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Artyom+Gadetsky%2C+Ilya+Yakubovskiy%2C+and+Dmitry+Vetrov%2E+2018%2E+Conditional+generators+of+words+definitions%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Vol)
19. Hiram Calvo, Oscar Méndez, and Marco A. Moreno-Armendáriz. 2016. Integrated concept blending with vector space models. Comput. Speech Lang., 40(C):79–96. [find](https://scholar.google.com/scholar?q=Hiram+Calvo%2C+Oscar+M%C3%A9ndez%2C+and+Marco+A%2E+Moreno%2DArmend%C3%A1riz%2E+2016%2E+Integrated+concept+blending+with+vector+space+models%2E+Comput%2E+Speech+Lang%2E%2C+40%28C%29%3A79%E2%80%9396%2E)
20. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Rafal Cerniavski and Sara Stymne. 2022. Uppsala university at SemEval-2022 Task 1: Multilingualism in… [find](https://scholar.google.com/scholar?q=Ian+Goodfellow%2C+Jean+Pouget%2DAbadie%2C+Mehdi+Mirza%2C+Bing+Xu%2C+David+Warde%2DFarley%2C+Sherjil+Ozair%2C+Aaron+Courville%2C+and+Yoshua+Bengio%2E+2014%2E+Generative+adversarial+nets%2E+In+Advances+in+Neural+Information+Ra)
21. Slovenia. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Slovenia%2E+European+Language+Resources+Association+%28ELRA%29%2E)
22. E. Dario Gutiérrez, Roger Levy, and Benjamin Bergen. 2016. Finding non-arbitrary form-meaning systematicity using string-metric learning for kernel regression. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2379–2388, Berlin,… [find](https://scholar.google.com/scholar?q=E%2E+Dario+Guti%C3%A9rrez%2C+Roger+Levy%2C+and+Benjamin+Bergen%2E+2016%2E+Finding+non%2Darbitrary+form%2Dmeaning+systematicity+using+string%2Dmetric+learning+for+kernel+regression%2E+In+Proceedings+of+the+54th+Annual+Meetin)
23. Arman Malekzadeh, Amin Gheibi, and Ali Mohades. 2021. PREDICT: persian reverse dictionary. CoRR, abs/2105.00309. — [arXiv:2105.00309](https://arxiv.org/abs/2105.00309) [find](https://scholar.google.com/scholar?q=Arman+Malekzadeh%2C+Amin+Gheibi%2C+and+Ali+Mohades%2E+2021%2E+PREDICT%3A+persian+reverse+dictionary%2E+CoRR%2C+abs%2F2105%2E00309%2E+%E2%80%94+%5BarXiv%3A2105%2E00309%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2105%2E00309%29)
24. Oscar Méndez, Hiram Calvo, and Marco A. Moreno-Armendáriz. 2013. A reverse dictionary based on semantic analysis using wordnet. In Advances in Artificial Intelligence and Its Applications, pages 275– 285, Berlin, Heidelberg. Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Oscar+M%C3%A9ndez%2C+Hiram+Calvo%2C+and+Marco+A%2E+Moreno%2DArmend%C3%A1riz%2E+2013%2E+A+reverse+dictionary+based+on+semantic+analysis+using+wordnet%2E+In+Advances+in+Artificial+Intelligence+and+Its+Applications%2C+pages+275%E2%80%93+)
25. Zellig Harris. 1954. Distributional structure. Word, 10(23):146–162. [find](https://scholar.google.com/scholar?q=Zellig+Harris%2E+1954%2E+Distributional+structure%2E+Word%2C+10%2823%29%3A146%E2%80%93162%2E)
26. Timothee Mickus, Mathieu Constant, and Denis Paperno. 2020a. Génération automatique de définitions pour le français (definition modeling in French). In Actes de la 6e conférence conjointe Journées d’Études sur la Parole (JEP, 33e édition), Traitement Automatique des Langues Naturelles (TALN, 27e édi… [find](https://scholar.google.com/scholar?q=Timothee+Mickus%2C+Mathieu+Constant%2C+and+Denis+Paperno%2E+2020a%2E+G%C3%A9n%C3%A9ration+automatique+de+d%C3%A9finitions+pour+le+fran%C3%A7ais+%28definition+modeling+in+French%29%2E+In+Actes+de+la+6e+conf%C3%A9rence+conjointe+Journ%C3%A9es+d%E2%80%99%C3%89)
27. Felix Hill, Kyunghyun Cho, Anna Korhonen, and Yoshua Bengio. 2016. [Learning to understand phrases by embedding the dictionary](https://md.research.elamur.ai/free/en/papers/hill-2016-learning-understand-phrases-embedding-dictionary/). Transactions of the Association for Computational Linguistics, 4:17– 30.
28. Arman Kabiri and Paul Cook. 2020. Evaluating a multi-sense definition generation model for multiple languages. In Text, Speech, and Dialogue, pages 153– 161, Cham. Springer International Publishing. [find](https://scholar.google.com/scholar?q=Arman+Kabiri+and+Paul+Cook%2E+2020%2E+Evaluating+a+multi%2Dsense+definition+generation+model+for+multiple+languages%2E+In+Text%2C+Speech%2C+and+Dialogue%2C+pages+153%E2%80%93+161%2C+Cham%2E+Springer+International+Publishing%2E)
29. Cunliang Kong, Yujie Wang, Ruining Chong, Liner Yang, Hengyuan Zhang, Erhong Yang, and Yaping Huang. 2022. BLCU-ICALL at SemEval-2022 Task 1: Cross-attention multitasking framework for definition modeling. [find](https://scholar.google.com/scholar?q=Cunliang+Kong%2C+Yujie+Wang%2C+Ruining+Chong%2C+Liner+Yang%2C+Hengyuan+Zhang%2C+Erhong+Yang%2C+and+Yaping+Huang%2E+2022%2E+BLCU%2DICALL+at+SemEval%2D2022+Task+1%3A+Cross%2Dattention+multitasking+framework+for+definition+mode)
30. Timothee Mickus, Mathieu Constant, and Denis Paperno. 2021. About neural networks and writing definitions. Dictionaries: Journal of the Dictionary Society of North America, 42(2). [find](https://scholar.google.com/scholar?q=Timothee+Mickus%2C+Mathieu+Constant%2C+and+Denis+Paperno%2E+2021%2E+About+neural+networks+and+writing+definitions%2E+Dictionaries%3A+Journal+of+the+Dictionary+Society+of+North+America%2C+42%282%29%2E)
31. Timothee Mickus, Denis Paperno, Mathieu Constant, and Kees van Deemter. 2020b. What do you mean, BERT? In Proceedings of the Society for Computation in Linguistics 2020, pages 279–290, New York, New York. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Timothee+Mickus%2C+Denis+Paperno%2C+Mathieu+Constant%2C+and+Kees+van+Deemter%2E+2020b%2E+What+do+you+mean%2C+BERT%3F+In+Proceedings+of+the+Society+for+Computation+in+Linguistics+2020%2C+pages+279%E2%80%93290%2C+New+York%2C+New+Y)
32. Damir Korenčić and Ivan Grubišić. 2022. IRB-NLP at SemEval-2022 Task 1: Exploring the relationship between words and their semantic representations. [find](https://scholar.google.com/scholar?q=Damir+Korenc%CC%8Cic%CC%81+and+Ivan+Grubi%C5%A1ic%CC%81%2E+2022%2E+IRB%2DNLP+at+SemEval%2D2022+Task+1%3A+Exploring+the+relationship+between+words+and+their+semantic+representations%2E)
33. Andrei Kutuzov. 2017. Arbitrariness of linguistic sign questioned: Correlation between word form and meaning in russian. [find](https://scholar.google.com/scholar?q=Andrei+Kutuzov%2E+2017%2E+Arbitrariness+of+linguistic+sign+questioned%3A+Correlation+between+word+form+and+meaning+in+russian%2E)
34. Timothee Mickus, Denis Paperno, and Matthieu Constant. 2019. Mark my word: A sequence-to-sequence approach to definition modeling. In Proceedings of the First NLPL Workshop on Deep Learning for Natural Language Processing, pages 1–11, Turku, Finland. Linköping University Electronic Press. [find](https://scholar.google.com/scholar?q=Timothee+Mickus%2C+Denis+Paperno%2C+and+Matthieu+Constant%2E+2019%2E+Mark+my+word%3A+A+sequence%2Dto%2Dsequence+approach+to+definition+modeling%2E+In+Proceedings+of+the+First+NLPL+Workshop+on+Deep+Learning+for+Natura)
35. Alessandro Lenci. 2018. Distributional models of word meaning. Annual Review of Linguistics, 4(1):151– 171. [find](https://scholar.google.com/scholar?q=Alessandro+Lenci%2E+2018%2E+Distributional+models+of+word+meaning%2E+Annual+Review+of+Linguistics%2C+4%281%29%3A151%E2%80%93+171%2E)
36. Bin Li, Yixuan Weng, Fei Xia, Shizhu He, Bin Sun, and Shutao Li. 2022. Lingjing at SemEval-2022 Task 1: Multi-task self-supervised pre-training for multilingual reverse dictionary. [find](https://scholar.google.com/scholar?q=Bin+Li%2C+Yixuan+Weng%2C+Fei+Xia%2C+Shizhu+He%2C+Bin+Sun%2C+and+Shutao+Li%2E+2022%2E+Lingjing+at+SemEval%2D2022+Task+1%3A+Multi%2Dtask+self%2Dsupervised+pre%2Dtraining+for+multilingual+reverse+dictionary%2E)
37. Tomas Mikolov, Kai Chen, Greg S. Corrado, and Jeffrey Dean. 2013. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/).
38. Yinqiao Li, Chi Hu, Yuhao Zhang, Nuo Xu, Yufan Jiang, Tong Xiao, Jingbo Zhu, Tongran Liu, and Changliang Li. 2020. Learning architectures from an extended search space for language modeling. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6629–6639,… [find](https://scholar.google.com/scholar?q=Yinqiao+Li%2C+Chi+Hu%2C+Yuhao+Zhang%2C+Nuo+Xu%2C+Yufan+Jiang%2C+Tong+Xiao%2C+Jingbo+Zhu%2C+Tongran+Liu%2C+and+Changliang+Li%2E+2020%2E+Learning+architectures+from+an+extended+search+space+for+language+modeling%2E+In+Procee)
39. Eduards Mukans, Gus Strazds, and Guntis Barzdins. 2022. RIGA at SemEval-2022 Task 1: Scaling recurrent neural networks for CODWOE dictionary modeling. [find](https://scholar.google.com/scholar?q=Eduards+Mukans%2C+Gus+Strazds%2C+and+Guntis+Barzdins%2E+2022%2E+RIGA+at+SemEval%2D2022+Task+1%3A+Scaling+recurrent+neural+networks+for+CODWOE+dictionary+modeling%2E)
40. Chin-Yew Lin. 2004. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
41. Thanapon Noraset, Chen Liang, Larry Birnbaum, and Doug Downey. 2017. Definition modeling: Learning to define word embeddings in natural language. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, pages 3259–3266. AAAI P… [find](https://scholar.google.com/scholar?q=Thanapon+Noraset%2C+Chen+Liang%2C+Larry+Birnbaum%2C+and+Doug+Downey%2E+2017%2E+Definition+modeling%3A+Learning+to+define+word+embeddings+in+natural+language%2E+In+Proceedings+of+the+Thirty%2DFirst+AAAI+Conference+on+)
42. Pierre Lison and Jörg Tiedemann. 2016. [OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles](https://md.research.elamur.ai/free/en/papers/lison-2016-opensubtitles2016-extracting-large-parallel-corpora/). In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 923–929, Portorož, Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Ji…
43. William Timkey and Marten van Schijndel. 2021. [All bark and no bite: Rogue dimensions in transformer language models obscure representational quality](https://md.research.elamur.ai/free/en/papers/timkey-2021-all-bark-no-bite-rogue/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4527–4546, Online and Punta Cana, Dominican Republic…
44. Tiago Pimentel, Arya D. McCarthy, Damian Blasi, Brian Roark, and Ryan Cotterell. 2019. Meaning to form: Measuring systematicity as information. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1751– 1764, Florence, Italy. Association for Computational… [find](https://scholar.google.com/scholar?q=Tiago+Pimentel%2C+Arya+D%2E+McCarthy%2C+Damian+Blasi%2C+Brian+Roark%2C+and+Ryan+Cotterell%2E+2019%2E+Meaning+to+form%3A+Measuring+systematicity+as+information%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Associa)
45. Thi Hong Hanh Tran, Matej Martinc, Matthew Purver, and Senja Pollak. 2022. JSI at SemEval-2022 Task 1: CODWOE - reverse dictionary: Monolingual and cross-lingual approaches. [find](https://scholar.google.com/scholar?q=Thi+Hong+Hanh+Tran%2C+Matej+Martinc%2C+Matthew+Purver%2C+and+Senja+Pollak%2E+2022%2E+JSI+at+SemEval%2D2022+Task+1%3A+CODWOE+%2D+reverse+dictionary%3A+Monolingual+and+cross%2Dlingual+approaches%2E)
46. Fanchao Qi, Lei Zhang, Yanhui Yang, Zhiyuan Liu, and Maosong Sun. 2020. Wantwords: An open-source online reverse dictionary system. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 175–181. [find](https://scholar.google.com/scholar?q=Fanchao+Qi%2C+Lei+Zhang%2C+Yanhui+Yang%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2020%2E+Wantwords%3A+An+open%2Dsource+online+reverse+dictionary+system%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natura)
47. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Process…
48. Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45–50, Valletta, Malta. ELRA. [http://is.muni.cz/](http://is.muni.cz/) publication/884893/en. [↗](http://is.muni.cz/)
49. Zhiyong Wang, Ge Zhang, and Nineli Lashkarashvili. 2022. 1cademy at SemEval-2022 Task 1: Investigating the effectiveness of multilingual, multitask, and language-agnostic tricks for the reverse dictionary task. [find](https://scholar.google.com/scholar?q=Zhiyong+Wang%2C+Ge+Zhang%2C+and+Nineli+Lashkarashvili%2E+2022%2E+1cademy+at+SemEval%2D2022+Task+1%3A+Investigating+the+effectiveness+of+multilingual%2C+multitask%2C+and+language%2Dagnostic+tricks+for+the+reverse+dictio)
50. Ferdinand de Saussure. 1916. Cours de linguistique générale, 1995 edition. Payot & Rivage, Paris. [find](https://scholar.google.com/scholar?q=Ferdinand+de+Saussure%2E+1916%2E+Cours+de+linguistique+g%C3%A9n%C3%A9rale%2C+1995+edition%2E+Payot+%26+Rivage%2C+Paris%2E)
51. Hang Yan, Xiaonan Li, Xipeng Qiu, and Bocao Deng. 2020. BERT for monolingual and cross-lingual reverse dictionary. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4329–4338, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hang+Yan%2C+Xiaonan+Li%2C+Xipeng+Qiu%2C+and+Bocao+Deng%2E+2020%2E+BERT+for+monolingual+and+cross%2Dlingual+reverse+dictionary%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020%2C+pages+4329%E2%80%93)
52. Gilles Sérasset. 2012. [Dbnary: Wiktionary as a LMF based multilingual RDF network](https://md.research.elamur.ai/free/en/papers/serasset-2012-dbnary-wiktionary-as-lmf-based/). In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 2466– 2472, Istanbul, Turkey. European Language Resources Association (ELRA).
53. Liner Yang, Cunliang Kong, Yun Chen, Yang Liu, Qinan Fan, and Erhong Yang. 2019. Incorporating sememes into chinese definition modeling. ArXiv preprint : 1905.06512. — [arXiv:1905.06512](https://arxiv.org/abs/1905.06512) [find](https://scholar.google.com/scholar?q=Liner+Yang%2C+Cunliang+Kong%2C+Yun+Chen%2C+Yang+Liu%2C+Qinan+Fan%2C+and+Erhong+Yang%2E+2019%2E+Incorporating+sememes+into+chinese+definition+modeling%2E+ArXiv+preprint+%3A+1905%2E06512%2E+%E2%80%94+%5BarXiv%3A1905%2E06512%5D%28https%3A%2F%2Farxiv)
54. Ryan Shaw, Anindya Datta, Debra VanderMeer, and Kaushik Dutta. 2013. Building a scalable database-driven reverse dictionary. IEEE Transactions on Knowledge and Data Engineering, 25(3):528–540. [find](https://scholar.google.com/scholar?q=Ryan+Shaw%2C+Anindya+Datta%2C+Debra+VanderMeer%2C+and+Kaushik+Dutta%2E+2013%2E+Building+a+scalable+database%2Ddriven+reverse+dictionary%2E+IEEE+Transactions+on+Knowledge+and+Data+Engineering%2C+25%283%29%3A528%E2%80%93540%2E)
55. David Yenicelik, Florian Schmidt, and Yannic Kilcher. 2020. How does BERT capture semantics? a closer look at polysemous words. In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 156–162, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=David+Yenicelik%2C+Florian+Schmidt%2C+and+Yannic+Kilcher%2E+2020%2E+How+does+BERT+capture+semantics%3F+a+closer+look+at+polysemous+words%2E+In+Proceedings+of+the+Third+BlackboxNLP+Workshop+on+Analyzing+and+Interp)
56. Bushra Siddique and Mirza Mohd Sufyan Beg. 2019. A review of reverse dictionary: Finding words from concept description. In Next Generation Computing Technologies on Computational Intelligence, pages 128–139, Singapore. Springer Singapore. [find](https://scholar.google.com/scholar?q=Bushra+Siddique+and+Mirza+Mohd+Sufyan+Beg%2E+2019%2E+A+review+of+reverse+dictionary%3A+Finding+words+from+concept+description%2E+In+Next+Generation+Computing+Technologies+on+Computational+Intelligence%2C+pages+)
57. Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Jasper+Snoek%2C+Hugo+Larochelle%2C+and+Ryan+P+Adams%2E+2012%2E+Practical+bayesian+optimization+of+machine+learning+algorithms%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+25%2E+Curran+Associate)
58. Fabio Massimo Zanzotto, Ioannis Korkontzelos, Francesca Fallucchi, and Suresh Manandhar. 2010. [Estimating linear models for compositional distributional semantics](https://md.research.elamur.ai/free/en/papers/zanzotto-2010-estimating-linear-models-compositional-distributional/). In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1263–1271, Beijing, China.
59. Coling 2010 Organizing Committee. [find](https://scholar.google.com/scholar?q=Coling+2010+Organizing+Committee%2E)
60. Aditya Srivastava and Vemulapati Harsha Vardhan. 2022. TLDR at SemEval-2022 Task 1: Using transformers to learn dictionaries and representations. [find](https://scholar.google.com/scholar?q=Aditya+Srivastava+and+Vemulapati+Harsha+Vardhan%2E+2022%2E+TLDR+at+SemEval%2D2022+Task+1%3A+Using+transformers+to+learn+dictionaries+and+representations%2E)
61. Haitong Zhang, Yongping Du, Jiaxin Sun, and Qingxiao Li. 2020a. Improving interpretability of word embeddings by generating definition and usage. Expert Systems with Applications, 160:113633. [find](https://scholar.google.com/scholar?q=Haitong+Zhang%2C+Yongping+Du%2C+Jiaxin+Sun%2C+and+Qingxiao+Li%2E+2020a%2E+Improving+interpretability+of+word+embeddings+by+generating+definition+and+usage%2E+Expert+Systems+with+Applications%2C+160%3A113633%2E)
62. Sushrut Thorat and Varad Choudhari. 2016. Implementing a reverse dictionary, based on word definitions, using a node-graph architecture. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2797–2806, Osaka, Japan. [find](https://scholar.google.com/scholar?q=Sushrut+Thorat+and+Varad+Choudhari%2E+2016%2E+Implementing+a+reverse+dictionary%2C+based+on+word+definitions%2C+using+a+node%2Dgraph+architecture%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Conferenc)
63. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
64. Lei Zhang, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Qun Liu, and Maosong Sun. 2020b. Multi-channel reverse dictionary model. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 312–319. [find](https://scholar.google.com/scholar?q=Lei+Zhang%2C+Fanchao+Qi%2C+Zhiyuan+Liu%2C+Yasheng+Wang%2C+Qun+Liu%2C+and+Maosong+Sun%2E+2020b%2E+Multi%2Dchannel+reverse+dictionary+model%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+pages+312%E2%80%933)
65. Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, and Steffen Eger. 2019. [MoverScore: Text generation evaluating with contextualized embeddings and earth mover distance](https://md.research.elamur.ai/free/en/papers/zhao-2019-moverscore-text-generation-evaluating-contextualized/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International…

*Updated 24 October 2026*