---
title: "Uppsala University at SemEval-2022 Task 1: Can Foreign Entries"
description: "The paper asks whether training material in other languages can help an English reverse dictionary when supervision is scarce. 14 references."
authors: ["Rafal Cerniavski", "Sara Stymne"]
year: 2022
venue: "Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)"
source: "https://aclanthology.org/2022.semeval-1.10/"
doi: "10.18653/v1/2022.semeval-1.10"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/cerniavski-2022-uppsala-university-semeval-2022-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/cerniavski-2022-uppsala-university-semeval-2022-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/8/) / Uppsala University at SemEval-2022 Task 1: Can Foreign Entries Enhance an English Reverse Dictionary?

# Uppsala University at SemEval-2022 Task 1: Can Foreign Entries Enhance an English Reverse Dictionary?

**Authors:** Rafal Cerniavski and [Sara Stymne](https://md.research.elamur.ai/free/en/authors/sara-stymne/) · **Year:** 2022 · **Venue:** [Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2022](https://md.research.elamur.ai/free/en/venues/semeval/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.semeval-1.10/) · [PDF](https://aclanthology.org/2022.semeval-1.10.pdf) · [DOI 10.18653/v1/2022.semeval-1.10](https://doi.org/10.18653/v1/2022.semeval-1.10) · [OpenAlex](https://openalex.org/W4287887995)

**Citations:** 2 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks whether training material in other languages can help an English reverse dictionary when supervision is scarce. The setting is the reverse-dictionary track of SemEval-2022 Task 1 (CODWOE). In this track the system receives a gloss and must output an embedding vector for the word it defines, not the word itself.

The authors used the official Transformer baseline with no changes. They first checked which embedding type works best and chose char. They then ran two experiments:
- **Joint training:** the English training set was combined with the French and/or Russian sets.
- **Alignment plus translation:** French and Russian embeddings were mapped into the English space with unsupervised MUSE. Some of their glosses were machine-translated with IBM Watson and added to the English training data.

Joint models performed roughly on par with monolingual ones, so mixing languages did no harm but brought no clear gain. Aligned Russian entries with translated glosses gave a small improvement on English. French gave no comparable benefit.

The system was not officially submitted because it relied on external MT. The work still shows how much cross-lingual data can realistically contribute to reverse dictionaries.

## Key points
- **Embedding choice:** on English (mean of 3 runs), char had the lowest MSE (0.156) with very low variance. Electra scored highest on COS (0.840) but its MSE was far higher (1.846). Char was used for all later experiments.
- **Joint training:** on the trial set, E+R lowered English MSE from 0.179 to 0.147 and raised CRK from 0.458 to 0.498. On the test set the change was negligible (MSE 0.178). Adding English to French or Russian slightly hurt those languages.
- **Alignment setup:** MUSE ran unsupervised for 5 epochs, with bilingual refinement and evaluation disabled and the English vectors kept fixed. The first 4,500 foreign glosses were translated and re-tokenized with spaCy.
- **Russian result:** English + Russian with aligned vectors and translated glosses reached MSE 0.135, COS 0.811 and CRK 0.501, against a baseline of 0.156 / 0.810 / 0.469. Alignment without translation mostly added noise.
- **Test-set ranks for the joint char models:**
  - French: 3rd on CRK.
  - Russian: 7th on both MSE and COS.
  - English: 9th on CRK.

## Interesting ideas
- Each baseline was retrained several times to measure run-to-run spread before any comparison, so small differences could be dismissed as chance.
- When mapping foreign vectors onto English, the English space was frozen so its quality could not degrade; only the newcomer moved.
- Translated glosses were re-tokenized with the same tool as the original data, keeping added material consistent with the native training set.
- The authors noticed that the alignment tool had silently cut vector precision, and trained a control model with restored values to test whether this mattered.
- The paper frames "no loss" from pooling unrelated data as a finding: the model kept the languages apart internally.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Mova's evaluation could report the spread across seeds for each parser or reader, and accept a gain only when it exceeds that spread.
- **I2** (world): when Wikidata or FrameNet-derived resources are linked to the WordNet-based ontology, WordNet could stay as the fixed anchor and other sources would be mapped onto it.
- **I3** (tokens): silver or translated text should pass through the en crate's own tokenizer before training, so tokenization mismatches do not appear as parser errors.
- **I4** (learn): pipeline side effects such as rounding or format conversion can be checked by retraining with the side effect undone, a cheap negative control.
- **I5** (tree): before mixing treebanks from different UD dialects in gparse training, check that per-dialect LAS does not drop, and treat that as the first success criterion.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Rafal Cerniavski and Sara Stymne. 2022. Uppsala University at SemEval-2022 Task 1: Can Foreign Entries Enhance an English Reverse Dictionary?. Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{cerniavski2022uppsala,
  title = {{Uppsala University at SemEval-2022 Task 1: Can Foreign Entries Enhance an English Reverse Dictionary?}},
  author = {Rafal Cerniavski and Sara Stymne},
  booktitle = {Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)},
  year = {2022},
  doi = {10.18653/v1/2022.semeval-1.10},
  url = {https://doi.org/10.18653/v1/2022.semeval-1.10}
}
```

- [← previous Uncovering More Shallow Heuristics: Probing the Natural Language Inference Capacities of Transformer-Based Pre-Trained Language Models Using Syllogistic Patterns](https://md.research.elamur.ai/free/en/papers/gubelmann-2022-uncovering-more-shallow-heuristics-probing/)
- [next → Wonder, beauty, ability and the natural world: The experience of wonder as a positive aesthetic emotion in Old English verse](https://md.research.elamur.ai/free/en/papers/gomez-2022-wonder-beauty-ability-natural-world/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/8/)

## References

14 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
2. Junqing He, Long Wu, Xuemin Zhao, and Yonghong Yan. 2017. HCCL at SemEval-2017 task 2: Combining multilingual word embeddings and transliteration model for semantic similarity. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 220–225, Vancouver, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Junqing+He%2C+Long+Wu%2C+Xuemin+Zhao%2C+and+Yonghong+Yan%2E+2017%2E+HCCL+at+SemEval%2D2017+task+2%3A+Combining+multilingual+word+embeddings+and+transliteration+model+for+semantic+similarity%2E+In+Proceedings+of+the+1)
3. Sergio Jimenez, George Dueñas, Lorena Gaitan, and Jorge Segura. 2017. RUFINO at SemEval-2017 task 2: Cross-lingual lexical similarity by extending PMI and word embeddings systems with a Swadesh’slike list. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 239–244, Vancouver, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sergio+Jimenez%2C+George+Due%C3%B1as%2C+Lorena+Gaitan%2C+and+Jorge+Segura%2E+2017%2E+RUFINO+at+SemEval%2D2017+task+2%3A+Cross%2Dlingual+lexical+similarity+by+extending+PMI+and+word+embeddings+systems+with+a+Swadesh%E2%80%99slike+)
4. Lei Zhang, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Qun Liu, and Maosong Sun. 2020. Multi-channel reverse dictionary model. arXiv preprint, arXiv:1912.08441. [↗](https://arxiv.org/abs/1912.08441)
5. Michael Zock and Slaven Bilac. 2004. Word lookup on the basis of associations : from an idea to a roadmap. In Proceedings of the Workshop on Enhancing and Using Electronic Dictionaries, pages 29–35, Geneva, Switzerland. COLING. [find](https://scholar.google.com/scholar?q=Michael+Zock+and+Slaven+Bilac%2E+2004%2E+Word+lookup+on+the+basis+of+associations+%3A+from+an+idea+to+a+roadmap%2E+In+Proceedings+of+the+Workshop+on+Enhancing+and+Using+Electronic+Dictionaries%2C+pages+29%E2%80%9335%2C+G)
6. Dan Kondratyuk and Milan Straka. 2019. [75 languages, 1 model: Parsing Universal Dependencies universally](https://md.research.elamur.ai/free/en/papers/kondratyuk-2019-75-languages-1-model-parsing/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–2795, Hong Kong, China. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d19-1279)
7. Guillaume Lample, Alexis Conneau, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018. Word translation without parallel data. In International Conference on Learning Representations (ICLR), Vancouver, Canada. [↗](https://doi.org/10.48550/arxiv.1710.04087)
8. Timothee Mickus, Denis Paperno, Mathieu Constant, and Kees van Deemter. 2022. [SemEval-2022 Task 1: CODWOE – COmparing Dictionaries and WOrd Embeddings](https://md.research.elamur.ai/free/en/papers/mickus-2022-codwoe-dictionaries-embeddings/). In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). Association for Computational Linguistics.
9. Ozan Polatbilek. 2020. Enriching Contextual Word Embeddings with Character Information. Ph.D. thesis, Izmir Institute of Technology (Turkey). [find](https://scholar.google.com/scholar?q=Ozan+Polatbilek%2E+2020%2E+Enriching+Contextual+Word+Embeddings+with+Character+Information%2E+Ph%2ED%2E+thesis%2C+Izmir+Institute+of+Technology+%28Turkey%29%2E)
10. Fanchao Qi, Lei Zhang, Yanhui Yang, Zhiyuan Liu, and Maosong Sun. 2020. WantWords: An open-source online reverse dictionary system. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 175–181, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Fanchao+Qi%2C+Lei+Zhang%2C+Yanhui+Yang%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2020%2E+WantWords%3A+An+open%2Dsource+online+reverse+dictionary+system%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natura)
11. Sebastian Ruder, Ivan Vulić, and Anders Søgaard. 2019. A survey of cross-lingual word embedding models. J. Artif. Int. Res., 65(1):569–630. [↗](https://doi.org/10.1613/jair.1.11640)
12. Aaron Smith, Bernd Bohnet, Miryam de Lhoneux, Joakim Nivre, Yan Shao, and Sara Stymne. 2018. 82 treebanks, 34 models: Universal Dependency parsing with multi-treebank models. In Proceedings of the [CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies](https://md.research.elamur.ai/free/en/papers/zeman-2018-conll-2018-shared-task-multilingual/), pages 113–123, Brussels, Belgium. Association for Computational Linguistics.
13. Iulia Turc, Kenton Lee, Jacob Eisenstein, Ming-Wei Chang, and Kristina Toutanova. 2021. Revisiting the primacy of English in zero-shot cross-lingual transfer. arXiv preprint, arXiv:2106.16171. [↗](https://arxiv.org/abs/2106.16171)
14. Hang Yan, Xiaonan Li, Xipeng Qiu, and Bocao Deng. 2020. BERT for monolingual and cross-lingual reverse dictionary. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4329–4338, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hang+Yan%2C+Xiaonan+Li%2C+Xipeng+Qiu%2C+and+Bocao+Deng%2E+2020%2E+BERT+for+monolingual+and+cross%2Dlingual+reverse+dictionary%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020%2C+pages+4329%E2%80%93)

*Updated 1 November 2026*