---
title: "Insights from Transfer Learning Experiments — Mujko 2026"
description: "The paper tests whether Word-in-Context (WiC) and Word Sense Disambiguation (WSD), two related ways of modelling word meaning in context, benefit from…"
authors: ["Alp Mujko", "Dominik Schlechtweg"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.785/"
doi: "10.63317/2nnxnn5jfjan"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/mujko-2026-insights-transfer-learning-experiments-word/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/mujko-2026-insights-transfer-learning-experiments-word/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/3/) / Insights from Transfer Learning Experiments with Word-in-Context and Word Sense Disambiguation Models

# Insights from Transfer Learning Experiments with Word-in-Context and Word Sense Disambiguation Models

**Authors:** Alp Mujko, [Dominik Schlechtweg](https://md.research.elamur.ai/free/authors/dominik-schlechtweg/) · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.785/) · [PDF](https://aclanthology.org/2026.lrec-1.785.pdf) · [DOI 10.63317/2nnxnn5jfjan](https://doi.org/10.63317/2nnxnn5jfjan) · [OpenAlex](https://openalex.org/W7160068366)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
The paper tests whether Word-in-Context (WiC) and Word Sense Disambiguation (WSD), two related ways of modelling word meaning in context, benefit from being learned by one model. Using only English data, the authors fine-tune a Sentence Transformer built on xlm-roberta-large with mean pooling and Contrastive Loss (margin 0.5). Target words, and glosses in the WSD data, are wrapped in special `<t>…</t>` tokens. WiC labels come from thresholding the cosine similarity of two context embeddings. WSD is treated as finding the gloss embedding closest to the context. Training data are Pil-WiC, MCL-WiC and XL-Lexeme, plus FEWS recast as balanced sentence–gloss pairs. FEWS is used either downsampled to each WiC dataset's size or in full (about 200K pairs). The authors compare single-task, joint, merged-WiC and cross-task setups. Joint training never meaningfully harms WiC and helps most where WiC data is scarce. WSD improves from added WiC data only when FEWS is downsampled. Each task's model transfers clearly to the other, which supports unified training for sense disambiguation in low-resource settings.

## Key points
- Data sizes: Pil-WiC has 5,428 training pairs, MCL-WiC 8,000 and XL-Lexeme 13,428. FEWS was expanded from 101,459 positive instances to 202,918 balanced instances, and evaluation uses its zero-shot splits.
- Adding the full FEWS data raised Pil-WiC accuracy from .681 to .714 (p = 0.004). Merging all WiC sets lifted MCL-WiC from .889 to .906.
- With limited WSD data, adding MCL-WiC improved FEWS zero-shot accuracy from .527 to .575 (p = 0.008). With the full FEWS data, adding WiC gave no gain (about .67 either way).
- Cross-task transfer:
  - A WiC-only model trained on Pil-WiC reached .475 on FEWS, against .309 for the untrained model; the merged-WiC model reached .503.
  - The FEWS-200K model scored about .69 on Pil-WiC, roughly matching in-task training.
- Results average only three runs per configuration, so statistical power is low. Training took about 50–60 GPU-hours on RTX A6000 cards.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Word sense disambiguation](https://md.research.elamur.ai/free/topics/word-sense-disambiguation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mujko2026insights,
  title = {{Insights from Transfer Learning Experiments with Word-in-Context and Word Sense Disambiguation Models}},
  author = {Mujko, Alp and Schlechtweg, Dominik},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2nnxnn5jfjan},
  url = {https://doi.org/10.63317/2nnxnn5jfjan}
}
```

- [← previous Information Asymmetry across Language Varieties: A Case Study on Cantonese-Mandarin and Bavarian-German QA](https://md.research.elamur.ai/free/papers/pei-2026-information-asymmetry-across-language-varieties/)
- [next → Interaction in Biblical Hebrew Narrative: A Conversation-Analytic Approach](https://md.research.elamur.ai/free/papers/hbo-2026-patty-interaction-biblical-hebrew-narrative-conversation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/3/)

## References

21 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Edoardo Barba, Luigi Procopio, and Roberto Navigli. 2021. ConSeC: Word sense disambiguation as continuous sense comprehension. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1492–1503, Online and Punta Cana, Dominican Republic. Association for Comput… [find](https://scholar.google.com/scholar?q=Edoardo+Barba%2C+Luigi+Procopio%2C+and+Roberto+Navigli%2E+2021%2E+ConSeC%3A+Word+sense+disambiguation+as+continuous+sense+comprehension%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Lan)
2. Terra Blevins, Mandar Joshi, and Luke Zettlemoyer. 2021. FEWS: Large-scale, low-shot word sense disambiguation with the dictionary. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 455–465, Online. Association for Comp… [find](https://scholar.google.com/scholar?q=Terra+Blevins%2C+Mandar+Joshi%2C+and+Luke+Zettlemoyer%2E+2021%2E+FEWS%3A+Large%2Dscale%2C+low%2Dshot+word+sense+disambiguation+with+the+dictionary%2E+In+Proceedings+of+the+16th+Conference+of+the+European+Chapter+of+the)
3. Pierluigi Cassotti, Lucia Siciliani, Marco DeGemmis, Giovanni Semeraro, and Pierpaolo Basile. 2023. XL-LEXEME: WiC pretrained model for cross-lingual LEXical sEMantic changE. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 15… [find](https://scholar.google.com/scholar?q=Pierluigi+Cassotti%2C+Lucia+Siciliani%2C+Marco+DeGemmis%2C+Giovanni+Semeraro%2C+and+Pierpaolo+Basile%2E+2023%2E+XL%2DLEXEME%3A+WiC+pretrained+model+for+cross%2Dlingual+LEXical+sEMantic+changE%2E+In+Proceedings+of+the+61s)
4. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E)
5. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440–8451, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Unsupervised+cross%2Dlingual+representation+learning+at+scale%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+8440%E2%80%938451%2C+Online%2E+Association+for+Comput)
6. Bradley Hauer and Grzegorz Kondrak. 2022. WiC = TSV = WSD: On the equivalence of three semantic tasks. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2478–2486, Seattle, United States. Associati… [find](https://scholar.google.com/scholar?q=Bradley+Hauer+and+Grzegorz+Kondrak%2E+2022%2E+WiC+%3D+TSV+%3D+WSD%3A+On+the+equivalence+of+three+semantic+tasks%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Chapter+of+the+Association+for+Comput)
7. Luyao Huang, Chi Sun, Xipeng Qiu, and Xuanjing Huang. 2019. GlossBERT: BERT for word sense disambiguation with gloss knowledge. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-I… [find](https://scholar.google.com/scholar?q=Luyao+Huang%2C+Chi+Sun%2C+Xipeng+Qiu%2C+and+Xuanjing+Huang%2E+2019%2E+GlossBERT%3A+BERT+for+word+sense+disambiguation+with+gloss+knowledge%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+La) [our card](https://md.research.elamur.ai/free/papers/huang-2019-glossbert-bert-word-sense-disambiguation/)
8. Nancy Ide and Jean Véronis. 1998. Introduction to the special issue on word sense disambiguation: The state of the art. Computational Linguistics, 24(1):1–40. [find](https://scholar.google.com/scholar?q=Nancy+Ide+and+Jean+V%C3%A9ronis%2E+1998%2E+Introduction+to+the+special+issue+on+word+sense+disambiguation%3A+The+state+of+the+art%2E+Computational+Linguistics%2C+24%281%29%3A1%E2%80%9340%2E)
9. Qianchu Liu, Edoardo Maria Ponti, Diana McCarthy, Ivan Vulić, and Anna Korhonen. 2021. AM2iCo: Evaluating word meaning in context across low-resource languages with adversarial examples. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7151–7162, Onlin… [find](https://scholar.google.com/scholar?q=Qianchu+Liu%2C+Edoardo+Maria+Ponti%2C+Diana+McCarthy%2C+Ivan+Vuli%C4%87%2C+and+Anna+Korhonen%2E+2021%2E+AM2iCo%3A+Evaluating+word+meaning+in+context+across+low%2Dresource+languages+with+adversarial+examples%2E+In+Proceeding)
10. Daniel Loureiro and Alípio Jorge. 2019. LIAAD at SemDeep-5 challenge: Word-in-context (WiC). In Proceedings of the 5th Workshop on Semantic Deep Learning (SemDeep-5), pages 1–5, Macau, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Daniel+Loureiro+and+Al%C3%ADpio+Jorge%2E+2019%2E+LIAAD+at+SemDeep%2D5+challenge%3A+Word%2Din%2Dcontext+%28WiC%29%2E+In+Proceedings+of+the+5th+Workshop+on+Semantic+Deep+Learning+%28SemDeep%2D5%29%2C+pages+1%E2%80%935%2C+Macau%2C+China%2E+Associat)
11. Federico Martelli, Najla Kalach, Gabriele Tola, and Roberto Navigli. 2021. SemEval-2021 task 2: Multilingual and cross-lingual word-in-context disambiguation (MCL-WiC). In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 24–36, Online. Association for Compu… [find](https://scholar.google.com/scholar?q=Federico+Martelli%2C+Najla+Kalach%2C+Gabriele+Tola%2C+and+Roberto+Navigli%2E+2021%2E+SemEval%2D2021+task+2%3A+Multilingual+and+cross%2Dlingual+word%2Din%2Dcontext+disambiguation+%28MCL%2DWiC%29%2E+In+Proceedings+of+the+15th+Inte)
12. Roberto Navigli. 2009. Word sense disambiguation: A survey. ACM Comput. Surv., 41(2). [find](https://scholar.google.com/scholar?q=Roberto+Navigli%2E+2009%2E+Word+sense+disambiguation%3A+A+survey%2E+ACM+Comput%2E+Surv%2E%2C+41%282%29%2E)
13. Mohammad Taher Pilehvar and Jose Camacho-Collados. 2019. WiC: the word-in-context dataset for evaluating context-sensitive meaning representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volum… [find](https://scholar.google.com/scholar?q=Mohammad+Taher+Pilehvar+and+Jose+Camacho%2DCollados%2E+2019%2E+WiC%3A+the+word%2Din%2Dcontext+dataset+for+evaluating+context%2Dsensitive+meaning+representations%2E+In+Proceedings+of+the+2019+Conference+of+the+North+A) [our card](https://md.research.elamur.ai/free/papers/pilehvar-2019-wic-word-context-dataset-evaluating/)
14. Alessandro Raganato, Jose Camacho-Collados, and Roberto Navigli. 2017. Word sense disambiguation: A unified evaluation framework and empirical comparison. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 99–1… [find](https://scholar.google.com/scholar?q=Alessandro+Raganato%2C+Jose+Camacho%2DCollados%2C+and+Roberto+Navigli%2E+2017%2E+Word+sense+disambiguation%3A+A+unified+evaluation+framework+and+empirical+comparison%2E+In+Proceedings+of+the+15th+Conference+of+the+) [our card](https://md.research.elamur.ai/free/papers/onto-2017-raganato-word-sense-disambiguation-unified-evaluation/)
15. Alessandro Raganato, Tommaso Pasini, Jose Camacho-Collados, and Mohammad Taher Pilehvar. 2020. XL-WiC: A multilingual benchmark for evaluating semantic contextualization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7193–7206, Online. Assoc… [find](https://scholar.google.com/scholar?q=Alessandro+Raganato%2C+Tommaso+Pasini%2C+Jose+Camacho%2DCollados%2C+and+Mohammad+Taher+Pilehvar%2E+2020%2E+XL%2DWiC%3A+A+multilingual+benchmark+for+evaluating+semantic+contextualization%2E+In+Proceedings+of+the+2020+Co)
16. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2019%2E+Sentence%2DBERT%3A+Sentence+embeddings+using+Siamese+BERT%2Dnetworks%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the)
17. Dominik Schlechtweg, Tejaswi Choppa, Wei Zhao, and Michael Roth. 2025. CoMeDi shared task: Median judgment classification & mean disagreement ranking with ordinal word-in-context judgments. In Proceedings of Context and Meaning: Navigating Disagreements in NLP Annotation, pages 33–47, Abu Dhabi, UAE… [find](https://scholar.google.com/scholar?q=Dominik+Schlechtweg%2C+Tejaswi+Choppa%2C+Wei+Zhao%2C+and+Michael+Roth%2E+2025%2E+CoMeDi+shared+task%3A+Median+judgment+classification+%26+mean+disagreement+ranking+with+ordinal+word%2Din%2Dcontext+judgments%2E+In+Proceed)
18. Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky, and Nina Tahmasebi. 2020. SemEval-2020 task 1: Unsupervised lexical semantic change detection. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1–23, Barcelona (online). International Committee for Com… [find](https://scholar.google.com/scholar?q=Dominik+Schlechtweg%2C+Barbara+McGillivray%2C+Simon+Hengchen%2C+Haim+Dubossarsky%2C+and+Nina+Tahmasebi%2E+2020%2E+SemEval%2D2020+task+1%3A+Unsupervised+lexical+semantic+change+detection%2E+In+Proceedings+of+the+Fourtee) [our card](https://md.research.elamur.ai/free/papers/schlechtweg-2020-semeval-2020-task-1-unsupervised/)
19. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stickier benchmark for general-purpose language understanding systems. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Alex+Wang%2C+Yada+Pruksachatkun%2C+Nikita+Nangia%2C+Amanpreet+Singh%2C+Julian+Michael%2C+Felix+Hill%2C+Omer+Levy%2C+and+Samuel+Bowman%2E+2019%2E+Superglue%3A+A+stickier+benchmark+for+general%2Dpurpose+language+understandin) [our card](https://md.research.elamur.ai/free/papers/wang-2019-superglue-stickier-benchmark-general-purpose/)
20. Sachin Yadav and Dominik Schlechtweg. 2025. XL-DURel: Finetuning sentence transformers for ordinal word-in-context classification. In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Comput… [find](https://scholar.google.com/scholar?q=Sachin+Yadav+and+Dominik+Schlechtweg%2E+2025%2E+XL%2DDURel%3A+Finetuning+sentence+transformers+for+ordinal+word%2Din%2Dcontext+classification%2E+In+Proceedings+of+the+14th+International+Joint+Conference+on+Natural+)
21. Tadej Škvorc and Marko Robnik-Šikonja. 2025. Solving word-sense disambiguation and word-sense induction with dictionary examples. [find](https://scholar.google.com/scholar?q=Tadej+%C5%A0kvorc+and+Marko+Robnik%2D%C5%A0ikonja%2E+2025%2E+Solving+word%2Dsense+disambiguation+and+word%2Dsense+induction+with+dictionary+examples%2E)