---
title: "EVWSD-ITA at EVALITA 2026 — paper by Musacchio et al. (2026)"
description: "EVWSD-ITA is a shared task at EVALITA 2026 that extends Visual Word Sense Disambiguation… 20 references, cited by 1 paper here."
authors: ["Elio Musacchio", "Lucia Siciliani", "Pierpaolo Basile", "Giovanni Semeraro"]
year: 2026
venue: "Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)"
source: "https://aclanthology.org/2026.evalita-1.47/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/musacchio-2026-evwsd-ita-evalita-2026-overview/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/musacchio-2026-evwsd-ita-evalita-2026-overview/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) / EVWSD-ITA at EVALITA 2026: Overview of the Enhanced Visual Word Sense Disambiguation for Italian Task

# EVWSD-ITA at EVALITA 2026: Overview of the Enhanced Visual Word Sense Disambiguation for Italian Task

**Authors:** [Elio Musacchio](https://md.research.elamur.ai/free/en/authors/elio-musacchio/), Lucia Siciliani, [Pierpaolo Basile](https://md.research.elamur.ai/free/en/authors/pierpaolo-basile/) and [Giovanni Semeraro](https://md.research.elamur.ai/free/en/authors/giovanni-semeraro/) · **Year:** 2026 · **Venue:** [Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)](https://md.research.elamur.ai/free/en/venues/evalita/) ([Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop 2026](https://md.research.elamur.ai/free/en/venues/evalita/2026/)) · **Publisher:** [CEUR Workshop Proceedings](https://md.research.elamur.ai/free/en/publishers/ceur-workshop-proceedings/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.evalita-1.47/) · [PDF](https://aclanthology.org/2026.evalita-1.47.pdf) · [OpenAlex](https://openalex.org/W7212412634)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
EVWSD-ITA is a shared task at EVALITA 2026 that extends Visual Word Sense Disambiguation (VWSD) to fine-grained distinctions in Italian. In the original SemEval 2023 VWSD setup, wrong candidate images usually belong to unrelated senses, so coarse semantic matching is often enough. In EVWSD-ITA, a system receives a short query with a target word and context words. It must identify the single correct image among ten candidates. The distractors mix two kinds of images: other senses of the same lemma, and "hard negatives". Hard negatives are co-hyponyms that share the target's hypernym, such as a harpsichord instead of a piano.

The data come from BabelNet synsets, their glosses and their "best image". The organizers provide a large automatically built train set. The test set was validated by hand to remove near-identical senses, visually indistinguishable images and abstract concepts, and its queries were written manually.

Only one team, UniTor, participated. Its pipeline combined LLM-generated questions, VQA confidence and CLIP similarity, and it clearly outperformed the multilingual CLIP baseline. The results suggest that multimodal LLMs help with fine-grained visual sense distinctions.

## Key points
- **Task format:** each query has ten candidate images and exactly one correct answer. There are two separate leaderboards, HIT@1 and MRR, because systems that only predict an answer cannot produce the ranking MRR requires.
- **Dataset construction:** the data are built from BabelNet.
  - One gloss is chosen per synset, with sources prioritized in this order: WordNet, Wiktionary, Wikipedia, Wikidata, then other sources.
  - Each query combines the target lemma, a lemma of its hypernym and a word from the gloss.
  - After filtering, test instances with fewer than ten candidates were filled up with random synsets taken from other instances.
- **Baseline:** zero-shot clip-ViT-B-32-multilingual-v1 (sentence-transformers), ranking images by cosine similarity, scored HIT@1 .4505 and MRR .6244.
- **Best system:** UniTor submitted three runs.
  - Its best run, Score VQA + CLIP (Query) + CLIP (Description), reached HIT@1 .7117 and MRR .8182.
  - The pipeline used GPT-5.1 to generate questions and Qwen3-VL Yes/No answer probabilities for VQA scoring.
- **Other UniTor runs:** the run without the description score achieved MRR .8100. The stricter "Two-Step" question parsing variant scored lowest (.6667 HIT@1 / .7770 MRR) but still beat the baseline.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works · [🔀 Word sense disambiguation](https://md.research.elamur.ai/free/en/topics/word-sense-disambiguation/) · 87 works

## Cite

Cite as: Elio Musacchio, Lucia Siciliani, Pierpaolo Basile and Giovanni Semeraro. 2026. EVWSD-ITA at EVALITA 2026: Overview of the Enhanced Visual Word Sense Disambiguation for Italian Task. Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{musacchio2026evwsd,
  title = {{EVWSD-ITA at EVALITA 2026: Overview of the Enhanced Visual Word Sense Disambiguation for Italian Task}},
  author = {Elio Musacchio, Lucia Siciliani, Pierpaolo Basile and Giovanni Semeraro},
  booktitle = {Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)},
  year = {2026},
  url = {https://aclanthology.org/2026.evalita-1.47/}
}
```

- [← previous EvolvR: Self-Evolving Pairwise Reasoning for Story Evaluation to Enhance Generation](https://md.research.elamur.ai/free/en/papers/wang-2026-evolvr-self-evolving-pairwise-reasoning/)
- [next → ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution Reasoning](https://md.research.elamur.ai/free/en/papers/tang-2026-execverify-white-box-rl-verifiable/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/)

## References

20 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. R. Navigli, Word sense disambiguation: A survey, ACM computing surveys (CSUR) 41 (2009) 1–69. [find](https://scholar.google.com/scholar?q=R%2E+Navigli%2C+Word+sense+disambiguation%3A+A+survey%2C+ACM+computing+surveys+%28CSUR%29+41+%282009%29+1%E2%80%9369%2E)
2. M. Bevilacqua, T. Pasini, A. Raganato, R. Navigli, Recent trends in word sense disambiguation: A survey, in: International joint conference on artificial intelligence, International Joint Conference on Artificial Intelligence, Inc, 2021, pp. 4330–4338. [find](https://scholar.google.com/scholar?q=M%2E+Bevilacqua%2C+T%2E+Pasini%2C+A%2E+Raganato%2C+R%2E+Navigli%2C+Recent+trends+in+word+sense+disambiguation%3A+A+survey%2C+in%3A+International+joint+conference+on+artificial+intelligence%2C+International+Joint+Conference+o)
3. D. Sumanathilaka, N. Micallef, J. Hough, Assessing gpt’s potential for word sense disambiguation: A quantitative evaluation on prompt engineering techniques, in: 2024 IEEE 15th Control and System Graduate Research Colloquium (ICSGRC), IEEE, 2024, pp. 204–209. [find](https://scholar.google.com/scholar?q=D%2E+Sumanathilaka%2C+N%2E+Micallef%2C+J%2E+Hough%2C+Assessing+gpt%E2%80%99s+potential+for+word+sense+disambiguation%3A+A+quantitative+evaluation+on+prompt+engineering+techniques%2C+in%3A+2024+IEEE+15th+Control+and+System+Grad)
4. A. Raganato, I. Calixto, A. Ushio, J. Camacho-Collados, M. T. Pilehvar, SemEval-2023 task 1: Visual word sense disambiguation, in: A. K. Ojha, A. S. Doğruöz, G. Da San Martino, H. Tayyar Madabushi, R. Kumar, E. Sartori (Eds.), Proceedings of the 17th International Workshop on Semantic Evaluation (Se… [find](https://scholar.google.com/scholar?q=A%2E+Raganato%2C+I%2E+Calixto%2C+A%2E+Ushio%2C+J%2E+Camacho%2DCollados%2C+M%2E+T%2E+Pilehvar%2C+SemEval%2D2023+task+1%3A+Visual+word+sense+disambiguation%2C+in%3A+A%2E+K%2E+Ojha%2C+A%2E+S%2E+Do%C4%9Fru%C3%B6z%2C+G%2E+Da+San+Martino%2C+H%2E+Tayyar+Madabushi%2C+R%2E)
5. F. Cutugno, A. Miaschi, A. P. Aprosio, G. Rambelli, L. Siciliani, M. A. Stranisci, Evalita 2026: Overview of the 9th evaluation campaign of natural language processing and speech tools for italian, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for I… [find](https://scholar.google.com/scholar?q=F%2E+Cutugno%2C+A%2E+Miaschi%2C+A%2E+P%2E+Aprosio%2C+G%2E+Rambelli%2C+L%2E+Siciliani%2C+M%2E+A%2E+Stranisci%2C+Evalita+2026%3A+Overview+of+the+9th+evaluation+campaign+of+natural+language+processing+and+speech+tools+for+italian%2C+in)
6. T. Pasini, A. Raganato, R. Navigli, Xl-wsd: An extra-large and cross-lingual evaluation framework for word sense disambiguation, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 2021, pp. 13648–13656. [find](https://scholar.google.com/scholar?q=T%2E+Pasini%2C+A%2E+Raganato%2C+R%2E+Navigli%2C+Xl%2Dwsd%3A+An+extra%2Dlarge+and+cross%2Dlingual+evaluation+framework+for+word+sense+disambiguation%2C+in%3A+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volu)
7. L. Huang, C. Sun, X. Qiu, X.-J. Huang, [Glossbert: Bert for word sense disambiguation with gloss knowledge](https://md.research.elamur.ai/free/en/papers/huang-2019-glossbert-bert-word-sense-disambiguation/), in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 35…
8. J. H. Yae, N. C. Skelly, N. C. Ranly, P. M. LaCasse, Leveraging large language models for word sense disambiguation, Neural Computing and Applications 37 (2025) 4093–4110. [find](https://scholar.google.com/scholar?q=J%2E+H%2E+Yae%2C+N%2E+C%2E+Skelly%2C+N%2E+C%2E+Ranly%2C+P%2E+M%2E+LaCasse%2C+Leveraging+large+language+models+for+word+sense+disambiguation%2C+Neural+Computing+and+Applications+37+%282025%29+4093%E2%80%934110%2E)
9. D. Meconi, S. Stirpe, F. Martelli, L. Lavalle, R. Navigli, [Do large language models understand word senses?](https://md.research.elamur.ai/free/en/papers/meconi-2025-llms-word-senses/), in: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 33885–33904.
10. A. Kritharoula, M. Lymperaiou, G. Stamou, Large language models and multimodal retrieval for visual word sense disambiguation, in: H. Bouamor, J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, S… [find](https://scholar.google.com/scholar?q=A%2E+Kritharoula%2C+M%2E+Lymperaiou%2C+G%2E+Stamou%2C+Large+language+models+and+multimodal+retrieval+for+visual+word+sense+disambiguation%2C+in%3A+H%2E+Bouamor%2C+J%2E+Pino%2C+K%2E+Bali+%28Eds%2E%29%2C+Proceedings+of+the+2023+Conferen)
11. S. Kwon, R. Garodia, M. Lee, Z. Yang, H. Yu, Vision meets definitions: Unsupervised visual word sense disambiguation incorporating gloss information, in: A. Rogers, J. Boyd-Graber, N. Okazaki (Eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: L… [find](https://scholar.google.com/scholar?q=S%2E+Kwon%2C+R%2E+Garodia%2C+M%2E+Lee%2C+Z%2E+Yang%2C+H%2E+Yu%2C+Vision+meets+definitions%3A+Unsupervised+visual+word+sense+disambiguation+incorporating+gloss+information%2C+in%3A+A%2E+Rogers%2C+J%2E+Boyd%2DGraber%2C+N%2E+Okazaki+%28Eds%2E%29%2C+)
12. X. Zhang, T. Zhen, J. Zhang, Y. Wang, S. Liu, Srcb at semeval-2023 task 1: Prompt based and cross-modal retrieval enhanced visual word sense disambiguation, in: Proceedings of the 17th international workshop on semantic evaluation (SemEval-2023), 2023, pp. 439–446. [find](https://scholar.google.com/scholar?q=X%2E+Zhang%2C+T%2E+Zhen%2C+J%2E+Zhang%2C+Y%2E+Wang%2C+S%2E+Liu%2C+Srcb+at+semeval%2D2023+task+1%3A+Prompt+based+and+cross%2Dmodal+retrieval+enhanced+visual+word+sense+disambiguation%2C+in%3A+Proceedings+of+the+17th+international+w)
13. Q. Yang, X. Wang, Y. Li, L.-K. Lee, F. L. Wang, T. Hao, Mta: A lightweight multilingual text alignment model for cross-language visual word sense disambiguation, in: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2024, pp. 12166–12170. [find](https://scholar.google.com/scholar?q=Q%2E+Yang%2C+X%2E+Wang%2C+Y%2E+Li%2C+L%2E%2DK%2E+Lee%2C+F%2E+L%2E+Wang%2C+T%2E+Hao%2C+Mta%3A+A+lightweight+multilingual+text+alignment+model+for+cross%2Dlanguage+visual+word+sense+disambiguation%2C+in%3A+ICASSP+2024%2D2024+IEEE+Internationa)
14. Y. Laba, Y. Mohytych, I. Rohulia, H. Kyryleyza, H. Dydyk-Meush, O. Dobosevych, R. Hryniv, Ukrainian visual word sense disambiguation benchmark, in: Proceedings of the Third Ukrainian Natural Language Processing Workshop (UNLP)@ LREC-COLING 2024, 2024, pp. 61–66. [find](https://scholar.google.com/scholar?q=Y%2E+Laba%2C+Y%2E+Mohytych%2C+I%2E+Rohulia%2C+H%2E+Kyryleyza%2C+H%2E+Dydyk%2DMeush%2C+O%2E+Dobosevych%2C+R%2E+Hryniv%2C+Ukrainian+visual+word+sense+disambiguation+benchmark%2C+in%3A+Proceedings+of+the+Third+Ukrainian+Natural+Language+)
15. I. Setitra, P. Rajapaksha, A. K. Myat, N. Crespi, Leveraging ensemble deep models and llm for visual polysemy and word sense disambiguation, Multimedia Tools and Applications (2025) 1–33. [find](https://scholar.google.com/scholar?q=I%2E+Setitra%2C+P%2E+Rajapaksha%2C+A%2E+K%2E+Myat%2C+N%2E+Crespi%2C+Leveraging+ensemble+deep+models+and+llm+for+visual+polysemy+and+word+sense+disambiguation%2C+Multimedia+Tools+and+Applications+%282025%29+1%E2%80%9333%2E)
16. E. Musacchio, L. Siciliani, P. Basile, G. Semeraro, Assessing and improving the multilingual visual word sense disambiguation ability of vision-language models, in: ECAI 2025, IOS Press, 2025, pp. 4145 – 4152. [find](https://scholar.google.com/scholar?q=E%2E+Musacchio%2C+L%2E+Siciliani%2C+P%2E+Basile%2C+G%2E+Semeraro%2C+Assessing+and+improving+the+multilingual+visual+word+sense+disambiguation+ability+of+vision%2Dlanguage+models%2C+in%3A+ECAI+2025%2C+IOS+Press%2C+2025%2C+pp%2E+414)
17. J. Robinson, C.-Y. Chuang, S. Sra, S. Jegelka, Contrastive learning with hard negative samples, arXiv preprint arXiv:2010.04592 (2020). — [arXiv:2010.04592](https://arxiv.org/abs/2010.04592) [find](https://scholar.google.com/scholar?q=J%2E+Robinson%2C+C%2E%2DY%2E+Chuang%2C+S%2E+Sra%2C+S%2E+Jegelka%2C+Contrastive+learning+with+hard+negative+samples%2C+arXiv+preprint+arXiv%3A2010%2E04592+%282020%29%2E+%E2%80%94+%5BarXiv%3A2010%2E04592%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2010%2E04592%29)
18. R. Navigli, S. P. Ponzetto, [Babelnet: Building a very large multilingual semantic network](https://md.research.elamur.ai/free/en/papers/navigli-2010-babelnet-building-very-large-multilingual/), in: Proceedings of the 48th annual meeting of the association for computational linguistics, 2010, pp. 216–225.
19. N. Reimers, I. Gurevych, [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/), in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, 2019. URL: http://arxiv.org/abs/1908.10084. [↗](http://arxiv.org/abs/1908.10084)
20. C. D. Hromei, A. Scaiella, D. Croce, R. Basili, Unitor at evwsd-ita: Zero-shot visual word sense disambiguation via visual question-answering, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari,… [find](https://scholar.google.com/scholar?q=C%2E+D%2E+Hromei%2C+A%2E+Scaiella%2C+D%2E+Croce%2C+R%2E+Basili%2C+Unitor+at+evwsd%2Dita%3A+Zero%2Dshot+visual+word+sense+disambiguation+via+visual+question%2Danswering%2C+in%3A+Proceedings+of+the+Ninth+Evaluation+Campaign+of+Natur)

*Updated 1 November 2026*