---
title: "Ambirig at SemEval-2026 Task 5 — paper by Roy (2026)"
description: "SemEval-2026 Task 5 asks systems to score how plausible a given sense of a homonym is within a… 10 references, cited by 1 paper here."
authors: ["Soumyajit Roy"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.84/"
doi: "10.18653/v1/2026.semeval-1.84"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/roy-2026-ambirig-semeval-2026-task-5/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/roy-2026-ambirig-semeval-2026-task-5/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/10/) / Ambirig at SemEval-2026 Task 5: Distributional Ordinal Modelling for Ambiguous Word Senses in Narrative Contexts

# Ambirig at SemEval-2026 Task 5: Distributional Ordinal Modelling for Ambiguous Word Senses in Narrative Contexts

**Authors:** [Soumyajit Roy](https://md.research.elamur.ai/free/en/authors/soumyajit-roy/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.84/) · [PDF](https://aclanthology.org/2026.semeval-1.84.pdf) · [DOI 10.18653/v1/2026.semeval-1.84](https://doi.org/10.18653/v1/2026.semeval-1.84) · [OpenAlex](https://openalex.org/W7166847739)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
SemEval-2026 Task 5 asks systems to score how plausible a given sense of a homonym is within a short English story, on a 1–5 scale. Human raters often disagree on these scores. The author frames the task as predicting an ordinal distribution over ratings rather than a single number. The system is a GlossBERT-style cross-encoder built on DeBERTa-v3-base. Its input pairs the sense definition and an example usage with the story, and the target word is wrapped in quotes. A softmax head outputs probabilities for the five ratings. It is trained against the empirical annotator vote distribution using a squared Earth Mover's Distance (EMD) loss over CDFs. With only 2,280 training items, the model reached Spearman 0.593 and accuracy 0.730 on the development set, and 0.49 and 66.6% on the blind test set. It outperformed MSE regression, contrastive, CORAL and entropy-based variants. TAPT, NLI initialisation and complementarity constraints all lowered results. The study suggests that, for truly ambiguous data, a loss that preserves disagreement matters more than model scale or auxiliary training.

## Key points
- Data: 2,280 training and 588 development instances. Each instance has a three-sentence precontext, an ambiguous sentence, an optional ending and a candidate sense.
- Setup: microsoft/deberta-v3-base, AdamW with lr 1.5e-5, batch 8, 5 epochs and 300-token inputs. DeBERTa-Large (435M) overfit.
- Development-set results:
  - EMD: ρ = 0.593, Acc = 0.730.
  - Entropy loss: 0.586.
  - CORAL: 0.518.
  - MSE regression: ≈0.00, because it collapsed to predicting around 3.4–3.5.
- Failed additions:
  - Complementarity constraint: ρ = 0.135.
  - TAPT: ρ = 0.501.
  - NLI initialisation (cross-encoder/nli-deberta-v3-base): ρ = 0.447.
  - Adding a pairwise ranking loss brought no gain.
- On high-variance items, EMD kept a lower MAE than a Cross-Entropy baseline. Code is announced at github.com/roysammy/ambirig.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🔀 Word sense disambiguation](https://md.research.elamur.ai/free/en/topics/word-sense-disambiguation/) · 87 works

## Cite

Cite as: Soumyajit Roy. 2026. Ambirig at SemEval-2026 Task 5: Distributional Ordinal Modelling for Ambiguous Word Senses in Narrative Contexts. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{roy2026ambirig,
  title = {{Ambirig at SemEval-2026 Task 5: Distributional Ordinal Modelling for Ambiguous Word Senses in Narrative Contexts}},
  author = {Soumyajit Roy},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.84},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.84}
}
```

- [← previous AMATI at BEA 2026 Shared Task 2: Automatic Short Answer Grading with Inductive Logic Programming and a Large Language Model](https://md.research.elamur.ai/free/en/papers/willis-2026-amati-bea-2026-shared-task/)
- [next → Among Us: Measuring and Mitigating Malicious Contributions in Model Collaboration Systems](https://md.research.elamur.ai/free/en/papers/yang-2026-among-us-measuring-mitigating-malicious/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/10/)

## References

10 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2009. Evaluation measures for ordinal regression. In 2009 Ninth International Conference on Intelligent Systems Design and Applications, pages 283–287. [find](https://scholar.google.com/scholar?q=Stefano+Baccianella%2C+Andrea+Esuli%2C+and+Fabrizio+Sebastiani%2E+2009%2E+Evaluation+measures+for+ordinal+regression%2E+In+2009+Ninth+International+Conference+on+Intelligent+Systems+Design+and+Applications%2C+pag)
2. Michele Bevilacqua, Tommaso Pasini, Alessandro Raganato, and Roberto Navigli. 2021. Recent trends in word sense disambiguation: A survey. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 4330– 4338. International Joint Conferences on Artifici… [find](https://scholar.google.com/scholar?q=Michele+Bevilacqua%2C+Tommaso+Pasini%2C+Alessandro+Raganato%2C+and+Roberto+Navigli%2E+2021%2E+Recent+trends+in+word+sense+disambiguation%3A+A+survey%2E+In+Proceedings+of+the+Thirtieth+International+Joint+Conference)
3. Janosch Gehring, Selina Meyer, and Michael Roth. 2026. SemEval-2026 task 5: Rating plausibility of word senses in ambiguous stories through narrative understanding. In Proceedings of the 20th International Workshop on Semantic Evaluation, San Diego, California. Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=Janosch+Gehring%2C+Selina+Meyer%2C+and+Michael+Roth%2E+2026%2E+SemEval%2D2026+task+5%3A+Rating+plausibility+of+word+senses+in+ambiguous+stories+through+narrative+understanding%2E+In+Proceedings+of+the+20th+Internat)
4. Xin Geng and Quan Zhao. 2014. Label distribution learning. CoRR, abs/1408.6027. — [arXiv:1408.6027](https://arxiv.org/abs/1408.6027) [find](https://scholar.google.com/scholar?q=Xin+Geng+and+Quan+Zhao%2E+2014%2E+Label+distribution+learning%2E+CoRR%2C+abs%2F1408%2E6027%2E+%E2%80%94+%5BarXiv%3A1408%2E6027%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1408%2E6027%29)
5. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, Online. A… [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Ana+Marasovic%CC%81%2C+Swabha+Swayamdipta%2C+Kyle+Lo%2C+Iz+Beltagy%2C+Doug+Downey%2C+and+Noah+A%2E+Smith%2E+2020%2E+Don%E2%80%99t+stop+pretraining%3A+Adapt+language+models+to+domains+and+tasks%2E+In+Proceedings+of+)
6. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. [DeBERTav3: Improving deBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/). In The Eleventh International Conference on Learning Representations.
7. Le Hou, Chen-Ping Yu, and Dimitris Samaras. 2017. Squared earth mover’s distance-based loss for training deep neural networks. Preprint, arXiv:1611.05916. — [arXiv:1611.05916](https://arxiv.org/abs/1611.05916) [find](https://scholar.google.com/scholar?q=Le+Hou%2C+Chen%2DPing+Yu%2C+and+Dimitris+Samaras%2E+2017%2E+Squared+earth+mover%E2%80%99s+distance%2Dbased+loss+for+training+deep+neural+networks%2E+Preprint%2C+arXiv%3A1611%2E05916%2E+%E2%80%94+%5BarXiv%3A1611%2E05916%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F16)
8. Luyao Huang, Chi Sun, Xipeng Qiu, and Xuanjing Huang. 2019. [GlossBERT: BERT for word sense disambiguation with gloss knowledge](https://md.research.elamur.ai/free/en/papers/huang-2019-glossbert-bert-word-sense-disambiguation/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-I…
9. Ling Li and Hsuan-tien Lin. 2006. Ordinal regression by extended binary classification. In Advances in Neural Information Processing Systems, volume 19. MIT Press. [find](https://scholar.google.com/scholar?q=Ling+Li+and+Hsuan%2Dtien+Lin%2E+2006%2E+Ordinal+regression+by+extended+binary+classification%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+19%2E+MIT+Press%2E)
10. Roberto Navigli. 2009. Word sense disambiguation: A survey. ACM Comput. Surv., 41(2). [find](https://scholar.google.com/scholar?q=Roberto+Navigli%2E+2009%2E+Word+sense+disambiguation%3A+A+survey%2E+ACM+Comput%2E+Surv%2E%2C+41%282%29%2E)

*Updated 31 October 2026*