---
title: "Team p1 at SemEval-2026 Task 5 — paper by Rajpoot (2026)"
description: "This paper presents a submission to SemEval 2026 Task 5. 21 references, cited by 1 paper here."
authors: ["Pawan Rajpoot"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.388/"
doi: "10.18653/v1/2026.semeval-1.388"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rajpoot-2026-team-p1-semeval-2026-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rajpoot-2026-team-p1-semeval-2026-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/) / Team p1 at SemEval-2026 Task 5: Semantic Bridge - Augmented Encoding for Word Sense Plausibility

# Team p1 at SemEval-2026 Task 5: Semantic Bridge - Augmented Encoding for Word Sense Plausibility

**Authors:** [Pawan Rajpoot](https://md.research.elamur.ai/free/en/authors/pawan-kumar-rajpoot/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.388/) · [PDF](https://aclanthology.org/2026.semeval-1.388.pdf) · [DOI 10.18653/v1/2026.semeval-1.388](https://doi.org/10.18653/v1/2026.semeval-1.388) · [OpenAlex](https://openalex.org/W7166820274)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
This paper presents a submission to SemEval 2026 Task 5. The task asks systems to score, on a 1–5 scale, how well a given sense of an ambiguous English homonym fits a short story. A prediction counts as correct if it lies within max(SD, 1.0) of the mean of five annotators' ratings. Homonyms do not overlap between train, dev and test.

Rather than asking an LLM for the final number, the author uses LLMs as feature producers. DeepSeek-Chat-V3 generates chain-of-thought fit labels (POOR/PARTIAL/STRONG) in English, Hindi and Korean, gives 4-shot ratings, and translates stories into Hindi and Korean. The idea is that translation forces a commitment to one sense. These reasoning outputs are distilled into Qwen-2.5-7B with LoRA.

System A feeds eight numeric features into XGBoost with Huber loss. System B fine-tunes the bge-reranker-v2-m3 cross-encoder on Korean stories plus English and Korean reasoning text.

An equal-weight average of the two reaches 92.37% test accuracy, above the estimated 89.2% human ceiling. This suggests that representing the same LLM signals in different ways yields complementary errors.

## Key points
- **Dataset:** 3,798 story–sense pairs (2,280 train / 588 dev / 930 test) covering 361 unique homonyms, disjoint across splits. Mean annotator SD is 0.95.
- **Test accuracy:** System A scores 89.78%, System B (retrained on train+dev) 89.35%, and the 50/50 ensemble 859/930 = 92.37%. The ensemble reaches Spearman ρ = 0.8384 and MAE = 0.53.
- **DeepSeek-V3 baselines:** 4-shot prompting reaches 81.6%. The author's zero-shot run reached 71.29%, versus the 79.0% reported for the benchmark.
- **Ablations:**
  - System B on dev scores 71.43% with no enrichment, 86.90% with English only, and 88.27% with English + Korean (best). Adding Hindi lowers it to 85.88%.
  - In System A, the 4-shot score alone as a regressor input gives only 76.88%.
- **Error analysis:** The ensemble makes 71 errors. 82% are shrinkage toward the scale midpoint, and 54% miss the tolerance by under 0.3. "disappear" is the worst homonym.
- **Released resources:** Code is at github.com/pawan2411/semeval26-task5, and the fine-tuned model is pawan2411/semstage7b on Hugging Face.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🔀 Word sense disambiguation](https://md.research.elamur.ai/free/en/topics/word-sense-disambiguation/) · 83 works

## Cite

Cite as: Pawan Rajpoot. 2026. Team p1 at SemEval-2026 Task 5: Semantic Bridge - Augmented Encoding for Word Sense Plausibility. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rajpoot2026team,
  title = {{Team p1 at SemEval-2026 Task 5: Semantic Bridge - Augmented Encoding for Word Sense Plausibility}},
  author = {Pawan Rajpoot},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.388},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.388}
}
```

- [← previous Team Oryu@DravidianLangTech 2026: A Multilingual Transformer Approach for Hope Speech Detection in Code-Mixed Tulu](https://md.research.elamur.ai/free/en/papers/moni-2026-team-oryu-dravidianlangtech-2026-multilingual/)
- [next → Team QUESPA System Submission for the IWSLT 2026 Dialectal and Low-resource Speech Translation Task](https://md.research.elamur.ai/free/en/papers/ortega-2026-team-quespa-system-submission-iwslt/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/)

## References

21 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Carmen Banea, Rada Mihalcea, and Janyce Wiebe. 2011. Word sense disambiguation with multilingual features. In Proceedings of the Ninth International Conference on Computational Semantics (IWCS 2011). [find](https://scholar.google.com/scholar?q=Carmen+Banea%2C+Rada+Mihalcea%2C+and+Janyce+Wiebe%2E+2011%2E+Word+sense+disambiguation+with+multilingual+features%2E+In+Proceedings+of+the+Ninth+International+Conference+on+Computational+Semantics+%28IWCS+2011%29%2E)
2. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. [A large annotated corpus for learning natural language inference](https://md.research.elamur.ai/free/en/papers/bowman-2015-large-annotated-corpus-learning-natural/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642. Association for Computational Linguistics.
3. Katrin Erk and Diana McCarthy. 2009. Graded word sense assignment. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 440–449. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Katrin+Erk+and+Diana+McCarthy%2E+2009%2E+Graded+word+sense+assignment%2E+In+Proceedings+of+the+2009+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+440%E2%80%93449%2E+Association+for+Computation)
4. Janosch Gehring, Selina Meyer, and Michael Roth. 2026. SemEval-2026 task 5: Rating plausibility of word senses in ambiguous stories through narrative understanding. In Proceedings of the 20th International Workshop on Semantic Evaluation, San Diego, California. Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=Janosch+Gehring%2C+Selina+Meyer%2C+and+Michael+Roth%2E+2026%2E+SemEval%2D2026+task+5%3A+Rating+plausibility+of+word+senses+in+ambiguous+stories+through+narrative+understanding%2E+In+Proceedings+of+the+20th+Internat)
5. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, pages 1321–1330. [find](https://scholar.google.com/scholar?q=Chuan+Guo%2C+Geoff+Pleiss%2C+Yu+Sun%2C+and+Kilian+Q%2E+Weinberger%2E+2017%2E+On+calibration+of+modern+neural+networks%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning%2C+pages+1321%E2%80%931330%2E)
6. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). arXiv preprint arXiv:1503.02531.
7. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shanen Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
8. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. [BGE M3-Embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). arXiv preprint arXiv:2402.03216.
9. Luyao Huang, Chi Sun, Xipeng Qiu, and Xuanjing Huang. 2019. [GlossBERT: BERT for word sense disambiguation with gloss knowledge](https://md.research.elamur.ai/free/en/papers/huang-2019-glossbert-bert-word-sense-disambiguation/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-I…
10. Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794. ACM. [find](https://scholar.google.com/scholar?q=Tianqi+Chen+and+Carlos+Guestrin%2E+2016%2E+XGBoost%3A+A+scalable+tree+boosting+system%2E+In+Proceedings+of+the+22nd+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+and+Data+Mining%2C+pages+785%E2%80%93794%2E+A)
11. Els Lefever and Véronique Hoste. 2010. SemEval-2010 task 3: Cross-lingual word sense disambiguation. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 15–20. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Els+Lefever+and+V%C3%A9ronique+Hoste%2E+2010%2E+SemEval%2D2010+task+3%3A+Cross%2Dlingual+word+sense+disambiguation%2E+In+Proceedings+of+the+5th+International+Workshop+on+Semantic+Evaluation%2C+pages+15%E2%80%9320%2E+Association+f)
12. Alexis Conneau, Karttikeya Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Karttikeya+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlin)
13. DeepSeek-AI. 2024. DeepSeek-V3 technical report. arXiv preprint arXiv:2412.19437. [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2E+2024%2E+DeepSeek%2DV3+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E19437%2E)
14. Mona Diab and Philip Resnik. 2002. An unsupervised method for word sense tagging using parallel corpora. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 255–262. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mona+Diab+and+Philip+Resnik%2E+2002%2E+An+unsupervised+method+for+word+sense+tagging+using+parallel+corpora%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pag)
15. Thomas G. Dietterich. 2000. Ensemble methods in machine learning. In International Workshop on Multiple Classifier Systems, pages 1–15. Springer. [find](https://scholar.google.com/scholar?q=Thomas+G%2E+Dietterich%2E+2000%2E+Ensemble+methods+in+machine+learning%2E+In+International+Workshop+on+Multiple+Classifier+Systems%2C+pages+1%E2%80%9315%2E+Springer%2E)
16. Daniel Loureiro, Jose Camacho-Collados, and Luis Espinosa-Anke. 2021. Analysis and evaluation of language models for word sense disambiguation. Computational Linguistics, 47(2):387–443. [find](https://scholar.google.com/scholar?q=Daniel+Loureiro%2C+Jose+Camacho%2DCollados%2C+and+Luis+Espinosa%2DAnke%2E+2021%2E+Analysis+and+evaluation+of+language+models+for+word+sense+disambiguation%2E+Computational+Linguistics%2C+47%282%29%3A387%E2%80%93443%2E)
17. George A. Miller. 1995. [WordNet: A lexical database for English](https://md.research.elamur.ai/free/en/papers/miller-1994-wordnet/). Communications of the ACM, 38(11):39– 41.
18. Roberto Navigli. 2009. Word sense disambiguation: A survey. ACM Computing Surveys, 41(2):1–69. [find](https://scholar.google.com/scholar?q=Roberto+Navigli%2E+2009%2E+Word+sense+disambiguation%3A+A+survey%2E+ACM+Computing+Surveys%2C+41%282%29%3A1%E2%80%9369%2E)
19. John C. Platt. 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers, pages 61–74. MIT Press. [find](https://scholar.google.com/scholar?q=John+C%2E+Platt%2E+1999%2E+Probabilistic+outputs+for+support+vector+machines+and+comparisons+to+regularized+likelihood+methods%2E+In+Advances+in+Large+Margin+Classifiers%2C+pages+61%E2%80%9374%2E+MIT+Press%2E)
20. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems, volume 35.
21. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Wang, Bowen Zheng, Chengyuan Yu, Dayiheng Li, and 1 others. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Wang%2C+Bowen+Zheng%2C+Chengyuan+Yu%2C+Dayiheng+Li%2C+and+1+others%2E+2024%2E+Qwen2%2E5+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E15115%2E)

*Updated 31 October 2026*