---
title: "Word Sense Clustering and Clusterability — McCarthy 2016"
description: "The authors define this property as partitionability and estimate it through clusterability , meaning how… 50 references, cited by 23 papers here."
authors: ["Diana McCarthy", "Marianna Apidianaki", "Katrin Erk"]
year: 2016
venue: "Computational Linguistics"
source: "https://aclanthology.org/J16-2003/"
doi: "10.1162/coli_a_00247"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mccarthy-2016-word-sense-clustering-clusterability/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mccarthy-2016-word-sense-clustering-clusterability/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/) / Word Sense Clustering and Clusterability

# Word Sense Clustering and Clusterability

**Authors:** [Diana McCarthy](https://md.research.elamur.ai/free/en/authors/diana-mccarthy/), [Marianna Apidianaki](https://md.research.elamur.ai/free/en/authors/marianna-apidianaki/) and [Katrin Erk](https://md.research.elamur.ai/free/en/authors/katrin-erk/) · **Year:** 2016 · **Venue:** [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/) ([Computational Linguistics 2016](https://md.research.elamur.ai/free/en/venues/cl/2016/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/J16-2003/) · [PDF](https://aclanthology.org/J16-2003.pdf) · [DOI 10.1162/coli\_a\_00247](https://doi.org/10.1162/coli_a_00247) · [OpenAlex](https://openalex.org/W2303034160)

**Citations:** 47 citations (OpenAlex, 2026-10-14) · [23](https://md.research.elamur.ai/free/en/papers/mccarthy-2016-word-sense-clustering-clusterability/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Word sense disambiguation (WSD) and word sense induction (WSI) usually assume that a lemma's occurrences can be split cleanly into senses. In practice, some words have sharply separated meanings (crane), others are vague (thing), and many polysemous words fall in between. The authors define this property as *partitionability* and estimate it through *clusterability*, meaning how much cluster structure exists in a lemma's usage data. They represented instances using manual annotations: English paraphrases from LEXSUB and translations from CLLS. Two families of measures were tested. The first, intra-clust, uses k-means-based metrics from machine learning theory. The second, inter-clust, checks whether clusterings built from paraphrases and from translations agree. Gold estimates of partitionability came from the Usim graded similarity data set. Once polysemy is controlled for, clusterability generally tracks partitionability. The intra-clust metrics perform best and beat a soft-clustering overlap baseline. This matters because clusterability could indicate whether a word suits hard senses, soft clustering, or no clustering at all, and it could help plan annotation effort.

## Key points
- **Gold standards from Usim:** Uiaa is inter-annotator agreement. Umid is the share of mid-scale similarity judgments. Higher clusterability is expected to raise Uiaa and lower Umid.
- **Intra-clust measures:** variance ratio (VR), worst pair ratio (WPR) and separability (SEP). The number of clusters k for these is supplied by a graph-based clustering. VR and SEP gave the most promising results.
- **Inter-clust measures:** V and pF compare LEXSUB-based and CLLS-based clusterings. Their results were less consistent and did not always beat the baseline. The authors attribute this to their reliance on CLLS, which is generally weaker than LEXSUB.
- **Example correlations:** on LEXSUB, low-polysemy band (29 lemmas), VR gave Spearman ρ ≈ −0.61 with Umid and ≈ 0.40 with Uiaa, both in the expected direction. Polysemy bands were analysed only if they contained at least five lemmas.
- **Future work:** the authors plan to apply the measures to automatically generated paraphrases, translations and embeddings, and to compare the outputs of different clustering algorithms.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 256 works · 123,373 citations
- [The University of Texas at Austin](https://md.research.elamur.ai/free/en/universities/the-university-of-texas-at-austin/) — University, Austin, United States · 119 works · 57,950 citations
- [Centre National de la Recherche Scientifique](https://md.research.elamur.ai/free/en/labs/centre-national-de-la-recherche-scientifique/) — Research institute, Paris, France · 169 works · 62,488 citations
- [Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur](https://md.research.elamur.ai/free/en/labs/laboratoire-d-informatique-pour-la-mecanique-et-les-sciences-de-l-ingenieur/) — Research institute, Orsay, France · 25 works · 7,462 citations
- [Université Paris-Saclay](https://md.research.elamur.ai/free/en/universities/universite-paris-saclay/) — University, Gif-sur-Yvette, France · 48 works · 6,336 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [🔀 Word sense disambiguation](https://md.research.elamur.ai/free/en/topics/word-sense-disambiguation/) · 87 works · [📖 Lexical semantics & word embeddings](https://md.research.elamur.ai/free/en/topics/lexical-semantics/) · 65 works

## Cite

Cite as: Diana McCarthy, Marianna Apidianaki and Katrin Erk. 2016. Word Sense Clustering and Clusterability. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{mccarthy2016word,
  title = {{Word Sense Clustering and Clusterability}},
  author = {Diana McCarthy, Marianna Apidianaki and Katrin Erk},
  journal = {Computational Linguistics},
  year = {2016},
  doi = {10.1162/coli\_a\_00247},
  url = {https://doi.org/10.1162/coli_a_00247}
}
```

- [← previous Integer Linear Programming for Discourse Parsing](https://md.research.elamur.ai/free/en/papers/perret-2016-integer-linear-programming-discourse-parsing/)
- [next → rstWeb - A Browser-based Annotation Interface for Rhetorical Structure Theory and Discourse Relations](https://md.research.elamur.ai/free/en/papers/zeldes-2016-rstweb-browser-based-annotation-interface/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/)

## References

50 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ackerman, Margareta and Shai Ben-David. 2009a. Clusterability: A theoretical study. Journal of Machine Learning Research -Proceedings Track, 5:1–8. [find](https://scholar.google.com/scholar?q=Ackerman%2C+Margareta+and+Shai+Ben%2DDavid%2E+2009a%2E+Clusterability%3A+A+theoretical+study%2E+Journal+of+Machine+Learning+Research+%2DProceedings+Track%2C+5%3A1%E2%80%938%2E)
2. Ackerman, Margareta and Shai Ben-David. 2009b. Clusterability: A theoretical study. In Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS), pages 1–8, Clearwater Beach, FL. Agirre, Eneko and Philip Edmonds, editors. 2006. [find](https://scholar.google.com/scholar?q=Ackerman%2C+Margareta+and+Shai+Ben%2DDavid%2E+2009b%2E+Clusterability%3A+A+theoretical+study%2E+In+Proceedings+of+the+Twelfth+International+Conference+on+Artificial+Intelligence+and+Statistics+%28AISTATS%29%2C+pages+1%E2%80%93)
3. Word Sense Disambiguation, Algorithms and Applications. Springer. Apidianaki, Marianna. 2008. Translation-oriented word sense induction based on parallel corpora. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), pages 3269–3275, Marrakech. [find](https://scholar.google.com/scholar?q=Word+Sense+Disambiguation%2C+Algorithms+and+Applications%2E+Springer%2E+Apidianaki%2C+Marianna%2E+2008%2E+Translation%2Doriented+word+sense+induction+based+on+parallel+corpora%2E+In+Proceedings+of+the+Sixth+Internati)
4. Apidianaki, Marianna. 2009. Data-driven semantic analysis for multilingual WSD and lexical selection in translation. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL’09), pages 77–85, Athens. [find](https://scholar.google.com/scholar?q=Apidianaki%2C+Marianna%2E+2009%2E+Data%2Ddriven+semantic+analysis+for+multilingual+WSD+and+lexical+selection+in+translation%2E+In+Proceedings+of+the+12th+Conference+of+the+European+Chapter+of+the+Association+fo)
5. Apidianaki, Marianna, Emilia Verzeni, and Diana McCarthy. 2014. Semantic clustering of pivot paraphrases. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4270–4275, Reykjavik. Artiles, Javier, Enrique Amigó, and Julio Gonzalo. 2009. The rol… [find](https://scholar.google.com/scholar?q=Apidianaki%2C+Marianna%2C+Emilia+Verzeni%2C+and+Diana+McCarthy%2E+2014%2E+Semantic+clustering+of+pivot+paraphrases%2E+In+Proceedings+of+the+Ninth+International+Conference+on+Language+Resources+and+Evaluation+%28LRE)
6. Bannard, Colin and Chris Callison-Burch. 2005. [Paraphrasing with bilingual parallel corpora](https://md.research.elamur.ai/free/en/papers/bannard-2005-paraphrasing-bilingual-parallel-corpora/). In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 597–604, Ann Arbor, MI.
7. Bansal, Mohit, John DeNero, and Dekang Lin. 2012. Unsupervised translation sense clustering. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL: HLT), pages 773–782, Montréal. [find](https://scholar.google.com/scholar?q=Bansal%2C+Mohit%2C+John+DeNero%2C+and+Dekang+Lin%2E+2012%2E+Unsupervised+translation+sense+clustering%2E+In+Proceedings+of+the+2012+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Li)
8. Biemann, Chris and Valerie Nygaard. 2010. Crowdsourcing WordNet. In Proceedings of the 5th Global WordNet Conference, Mumbai. [find](https://scholar.google.com/scholar?q=Biemann%2C+Chris+and+Valerie+Nygaard%2E+2010%2E+Crowdsourcing+WordNet%2E+In+Proceedings+of+the+5th+Global+WordNet+Conference%2C+Mumbai%2E)
9. Carpuat, Marine and Dekai Wu. 2007. [Improving statistical machine translation using word sense disambiguation](https://md.research.elamur.ai/free/en/papers/carpuat-2007-improving-statistical-machine-translation-word/). In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 61–72, Prague.
10. Chen, Jinying and Martha Palmer. 2009. Improving English verb sense disambiguation performance with linguistically motivated features and clear sense distinction boundaries. Journal of Language Resources and Evaluation, Special Issue on SemEval-2007, 43:181–208. [find](https://scholar.google.com/scholar?q=Chen%2C+Jinying+and+Martha+Palmer%2E+2009%2E+Improving+English+verb+sense+disambiguation+performance+with+linguistically+motivated+features+and+clear+sense+distinction+boundaries%2E+Journal+of+Language+Resour)
11. Copestake, Ann and Ted Briscoe. 1995. Semi-productive polysemy and sense extension. Journal of Semantics, 12:15–67. [find](https://scholar.google.com/scholar?q=Copestake%2C+Ann+and+Ted+Briscoe%2E+1995%2E+Semi%2Dproductive+polysemy+and+sense+extension%2E+Journal+of+Semantics%2C+12%3A15%E2%80%9367%2E)
12. Cruse, D. A. 2000. Aspects of the microstructure of word meanings. In Yael Ravin and Claudia Leacock, editors, Polysemy: Theoretical and Computational Approaches. Oxford University Press, pages 30–51. [find](https://scholar.google.com/scholar?q=Cruse%2C+D%2E+A%2E+2000%2E+Aspects+of+the+microstructure+of+word+meanings%2E+In+Yael+Ravin+and+Claudia+Leacock%2C+editors%2C+Polysemy%3A+Theoretical+and+Computational+Approaches%2E+Oxford+University+Press%2C+pages+30%E2%80%9351%2E)
13. Di Marco, Antonio and Roberto Navigli. 2013. [Clustering and diversifying Web search results with graph-based word sense induction](https://md.research.elamur.ai/free/en/papers/marco-2013-clustering-diversifying-web-search-results/). Computational Linguistics, 39(3):709–754.
14. Dyvik, Helge. 1998. Translations as semantic mirrors: From parallel corpus to Wordnet. In Proceedings of the Workshop Multilinguality in the Lexicon II at the 13th Biennial European Conference on Artificial Intelligence (ECAI’98), pages 24–44, Brighton. [find](https://scholar.google.com/scholar?q=Dyvik%2C+Helge%2E+1998%2E+Translations+as+semantic+mirrors%3A+From+parallel+corpus+to+Wordnet%2E+In+Proceedings+of+the+Workshop+Multilinguality+in+the+Lexicon+II+at+the+13th+Biennial+European+Conference+on+Arti)
15. Eom, Soojeong, Markus Dickinson, and Graham Katz. 2012. Using semi-experts to derive judgments on word sense alignment: A pilot study. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 605–611, Istanbul. [find](https://scholar.google.com/scholar?q=Eom%2C+Soojeong%2C+Markus+Dickinson%2C+and+Graham+Katz%2E+2012%2E+Using+semi%2Dexperts+to+derive+judgments+on+word+sense+alignment%3A+A+pilot+study%2E+In+Proceedings+of+the+Eighth+International+Conference+on+Language)
16. Epter, Scott, Mukkai Krishnamoorthy, and Mohammed Zaki. 1999. Clusterability detection and initial seed selection in large data sets. Technical Report 99-6, Rensselaer Polytechnic Institute, Computer Science Department, Troy, NY. [find](https://scholar.google.com/scholar?q=Epter%2C+Scott%2C+Mukkai+Krishnamoorthy%2C+and+Mohammed+Zaki%2E+1999%2E+Clusterability+detection+and+initial+seed+selection+in+large+data+sets%2E+Technical+Report+99%2D6%2C+Rensselaer+Polytechnic+Institute%2C+Computer+)
17. Erk, Katrin, Diana McCarthy, and Nick Gaylord. 2009. Investigations on word senses and word usages. In Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural… [find](https://scholar.google.com/scholar?q=Erk%2C+Katrin%2C+Diana+McCarthy%2C+and+Nick+Gaylord%2E+2009%2E+Investigations+on+word+senses+and+word+usages%2E+In+Proceedings+of+the+47th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+4)
18. Erk, Katrin, Diana McCarthy, and Nick Gaylord. 2013. [Measuring word meaning in context](https://md.research.elamur.ai/free/en/papers/erk-2013-measuring-word-meaning-context/). Computational Linguistics, 39(3):511–554. Fellbaum, Christiane, editor. 1998.
19. WordNet, An Electronic Lexical Database. The MIT Press, Cambridge, MA. Goldberg, Mark K., Mykola Hayvanovych, and Malik Magdon-Ismail. 2010. Measuring similarity between sets of overlapping clusters. In SocialCom/PASSAT, pages 303–308, Minneapolis, MN. [find](https://scholar.google.com/scholar?q=WordNet%2C+An+Electronic+Lexical+Database%2E+The+MIT+Press%2C+Cambridge%2C+MA%2E+Goldberg%2C+Mark+K%2E%2C+Mykola+Hayvanovych%2C+and+Malik+Magdon%2DIsmail%2E+2010%2E+Measuring+similarity+between+sets+of+overlapping+clusters%2E+)
20. Hanks, Patrick. 2000. Do word meanings exist? Computers and the Humanities. Senseval Special Issue, 34(1–2):205–215. [find](https://scholar.google.com/scholar?q=Hanks%2C+Patrick%2E+2000%2E+Do+word+meanings+exist%3F+Computers+and+the+Humanities%2E+Senseval+Special+Issue%2C+34%281%E2%80%932%29%3A205%E2%80%93215%2E)
21. Hovy, Eduard, Mitch Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: The 90% solution. In Proceedings of the 2006 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL: HLT), pages 57–60, New York, NY… [find](https://scholar.google.com/scholar?q=Hovy%2C+Eduard%2C+Mitch+Marcus%2C+Martha+Palmer%2C+Lance+Ramshaw%2C+and+Ralph+Weischedel%2E+2006%2E+OntoNotes%3A+The+90%25+solution%2E+In+Proceedings+of+the+2006+Conference+of+the+North+American+Chapter+of+the+Associatio)
22. Ide, Nancy, Tomaž Erjavec, and Dan Tufiş. 2002. Sense discrimination with parallel corpora. In Proceedings of the ACL’02 Workshop on Word Sense Disambiguation: Recent Successes and Future Directions, pages 54–60, Philadelphia, PA. [find](https://scholar.google.com/scholar?q=Ide%2C+Nancy%2C+Tomaz%CC%8C+Erjavec%2C+and+Dan+Tufis%CC%A7%2E+2002%2E+Sense+discrimination+with+parallel+corpora%2E+In+Proceedings+of+the+ACL%E2%80%9902+Workshop+on+Word+Sense+Disambiguation%3A+Recent+Successes+and+Future+Directions)
23. Ide, Nancy and Yorick Wilks. 2006. Making sense about sense. In Eneko Agirre and Phil Edmonds, editors, Word Sense Disambiguation, Algorithms and Applications. Springer, pages 47–73. [find](https://scholar.google.com/scholar?q=Ide%2C+Nancy+and+Yorick+Wilks%2E+2006%2E+Making+sense+about+sense%2E+In+Eneko+Agirre+and+Phil+Edmonds%2C+editors%2C+Word+Sense+Disambiguation%2C+Algorithms+and+Applications%2E+Springer%2C+pages+47%E2%80%9373%2E)
24. Jurgens, David and Ioannis Klapaftis. 2013. Semeval-2013 task 13: Word sense induction for graded and non-graded senses. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages… [find](https://scholar.google.com/scholar?q=Jurgens%2C+David+and+Ioannis+Klapaftis%2E+2013%2E+Semeval%2D2013+task+13%3A+Word+sense+induction+for+graded+and+non%2Dgraded+senses%2E+In+Second+Joint+Conference+on+Lexical+and+Computational+Semantics+%28%2ASEM%29%2C+Volum)
25. Kilgarriff, Adam. 1998. [‘I don’t believe in word senses’](https://md.research.elamur.ai/free/en/papers/kilgarriff-1997-word-senses/). Computers and the Humanities, 31(2):91–113.
26. Kilgarriff, Adam. 2006. Word Senses. In Eneko Agirre and Phil Edmonds, editors, Word Sense Disambiguation, Algorithms and Applications. Springer, pages 29–46. [find](https://scholar.google.com/scholar?q=Kilgarriff%2C+Adam%2E+2006%2E+Word+Senses%2E+In+Eneko+Agirre+and+Phil+Edmonds%2C+editors%2C+Word+Sense+Disambiguation%2C+Algorithms+and+Applications%2E+Springer%2C+pages+29%E2%80%9346%2E)
27. Krippendorff, Klaus. 1980. Content Analysis: An Introduction to Methodology. [find](https://scholar.google.com/scholar?q=Krippendorff%2C+Klaus%2E+1980%2E+Content+Analysis%3A+An+Introduction+to+Methodology%2E)
28. Sage Publications, Inc., Beverly Hills, CA. Lakoff, George. 1987. Women, Fire, and Dangerous Things: What Categories Reveal about the Mind. University of Chicago Press. [find](https://scholar.google.com/scholar?q=Sage+Publications%2C+Inc%2E%2C+Beverly+Hills%2C+CA%2E+Lakoff%2C+George%2E+1987%2E+Women%2C+Fire%2C+and+Dangerous+Things%3A+What+Categories+Reveal+about+the+Mind%2E+University+of+Chicago+Press%2E)
29. Landes, Shari, Claudia Leacock, and I. Tengi Randee. 1998. Building Semantic Concordances. In Christiane Fellbaum, editor, WordNet: an Electronic Lexical Database. MIT Press, Cambridge, MA, pages 199–237. [find](https://scholar.google.com/scholar?q=Landes%2C+Shari%2C+Claudia+Leacock%2C+and+I%2E+Tengi+Randee%2E+1998%2E+Building+Semantic+Concordances%2E+In+Christiane+Fellbaum%2C+editor%2C+WordNet%3A+an+Electronic+Lexical+Database%2E+MIT+Press%2C+Cambridge%2C+MA%2C+pages+199%E2%80%93)
30. Lefever, Els and Veronique Hoste. 2010. SemEval-2007 Task 3: Cross-lingual word sense disambiguation. In Proceedings of the Fifth International Workshop on Semantic Evaluations (SemEval-2010), pages 15–20, Uppsala. [find](https://scholar.google.com/scholar?q=Lefever%2C+Els+and+Veronique+Hoste%2E+2010%2E+SemEval%2D2007+Task+3%3A+Cross%2Dlingual+word+sense+disambiguation%2E+In+Proceedings+of+the+Fifth+International+Workshop+on+Semantic+Evaluations+%28SemEval%2D2010%29%2C+pages+1)
31. Manandhar, Suresh, Ioannis Klapaftis, Dmitriy Dligach, and Sameer Pradhan. 2010. SemEval-2010 Task 14: Word sense induction and disambiguation. In Proceedings of the Fifth International Workshop on Semantic Evaluation (SemEval-2010), pages 63–68, Uppsala. [find](https://scholar.google.com/scholar?q=Manandhar%2C+Suresh%2C+Ioannis+Klapaftis%2C+Dmitriy+Dligach%2C+and+Sameer+Pradhan%2E+2010%2E+SemEval%2D2010+Task+14%3A+Word+sense+induction+and+disambiguation%2E+In+Proceedings+of+the+Fifth+International+Workshop+on+Se)
32. McCarthy, Diana. 2009. Word sense disambiguation: An overview. Language and Linguistics Compass, 3(2):537–558. [find](https://scholar.google.com/scholar?q=McCarthy%2C+Diana%2E+2009%2E+Word+sense+disambiguation%3A+An+overview%2E+Language+and+Linguistics+Compass%2C+3%282%29%3A537%E2%80%93558%2E)
33. McCarthy, Diana. 2011. Measuring similarity of word meaning in context with lexical substitutes and translations. In Alexander Gelbukh, editor, Computational Linguistics and Intelligent Text Processing, CICLing 2011, Pt. I (Lecture Notes in Computer Science, LNTCS 6608). Springer, pages 238–252. [find](https://scholar.google.com/scholar?q=McCarthy%2C+Diana%2E+2011%2E+Measuring+similarity+of+word+meaning+in+context+with+lexical+substitutes+and+translations%2E+In+Alexander+Gelbukh%2C+editor%2C+Computational+Linguistics+and+Intelligent+Text+Processin)
34. McCarthy, Diana and Roberto Navigli. 2007. [SemEval-2007 Task 10: English lexical substitution task](https://md.research.elamur.ai/free/en/papers/mccarthy-2007-semeval-2007-task-10-english/). In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 48–53, Prague.
35. McCarthy, Diana and Roberto Navigli. 2009. The English lexical substitution task. Language Resources and Evaluation Special Issue on Computational Semantic Analysis of Language: SemEval-2007 and Beyond, 43(2):139–159. [find](https://scholar.google.com/scholar?q=McCarthy%2C+Diana+and+Roberto+Navigli%2E+2009%2E+The+English+lexical+substitution+task%2E+Language+Resources+and+Evaluation+Special+Issue+on+Computational+Semantic+Analysis+of+Language%3A+SemEval%2D2007+and+Beyon)
36. Mihalcea, Rada, Ravi Sinha, and Diana McCarthy. 2010. SemEval-2010 Task 2: Cross-lingual lexical substitution. In Proceedings of the Fifth International Workshop on Semantic Evaluation (SemEval-2010), pages 9–14, Uppsala. [find](https://scholar.google.com/scholar?q=Mihalcea%2C+Rada%2C+Ravi+Sinha%2C+and+Diana+McCarthy%2E+2010%2E+SemEval%2D2010+Task+2%3A+Cross%2Dlingual+lexical+substitution%2E+In+Proceedings+of+the+Fifth+International+Workshop+on+Semantic+Evaluation+%28SemEval%2D2010%29%2C)
37. Mitchell, Jeff and Mirella Lapata. 2008. Vector-based models of semantic composition. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL’08: HLT), pages 236–244, Columbus, OH. [find](https://scholar.google.com/scholar?q=Mitchell%2C+Jeff+and+Mirella+Lapata%2E+2008%2E+Vector%2Dbased+models+of+semantic+composition%2E+In+Proceedings+of+the+46th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%3A+Human+Language+Technol)
38. Navigli, Roberto. 2009. Word sense disambiguation: A survey. ACM Computing Surveys, 41(2):1–69. [find](https://scholar.google.com/scholar?q=Navigli%2C+Roberto%2E+2009%2E+Word+sense+disambiguation%3A+A+survey%2E+ACM+Computing+Surveys%2C+41%282%29%3A1%E2%80%9369%2E)
39. Ostrovsky, Rafail, Yuval Rabani, Leonard J. Schulman, and Chaitanya Swamy. 2006. The effectiveness of Lloyd-type methods for the k-means problem. In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pages 165–176, Berkeley, CA. [find](https://scholar.google.com/scholar?q=Ostrovsky%2C+Rafail%2C+Yuval+Rabani%2C+Leonard+J%2E+Schulman%2C+and+Chaitanya+Swamy%2E+2006%2E+The+effectiveness+of+Lloyd%2Dtype+methods+for+the+k%2Dmeans+problem%2E+In+Proceedings+of+the+47th+Annual+IEEE+Symposium+on+Fo)
40. Palmer, Martha, Hoa Trang Dang, and Joseph Rosenzweig. 2000. [Semantic tagging for the Penn treebank](https://md.research.elamur.ai/free/en/papers/palmer-2000-semantic-tagging-penn-treebank/). In Proceedings of the Second International Conference on Language Resources and Evaluation (LREC’00), pages 699–704, Athens.
41. Passonneau, Rebecca, Ansaf Salleb-Aouissi, Vikas Bhardwaj, and Nancy Ide. 2010. Word sense annotation of polysemous words by multiple annotators. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), pages 3244–3249, Valleta. [find](https://scholar.google.com/scholar?q=Passonneau%2C+Rebecca%2C+Ansaf+Salleb%2DAouissi%2C+Vikas+Bhardwaj%2C+and+Nancy+Ide%2E+2010%2E+Word+sense+annotation+of+polysemous+words+by+multiple+annotators%2E+In+Proceedings+of+the+Seventh+International+Conference)
42. Passonneau, Rebecca J., Collin F. Baker, Christiane Fellbaum, and Nancy Ide. 2012. The MASC word sense corpus. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 3025–3030, Istanbul. [find](https://scholar.google.com/scholar?q=Passonneau%2C+Rebecca+J%2E%2C+Collin+F%2E+Baker%2C+Christiane+Fellbaum%2C+and+Nancy+Ide%2E+2012%2E+The+MASC+word+sense+corpus%2E+In+Proceedings+of+the+Eighth+International+Conference+on+Language+Resources+and+Evaluatio)
43. Pedersen, Ted. 2006. Unsupervised corpus-based methods for WSD. In Eneko Agirre and Phil Edmonds, editors, Word Sense Disambiguation, Algorithms and Applications. Springer, pages 131–166. [find](https://scholar.google.com/scholar?q=Pedersen%2C+Ted%2E+2006%2E+Unsupervised+corpus%2Dbased+methods+for+WSD%2E+In+Eneko+Agirre+and+Phil+Edmonds%2C+editors%2C+Word+Sense+Disambiguation%2C+Algorithms+and+Applications%2E+Springer%2C+pages+131%E2%80%93166%2E)
44. Resnik, Philip and David Yarowsky. 1997. [A perspective on word sense disambiguation methods and their evaluation](https://md.research.elamur.ai/free/en/papers/resnik-1997-perspective-word-sense-disambiguation-methods/). In Proceedings of the ACL SIGLEX Workshop on Tagging Text with Lexical Semantics: Why, What, and How, pages 79–86, Washington, DC.
45. Resnik, Philip and David Yarowsky. 2000. Distinguishing systems and distinguishing senses: New evaluation methods for word sense disambiguation. Natural Language Engineering, 5(3):113–133. [find](https://scholar.google.com/scholar?q=Resnik%2C+Philip+and+David+Yarowsky%2E+2000%2E+Distinguishing+systems+and+distinguishing+senses%3A+New+evaluation+methods+for+word+sense+disambiguation%2E+Natural+Language+Engineering%2C+5%283%29%3A113%E2%80%93133%2E)
46. Rosenberg, Andrew and Julia Hirschberg. 2007. [V-measure: A conditional entropy-based external cluster evaluation measure](https://md.research.elamur.ai/free/en/papers/rosenberg-2007-vmeasure-conditional-entropy-cluster-evaluation/). In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 410–420, Prague.…
47. Sun, Lin and Anna Korhonen. 2009. Improving verb clustering with automatically acquired selectional preferences. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP’09), pages 638–647, Singapore. [find](https://scholar.google.com/scholar?q=Sun%2C+Lin+and+Anna+Korhonen%2E+2009%2E+Improving+verb+clustering+with+automatically+acquired+selectional+preferences%2E+In+Proceedings+of+the+2009+Conference+on+Empirical+Methods+in+Natural+Language+Processi)
48. Tuggy, David H. 1993. Ambiguity, Polysemy and Vagueness. Cognitive Linguistics, 4(2):273–290. Véronis, Jean. 2004. HyperLex: Lexical Cartography for Information Retrieval. Computer Speech & Language, 18(3):223–252. [find](https://scholar.google.com/scholar?q=Tuggy%2C+David+H%2E+1993%2E+Ambiguity%2C+Polysemy+and+Vagueness%2E+Cognitive+Linguistics%2C+4%282%29%3A273%E2%80%93290%2E+Ve%CC%81ronis%2C+Jean%2E+2004%2E+HyperLex%3A+Lexical+Cartography+for+Information+Retrieval%2E+Computer+Speech+%26+Language%2C)
49. Yuret, Deniz. 2007. KU: Word sense disambiguation by substitution. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 207–214, Prague. [find](https://scholar.google.com/scholar?q=Yuret%2C+Deniz%2E+2007%2E+KU%3A+Word+sense+disambiguation+by+substitution%2E+In+Proceedings+of+the+Fourth+International+Workshop+on+Semantic+Evaluations+%28SemEval%2D2007%29%2C+pages+207%E2%80%93214%2C+Prague%2E)
50. Zhang, Bin. 2001. Dependence of clustering algorithm performance on clustered-ness of data. Technical report HP-2001-91, Hewlett-Packard Labs. [find](https://scholar.google.com/scholar?q=Zhang%2C+Bin%2E+2001%2E+Dependence+of+clustering+algorithm+performance+on+clustered%2Dness+of+data%2E+Technical+report+HP%2D2001%2D91%2C+Hewlett%2DPackard+Labs%2E)

*Updated 30 October 2026*