---
title: "Word Sense Induction for Novel Sense Detection — Lau 2012"
description: "Lau et al. tackle word sense induction (WSI), the task of discovering a word's senses from unlabelled usage rather… 27 references, cited by 70 papers here."
authors: ["Jey Han Lau", "Paul Cook", "Diana McCarthy", "David Newman", "Timothy Baldwin"]
year: 2012
venue: "Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics"
source: "https://aclanthology.org/E12-1060/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lau-2012-word-sense-induction-novel-sense/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lau-2012-word-sense-induction-novel-sense/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2012](https://md.research.elamur.ai/free/en/papers/2012/) / Word Sense Induction for Novel Sense Detection

# Word Sense Induction for Novel Sense Detection

**Authors:** [Jey Han Lau](https://md.research.elamur.ai/free/en/authors/jey-han-lau/), [Paul Cook](https://md.research.elamur.ai/free/en/authors/paul-cook/), [Diana McCarthy](https://md.research.elamur.ai/free/en/authors/diana-mccarthy/), [David Newman](https://md.research.elamur.ai/free/en/authors/david-newman/) and [Timothy Baldwin](https://md.research.elamur.ai/free/en/authors/timothy-baldwin/) · **Year:** 2012 · **Venue:** Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/E12-1060/) · [PDF](https://aclanthology.org/E12-1060.pdf) · [OpenAlex](https://openalex.org/W1897587124)

**Citations:** 117 citations (OpenAlex, 2026-10-14) · [70](https://md.research.elamur.ai/free/en/papers/lau-2012-word-sense-induction-novel-sense/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
Lau et al. tackle word sense induction (WSI), the task of discovering a word's senses from unlabelled usage rather than from a fixed inventory. They also show that WSI can serve a practical lexicographic purpose: detecting words that have acquired new senses. Each target word gets its own topic model. Every context (a sentence or short fragment) is treated as a "document", and the dominant topic of that context is taken as its sense. Features are lemmatised bag-of-words, plus optional positional words and dependency relations involving the target. A standard LDA model with a preset number of topics is compared against a Hierarchical Dirichlet Process (HDP), which decides how many senses each word needs. On SemEval-2010, HDP beats LDA even when LDA is given the gold number of senses, and it surpasses the best system from the original shared task. It also outperforms other topic-model approaches on SemEval-2007. In the novel sense experiments, BNC serves as the reference corpus and a ukWaC sample as the newer one. The HDP model separates words that gained a sense from distractor words and often isolates the tokens of the new sense.

## Key points
- SemEval-2010 WSI data: 100 target words (50 nouns, 50 verbs), about 163K web training instances and about 9K news test instances. Evaluation is WSD F-score after mapping induced senses to gold senses, using 80%/20% mapping/evaluation splits averaged over 5 random runs.
- Overall F-scores:
  - HDP+Position: 0.68
  - LDA with oracle per-word T: 0.64
  - LDA with fixed T (7 for nouns, 3 for verbs): 0.63
  - UoY benchmark: 0.62
  - MFS baseline: 0.59
  - Adding dependency features to HDP brought no gain.
- A novelty score computed from the induced senses ranks lemmas with a new sense above distractors (one-sided Wilcoxon rank sum test, p \< .05). A baseline based on the ratio of word frequencies between the two corpora was not significant.
- Token-level detection of the new sense (Nov method) reached F-scores of 0.95 for *export* and 0.92 for *worm*, but only 0.45 for *domain* and 0.00 for *poster*.
- Oracle topic selection lifted *poster* to 0.62. Allowing the oracle to pick several topics raised *domain* to 0.92, because its new "Internet domain" sense was split across two topics.

## Interesting ideas
- **One model per target word, with context windows as documents.** Training a separate topic model for each word over its short contexts turns a generic document-clustering tool into a sense clusterer. Each sense then comes with an interpretable weighted word list, which helps when inspecting results for any language.
- **Let a non-parametric model choose the granularity.** HDP beat LDA even when LDA was handed the true sense count. This suggests that fixing the number of clusters can hurt even when the "correct" number is known, which is worth considering for any clustering task where the categories are uncertain.
- **Corpus comparison through induced senses rather than raw frequency.** Inducing senses on pooled data from two corpora and checking which senses are concentrated in the newer one exposed semantic change that frequency ratios missed. The same approach could track meaning shifts across time periods, regions or domains.
- **Oracle upper bounds to locate the bottleneck.** Comparing the automatic sense selection against oracle single-topic and oracle multi-topic choices showed that most errors came from the selection heuristic, not from the induced clusters. This kind of diagnostic separates pipeline stages cleanly in many evaluation setups.
- **Watch for genre mismatch in corpus comparison.** The top false positive, *guess*, came from forum-style usage that was common in ukWaC but rare in BNC. The lesson for any diachronic or cross-corpus study is that the corpora need to be comparable in genre, or register differences will look like new meanings.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [🔀 Word sense disambiguation](https://md.research.elamur.ai/free/en/topics/word-sense-disambiguation/)

## Institutions

- [University of California, Irvine](https://md.research.elamur.ai/free/en/universities/university-of-california-irvine/) — University, Irvine, United States · 22 works · 35,120 citations
- [The University of Melbourne](https://md.research.elamur.ai/free/en/universities/the-university-of-melbourne/) — University, Melbourne, Australia · 47 works · 19,957 citations
- [Data61](https://md.research.elamur.ai/free/en/labs/data61/) — Research institute, Sydney, Australia · 9 works · 2,187 citations
- [Lexical Computing](https://md.research.elamur.ai/free/en/companies/lexical-computing/) — Company, Brighton · 3 works · 472 citations

## Cite

Cite as: Jey Han Lau, Paul Cook, Diana McCarthy, David Newman and Timothy Baldwin. 2012. Word Sense Induction for Novel Sense Detection. Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lau2012word,
  title = {{Word Sense Induction for Novel Sense Detection}},
  author = {Jey Han Lau, Paul Cook, Diana McCarthy, David Newman and Timothy Baldwin},
  booktitle = {Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics},
  year = {2012},
  url = {https://aclanthology.org/E12-1060/}
}
```

- [← previous Paraphrasing for Style](https://md.research.elamur.ai/free/en/papers/xu-2012-paraphrasing-style/)
- [next → ISO 24617-2: A semantically-based standard for dialogue annotation](https://md.research.elamur.ai/free/en/papers/bunt-2012-iso-24617-2-semantically-based/)
- [↑ back to papers of 2012](https://md.research.elamur.ai/free/en/papers/2012/)

## References

27 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eneko Agirre and Aitor Soroa. 2007. SemEval-2007 Task 02: Evaluating word sense induction and discrimination systems. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 7–12, Prague, Czech Republic. [find](https://scholar.google.com/scholar?q=Eneko+Agirre+and+Aitor+Soroa%2E+2007%2E+SemEval%2D2007+Task+02%3A+Evaluating+word+sense+induction+and+discrimination+systems%2E+In+Proceedings+of+the+Fourth+International+Workshop+on+Semantic+Evaluations+%28SemEv)
2. John Ayto. 2006. Movers and Shakers: A Chronology of Words that Shaped our Age. Oxford University Press, Oxford. [find](https://scholar.google.com/scholar?q=John+Ayto%2E+2006%2E+Movers+and+Shakers%3A+A+Chronology+of+Words+that+Shaped+our+Age%2E+Oxford+University+Press%2C+Oxford%2E)
3. David Bamman and Gregory Crane. 2011. Measuring historical word sense variation. In Proceedings of the 2011 Joint International Conference on Digital Libraries (JCDL 2011), pages 1–10, Ottawa, Canada. [find](https://scholar.google.com/scholar?q=David+Bamman+and+Gregory+Crane%2E+2011%2E+Measuring+historical+word+sense+variation%2E+In+Proceedings+of+the+2011+Joint+International+Conference+on+Digital+Libraries+%28JCDL+2011%29%2C+pages+1%E2%80%9310%2C+Ottawa%2C+Canada%2E)
4. D. Blei, A. Ng, and M. Jordan. 2003. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993–1022. [our card](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/)
5. S. Brody and M. Lapata. 2009. Bayesian word sense induction. pages 103–111, Athens, Greece. [our card](https://md.research.elamur.ai/free/en/papers/brody-2009-bayesian-word-sense-induction/)
6. Lou Burnard. 2000. The British National Corpus Users Reference Guide. Oxford University Computing Services. [find](https://scholar.google.com/scholar?q=Lou+Burnard%2E+2000%2E+The+British+National+Corpus+Users+Reference+Guide%2E+Oxford+University+Computing+Services%2E)
7. Paul Cook and Suzanne Stevenson. 2010. Automatically identifying changes in the semantic orientation of words. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC 2010), pages 28–34, Valletta, Malta. [find](https://scholar.google.com/scholar?q=Paul+Cook+and+Suzanne+Stevenson%2E+2010%2E+Automatically+identifying+changes+in+the+semantic+orientation+of+words%2E+In+Proceedings+of+the+Seventh+International+Conference+on+Language+Resources+and+Evaluati)
8. Marie-Catherine De Marneffe, Bill Maccartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. Genoa, Italy. [our card](https://md.research.elamur.ai/free/en/papers/marneffe-2006-generating-typed-dependency-parses-phrase/)
9. Christiane Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2C+editor%2E+1998%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
10. Adriano Ferraresi, Eros Zanchetta, Marco Baroni, and Silvia Bernardini. 2008. Introducing and evaluating ukwac, a very large web-derived corpus of english. In Proceedings of the 4th Web as Corpus Workshop: Can we beat Google, pages 47–54, Marrakech, Morocco. [find](https://scholar.google.com/scholar?q=Adriano+Ferraresi%2C+Eros+Zanchetta%2C+Marco+Baroni%2C+and+Silvia+Bernardini%2E+2008%2E+Introducing+and+evaluating+ukwac%2C+a+very+large+web%2Dderived+corpus+of+english%2E+In+Proceedings+of+the+4th+Web+as+Corpus+Work)
11. William A. Gale, Kenneth W. Church, and David Yarowsky. 1992. One sense per discourse. pages 233–237. [find](https://scholar.google.com/scholar?q=William+A%2E+Gale%2C+Kenneth+W%2E+Church%2C+and+David+Yarowsky%2E+1992%2E+One+sense+per+discourse%2E+pages+233%E2%80%93237%2E)
12. Kristina Gulordava and Marco Baroni. 2011. A distributional similarity approach to the detection of semantic change in the Google Books Ngram corpus. In Proceedings of the GEMS 2011 Workshop on GEometrical Models of Natural Language Semantics, pages 67–71, Edinburgh, Scotland. [our card](https://md.research.elamur.ai/free/en/papers/gulordava-2011-distributional-similarity-approach-detection-semantic/)
13. Adam Kilgarriff and David Tugwell. 2002. Sketching words. In Marie-Hélène Corréard, editor, Lexicography and Natural Language Processing: A Festschrift in Honour of B. T. S. Atkins, pages 125– 137. Euralex, Grenoble, France. [find](https://scholar.google.com/scholar?q=Adam+Kilgarriff+and+David+Tugwell%2E+2002%2E+Sketching+words%2E+In+Marie%2DHe%CC%81le%CC%80ne+Corre%CC%81ard%2C+editor%2C+Lexicography+and+Natural+Language+Processing%3A+A+Festschrift+in+Honour+of+B%2E+T%2E+S%2E+Atkins%2C+pages+125%E2%80%93+137%2E)
14. Dan Klein and Christopher D. Manning. 2003. Fast exact inference with a factored model for natural language parsing. In Advances in Neural Information Processing Systems 15 (NIPS 2002), pages 3– 10, Whistler, Canada. [find](https://scholar.google.com/scholar?q=Dan+Klein+and+Christopher+D%2E+Manning%2E+2003%2E+Fast+exact+inference+with+a+factored+model+for+natural+language+parsing%2E+In+Advances+in+Neural+Information+Processing+Systems+15+%28NIPS+2002%29%2C+pages+3%E2%80%93+10%2C+W)
15. Ioannis Korkontzelos and Suresh Manandhar. 2010. Uoy: Graphs of unambiguous vertices for word sense induction and disambiguation. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 355–358, Uppsala, Sweden. [find](https://scholar.google.com/scholar?q=Ioannis+Korkontzelos+and+Suresh+Manandhar%2E+2010%2E+Uoy%3A+Graphs+of+unambiguous+vertices+for+word+sense+induction+and+disambiguation%2E+In+Proceedings+of+the+5th+International+Workshop+on+Semantic+Evaluatio)
16. Suresh Manandhar, Ioannis Klapaftis, Dmitriy Dligach, and Sameer Pradhan. 2010. SemEval-2010 Task 14: Word sense induction & disambiguation. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 63–68, Uppsala, Sweden. [our card](https://md.research.elamur.ai/free/en/papers/manandhar-2010-semeval-2010-task-14-word/)
17. Roberto Navigli and Giuseppe Crisafulli. 2010. Inducing word senses to improve web search result clustering. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 116–126, Cambridge, USA. [find](https://scholar.google.com/scholar?q=Roberto+Navigli+and+Giuseppe+Crisafulli%2E+2010%2E+Inducing+word+senses+to+improve+web+search+result+clustering%2E+In+Proceedings+of+the+2010+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+)
18. Zheng-Yu Niu, Dong-Hong Ji, and Chew-Lim Tan. 2007. I2R: Three systems for word sense discrimination, chinese word sense disambiguation, and english word sense disambiguation. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 177–182, Prague, Czech Rep… [find](https://scholar.google.com/scholar?q=Zheng%2DYu+Niu%2C+Dong%2DHong+Ji%2C+and+Chew%2DLim+Tan%2E+2007%2E+I2R%3A+Three+systems+for+word+sense+discrimination%2C+chinese+word+sense+disambiguation%2C+and+english+word+sense+disambiguation%2E+In+Proceedings+of+the+Fo)
19. Sebastian Padó and Mirella Lapata. 2007. Dependency-based construction of semantic space models. Comput. Linguist., 33:161–199. [our card](https://md.research.elamur.ai/free/en/papers/pado-2007-dependency-based-construction-semantic-space/)
20. Yves Peirsman, Dirk Geeraerts, and Dirk Speelman. 2010. The automatic identification of lexical variation between language varieties. Natural Language Engineering, 16(4):469–491. [find](https://scholar.google.com/scholar?q=Yves+Peirsman%2C+Dirk+Geeraerts%2C+and+Dirk+Speelman%2E+2010%2E+The+automatic+identification+of+lexical+variation+between+language+varieties%2E+Natural+Language+Engineering%2C+16%284%29%3A469%E2%80%93491%2E)
21. Eyal Sagi, Stefan Kaufmann, and Brady Clark. 2009. Semantic density analysis: Comparing word meaning across time and space. In Proceedings of the EACL 2009 Workshop on GEMS: GEometrical Models of Natural Language Semantics, pages 104– 111, Athens, Greece. [find](https://scholar.google.com/scholar?q=Eyal+Sagi%2C+Stefan+Kaufmann%2C+and+Brady+Clark%2E+2009%2E+Semantic+density+analysis%3A+Comparing+word+meaning+across+time+and+space%2E+In+Proceedings+of+the+EACL+2009+Workshop+on+GEMS%3A+GEometrical+Models+of+Natu)
22. Helmut Schmid. 1994. Probabilistic part-of-speech tagging using decision trees. In Proceedings of the International Conference on New Methods in Language Processing, pages 44–49, Manchester, UK. [find](https://scholar.google.com/scholar?q=Helmut+Schmid%2E+1994%2E+Probabilistic+part%2Dof%2Dspeech+tagging+using+decision+trees%2E+In+Proceedings+of+the+International+Conference+on+New+Methods+in+Language+Processing%2C+pages+44%E2%80%9349%2C+Manchester%2C+UK%2E)
23. Hinrich Schutze. 1998. Automatic word sense discrimination. Computational Linguistics, 24(1):97– 123. [our card](https://md.research.elamur.ai/free/en/papers/schutze-1998-automatic-word-sense-discrimination/)
24. Catherine Soanes and Angus Stevenson, editors. 2008. The Concise Oxford English Dictionary. Oxford University Press, eleventh (revised) edition. Oxford Reference Online. [find](https://scholar.google.com/scholar?q=Catherine+Soanes+and+Angus+Stevenson%2C+editors%2E+2008%2E+The+Concise+Oxford+English+Dictionary%2E+Oxford+University+Press%2C+eleventh+%28revised%29+edition%2E+Oxford+Reference+Online%2E)
25. Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei. 2006. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 101:1566– 1581. [find](https://scholar.google.com/scholar?q=Y%2E+W%2E+Teh%2C+M%2E+I%2E+Jordan%2C+M%2E+J%2E+Beal%2C+and+D%2E+M%2E+Blei%2E+2006%2E+Hierarchical+Dirichlet+processes%2E+Journal+of+the+American+Statistical+Association%2C+101%3A1566%E2%80%93+1581%2E)
26. Della Thompson, editor. 1995. The Concise Oxford Dictionary of Current English. Oxford University Press, Oxford, ninth edition. [find](https://scholar.google.com/scholar?q=Della+Thompson%2C+editor%2E+1995%2E+The+Concise+Oxford+Dictionary+of+Current+English%2E+Oxford+University+Press%2C+Oxford%2C+ninth+edition%2E)
27. Xuchen Yao and Benjamin Van Durme. 2011. Non-parametric bayesian word sense induction. In Proceedings of TextGraphs-6: Graph-based Methods for Natural Language Processing, pages 10–14, Portland, Oregon. [find](https://scholar.google.com/scholar?q=Xuchen+Yao+and+Benjamin+Van+Durme%2E+2011%2E+Non%2Dparametric+bayesian+word+sense+induction%2E+In+Proceedings+of+TextGraphs%2D6%3A+Graph%2Dbased+Methods+for+Natural+Language+Processing%2C+pages+10%E2%80%9314%2C+Portland%2C+Orego)