---
title: "Integrating Multiple Knowledge Sources to — Ng 1996"
description: "This paper addresses word sense disambiguation (WSD), the task of choosing the correct dictionary sense of a content word in context, using WORDNET senses."
authors: ["Hwee Tou Ng", "Hian Beng Lee"]
year: 1996
venue: "34th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P96-1006/"
doi: "10.3115/981863.981869"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ng-1996-integrating-multiple-knowledge-sources-disambiguate/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ng-1996-integrating-multiple-knowledge-sources-disambiguate/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1996](https://md.research.elamur.ai/free/en/papers/1996/) / Integrating Multiple Knowledge Sources to Disambiguate Word Sense: An Exemplar-Based Approach

# Integrating Multiple Knowledge Sources to Disambiguate Word Sense: An Exemplar-Based Approach

**Authors:** [Hwee Tou Ng](https://md.research.elamur.ai/free/en/authors/hwee-tou-ng/), Hian Beng Lee · **Year:** 1996 · **Venue:** 34th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P96-1006/) · [PDF](https://aclanthology.org/P96-1006.pdf) · [DOI 10.3115/981863.981869](https://doi.org/10.3115/981863.981869) · [OpenAlex](https://openalex.org/W2157025692)

**Citations:** 408 citations (openalex-snapshot · 2026-10-14) · [94](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
This paper addresses word sense disambiguation (WSD), the task of choosing the correct dictionary sense of a content word in context, using WORDNET senses. The authors present LEXAS, a supervised system that trains a separate exemplar-based (nearest-neighbour) classifier for each word, partly built on the public PEBLS learner. Rather than depending on one type of clue, each example combines several feature types: POS tags of the three words on each side, the target's morphological form, sentence-level keywords, local collocations at fixed offsets, and, for nouns, the verb that takes the noun as its object. On the widely used "interest" dataset from Bruce and Wiebe, LEXAS reaches 87.4% mean accuracy, compared with their 78%. To check whether the approach scales, the authors built a new corpus of 192,800 occurrences hand-tagged with WORDNET 1.5 senses, covering 191 frequent and highly ambiguous nouns and verbs. On this harder, noisier data, LEXAS still outperforms most-frequent-sense baselines. The authors describe this as the first large-scale WSD evaluation with fine-grained senses that beats that heuristic.

## Key points
- LEXAS reaches 87.4% on the "interest" noun dataset of Bruce and Wiebe (1994), compared with the 78% they reported.
- The new corpus contains 192,800 occurrences tagged with WORDNET 1.5 senses for 191 nouns and verbs. Linguistics undergraduates produced it with roughly one man-year of effort.
- Results on two held-out test sets:
  - **BC50** (7,119 occurrences from 50 Brown corpus files): Sense 1 scores 40.5%, Most Frequent 47.1%, LEXAS 54.0%.
  - **WSJ6** (14,139 occurrences from 6 WSJ files): Sense 1 scores 44.8%, Most Frequent 63.7%, LEXAS 68.6%.
- Annotation is noisy. Of 5,317 occurrences that also appear in SEMCOR, only about 57% received the same sense in both resources.
- Keywords and collocations are selected with thresholds on conditional probability and frequency. For "interest" the settings were M1 = 0.8, M2 = 5 and M3 = 5.

## Interesting ideas
- **One feature-selection recipe for several clue types.** Keywords, collocations and verb-object pairs are all chosen with the same three conditions: strong prediction of some sense, a minimum count, and a cap per sense. This gives a simple and interpretable way to prune large sparse feature spaces in any classification task.
- **Collocations as offset windows.** Each feature is the exact word string spanning a fixed left/right offset range around the target. This captures function-word patterns such as "in the interest of", which bag-of-words features miss because those words appear everywhere.
- **Summing all evidence instead of using the single best clue.** The authors contrast their approach with decision lists, which classify using only one rule. Pooling every feature through a similarity measure can be more robust when individual clues are weak or contradictory.
- **Cheap syntax via chunk boundaries.** The verb-object relation is approximated as a noun at the end of a noun group immediately preceded by a verb. A shallow heuristic like this can add syntactic signal without a full parser, which is useful for lower-resource languages.
- **Two separate baselines and per-genre test sets.** The evaluation reports both the dictionary's first sense and the most frequent sense in training data, and tests separately on homogeneous text (WSJ) and mixed-genre text (Brown). Measuring inter-annotator agreement against SEMCOR also gives a realistic ceiling for the task.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Word sense disambiguation](https://md.research.elamur.ai/free/en/topics/word-sense-disambiguation/)

## Cited by

Cited by

94 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 93 not yet.

- [Random Walks for Knowledge-Based Word Sense Disambiguation](https://md.research.elamur.ai/free/en/papers/agirre-2014-random-walks-knowledge-based-word/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ng1996integrating,
  title = {{Integrating Multiple Knowledge Sources to Disambiguate Word Sense: An Exemplar-Based Approach}},
  author = {Ng, Hwee Tou and Lee, Hian Beng},
  booktitle = {34th Annual Meeting of the Association for Computational Linguistics},
  year = {1996},
  doi = {10.3115/981863.981869},
  url = {https://doi.org/10.3115/981863.981869}
}
```

- [← previous Chart Generation](https://md.research.elamur.ai/free/en/papers/kay-1996-chart-generation/)
- [next → Interrogatives in Biblical Hebrew: a case study in the Joseph narrative](https://md.research.elamur.ai/free/en/papers/hbo-1996-meister-interrogatives-biblical-hebrew-case-study/)
- [↑ back to papers of 1996](https://md.research.elamur.ai/free/en/papers/1996/)

## References

23 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ezra Black. 1988. An experiment in computational discrimination of English word senses. IBM Journal of Research and Development, 32(2):185-194. [find](https://scholar.google.com/scholar?q=Ezra+Black%2E+1988%2E+An+experiment+in+computational+discrimination+of+English+word+senses%2E+IBM+Journal+of+Research+and+Development%2C+32%282%29%3A185%2D194%2E)
2. Eric Brill. 1992. A simple rule-based part of speech tagger. In Proceedings of the Third Conference on Applied Natural Language Processing, pages 152-155. [find](https://scholar.google.com/scholar?q=Eric+Brill%2E+1992%2E+A+simple+rule%2Dbased+part+of+speech+tagger%2E+In+Proceedings+of+the+Third+Conference+on+Applied+Natural+Language+Processing%2C+pages+152%2D155%2E) [our card](https://md.research.elamur.ai/free/en/papers/brill-1992-simple-rule-based-part-speech/)
3. Rebecca Bruce and Janyce Wiebe. 1994. Word-sense disambiguation using decomposable models. In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, Las Cruces, New Mexico. [find](https://scholar.google.com/scholar?q=Rebecca+Bruce+and+Janyce+Wiebe%2E+1994%2E+Word%2Dsense+disambiguation+using+decomposable+models%2E+In+Proceedings+of+the+32nd+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+Las+Cruces%2C+New+M)
4. Claire Cardie. 1993. A case-based approach to knowledge acquisition for domain-specific sentence analysis. In Proceedings of the Eleventh National Conference on Artificial Intelligence, pages 798-803, Washington, DC. [find](https://scholar.google.com/scholar?q=Claire+Cardie%2E+1993%2E+A+case%2Dbased+approach+to+knowledge+acquisition+for+domain%2Dspecific+sentence+analysis%2E+In+Proceedings+of+the+Eleventh+National+Conference+on+Artificial+Intelligence%2C+pages+798%2D803%2C)
5. Y. Choueka and S. Lusignan. 1985. Disambiguation by short contexts. Computers and the Humanities, 19:147-157. [find](https://scholar.google.com/scholar?q=Y%2E+Choueka+and+S%2E+Lusignan%2E+1985%2E+Disambiguation+by+short+contexts%2E+Computers+and+the+Humanities%2C+19%3A147%2D157%2E)
6. Scott Cost and Steven Salzberg. 1993. A weighted nearest neighbor algorithm for learning with symbolic features. Machine Learning, 10(1):57-78. [find](https://scholar.google.com/scholar?q=Scott+Cost+and+Steven+Salzberg%2E+1993%2E+A+weighted+nearest+neighbor+algorithm+for+learning+with+symbolic+features%2E+Machine+Learning%2C+10%281%29%3A57%2D78%2E)
7. Doug Cutting, Julian Kupiec, Jan Pedersen, and Penelope Sibun. 1992. A practical part-of-speech tagger. In Proceedings of the Third Conference on Applied Natural Language Processing, pages 133-140. [find](https://scholar.google.com/scholar?q=Doug+Cutting%2C+Julian+Kupiec%2C+Jan+Pedersen%2C+and+Penelope+Sibun%2E+1992%2E+A+practical+part%2Dof%2Dspeech+tagger%2E+In+Proceedings+of+the+Third+Conference+on+Applied+Natural+Language+Processing%2C+pages+133%2D140%2E) [our card](https://md.research.elamur.ai/free/en/papers/cutting-1992-practical-part-speech-tagger/)
8. William Gale, Kenneth Ward Church, and David Yarowsky. 1992. Estimating upper and lower bounds on the performance of word-sense disambiguation programs. In Proceedings of the 30th Annual Meeting of the Association for Computational Linguistics, Newark, Delaware. [find](https://scholar.google.com/scholar?q=William+Gale%2C+Kenneth+Ward+Church%2C+and+David+Yarowsky%2E+1992%2E+Estimating+upper+and+lower+bounds+on+the+performance+of+word%2Dsense+disambiguation+programs%2E+In+Proceedings+of+the+30th+Annual+Meeting+of+th)
9. Graeme Hirst. 1987. Semantic Interpretation and the Resolution of Ambiguity. Cambridge University Press, Cambridge. 1 9 7 5 . Computer Recognition of English Word Senses. North-Edward Kelly and Phillip Stone. [find](https://scholar.google.com/scholar?q=Graeme+Hirst%2E+1987%2E+Semantic+Interpretation+and+the+Resolution+of+Ambiguity%2E+Cambridge+University+Press%2C+Cambridge%2E+1+9+7+5+%2E+Computer+Recognition+of+English+Word+Senses%2E+North%2DEdward+Kelly+and+Philli)
10. Claudia Leacock, Geoffrey Towell, and Ellen Voorhees. 1993. Corpus-based statistical sense resolution. In Proceedings of the ARPA Human Language Technology Workshop. [find](https://scholar.google.com/scholar?q=Claudia+Leacock%2C+Geoffrey+Towell%2C+and+Ellen+Voorhees%2E+1993%2E+Corpus%2Dbased+statistical+sense+resolution%2E+In+Proceedings+of+the+ARPA+Human+Language+Technology+Workshop%2E)
11. Alpha K. Luk. 1995. Statistical sense disambiguation with relatively small corpora using dictionary definitions. In Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, Cambridge, Massachusetts. [find](https://scholar.google.com/scholar?q=Alpha+K%2E+Luk%2E+1995%2E+Statistical+sense+disambiguation+with+relatively+small+corpora+using+dictionary+definitions%2E+In+Proceedings+of+the+33rd+Annual+Meeting+of+the+Association+for+Computational+Linguist)
12. Susan W. McRoy 1992. Using multiple knowledge sources for word sense discrimination. Computational Linguistics, 18(1):1-30. [find](https://scholar.google.com/scholar?q=Susan+W%2E+McRoy+1992%2E+Using+multiple+knowledge+sources+for+word+sense+discrimination%2E+Computational+Linguistics%2C+18%281%29%3A1%2D30%2E)
13. George A. Miller, Ed. 1990. WordNet: An on-line lexical database. International Journal of Lezi-cography, 3(4):235-312. [find](https://scholar.google.com/scholar?q=George+A%2E+Miller%2C+Ed%2E+1990%2E+WordNet%3A+An+on%2Dline+lexical+database%2E+International+Journal+of+Lezi%2Dcography%2C+3%284%29%3A235%2D312%2E)
14. George A. Miller, Martin Chodorow, Shari Landes, Claudia Leacock, and Robert G. Thomas. 1994. Using a semantic concordance for sense identification. In Proceedings of the ARPA Human Language Technology Workshop. [find](https://scholar.google.com/scholar?q=George+A%2E+Miller%2C+Martin+Chodorow%2C+Shari+Landes%2C+Claudia+Leacock%2C+and+Robert+G%2E+Thomas%2E+1994%2E+Using+a+semantic+concordance+for+sense+identification%2E+In+Proceedings+of+the+ARPA+Human+Language+Technolog)
15. Paul Procter et al. 1978. Longman Dictionary of Contemporary English. [find](https://scholar.google.com/scholar?q=Paul+Procter+et+al%2E+1978%2E+Longman+Dictionary+of+Contemporary+English%2E)
16. John Rachlin and Steven Salzberg. 1993. PEBLS 3.0 User's Guide. [find](https://scholar.google.com/scholar?q=John+Rachlin+and+Steven+Salzberg%2E+1993%2E+PEBLS+3%2E0+User%27s+Guide%2E)
17. C Stanfill and David Waltz. 1986. Toward memory-based reasoning. Communications of the ACM, 29(12):1213-1228. [find](https://scholar.google.com/scholar?q=C+Stanfill+and+David+Waltz%2E+1986%2E+Toward+memory%2Dbased+reasoning%2E+Communications+of+the+ACM%2C+29%2812%29%3A1213%2D1228%2E)
18. Yorick Wilks, Dan Fass, Cheng-Ming Guo, James E. McDonald, Tony Plate, and Brian M. Slator. 1990. Providing machine tractable dictionary tools. Machine Translation, 5(2):99-154. [find](https://scholar.google.com/scholar?q=Yorick+Wilks%2C+Dan+Fass%2C+Cheng%2DMing+Guo%2C+James+E%2E+McDonald%2C+Tony+Plate%2C+and+Brian+M%2E+Slator%2E+1990%2E+Providing+machine+tractable+dictionary+tools%2E+Machine+Translation%2C+5%282%29%3A99%2D154%2E)
19. David Yarowsky. 1992. Word-sense disambiguation using statistical models of Roger's categories trained on large corpora. In Proceedings of the Fifteenth International Conference on Computational Linguistics, pages 454-460, Nantes, France. [find](https://scholar.google.com/scholar?q=David+Yarowsky%2E+1992%2E+Word%2Dsense+disambiguation+using+statistical+models+of+Roger%27s+categories+trained+on+large+corpora%2E+In+Proceedings+of+the+Fifteenth+International+Conference+on+Computational+Lingu)
20. David Yarowsky. 1993. One sense per collocation. In Proceedings of the ARPA Human Language Technology Workshop. [find](https://scholar.google.com/scholar?q=David+Yarowsky%2E+1993%2E+One+sense+per+collocation%2E+In+Proceedings+of+the+ARPA+Human+Language+Technology+Workshop%2E)
21. David Yarowsky. 1994. Decision lists for lexical ambiguity resolution: Application to accent restoration in Spanish and French. In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, Las Cruces, New Mexico. [find](https://scholar.google.com/scholar?q=David+Yarowsky%2E+1994%2E+Decision+lists+for+lexical+ambiguity+resolution%3A+Application+to+accent+restoration+in+Spanish+and+French%2E+In+Proceedings+of+the+32nd+Annual+Meeting+of+the+Association+for+Computa) [our card](https://md.research.elamur.ai/free/en/papers/yarowsky-1994-decision-lists-lexical-ambiguity-resolution/)
22. David Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, Cambridge, Massachusetts. [find](https://scholar.google.com/scholar?q=David+Yarowsky%2E+1995%2E+Unsupervised+word+sense+disambiguation+rivaling+supervised+methods%2E+In+Proceedings+of+the+33rd+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+Cambridge%2C+Massach) [our card](https://md.research.elamur.ai/free/en/papers/yarowsky-1995-unsupervised-word-sense-disambiguation-rivaling/)
23. Uri Zernik. 1990. Tagging word senses in corpus: the needle in the haystack revisited. Technical Report 90CRD198, GE R&D Center. [find](https://scholar.google.com/scholar?q=Uri+Zernik%2E+1990%2E+Tagging+word+senses+in+corpus%3A+the+needle+in+the+haystack+revisited%2E+Technical+Report+90CRD198%2C+GE+R%26D+Center%2E)