---
title: "Automatically Labeling Semantic Classes — Pantel 2004"
description: "Pantel and Ravichandran present a top-down method that assigns names to such… 68% precision. 23 references, cited by 63 papers here."
authors: ["Patrick Pantel", "Deepak Ravichandran"]
year: 2004
venue: "Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004"
source: "https://aclanthology.org/N04-1041/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pantel-2004-automatically-labeling-semantic-classes/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pantel-2004-automatically-labeling-semantic-classes/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/) / Automatically Labeling Semantic Classes

# Automatically Labeling Semantic Classes

**Authors:** [Patrick Pantel](https://md.research.elamur.ai/free/en/authors/patrick-pantel/) and [Deepak Ravichandran](https://md.research.elamur.ai/free/en/authors/deepak-ravichandran/) · **Year:** 2004 · **Venue:** [Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2004](https://md.research.elamur.ai/free/en/venues/naacl/2004/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/N04-1041/) · [PDF](https://aclanthology.org/N04-1041.pdf) · [OpenAlex](https://openalex.org/W69597389)

**Citations:** 214 citations (OpenAlex, 2026-10-14) · [63](https://md.research.elamur.ai/free/en/papers/pantel-2004-automatically-labeling-semantic-classes/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Clustering systems such as CBC can group words into semantic classes, for example diseases, hockey goalies or colors, but they leave these classes unnamed. Pantel and Ravichandran present a top-down method that assigns names to such classes and then uses those names to produce is-a (hyponym) pairs. Each word is described by grammatical contexts weighted with discounted pointwise mutual information. For every class, a small "committee" of highly similar core members is chosen, and their vectors are averaged into a grammatical signature. The system then searches this signature for simple syntactic patterns, such as apposition, that point to a class name, and outputs a ranked list of candidate labels. Every instance in a labeled class becomes a hyponym of the label. The method named 98.5% of 1,432 CBC noun concepts, with an MRR of 77.1% in human evaluation. It extracted 159,000 hyponyms at 68% precision, rising to 81.5% for proper names. The work matters because it turns unnamed clusters into usable taxonomic knowledge that helps question answering.

## Key points
- The labeling algorithm accepts word clusters from any source. The experiments use CBC output, and the result is a ranked list of names for each class.
- The pipeline has three phases:
  - word feature vectors built from grammatical contexts, weighted by PMI with a discount that counters PMI's bias toward rare items;
  - committee selection through group-average clustering, scored by cluster size × average pairwise similarity;
  - pattern search over the averaged signature, e.g. the N:appo:N relation, which yields goaltender/goalie for a cluster of NHL goalies.
- Results on 1,432 noun concepts:
  - 98.5% received a label, with an MRR of 77.1%;
  - 159,000 hyponyms were extracted at 68% precision;
  - 65,000 proper-name hyponyms reached 81.5% precision.
- On 50 TREC-2003 definition questions, the system's top-5 answers were correct 60% of the time (strict) and 62% (lenient). WordNet scored 38%, and Fleischman et al. scored 42% and 44%.
- Semantic indexing of the TREC-2002 collection with the extracted hyponyms helped on 179 TREC-2003 questions that have an explicit answer type:
  - correct top-1 passages rose from 36 to 43, roughly a 20% gain;
  - correct top-100 passages rose from 131 to 134.

## Interesting ideas
- **Label the class, not each word.** Searching a pooled signature for a whole class avoids the sparse and ambiguous evidence of individual words. A rare instance can inherit a hypernym simply by belonging to the class, which could help low-resource languages where many words appear only a few times.
- **Represent a class by its committee.** Averaging only the tight core of a cluster gives a cleaner profile than averaging every member, because noisy or polysemous members are left out. The same idea could work as a general denoising step for prototype vectors in other clustering tasks.
- **Score candidate clusters by size × average similarity.** This simple metric rewards groups that are both large and coherent. It is a cheap heuristic that could be reused for selecting seed sets elsewhere.
- **Discount PMI by frequency.** Multiplying PMI by a frequency-based factor damps the inflated scores of rare words and features. This is useful in any distributional pipeline built on co-occurrence counts.
- **Measure recall indirectly through QA tasks.** Recall for knowledge extraction is hard to measure directly. Running definition questions and passage retrieval with and without the resource gives a practical, application-based proxy for coverage.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Patrick Pantel and Deepak Ravichandran. 2004. Automatically Labeling Semantic Classes. Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pantel2004automatically,
  title = {{Automatically Labeling Semantic Classes}},
  author = {Patrick Pantel and Deepak Ravichandran},
  booktitle = {Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004},
  year = {2004},
  url = {https://aclanthology.org/N04-1041/}
}
```

- [← previous Identifying Agreement and Disagreement in Conversational Speech: Use of Bayesian Networks to Model Pragmatic Dependencies](https://md.research.elamur.ai/free/en/papers/galley-2004-identifying-agreement-disagreement-conversational-speech/)
- [next → CorMet: A Computational, Corpus-Based Conventional Metaphor Extraction System](https://md.research.elamur.ai/free/en/papers/mason-2004-cormet-computational-corpus-based-conventional/)
- [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/)

## References

23 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Barwise, J. and Perry, J. 1985. Semantic innocence and uncompromising situations. In: Martinich, A. P. (ed.) The Philosophy of Language. New York: Oxford University Press. pp. 401–413. [find](https://scholar.google.com/scholar?q=Barwise%2C+J%2E+and+Perry%2C+J%2E+1985%2E+Semantic+innocence+and+uncompromising+situations%2E+In%3A+Martinich%2C+A%2E+P%2E+%28ed%2E%29+The+Philosophy+of+Language%2E+New+York%3A+Oxford+University+Press%2E+pp%2E+401%E2%80%93413%2E)
2. Berland, M. and E. Charniak, 1999. [Finding parts in very large corpora](https://md.research.elamur.ai/free/en/papers/berland-1999-finding-parts-very-large-corpora/). In ACL-1999. pp. 57–64. College Park, MD.
3. Church, K. and Hanks, P. 1989. [Word association norms, mutual information, and lexicography](https://md.research.elamur.ai/free/en/papers/church-1990-word-association-norms-mutual-information/). In Proceedings of ACL-89. pp. 76–83. Vancouver, Canada.
4. Fleischman, M.; Hovy, E.; and Echihabi, A. 2003. Offline strategies for online question answering: Answering questions before they are asked. In Proceedings of ACL-03. pp. 1–7. Sapporo, Japan. [find](https://scholar.google.com/scholar?q=Fleischman%2C+M%2E%3B+Hovy%2C+E%2E%3B+and+Echihabi%2C+A%2E+2003%2E+Offline+strategies+for+online+question+answering%3A+Answering+questions+before+they+are+asked%2E+In+Proceedings+of+ACL%2D03%2E+pp%2E+1%E2%80%937%2E+Sapporo%2C+Japan%2E)
5. Girju, R.; Badulescu, A.; and Moldovan, D. 2003. [Learning semantic constraints for the automatic discovery of part-whole relations](https://md.research.elamur.ai/free/en/papers/girju-2003-learning-semantic-constraints-automatic-discovery/). In Proceedings of HLT/NAACL-03. pp. 80–87. Edmonton, Canada.
6. Han, J. and Kamber, M. 2001. Data Mining – Concepts and Techniques. Morgan Kaufmann. [find](https://scholar.google.com/scholar?q=Han%2C+J%2E+and+Kamber%2C+M%2E+2001%2E+Data+Mining+%E2%80%93+Concepts+and+Techniques%2E+Morgan+Kaufmann%2E)
7. Hearst, M. 1992. Automatic acquisition of hyponyms from large text corpora. In COLING-92. pp. 539–545. Nantes, France. [find](https://scholar.google.com/scholar?q=Hearst%2C+M%2E+1992%2E+Automatic+acquisition+of+hyponyms+from+large+text+corpora%2E+In+COLING%2D92%2E+pp%2E+539%E2%80%93545%2E+Nantes%2C+France%2E)
8. Hindle, D. 1990. [Noun classification from predicate-argument structures](https://md.research.elamur.ai/free/en/papers/hindle-1990-noun-classification-predicate-argument-structures/). In Proceedings of ACL-90. pp. 268–275. Pittsburgh, PA.
9. Leacock, C.; Chodorow, M.; and Miller; G. A. 1998. [Using corpus statistics and WordNet relations for sense identification](https://md.research.elamur.ai/free/en/papers/leacock-1998-corpus-statistics-wordnet-relations-sense/). Computational Linguistics, 24(1):147–165.
10. Lenat, D. 1995. CYC: A large-scale investment in knowledge infrastructure. Communications of the ACM, 38(11):33–38. [find](https://scholar.google.com/scholar?q=Lenat%2C+D%2E+1995%2E+CYC%3A+A+large%2Dscale+investment+in+knowledge+infrastructure%2E+Communications+of+the+ACM%2C+38%2811%29%3A33%E2%80%9338%2E)
11. Lin, D. 1994. Principar - an efficient, broad-coverage, principle-based parser. Proceedings of COLING-94. pp. 42–48. Kyoto, Japan. [find](https://scholar.google.com/scholar?q=Lin%2C+D%2E+1994%2E+Principar+%2D+an+efficient%2C+broad%2Dcoverage%2C+principle%2Dbased+parser%2E+Proceedings+of+COLING%2D94%2E+pp%2E+42%E2%80%9348%2E+Kyoto%2C+Japan%2E)
12. Lin, D. 1998. [Automatic retrieval and clustering of similar words](https://md.research.elamur.ai/free/en/papers/lin-1998-automatic-retrieval-clustering-similar-words/). In Proceedings of COLING/ACL-98. pp. 768–774. Montreal, Canada.
13. Lin, D. and Pantel, P. 2001. Induction of semantic classes from natural language text. In Proceedings of SIGKDD-01. pp. 317–322. San Francisco, CA. [find](https://scholar.google.com/scholar?q=Lin%2C+D%2E+and+Pantel%2C+P%2E+2001%2E+Induction+of+semantic+classes+from+natural+language+text%2E+In+Proceedings+of+SIGKDD%2D01%2E+pp%2E+317%E2%80%93322%2E+San+Francisco%2C+CA%2E)
14. Mann, G. S. 2002. Fine-Grained Proper Noun Ontologies for Question Answering. SemaNet’ 02: Building and Using Semantic Networks, Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=Mann%2C+G%2E+S%2E+2002%2E+Fine%2DGrained+Proper+Noun+Ontologies+for+Question+Answering%2E+SemaNet%E2%80%99+02%3A+Building+and+Using+Semantic+Networks%2C+Taipei%2C+Taiwan%2E)
15. Miller, G. 1990. WordNet: An online lexical database. International Journal of Lexicography, 3(4). [find](https://scholar.google.com/scholar?q=Miller%2C+G%2E+1990%2E+WordNet%3A+An+online+lexical+database%2E+International+Journal+of+Lexicography%2C+3%284%29%2E)
16. Pasca, M. and Harabagiu, S. 2001. The informative role of WordNet in Open-Domain Question Answering. In Proceedings of NAACL-01 Workshop on WordNet and Other Lexical Resources. pp. 138–143. Pittsburgh, PA. [find](https://scholar.google.com/scholar?q=Pasca%2C+M%2E+and+Harabagiu%2C+S%2E+2001%2E+The+informative+role+of+WordNet+in+Open%2DDomain+Question+Answering%2E+In+Proceedings+of+NAACL%2D01+Workshop+on+WordNet+and+Other+Lexical+Resources%2E+pp%2E+138%E2%80%93143%2E+Pittsburgh)
17. Pantel, P. and Lin, D. 2002. Discovering Word Senses from Text. In Proceedings of SIGKDD-02. pp. 613–619. Edmonton, Canada. [find](https://scholar.google.com/scholar?q=Pantel%2C+P%2E+and+Lin%2C+D%2E+2002%2E+Discovering+Word+Senses+from+Text%2E+In+Proceedings+of+SIGKDD%2D02%2E+pp%2E+613%E2%80%93619%2E+Edmonton%2C+Canada%2E)
18. Riloff, E. and Shepherd, J. 1997. [A corpus-based approach for building semantic lexicons](https://md.research.elamur.ai/free/en/papers/riloff-1997-corpus-based-approach-building-semantic/). In Proceedings of EMNLP-1997.
19. Riloff, E. and Jones, R. 1999. Learning dictionaries for information extraction by multi-level bootstrapping. In Proceedings of AAAI-99. pp. 474–479. Orlando, FL. [find](https://scholar.google.com/scholar?q=Riloff%2C+E%2E+and+Jones%2C+R%2E+1999%2E+Learning+dictionaries+for+information+extraction+by+multi%2Dlevel+bootstrapping%2E+In+Proceedings+of+AAAI%2D99%2E+pp%2E+474%E2%80%93479%2E+Orlando%2C+FL%2E)
20. Salton, G. and McGill, M. J. 1983. Introduction to Modern Information Retrieval. [find](https://scholar.google.com/scholar?q=Salton%2C+G%2E+and+McGill%2C+M%2E+J%2E+1983%2E+Introduction+to+Modern+Information+Retrieval%2E)
21. McGraw Hill Schank, R. and Abelson, R. 1977. Scripts, Plans, Goals and Understanding: An Inquiry into Human Knowledge Structures. Lawrence Erlbaum Associates. [find](https://scholar.google.com/scholar?q=McGraw+Hill+Schank%2C+R%2E+and+Abelson%2C+R%2E+1977%2E+Scripts%2C+Plans%2C+Goals+and+Understanding%3A+An+Inquiry+into+Human+Knowledge+Structures%2E+Lawrence+Erlbaum+Associates%2E)
22. Siegel, S. and Castellan Jr., N. J. 1988. Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill. [find](https://scholar.google.com/scholar?q=Siegel%2C+S%2E+and+Castellan+Jr%2E%2C+N%2E+J%2E+1988%2E+Nonparametric+Statistics+for+the+Behavioral+Sciences%2E+McGraw%2DHill%2E)
23. Voorhees, E. 2003. Overview of the question answering track. To appear in Proceedings of TREC-12 Conference. NIST, Gaithersburg, MD. [find](https://scholar.google.com/scholar?q=Voorhees%2C+E%2E+2003%2E+Overview+of+the+question+answering+track%2E+To+appear+in+Proceedings+of+TREC%2D12+Conference%2E+NIST%2C+Gaithersburg%2C+MD%2E)

*Updated 24 October 2026*