--- title: "Automatic construction of a hypernym-labeled noun hierarchy from text — Caraballo 1999" description: "WordNet is built by hand and covers domain-specific text, such as MUC material, poorly. 9 references, cited by 68 papers here." authors: ["Sharon A. Caraballo"] year: 1999 venue: "Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics" source: "https://aclanthology.org/P99-1016/" doi: "10.3115/1034678.1034705" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/caraballo-1999-automatic-construction-hypernym-labeled-noun/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/caraballo-1999-automatic-construction-hypernym-labeled-noun/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1999](https://md.research.elamur.ai/free/en/papers/1999/) / Automatic construction of a hypernym-labeled noun hierarchy from text # Automatic construction of a hypernym-labeled noun hierarchy from text **Authors:** Sharon A. Caraballo · **Year:** 1999 · **Venue:** Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/P99-1016/) · [PDF](https://aclanthology.org/P99-1016.pdf) · [DOI 10.3115/1034678.1034705](https://doi.org/10.3115/1034678.1034705) · [OpenAlex](https://openalex.org/W2100377551) **Citations:** 373 citations (OpenAlex, 2026-10-14) · [68](https://md.research.elamur.ai/free/en/papers/caraballo-1999-automatic-construction-hypernym-labeled-noun/cited-by/) in our library, of the 140,884 papers whose bibliographies we read ## Summary WordNet is built by hand and covers domain-specific text, such as MUC material, poorly. Caraballo asks whether a WordNet-like noun taxonomy with hypernym labels can be induced from a raw corpus alone, without other lexical resources. The data are parsed Wall Street Journal text: Penn Treebank files 2–21 plus the 1987 WSJ, parsed with the Charniak et al. (1998) parser. Head nouns that co-occur in NP coordinations and appositives give each noun a count vector. Nouns are then merged bottom-up by cosine similarity with size-weighted average linkage, in memory-bounded batches. Next, Hearst-style "X, Y, and other Zs" patterns supply hypernym evidence. Votes from the leaves are pooled at each internal node, and the three strongest labels are kept. A pruning pass then collapses redundant binary structure. The result is a tree of 20,014 nouns under 654 internal nodes. Human judges accepted the top label for 33% of sampled nouns under the strictest criterion. Under the loosest criterion, 60% of nouns received at least one acceptable label. This is on par with seed-based lexicon builders that work on hand-chosen categories, which suggests domain-tailored taxonomies can be learned automatically. ## Key points - **Clustering data and representation:** - About 50,000 stemmed head nouns were extracted. - Only nouns with vector length ≥2 were kept, leaving roughly 20,000 leaves. - Group similarity is the mean pairwise cosine between members. - **Memory limit:** a full 50,000×50,000 similarity matrix was infeasible. Instead, 5,000 nouns are clustered down to 2,500 nodes, then 2,500 new nouns are added, and the cycle repeats. - **Compression:** internal nodes fell from 20,013 to 654. The 1,288 nouns with too little evidence, mostly proper names, ended up directly under the root. The root itself is labeled "product/analyst/official". - **Evaluation design:** - 10 random nodes, each with ≥20 nouns, were sampled, with 20 nouns per node. - Three NLP students rated every noun–hypernym pair. - 5 hidden "noise" nouns per cluster tested judge leniency. - **Results compared with prior work:** - Top label accepted by at least one judge: 39%, versus about 31% for Riloff and Shepherd (1997). - Roark and Charniak (1998) report 20–40% for hand-selected categories. ## Interesting ideas - **Label clusters, not single nouns.** Pooling pattern-based hypernym hits over all of a cluster's descendants smooths out the noisy, context-dependent pairs that Hearst patterns produce. Requiring support from at least two descendants adds a cheap evidence threshold. The same approach suits any weak-label aggregation over clusters. - **Rolling-batch agglomerative clustering.** Clustering a fixed-size working set down to half its size, then topping it up with new items, keeps memory bounded. This is a practical approximation whenever a full pairwise similarity matrix will not fit. - **One simple pruning rule.** A depth-first pass removes any child node that is unlabeled or carries the same top-three labels as its parent. This turns an overly deep binary dendrogram into a readable n-ary taxonomy, and it works for any hierarchical clustering output. - **Hidden distractors in human evaluation.** Randomly drawn noise items mixed into each cluster reveal whether annotators are too generous. This is an inexpensive sanity check for any judgment-based evaluation. - **Several candidate labels per node.** Listing up to three labels per node covers near-synonym alternation (e.g., "country"/"nation") and clusters that no single word describes. Reporting both strict and lenient scores, by majority or any judge, makes the comparison with earlier systems fair. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [📖 Lexical semantics & word embeddings](https://md.research.elamur.ai/free/en/topics/lexical-semantics/) ## Institutions - [Brown University](https://md.research.elamur.ai/free/en/universities/brown-university/) — University, Providence, United States · 40 works · 21,721 citations ## Cite Cite as: Sharon A. Caraballo. 1999. Automatic construction of a hypernym-labeled noun hierarchy from text. Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{caraballo1999automatic, title = {{Automatic construction of a hypernym-labeled noun hierarchy from text}}, author = {Sharon A. Caraballo}, booktitle = {Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics}, year = {1999}, doi = {10.3115/1034678.1034705}, url = {https://doi.org/10.3115/1034678.1034705} } ``` - [← previous Information Fusion in the Context of Multi-Document Summarization](https://md.research.elamur.ai/free/en/papers/barzilay-1999-information-fusion-context-multi-document/) - [next → An Efficient Method for Determining Bilingual Word Classes](https://md.research.elamur.ai/free/en/papers/och-1999-efficient-method-determining-bilingual-word/) - [↑ back to papers of 1999](https://md.research.elamur.ai/free/en/papers/1999/) ## References 9 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Thomas Ahlswede and Martha Evens. 1988. Parsing vs. text processing in the analysis of dictionary definitions. In Proceedings of the 29th Annual Meeting of the Association for Computational Linguistics, pages 217-224. [find](https://scholar.google.com/scholar?q=Thomas+Ahlswede+and+Martha+Evens%2E+1988%2E+Parsing+vs%2E+text+processing+in+the+analysis+of+dictionary+definitions%2E+In+Proceedings+of+the+29th+Annual+Meeting+of+the+Association+for+Computational+Linguistic) 2. Peter F. Brown, Vincent J. Della Pietra, Peter V. DeSouza, Jennifer C. Lai, and Robert L. Mercer. 1992. Class-based ngram models of natural language. Computational Linguistics, 18:467-479. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Vincent+J%2E+Della+Pietra%2C+Peter+V%2E+DeSouza%2C+Jennifer+C%2E+Lai%2C+and+Robert+L%2E+Mercer%2E+1992%2E+Class%2Dbased+ngram+models+of+natural+language%2E+Computational+Linguistics%2C+18%3A467%2D479%2E) 3. Eugene Charniak, Sharon Goldwater, and Mark Johnson. 1998. Edge-based best-first chart parsing. In Proceedings of the Sixth Workshop on Very Large Corpora, pages 127-133. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Eugene+Charniak%2C+Sharon+Goldwater%2C+and+Mark+Johnson%2E+1998%2E+Edge%2Dbased+best%2Dfirst+chart+parsing%2E+In+Proceedings+of+the+Sixth+Workshop+on+Very+Large+Corpora%2C+pages+127%2D133%2E+Association+for+Computational) 4. Christiane Fellbaum, editor. 1998. Word-Net: An Electronic Lexical Database. MIT Press. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2C+editor%2E+1998%2E+Word%2DNet%3A+An+Electronic+Lexical+Database%2E+MIT+Press%2E) 5. Marti A. Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of the Fourteenth International Conference on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Marti+A%2E+Hearst%2E+1992%2E+Automatic+acquisition+of+hyponyms+from+large+text+corpora%2E+In+Proceedings+of+the+Fourteenth+International+Conference+on+Computational+Linguistics%2E) 6. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19:313-330. [our card](https://md.research.elamur.ai/free/en/papers/marcus-1993-building-large-annotated-corpus-english/) 7. Fernando Pereira, Naftali Tishby, and Lillian Lee. 1993. Distributional clustering of English words. In Proceedings of the 31st Annual Meeting of the Association for Computational Linguistics, pages 183-190. [our card](https://md.research.elamur.ai/free/en/papers/pereira-1993-distributional-clustering-english-words/) 8. Ellen Riloff and Jessica Shepherd. 1997. A corpus-based approach for building semantic lexicons. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing, pages 117-124. [our card](https://md.research.elamur.ai/free/en/papers/riloff-1997-corpus-based-approach-building-semantic/) 9. Brian Roark and Eugene Charniak. 1998. Noun-phrase co-occurrence statistics for semi-automatic semantic lexicon construction. In COLING-ACL '98: 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics: Proceedings of the Con… [find](https://scholar.google.com/scholar?q=Brian+Roark+and+Eugene+Charniak%2E+1998%2E+Noun%2Dphrase+co%2Doccurrence+statistics+for+semi%2Dautomatic+semantic+lexicon+construction%2E+In+COLING%2DACL+%2798%3A+36th+Annual+Meeting+of+the+Association+for+Computationa)