---
title: "Inducing a Semantically Annotated Lexicon via EM-Based"
description: "Most earlier work on learning selectional restrictions depends on a fixed semantic taxonomy such as… 17 references, cited by 72 papers here."
authors: ["Mats Rooth", "Stefan Riezler", "Detlef Prescher", "Glenn Carroll", "Franz Beil"]
year: 1999
venue: "Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P99-1014/"
doi: "10.3115/1034678.1034703"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rooth-1999-inducing-semantically-annotated-lexicon-em/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rooth-1999-inducing-semantically-annotated-lexicon-em/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1999](https://md.research.elamur.ai/free/en/papers/1999/) / Inducing a Semantically Annotated Lexicon via EM-Based Clustering

# Inducing a Semantically Annotated Lexicon via EM-Based Clustering

**Authors:** [Mats Rooth](https://md.research.elamur.ai/free/en/authors/mats-rooth/), [Stefan Riezler](https://md.research.elamur.ai/free/en/authors/stefan-riezler/), Detlef Prescher, Glenn Carroll and Franz Beil · **Year:** 1999 · **Venue:** [Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 1999](https://md.research.elamur.ai/free/en/venues/acl/1999/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P99-1014/) · [PDF](https://aclanthology.org/P99-1014.pdf) · [DOI 10.3115/1034678.1034703](https://doi.org/10.3115/1034678.1034703) · [OpenAlex](https://openalex.org/W2119610041)

**Citations:** 151 citations (OpenAlex, 2026-10-14) · [72](https://md.research.elamur.ai/free/en/papers/rooth-1999-inducing-semantically-annotated-lexicon-em/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Most earlier work on learning selectional restrictions depends on a fixed semantic taxonomy such as an entailment hierarchy. Such resources exist for only a few languages, and their suitability for describing word meaning is debatable. Rooth et al. instead derive semantic slot labels for subcategorization frames directly from unannotated text. They parse the British National Corpus using Carroll and Rooth's (1998) probabilistic grammar, a context-free one lexicalized on heads, and extract verb–noun pairs for subject and object relations. They then fit a latent class model with EM, in which a hidden class jointly generates a verb-slot and a noun. A second EM step assigns class labels to the slots of whole frames. The induced classes are often semantically coherent, for example scalar change or communicative action. The models are tested on a pseudo-disambiguation task. The authors also show how the labels can serve as relation constants in decompositional lexical entries, which captures alternations such as transitive versus intransitive *increase*. The method needs no hand-built hierarchy, so it can be applied to any language that has a large corpus, morphology tools and a robust parser.

## Key points
- **Training data:** 1,280,715 verb–noun pair tokens (608,850 types), taken from maximal-probability parses of the British National Corpus (117 million words).
- **Model:** p(v,n) is a sum over classes c of p(c)·p(v|c)·p(n|c). Each "verb" is really a verb paired with a slot: `as:s` (intransitive subject), `aso:s` (transitive subject) or `aso:o` (transitive object).
- **Example classes:** in a 35-class model, class 17 groups scalar-change verbs (increase, fall, rise, reduce) with things that move along scales. Class 5 links communicative-action verbs with human agents.
- **Evaluation setup:**
  - 3,000 pairs were held out, leaving 1,178,698 training tokens.
  - Each held-out noun was paired with a frequency-sampled verb that never occurs with it, giving 1,337 test triples.
  - Only words with frequency between 30 and 3,000 were kept.
  - 3 × 10 × 6 models were compared, varying starting values, number of classes and number of iterations.
- **Linguistic outcome:** transitive and intransitive *increase* share the component R17 ∧ increase17. This means the diathesis alternation was learned. The authors expect a near-complete lexicon to need a few hundred classes.

## Interesting ideas
- **Slot-specific verb units:** Clustering a verb together with its grammatical slot, rather than the bare verb, lets one model capture how different argument positions select different noun types. The same trick could suit Ukrainian, where case marking makes slots easy to identify.
- **Pseudo-disambiguation as cheap evaluation:** The model must pick the real verb over a never-seen distractor verb for a held-out noun, so no gold annotation is needed. This is a reusable way to test any selectional-preference or embedding model.
- **Frequency filters on test items:** Restricting test words to a middle frequency band keeps the test away from both sparse noise and trivially frequent words. This is a useful design choice for any lexical benchmark built automatically.
- **Sense-indexed lexical roots:** Tagging a word's root with its induced class index keeps homonyms apart, such as *decline* a proposal versus *decline* five percent. Related uses of one sense still share structure, which could help any sense-aware lexicon.
- **Probabilistic outputs for reuse:** Because the classes come from a simple probabilistic model, the learned entries can be plugged into head-lexicalized parsers or language models. This makes the induced lexicon a component that other systems can use, not just a stand-alone resource.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Stuttgart](https://md.research.elamur.ai/free/en/universities/university-of-stuttgart/) — University, Stuttgart, Germany · 75 works · 46,113 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 798 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 463 works

## Cite

Cite as: Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll and Franz Beil. 1999. Inducing a Semantically Annotated Lexicon via EM-Based Clustering. Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rooth1999inducing,
  title = {{Inducing a Semantically Annotated Lexicon via EM-Based Clustering}},
  author = {Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll and Franz Beil},
  booktitle = {Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics},
  year = {1999},
  doi = {10.3115/1034678.1034703},
  url = {https://doi.org/10.3115/1034678.1034703}
}
```

- [← previous Simplifying Text for Language-Impaired Readers](https://md.research.elamur.ai/free/en/papers/carroll-1999-simplifying-text-language-impaired-readers/)
- [next → Noun Phrase Coreference as Clustering](https://md.research.elamur.ai/free/en/papers/cardie-1999-noun-phrase-coreference-as-clustering/)
- [↑ back to papers of 1999](https://md.research.elamur.ai/free/en/papers/1999/)

## References

17 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Leonard E. Baum, Ted Petrie, George Soules, and Norman Weiss. 1970. A maximization technique occuring in the statistical analysis of probabilistic functions of Markov chains. The Annals of Mathematical Statistics, 41(1):164-171. [find](https://scholar.google.com/scholar?q=Leonard+E%2E+Baum%2C+Ted+Petrie%2C+George+Soules%2C+and+Norman+Weiss%2E+1970%2E+A+maximization+technique+occuring+in+the+statistical+analysis+of+probabilistic+functions+of+Markov+chains%2E+The+Annals+of+Mathematica)
2. Franz Beil, Glenn Carroll, Detlef Prescher, Stefan Riezler, and Mats Rooth. 1999. Inside-outside estimation of a lexicalized PCFG for German. In Proceedings of the 37th Annual Meeting of the A CL, Maryland. [find](https://scholar.google.com/scholar?q=Franz+Beil%2C+Glenn+Carroll%2C+Detlef+Prescher%2C+Stefan+Riezler%2C+and+Mats+Rooth%2E+1999%2E+Inside%2Doutside+estimation+of+a+lexicalized+PCFG+for+German%2E+In+Proceedings+of+the+37th+Annual+Meeting+of+the+A+CL%2C+Mar)
3. Glenn Carroll and Mats Rooth. 1998. Valence induction with a head-lexicalized PCFG. In Proceedings of EMNLP-3, Granada. [find](https://scholar.google.com/scholar?q=Glenn+Carroll+and+Mats+Rooth%2E+1998%2E+Valence+induction+with+a+head%2Dlexicalized+PCFG%2E+In+Proceedings+of+EMNLP%2D3%2C+Granada%2E)
4. Ido Dagan, Lillian Lee, and Fernando Pereira. to appear. Similarity-based models of word cooccurence probabilities. Machine Learning. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Lillian+Lee%2C+and+Fernando+Pereira%2E+to+appear%2E+Similarity%2Dbased+models+of+word+cooccurence+probabilities%2E+Machine+Learning%2E)
5. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(B):1-38. [find](https://scholar.google.com/scholar?q=A%2E+P%2E+Dempster%2C+N%2E+M%2E+Laird%2C+and+D%2E+B%2E+Rubin%2E+1977%2E+Maximum+likelihood+from+incomplete+data+via+the+EM+algorithm%2E+Journal+of+the+Royal+Statistical+Society%2C+39%28B%29%3A1%2D38%2E)
6. Jerry A. Fodor. 1998. Concepts : Where Cogni-tire Science Went Wrong. Oxford Cognitive Science Series, Oxford. [find](https://scholar.google.com/scholar?q=Jerry+A%2E+Fodor%2E+1998%2E+Concepts+%3A+Where+Cogni%2Dtire+Science+Went+Wrong%2E+Oxford+Cognitive+Science+Series%2C+Oxford%2E)
7. K. Hale and S.J. Keyser. 1993. Argument structure and the lexical expression of syntactic relations. In K. Hale and S.J. Keyser, editors, The View from Building 20. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=K%2E+Hale+and+S%2EJ%2E+Keyser%2E+1993%2E+Argument+structure+and+the+lexical+expression+of+syntactic+relations%2E+In+K%2E+Hale+and+S%2EJ%2E+Keyser%2C+editors%2C+The+View+from+Building+20%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
8. Thomas Hofmann and Jan Puzicha. 1998. Unsupervised learning from dyadic data. Technical Report TR-98-042, International Computer Science Insitute, Berkeley, CA. [find](https://scholar.google.com/scholar?q=Thomas+Hofmann+and+Jan+Puzicha%2E+1998%2E+Unsupervised+learning+from+dyadic+data%2E+Technical+Report+TR%2D98%2D042%2C+International+Computer+Science+Insitute%2C+Berkeley%2C+CA%2E)
9. Murat Kural. 1996. Verb Incorporation and Elementary Predicates. Ph.D. thesis, University of California, Los Angeles. [find](https://scholar.google.com/scholar?q=Murat+Kural%2E+1996%2E+Verb+Incorporation+and+Elementary+Predicates%2E+Ph%2ED%2E+thesis%2C+University+of+California%2C+Los+Angeles%2E)
10. Beth Levin. 1993. English Verb Classes and Alternations. A Preliminary Investigation. The University of Chicago Press, Chicago/London. [find](https://scholar.google.com/scholar?q=Beth+Levin%2E+1993%2E+English+Verb+Classes+and+Alternations%2E+A+Preliminary+Investigation%2E+The+University+of+Chicago+Press%2C+Chicago%2FLondon%2E)
11. Fernando Pereira, Naftali Tishby, and Lillian Lee. 1993. [Distributional clustering of english words](https://md.research.elamur.ai/free/en/papers/pereira-1993-distributional-clustering-english-words/). In Proceedings of the 31th Annual Meeting of the A CL, Columbus, Ohio.
12. Philip Resnik. 1993. Selection and information: A class-bases approach to lexical relationships. Ph.D. thesis, University of Pennsylvania, CIS Department. [find](https://scholar.google.com/scholar?q=Philip+Resnik%2E+1993%2E+Selection+and+information%3A+A+class%2Dbases+approach+to+lexical+relationships%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2C+CIS+Department%2E)
13. Francecso Ribas. 1994. An experiment on learning appropriate selectional restrictions from a parsed corpus. In Proceedings of COLING-9~, Kyoto, Japan. [find](https://scholar.google.com/scholar?q=Francecso+Ribas%2E+1994%2E+An+experiment+on+learning+appropriate+selectional+restrictions+from+a+parsed+corpus%2E+In+Proceedings+of+COLING%2D9%7E%2C+Kyoto%2C+Japan%2E)
14. Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll, and Franz Beil. 1998. EM-based clustering for NLP applications. In Inducing Lexicons with the EM Algorithm. AIMS Report 4(3), Institut fiir Maschinelle Sprachverarbeitung, Universit~t Stuttgart. [find](https://scholar.google.com/scholar?q=Mats+Rooth%2C+Stefan+Riezler%2C+Detlef+Prescher%2C+Glenn+Carroll%2C+and+Franz+Beil%2E+1998%2E+EM%2Dbased+clustering+for+NLP+applications%2E+In+Inducing+Lexicons+with+the+EM+Algorithm%2E+AIMS+Report+4%283%29%2C+Institut+fiir+)
15. Mats Rooth. Ms. Two-dimensional clusters in grammatical relations. In Symposium on Representation and Acquisition of Lexical Knowledge: Polysemy, Ambiguity, and Generativity. AAAI 1995 Spring Symposium Series, Stanford University. [find](https://scholar.google.com/scholar?q=Mats+Rooth%2E+Ms%2E+Two%2Ddimensional+clusters+in+grammatical+relations%2E+In+Symposium+on+Representation+and+Acquisition+of+Lexical+Knowledge%3A+Polysemy%2C+Ambiguity%2C+and+Generativity%2E+AAAI+1995+Spring+Symposiu)
16. Lawrence K. Saul and Fernando Pereira. 1997. Aggregate and mixed-order Markov models for statistical language processing. In Proceedings of EMNLP-2. [find](https://scholar.google.com/scholar?q=Lawrence+K%2E+Saul+and+Fernando+Pereira%2E+1997%2E+Aggregate+and+mixed%2Dorder+Markov+models+for+statistical+language+processing%2E+In+Proceedings+of+EMNLP%2D2%2E)
17. Helmut Schuhmacher. 1986. Verben in Feldern. Valenzw5rterbuch zur Syntax und Semantik deutscher Verben. de Gruyter, Berlin. [find](https://scholar.google.com/scholar?q=Helmut+Schuhmacher%2E+1986%2E+Verben+in+Feldern%2E+Valenzw5rterbuch+zur+Syntax+und+Semantik+deutscher+Verben%2E+de+Gruyter%2C+Berlin%2E)

*Updated 24 October 2026*