---
title: "A Linear Observed Time Statistical Parser Based on Maximum"
description: "The paper tackles full syntactic parsing of English. 87% precision. 11 references, cited by 88 papers here."
authors: ["Adwait Ratnaparkhi"]
year: 1997
venue: "Second Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/W97-0301/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ratnaparkhi-1997-linear-observed-time-statistical-parser/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ratnaparkhi-1997-linear-observed-time-statistical-parser/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1997](https://md.research.elamur.ai/free/en/papers/1997/) / A Linear Observed Time Statistical Parser Based on Maximum Entropy Models

# A Linear Observed Time Statistical Parser Based on Maximum Entropy Models

**Authors:** [Adwait Ratnaparkhi](https://md.research.elamur.ai/free/en/authors/adwait-ratnaparkhi/) · **Year:** 1997 · **Venue:** [Second Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 1997](https://md.research.elamur.ai/free/en/venues/emnlp/1997/)) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W97-0301/) · [PDF](https://aclanthology.org/W97-0301.pdf) · [OpenAlex](https://openalex.org/W1810157568)

**Citations:** 114 citations (OpenAlex, 2026-10-14) · [88](https://md.research.elamur.ai/free/en/papers/ratnaparkhi-1997-linear-observed-time-statistical-parser/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tackles full syntactic parsing of English. It aims for parse quality above earlier statistical parsers while keeping feature engineering light and decoding fast. Ratnaparkhi recasts parsing as a sequence of small decisions made in three left-to-right passes: POS tagging, flat-chunk detection, and then an alternation of constituent-building and completeness-checking steps. A separate maximum entropy model scores each kind of decision. Its features are generated automatically from short templates over the local context, such as neighbouring words, tags, constituent labels and head words. A simple breadth-first search keeps only the top-scoring partial derivations. Trained and tested on the Penn Treebank (Wall St. Journal), the parser reaches about 87% precision and 86% recall. This beats the best results previously reported, from Collins (1996) and Magerman's SPATTER (1995). Its running time grows linearly with sentence length in practice. The parser also outputs several ranked parses, and an oracle choosing the best of the top 20 would score about 93%. That gap points to reranking as a valuable research direction.

## Key points
- Accuracy on section 23 of the WSJ part of the Penn Treebank is roughly 87% precision and 86% recall, measured with PARSEVAL and with the looser measures used by Collins (1996) and Magerman (1995).
- Four procedures (TAG, CHUNK, BUILD, CHECK) run in three passes. Their actions are designed so that every complete parse tree corresponds to exactly one derivation.
- BUILD labels each tree as Start X or Join X. CHECK answers Yes or No on whether the newest proposed constituent is finished, which plays the role of reduce versus shift in a shift-reduce parser.
- Feature templates cover head words, which are found with Magerman's head-finding rules. Each lexical predicate also gets backed-off variants that drop one or more words, which helps with rare words.
- An oracle that picks the best of the top 20 parses would reach about 93% precision and recall.

## Interesting ideas
- **Splitting a reduce into incremental decisions.** Instead of committing to a whole CFG rule at once, the parser attaches trees one by one and then asks a yes/no completeness question. Small, local classification steps like these are easier to model and could suit other structured-prediction tasks, such as transition-based parsers for UD treebanks.
- **Back-off expressed as features.** Sparse lexical predicates are paired with less specific versions that omit words, and the maximum entropy model learns how much to trust each one. This avoids hand-tuned smoothing schemes like the one in Collins's bigram parser.
- **Oracle upper bound for reranking.** Measuring how good the best parse in the top N could be gives a ceiling for any reranker. This tells you whether building a second-stage model is worth the effort, and the check works for any system that produces n-best output.
- **Chunking as a reusable intermediate layer.** Flat phrases are found first with the same Start/Join tagging scheme, and later passes then work on a shorter forest. A staged pipeline like this can simplify parsers for languages or domains with limited resources.
- **Plain words instead of word clusters.** Unlike SPATTER, the model uses raw words in its predicates, which removes a costly clustering step. Adding new kinds of information, such as punctuation cues, only requires writing another template.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Pennsylvania](https://md.research.elamur.ai/free/en/universities/university-of-pennsylvania/) — University, Philadelphia, United States · 170 works · 60,739 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Adwait Ratnaparkhi. 1997. A Linear Observed Time Statistical Parser Based on Maximum Entropy Models. Second Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ratnaparkhi1997linear,
  title = {{A Linear Observed Time Statistical Parser Based on Maximum Entropy Models}},
  author = {Adwait Ratnaparkhi},
  booktitle = {Second Conference on Empirical Methods in Natural Language Processing},
  year = {1997},
  url = {https://aclanthology.org/W97-0301/}
}
```

- [← previous A Corpus-Based Approach for Building Semantic Lexicons](https://md.research.elamur.ai/free/en/papers/riloff-1997-corpus-based-approach-building-semantic/)
- [next → IGTree: Using Trees for Compression and Classification in Lazy Learning Algorithms](https://md.research.elamur.ai/free/en/papers/daelemans-1997-igtree-trees-compression-classification-lazy/)
- [↑ back to papers of 1997](https://md.research.elamur.ai/free/en/papers/1997/)

## References

11 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Berger, Adam, Stephen A. Della Pietra, and Vincent J. Della Pietra. 1996. [A Maximum Entropy Approach to Natural Language Processing](https://md.research.elamur.ai/free/en/papers/berger-1996-maximum-entropy-approach-natural-language/). Computational Linguistics, 22(1):39-71.
2. Black, Ezra et al. 1991. A Procedure for Quantitatively Comparing the Syntactic Coverage of English Grammars. In Proceedings of the February 1991 DARPA Speech and Natural Language Workshop, pages 306-311. [find](https://scholar.google.com/scholar?q=Black%2C+Ezra+et+al%2E+1991%2E+A+Procedure+for+Quantitatively+Comparing+the+Syntactic+Coverage+of+English+Grammars%2E+In+Proceedings+of+the+February+1991+DARPA+Speech+and+Natural+Language+Workshop%2C+pages+306%2D)
3. Black, Ezra, Fred Jelinek, John Lafferty, David M. Magerman, Robert Mercer, and Salim Roukos. 1993. [Towards History-based Grammars: Using Richer Models for Probabilistic Parsing](https://md.research.elamur.ai/free/en/papers/black-1993-history-based-grammars-richer-models/). In Proceedings of the 31st Annual Meeting of the ACL, Columbus, Ohio.
4. Collins, Michael John. 1996. A New Statistical Parser Based on Bigram Lexical Dependencies. In Proceedings of the 34th Annual Meeting of the ACL. [find](https://scholar.google.com/scholar?q=Collins%2C+Michael+John%2E+1996%2E+A+New+Statistical+Parser+Based+on+Bigram+Lexical+Dependencies%2E+In+Proceedings+of+the+34th+Annual+Meeting+of+the+ACL%2E)
5. Darroch, J. N. and D. Ratcliff. 1972. Generalized Iterative Scaling for Log-Linear Models. The Annals of Mathematical Statistics, 43(5):1470-1480. [find](https://scholar.google.com/scholar?q=Darroch%2C+J%2E+N%2E+and+D%2E+Ratcliff%2E+1972%2E+Generalized+Iterative+Scaling+for+Log%2DLinear+Models%2E+The+Annals+of+Mathematical+Statistics%2C+43%285%29%3A1470%2D1480%2E)
6. Jelinek, Fred, John Lafferty, David M. Magerman, Robert Mercer, Adwait Ratnaparkhi, and Salim Roukos. 1994. Decision Tree Parsing using a Hidden Derivational Model. In Proceedings of the Human Language Technology Workshop, pages 272-277. ARPA. [find](https://scholar.google.com/scholar?q=Jelinek%2C+Fred%2C+John+Lafferty%2C+David+M%2E+Magerman%2C+Robert+Mercer%2C+Adwait+Ratnaparkhi%2C+and+Salim+Roukos%2E+1994%2E+Decision+Tree+Parsing+using+a+Hidden+Derivational+Model%2E+In+Proceedings+of+the+Human+Languag)
7. Lau, Ray, Ronald Rosenfeld, and Salim Roukos. 1993. Adaptive Language Modeling Using The Maximum Entropy Principle. In Proceedings of the Human Language Technology Workshop, pages 108-113. ARPA. [find](https://scholar.google.com/scholar?q=Lau%2C+Ray%2C+Ronald+Rosenfeld%2C+and+Salim+Roukos%2E+1993%2E+Adaptive+Language+Modeling+Using+The+Maximum+Entropy+Principle%2E+In+Proceedings+of+the+Human+Language+Technology+Workshop%2C+pages+108%2D113%2E+ARPA%2E)
8. Magerman, David M. 1995. [Statistical Decision-Tree Models for Parsing](https://md.research.elamur.ai/free/en/papers/magerman-1995-statistical-decision-tree-models-parsing/). In Proceedings of the 33rd Annual Meeting of the ACL.
9. Marcus, Mitchell P., Beatrice Santorini, and Mary Ann Marcinkiewicz. 1994. [Building a large annotated corpus of English: the Penn Treebank](https://md.research.elamur.ai/free/en/papers/marcus-1993-building-large-annotated-corpus-english/). Computational Linguistics, 19(2):313-330.
10. Ratnaparkhi, Adwait. 1996. A Maximum Entropy Part of Speech Tagger. In Conference on Empirical Methods in Natural Language Processing, University of Pennsylvania, May 17-18. [find](https://scholar.google.com/scholar?q=Ratnaparkhi%2C+Adwait%2E+1996%2E+A+Maximum+Entropy+Part+of+Speech+Tagger%2E+In+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+University+of+Pennsylvania%2C+May+17%2D18%2E)
11. Ratnaparkhi, Adwait. 1997. A Simple Introduction to Maximum Entropy Models for Natural Language Processing. Technical Report 97-08, Institute for Research in Cognitive Science, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Ratnaparkhi%2C+Adwait%2E+1997%2E+A+Simple+Introduction+to+Maximum+Entropy+Models+for+Natural+Language+Processing%2E+Technical+Report+97%2D08%2C+Institute+for+Research+in+Cognitive+Science%2C+University+of+Pennsylva)

*Updated 24 October 2026*