--- title: "A Maximum-Entropy-Inspired Parser — Charniak 2000" description: "The paper addresses how to make a generative statistical parser for Penn treebank-style trees more accurate, given sparse training data." authors: ["Eugene Charniak"] year: 2000 venue: "1st Meeting of the North American Chapter of the Association for Computational Linguistics" source: "https://aclanthology.org/A00-2018/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/charniak-2000-maximum-entropy-inspired-parser/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/charniak-2000-maximum-entropy-inspired-parser/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2000](https://md.research.elamur.ai/free/papers/2000/) / A Maximum-Entropy-Inspired Parser # A Maximum-Entropy-Inspired Parser **Authors:** [Eugene Charniak](https://md.research.elamur.ai/free/authors/eugene-charniak/) · **Year:** 2000 · **Venue:** 1st Meeting of the North American Chapter of the Association for Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/A00-2018/) · [PDF](https://aclanthology.org/A00-2018.pdf) · [OpenAlex](https://openalex.org/W1535015163) **Citations:** 1454 citations (openalex-snapshot · 2026-10-14) · [594](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary The paper addresses how to make a generative statistical parser for Penn treebank-style trees more accurate, given sparse training data. Charniak's model builds a parse top-down. For each constituent it predicts three things in order: the head's pre-terminal (tag), the lexical head, and the expansion. Expansions come from a lexicalized Markov grammar rather than fixed treebank rules. The main contribution is a way to condition and smooth these probabilities that borrows from log-linear modelling. Each probability is a product of ratios, and each ratio adjusts the estimate as more context is added. This lets the author test and combine many conditioning events without heavy re-engineering. Trained and tested on the standard Wall Street Journal treebank sections, the parser reaches 90.1% average precision/recall on sentences of up to 40 words and 89.5% on sentences of up to 100 words. That is a 13% error reduction over the best earlier single parser (Collins). The result also refutes the idea that treebank-trained parsers had hit a performance ceiling. ## Key points - Results on the standard WSJ Penn treebank sections: precision/recall reaches 90.1% when sentences are capped at ≤40 words, and 89.5% when the cap is ≤100 words. This is about 13% less error than Collins's parser [9]. The conclusion prints 91.1% for the ≤40 setting, which conflicts with the abstract and introduction. - Predicting the head's pre-terminal before the head word itself raised labeled precision/recall by nearly 2%, an effect the author did not anticipate. - The conditioning context for the head tag includes: - the parent's label, head tag and head word; - the label of the left sibling; - the grandparent's label. - Comparison of expansion models: - a first-order Markov grammar was slightly worse than a treebank grammar; - a second-order one was slightly better; - a third-order one was clearly better. - The maximum-entropy-inspired smoothing gave a small but real gain over deleted interpolation. The non-terminal set was extended with the symbols aux and auxg for certain auxiliary verbs. ## Interesting ideas - **Smoothing as a product of ratios.** Start from a coarse estimate, then multiply by factors of the form "estimate with more context divided by estimate with less context." This gives much of the flexibility of log-linear models without iterative training, and suits any sparse conditional estimation task, such as tagging or language modelling. - **Context chains that skip unrelated features.** Each added factor can condition only on the features that matter for it, instead of on everything added before. This reduces fragmentation of the data when the features come from unrelated sources. - **Predicting a coarse category first.** Guessing the tag before the word gives a cheap intermediate variable that sharpens the later prediction. The same decomposition could help other generative models with large vocabularies. - **Markov-generated rule right-hand sides.** Generating the head child first and then its left and right siblings outward, with tunable order, means unseen rules can still get probability. Treating the order as a parameter to sweep made for a clean ablation. - **Choosing a model for flexibility.** The author favored a model in which changing features needs no new code. This made fast experimentation possible across conditioning events and grammar types, a useful principle for research codebases. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) Topics: [Treebanks](https://md.research.elamur.ai/free/topics/treebanks/) ## Cited by Cited by 594 papers of our library cite it (of the 140673 whose bibliographies we read); 69 of them have a page here, 525 not yet. - [Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples](https://md.research.elamur.ai/free/papers/acl-2018-joshi-extending-parser-distant-domains-few/) - [Cross-Domain Generalization of Neural Constituency Parsers](https://md.research.elamur.ai/free/papers/acl-2019-fried-cross-domain-generalization-neural-constituency/) - [Corpus Variation and Parser Performance](https://md.research.elamur.ai/free/papers/aclbulk-2001-gildea-corpus-variation-parser-performance/) - [Supersense Tagging of Unknown Nouns in WordNet](https://md.research.elamur.ai/free/papers/aclbulk-2003-ciaramita-supersense-tagging-unknown-nouns-wordnet/) - [Accurate Unlexicalized Parsing](https://md.research.elamur.ai/free/papers/aclbulk-2003-klein-accurate-unlexicalized-parsing/) - [Robust VPE Detection using Automatically Parsed Text](https://md.research.elamur.ai/free/papers/aclbulk-2004-nielsen-robust-vpe-detection-automatically-parsed/) - [Verb Phrase Ellipsis detection using Automatically Parsed Text](https://md.research.elamur.ai/free/papers/aclbulk-2004-nielsen-verb-phrase-ellipsis-detection-automatically/) - [Deterministic Dependency Parsing of English Text](https://md.research.elamur.ai/free/papers/aclbulk-2004-nivre-deterministic-dependency-parsing-english-text/) - [Probabilistic Disambiguation Models for Wide-Coverage HPSG Parsing](https://md.research.elamur.ai/free/papers/aclbulk-2005-miyao-probabilistic-disambiguation-models-wide-coverage/) - [Effective Self-Training for Parsing](https://md.research.elamur.ai/free/papers/aclbulk-2006-mcclosky-effective-self-training-parsing/) - [Learning Accurate, Compact, and Interpretable Tree Annotation](https://md.research.elamur.ai/free/papers/aclbulk-2006-petrov-learning-accurate-compact-interpretable-tree/) - [Integrated Morphological and Syntactic Disambiguation for Modern Hebrew](https://md.research.elamur.ai/free/papers/aclbulk-2006-tsarfaty-integrated-morphological-syntactic-disambiguation-modern/) - [Formalism-Independent Parser Evaluation with CCG and DepBank](https://md.research.elamur.ai/free/papers/aclbulk-2007-clark-formalism-independent-parser-evaluation-ccg/) - [Task-oriented Evaluation of Syntactic Parsers and Their Representations](https://md.research.elamur.ai/free/papers/aclbulk-2008-miyao-task-oriented-evaluation-syntactic-parsers/) - [Unbounded Dependency Recovery for Parser Evaluation](https://md.research.elamur.ai/free/papers/aclbulk-2009-rimell-unbounded-dependency-recovery-parser-evaluation/) - [Parser Showdown at the Wall Street Corral: An Empirical Investigation of Error Types in Parser Output](https://md.research.elamur.ai/free/papers/aclbulk-2012-kummerfeld-parser-showdown-wall-street-corral/) - [Using the Penn Treebank to Evaluate Non-Treebank Parsers](https://md.research.elamur.ai/free/papers/bank-2004-ringger-penn-treebank-evaluate-non-treebank/) - [Fully Parsing the Penn Treebank](https://md.research.elamur.ai/free/papers/bank-2006-gabbard-fully-parsing-penn-treebank/) - [CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank](https://md.research.elamur.ai/free/papers/bank-2007-hockenmaier-ccgbank-corpus-ccg-derivations-dependency/) - [Parsing to Stanford Dependencies: Trade-offs between Speed and Accuracy](https://md.research.elamur.ai/free/papers/bank-2010-cer-parsing-stanford-dependencies-trade-offs/) - [Parsing Noun Phrases in the Penn Treebank](https://md.research.elamur.ai/free/papers/bank-2011-vadas-parsing-noun-phrases-penn-treebank/) - [Modeling Local Coherence: An Entity-Based Approach](https://md.research.elamur.ai/free/papers/barzilay-2008-modeling-local-coherence-entity-based/) - [Robust Accurate Statistical Annotation of General Text](https://md.research.elamur.ai/free/papers/briscoe-2002-robust-accurate-statistical-annotation-general/) - [Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling](https://md.research.elamur.ai/free/papers/carreras-2005-introduction-conll-2005-shared-task/) - [Head-Driven Statistical Models for Natural Language Parsing](https://md.research.elamur.ai/free/papers/cl-2003-collins-head-driven-statistical-models-natural/) - [Feature Forest Models for Probabilistic HPSG Parsing](https://md.research.elamur.ai/free/papers/cl-2008-miyao-feature-forest-models-probabilistic-hpsg/) - [Czech-English Dependency Tree-based Machine Translation](https://md.research.elamur.ai/free/papers/cmejrek-2003-czech-english-dependency-tree-based/) - [Discriminative Reranking for Natural Language Parsing](https://md.research.elamur.ai/free/papers/collins-2000-discriminative-reranking-natural-language-parsing/) - [Natural Language Processing (almost) from Scratch](https://md.research.elamur.ai/free/papers/collobert-2011-natural-language-processing-almost-scratch/) - [Online Learning of Approximate Dependency Parsing Algorithms](https://md.research.elamur.ai/free/papers/continual-2006-mcdonald-online-learning-approximate-dependency-parsing/) - [Translation Divergences in Chinese–English Machine Translation: An Empirical Investigation](https://md.research.elamur.ai/free/papers/deng-2017-translation-divergences-chinese-english-machine/) - [A Factored Functional Dependency Transformation of the English Penn Treebank for Probabilistic Surface Generation](https://md.research.elamur.ai/free/papers/geary-2006-factored-functional-dependency-transformation-english/) - [Deep Semantic Role Labeling: What Works and What's Next](https://md.research.elamur.ai/free/papers/he-2017-deep-semantic-role-labeling-what/) - [Capturing Translational Divergences with a Statistical Tree-to-Tree Aligner](https://md.research.elamur.ai/free/papers/hearne-2007-capturing-translational-divergences-statistical-tree/) - [Fast Rhetorical Structure Theory Discourse Parsing](https://md.research.elamur.ai/free/papers/heilman-2015-fast-rhetorical-structure-theory-discourse/) - [HILDA: A Discourse Parser Using Support Vector Machine Classification](https://md.research.elamur.ai/free/papers/hernault-2010-hilda-discourse-parser-support-vector/) - [Statistical Syntax-Directed Translation with Extended Domain of Locality](https://md.research.elamur.ai/free/papers/huang-2006-statistical-syntax-directed-translation-extended/) - [Acquiring the Meaning of Discourse Markers](https://md.research.elamur.ai/free/papers/hutchinson-2004-acquiring-meaning-discourse-markers/) - [CODRA: A Novel Discriminative Framework for Rhetorical Analysis](https://md.research.elamur.ai/free/papers/joty-2015-codra-novel-discriminative-framework-rhetorical/) - [Parsing Any Domain English text to CoNLL dependencies](https://md.research.elamur.ai/free/papers/kolachina-2012-parsing-any-domain-english-text/) - [A Constituent-Based Approach to Argument Labeling with Joint Inference in Discourse Parsing](https://md.research.elamur.ai/free/papers/kong-2014-constituent-based-approach-argument-labeling/) - [Statistical Parsing by Machine Learning from a Classical Arabic Treebank](https://md.research.elamur.ai/free/papers/langstream-2015-dukes-statistical-parsing-machine-learning-classical/) - [Inferring Sentence-internal Temporal Relations](https://md.research.elamur.ai/free/papers/lapata-2004-inferring-sentence-internal-temporal-relations/) - [Parsing Ungrammatical Input: An Evaluation Procedure](https://md.research.elamur.ai/free/papers/lingbulk-2004-foster-parsing-ungrammatical-input-evaluation-procedure/) - [Adapting WSJ-Trained Parsers to the British National Corpus Using In-Domain Self-Training](https://md.research.elamur.ai/free/papers/lingbulk-2007-foster-adapting-wsj-trained-parsers-british/) - [Tree-distance and some other variants of evalb](https://md.research.elamur.ai/free/papers/lingbulk-2008-emms-tree-distance-some-other-variants/) - [Parser Evaluation and the BNC: Evaluating 4 constituency parsers with 3 metrics](https://md.research.elamur.ai/free/papers/lingbulk-2008-foster-parser-evaluation-bnc-evaluating-4/) - [Towards Semi-Automated Annotation for Prepositional Phrase Attachment](https://md.research.elamur.ai/free/papers/lingbulk-2010-rosenthal-semi-automated-annotation-prepositional-phrase/) - [What Causes the Failure of Explicit to Implicit Discourse Relation Recognition?](https://md.research.elamur.ai/free/papers/liu-2024-what-causes-failure-explicit-implicit/) - [Generating Typed Dependency Parses from Phrase Structure Parses](https://md.research.elamur.ai/free/papers/map-2006-marneffe-generating-typed-dependency-parses-phrase/) - [An Unsupervised Approach to Recognizing Discourse Relations](https://md.research.elamur.ai/free/papers/marcu-2002-unsupervised-approach-recognizing-discourse-relations/) - [Multi-Source Transfer of Delexicalized Dependency Parsers](https://md.research.elamur.ai/free/papers/mcdonald-2011-multi-source-transfer-delexicalized-dependency/) - [Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking](https://md.research.elamur.ai/free/papers/narrative-2005-charniak-coarse-fine-n-best-parsing/) - [Online Large-Margin Training of Dependency Parsers](https://md.research.elamur.ai/free/papers/narrative-2005-mcdonald-online-large-margin-training-dependency/) - [Incrementality in Deterministic Dependency Parsing](https://md.research.elamur.ai/free/papers/nivre-2004-incrementality-deterministic-dependency-parsing/) - [Memory-Based Dependency Parsing](https://md.research.elamur.ai/free/papers/nivre-2004-memory-based-dependency-parsing/) - [Wide-Coverage Efficient Statistical Parsing with CCG and Log-Linear Models](https://md.research.elamur.ai/free/papers/nlpgap-2007-clark-wide-coverage-efficient-statistical-parsing/) - [The Proposition Bank: An Annotated Corpus of Semantic Roles](https://md.research.elamur.ai/free/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/) - [Syntax-based Alignment of Multiple Translations: Extracting Paraphrases and Generating New Sentences](https://md.research.elamur.ai/free/papers/pang-2003-syntax-based-alignment-multiple-translations/) - [Statistical Dependency Analysis with Support Vector Machines](https://md.research.elamur.ai/free/papers/parsers-2003-yamada-statistical-dependency-analysis-support-vector/) - [Parsing English into Abstract Meaning Representation Using Syntax-Based Machine Translation](https://md.research.elamur.ai/free/papers/pust-2015-parsing-english-into-abstract-meaning/) - [A Public Reference Implementation of the RAP Anaphora Resolution Algorithm](https://md.research.elamur.ai/free/papers/qiu-2004-public-reference-implementation-rap-anaphora/) - [A Classifier-Based Parser with Linear Run-Time Complexity](https://md.research.elamur.ai/free/papers/sagae-2005-classifier-based-parser-linear-run/) - [Sentence Level Discourse Parsing using Syntactic and Lexical Information](https://md.research.elamur.ai/free/papers/soricut-2003-sentence-level-discourse-parsing-syntactic/) - [Coreference Resolution with and without Linguistic Knowledge](https://md.research.elamur.ai/free/papers/uryupina-2006-coreference-resolution-without-linguistic-knowledge/) - [Error Analysis for Learning-based Coreference Resolution](https://md.research.elamur.ai/free/papers/uryupina-2008-error-analysis-learning-based-coreference/) - [Not an Interlingua, But Close: Comparison of English AMRs to Chinese and Czech](https://md.research.elamur.ai/free/papers/xue-2014-not-interlingua-but-close-comparison/) - [Kernel-Based Pronoun Resolution with Structured Syntactic Knowledge](https://md.research.elamur.ai/free/papers/yang-2006-kernel-based-pronoun-resolution-structured/) - [SciDTB: Discourse Dependency TreeBank for Scientific Abstracts](https://md.research.elamur.ai/free/papers/yang-2018-scidtb-discourse-dependency-treebank-scientific/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{charniak2000maximum, title = {{A Maximum-Entropy-Inspired Parser}}, author = {Charniak, Eugene}, booktitle = {1st Meeting of the North American Chapter of the Association for Computational Linguistics}, year = {2000}, url = {https://aclanthology.org/A00-2018/} } ``` - [next → A Morphologically Sensitive Clustering Algorithm for Identifying Arabic Roots](https://md.research.elamur.ai/free/papers/aclbulk-2000-roeck-morphologically-sensitive-clustering-algorithm-identifying/) - [↑ back to papers of 2000](https://md.research.elamur.ai/free/papers/2000/)