---
title: "Intricacies of Collins' Parsing Model — Bikel 2004"
description: "Collins' lexicalized generative parsing models set accuracy benchmarks on the Penn Treebank and became the basis for much later work."
authors: ["Daniel M. Bikel"]
year: 2004
venue: "Computational Linguistics"
source: "https://aclanthology.org/J04-4004/"
doi: "10.1162/0891201042544929"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bikel-2004-intricacies-collins-parsing-model/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bikel-2004-intricacies-collins-parsing-model/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/) / Intricacies of Collins' Parsing Model

# Intricacies of Collins' Parsing Model

**Authors:** [Daniel M. Bikel](https://md.research.elamur.ai/free/en/authors/daniel-m-bikel/) · **Year:** 2004 · **Venue:** Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/J04-4004/) · [PDF](https://aclanthology.org/J04-4004.pdf) · [DOI 10.1162/0891201042544929](https://doi.org/10.1162/0891201042544929) · [OpenAlex](https://openalex.org/W1985754308)

**Citations:** 265 citations (openalex-snapshot · 2026-10-14) · [104](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Collins' lexicalized generative parsing models set accuracy benchmarks on the Penn Treebank and became the basis for much later work. Researchers who rebuilt them from the published descriptions, however, could not match the reported scores. Bikel built a multilingual engine for generative statistical parsing, set it up as Collins' Model 2, and found that a version built only from the thesis fell well short on Section 00 of the Penn Treebank. Through reverse engineering, study of Collins' code and discussion with Collins, he documented every unpublished detail needed to reproduce the benchmarks. These include an extensive training-tree preprocessing pipeline and smoothing and back-off choices. Leaving them out raises relative error by 11%.

He also proposes a simpler way to handle punctuation and conjunction that performs equally well. He then re-examines familiar features. Bilexical dependencies contribute very little, and head choice matters less than people assumed. Most of the benefit of lexicalization seems to come from generating unlexicalized structure conditioned on the headword and its POS tag. The work matters for replicability and for guiding the design of newer parsing models.

## Key points
- A clean-room implementation of Collins' Model 2 based only on published details has 11% more relative error than the full-detail version. An earlier Model 1 reimplementation by Gildea (2001) showed about 16.7% more relative error.
- The paper lists 11 ordered preprocessing steps for training trees. Among them are:
  - base-NP (NPB) insertion and repair
  - null-element removal
  - punctuation raising
  - argument identification
  - head-finding
- It refines the definitions of coordinated phrases and of when to insert an extra NP level, beyond what Collins' 1999 thesis specifies.
- Bilexical dependencies are rarely used by the model, and head choice has limited impact on parsing accuracy.
- The parsing engine developed for this work (Bikel 2002) is publicly available, and the evaluation uses Penn Treebank Section 00 as the development set.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Treebanks](https://md.research.elamur.ai/free/en/topics/treebanks/)

## Cited by

Cited by

104 papers of our library cite it (of the 140884 whose bibliographies we read); 12 of them have a page here, 92 not yet.

- [Construct State Modification in the Arabic Treebank](https://md.research.elamur.ai/free/en/papers/aclbulk-2008-gabbard-construct-state-modification-arabic-treebank/)
- [Task-oriented Evaluation of Syntactic Parsers and Their Representations](https://md.research.elamur.ai/free/en/papers/aclbulk-2008-miyao-task-oriented-evaluation-syntactic-parsers/)
- ["cba to check the spelling": Investigating Parser Performance on Discussion Forum Posts](https://md.research.elamur.ai/free/en/papers/aclbulk-2010-foster-cba-check-spelling-investigating-parser/)
- [Better Arabic Parsing: Baselines, Evaluations, and Analysis](https://md.research.elamur.ai/free/en/papers/aclbulk-2010-green-better-arabic-parsing-baselines-evaluations/)
- [Parser Showdown at the Wall Street Corral: An Empirical Investigation of Error Types in Parser Output](https://md.research.elamur.ai/free/en/papers/aclbulk-2012-kummerfeld-parser-showdown-wall-street-corral/)
- [Prepositional Phrase Attachment without Oracles](https://md.research.elamur.ai/free/en/papers/cl-2007-atterer-squibs-prepositional-phrase-attachment-without/)
- [Overview of BioNLP'09 Shared Task on Event Extraction](https://md.research.elamur.ai/free/en/papers/kim-2009-overview-bionlp-09-shared-task/)
- [Statistical Parsing by Machine Learning from a Classical Arabic Treebank](https://md.research.elamur.ai/free/en/papers/langstream-2015-dukes-statistical-parsing-machine-learning-classical/)
- [Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking](https://md.research.elamur.ai/free/en/papers/narrative-2005-charniak-coarse-fine-n-best-parsing/)
- [Online Large-Margin Training of Dependency Parsers](https://md.research.elamur.ai/free/en/papers/narrative-2005-mcdonald-online-large-margin-training-dependency/)
- [The Importance of Syntactic Parsing and Inference in Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/punyakanok-2008-importance-syntactic-parsing-inference-semantic/)
- [Does Universal Dependencies need a parsing representation? An investigation of English](https://md.research.elamur.ai/free/en/papers/silveira-2015-does-universal-dependencies-need-parsing/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{bikel2004intricacies,
  title = {{Intricacies of Collins' Parsing Model}},
  author = {Bikel, Daniel M.},
  journal = {Computational Linguistics},
  year = {2004},
  doi = {10.1162/0891201042544929},
  url = {https://doi.org/10.1162/0891201042544929}
}
```

- [← previous Inferring Sentence-internal Temporal Relations](https://md.research.elamur.ai/free/en/papers/lapata-2004-inferring-sentence-internal-temporal-relations/)
- [next → Introduction to the Bio-entity Recognition Task at JNLPBA](https://md.research.elamur.ai/free/en/papers/collier-2004-introduction-bio-entity-recognition-task/)
- [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/)

## References

27 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Baker, J. K. 1979. Trainable grammars for speech recognition. In Spring Conference of the Acoustical Society of America, pages 547–550, Boston. [find](https://scholar.google.com/scholar?q=Baker%2C+J%2E+K%2E+1979%2E+Trainable+grammars+for+speech+recognition%2E+In+Spring+Conference+of+the+Acoustical+Society+of+America%2C+pages+547%E2%80%93550%2C+Boston%2E)
2. Bies, A. 1995. Bracketing guidelines for Treebank II style Penn Treebank Project. Available at ftp://ftp.cis.upenn.edu/pub/ treebank/doc/manual/root.ps.gz. [find](https://scholar.google.com/scholar?q=Bies%2C+A%2E+1995%2E+Bracketing+guidelines+for+Treebank+II+style+Penn+Treebank+Project%2E+Available+at+ftp%3A%2F%2Fftp%2Ecis%2Eupenn%2Eedu%2Fpub%2F+treebank%2Fdoc%2Fmanual%2Froot%2Eps%2Egz%2E)
3. Bikel, Daniel M. 2000. A statistical model for parsing and word-sense disambiguation. In Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, Hong Kong, October. [find](https://scholar.google.com/scholar?q=Bikel%2C+Daniel+M%2E+2000%2E+A+statistical+model+for+parsing+and+word%2Dsense+disambiguation%2E+In+Joint+SIGDAT+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+Very+Large+Corpora%2C+Hong+Kong%2C+)
4. Bikel, Daniel M. 2002. Design of a multi-lingual, parallel-processing statistical parsing engine. In Proceedings of HLT2002, San Diego. [find](https://scholar.google.com/scholar?q=Bikel%2C+Daniel+M%2E+2002%2E+Design+of+a+multi%2Dlingual%2C+parallel%2Dprocessing+statistical+parsing+engine%2E+In+Proceedings+of+HLT2002%2C+San+Diego%2E)
5. Bikel, Daniel M. and David Chiang. 2000. Two statistical parsing models applied to the Chinese Treebank. In Martha Palmer, Mitch Marcus, Aravind Joshi, and Fei Xia, editors, Proceedings of the Second Chinese Language Processing Workshop, pages 1–6, Hong Kong. [find](https://scholar.google.com/scholar?q=Bikel%2C+Daniel+M%2E+and+David+Chiang%2E+2000%2E+Two+statistical+parsing+models+applied+to+the+Chinese+Treebank%2E+In+Martha+Palmer%2C+Mitch+Marcus%2C+Aravind+Joshi%2C+and+Fei+Xia%2C+editors%2C+Proceedings+of+the+Second+)
6. Bikel, Daniel M., Richard Schwartz, Ralph Weischedel, and Scott Miller. 1997. Nymble: A high-performance learning name-finder. In Fifth Conference on Applied Natural Language Processing, pages 194–201, Washington, DC. [find](https://scholar.google.com/scholar?q=Bikel%2C+Daniel+M%2E%2C+Richard+Schwartz%2C+Ralph+Weischedel%2C+and+Scott+Miller%2E+1997%2E+Nymble%3A+A+high%2Dperformance+learning+name%2Dfinder%2E+In+Fifth+Conference+on+Applied+Natural+Language+Processing%2C+pages+194%E2%80%93201) [our card](https://md.research.elamur.ai/free/en/papers/bikel-1997-nymble-high-performance-learning-name/)
7. Black, Ezra, Frederick Jelinek, John Lafferty, David Magerman, Robert Mercer, and Salim Roukos. 1992. Towards history-based grammars: Using richer models for probabilistic parsing. In Proceedings of the Fifth DARPA Speech and Natural Language Workshop, Harriman, NY. [find](https://scholar.google.com/scholar?q=Black%2C+Ezra%2C+Frederick+Jelinek%2C+John+Lafferty%2C+David+Magerman%2C+Robert+Mercer%2C+and+Salim+Roukos%2E+1992%2E+Towards+history%2Dbased+grammars%3A+Using+richer+models+for+probabilistic+parsing%2E+In+Proceedings+of+t)
8. Booth, T. L. and R. A. Thompson. 1973. Applying probability measures to abstract languages. IEEE Transactions on Computers, volume C-22: 442–450. [find](https://scholar.google.com/scholar?q=Booth%2C+T%2E+L%2E+and+R%2E+A%2E+Thompson%2E+1973%2E+Applying+probability+measures+to+abstract+languages%2E+IEEE+Transactions+on+Computers%2C+volume+C%2D22%3A+442%E2%80%93450%2E)
9. Chiang, David and Daniel M. Bikel. 2002. Recovering latent information in treebanks. In Proceedings of COLING’02, Taipei. [find](https://scholar.google.com/scholar?q=Chiang%2C+David+and+Daniel+M%2E+Bikel%2E+2002%2E+Recovering+latent+information+in+treebanks%2E+In+Proceedings+of+COLING%E2%80%9902%2C+Taipei%2E)
10. Cohen, Paul R. 1995. Empirical Methods for Artificial Intelligence. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Cohen%2C+Paul+R%2E+1995%2E+Empirical+Methods+for+Artificial+Intelligence%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
11. Collins, Michael. 1996. A new statistical parser based on bigram lexical dependencies. In Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics, pages 184–191, Santa Cruz, CA. [find](https://scholar.google.com/scholar?q=Collins%2C+Michael%2E+1996%2E+A+new+statistical+parser+based+on+bigram+lexical+dependencies%2E+In+Proceedings+of+the+34th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+184%E2%80%93191%2C+Santa+)
12. Collins, Michael. 1997. Three generative, lexicalised models for statistical parsing. In Proceedings of ACL-EACL ’97, pages 16–23, Madrid. [find](https://scholar.google.com/scholar?q=Collins%2C+Michael%2E+1997%2E+Three+generative%2C+lexicalised+models+for+statistical+parsing%2E+In+Proceedings+of+ACL%2DEACL+%E2%80%9997%2C+pages+16%E2%80%9323%2C+Madrid%2E) [our card](https://md.research.elamur.ai/free/en/papers/collins-1997-three-generative-lexicalised-models-statistical/)
13. Collins, Michael. 1999. Head-Driven Statistical Models for Natural Language Parsing. Ph.D. thesis, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Collins%2C+Michael%2E+1999%2E+Head%2DDriven+Statistical+Models+for+Natural+Language+Parsing%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2E)
14. Collins, Michael. 2000. Discriminative reranking for natural language parsing. In International Conference on Machine Learning, Stanford University, Stanford, CA. [find](https://scholar.google.com/scholar?q=Collins%2C+Michael%2E+2000%2E+Discriminative+reranking+for+natural+language+parsing%2E+In+International+Conference+on+Machine+Learning%2C+Stanford+University%2C+Stanford%2C+CA%2E) [our card](https://md.research.elamur.ai/free/en/papers/collins-2000-discriminative-reranking-natural-language-parsing/)
15. Collins, Michael and Nigel Duffy. 2002. New ranking algorithms for parsing and tagging: Kernels over discrete structures, and the voted perceptron. In Proceedings of ACL-02, pages 263–270, Philadelphia. [find](https://scholar.google.com/scholar?q=Collins%2C+Michael+and+Nigel+Duffy%2E+2002%2E+New+ranking+algorithms+for+parsing+and+tagging%3A+Kernels+over+discrete+structures%2C+and+the+voted+perceptron%2E+In+Proceedings+of+ACL%2D02%2C+pages+263%E2%80%93270%2C+Philadelphi)
16. Eisner, Jason. 1996. Three new probabilistic models for dependency parsing: An exploration. In Proceedings of the 16th International Conference on Computational Linguistics (COLING-96), pages 340–345, Copenhagen, August. [find](https://scholar.google.com/scholar?q=Eisner%2C+Jason%2E+1996%2E+Three+new+probabilistic+models+for+dependency+parsing%3A+An+exploration%2E+In+Proceedings+of+the+16th+International+Conference+on+Computational+Linguistics+%28COLING%2D96%29%2C+pages+340%E2%80%93345%2C) [our card](https://md.research.elamur.ai/free/en/papers/ast-1996-eisner-three-new-probabilistic-models-dependency/)
17. Gildea, Daniel. 2001. Corpus variation and parser performance. In Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing, Pittsburgh. [find](https://scholar.google.com/scholar?q=Gildea%2C+Daniel%2E+2001%2E+Corpus+variation+and+parser+performance%2E+In+Proceedings+of+the+2001+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+Pittsburgh%2E) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2001-gildea-corpus-variation-parser-performance/)
18. Gildea, Daniel and Daniel Jurafsky. 2000. Automatic labeling of semantic roles. In Proceedings of ACL 2000, Hong Kong. [find](https://scholar.google.com/scholar?q=Gildea%2C+Daniel+and+Daniel+Jurafsky%2E+2000%2E+Automatic+labeling+of+semantic+roles%2E+In+Proceedings+of+ACL+2000%2C+Hong+Kong%2E) [our card](https://md.research.elamur.ai/free/en/papers/gildea-2002-automatic-labeling-semantic-roles/)
19. Gildea, Daniel and Martha Palmer. 2002. The necessity of parsing for predicate argument recognition. In Proceedings of ACL 2002, Philadelphia. [find](https://scholar.google.com/scholar?q=Gildea%2C+Daniel+and+Martha+Palmer%2E+2002%2E+The+necessity+of+parsing+for+predicate+argument+recognition%2E+In+Proceedings+of+ACL+2002%2C+Philadelphia%2E)
20. Goodman, Joshua. 1997. Global thresholding and multiple-pass parsing. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing, Brown University, Providence, RI. [find](https://scholar.google.com/scholar?q=Goodman%2C+Joshua%2E+1997%2E+Global+thresholding+and+multiple%2Dpass+parsing%2E+In+Proceedings+of+the+Second+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+Brown+University%2C+Providence%2C+RI%2E)
21. Henderson, John C. and Eric Brill. 1999. Exploiting diversity in natural language processing: Combining parsers. In Proceedings of the Fourth Conference on Empirical Methods in Natural Language Processing, College Park, MD. [find](https://scholar.google.com/scholar?q=Henderson%2C+John+C%2E+and+Eric+Brill%2E+1999%2E+Exploiting+diversity+in+natural+language+processing%3A+Combining+parsers%2E+In+Proceedings+of+the+Fourth+Conference+on+Empirical+Methods+in+Natural+Language+Proces)
22. Hwa, Rebecca. 2001. On minimizing training corpus for parser acquisition. In Proceedings of the Fifth Computational Natural Language Learning Workshop, Toulouse, France, July. [find](https://scholar.google.com/scholar?q=Hwa%2C+Rebecca%2E+2001%2E+On+minimizing+training+corpus+for+parser+acquisition%2E+In+Proceedings+of+the+Fifth+Computational+Natural+Language+Learning+Workshop%2C+Toulouse%2C+France%2C+July%2E)
23. Hwa, Rebecca, Philip Resnik, and Amy Weinberg. 2002. Breaking the resource bottleneck for multilingual parsing. In Workshop on Linguistic Knowledge Acquisition and Representation: Bootstrapping Annotated Language Data, Third International Conference on Language Resources and Evaluation (LREC-2002),… [find](https://scholar.google.com/scholar?q=Hwa%2C+Rebecca%2C+Philip+Resnik%2C+and+Amy+Weinberg%2E+2002%2E+Breaking+the+resource+bottleneck+for+multilingual+parsing%2E+In+Workshop+on+Linguistic+Knowledge+Acquisition+and+Representation%3A+Bootstrapping+Annota)
24. Lari, K. and S. J. Young. 1990. The estimation of stochastic context-free grammars using the Inside-Outside algorithm. Computer Speech and Language, 4:35–56. [find](https://scholar.google.com/scholar?q=Lari%2C+K%2E+and+S%2E+J%2E+Young%2E+1990%2E+The+estimation+of+stochastic+context%2Dfree+grammars+using+the+Inside%2DOutside+algorithm%2E+Computer+Speech+and+Language%2C+4%3A35%E2%80%9356%2E)
25. Marcus, Mitchell P., Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19:313–330. [find](https://scholar.google.com/scholar?q=Marcus%2C+Mitchell+P%2E%2C+Beatrice+Santorini%2C+and+Mary+Ann+Marcinkiewicz%2E+1993%2E+Building+a+large+annotated+corpus+of+English%3A+The+Penn+Treebank%2E+Computational+Linguistics%2C+19%3A313%E2%80%93330%2E) [our card](https://md.research.elamur.ai/free/en/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
26. Ratnaparkhi, Adwait. 1996. A maximum entropy model for part-of-speech tagging. In Conference on Empirical Methods in Natural Language Processing, University of Pennsylvania, Philadelphia, May. [find](https://scholar.google.com/scholar?q=Ratnaparkhi%2C+Adwait%2E+1996%2E+A+maximum+entropy+model+for+part%2Dof%2Dspeech+tagging%2E+In+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+University+of+Pennsylvania%2C+Philadelphia%2C+May%2E) [our card](https://md.research.elamur.ai/free/en/papers/ratnaparkhi-1996-maximum-entropy-model-part-speech/)
27. Witten, I. T. and T. C. Bell. 1991. The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression. IEEE Transactions on Information Theory 37: 1085–1094. [find](https://scholar.google.com/scholar?q=Witten%2C+I%2E+T%2E+and+T%2E+C%2E+Bell%2E+1991%2E+The+zero%2Dfrequency+problem%3A+Estimating+the+probabilities+of+novel+events+in+adaptive+text+compression%2E+IEEE+Transactions+on+Information+Theory+37%3A+1085%E2%80%931094%2E)