---
title: "Introduction to the CoNLL-2000 Shared Task — Sang 2000"
description: "The paper presents the CoNLL-2000 shared task on text chunking, which means splitting sentences into non-overlapping, syntactically coherent word groups…"
authors: ["Erik F. Tjong Kim Sang", "Sabine Buchholz"]
year: 2000
venue: "Fourth Conference on Computational Natural Language Learning and the Second Learning Language in Logic Workshop"
source: "https://aclanthology.org/W00-0726/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/sang-2000-introduction-conll-2000-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/sang-2000-introduction-conll-2000-shared-task/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2000](https://md.research.elamur.ai/free/papers/2000/) / Introduction to the CoNLL-2000 Shared Task Chunking

# Introduction to the CoNLL-2000 Shared Task Chunking

**Authors:** Erik F. Tjong Kim Sang, [Sabine Buchholz](https://md.research.elamur.ai/free/authors/sabine-buchholz/) · **Year:** 2000 · **Venue:** Fourth Conference on Computational Natural Language Learning and the Second Learning Language in Logic Workshop · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W00-0726/) · [PDF](https://aclanthology.org/W00-0726.pdf) · [OpenAlex](https://openalex.org/W7029250385)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14) · [276](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper presents the CoNLL-2000 shared task on text chunking, which means splitting sentences into non-overlapping, syntactically coherent word groups such as noun, verb and prepositional phrases. Earlier work concentrated almost entirely on base noun phrases. This task extends the problem to eleven chunk types. No corpus was annotated directly for arbitrary chunks, so the organizers derived chunks automatically from parse trees in the WSJ portion of the Penn Treebank II. The rule they used is that a chunk runs from its left edge up to and including the syntactic head, with premodifiers but no arguments or postmodifiers. Sections 15–18 served for training and section 20 for testing. Tokens carry POS tags from the Brill tagger rather than gold tags, and chunks are encoded with B/I/O labels. Eleven participating systems were compared: rule-based, memory-based, statistical and combined. All of them beat a simple POS-based baseline. The top system, an SVM combination by Kudoh and Matsumoto, reached F(β=1) = 93.48. The resulting dataset and conversion script became a standard benchmark for shallow parsing.

## Key points
- **Data:** WSJ sections 15–18 of the Penn Treebank II for training and section 20 for testing, the same split as the Ramshaw and Marcus (1995) base-NP data. The training set has 211,727 tokens and 106,978 chunks.
- **Chunk types (11):** NP (51% of chunks), VP (20%), PP (20%), ADVP (4%), SBAR, ADJP, PRT, CONJP, INTJ, LST and UCP. UCP is the rarest, with only 2 instances in training.
- **Encoding:** each token gets a B-X tag (chunk-initial), an I-X tag (inside a chunk) or O (outside any chunk). This turns chunking into a per-word tagging problem.
- **Evaluation:** precision, recall and F(β=1), with the F-score as the optimization target. The baseline assigns each POS tag its most frequent chunk tag.
- **Systems:** eleven in four families: rule-based, memory-based, statistical (Markov models/HMMs, maximum entropy) and combined. The winner, Kudoh & Matsumoto, used 231 SVM classifiers merged by dynamic programming and scored F = 93.48.

## Interesting ideas
- **Deriving a new task from an existing treebank:** the chunk annotation was produced by a script from existing parse trees, with a head-based rule deciding chunk boundaries. Any language with a treebank could build a cheap shallow-parsing dataset the same way without fresh manual annotation.
- **Releasing the conversion script:** the configurable converter was published with the data, so others can change design decisions and measure how those choices affect task difficulty. This makes annotation conventions an object of study rather than a fixed assumption.
- **Predicted POS tags instead of gold tags:** inputs were tagged by a standard POS tagger instead of using treebank tags. Scores therefore reflect realistic conditions on raw text, which is a sensible default for any pipeline benchmark.
- **IOB encoding of spans:** representing non-overlapping spans as per-token B/I/O labels lets any sequence tagger handle segmentation. The same trick applies to NER, multiword expressions or address parsing.
- **Ensembles over different output representations:** several top systems gained by combining classifiers, for example by majority voting over learners trained on different chunk-tag encodings. Combining diverse views of the same task is a cheap way to beat any single model.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Treebanks](https://md.research.elamur.ai/free/topics/treebanks/) · [Part-of-speech tagging](https://md.research.elamur.ai/free/topics/part-of-speech-tagging/)

## Cited by

Cited by

276 papers of our library cite it (of the 140673 whose bibliographies we read); 17 of them have a page here, 259 not yet.

- [Noun Phrase Chunking in Hebrew: Influence of Lexical and Morphological Features](https://md.research.elamur.ai/free/papers/aclbulk-2006-goldberg-noun-phrase-chunking-hebrew-influence/)
- [SVM Model Tampering and Anchored Learning: A Case Study in Hebrew NP Chunking](https://md.research.elamur.ai/free/papers/aclbulk-2007-goldberg-svm-model-tampering-anchored-learning/)
- [Contextual String Embeddings for Sequence Labeling](https://md.research.elamur.ai/free/papers/akbik-2018-contextual-string-embeddings-sequence-labeling/)
- [FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP](https://md.research.elamur.ai/free/papers/akbik-2019-flair-easy-use-framework-state/)
- [Semi-Supervised Sequence Modeling with Cross-View Training](https://md.research.elamur.ai/free/papers/clark-2018-semi-supervised-sequence-modeling-cross/)
- [What Does BERT Learn about the Structure of Language?](https://md.research.elamur.ai/free/papers/jawahar-2019-what-does-bert-learn-about/)
- [Datasets: A Community Library for Natural Language Processing](https://md.research.elamur.ai/free/papers/lhoest-2021-datasets-community-library-natural-language/)
- [Towards Semi-Automated Annotation for Prepositional Phrase Attachment](https://md.research.elamur.ai/free/papers/lingbulk-2010-rosenthal-semi-automated-annotation-prepositional-phrase/)
- [Linguistic Knowledge and Transferability of Contextual Representations](https://md.research.elamur.ai/free/papers/liu-2019-linguistic-knowledge-transferability-contextual-representations/)
- [EDDA-Coordinata: An Annotated Dataset of Historical Geographic Coordinates](https://md.research.elamur.ai/free/papers/moncla-2026-edda-coordinata-annotated-dataset-historical/)
- [The Proposition Bank: An Annotated Corpus of Semantic Roles](https://md.research.elamur.ai/free/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/)
- [Semi-supervised sequence tagging with bidirectional language models](https://md.research.elamur.ai/free/papers/peters-2017-semi-supervised-sequence-tagging-bidirectional/)
- [Dissecting Contextual Word Embeddings: Architecture and Representation](https://md.research.elamur.ai/free/papers/peters-2018-dissecting-contextual-word-embeddings-architecture/)
- [The Importance of Syntactic Parsing and Inference in Semantic Role Labeling](https://md.research.elamur.ai/free/papers/punyakanok-2008-importance-syntactic-parsing-inference-semantic/)
- [The Possible, the Plausible, and the Desirable: Event-Based Modality Detection for Language Processing](https://md.research.elamur.ai/free/papers/pyatkin-2021-possible-plausible-desirable-event-based/)
- [Word Representations: A Simple and General Method for Semi-Supervised Learning](https://md.research.elamur.ai/free/papers/turian-2010-word-representations-simple-general-method/)
- [Investigating NP-Chunking with Universal Dependencies for English](https://md.research.elamur.ai/free/papers/ud-2018-lacroix-investigating-np-chunking-universal-dependencies/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sang2000introduction,
  title = {{Introduction to the CoNLL-2000 Shared Task Chunking}},
  author = {Sang, Erik F. Tjong Kim and Buchholz, Sabine},
  booktitle = {Fourth Conference on Computational Natural Language Learning and the Second Learning Language in Logic Workshop},
  year = {2000},
  url = {https://aclanthology.org/W00-0726/}
}
```

- [← previous Integrating Subject Field Codes into WordNet](https://md.research.elamur.ai/free/papers/magnini-2000-integrating-subject-field-codes-into/)
- [next → Multilingual Coreference Resolution](https://md.research.elamur.ai/free/papers/harabagiu-2000-multilingual-coreference-resolution/)
- [↑ back to papers of 2000](https://md.research.elamur.ai/free/papers/2000/)

## References

22 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Steven Abney. 1991. Parsing by chunks. In Principle-Based Parsing. Kluwer Academic Publishers. [find](https://scholar.google.com/scholar?q=Steven+Abney%2E+1991%2E+Parsing+by+chunks%2E+In+Principle%2DBased+Parsing%2E+Kluwer+Academic+Publishers%2E)
2. Ann Bies, Mark Ferguson, Karen Katz, and Robert MacIntyre. 1995. Bracket Guidelines /or Treebank H Style Penn Treebank Project. Penn Treebank II cdrom. [find](https://scholar.google.com/scholar?q=Ann+Bies%2C+Mark+Ferguson%2C+Karen+Katz%2C+and+Robert+MacIntyre%2E+1995%2E+Bracket+Guidelines+%2For+Treebank+H+Style+Penn+Treebank+Project%2E+Penn+Treebank+II+cdrom%2E)
3. Eric Brill. 1994. Some advances in rule-based part of speech tagging. In Proceedings o] the Twelfth National Con/erence on Artificial Intelligence (AAAI-9~). Seattle, Washington. [find](https://scholar.google.com/scholar?q=Eric+Brill%2E+1994%2E+Some+advances+in+rule%2Dbased+part+of+speech+tagging%2E+In+Proceedings+o%5D+the+Twelfth+National+Con%2Ference+on+Artificial+Intelligence+%28AAAI%2D9%7E%29%2E+Seattle%2C+Washington%2E)
4. Sabine Buchholz, Jorn Veenstra, and Walter Daelemans. 1999. Cascaded grammatical :relation assignment. In Proceedings o] EMNLP/VLC-99. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sabine+Buchholz%2C+Jorn+Veenstra%2C+and+Walter+Daelemans%2E+1999%2E+Cascaded+grammatical+%3Arelation+assignment%2E+In+Proceedings+o%5D+EMNLP%2FVLC%2D99%2E+Association+for+Computational+Linguistics%2E)
5. John Carroll, Ted Briscoe, Glenn Carroll, Marc Light, Dethleff Prescher, Mats Rooth, Stefano Federici, Simonetta Montemagni, Vito Pirrelli, Irina Prodanof, and Massimo Vanocchi. 1997. Phrasal Parsing Software. Sparkle Work Package 3, Deliverable D3.2. [find](https://scholar.google.com/scholar?q=John+Carroll%2C+Ted+Briscoe%2C+Glenn+Carroll%2C+Marc+Light%2C+Dethleff+Prescher%2C+Mats+Rooth%2C+Stefano+Federici%2C+Simonetta+Montemagni%2C+Vito+Pirrelli%2C+Irina+Prodanof%2C+and+Massimo+Vanocchi%2E+1997%2E+Phrasal+Parsing+)
6. Kenneth Ward Church. 1988. A stochastic parts program and noun phrase parser for unrestricted text. In Second Con]erence on Applied Natural Language Processing. Austin, Texas. [find](https://scholar.google.com/scholar?q=Kenneth+Ward+Church%2E+1988%2E+A+stochastic+parts+program+and+noun+phrase+parser+for+unrestricted+text%2E+In+Second+Con%5Derence+on+Applied+Natural+Language+Processing%2E+Austin%2C+Texas%2E)
7. H@rve D@jean. 2000. Learning syntactic structures with xml. In Proceedings o/ CoN~LL-2000 and LLL-2000. Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=H%40rve+D%40jean%2E+2000%2E+Learning+syntactic+structures+with+xml%2E+In+Proceedings+o%2F+CoN%7ELL%2D2000+and+LLL%2D2000%2E+Lisbon%2C+Portugal%2E)
8. Christer Johansson. 2000. A context sensitive maximum likelihood approach to chunking. In Proceedings o] CoNLL-2000 and LLL-2000. [find](https://scholar.google.com/scholar?q=Christer+Johansson%2E+2000%2E+A+context+sensitive+maximum+likelihood+approach+to+chunking%2E+In+Proceedings+o%5D+CoNLL%2D2000+and+LLL%2D2000%2E)
9. Lisbon, Portugal. Rob Koeling. 2000. Chunking with maximum entropy models. In Proceedings o/ CoNLL-2000 and LLL-2000. [find](https://scholar.google.com/scholar?q=Lisbon%2C+Portugal%2E+Rob+Koeling%2E+2000%2E+Chunking+with+maximum+entropy+models%2E+In+Proceedings+o%2F+CoNLL%2D2000+and+LLL%2D2000%2E)
10. Lisbon, Portugal. Taku Kudoh and Yuji Matsumoto. 2000. Use of support vector learning for chunk identification. In Proceedings o~ CoNLL-2000 and LLL-2000. [find](https://scholar.google.com/scholar?q=Lisbon%2C+Portugal%2E+Taku+Kudoh+and+Yuji+Matsumoto%2E+2000%2E+Use+of+support+vector+learning+for+chunk+identification%2E+In+Proceedings+o%7E+CoNLL%2D2000+and+LLL%2D2000%2E)
11. Lisbon, Portugal. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: the penn treebank. Computational Linguistics, 19(2). [find](https://scholar.google.com/scholar?q=Lisbon%2C+Portugal%2E+Mitchell+P%2E+Marcus%2C+Beatrice+Santorini%2C+and+Mary+Ann+Marcinkiewicz%2E+1993%2E+Building+a+large+annotated+corpus+of+english%3A+the+penn+treebank%2E+Computational+Linguistics%2C+19%282%29%2E) [our card](https://md.research.elamur.ai/free/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
12. Miles Osborne. 2000. Shallow parsing as part-of-speech tagging. In Proceedings o] CoNLL-2000 and LLL-2000. Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Miles+Osborne%2E+2000%2E+Shallow+parsing+as+part%2Dof%2Dspeech+tagging%2E+In+Proceedings+o%5D+CoNLL%2D2000+and+LLL%2D2000%2E+Lisbon%2C+Portugal%2E)
13. Ferran Pla, Antonio Molina, and Natividad Prieto. 2000. Improving chunking by means of lexical-contextual information in statistical language models. In Proceedings o] CoNLL-2000 and LLL-2000. Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Ferran+Pla%2C+Antonio+Molina%2C+and+Natividad+Prieto%2E+2000%2E+Improving+chunking+by+means+of+lexical%2Dcontextual+information+in+statistical+language+models%2E+In+Proceedings+o%5D+CoNLL%2D2000+and+LLL%2D2000%2E+Lisbon%2C)
14. Lance A. Ramshaw and Mitchell P. Marcus. 1995. Text chunking using transformation-based learning. In Proceedings o] the Third A CL Workshop on Very Large Corpora. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Lance+A%2E+Ramshaw+and+Mitchell+P%2E+Marcus%2E+1995%2E+Text+chunking+using+transformation%2Dbased+learning%2E+In+Proceedings+o%5D+the+Third+A+CL+Workshop+on+Very+Large+Corpora%2E+Association+for+Computational+Linguis)
15. Adwait Ratnaparkhi. 1998. Maximum Entropy Models ]or Natural Language Ambiguity Resolution. PhD thesis Computer and Information Science, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Adwait+Ratnaparkhi%2E+1998%2E+Maximum+Entropy+Models+%5Dor+Natural+Language+Ambiguity+Resolution%2E+PhD+thesis+Computer+and+Information+Science%2C+University+of+Pennsylvania%2E)
16. Erik F. Tjong Kim Sang. 2000. Text chunking by system combination. In Proceedings of CoNLL-2000 and LLL-2000. Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Erik+F%2E+Tjong+Kim+Sang%2E+2000%2E+Text+chunking+by+system+combination%2E+In+Proceedings+of+CoNLL%2D2000+and+LLL%2D2000%2E+Lisbon%2C+Portugal%2E)
17. Hans van Halteren. 2000. Chunking with wpdv models. In Proceedings o/ CoNLL-2000 and LLL-2000. Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Hans+van+Halteren%2E+2000%2E+Chunking+with+wpdv+models%2E+In+Proceedings+o%2F+CoNLL%2D2000+and+LLL%2D2000%2E+Lisbon%2C+Portugal%2E)
18. C.J. van Rijsbergen. 1975. In/ormation Retrieval. Buttersworth. [find](https://scholar.google.com/scholar?q=C%2EJ%2E+van+Rijsbergen%2E+1975%2E+In%2Formation+Retrieval%2E+Buttersworth%2E)
19. Jorn Veenstra and Antal van den Bosch. 2000. Single-classifier memory-based phrase chunking. In Proceedings o] CoNLL-2000 and LLL-2000. Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Jorn+Veenstra+and+Antal+van+den+Bosch%2E+2000%2E+Single%2Dclassifier+memory%2Dbased+phrase+chunking%2E+In+Proceedings+o%5D+CoNLL%2D2000+and+LLL%2D2000%2E+Lisbon%2C+Portugal%2E)
20. Jorn Veenstra. 1999. Memory-based text chunking. In Nikos Fakotakis, editor, Machine learning in human language technology, workshop at ACAI 99. [find](https://scholar.google.com/scholar?q=Jorn+Veenstra%2E+1999%2E+Memory%2Dbased+text+chunking%2E+In+Nikos+Fakotakis%2C+editor%2C+Machine+learning+in+human+language+technology%2C+workshop+at+ACAI+99%2E)
21. Marc Vilain and David Day. 2000. Phrase parsing with rule sequence processors: an application to the shared conll task. In Proceedings o/CoNLL-2000 and LLL-2000. Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Marc+Vilain+and+David+Day%2E+2000%2E+Phrase+parsing+with+rule+sequence+processors%3A+an+application+to+the+shared+conll+task%2E+In+Proceedings+o%2FCoNLL%2D2000+and+LLL%2D2000%2E+Lisbon%2C+Portugal%2E)
22. GuoDong Zhou, Jian Su, and TongGuan Tey. 2000. Hybrid text chunking. In Proceedings o/CoNLL-2000 and LLL-2000. Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=GuoDong+Zhou%2C+Jian+Su%2C+and+TongGuan+Tey%2E+2000%2E+Hybrid+text+chunking%2E+In+Proceedings+o%2FCoNLL%2D2000+and+LLL%2D2000%2E+Lisbon%2C+Portugal%2E)