---
title: "Introduction to the CoNLL-2002 Shared Task — F 2002"
description: "This paper introduces the CoNLL-2002 shared task, which asked participants to build named entity recognizers that include a machine learning component…"
authors: ["Tjong Kim Sang, Erik F."]
year: 2002
venue: "COLING-02: The 6th Conference on Natural Language Learning 2002 (CoNLL-2002)"
source: "https://aclanthology.org/W02-2024/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/sang-2002-introduction-conll-2002-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/sang-2002-introduction-conll-2002-shared-task/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2002](https://md.research.elamur.ai/free/papers/2002/) / Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition

# Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition

**Authors:** Tjong Kim Sang, Erik F. · **Year:** 2002 · **Venue:** COLING-02: The 6th Conference on Natural Language Learning 2002 (CoNLL-2002) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W02-2024/) · [PDF](https://aclanthology.org/W02-2024.pdf)

**Citations:** [291](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
This paper introduces the CoNLL-2002 shared task, which asked participants to build named entity recognizers that include a machine learning component and that work across languages. The organizers supplied annotated newswire data for Spanish and Dutch. Each language has training, development and test files, so systems can be tuned without seeing the test set. Four entity classes are tagged: PER, ORG, LOC and MISC. Tokens are labelled with a variant of the IOB scheme, and entities are treated as flat, with no nesting or overlap. Systems are scored with F(β=1), and a predicted entity counts only if its span and type exactly match the annotation. The organizers especially encouraged the use of unannotated data. Twelve systems took part, and all of them beat a simple lexical-lookup baseline. The winner was a boosted decision tree approach (AdaBoost over fixed-depth trees, Carreras et al.). It reached F=81.39 on Spanish and 77.05 on Dutch. The task set up a reusable multilingual NER benchmark and evaluation protocol that later work built on.

## Key points
- **Spanish data:** EFE News Agency wire articles from May 2000, annotated by TALP (UPC) and CLiC (UB) under the NAMIC project. The files contain only words and entity tags, with 273,037, 54,837 and 53,049 lines for train, dev and test.
- **Dutch data:** four 2000 issues of the Belgian newspaper "De Morgen", annotated in the Atranos project following the MITRE/SAIC guidelines. The files add automatically assigned part-of-speech tags and mark article boundaries with `-DOCSTART-`. They have 218,737, 40,656 and 74,189 lines.
- **Baseline:** it tags only phrases that had a single unambiguous class in the training data and prefers the longest match. Every participating system outperformed it.
- **Top results:**
  - Carreras et al.: 81.39 (Spanish) / 77.05 (Dutch).
  - Florian, a stack of TBL, SNoW and forward-backward: 79.05 / 74.99.
  - Wu et al., AdaBoost.MH on decision stumps: 76.61 / 75.36.
  - The lowest system scored roughly 57–61.
- **Approaches used:** SVMs, transformation-based learning, AdaBoost, maximum entropy, HMMs and Markov models, memory-based learning, and character tries. Most strong systems separated boundary detection from entity classification.

## Interesting ideas
- **Separate development split:** keeping tuning data apart from the final test set prevents systems from overfitting to the evaluation. It is a simple discipline worth enforcing in any shared benchmark, including a Ukrainian NER one.
- **Two-step decomposition:** many entries first found entity spans without types and then assigned classes. This lets each subproblem use its own features and learner, which can help when labelled data is scarce.
- **Gazetteers versus induced lexicons:** Burger et al. found that word classes derived from the available data helped more than an external list of about 250,000 names. Malouf also saw no gain from a person-name list. This suggests trying lexical resources induced from your own data before investing in curated name lists.
- **Document-level consistency:** Cucerzan and Yarowsky used "one sense per discourse", and Malouf added a feature for whether a word had appeared earlier in the text as part of a person name. Both exploit the tendency of a name to keep its type throughout an article, which is cheap to add for news-like text.
- **Stacking and cascades:** Tsukamoto et al., Florian and Tjong Kim Sang each fed one learner's output into the next. Their papers report clear gains over single models, so this is a generic, transferable way to boost sequence labellers.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Named entity recognition](https://md.research.elamur.ai/free/topics/named-entity-recognition/)

## Cited by

Cited by

291 papers of our library cite it (of the 140673 whose bibliographies we read); 12 of them have a page here, 279 not yet.

- [Syntax-augmented Multilingual BERT for Cross-lingual Transfer](https://md.research.elamur.ai/free/papers/ahmad-2021-syntax-augmented-multilingual-bert-cross/)
- [BERTje: A Dutch BERT Model](https://md.research.elamur.ai/free/papers/de-2019-vries-bertje-dutch-bert-model/)
- [RobBERT: a Dutch RoBERTa-based Language Model](https://md.research.elamur.ai/free/papers/delobelle-2020-robbert-dutch-roberta-based-language/)
- [Few-NERD: A Few-shot Named Entity Recognition Dataset](https://md.research.elamur.ai/free/papers/ding-2021-few-nerd-few-shot-named/)
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://md.research.elamur.ai/free/papers/lai-2023-chatgpt-beyond-english-comprehensive-evaluation/)
- [EDDA-Coordinata: An Annotated Dataset of Historical Geographic Coordinates](https://md.research.elamur.ai/free/papers/moncla-2026-edda-coordinata-annotated-dataset-historical/)
- [Massively Multilingual Transfer for NER](https://md.research.elamur.ai/free/papers/rahimi-2019-massively-multilingual-transfer-ner/)
- [Cross-lingual Word Clusters for Direct Transfer of Linguistic Structure](https://md.research.elamur.ai/free/papers/tackstrom-2012-cross-lingual-word-clusters-direct/)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://md.research.elamur.ai/free/papers/ud-2020-qi-stanza-python-natural-language-processing/)
- [Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing](https://md.research.elamur.ai/free/papers/ud-2021-nguyen-trankit-light-weight-transformer-based/)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://md.research.elamur.ai/free/papers/wu-2019-beto-bentz-becas-surprising-cross/)
- [Named Entity Recognition as Dependency Parsing](https://md.research.elamur.ai/free/papers/yu-2020-named-entity-recognition-as-dependency/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{f2002introduction,
  title = {{Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition}},
  author = {F},
  booktitle = {COLING-02: The 6th Conference on Natural Language Learning 2002 (CoNLL-2002)},
  year = {2002},
  url = {https://aclanthology.org/W02-2024/}
}
```

- [← previous Improving Machine Learning Approaches to Coreference Resolution](https://md.research.elamur.ai/free/papers/ng-2002-improving-machine-learning-approaches-coreference/)
- [next → Learning Domain-Specific Transfer Rules: An Experiment with Korean to English Translation](https://md.research.elamur.ai/free/papers/lavoie-2002-learning-domain-specific-transfer-rules/)
- [↑ back to papers of 2002](https://md.research.elamur.ai/free/papers/2002/)

## References

13 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. William J. Black and Argyrios Vasilakopoulos. 2002. Language-independent named entity classi cation by modi ed transformation-based learning and by decision tree induction. In Proceedings of CoNLL-2002 . Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=William+J%2E+Black+and+Argyrios+Vasilakopoulos%2E+2002%2E+Language%2Dindependent+named+entity+classi+cation+by+modi+ed+transformation%2Dbased+learning+and+by+decision+tree+induction%2E+In+Proceedings+of+CoNLL%2D200)
2. John D. Burger, John C. Henderson, and William T. Morgan. 2002. Statistical named entity recognizer adaptation. In Proceedings of CoNLL-2002 . Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=John+D%2E+Burger%2C+John+C%2E+Henderson%2C+and+William+T%2E+Morgan%2E+2002%2E+Statistical+named+entity+recognizer+adaptation%2E+In+Proceedings+of+CoNLL%2D2002+%2E+Taipei%2C+Taiwan%2E)
3. Xavier Carreras, Llus Marques, and Llus Padro. 2002. Named entity extraction using adaboost. In Proceedings of CoNLL-2002 . Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=Xavier+Carreras%2C+Llu%13%10s+M%12arques%2C+and+Llu%13%10s+Padr%13o%2E+2002%2E+Named+entity+extraction+using+adaboost%2E+In+Proceedings+of+CoNLL%2D2002+%2E+Taipei%2C+Taiwan%2E)
4. Nancy Chinchor, Erica Brown, Lisa Ferro, and Patty Robinson. 1999. 1999 Named Entity Recognition Task De nition . MITRE and SAIC. [find](https://scholar.google.com/scholar?q=Nancy+Chinchor%2C+Erica+Brown%2C+Lisa+Ferro%2C+and+Patty+Robinson%2E+1999%2E+1999+Named+Entity+Recognition+Task+De+nition+%2E+MITRE+and+SAIC%2E)
5. Silviu Cucerzan and David Yarowsky. 2002. Language independent ner using a uni ed model of internal and contextual evidence. In Proceedings of CoNLL-2002 . Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=Silviu+Cucerzan+and+David+Yarowsky%2E+2002%2E+Language+independent+ner+using+a+uni+ed+model+of+internal+and+contextual+evidence%2E+In+Proceedings+of+CoNLL%2D2002+%2E+Taipei%2C+Taiwan%2E)
6. Walter Daelemans, Jakub Zavrel, Peter Berck, and Steven Gillis. 1996. Mbt: A memory-based part of speech tagger-generator. In Proceedings of the Fourth Workshop on Very Large Corpora , pages 14{27. Copenhagen, Denmark. [find](https://scholar.google.com/scholar?q=Walter+Daelemans%2C+Jakub+Zavrel%2C+Peter+Berck%2C+and+Steven+Gillis%2E+1996%2E+Mbt%3A+A+memory%2Dbased+part+of+speech+tagger%2Dgenerator%2E+In+Proceedings+of+the+Fourth+Workshop+on+Very+Large+Corpora+%2C+pages+14%7B27%2E+Co)
7. Radu Florian. 2002. Named entity recognition as a house of cards: Classi er stacking. In Proceedings of CoNLL-2002 . Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=Radu+Florian%2E+2002%2E+Named+entity+recognition+as+a+house+of+cards%3A+Classi+er+stacking%2E+In+Proceedings+of+CoNLL%2D2002+%2E+Taipei%2C+Taiwan%2E)
8. Martin Jansche. 2002. Named entity extraction with conditional markov models and classi ers. In Proceedings of CoNLL-2002 . Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=Martin+Jansche%2E+2002%2E+Named+entity+extraction+with+conditional+markov+models+and+classi+ers%2E+In+Proceedings+of+CoNLL%2D2002+%2E+Taipei%2C+Taiwan%2E)
9. Robert Malouf. 2002. Markov models for language-independent named entity recognition. In Proceedings of CoNLL-2002 . Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=Robert+Malouf%2E+2002%2E+Markov+models+for+language%2Dindependent+named+entity+recognition%2E+In+Proceedings+of+CoNLL%2D2002+%2E+Taipei%2C+Taiwan%2E)
10. Paul McNamee and James May eld. 2002. Entity extraction without language-speci c resources. In Proceedings of CoNLL-2002 . Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=Paul+McNamee+and+James+May+eld%2E+2002%2E+Entity+extraction+without+language%2Dspeci+c+resources%2E+In+Proceedings+of+CoNLL%2D2002+%2E+Taipei%2C+Taiwan%2E)
11. Jon Patrick, Casey Whitelaw, and Robert Munro. 2002. Slinerc: The sydney language-independent named entity recogniser and classi er. In Proceedings of CoNLL-2002 . Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=Jon+Patrick%2C+Casey+Whitelaw%2C+and+Robert+Munro%2E+2002%2E+Slinerc%3A+The+sydney+language%2Dindependent+named+entity+recogniser+and+classi+er%2E+In+Proceedings+of+CoNLL%2D2002+%2E+Taipei%2C+Taiwan%2E)
12. Lance A. Ramshaw and Mitchell P. Marcus. 1995. Text chunking using transformation-based learning. In Proceedings of the Third ACL Workshop on Very Large Corpora , pages 82{94. Cambridge, MA, USA. [find](https://scholar.google.com/scholar?q=Lance+A%2E+Ramshaw+and+Mitchell+P%2E+Marcus%2E+1995%2E+Text+chunking+using+transformation%2Dbased+learning%2E+In+Proceedings+of+the+Third+ACL+Workshop+on+Very+Large+Corpora+%2C+pages+82%7B94%2E+Cambridge%2C+MA%2C+USA%2E)
13. Erik F. Tjong Kim Sang. 2002. Memory-based [find](https://scholar.google.com/scholar?q=Erik+F%2E+Tjong+Kim+Sang%2E+2002%2E+Memory%2Dbased)