---
title: "Neural Architectures for Nested NER through — Straková 2019"
description: "Nested named entity recognition deals with mentions that overlap or sit inside one another."
authors: ["Jana Straková", "Milan Straka", "Jan Hajic"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1527/"
doi: "10.18653/v1/p19-1527"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/strakova-2019-neural-architectures-nested-ner-through/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/strakova-2019-neural-architectures-nested-ner-through/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/2/) / Neural Architectures for Nested NER through Linearization

# Neural Architectures for Nested NER through Linearization

**Authors:** [Jana Straková](https://md.research.elamur.ai/free/en/authors/jana-strakova/), [Milan Straka](https://md.research.elamur.ai/free/en/authors/milan-straka/), [Jan Hajic](https://md.research.elamur.ai/free/en/authors/jan-hajic/) · **Year:** 2019 · **Venue:** Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1527/) · [PDF](https://aclanthology.org/P19-1527.pdf) · [DOI 10.18653/v1/p19-1527](https://doi.org/10.18653/v1/p19-1527) · [OpenAlex](https://openalex.org/W2952594430)

**Citations:** 309 citations (OpenAlex, 2026-10-14) · [91](https://md.research.elamur.ai/free/en/papers/strakova-2019-neural-architectures-nested-ner-through/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Nested named entity recognition deals with mentions that overlap or sit inside one another. For example, a country name can appear within a longer organization name. Earlier systems handled this by building explicit structures such as constituency trees, hypergraphs or stacked recognizers. This paper avoids any such structure. It turns the nesting into per-token label sequences using an extended BILOU scheme and lets the network infer the relationships on its own.

The authors propose two encoder–decoder models, both with a BiLSTM encoder:
- **LSTM-CRF:** each combination of nested labels is treated as a single multilabel, so an ordinary flat NER pipeline can be reused.
- **seq2seq:** an LSTM decoder with hard attention on the current word outputs that word's labels one at a time, then emits an end-of-word symbol.

The models are tested on four nested corpora (ACE-2004, ACE-2005, GENIA, Czech CNEC) and on CoNLL flat NER data. Even without contextual embeddings, seq2seq beats the previous best results on all four nested corpora. Adding ELMo, BERT and Flair raises scores further. The work shows that simple linearization can match or outperform structure-heavy approaches.

## Key points
- **Encoding:** Labels of overlapping entities are joined into one multilabel per token. Entities that start earlier come first, and among entities with the same start, longer ones come first. Decoding merges labels across neighboring tokens by their position in the multilabel.
- **Baseline seq2seq results (no contextual embeddings):** F1 of 77.08 on ACE-2004, 75.36 on ACE-2005, 76.44 on GENIA and 82.96 on CNEC 1.0. The previous best results were 75.1, 74.5, 75.1 and 81.20.
- **Best results (seq2seq+BERT+Flair):** F1 of 84.40 on ACE-2004, 84.33 on ACE-2005, 78.31 on GENIA and 86.88 on CNEC 1.0.
- **Which model suits which data:** seq2seq works best on heavily nested data such as ACE, where about 39% of training sentences contain overlaps, compared with 22% in GENIA. LSTM-CRF is preferable for flat or less nested corpora.
- **Flat NER:** New state-of-the-art F1 on CoNLL-2003 English (93.38), CoNLL-2002 Dutch (92.69) and CoNLL-2002 Spanish (88.81). CoNLL-2003 German remains below Flair's 88.32.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{strakova2019neural,
  title = {{Neural Architectures for Nested NER through Linearization}},
  author = {Straková, Jana and Straka, Milan and Hajic, Jan},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1527},
  url = {https://doi.org/10.18653/v1/p19-1527}
}
```

- [← previous How to Fine-Tune BERT for Text Classification?](https://md.research.elamur.ai/free/en/papers/sun-2019-fine-tune-bert-text-classification/)
- [next → Disentangled Representation Learning for Non-Parallel Text Style Transfer](https://md.research.elamur.ai/free/en/papers/john-2019-disentangled-representation-learning-non-parallel/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/2/)

## References

23 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018. Contextual String Embeddings for Sequence Labeling. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1638–1649. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alan+Akbik%2C+Duncan+Blythe%2C+and+Roland+Vollgraf%2E+2018%2E+Contextual+String+Embeddings+for+Sequence+Labeling%2E+In+Proceedings+of+the+27th+International+Conference+on+Computational+Linguistics%2C+pages+1638%E2%80%931) [our card](https://md.research.elamur.ai/free/en/papers/akbik-2018-contextual-string-embeddings-sequence-labeling/)
2. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5:135–146. [find](https://scholar.google.com/scholar?q=Piotr+Bojanowski%2C+Edouard+Grave%2C+Armand+Joulin%2C+and+Tomas+Mikolov%2E+2017%2E+Enriching+Word+Vectors+with+Subword+Information%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+5%3A135%E2%80%93146%2E)
3. KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. CoRR. [find](https://scholar.google.com/scholar?q=KyungHyun+Cho%2C+Bart+van+Merrienboer%2C+Dzmitry+Bahdanau%2C+and+Yoshua+Bengio%2E+2014%2E+On+the+Properties+of+Neural+Machine+Translation%3A+Encoder%2DDecoder+Approaches%2E+CoRR%2E) [our card](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/)
4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+BERT%3A+Pre%2Dtraining+of+Deep+Bidirectional+Transformers+for+Language+Understanding%2E)
5. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The Automatic Content Extraction (ACE) program-tasks, data, and evaluation. Proceedings of LREC, 2. [find](https://scholar.google.com/scholar?q=George+Doddington%2C+Alexis+Mitchell%2C+Mark+Przybocki%2C+Lance+Ramshaw%2C+Stephanie+Strassel%2C+and+Ralph+Weischedel%2E+2004%2E+The+Automatic+Content+Extraction+%28ACE%29+program%2Dtasks%2C+data%2C+and+evaluation%2E+Proceedin) [our card](https://md.research.elamur.ai/free/en/papers/doddington-2004-automatic-content-extraction-ace-program/)
6. Jenny Rose Finkel and Christopher D. Manning. 2009. Nested Named Entity Recognition. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1, EMNLP ’09, pages 141–150. [find](https://scholar.google.com/scholar?q=Jenny+Rose+Finkel+and+Christopher+D%2E+Manning%2E+2009%2E+Nested+Named+Entity+Recognition%2E+In+Proceedings+of+the+2009+Conference+on+Empirical+Methods+in+Natural+Language+Processing%3A+Volume+1+%2D+Volume+1%2C+EMN) [our card](https://md.research.elamur.ai/free/en/papers/finkel-2009-nested-named-entity-recognition/)
7. Dan Gillick, Cliff Brunk, Oriol Vinyals, and Amarnag Subramanya. 2016. Multilingual language processing from bytes. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1296–1306. Association for Comp… [find](https://scholar.google.com/scholar?q=Dan+Gillick%2C+Cliff+Brunk%2C+Oriol+Vinyals%2C+and+Amarnag+Subramanya%2E+2016%2E+Multilingual+language+processing+from+bytes%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+Associati)
8. Alex Graves and Jürgen Schmidhuber. 2005. Frame-wise phoneme classification with bidirectional lstm and other neural network architectures. Neural Networks, pages 5–6. [find](https://scholar.google.com/scholar?q=Alex+Graves+and+Ju%CC%88rgen+Schmidhuber%2E+2005%2E+Frame%2Dwise+phoneme+classification+with+bidirectional+lstm+and+other+neural+network+architectures%2E+Neural+Networks%2C+pages+5%E2%80%936%2E)
9. Meizhi Ju, Makoto Miwa, and Sophia Ananiadou. 2018. A neural layered model for nested named entity recognition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1446–1459.… [find](https://scholar.google.com/scholar?q=Meizhi+Ju%2C+Makoto+Miwa%2C+and+Sophia+Ananiadou%2E+2018%2E+A+neural+layered+model+for+nested+named+entity+recognition%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter+of+the+Association+f) [our card](https://md.research.elamur.ai/free/en/papers/ju-2018-neural-layered-model-nested-named/)
10. Arzoo Katiyar and Claire Cardie. 2018. Nested named entity recognition revisited. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 861–871. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Arzoo+Katiyar+and+Claire+Cardie%2E+2018%2E+Nested+named+entity+recognition+revisited%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+)
11. Jing-Dong Kim, Tomoto Ohta, Yuka Tateisi, and Jun’ichi Tsujii. 2003. GENIA corpus—A semantically annotated corpus for bio-textmining. Bioinformatics (Oxford, England), 19 Suppl 1:180–182. [find](https://scholar.google.com/scholar?q=Jing%2DDong+Kim%2C+Tomoto+Ohta%2C+Yuka+Tateisi%2C+and+Jun%E2%80%99ichi+Tsujii%2E+2003%2E+GENIA+corpus%E2%80%94A+semantically+annotated+corpus+for+bio%2Dtextmining%2E+Bioinformatics+%28Oxford%2C+England%29%2C+19+Suppl+1%3A180%E2%80%93182%2E)
12. Diederik Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Diederik+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+Method+for+Stochastic+Optimization%2E+International+Conference+on+Learning+Representations%2E)
13. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural Architectures for Named Entity Recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, page… [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Miguel+Ballesteros%2C+Sandeep+Subramanian%2C+Kazuya+Kawakami%2C+and+Chris+Dyer%2E+2016%2E+Neural+Architectures+for+Named+Entity+Recognition%2E+In+Proceedings+of+the+2016+Conference+of+the+North+)
14. Wang Ling, Tiago Luı́s, Luı́s Marujo, Ramón Fernandez Astudillo, Silvio Amir, Chris Dyer, Alan W. Black, and Isabel Trancoso. 2015. Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation. CoRR. [find](https://scholar.google.com/scholar?q=Wang+Ling%2C+Tiago+Lu%C4%B1%CC%81s%2C+Lu%C4%B1%CC%81s+Marujo%2C+Ramo%CC%81n+Fernandez+Astudillo%2C+Silvio+Amir%2C+Chris+Dyer%2C+Alan+W%2E+Black%2C+and+Isabel+Trancoso%2E+2015%2E+Finding+Function+in+Form%3A+Compositional+Character+Models+for+Open+V) [our card](https://md.research.elamur.ai/free/en/papers/ling-2015-finding-function-form-compositional-character/)
15. Jiangming Liu and Yue Zhang. 2017. Shift-Reduce Constituent Parsing with Neural Lookahead Features. Transactions of the Association for Computational Linguistics, 5:45–58. [find](https://scholar.google.com/scholar?q=Jiangming+Liu+and+Yue+Zhang%2E+2017%2E+Shift%2DReduce+Constituent+Parsing+with+Neural+Lookahead+Features%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+5%3A45%E2%80%9358%2E)
16. Wei Lu and Dan Roth. 2015. Joint mention extraction and classification with mention hypergraphs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 857–867. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wei+Lu+and+Dan+Roth%2E+2015%2E+Joint+mention+extraction+and+classification+with+mention+hypergraphs%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+857%E2%80%9386)
17. Aldrian Obaja Muis and Wei Lu. 2017. Labeling Gaps Between Words: Recognizing Overlapping Mentions with Mention Separators. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2608–2618. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aldrian+Obaja+Muis+and+Wei+Lu%2E+2017%2E+Labeling+Gaps+Between+Words%3A+Recognizing+Overlapping+Mentions+with+Mention+Separators%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Langua)
18. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep Contextualized Word Representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… [find](https://scholar.google.com/scholar?q=Matthew+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+Contextualized+Word+Representations%2E+In+Proceedings+of+the+2018+Conference+of+t)
19. Lev Ratinov and Dan Roth. 2009. Design Challenges and Misconceptions in Named Entity Recognition. In CoNLL ’09: Proceedings of the Thirteenth Conference on Computational Natural Language Learning, pages 147–155. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Lev+Ratinov+and+Dan+Roth%2E+2009%2E+Design+Challenges+and+Misconceptions+in+Named+Entity+Recognition%2E+In+CoNLL+%E2%80%9909%3A+Proceedings+of+the+Thirteenth+Conference+on+Computational+Natural+Language+Learning%2C+pag) [our card](https://md.research.elamur.ai/free/en/papers/ratinov-2009-design-challenges-misconceptions-named-entity/)
20. Jana Straková, Milan Straka, and Jan Hajič. 2016. Neural Networks for Featureless Named Entity Recognition in Czech. In Text, Speech, and Dialogue: 19th International Conference, TSD 2016, Brno , Czech Republic, September 12-16, 2016, Proceedings, pages 173–181. Springer International Publishing. [find](https://scholar.google.com/scholar?q=Jana+Strakova%CC%81%2C+Milan+Straka%2C+and+Jan+Hajic%CC%8C%2E+2016%2E+Neural+Networks+for+Featureless+Named+Entity+Recognition+in+Czech%2E+In+Text%2C+Speech%2C+and+Dialogue%3A+19th+International+Conference%2C+TSD+2016%2C+Brno+%2C+Cz)
21. Erik F. Tjong Kim Sang. 2002. Introduction to the CoNLL-2002 Shared Task: Language-independent Named Entity Recognition. In Proceedings of the 6th Conference on Natural Language Learning -Volume 20, COLING-02, pages 1–4, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Erik+F%2E+Tjong+Kim+Sang%2E+2002%2E+Introduction+to+the+CoNLL%2D2002+Shared+Task%3A+Language%2Dindependent+Named+Entity+Recognition%2E+In+Proceedings+of+the+6th+Conference+on+Natural+Language+Learning+%2DVolume+20%2C+C) [our card](https://md.research.elamur.ai/free/en/papers/sang-2002-introduction-conll-2002-shared-task/)
22. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. In Proceedings of CoNLL-2003, pages 142– 147. Edmonton, Canada. [find](https://scholar.google.com/scholar?q=Erik+F%2E+Tjong+Kim+Sang+and+Fien+De+Meulder%2E+2003%2E+Introduction+to+the+CoNLL%2D2003+Shared+Task%3A+Language%2DIndependent+Named+Entity+Recognition%2E+In+Proceedings+of+CoNLL%2D2003%2C+pages+142%E2%80%93+147%2E+Edmonton%2C+Can)
23. Bailin Wang and Wei Lu. 2018. Neural segmental hypergraphs for overlapping mention recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 204–214. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bailin+Wang+and+Wei+Lu%2E+2018%2E+Neural+segmental+hypergraphs+for+overlapping+mention+recognition%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+204%E2%80%93214)