---
title: "Contextual String Embeddings for Sequence — Akbik 2018"
description: "Sequence labeling tasks such as NER, chunking and PoS tagging depend heavily on word embeddings."
authors: ["Alan Akbik", "Duncan Blythe", "Roland Vollgraf"]
year: 2018
venue: "Proceedings of the 27th International Conference on Computational Linguistics"
source: "https://aclanthology.org/C18-1139/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/akbik-2018-contextual-string-embeddings-sequence-labeling/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/akbik-2018-contextual-string-embeddings-sequence-labeling/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/) / Contextual String Embeddings for Sequence Labeling

# Contextual String Embeddings for Sequence Labeling

**Authors:** [Alan Akbik](https://md.research.elamur.ai/free/en/authors/alan-akbik/), [Duncan Blythe](https://md.research.elamur.ai/free/en/authors/duncan-blythe/), [Roland Vollgraf](https://md.research.elamur.ai/free/en/authors/roland-vollgraf/) · **Year:** 2018 · **Venue:** Proceedings of the 27th International Conference on Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/C18-1139/) · [PDF](https://aclanthology.org/C18-1139.pdf) · [OpenAlex](https://openalex.org/W2880875857)

**Citations:** 924 citations (openalex-snapshot · 2026-10-14) · [466](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Sequence labeling tasks such as NER, chunking and PoS tagging depend heavily on word embeddings. Each existing type covers only part of what is useful. Pre-trained word vectors ignore context, character features are learned only from task data, and contextual embeddings from word-level LMs are costly to train. The authors introduce contextual string embeddings, built from a bidirectional character-level LSTM language model pre-trained on unlabeled text. For each word, they concatenate two hidden states: the forward model's state right after the word's final character and the backward model's state just before its first character. The resulting vectors change with the surrounding text and treat words purely as character sequences, which helps with rare or misspelled forms. Fed into a standard BiLSTM-CRF tagger, they set new state-of-the-art results on CoNLL03 NER for English and German. They also slightly beat prior bests on CoNLL2000 chunking and Penn treebank PoS tagging. The character LM is cheap enough to retrain for new languages or domains, and all code and models are released in the Flair framework.

## Key points
- CoNLL03 NER: 93.09 F1 for English (previous best 92.22, Peters et al., 2018) and about 88.3 F1 for German (previous best 78.76, Lample et al., 2016), a gain of roughly 9.5 points.
- On saturated tasks the gains are small: 96.72 on CoNLL2000 chunking (vs. 96.37) and 97.85 accuracy on Penn treebank PoS tagging (vs. 97.64).
- When the BiLSTM is swapped for a simple feedforward map (Map-CRF), scores drop only about 3% on average with the proposed embeddings. With classic embeddings such as GloVe, Komninos or FastText, the drop is about 20%. This suggests the new vectors carry rich information on their own.
- Nearest-neighbor checks on the word "Washington" in CoNLL03 show the embeddings separate person, place, legislative body and sports team readings depending on context.
- The character LM trains in about 1 week on 1 GPU, compared with 5 weeks on 32 GPUs for the word-level LM used by Peters et al. Code and models are available at github.com/zalandoresearch/flair.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/)

## Cited by

Cited by

466 papers of our library cite it (of the 140673 whose bibliographies we read); 21 of them have a page here, 445 not yet.

- [FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP](https://md.research.elamur.ai/free/en/papers/akbik-2019-flair-easy-use-framework-state/)
- [Pooled Contextualized Embeddings for Named Entity Recognition](https://md.research.elamur.ai/free/en/papers/akbik-2019-pooled-contextualized-embeddings-named-entity/)
- [AraCOVID19-MFH: Arabic COVID-19 Multi-label Fake News and Hate Speech Detection Dataset](https://md.research.elamur.ai/free/en/papers/ameur-2021-aracovid19-mfh-arabic-covid-19/)
- [AraCOVID19-SSD: Arabic COVID-19 Sentiment and Sarcasm Detection Dataset](https://md.research.elamur.ai/free/en/papers/ameur-2021-aracovid19-ssd-arabic-covid-19/)
- [Universal Dependencies for Sindhi](https://md.research.elamur.ai/free/en/papers/bauer-2025-universal-dependencies-sindhi/)
- [German's Next Language Model](https://md.research.elamur.ai/free/en/papers/chan-2020-german-s-next-language-model/)
- [Acquiring a Formality-Informed Lexical Resource for Style Analysis](https://md.research.elamur.ai/free/en/papers/eder-2021-acquiring-formality-informed-lexical-resource/)
- [Probing for idiomaticity in vector space models](https://md.research.elamur.ai/free/en/papers/garcia-2021-probing-idiomaticity-vector-space-models/)
- [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://md.research.elamur.ai/free/en/papers/hedderich-2021-survey-recent-approaches-natural-language/)
- [Automatic Dialect Density Estimation for African American English](https://md.research.elamur.ai/free/en/papers/johnson-2022-automatic-dialect-density-estimation-african/)
- [Whit's the Richt Pairt o Speech: PoS tagging for Scots](https://md.research.elamur.ai/free/en/papers/lameris-2021-whit-s-richt-pairt-o/)
- [CamemBERT: a Tasty French Language Model](https://md.research.elamur.ai/free/en/papers/martin-2019-camembert-tasty-french-language-model/)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/)
- [Pre-trained Models for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/qiu-2020-pre-trained-models-natural-language/)
- [From sunblock to softblock: Analyzing the correlates of neology in published writing and on social media](https://md.research.elamur.ai/free/en/papers/ryskina-2026-sunblock-softblock-analyzing-correlates-neology/)
- [Neural Architectures for Nested NER through Linearization](https://md.research.elamur.ai/free/en/papers/strakova-2019-neural-architectures-nested-ner-through/)
- [A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages](https://md.research.elamur.ai/free/en/papers/suarez-2020-monolingual-approach-contextualized-word-embeddings/)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://md.research.elamur.ai/free/en/papers/ud-2020-qi-stanza-python-natural-language-processing/)
- [Introducing various Semantic Models for Amharic: Experimentation and Evaluation with multiple Tasks and Datasets](https://md.research.elamur.ai/free/en/papers/yimam-2020-introducing-various-semantic-models-amharic/)
- [Named Entity Recognition as Dependency Parsing](https://md.research.elamur.ai/free/en/papers/yu-2020-named-entity-recognition-as-dependency/)
- [GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer](https://md.research.elamur.ai/free/en/papers/zaratiana-2024-gliner-generalist-model-named-entity/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{akbik2018contextual,
  title = {{Contextual String Embeddings for Sequence Labeling}},
  author = {Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {Proceedings of the 27th International Conference on Computational Linguistics},
  year = {2018},
  url = {https://aclanthology.org/C18-1139/}
}
```

- [← previous Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations](https://md.research.elamur.ai/free/en/papers/kobayashi-2018-contextual-augmentation-data-augmentation-words/)
- [next → Continual Lifelong Learning with Neural Networks: A Review](https://md.research.elamur.ai/free/en/papers/parisi-2018-continual-lifelong-learning-neural-networks/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/)

## References

33 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2013. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005. [find](https://scholar.google.com/scholar?q=Ciprian+Chelba%2C+Tomas+Mikolov%2C+Mike+Schuster%2C+Qi+Ge%2C+Thorsten+Brants%2C+Phillipp+Koehn%2C+and+Tony+Robinson%2E+2013%2E+One+billion+word+benchmark+for+measuring+progress+in+statistical+language+modeling%2E+arXiv) [our card](https://md.research.elamur.ai/free/en/papers/chelba-2013-one-billion-word-benchmark-measuring/)
2. Jinho D Choi. 2016. Dynamic feature induction: The last gist to the state-of-the-art. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 271–281. [find](https://scholar.google.com/scholar?q=Jinho+D+Choi%2E+2016%2E+Dynamic+feature+induction%3A+The+last+gist+to+the+state%2Dof%2Dthe%2Dart%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguisti)
3. Michael Collins. 2002. Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10, pages 1–8. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+2002%2E+Discriminative+training+methods+for+hidden+markov+models%3A+Theory+and+experiments+with+perceptron+algorithms%2E+In+Proceedings+of+the+ACL%2D02+conference+on+Empirical+methods+in+natu) [our card](https://md.research.elamur.ai/free/en/papers/collins-2002-discriminative-training-methods-hidden-markov/)
4. Dan Gillick, Cliff Brunk, Oriol Vinyals, and Amarnag Subramanya. 2015. Multilingual language processing from bytes. arXiv preprint arXiv:1512.00103. [find](https://scholar.google.com/scholar?q=Dan+Gillick%2C+Cliff+Brunk%2C+Oriol+Vinyals%2C+and+Amarnag+Subramanya%2E+2015%2E+Multilingual+language+processing+from+bytes%2E+arXiv+preprint+arXiv%3A1512%2E00103%2E)
5. Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. Learning word vectors for 157 languages. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018). [find](https://scholar.google.com/scholar?q=Edouard+Grave%2C+Piotr+Bojanowski%2C+Prakhar+Gupta%2C+Armand+Joulin%2C+and+Tomas+Mikolov%2E+2018%2E+Learning+word+vectors+for+157+languages%2E+In+Proceedings+of+the+International+Conference+on+Language+Resources+an) [our card](https://md.research.elamur.ai/free/en/papers/grave-2018-learning-word-vectors-157-languages/)
6. Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850. [find](https://scholar.google.com/scholar?q=Alex+Graves%2E+2013%2E+Generating+sequences+with+recurrent+neural+networks%2E+arXiv+preprint+arXiv%3A1308%2E0850%2E)
7. Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, and Richard Socher. 2016. A joint many-task model: Growing a neural network for multiple nlp tasks. arXiv preprint arXiv:1611.01587. [find](https://scholar.google.com/scholar?q=Kazuma+Hashimoto%2C+Caiming+Xiong%2C+Yoshimasa+Tsuruoka%2C+and+Richard+Socher%2E+2016%2E+A+joint+many%2Dtask+model%3A+Growing+a+neural+network+for+multiple+nlp+tasks%2E+arXiv+preprint+arXiv%3A1611%2E01587%2E) [our card](https://md.research.elamur.ai/free/en/papers/hashimoto-2016-joint-many-task-model-growing/)
8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%931780%2E)
9. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991. [https://github.com/zalandoresearch/flair](https://github.com/zalandoresearch/flair) [↗](https://github.com/zalandoresearch/flair) [our card](https://md.research.elamur.ai/free/en/papers/huang-2015-bidirectional-lstm-crf-models-sequence/)
10. Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. 2016. Exploring the limits of language modeling. arXiv preprint arXiv:1602.02410. [find](https://scholar.google.com/scholar?q=Rafal+Jozefowicz%2C+Oriol+Vinyals%2C+Mike+Schuster%2C+Noam+Shazeer%2C+and+Yonghui+Wu%2E+2016%2E+Exploring+the+limits+of+language+modeling%2E+arXiv+preprint+arXiv%3A1602%2E02410%2E)
11. Andrej Karpathy, Justin Johnson, and Li Fei-Fei. 2015. Visualizing and understanding recurrent networks. arXiv preprint arXiv:1506.02078. [find](https://scholar.google.com/scholar?q=Andrej+Karpathy%2C+Justin+Johnson%2C+and+Li+Fei%2DFei%2E+2015%2E+Visualizing+and+understanding+recurrent+networks%2E+arXiv+preprint+arXiv%3A1506%2E02078%2E) [our card](https://md.research.elamur.ai/free/en/papers/karpathy-2015-visualizing-understanding-recurrent-networks/)
12. Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. 2015. Character-aware neural language models. arXiv preprint arXiv:1508.06615. [find](https://scholar.google.com/scholar?q=Yoon+Kim%2C+Yacine+Jernite%2C+David+Sontag%2C+and+Alexander+M+Rush%2E+2015%2E+Character%2Daware+neural+language+models%2E+arXiv+preprint+arXiv%3A1508%2E06615%2E) [our card](https://md.research.elamur.ai/free/en/papers/kim-2015-character-aware-neural-language-models/)
13. Alexandros Komninos and Suresh Manandhar. 2016. Dependency based embeddings for sentence classification tasks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1490–1500. [find](https://scholar.google.com/scholar?q=Alexandros+Komninos+and+Suresh+Manandhar%2E+2016%2E+Dependency+based+embeddings+for+sentence+classification+tasks%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+Association+fo)
14. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360. [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Miguel+Ballesteros%2C+Sandeep+Subramanian%2C+Kazuya+Kawakami%2C+and+Chris+Dyer%2E+2016%2E+Neural+architectures+for+named+entity+recognition%2E+arXiv+preprint+arXiv%3A1603%2E01360%2E)
15. Liyuan Liu, Jingbo Shang, Frank Xu, Xiang Ren, Huan Gui, Jian Peng, and Jiawei Han. 2017. Empower sequence labeling with task-aware neural language model. arXiv preprint arXiv:1709.04109. [find](https://scholar.google.com/scholar?q=Liyuan+Liu%2C+Jingbo+Shang%2C+Frank+Xu%2C+Xiang+Ren%2C+Huan+Gui%2C+Jian+Peng%2C+and+Jiawei+Han%2E+2017%2E+Empower+sequence+labeling+with+task%2Daware+neural+language+model%2E+arXiv+preprint+arXiv%3A1709%2E04109%2E) [our card](https://md.research.elamur.ai/free/en/papers/liu-2017-empower-sequence-labeling-task-aware/)
16. Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional LSTMs-CNNs-CRF. arXiv preprint arXiv:1603.01354. [find](https://scholar.google.com/scholar?q=Xuezhe+Ma+and+Eduard+Hovy%2E+2016%2E+End%2Dto%2Dend+sequence+labeling+via+bi%2Ddirectional+LSTMs%2DCNNs%2DCRF%2E+arXiv+preprint+arXiv%3A1603%2E01354%2E) [our card](https://md.research.elamur.ai/free/en/papers/ma-2016-end-end-sequence-labeling-bi/)
17. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60. [find](https://scholar.google.com/scholar?q=Christopher+Manning%2C+Mihai+Surdeanu%2C+John+Bauer%2C+Jenny+Finkel%2C+Steven+Bethard%2C+and+David+McClosky%2E+2014%2E+The+Stanford+CoreNLP+natural+language+processing+toolkit%2E+In+Association+for+Computational+Ling) [our card](https://md.research.elamur.ai/free/en/papers/manning-2014-stanford-corenlp-natural-language-processing/)
18. Scott Menard. 2018. Applied logistic regression analysis, volume 106. SAGE publications. [find](https://scholar.google.com/scholar?q=Scott+Menard%2E+2018%2E+Applied+logistic+regression+analysis%2C+volume+106%2E+SAGE+publications%2E)
19. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+S+Corrado%2C+and+Jeff+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Advances+in+neural+information+processing+)
20. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Proceedings+of+the+2014+conference+on+empirical+methods+in+natural+language+process)
21. Matthew Peters, Waleed Ammar, Chandra Bhagavatula, and Russell Power. 2017. Semi-supervised sequence tagging with bidirectional language models. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1756–1765, Vancouver, Canada, Jul… [find](https://scholar.google.com/scholar?q=Matthew+Peters%2C+Waleed+Ammar%2C+Chandra+Bhagavatula%2C+and+Russell+Power%2E+2017%2E+Semi%2Dsupervised+sequence+tagging+with+bidirectional+language+models%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Associ) [our card](https://md.research.elamur.ai/free/en/papers/peters-2017-semi-supervised-sequence-tagging-bidirectional/)
22. Matthew Peters, Mark Neumann, and Christopher Clark Kenton Lee Luke Zettlemoyer Mohit Iyyer, Matt Gardner. 2018. Deep contextualized word representations. 6th International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Matthew+Peters%2C+Mark+Neumann%2C+and+Christopher+Clark+Kenton+Lee+Luke+Zettlemoyer+Mohit+Iyyer%2C+Matt+Gardner%2E+2018%2E+Deep+contextualized+word+representations%2E+6th+International+Conference+on+Learning+Repr)
23. Yanjun Qi, Ronan Collobert, Pavel Kuksa, Koray Kavukcuoglu, and Jason Weston. 2009. Combining labeled and unlabeled data with word-class distribution learning. In Proceedings of the 18th ACM conference on Information and knowledge management, pages 1737–1740. ACM. [find](https://scholar.google.com/scholar?q=Yanjun+Qi%2C+Ronan+Collobert%2C+Pavel+Kuksa%2C+Koray+Kavukcuoglu%2C+and+Jason+Weston%2E+2009%2E+Combining+labeled+and+unlabeled+data+with+word%2Dclass+distribution+learning%2E+In+Proceedings+of+the+18th+ACM+conferenc)
24. Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. 2017. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Rafal+Jozefowicz%2C+and+Ilya+Sutskever%2E+2017%2E+Learning+to+generate+reviews+and+discovering+sentiment%2E+arXiv+preprint+arXiv%3A1704%2E01444%2E) [our card](https://md.research.elamur.ai/free/en/papers/radford-2017-learning-generate-reviews-discovering-sentiment/)
25. Nils Reimers and Iryna Gurevych. 2017. Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 338–348, Copenhagen, Denmark, 09. [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2017%2E+Reporting+Score+Distributions+Makes+a+Difference%3A+Performance+Study+of+LSTM%2Dnetworks+for+Sequence+Tagging%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Met) [our card](https://md.research.elamur.ai/free/en/papers/reimers-2017-reporting-score-distributions-makes-difference/)
26. Ronald Rosenfeld. 2000. Two decades of statistical language modeling: Where do we go from here? Proceedings of the IEEE, 88(8):1270–1278. [find](https://scholar.google.com/scholar?q=Ronald+Rosenfeld%2E+2000%2E+Two+decades+of+statistical+language+modeling%3A+Where+do+we+go+from+here%3F+Proceedings+of+the+IEEE%2C+88%288%29%3A1270%E2%80%931278%2E)
27. Dominic Seyler, Tatiana Dembelova, Luciano Del Corro, Johannes Hoffart, and Gerhard Weikum. 2017. Knowner: Incremental multilingual knowledge in named entity recognition. arXiv preprint arXiv:1709.03544. — [arXiv:1709.03544](https://arxiv.org/abs/1709.03544) [find](https://scholar.google.com/scholar?q=Dominic+Seyler%2C+Tatiana+Dembelova%2C+Luciano+Del+Corro%2C+Johannes+Hoffart%2C+and+Gerhard+Weikum%2E+2017%2E+Knowner%3A+Incremental+multilingual+knowledge+in+named+entity+recognition%2E+arXiv+preprint+arXiv%3A1709%2E035)
28. Anders Søgaard and Yoav Goldberg. 2016. Deep multi-task learning with low level tasks supervised at lower layers. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 231–235. [find](https://scholar.google.com/scholar?q=Anders+S%C3%B8gaard+and+Yoav+Goldberg%2E+2016%2E+Deep+multi%2Dtask+learning+with+low+level+tasks+supervised+at+lower+layers%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguis) [our card](https://md.research.elamur.ai/free/en/papers/sogaard-2016-deep-multi-task-learning-low/)
29. Ilya Sutskever, James Martens, and Geoffrey E Hinton. 2011. Generating text with recurrent neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 1017–1024. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+James+Martens%2C+and+Geoffrey+E+Hinton%2E+2011%2E+Generating+text+with+recurrent+neural+networks%2E+In+Proceedings+of+the+28th+International+Conference+on+Machine+Learning+%28ICML%2D11%29%2C+pages+101)
30. Ilya Sutskever, Oriol Vinyals, and Quoc Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+neural+information+processing+systems%2C+pages+3104%E2%80%933112%2E)
31. Erik F Tjong Kim Sang and Sabine Buchholz. 2000. Introduction to the CoNLL-2000 shared task: Chunking. In Proceedings of the 2nd workshop on Learning language in logic and the 4th conference on Computational natural language learning-Volume 7, pages 127–132. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Erik+F+Tjong+Kim+Sang+and+Sabine+Buchholz%2E+2000%2E+Introduction+to+the+CoNLL%2D2000+shared+task%3A+Chunking%2E+In+Proceedings+of+the+2nd+workshop+on+Learning+language+in+logic+and+the+4th+conference+on+Comput) [our card](https://md.research.elamur.ai/free/en/papers/sang-2000-introduction-conll-2000-shared-task/)
32. Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4, pages 142–147. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Erik+F+Tjong+Kim+Sang+and+Fien+De+Meulder%2E+2003%2E+Introduction+to+the+CoNLL%2D2003+shared+task%3A+Language%2Dindependent+named+entity+recognition%2E+In+Proceedings+of+the+seventh+conference+on+Natural+language)
33. Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329. [find](https://scholar.google.com/scholar?q=Wojciech+Zaremba%2C+Ilya+Sutskever%2C+and+Oriol+Vinyals%2E+2014%2E+Recurrent+neural+network+regularization%2E+arXiv+preprint+arXiv%3A1409%2E2329%2E) [our card](https://md.research.elamur.ai/free/en/papers/zaremba-2014-recurrent-neural-network-regularization/)