---
title: "Context Sensitive Neural Lemmatization with Lematus (2018)"
description: "Bergmanis and Goldwater present Lematus, which adapts the Nematus attentional encoder-decoder… 30 references, cited by 33 papers here."
authors: ["Toms Bergmanis", "Sharon Goldwater"]
year: 2018
venue: "Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)"
source: "https://aclanthology.org/N18-1126/"
doi: "10.18653/v1/n18-1126"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bergmanis-2018-context-sensitive-neural-lemmatization-lematus/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bergmanis-2018-context-sensitive-neural-lemmatization-lematus/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/) / Context Sensitive Neural Lemmatization with Lematus

# Context Sensitive Neural Lemmatization with Lematus

**Authors:** Toms Bergmanis and [Sharon Goldwater](https://md.research.elamur.ai/free/en/authors/sharon-goldwater/) · **Year:** 2018 · **Venue:** [Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2018](https://md.research.elamur.ai/free/en/venues/naacl/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N18-1126/) · [PDF](https://aclanthology.org/N18-1126.pdf) · [DOI 10.18653/v1/n18-1126](https://doi.org/10.18653/v1/n18-1126) · [OpenAlex](https://openalex.org/W2804387108)

**Citations:** 57 citations (OpenAlex, 2026-10-14) · [33](https://md.research.elamur.ai/free/en/papers/bergmanis-2018-context-sensitive-neural-lemmatization-lematus/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Lemmatizers must cope with two difficulties: wordforms never seen in training, and forms whose lemma depends on the sentence (e.g., Latvian "ceļu" may come from a verb or two nouns). Earlier context-aware systems rarely measured how much context actually contributes. Bergmanis and Goldwater present Lematus, which adapts the Nematus attentional encoder-decoder to generate lemmas character by character. Its input is the target word surrounded by a window of neighbouring characters. Unlike prior systems, it needs only lemmatized text, with no POS tags, edit trees or external embeddings. The authors test it on 20 typologically diverse UD treebanks, using both full training sets and a reduced setting of 10k tokens. They compare it against a context-free variant and three baselines: Morfette, Lemming and Ch-2017. Adding context consistently improves accuracy, mostly for ambiguous forms. The context-free model is already competitive, especially on unseen words. Lematus beats Lemming with full data and matches it with 10k tokens. The study suggests that context matters most for languages with high ambiguity.

## Key points
- **Data and setup:** UD Treebank v2.0 standard splits for 20 languages, with Dutch taken from v2.1. Each language is tested with full data and with only the first 10k training tokens.
- **Architecture:** A 2-layer bidirectional GRU encoder feeds a decoder built from a conditional GRU and a GRU, using 100 hidden units and 300-dimensional embeddings. The model uses 20 context characters on each side (Lematus 20-Ch) or none (0-Ch).
- **Effect of context (full data):** Accuracy on ambiguous words rises from 88.8% to 92.4%, while unseen words improve only from 83.6% to 84.3%. Even 0-Ch outperforms all baselines on unseen words.
- **Low-resource setting:** With 10k tokens, most systems lose 5.6–6.8 points. Ch-2017 loses about 10 points, and Lematus 20-Ch and Lemming are statistically equivalent.
- **Cross-lingual analysis:**
  - A higher share of unseen words predicts lower accuracy (rank R = −0.73).
  - Higher ambiguity is associated with higher accuracy (R = 0.44), likely because ambiguity and productivity are anti-correlated.
  - Training-set size does not significantly predict accuracy on unseen words (R = 0.32).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) — University, Edinburgh, United Kingdom · 359 works · 77,352 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧩 Morphology](https://md.research.elamur.ai/free/en/topics/morphology/) · 405 works

💡 Concepts (2): [💡 Lemma](https://md.research.elamur.ai/free/en/concepts/lemma/) · 24 works · [💡 Lemmatization](https://md.research.elamur.ai/free/en/concepts/lemmatization/) · 20 works

## Cite

Cite as: Toms Bergmanis and Sharon Goldwater. 2018. Context Sensitive Neural Lemmatization with Lematus. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{bergmanis2018context,
  title = {{Context Sensitive Neural Lemmatization with Lematus}},
  author = {Toms Bergmanis and Sharon Goldwater},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1126},
  url = {https://doi.org/10.18653/v1/n18-1126}
}
```

- [← previous ClaimRank: Detecting Check-Worthy Claims in Arabic and English](https://md.research.elamur.ai/free/en/papers/jaradat-2018-claimrank-detecting-check-worthy-claims/)
- [next → RankME: Reliable Human Ratings for Natural Language Generation](https://md.research.elamur.ai/free/en/papers/novikova-2018-rankme-reliable-human-ratings-natural/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/)

## References

30 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Roee Aharoni, Yoav Goldberg, and Yonatan Belinkov. 2016. Improving sequence to sequence learning for morphological inflection generation: The BIU-MIT systems for the SIGMORPHON 2016 shared task for morphological reinflection. In Proceedings of the 14th SIGMORPHON Workshop on Computational Research i… [find](https://scholar.google.com/scholar?q=Roee+Aharoni%2C+Yoav+Goldberg%2C+and+Yonatan+Belinkov%2E+2016%2E+Improving+sequence+to+sequence+learning+for+morphological+inflection+generation%3A+The+BIU%2DMIT+systems+for+the+SIGMORPHON+2016+shared+task+for+mo)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). In Proceedings of ICLR.
3. Toms Bergmanis, Katharina Kann, Hinrich Schütze, and Sharon Goldwater. 2017. Training data augmentation for low-resource morphological inflection. In Proceedings of the CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection. Association for Computational Linguistics, Vancouver, Can… [find](https://scholar.google.com/scholar?q=Toms+Bergmanis%2C+Katharina+Kann%2C+Hinrich+Schu%CC%88tze%2C+and+Sharon+Goldwater%2E+2017%2E+Training+data+augmentation+for+low%2Dresource+morphological+inflection%2E+In+Proceedings+of+the+CoNLL%2DSIGMORPHON+2017+Shared+T)
4. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. [Enriching word vectors with subword information](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/). Transactions of the Association for Computational Linguistics 5:135–146.
5. Abhisek Chakrabarty, Onkar Arun Pandit, and Utpal Garain. 2017. Context sensitive lemmatization using two successive bidirectional gated recurrent networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computation… [find](https://scholar.google.com/scholar?q=Abhisek+Chakrabarty%2C+Onkar+Arun+Pandit%2C+and+Utpal+Garain%2E+2017%2E+Context+sensitive+lemmatization+using+two+successive+bidirectional+gated+recurrent+networks%2E+In+Proceedings+of+the+55th+Annual+Meeting+o)
6. Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. [On the properties of neural machine translation: Encoder-decoder approaches](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/). In The Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation. Association for Computational Linguistics, Doha, Qatar.
7. Grzegorz Chrupała. 2006. Simple data-driven context-sensitive lemmatization. Procesamiento del Lenguaje Natural 37. [find](https://scholar.google.com/scholar?q=Grzegorz+Chrupa%C5%82a%2E+2006%2E+Simple+data%2Ddriven+context%2Dsensitive+lemmatization%2E+Procesamiento+del+Lenguaje+Natural+37%2E)
8. Grzegorz Chrupała, Georgiana Dinu, and Josef van Genabith. 2008. [Learning morphology with morfette](https://md.research.elamur.ai/free/en/papers/chrupala-2008-learning-morphology-morfette/). In Bente Maegaard Joseph Mariani Jan Odijk Stelios Piperidis Daniel Tapias Nicoletta Calzolari (Conference Chair), Khalid Choukri, editor, Proceedings of the Sixth International Conference on Language…
9. Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sandra Kübler, David Yarowsky, Jason Eisner, and Mans Hulden. 2017. [find](https://scholar.google.com/scholar?q=Ryan+Cotterell%2C+Christo+Kirov%2C+John+Sylak%2DGlassman%2C+Ge%CC%81raldine+Walther%2C+Ekaterina+Vylomova%2C+Patrick+Xia%2C+Manaal+Faruqui%2C+Sandra+Ku%CC%88bler%2C+David+Yarowsky%2C+Jason+Eisner%2C+and+Mans+Hulden%2E+2017%2E)
10. [Conll-sigmorphon 2017 shared task: Universal morphological reinflection in 52 languages](https://md.research.elamur.ai/free/en/papers/cotterell-2017-sigmorphon-reinflection-52/). In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection. Association for Computational Linguistics, Vancouver, pages 1–30.
11. Ryan Cotterell, Christo Kirov, John Sylak-Glassman, David Yarowsky, Jason Eisner, and Mans Hulden. 2016. [find](https://scholar.google.com/scholar?q=Ryan+Cotterell%2C+Christo+Kirov%2C+John+Sylak%2DGlassman%2C+David+Yarowsky%2C+Jason+Eisner%2C+and+Mans+Hulden%2E+2016%2E)
12. The SIGMORPHON 2016 shared taskmor-phological reinflection. In Proceedings of the 14th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology. pages 10–22. [find](https://scholar.google.com/scholar?q=The+SIGMORPHON+2016+shared+taskmor%2Dphological+reinflection%2E+In+Proceedings+of+the+14th+SIGMORPHON+Workshop+on+Computational+Research+in+Phonetics%2C+Phonology%2C+and+Morphology%2E+pages+10%E2%80%9322%2E)
13. Tomaž Erjavec and Sasčo Džeroski. 2004. Machine learning of morphosyntactic structure: Lemmatizing unknown slovene words. Applied Artificial Intelligence 18(1):17–41. [find](https://scholar.google.com/scholar?q=Tomaz%CC%8C+Erjavec+and+Sasc%CC%8Co+Dz%CC%8Ceroski%2E+2004%2E+Machine+learning+of+morphosyntactic+structure%3A+Lemmatizing+unknown+slovene+words%2E+Applied+Artificial+Intelligence+18%281%29%3A17%E2%80%9341%2E)
14. Nerea Ezeiza, Iñaki Alegria, José Marı́a Arriola, Rubén Urizar, and Itziar Aduriz. 1998. Combining stochastic and rule-based methods for disambiguation in agglutinative languages. In Proceedings of the 17th international conference on Computational linguistics-Volume 1. Association for Computatio… [find](https://scholar.google.com/scholar?q=Nerea+Ezeiza%2C+In%CC%83aki+Alegria%2C+Jose%CC%81+Mar%C4%B1%CC%81a+Arriola%2C+Rube%CC%81n+Urizar%2C+and+Itziar+Aduriz%2E+1998%2E+Combining+stochastic+and+rule%2Dbased+methods+for+disambiguation+in+agglutinative+languages%2E+In+Proceedings+of)
15. Philip Gage. 1994. A new algorithm for data compression. C Users J. 12(2):23–38. [find](https://scholar.google.com/scholar?q=Philip+Gage%2E+1994%2E+A+new+algorithm+for+data+compression%2E+C+Users+J%2E+12%282%29%3A23%E2%80%9338%2E)
16. Yarin Gal and Zoubin Ghahramani. 2016. [A theoretically grounded application of dropout in recurrent neural networks](https://md.research.elamur.ai/free/en/papers/gal-2015-theoretically-grounded-application-dropout-recurrent/). In Advances in neural information processing systems. pages 1019–1027.
17. Dilek Z. Hakkani-Tür, Kemal Oflazer, and Gökhan Tür. 2002. Statistical morphological disambiguation for agglutinative languages. Computers and the Humanities 36(4):381–410. [find](https://scholar.google.com/scholar?q=Dilek+Z%2E+Hakkani%2DTu%CC%88r%2C+Kemal+Oflazer%2C+and+Go%CC%88khan+Tu%CC%88r%2E+2002%2E+Statistical+morphological+disambiguation+for+agglutinative+languages%2E+Computers+and+the+Humanities+36%284%29%3A381%E2%80%93410%2E)
18. Sittichai Jiampojamarn, Colin Cherry, and Grzegorz Kondrak. 2008. Joint processing and discriminative training for letter-to-phoneme conversion. In Proceedings of ACL-08: HLT. Association for Computational Linguistics, Columbus, Ohio, pages 905– 913. [find](https://scholar.google.com/scholar?q=Sittichai+Jiampojamarn%2C+Colin+Cherry%2C+and+Grzegorz+Kondrak%2E+2008%2E+Joint+processing+and+discriminative+training+for+letter%2Dto%2Dphoneme+conversion%2E+In+Proceedings+of+ACL%2D08%3A+HLT%2E+Association+for+Computat)
19. Katharina Kann and Hinrich Schütze. 2016. MED: The LMU system for the SIGMORPHON 2016 shared task on morphological reinflection. In Proceedings of ACL. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Katharina+Kann+and+Hinrich+Schu%CC%88tze%2E+2016%2E+MED%3A+The+LMU+system+for+the+SIGMORPHON+2016+shared+task+on+morphological+reinflection%2E+In+Proceedings+of+ACL%2E+Association+for+Computational+Linguistics%2E)
20. Wang Ling, Chris Dyer, Alan W Black, Isabel Trancoso, Ramon Fermandez, Silvio Amir, Luis Marujo, and Tiago Luis. 2015. [Finding function in form: Compositional character models for open vocabulary word representation](https://md.research.elamur.ai/free/en/papers/ling-2015-finding-function-form-compositional-character/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Proc…
21. Thomas Müller, Ryan Cotterell, Alexander Fraser, and Hinrich Schütze. 2015. [Joint lemmatization and morphological tagging with lemming](https://md.research.elamur.ai/free/en/papers/muller-2015-joint-lemmatization-morphological-tagging-lemming/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 2268…
22. Joakim Nivre et al. 2017. Universal dependencies 2.0 CoNLL 2017 shared task development and test data. LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics, Charles University. [find](https://scholar.google.com/scholar?q=Joakim+Nivre+et+al%2E+2017%2E+Universal+dependencies+2%2E0+CoNLL+2017+shared+task+development+and+test+data%2E+LINDAT%2FCLARIN+digital+library+at+the+Institute+of+Formal+and+Applied+Linguistics%2C+Charles+Univers)
23. Kemal Oflazer and Ìlker Kuruöz. 1994. Tagging and morphological disambiguation of turkish text. In Proceedings of the fourth conference on Applied natural language processing. Association for Computational Linguistics, pages 144–149. [find](https://scholar.google.com/scholar?q=Kemal+Oflazer+and+I%CC%80lker+Kuruo%CC%88z%2E+1994%2E+Tagging+and+morphological+disambiguation+of+turkish+text%2E+In+Proceedings+of+the+fourth+conference+on+Applied+natural+language+processing%2E+Association+for+Comput)
24. Robert Östling. 2016. Morphological reinflection with convolutional neural networks. In Proceedings of the 14th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology. pages 23–26. [find](https://scholar.google.com/scholar?q=Robert+O%CC%88stling%2E+2016%2E+Morphological+reinflection+with+convolutional+neural+networks%2E+In+Proceedings+of+the+14th+SIGMORPHON+Workshop+on+Computational+Research+in+Phonetics%2C+Phonology%2C+and+Morphology%2E+)
25. Lutz Prechelt. 1998. Early stopping-but when? Neural Networks: Tricks of the trade pages 553–553. [find](https://scholar.google.com/scholar?q=Lutz+Prechelt%2E+1998%2E+Early+stopping%2Dbut+when%3F+Neural+Networks%3A+Tricks+of+the+trade+pages+553%E2%80%93553%2E)
26. Tim Salimans and Diederik P. Kingma. 2016. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. CoRR abs/1602.07868. [find](https://scholar.google.com/scholar?q=Tim+Salimans+and+Diederik+P%2E+Kingma%2E+2016%2E+Weight+normalization%3A+A+simple+reparameterization+to+accelerate+training+of+deep+neural+networks%2E+CoRR+abs%2F1602%2E07868%2E)
27. Rico Sennrich, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Läubli, Antonio Valerio Miceli Barone, Jozef Mokry, and Maria Nadejde. 2017. [Nematus: a toolkit for neural machine translation](https://md.research.elamur.ai/free/en/papers/sennrich-2017-nematus-toolkit-neural-machine-translation/). CoRR abs/1703.04357.
28. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, pages 1…
29. Clara Vania and Adam Lopez. 2017. From characters to words to in between: Do we capture morphology? In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, pages 2016–2027. [find](https://scholar.google.com/scholar?q=Clara+Vania+and+Adam+Lopez%2E+2017%2E+From+characters+to+words+to+in+between%3A+Do+we+capture+morphology%3F+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1)
30. Matthew D Zeiler. 2012. [ADADELTA: an adaptive learning rate method](https://md.research.elamur.ai/free/en/papers/zeiler-2012-adadelta-adaptive-learning-rate-method/). arXiv preprint arXiv:1212.5701 .

*Updated 29 October 2026*