---
title: "Nematus: a Toolkit for Neural Machine — Sennrich 2017"
description: "Nematus is an open-source toolkit for Neural Machine Translation (NMT) designed for strong translation quality, ease of use, and easy extension."
authors: ["Rico Sennrich", "Orhan Firat", "Kyunghyun Cho", "Alexandra Birch", "Barry Haddow", "Julian Hitschler", "Marcin Junczys-Dowmunt", "Samuel Läubli", "Antonio Valerio Miceli Barone", "Jozef Mokry", "Maria Nădejde"]
year: 2017
venue: "Proceedings of the Software Demonstrations of the 15th Conference of the European Chapter of the Association for Computational Linguistics"
source: "https://aclanthology.org/E17-3017/"
arxiv: "1703.04357"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sennrich-2017-nematus-toolkit-neural-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sennrich-2017-nematus-toolkit-neural-machine-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/) / Nematus: a Toolkit for Neural Machine Translation

# Nematus: a Toolkit for Neural Machine Translation

**Authors:** [Rico Sennrich](https://md.research.elamur.ai/free/en/authors/rico-sennrich/), [Orhan Firat](https://md.research.elamur.ai/free/en/authors/orhan-firat/), [Kyunghyun Cho](https://md.research.elamur.ai/free/en/authors/kyunghyun-cho/), [Alexandra Birch](https://md.research.elamur.ai/free/en/authors/alexandra-birch/), [Barry Haddow](https://md.research.elamur.ai/free/en/authors/barry-haddow/), Julian Hitschler, [Marcin Junczys-Dowmunt](https://md.research.elamur.ai/free/en/authors/marcin-junczys-dowmunt/), [Samuel Läubli](https://md.research.elamur.ai/free/en/authors/samuel-laubli/), [Antonio Valerio Miceli Barone](https://md.research.elamur.ai/free/en/authors/antonio-valerio-miceli-barone/), Jozef Mokry, Maria Nădejde · **Year:** 2017 · **Venue:** Proceedings of the Software Demonstrations of the 15th Conference of the European Chapter of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/E17-3017/) · [PDF](https://aclanthology.org/E17-3017.pdf) · [arXiv 1703.04357](https://arxiv.org/abs/1703.04357) · [OpenAlex](https://openalex.org/W2952688536)

**Citations:** 13 citations (openalex-snapshot · 2026-10-14) · [189](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Nematus is an open-source toolkit for Neural Machine Translation (NMT) designed for strong translation quality, ease of use, and easy extension. It grew out of the compact dl4mt-tutorial codebase, is written in Python on top of Theano, and follows the attentional encoder–decoder approach of Bahdanau et al. (2015) with several deliberate changes. The decoder starts from the averaged source annotations instead of the final backward encoder state. It uses a new conditional GRU with attention (cGRU_att), in which two GRU transitions surround the attention step. The decoding order is Look, Update, Generate, which simplifies the code. Training uses cross-entropy with several optimizers and also supports minimum risk training toward sentence-level MT metrics. The toolkit adds ensembling, rescoring, and visualisation tools. The paper reports no new benchmark scores. Its value is that the same codebase produced top WMT and IWSLT 2016 submissions and production systems, such as at WIPO, so its design choices matter to both researchers and practitioners.

## Key points
- Architecture changes from Bahdanau et al. (2015) include:
  - a tanh feedforward layer instead of maxout before the softmax;
  - no biases in the embedding layers;
  - optional recurrent Bayesian dropout;
  - factored inputs, with per-feature embeddings concatenated;
  - optional tied embedding matrices.
- In cGRU_att, recurrence happens only at the level of the whole layer, making it a deep-transition RNN rather than a stacked one.
- Supported optimizers are SGD, Adadelta, RmsProp, and Adam. Minimum risk training (MRT) can use smoothed sentence BLEU, METEOR, BEER, or mixtures of these, and early stopping is available.
- Built-in tools cover ensemble decoding (geometric mean of model distributions), beam search, corpus scoring, n-best rescoring, and visualisation of attention weights and beam search graphs.
- The code is available on GitHub (rsennrich/nematus) under a BSD license.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cited by

Cited by

189 papers of our library cite it (of the 140673 whose bibliographies we read); 18 of them have a page here, 171 not yet.

- [Towards String-To-Tree Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/aharoni-2017-string-tree-neural-machine-translation/)
- [Evaluating Discourse Phenomena in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/bawden-2017-evaluating-discourse-phenomena-neural-machine/)
- [Findings of the 2018 Conference on Machine Translation (WMT18)](https://md.research.elamur.ai/free/en/papers/bojar-2018-findings-2018-conference-machine-translation/)
- [Findings of the WMT 2019 Shared Task on Automatic Post-Editing](https://md.research.elamur.ai/free/en/papers/chatterjee-2019-findings-wmt-2019-shared-task/)
- [Lemmatization as a Classification Task: Results from Arabic across Multiple Genres](https://md.research.elamur.ai/free/en/papers/ctx-2025-saeed-lemmatization-as-classification-task-results/)
- [Copied Monolingual Data Improves Low-Resource Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/currey-2017-copied-monolingual-data-improves-low/)
- [Stronger Baselines for Trustable Results in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/denkowski-2017-stronger-baselines-trustable-results-neural/)
- [Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description](https://md.research.elamur.ai/free/en/papers/elliott-2017-findings-second-shared-task-multimodal/)
- [The WebNLG Challenge: Generating Text from RDF Data](https://md.research.elamur.ai/free/en/papers/gardent-2017-webnlg-challenge-generating-text-rdf/)
- [Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings](https://md.research.elamur.ai/free/en/papers/gonzales-2017-improving-word-sense-disambiguation-neural/)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://md.research.elamur.ai/free/en/papers/iyyer-2018-adversarial-example-generation-syntactically-controlled/)
- [Marian: Fast Neural Machine Translation in C++](https://md.research.elamur.ai/free/en/papers/junczys-2018-dowmunt-marian-fast-neural-machine-translation/)
- [The IIT Bombay English-Hindi Parallel Corpus](https://md.research.elamur.ai/free/en/papers/kunchukuttan-2018-iit-bombay-english-hindi-parallel/)
- [Learning How to Simplify From Explicit Labeling of Complex-Simplified Text Pairs](https://md.research.elamur.ai/free/en/papers/manchego-2017-learning-how-simplify-explicit-labeling/)
- [Data-Driven Sentence Simplification: Survey and Benchmark](https://md.research.elamur.ai/free/en/papers/manchego-2020-data-driven-sentence-simplification-survey/)
- [A Call for Clarity in Reporting BLEU Scores](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/)
- [The University of Edinburgh's Neural MT Systems for WMT17](https://md.research.elamur.ai/free/en/papers/sennrich-2017-university-edinburgh-s-neural-mt/)
- [Coreference and Coherence in Neural Machine Translation: A Study Using Oracle Experiments](https://md.research.elamur.ai/free/en/papers/stojanovski-2018-coreference-coherence-neural-machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sennrich2017nematus,
  title = {{Nematus: a Toolkit for Neural Machine Translation}},
  author = {Sennrich, Rico and Firat, Orhan and Cho, Kyunghyun and Birch, Alexandra and Haddow, Barry and Hitschler, Julian and Junczys-Dowmunt, Marcin and Läubli, Samuel and Barone, Antonio Valerio Miceli and Mokry, Jozef and Nădejde, Maria},
  booktitle = {Proceedings of the Software Demonstrations of the 15th Conference of the European Chapter of the Association for Computational Linguistics},
  year = {2017},
  eprint = {1703.04357},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1703.04357}
}
```

- [← previous Multi-task Attention-based Neural Networks for Implicit Discourse Relationship Representation and Identification](https://md.research.elamur.ai/free/en/papers/lan-2017-multi-task-attention-based-neural/)
- [next → Neural AMR: Sequence-to-Sequence Models for Parsing and Generation](https://md.research.elamur.ai/free/en/papers/konstas-2017-neural-amr-sequence-sequence-models/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/)

## References

23 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In Proceedings of the International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Neural+Machine+Translation+by+Jointly+Learning+to+Align+and+Translate%2E+In+Proceedings+of+the+International+Conference+on+Learning+Representati) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
2. Boxing Chen and Colin Cherry. 2014. A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 362–367, Baltimore, Maryland, USA. 4 Pre-trained models for 8 translation directions are available at http://sta… [find](https://scholar.google.com/scholar?q=Boxing+Chen+and+Colin+Cherry%2E+2014%2E+A+Systematic+Comparison+of+Smoothing+Techniques+for+Sentence%2DLevel+BLEU%2E+In+Proceedings+of+the+Ninth+Workshop+on+Statistical+Machine+Translation%2C+pages+362%E2%80%93367%2C+Bal) [our card](https://md.research.elamur.ai/free/en/papers/chen-2014-systematic-comparison-smoothing-techniques-sentence/)
3. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany. [find](https://scholar.google.com/scholar?q=Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers%29%2C+Berlin%2C+Germany%2E)
4. Michael Denkowski and Alon Lavie. 2011. Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 85–91, Edinburgh, Scotland. [find](https://scholar.google.com/scholar?q=Michael+Denkowski+and+Alon+Lavie%2E+2011%2E+Meteor+1%2E3%3A+Automatic+Metric+for+Reliable+Optimization+and+Evaluation+of+Machine+Translation+Systems%2E+In+Proceedings+of+the+Sixth+Workshop+on+Statistical+Machin) [our card](https://md.research.elamur.ai/free/en/papers/denkowski-2011-meteor-1-3-automatic-metric/)
5. Milos Stanojevic and Khalil Sima’an. 2014. BEER: BEtter Evaluation as Ranking. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 414–419, Baltimore, Maryland, USA. [find](https://scholar.google.com/scholar?q=Milos+Stanojevic+and+Khalil+Sima%E2%80%99an%2E+2014%2E+BEER%3A+BEtter+Evaluation+as+Ranking%2E+In+Proceedings+of+the+Ninth+Workshop+on+Statistical+Machine+Translation%2C+pages+414%E2%80%93419%2C+Baltimore%2C+Maryland%2C+USA%2E)
6. Salah El Hihi and Yoshua Bengio. 1995. Hierarchical Recurrent Neural Networks for Long-Term Dependencies. In Nips, volume 409. [find](https://scholar.google.com/scholar?q=Salah+El+Hihi+and+Yoshua+Bengio%2E+1995%2E+Hierarchical+Recurrent+Neural+Networks+for+Long%2DTerm+Dependencies%2E+In+Nips%2C+volume+409%2E)
7. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, pages 3104–3112, Montreal, Quebec, Canada. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V%2E+Le%2E+2014%2E+Sequence+to+Sequence+Learning+with+Neural+Networks%2E+In+Advances+in+Neural+Information+Processing+Systems+27%3A+Annual+Conference+on+Neural+Informatio)
8. Yarin Gal. 2015. A Theoretically Grounded Application of Dropout in Recurrent Neural Networks. ArXiv e-prints. [find](https://scholar.google.com/scholar?q=Yarin+Gal%2E+2015%2E+A+Theoretically+Grounded+Application+of+Dropout+in+Recurrent+Neural+Networks%2E+ArXiv+e%2Dprints%2E) [our card](https://md.research.elamur.ai/free/en/papers/gal-2015-theoretically-grounded-application-dropout-recurrent/)
9. Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850. [find](https://scholar.google.com/scholar?q=Alex+Graves%2E+2013%2E+Generating+sequences+with+recurrent+neural+networks%2E+arXiv+preprint+arXiv%3A1308%2E0850%2E)
10. Theano Development Team. 2016. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints, abs/1605.02688. [find](https://scholar.google.com/scholar?q=Theano+Development+Team%2E+2016%2E+Theano%3A+A+Python+framework+for+fast+computation+of+mathematical+expressions%2E+arXiv+e%2Dprints%2C+abs%2F1605%2E02688%2E) [our card](https://md.research.elamur.ai/free/en/papers/the-2016-theano-development-team-theano-python-framework-fast-computation/)
11. Hakan Inan, Khashayar Khosravi, and Richard Socher. 2016. Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling. CoRR, abs/1611.01462. [find](https://scholar.google.com/scholar?q=Hakan+Inan%2C+Khashayar+Khosravi%2C+and+Richard+Socher%2E+2016%2E+Tying+Word+Vectors+and+Word+Classifiers%3A+A+Loss+Framework+for+Language+Modeling%2E+CoRR%2C+abs%2F1611%2E01462%2E)
12. Tijmen Tieleman and Geoffrey Hinton. 2012. Lecture 6.5 - rmsprop. [find](https://scholar.google.com/scholar?q=Tijmen+Tieleman+and+Geoffrey+Hinton%2E+2012%2E+Lecture+6%2E5+%2D+rmsprop%2E)
13. WIPO. 2016. WIPO Develops Cutting-Edge Translation Tool For Patent Documents, Oct. [http://www.wipo.int/pressroom/en/](http://www.wipo.int/pressroom/en/) articles/2016/article\\_0014.html. [↗](http://www.wipo.int/pressroom/en/)
14. Marcin Junczys-Dowmunt and Alexandra Birch. 2016. The University of Edinburgh’s systems submission to the MT task at IWSLT. In The International Workshop on Spoken Language Translation (IWSLT), Seattle, USA. [find](https://scholar.google.com/scholar?q=Marcin+Junczys%2DDowmunt+and+Alexandra+Birch%2E+2016%2E+The+University+of+Edinburgh%E2%80%99s+systems+submission+to+the+MT+task+at+IWSLT%2E+In+The+International+Workshop+on+Spoken+Language+Translation+%28IWSLT%29%2C+Seattl)
15. Matthew D Zeiler. 2012. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. [find](https://scholar.google.com/scholar?q=Matthew+D+Zeiler%2E+2012%2E+ADADELTA%3A+an+adaptive+learning+rate+method%2E+arXiv+preprint+arXiv%3A1212%2E5701%2E) [our card](https://md.research.elamur.ai/free/en/papers/zeiler-2012-adadelta-adaptive-learning-rate-method/)
16. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [find](https://scholar.google.com/scholar?q=Diederik+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2E)
17. Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, and Jürgen Schmidhuber. 2016. Recurrent highway networks. arXiv preprint arXiv:1607.03474. [find](https://scholar.google.com/scholar?q=Julian+Georg+Zilly%2C+Rupesh+Kumar+Srivastava%2C+Jan+Koutn%C3%ADk%2C+and+J%C3%BCrgen+Schmidhuber%2E+2016%2E+Recurrent+highway+networks%2E+arXiv+preprint+arXiv%3A1607%2E03474%2E)
18. Razvan Pascanu, Çağlar Gülçehre, Kyunghyun Cho, and Yoshua Bengio. 2014. How to Construct Deep Recurrent Neural Networks. In International Conference on Learning Representations 2014 (Conference Track). [find](https://scholar.google.com/scholar?q=Razvan+Pascanu%2C+%C3%87ag%CC%86lar+G%C3%BCl%C3%A7ehre%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+How+to+Construct+Deep+Recurrent+Neural+Networks%2E+In+International+Conference+on+Learning+Representations+2014+%28Conference+Trac)
19. Ofir Press and Lior Wolf. 2017. Using the Output Embedding to Improve Language Models. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL), Valencia, Spain. [find](https://scholar.google.com/scholar?q=Ofir+Press+and+Lior+Wolf%2E+2017%2E+Using+the+Output+Embedding+to+Improve+Language+Models%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics+%28E) [our card](https://md.research.elamur.ai/free/en/papers/press-2017-output-embedding-improve-language-models/)
20. Jürgen Schmidhuber. 1992. Learning complex, extended sequences using the principle of history compression. Neural Computation, 4(2):234–242. [find](https://scholar.google.com/scholar?q=J%C3%BCrgen+Schmidhuber%2E+1992%2E+Learning+complex%2C+extended+sequences+using+the+principle+of+history+compression%2E+Neural+Computation%2C+4%282%29%3A234%E2%80%93242%2E)
21. Rico Sennrich and Barry Haddow. 2016. Linguistic Input Features Improve Neural Machine Translation. In Proceedings of the First Conference on Machine Translation, Volume 1: Research Papers, pages 83– 91, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Rico+Sennrich+and+Barry+Haddow%2E+2016%2E+Linguistic+Input+Features+Improve+Neural+Machine+Translation%2E+In+Proceedings+of+the+First+Conference+on+Machine+Translation%2C+Volume+1%3A+Research+Papers%2C+pages+83%E2%80%93+) [our card](https://md.research.elamur.ai/free/en/papers/sennrich-2016-linguistic-input-features-improve-neural/)
22. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Edinburgh Neural Machine Translation Systems for WMT 16. In Proceedings of the First Conference on Machine Translation, Volume 2: Shared Task Papers, pages 368–373, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Edinburgh+Neural+Machine+Translation+Systems+for+WMT+16%2E+In+Proceedings+of+the+First+Conference+on+Machine+Translation%2C+Volume+2%3A+Shared+Task+Pa) [our card](https://md.research.elamur.ai/free/en/papers/sennrich-2016-edinburgh-neural-machine-translation-systems/)
23. Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Minimum Risk Training for Neural Machine Translation. In [find](https://scholar.google.com/scholar?q=Shiqi+Shen%2C+Yong+Cheng%2C+Zhongjun+He%2C+Wei+He%2C+Hua+Wu%2C+Maosong+Sun%2C+and+Yang+Liu%2E+2016%2E+Minimum+Risk+Training+for+Neural+Machine+Translation%2E+In) [our card](https://md.research.elamur.ai/free/en/papers/shen-2015-minimum-risk-training-neural-machine/)