--- title: "Linguistic Input Features Improve Neural — Sennrich 2016" description: "Neural machine translation (NMT) learns from raw parallel text, so it is unclear whether explicit linguistic annotation still helps." authors: ["Rico Sennrich", "Barry Haddow"] year: 2016 venue: "Proceedings of the First Conference on Machine Translation: Volume 1, Research Papers" source: "https://aclanthology.org/W16-2209/" doi: "10.18653/v1/w16-2209" arxiv: "1606.02892" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/sennrich-2016-linguistic-input-features-improve-neural/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sennrich-2016-linguistic-input-features-improve-neural/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/) / Linguistic Input Features Improve Neural Machine Translation # Linguistic Input Features Improve Neural Machine Translation **Authors:** [Rico Sennrich](https://md.research.elamur.ai/free/en/authors/rico-sennrich/), [Barry Haddow](https://md.research.elamur.ai/free/en/authors/barry-haddow/) · **Year:** 2016 · **Venue:** Proceedings of the First Conference on Machine Translation: Volume 1, Research Papers · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/W16-2209/) · [PDF](https://aclanthology.org/W16-2209.pdf) · [DOI 10.18653/v1/w16-2209](https://doi.org/10.18653/v1/w16-2209) · [arXiv 1606.02892](https://arxiv.org/abs/1606.02892) · [OpenAlex](https://openalex.org/W2410082850) **Citations:** 68 citations (openalex-snapshot · 2026-10-14) · [162](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Neural machine translation (NMT) learns from raw parallel text, so it is unclear whether explicit linguistic annotation still helps. Sennrich and Haddow test this by extending the encoder of an attentional encoder–decoder model (Bahdanau-style, built on the dl4mt code and released as Nematus) so that each source token can carry several features. Each feature gets its own embedding, and the embeddings are concatenated into a vector whose length equals the original embedding size. The rest of the network is unchanged. The source-side features are lemmas, subword-position tags, morphological descriptions, POS tags and dependency labels. All are predicted automatically and copied onto every BPE fragment of a word. The authors train English↔German systems on WMT16 data and English→Romanian systems, then compare perplexity, BLEU and CHRF3 against plain baselines. Adding the features generally lowers perplexity and raises translation scores, including when synthetic back-translated data is used. The work shows that annotation remains useful for strong neural models and offers a simple, general way to add it. ## Key points - **Data and setup:** - English↔German training used about 4.2M WMT16 sentence pairs with joint BPE (89,500 merges). - newstest2013 served for validation; newstest2015 and newstest2016 were the test sets. - **Annotation tools and subword tags:** - English input was annotated with Stanford CoreNLP. - German input was annotated with ParZu, with lemmas from Zmorge. - Subword tags mark each BPE unit as word-beginning (B), inside (I), end (E), or a complete word (O). - **English→German (parallel data only):** combining all features raised BLEU from 23.0 to 23.8 (test15) and from 27.8 to 28.4 (test16). In single-feature tests, POS tags gave the largest gain, reaching 24.0 and 28.9. - **With synthetic training data:** - German→English BLEU improved from 37.5 to 38.5 on newstest2016. - English→German BLEU barely changed, though perplexity still dropped (49.7 to 48.4). - **English→Romanian:** - Features added about 1.0 BLEU in both settings: 23.8→24.8 with parallel data only, and 28.2→29.2 with synthetic data. - The best system was competitive with the top WMT16 submission (28.9). *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/) · [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) · [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) ## Cited by Cited by 162 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 149 not yet. - [Towards String-To-Tree Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/aharoni-2017-string-tree-neural-machine-translation/) - [Graph Convolutional Encoders for Syntax-aware Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/bastings-2017-graph-convolutional-encoders-syntax-aware/) - [Survey: Multiword Expression Processing: A Survey](https://md.research.elamur.ai/free/en/papers/constant-2017-survey-multiword-expression-processing-survey/) - [Learning to Parse and Translate Improves Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/eriguchi-2017-learning-parse-translate-improves-neural/) - [Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings](https://md.research.elamur.ai/free/en/papers/gonzales-2017-improving-word-sense-disambiguation-neural/) - [Improved Lexically Constrained Decoding for Translation and Monolingual Rewriting](https://md.research.elamur.ai/free/en/papers/hu-2019-improved-lexically-constrained-decoding-translation/) - [Scheduled Multi-Task Learning: From Syntax to Translation](https://md.research.elamur.ai/free/en/papers/kiperwasser-2018-scheduled-multi-task-learning-syntax/) - [A Rich Morphological Tagger for English: Exploring the Cross-Linguistic Tradeoff Between Morphology and Syntax](https://md.research.elamur.ai/free/en/papers/kirov-2017-rich-morphological-tagger-english-exploring/) - [OpenNMT: Open-Source Toolkit for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/klein-2017-opennmt-open-source-toolkit-neural/) - [Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/marcheggiani-2017-encoding-sentences-graph-convolutional-networks/) - [Nematus: a Toolkit for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/sennrich-2017-nematus-toolkit-neural-machine-translation/) - [Leveraging Rule-Based Machine Translation Knowledge for Under-Resourced Neural Machine Translation Models](https://md.research.elamur.ai/free/en/papers/torregrosa-2019-leveraging-rule-based-machine-translation/) - [Exploiting Cross-Sentence Context for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/wang-2017-exploiting-cross-sentence-context-neural/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{sennrich2016linguistic, title = {{Linguistic Input Features Improve Neural Machine Translation}}, author = {Sennrich, Rico and Haddow, Barry}, booktitle = {Proceedings of the First Conference on Machine Translation: Volume 1, Research Papers}, year = {2016}, doi = {10.18653/v1/w16-2209}, eprint = {1606.02892}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/w16-2209} } ``` - [← previous Leveraging FrameNet to Improve Automatic Event Detection](https://md.research.elamur.ai/free/en/papers/liu-2016-leveraging-framenet-improve-automatic-event/) - [next → Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing](https://md.research.elamur.ai/free/en/papers/dowmunt-2016-log-linear-combinations-monolingual-bilingual/) - [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/) ## References 4 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. form multi-source translation (Zoph and Knight, 2016). The main technical difference is that in our approach, the encoder and attention layers are shared between features, which we deem appropriate for the types of features that we tested. [find](https://scholar.google.com/scholar?q=form+multi%2Dsource+translation+%28Zoph+and+Knight%2C+2016%29%2E+The+main+technical+difference+is+that+in+our+approach%2C+the+encoder+and+attention+layers+are+shared+between+features%2C+which+we+deem+appropriate+fo) 2. Andrei Alexandrescu and Katrin Kirchhoff. 2006. Factored Neural Language Models. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pages 1–4, New York City, USA. Association for Computational Linguistics. \#\# Conclusion Dzmitry Bahdanau, Kyunghyu… [find](https://scholar.google.com/scholar?q=Andrei+Alexandrescu+and+Katrin+Kirchhoff%2E+2006%2E+Factored+Neural+Language+Models%2E+In+Proceedings+of+the+Human+Language+Technology+Conference+of+the+NAACL%2C+Companion+Volume%3A+Short+Papers%2C+pages+1%E2%80%934%2C+New) 3. In this paper we investigate whether linguistic input features are beneficial to neural machine translation, and our empirical evidence suggests that this is the case. [find](https://scholar.google.com/scholar?q=In+this+paper+we+investigate+whether+linguistic+input+features+are+beneficial+to+neural+machine+translation%2C+and+our+empirical+evidence+suggests+that+this+is+the+case%2E) 4. Danqi Chen and Christopher Manning. 2014. A Fast and Accurate Dependency Parser using Neural Networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 740–750, Doha, Qatar. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Danqi+Chen+and+Christopher+Manning%2E+2014%2E+A+Fast+and+Accurate+Dependency+Parser+using+Neural+Networks%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29) [our card](https://md.research.elamur.ai/free/en/papers/missing-2014-chen-fast-accurate-dependency-parser-neural/)