--- title: "Neural Machine Translation with Extended — Tiedemann 2017" description: "Conventional NMT systems translate each sentence in isolation, so they lose cues from neighbouring sentences that are needed for discourse phenomena such…" authors: ["Jörg Tiedemann", "Yves Scherrer"] year: 2017 venue: "Proceedings of the Third Workshop on Discourse in Machine Translation" source: "https://aclanthology.org/W17-4811/" doi: "10.18653/v1/w17-4811" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/tiedemann-2017-neural-machine-translation-extended-context/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tiedemann-2017-neural-machine-translation-extended-context/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/) / Neural Machine Translation with Extended Context # Neural Machine Translation with Extended Context **Authors:** [Jörg Tiedemann](https://md.research.elamur.ai/free/en/authors/jorg-tiedemann/), [Yves Scherrer](https://md.research.elamur.ai/free/en/authors/yves-scherrer/) · **Year:** 2017 · **Venue:** Proceedings of the Third Workshop on Discourse in Machine Translation · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/W17-4811/) · [PDF](https://aclanthology.org/W17-4811.pdf) · [DOI 10.18653/v1/w17-4811](https://doi.org/10.18653/v1/w17-4811) · [OpenAlex](https://openalex.org/W2749972266) **Citations:** 217 citations (openalex-snapshot · 2026-10-14) · [160](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Conventional NMT systems translate each sentence in isolation, so they lose cues from neighbouring sentences that are needed for discourse phenomena such as pronoun reference. Tiedemann and Scherrer present a pilot study that tests whether a standard attention-based encoder-decoder can use wider context when the architecture stays unchanged and only the training and test segments are enlarged. They use German→English movie subtitles from OpenSubtitles2016 and the Helsinki NMT system (HNMT). A sentence-level baseline is compared with two variants. In 2+1, the previous source unit is prepended, either with its tokens tagged by a cc prefix or separated by break tokens. In 2+2, both source and target segments are joined with their predecessors using a BREAK marker. The extended models stay robust in translation quality and learn to tell context apart from the segment being translated. They also attend across sentence boundaries in plausible ways. A manual check of the ambiguous pronoun sie/Sie shows small gains, mainly for plural readings, but these are not statistically significant. The study offers a simple, architecture-free starting point for document-level NMT. ## Key points - **Data:** OpenSubtitles2016 German–English covers 16,910 movies and TV series, with 400 movies held out for development and testing. The corpus has about 13.9M translation units (roughly 8 tokens each), of which 13.5M are training instances. The first 5,000 test units are used for automatic evaluation. - **Setup:** HNMT with BPE subwords is used. The 2+1 models use 30k BPE codes and a 60k model vocabulary, because the cc-prefixed context words double the vocabulary. The 2+2 model uses joint 60k BPE codes. Each system is decoded as an ensemble of its last four savepoints. BLEU and chrF3 serve as automatic metrics, and for 2+2 only the final segment is scored. - **Attention:** In the extended-source model, about 7.1% of the total attention weight goes to the preceding context. This share is small but not negligible. - **Pronoun evaluation:** 565 of the 1,143 sie/Sie occurrences were judged manually. Overall accuracy is 90.1% for the baseline and 91.7% for 2+2. For plural readings it rises from 69.8% to 79.1%. None of the differences is significant under χ² tests at p = 0.05. - **Limitations:** A one-unit context window can miss antecedents that appear further back, which sometimes causes errors the baseline avoids. It is also not yet clear that the observed gains come from cross-segment attention. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/) · [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) ## Cited by Cited by 160 papers of our library cite it (of the 140673 whose bibliographies we read); 12 of them have a page here, 148 not yet. - [Assessing the Role of Context in Chat Translation Evaluation: Is Context Helpful and Under What Conditions?](https://md.research.elamur.ai/free/en/papers/agrawal-2024-assessing-role-context-chat-translation/) - [Evaluating Discourse Phenomena in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/bawden-2017-evaluating-discourse-phenomena-neural-machine/) - [Findings of the WMT 2020 Shared Task on Chat Translation](https://md.research.elamur.ai/free/en/papers/farajian-2020-findings-wmt-2020-shared-task/) - [Large Language Models Effectively Leverage Document-level Context for Literary Translation, but Critical Errors Persist](https://md.research.elamur.ai/free/en/papers/karpinska-2023-large-language-models-effectively-leverage/) - [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://md.research.elamur.ai/free/en/papers/krishna-2023-paraphrasing-evades-detectors-ai-generated/) - [Document-level Neural MT: A Systematic Comparison](https://md.research.elamur.ai/free/en/papers/lopes-2020-document-level-neural-mt-systematic/) - [文档级神经机器翻译综述](https://md.research.elamur.ai/free/en/papers/lv-2024-document-level-neural/) - [Document-Level Neural Machine Translation with Hierarchical Attention Networks](https://md.research.elamur.ai/free/en/papers/miculicich-2018-document-level-neural-machine-translation/) - [Context-aware Neural Machine Translation with Coreference Information](https://md.research.elamur.ai/free/en/papers/ohtani-2019-context-aware-neural-machine-translation/) - [Context-Aware Neural Machine Translation Learns Anaphora Resolution](https://md.research.elamur.ai/free/en/papers/voita-2018-context-aware-neural-machine-translation/) - [When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion](https://md.research.elamur.ai/free/en/papers/voita-2019-when-good-translation-is-wrong/) - [Context-8: A Data Set for Evaluating Context Sensitivity in Machine Translation](https://md.research.elamur.ai/free/en/papers/wang-2026-context-8-data-set-evaluating/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{tiedemann2017neural, title = {{Neural Machine Translation with Extended Context}}, author = {Tiedemann, Jörg and Scherrer, Yves}, booktitle = {Proceedings of the Third Workshop on Discourse in Machine Translation}, year = {2017}, doi = {10.18653/v1/w17-4811}, url = {https://doi.org/10.18653/v1/w17-4811} } ``` - [← previous Neural AMR: Sequence-to-Sequence Models for Parsing and Generation](https://md.research.elamur.ai/free/en/papers/konstas-2017-neural-amr-sequence-sequence-models/) - [next → Neural Message Passing for Quantum Chemistry](https://md.research.elamur.ai/free/en/papers/gilmer-2017-neural-message-passing-quantum-chemistry/) - [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/) ## References 2 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. ArXiv e-prints . [find](https://scholar.google.com/scholar?q=Jimmy+Lei+Ba%2C+Jamie+Ryan+Kiros%2C+and+Geoffrey+E%2E+Hinton%2E+2016%2E+Layer+normalization%2E+ArXiv+e%2Dprints+%2E) [our card](https://md.research.elamur.ai/free/en/papers/ba-2016-layer-normalization/) 2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+CoRR+abs%2F1409%2E0473%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)