---
title: "Context Gates for Neural Machine Translation — Tu 2017"
description: "To fix this, they propose context gates: sigmoid layers that inspect both contexts at every decoding… 17 references, cited by 44 papers here."
authors: ["Zhaopeng Tu", "Yang Liu", "Zhengdong Lu", "Xiaohua Liu", "Hang Li"]
year: 2017
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/Q17-1007/"
doi: "10.1162/tacl_a_00048"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tu-2017-context-gates-neural-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tu-2017-context-gates-neural-machine-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/) / Context Gates for Neural Machine Translation

# Context Gates for Neural Machine Translation

**Authors:** [Zhaopeng Tu](https://md.research.elamur.ai/free/en/authors/zhaopeng-tu/), [Yang Liu](https://md.research.elamur.ai/free/en/authors/yang-liu/), [Zhengdong Lu](https://md.research.elamur.ai/free/en/authors/zhengdong-lu/), [Xiaohua Liu](https://md.research.elamur.ai/free/en/authors/xiaohua-liu/) and [Hang Li](https://md.research.elamur.ai/free/en/authors/hang-li/) · **Year:** 2017 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2017](https://md.research.elamur.ai/free/en/venues/tacl/2017/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/Q17-1007/) · [PDF](https://aclanthology.org/Q17-1007.pdf) · [DOI 10.1162/tacl\_a\_00048](https://doi.org/10.1162/tacl_a_00048) · [OpenAlex](https://openalex.org/W2964298349)

**Citations:** 109 citations (OpenAlex, 2026-10-14) · [44](https://md.research.elamur.ai/free/en/papers/tu-2017-context-gates-neural-machine-translation/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Attention-based NMT systems often produce output that reads smoothly but drops or distorts parts of the source sentence. Tu et al. trace this to how the decoder combines two information streams. The source context (the attention-weighted encoder representation) mainly drives adequacy. The target context (the previous decoder state plus the last emitted word) mainly drives fluency. The authors manually rescaled each stream in Chinese–English translation and examined output lengths and human judgments. Weakening the source side produced shorter, less faithful output, while weakening the target side triggered repetition loops. To fix this, they propose context gates: sigmoid layers that inspect both contexts at every decoding step and output element-wise weights. These weights decide how much of each stream enters the decoder state update. Adding the gates to a standard attention model (GroundHog) improved BLEU by 2.3 points on average, with the largest benefit on long sentences. The work matters because it offers a lightweight way to control the adequacy–fluency balance, and the idea could apply to other sequence-to-sequence tasks.

## Key points
- Average gain of +2.3 BLEU over the attention-based GroundHog baseline (Bahdanau et al., 2015). The code is released at github.com/tuzhaopeng/NMT.
- Diagnostic study: source and target contexts were scaled by ratios such as 0.5 and 0.8, and 200 randomly sampled test sentences were rated by humans. Halving the target context pushed most outputs to the length cap of three times the source length.
- Ablations show that each design choice adds to the gain. Per-element gating beats a single scalar gate. Gating both contexts beats gating only the source. Peephole input from the source context contributes +0.57 BLEU, and the previously generated word gives a further gain.
- Sentence-level gate weight correlates positively with quality improvement, more strongly for sources of 30 words or more. The adequacy correlation is 0.121 and the fluency correlation 0.168 for long sentences, versus 0.071 and 0.040 for shorter ones.
- A decoder using a plain RNN with context gates matches the standard GRU system while using half the parameters and decoding faster. The method is described as complementary to coverage models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 137 works · 83,262 citations
- [Huawei Technologies](https://md.research.elamur.ai/free/en/companies/huawei-technologies/) — Company, Shenzhen, China · 29 works · 9,991 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works

## Cite

Cite as: Zhaopeng Tu, Yang Liu, Zhengdong Lu, Xiaohua Liu and Hang Li. 2017. Context Gates for Neural Machine Translation. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{tu2017context,
  title = {{Context Gates for Neural Machine Translation}},
  author = {Zhaopeng Tu, Yang Liu, Zhengdong Lu, Xiaohua Liu and Hang Li},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2017},
  doi = {10.1162/tacl\_a\_00048},
  url = {https://doi.org/10.1162/tacl_a_00048}
}
```

- [← previous Effective Domain Mixing for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/britz-2017-effective-domain-mixing-neural-machine/)
- [next → NegBio: a high-performance tool for negation and uncertainty detection in radiology reports](https://md.research.elamur.ai/free/en/papers/peng-2017-negbio-high-performance-tool-negation/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/)

## References

17 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/).
2. ICLR 2015. Peter E. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–311. [find](https://scholar.google.com/scholar?q=ICLR+2015%2E+Peter+E%2E+Brown%2C+Stephen+A%2E+Della+Pietra%2C+Vincent+J%2E+Della+Pietra%2C+and+Robert+L%2E+Mercer%2E+1993%2E+The+mathematics+of+statistical+machine+translation%3A+Parameter+estimation%2E+Computational+Linguis)
3. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP 2014. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+van+Merrienboer%2C+Caglar+Gulcehre%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014%2E+Learning+phrase+representations+using+rnn+encoder%2Ddecoder+for+statistical+machine+transla)
4. Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv. Michael Collins, Philipp Koehn, and Ivona Kučerová. 2005. [Clause restructuring for statistical machine translation](https://md.research.elamur.ai/free/en/papers/collins-2005-clause-restructuring-statistical-machine-translation/). In ACL 2005. Felix A Ger…
5. Yangfeng Ji, Trevor Cohn, Lingpeng Kong, Chris Dyer, and Jacob Eisenstein. 2015. Document context language models. In ICLR 2015. [find](https://scholar.google.com/scholar?q=Yangfeng+Ji%2C+Trevor+Cohn%2C+Lingpeng+Kong%2C+Chris+Dyer%2C+and+Jacob+Eisenstein%2E+2015%2E+Document+context+language+models%2E+In+ICLR+2015%2E)
6. Nal Kalchbrenner and Phil Blunsom. 2013. [Recurrent continuous translation models](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/). In EMNLP 2013.
7. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: open source toolkit for statistical machine translation. In ACL… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondrej+Bojar%2C+Alexandra+Consta)
8. Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. [Effective approaches to attention-based neural machine translation](https://md.research.elamur.ai/free/en/papers/luong-2015-effective-approaches-attention-based-neural/). In EMNLP 2015.
9. Tomas Mikolov and Geoffrey Zweig. 2012. Context dependent recurrent neural network language model. In SLT 2012. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov+and+Geoffrey+Zweig%2E+2012%2E+Context+dependent+recurrent+neural+network+language+model%2E+In+SLT+2012%2E)
10. Franz J. Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. [find](https://scholar.google.com/scholar?q=Franz+J%2E+Och+and+Hermann+Ney%2E+2003%2E+A+systematic+comparison+of+various+statistical+alignment+models%2E+Computational+Linguistics%2C+29%281%29%3A19%E2%80%9351%2E)
11. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [BLEU: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In ACL 2002.
12. Matthew Snover, Nitin Madnani, Bonnie J Dorr, and Richard Schwartz. 2009. [Fluency, adequacy, or HTER?: exploring different human judgments with a tunable MT metric](https://md.research.elamur.ai/free/en/papers/snover-2009-fluency-adequacy-or-hter-exploring/). In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 259–268.
13. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In NIPS 2014. Kristina Toutanova, H. Tolga Ilhan, and Christopher D. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V%2E+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+NIPS+2014%2E+Kristina+Toutanova%2C+H%2E+Tolga+Ilhan%2C+and+Christopher+D%2E)
14. Manning. 2002. Extensions to HMM-based statistical word alignment models. In EMNLP 2012. [find](https://scholar.google.com/scholar?q=Manning%2E+2002%2E+Extensions+to+HMM%2Dbased+statistical+word+alignment+models%2E+In+EMNLP+2012%2E)
15. Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. [Modeling coverage for neural machine translation](https://md.research.elamur.ai/free/en/papers/tu-2016-modeling-coverage-neural-machine-translation/). In ACL 2016.
16. Tian Wang and Kyunghyun Cho. 2016. Larger-context language modelling with recurrent neural network. In ACL 2016. [find](https://scholar.google.com/scholar?q=Tian+Wang+and+Kyunghyun+Cho%2E+2016%2E+Larger%2Dcontext+language+modelling+with+recurrent+neural+network%2E+In+ACL+2016%2E)
17. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, and Yoshua Bengio. 2015. [Show, attend and tell: Neural image caption generation with visual attention](https://md.research.elamur.ai/free/en/papers/xu-2015-show-attend-tell-neural-image/). In ICML 2015.

*Updated 24 October 2026*