---
title: "Selective Attention for Context-aware Neural — Maruf 2019"
description: "Most context-aware neural MT systems look only at a handful of preceding sentences."
authors: ["Sameen Maruf", "André F. T. Martins", "Gholamreza Haffari"]
year: 2019
venue: "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)"
source: "https://aclanthology.org/N19-1313/"
doi: "10.18653/v1/n19-1313"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/maruf-2019-selective-attention-context-aware-neural/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/maruf-2019-selective-attention-context-aware-neural/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / Selective Attention for Context-aware Neural Machine Translation

# Selective Attention for Context-aware Neural Machine Translation

**Authors:** [Sameen Maruf](https://md.research.elamur.ai/free/en/authors/sameen-maruf/), [André F. T. Martins](https://md.research.elamur.ai/free/en/authors/andre-f-t-martins/), [Gholamreza Haffari](https://md.research.elamur.ai/free/en/authors/gholamreza-haffari/) · **Year:** 2019 · **Venue:** Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N19-1313/) · [PDF](https://aclanthology.org/N19-1313.pdf) · [DOI 10.18653/v1/n19-1313](https://doi.org/10.18653/v1/n19-1313) · [OpenAlex](https://openalex.org/W2923779212)

**Citations:** 159 citations (openalex-snapshot · 2026-10-14) · [104](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Most context-aware neural MT systems look only at a handful of preceding sentences. This limits how well they handle discourse phenomena that depend on the wider document, and it makes them hard to scale. Maruf, Martins and Haffari address this with a memory-efficient context module for the Transformer that can draw on the whole document.

Their main proposal is a top-down hierarchical attention. Sparsemax first picks out a small set of relevant context sentences. Attention over words inside those sentences is then rescaled by the sentence weights, so irrelevant parts of the hierarchy are effectively pruned. They also test simpler flat variants that attend over sentence-level or word-level context. The resulting document representation is merged through context gating:
- into the encoder when only source-side context is used;
- into the decoder when both source and target context are used.

Experiments cover three English→German corpora in two settings: online (past context only) and offline (past and future). The selective model consistently beats the sentence-level Transformer and usually outperforms two recent context-aware systems. Qualitative examples suggest the sparsity makes the model's context use more interpretable.

## Key points
- Datasets are TED Talks, News-Commentary and Europarl (English→German). Overall gains over the Transformer are +1.34, +2.06 and +1.18 BLEU respectively.
- Context-aware baselines are Zhang et al. (2018) and the HAN model of Miculicich et al. (2018). The proposed approach surpasses them in most configurations.
- Sparsemax replaces softmax at the sentence level, and optionally at the word level, so that unrelated sentences receive exactly zero weight.
- The Document-level Context Layer has a multi-head context attention sub-layer and a feed-forward sub-layer. Residual connections are omitted because they hurt BLEU.
- Training proceeds in two steps: sentence-level pre-training, then joint fine-tuning. Decoding uses a two-pass iterative strategy following Maruf and Haffari (2018).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cited by

Cited by

104 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 97 not yet.

- [Assessing the Role of Context in Chat Translation Evaluation: Is Context Helpful and Under What Conditions?](https://md.research.elamur.ai/free/en/papers/agrawal-2024-assessing-role-context-chat-translation/)
- [ETC: Encoding Long and Structured Inputs in Transformers](https://md.research.elamur.ai/free/en/papers/ainslie-2020-etc-encoding-long-structured-inputs/)
- [Adaptively Sparse Transformers](https://md.research.elamur.ai/free/en/papers/correia-2019-adaptively-sparse-transformers/)
- [Findings of the WMT 2020 Shared Task on Chat Translation](https://md.research.elamur.ai/free/en/papers/farajian-2020-findings-wmt-2020-shared-task/)
- [Document-level Neural MT: A Systematic Comparison](https://md.research.elamur.ai/free/en/papers/lopes-2020-document-level-neural-mt-systematic/)
- [文档级神经机器翻译综述](https://md.research.elamur.ai/free/en/papers/lv-2024-document-level-neural/)
- [Context-8: A Data Set for Evaluating Context Sensitivity in Machine Translation](https://md.research.elamur.ai/free/en/papers/wang-2026-context-8-data-set-evaluating/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{maruf2019selective,
  title = {{Selective Attention for Context-aware Neural Machine Translation}},
  author = {Maruf, Sameen and Martins, André F. T. and Haffari, Gholamreza},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1313},
  url = {https://doi.org/10.18653/v1/n19-1313}
}
```

- [← previous ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://md.research.elamur.ai/free/en/papers/neumann-2019-scispacy-fast-robust-models-biomedical/)
- [next → Semantic mapping of participants in legal discourse](https://md.research.elamur.ai/free/en/papers/hbo-2019-hojgaard-semantic-mapping-participants-legal-discourse/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

42 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+Proceedings+of+the+International+Conference+on+Learning+Representati) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
2. Rachel Bawden, Rico Sennrich, Alexandra Birch, and Barry Haddow. 2018. Evaluating discourse phenomena in neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Paper… [find](https://scholar.google.com/scholar?q=Rachel+Bawden%2C+Rico+Sennrich%2C+Alexandra+Birch%2C+and+Barry+Haddow%2E+2018%2E+Evaluating+discourse+phenomena+in+neural+machine+translation%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter) [our card](https://md.research.elamur.ai/free/en/papers/bawden-2017-evaluating-discourse-phenomena-neural-machine/)
3. Mauro Cettolo, Christian Girardi, and Marcello Federico. 2012. Wit 3 : Web inventory of transcribed and translated talks. In Proceedings of the 16 th Conference of the European Association for Machine Translation (EAMT), pages 261–268, Trento, Italy. [find](https://scholar.google.com/scholar?q=Mauro+Cettolo%2C+Christian+Girardi%2C+and+Marcello+Federico%2E+2012%2E+Wit+3+%3A+Web+inventory+of+transcribed+and+translated+talks%2E+In+Proceedings+of+the+16+th+Conference+of+the+European+Association+for+Machine)
4. Kyunghyun Cho, B van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. In Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (SSST-8). [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+B+van+Merrienboer%2C+Dzmitry+Bahdanau%2C+and+Yoshua+Bengio%2E+2014%2E+On+the+properties+of+neural+machine+translation%3A+Encoder%2Ddecoder+approaches%2E+In+Eighth+Workshop+on+Syntax%2C+Semantics+and+St) [our card](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/)
5. Jonathan H. Clark, Chris Dyer, Alon Lavie, and Noah A. Smith. 2011. Better hypothesis testing for statistical machine translation: Controlling for optimizer instability. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (Short Pap… [find](https://scholar.google.com/scholar?q=Jonathan+H%2E+Clark%2C+Chris+Dyer%2C+Alon+Lavie%2C+and+Noah+A%2E+Smith%2E+2011%2E+Better+hypothesis+testing+for+statistical+machine+translation%3A+Controlling+for+optimizer+instability%2E+In+Proceedings+of+the+49th+Ann)
6. Trevor Cohn, Cong Duy Vu Hoang, Ekaterina Vy-molova, Kaisheng Yao, Chris Dyer, and Gholamreza Haffari. 2016. Incorporating structural alignment biases into an attentional neural translation model. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human La… [find](https://scholar.google.com/scholar?q=Trevor+Cohn%2C+Cong+Duy+Vu+Hoang%2C+Ekaterina+Vy%2Dmolova%2C+Kaisheng+Yao%2C+Chris+Dyer%2C+and+Gholamreza+Haffari%2E+2016%2E+Incorporating+structural+alignment+biases+into+an+attentional+neural+translation+model%2E+In+)
7. Peter Dayan, Sham Kakade, and P. Read Montague. 2000. Learning and selective attention. Nature Neuroscience, 3:1218–1223. [find](https://scholar.google.com/scholar?q=Peter+Dayan%2C+Sham+Kakade%2C+and+P%2E+Read+Montague%2E+2000%2E+Learning+and+selective+attention%2E+Nature+Neuroscience%2C+3%3A1218%E2%80%931223%2E)
8. Alon Lavie and Abhaya Agarwal. 2007. Meteor: An automatic metric for mt evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 228–231. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alon+Lavie+and+Abhaya+Agarwal%2E+2007%2E+Meteor%3A+An+automatic+metric+for+mt+evaluation+with+high+levels+of+correlation+with+human+judgments%2E+In+Proceedings+of+the+Second+Workshop+on+Statistical+Machine+Tr) [our card](https://md.research.elamur.ai/free/en/papers/lavie-2007-meteor-automatic-metric-mt-evaluation/)
9. Zhengxian Gong, Min Zhang, and Guodong Zhou. 2011. Cache-based document-level statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 909–919. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhengxian+Gong%2C+Min+Zhang%2C+and+Guodong+Zhou%2E+2011%2E+Cache%2Dbased+document%2Dlevel+statistical+machine+translation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pag)
10. Chaitanya Malaviya, Pedro Ferreira, and André F. T. Martins. 2018. Sparse and constrained attention for neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 370–376. [find](https://scholar.google.com/scholar?q=Chaitanya+Malaviya%2C+Pedro+Ferreira%2C+and+Andre%CC%81+F%2E+T%2E+Martins%2E+2018%2E+Sparse+and+constrained+attention+for+neural+machine+translation%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Co)
11. Zhengxian Gong, Min Zhang, and Guodong Zhou. 2015. Document-level machine translation evaluation with gist consistency and text cohesion. In Proceedings of the Second Workshop on Discourse in Machine Translation, pages 33–40, Lisbon, Portugal. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhengxian+Gong%2C+Min+Zhang%2C+and+Guodong+Zhou%2E+2015%2E+Document%2Dlevel+machine+translation+evaluation+with+gist+consistency+and+text+cohesion%2E+In+Proceedings+of+the+Second+Workshop+on+Discourse+in+Machine+)
12. André F. T. Martins and Ramón Astudillo. 2016. From softmax to sparsemax: A sparse model of attention and multi-label classification. In Proceedings of The 33rd International Conference on Machine Learning, volume 48, pages 1614–1623, New York, New York, USA. PMLR. [find](https://scholar.google.com/scholar?q=Andre%CC%81+F%2E+T%2E+Martins+and+Ramo%CC%81n+Astudillo%2E+2016%2E+From+softmax+to+sparsemax%3A+A+sparse+model+of+attention+and+multi%2Dlabel+classification%2E+In+Proceedings+of+The+33rd+International+Conference+on+Machine+L)
13. Christian Hardmeier and Marcello Federico. 2010. Modelling pronominal anaphora in statistical machine translation. In International Workshop on Spoken Language Translation, pages 283–289. [find](https://scholar.google.com/scholar?q=Christian+Hardmeier+and+Marcello+Federico%2E+2010%2E+Modelling+pronominal+anaphora+in+statistical+machine+translation%2E+In+International+Workshop+on+Spoken+Language+Translation%2C+pages+283%E2%80%93289%2E) [our card](https://md.research.elamur.ai/free/en/papers/hardmeier-2010-modelling-pronominal-anaphora-statistical-machine/)
14. Sameen Maruf and Gholamreza Haffari. 2018. Document context neural machine translation with memory networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1275– 1284. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sameen+Maruf+and+Gholamreza+Haffari%2E+2018%2E+Document+context+neural+machine+translation+with+memory+networks%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+) [our card](https://md.research.elamur.ai/free/en/papers/maruf-2018-document-context-neural-machine-translation/)
15. Sebastien Jean, Stanislas Lauly, Orhan Firat, and Kyunghyun Cho. 2017. Does neural machine translation benefit from larger context? In arXiv:1704.05135. [find](https://scholar.google.com/scholar?q=Sebastien+Jean%2C+Stanislas+Lauly%2C+Orhan+Firat%2C+and+Kyunghyun+Cho%2E+2017%2E+Does+neural+machine+translation+benefit+from+larger+context%3F+In+arXiv%3A1704%2E05135%2E)
16. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Diederik+P%2E+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+Proceedings+of+the+International+Conference+on+Learning+Representations%2E)
17. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Conference Proceedings: the 10th Machine Translation Summit, pages 79–86. AAMT. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2005%2E+Europarl%3A+A+parallel+corpus+for+statistical+machine+translation%2E+In+Conference+Proceedings%3A+the+10th+Machine+Translation+Summit%2C+pages+79%E2%80%9386%2E+AAMT%2E)
18. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Pro… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondr%CC%8Cej+Bojar%2C+Alexandra+Const)
19. Shaohui Kuang, Deyi Xiong, Weihua Luo, and Guodong Zhou. 2018. Modeling coherence for neural machine translation with dynamic and topic caches. In Proceedings of the 27th International Conference on Computational Linguistics, pages 596–606. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shaohui+Kuang%2C+Deyi+Xiong%2C+Weihua+Luo%2C+and+Guodong+Zhou%2E+2018%2E+Modeling+coherence+for+neural+machine+translation+with+dynamic+and+topic+caches%2E+In+Proceedings+of+the+27th+International+Conference+on+C)
20. Samuel Läubli, Rico Sennrich, and Martin Volk. 2018. Has machine translation achieved human parity? A case for document-level evaluation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 4791–4796. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Samuel+La%CC%88ubli%2C+Rico+Sennrich%2C+and+Martin+Volk%2E+2018%2E+Has+machine+translation+achieved+human+parity%3F+A+case+for+document%2Dlevel+evaluation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natu) [our card](https://md.research.elamur.ai/free/en/papers/laubli-2018-has-machine-translation-achieved-human/)
21. Sameen Maruf, André F. T. Martins, and Gholamreza Haffari. 2018. Contextual neural model for translating bilingual multi-speaker conversations. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 101–112, Brussels, Belgium. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Sameen+Maruf%2C+Andre%CC%81+F%2E+T%2E+Martins%2C+and+Gholamreza+Haffari%2E+2018%2E+Contextual+neural+model+for+translating+bilingual+multi%2Dspeaker+conversations%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Trans)
22. Laura Mascarell. 2017. Lexical chains meet word embeddings in document-level statistical machine translation. In Proceedings of the Third Workshop on Discourse in Machine Translation, pages 99– 109. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Laura+Mascarell%2E+2017%2E+Lexical+chains+meet+word+embeddings+in+document%2Dlevel+statistical+machine+translation%2E+In+Proceedings+of+the+Third+Workshop+on+Discourse+in+Machine+Translation%2C+pages+99%E2%80%93+109%2E+A)
23. Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, and James Henderson. 2018. Document-level neural machine translation with hierarchical attention networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2947–2954. [find](https://scholar.google.com/scholar?q=Lesly+Miculicich%2C+Dhananjay+Ram%2C+Nikolaos+Pappas%2C+and+James+Henderson%2E+2018%2E+Document%2Dlevel+neural+machine+translation+with+hierarchical+attention+networks%2E+In+Proceedings+of+the+2018+Conference+on+Em) [our card](https://md.research.elamur.ai/free/en/papers/miculicich-2018-document-level-neural-machine-translation/)
24. Lesly Miculicich Werlen and Andrei Popescu-Belis. 2017. Using coreference links to improve spanish-to-english machine translation. In Proceedings of the 2nd Workshop on Coreference Resolution Beyond OntoNotes (CORBON 2017), pages 30–40, Valencia, Spain. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Lesly+Miculicich+Werlen+and+Andrei+Popescu%2DBelis%2E+2017%2E+Using+coreference+links+to+improve+spanish%2Dto%2Denglish+machine+translation%2E+In+Proceedings+of+the+2nd+Workshop+on+Coreference+Resolution+Beyond+O) [our card](https://md.research.elamur.ai/free/en/papers/werlen-2017-coreference-links-improve-spanish-english/)
25. Mathias Müller, Annette Rios, Elena Voita, and Rico Sennrich. 2018. A large-scale test set for the evaluation of context-aware pronoun translation in neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 61–72, Belgium, Brussels. Associati… [find](https://scholar.google.com/scholar?q=Mathias+Mu%CC%88ller%2C+Annette+Rios%2C+Elena+Voita%2C+and+Rico+Sennrich%2E+2018%2E+A+large%2Dscale+test+set+for+the+evaluation+of+context%2Daware+pronoun+translation+in+neural+machine+translation%2E+In+Proceedings+of+the) [our card](https://md.research.elamur.ai/free/en/papers/muller-2018-large-scale-test-set-evaluation/)
26. Ramesh Nallapati, Bowen Zhou, Cı́cero Nogueira dos Santos, Çaglar Gülçehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In Proceedings of Conference on Natural Language Learning, pages 280–290. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ramesh+Nallapati%2C+Bowen+Zhou%2C+C%C4%B1%CC%81cero+Nogueira+dos+Santos%2C+C%CC%A7aglar+Gu%CC%88lc%CC%A7ehre%2C+and+Bing+Xiang%2E+2016%2E+Abstractive+text+summarization+using+sequence%2Dto%2Dsequence+RNNs+and+beyond%2E+In+Proceedings+of+Confer)
27. Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, Kevin Duh, Manaal Faruqui, Cynthia Gan, Dan Garrette, Yangfeng Ji, Lingpeng Kong, Adhiguna Kuncoro, Gaurav Kumar, Chaitanya Malaviya, Pau… [find](https://scholar.google.com/scholar?q=Graham+Neubig%2C+Chris+Dyer%2C+Yoav+Goldberg%2C+Austin+Matthews%2C+Waleed+Ammar%2C+Antonios+Anastasopoulos%2C+Miguel+Ballesteros%2C+David+Chiang%2C+Daniel+Clothiaux%2C+Trevor+Cohn%2C+Kevin+Duh%2C+Manaal+Faruqui%2C+Cynthia+Ga) [our card](https://md.research.elamur.ai/free/en/papers/neubig-2017-dynet-dynamic-neural-network-toolkit/)
28. Vlad Niculae and Mathieu Blondel. 2017. A regularized framework for sparse and structured neural attention. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 3338–3348. Curran Associates… [find](https://scholar.google.com/scholar?q=Vlad+Niculae+and+Mathieu+Blondel%2E+2017%2E+A+regularized+framework+for+sparse+and+structured+neural+attention%2E+In+I%2E+Guyon%2C+U%2E+V%2E+Luxburg%2C+S%2E+Bengio%2C+H%2E+Wallach%2C+R%2E+Fergus%2C+S%2E+Vishwanathan%2C+and+R%2E+Garnet)
29. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 311–318. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+BLEU%3A+A+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computat) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
30. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54 th Annual Meeting of the Association for Computational Linguistics, pages 1715–1725. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+Proceedings+of+the+54+th+Annual+Meeting+of+the+Association+for+Computational+Lin)
31. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems, pages 3104–3112. MIT Press. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V%2E+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Proceedings+of+the+27th+International+Conference+on+Neural+Information+Processing+Systems%2C+p)
32. Jörg Tiedemann. 2010. Context adaptation in statistical machine translation using models with exponentially decaying cache. In Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing, DANLP 2010, pages 8–15, Stroudsburg, PA, USA. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann%2E+2010%2E+Context+adaptation+in+statistical+machine+translation+using+models+with+exponentially+decaying+cache%2E+In+Proceedings+of+the+2010+Workshop+on+Domain+Adaptation+for+Natural+Langua)
33. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in OPUS. In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+OPUS%2E+In+Proceedings+of+the+8th+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9912%29%2C+Istanbul%2C+Turkey%2E+European+Langua)
34. Zhaopeng Tu, Yang Liu, Shuming Shi, and Tong Zhang. 2018. Learning to remember translation history with a continuous cache. Transactions of the Association for Computational Linguistics, 6:407– 420. [find](https://scholar.google.com/scholar?q=Zhaopeng+Tu%2C+Yang+Liu%2C+Shuming+Shi%2C+and+Tong+Zhang%2E+2018%2E+Learning+to+remember+translation+history+with+a+continuous+cache%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+6%3A407%E2%80%93+420%2E)
35. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30, pages 5998–6008. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+Neural+Information+Processi)
36. Elena Voita, Pavel Serdyukov, Rico Sennrich, and Ivan Titov. 2018. Context-aware neural machine translation learns anaphora resolution. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1264–1274. Association for Computational L… [find](https://scholar.google.com/scholar?q=Elena+Voita%2C+Pavel+Serdyukov%2C+Rico+Sennrich%2C+and+Ivan+Titov%2E+2018%2E+Context%2Daware+neural+machine+translation+learns+anaphora+resolution%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for) [our card](https://md.research.elamur.ai/free/en/papers/voita-2018-context-aware-neural-machine-translation/)
37. Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. 2017. Exploiting cross-sentence context for neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 2816–2821. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Longyue+Wang%2C+Zhaopeng+Tu%2C+Andy+Way%2C+and+Qun+Liu%2E+2017%2E+Exploiting+cross%2Dsentence+context+for+neural+machine+translation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Proc) [our card](https://md.research.elamur.ai/free/en/papers/wang-2017-exploiting-cross-sentence-context-neural/)
38. Yingce Xia, Xu Tan, Fei Tian, Tao Qin, Nenghai Yu, and Tie-Yan Liu. 2018. Model-level dual learning. In Proceedings of the 35th International Conference on Machine Learning, pages 5379–5388. [find](https://scholar.google.com/scholar?q=Yingce+Xia%2C+Xu+Tan%2C+Fei+Tian%2C+Tao+Qin%2C+Nenghai+Yu%2C+and+Tie%2DYan+Liu%2E+2018%2E+Model%2Dlevel+dual+learning%2E+In+Proceedings+of+the+35th+International+Conference+on+Machine+Learning%2C+pages+5379%E2%80%935388%2E)
39. Deyi Xiong, Yang Ding, Min Zhang, and Chew Lim Tan. 2013. Lexical chain based cohesion models for document-level statistical machine translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1563–1573. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Deyi+Xiong%2C+Yang+Ding%2C+Min+Zhang%2C+and+Chew+Lim+Tan%2E+2013%2E+Lexical+chain+based+cohesion+models+for+document%2Dlevel+statistical+machine+translation%2E+In+Proceedings+of+the+2013+Conference+on+Empirical+Met)
40. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 14… [find](https://scholar.google.com/scholar?q=Zichao+Yang%2C+Diyi+Yang%2C+Chris+Dyer%2C+Xiaodong+He%2C+Alex+Smola%2C+and+Eduard+Hovy%2E+2016%2E+Hierarchical+attention+networks+for+document+classification%2E+In+Proceedings+of+the+2016+Conference+of+the+North+Amer) [our card](https://md.research.elamur.ai/free/en/papers/yang-2016-hierarchical-attention-networks-document-classification/)
41. Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, and Yang Liu. 2018. Improving the transformer translation model with document-level context. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 533–542. Association for Computation… [find](https://scholar.google.com/scholar?q=Jiacheng+Zhang%2C+Huanbo+Luan%2C+Maosong+Sun%2C+Feifei+Zhai%2C+Jingfang+Xu%2C+Min+Zhang%2C+and+Yang+Liu%2E+2018%2E+Improving+the+transformer+translation+model+with+document%2Dlevel+context%2E+In+Proceedings+of+the+Confer) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2018-improving-transformer-translation-model-document/)
42. Jörg Tiedemann and Yves Scherrer. 2017. Neural machine translation with extended context. In Proceedings of the Third Workshop on Discourse in Machine Translation, pages 82–92. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann+and+Yves+Scherrer%2E+2017%2E+Neural+machine+translation+with+extended+context%2E+In+Proceedings+of+the+Third+Workshop+on+Discourse+in+Machine+Translation%2C+pages+82%E2%80%9392%2E+Association+for+Comput) [our card](https://md.research.elamur.ai/free/en/papers/tiedemann-2017-neural-machine-translation-extended-context/)