---
title: "Better Document-Level Machine Translation with Bayes' Rule"
description: "Translating whole documents requires cross-sentence context. 2.5 BLEU. 57 references, cited by 31 papers here."
authors: ["Lei Yu", "Laurent Sartran", "Wojciech Stokowiec", "Wang Ling", "Lingpeng Kong", "Phil Blunsom", "Chris Dyer"]
year: 2020
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2020.tacl-1.23/"
doi: "10.1162/tacl_a_00319"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yu-2020-better-document-level-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yu-2020-better-document-level-machine-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/) / Better Document-Level Machine Translation with Bayes' Rule

# Better Document-Level Machine Translation with Bayes' Rule

**Authors:** [Lei Yu](https://md.research.elamur.ai/free/en/authors/lei-yu/), [Laurent Sartran](https://md.research.elamur.ai/free/en/authors/laurent-sartran/), [Wojciech Stokowiec](https://md.research.elamur.ai/free/en/authors/wojciech-stokowiec/), [Wang Ling](https://md.research.elamur.ai/free/en/authors/wang-ling/), [Lingpeng Kong](https://md.research.elamur.ai/free/en/authors/lingpeng-kong/), [Phil Blunsom](https://md.research.elamur.ai/free/en/authors/phil-blunsom/) and [Chris Dyer](https://md.research.elamur.ai/free/en/authors/chris-dyer/) · **Year:** 2020 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2020](https://md.research.elamur.ai/free/en/venues/tacl/2020/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.tacl-1.23/) · [PDF](https://aclanthology.org/2020.tacl-1.23.pdf) · [DOI 10.1162/tacl\_a\_00319](https://doi.org/10.1162/tacl_a_00319) · [OpenAlex](https://openalex.org/W3038817008)

**Citations:** 41 citations (OpenAlex, 2026-10-14) · [31](https://md.research.elamur.ai/free/en/papers/yu-2020-better-document-level-machine-translation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Translating whole documents requires cross-sentence context. For example, Chinese marks tense and number less explicitly than English, so the right English form often depends on other sentences. However, parallel documents for training are scarce. The authors address this by applying Bayes' rule, the classic noisy channel approach. They split translation into two parts. The first is a document-level language model prior over target text, trained on plentiful monolingual documents. The second is a sentence-level reverse ("channel") translation model, trained only on parallel sentences. The channel model treats sentences independently, but once the source is observed, the language model prior links all target sentences in the posterior. Decoding works in two steps. A proposal model first produces candidate translations for each sentence. A left-to-right beam search then reranks these candidates by combining both models. On Chinese–English document translation, the method beat transformer baselines by roughly 2.5 BLEU, and human evaluation confirmed the gains. This matters because document-aware translation becomes possible without parallel documents.

## Key points
- The model factorizes p(y|x) ∝ p(x|y) × p(y). It uses a transformer-style autoregressive document language model and any sequence-to-sequence model as the channel. Each component can be retrained separately, for instance when more monolingual data becomes available.
- Two proposal models were compared: a sentence-level Transformer (Vaswani et al., 2017) and a document-context Transformer (Zhang et al., 2018).
- The sentence-independence assumption in the channel model loses less than it seems. The authors compare this to how naïve Bayes classifiers gain dependencies at inference time.
- Comparing the sent-reranker with the doc-reranker shows that document context in the language model is what drives the improvements.
- Error analysis shows the doc-reranker keeps tense, number and lexical choice consistent across a document better than the document transformer does.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 430 works · 424,363 citations
- [University of Oxford](https://md.research.elamur.ai/free/en/universities/university-of-oxford/) — University, Oxford, United Kingdom · 104 works · 157,183 citations
- [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) — Company, London, United Kingdom · 56 works · 78,880 citations
- [Polish-Japanese Academy of Information Technology](https://md.research.elamur.ai/free/en/universities/polish-japanese-academy-of-information-technology/) — University, Warsaw, Poland · 2 works · 74 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 1,087 works

## Cite

Cite as: Lei Yu, Laurent Sartran, Wojciech Stokowiec, Wang Ling, Lingpeng Kong, Phil Blunsom and Chris Dyer. 2020. Better Document-Level Machine Translation with Bayes' Rule. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{yu2020better,
  title = {{Better Document-Level Machine Translation with Bayes' Rule}},
  author = {Lei Yu, Laurent Sartran, Wojciech Stokowiec, Wang Ling, Lingpeng Kong, Phil Blunsom and Chris Dyer},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.1162/tacl\_a\_00319},
  url = {https://doi.org/10.1162/tacl_a_00319}
}
```

- [← previous "I'd rather just go to bed": Understanding Indirect Answers](https://md.research.elamur.ai/free/en/papers/louis-2020-i-d-rather-just-go/)
- [next → Finding the Optimal Vocabulary Size for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/gowda-2020-finding-optimal-vocabulary-size-neural/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/)

## References

57 references, 32 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Sergey+Edunov%2C+Myle+Ott%2C+Michael+Auli%2C+and+David+Grangier%2E+2018%2E+Understanding+back%2Dtranslation+at+scale%2E+In+Proceedings+of+EMNLP%2E)
2. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2019. [An effective approach to unsupervised machine translation](https://md.research.elamur.ai/free/en/papers/artetxe-2019-effective-approach-unsupervised-machine-translation/). In Proceedings of ACL.
3. William A. Gale and Kenneth W. Church. 1993. [A program for aligning sentences in bilingual corpora](https://md.research.elamur.ai/free/en/papers/gale-1993-program-aligning-sentences-bilingual-corpora/). Computational Linguistics, 19(1):75–102.
4. Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. [Unsupervised neural machine translation](https://md.research.elamur.ai/free/en/papers/artetxe-2017-unsupervised-neural-machine-translation/). In Proceedings of ICLR.
5. Çaglar Gülçehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loı̈c Barrault, Huei-Chi Lin, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2015. [On using monolingual corpora in neural machine translation](https://md.research.elamur.ai/free/en/papers/gulcehre-2015-using-monolingual-corpora-neural-machine/). CoRR, abs/1503.03535.
6. Rachel Bawden, Rico Sennrich, Alexandra Birch, and Barry Haddow. 2018. [Evaluating discourse phenomena in neural machine translation](https://md.research.elamur.ai/free/en/papers/bawden-2017-evaluating-discourse-phenomena-neural-machine/). In Proceedings of NAACL-HLT.
7. Peter F. Brown, Stephen Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–311. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Stephen+Della+Pietra%2C+Vincent+J%2E+Della+Pietra%2C+and+Robert+L%2E+Mercer%2E+1993%2E+The+mathematics+of+statistical+machine+translation%3A+Parameter+estimation%2E+Computational+Linguistics%2C+19%282%29%3A26)
8. Gholamreza Haffari and Sameen Maruf. 2018. [Document context neural machine translation with memory networks](https://md.research.elamur.ai/free/en/papers/maruf-2018-document-context-neural-machine-translation/). In Proceedings of ACL.
9. Yong Cheng, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. [Semi-supervised learning for neural machine translation](https://md.research.elamur.ai/free/en/papers/cheng-2016-semi-supervised-learning-neural-machine/). In Proceedings of ACL.
10. Sébastien Jean, Stanislas Lauly, Orhan Firat, and Kyunghyun Cho. 2017. Does neural machine translation benefit from larger context? CoRR, abs/1704.05135. [find](https://scholar.google.com/scholar?q=Se%CC%81bastien+Jean%2C+Stanislas+Lauly%2C+Orhan+Firat%2C+and+Kyunghyun+Cho%2E+2017%2E+Does+neural+machine+translation+benefit+from+larger+context%3F+CoRR%2C+abs%2F1704%2E05135%2E)
11. Alexandra Chronopoulou, Christos Baziotis, and Alexandros Potamianos. 2019. An embarrassingly simple approach for transfer learning from pretrained language models. In Proceedings of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Alexandra+Chronopoulou%2C+Christos+Baziotis%2C+and+Alexandros+Potamianos%2E+2019%2E+An+embarrassingly+simple+approach+for+transfer+learning+from+pretrained+language+models%2E+In+Proceedings+of+NAACL%2DHLT%2E)
12. Marcin Junczys-Dowmunt. 2019. [Microsoft translator at WMT 2019: Towards large-scale document-level neural machine translation](https://md.research.elamur.ai/free/en/papers/dowmunt-2019-microsoft-translator-wmt-2019-large/). In Proceedings of WMT.
13. Yoon Kim and Alexander M. Rush. 2016. [Sequence-level knowledge distillation](https://md.research.elamur.ai/free/en/papers/kim-2016-sequence-level-knowledge-distillation/). In Proceedings of EMNLP.
14. Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G. Carbonell, Quoc Viet Le, and Ruslan Salakhutdinov. 2019. Transformer-xl: Attentive language models beyond a fixed-length context. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Zihang+Dai%2C+Zhilin+Yang%2C+Yiming+Yang%2C+Jaime+G%2E+Carbonell%2C+Quoc+Viet+Le%2C+and+Ruslan+Salakhutdinov%2E+2019%2E+Transformer%2Dxl%3A+Attentive+language+models+beyond+a+fixed%2Dlength+context%2E+In+Proceedings+of+ACL%2E)
15. Diederik P. Kingma and Jimmy Ba. 2015. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In Proceedings of ICLR.
16. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of NAACL-HLT.
17. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proc… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondrej+Bojar%2C+Alexandra+Consta)
18. Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural Shaohui Kuang, Deyi Xiong, Weihua Luo, and Guodong Zhou. 2017. Cache-based document-level neural machine translation. CoRR, abs/1711.112… — [arXiv:1711.11221](https://arxiv.org/abs/1711.11221) [find](https://scholar.google.com/scholar?q=Li+Dong%2C+Nan+Yang%2C+Wenhui+Wang%2C+Furu+Wei%2C+Xiaodong+Liu%2C+Yu+Wang%2C+Jianfeng+Gao%2C+Ming+Zhou%2C+and+Hsiao%2DWuen+Hon%2E+2019%2E+Unified+language+model+pre%2Dtraining+for+natural+Shaohui+Kuang%2C+Deyi+Xiong%2C+Weihua+Lu)
19. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In Proceedings of NAACL.
20. Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. CoRR, abs/1901.07291. [find](https://scholar.google.com/scholar?q=Guillaume+Lample+and+Alexis+Conneau%2E+2019%2E+Cross%2Dlingual+language+model+pretraining%2E+CoRR%2C+abs%2F1901%2E07291%2E)
21. Matt Post. 2018. [A call for clarity in reporting BLEU scores](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/). In Proceedings of WMT.
22. Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2018a. [Unsupervised machine translation using monolingual corpora only](https://md.research.elamur.ai/free/en/papers/lample-2017-unsupervised-machine-translation-using-monolingual/). In Proceedings of ICLR.
23. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+and+Ilya+Sutskever%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
24. Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2018b. Phrase-based & neural unsupervised machine translation. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Myle+Ott%2C+Alexis+Conneau%2C+Ludovic+Denoyer%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2018b%2E+Phrase%2Dbased+%26+neural+unsupervised+machine+translation%2E+In+Proceedings+of+EMNLP%2E)
25. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8). [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+Blog%2C+1%288%29%2E)
26. Lidia Mangu, Eric Brill, and Andreas Stolcke. 2000. Finding consensus in speech recognition: word error minimization and other applications of confusion networks. Computer Speech & Language, 14(4). [find](https://scholar.google.com/scholar?q=Lidia+Mangu%2C+Eric+Brill%2C+and+Andreas+Stolcke%2E+2000%2E+Finding+consensus+in+speech+recognition%3A+word+error+minimization+and+other+applications+of+confusion+networks%2E+Computer+Speech+%26+Language%2C+14%284%29%2E)
27. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. Improving neural machine translation models with monolingual data. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016a%2E+Improving+neural+machine+translation+models+with+monolingual+data%2E+In+Proceedings+of+ACL%2E)
28. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of ACL.
29. Sameen Maruf, André F. T. Martins, and Gholamreza Haffari. 2019. [Selective attention for context-aware neural machine translation](https://md.research.elamur.ai/free/en/papers/maruf-2019-selective-attention-context-aware-neural/). In Proceedings of NAACL-HLT.
30. Ross D. Shachter. 1998. Bayes-ball: The rational pastime (for determining irrelevance and requisite information in belief networks and influence diagrams). In Proceedings of UAI. [find](https://scholar.google.com/scholar?q=Ross+D%2E+Shachter%2E+1998%2E+Bayes%2Dball%3A+The+rational+pastime+%28for+determining+irrelevance+and+requisite+information+in+belief+networks+and+influence+diagrams%29%2E+In+Proceedings+of+UAI%2E)
31. Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. [Learned in translation: Contextualized word vectors](https://md.research.elamur.ai/free/en/papers/mccann-2017-learned-translation-contextualized-word-vectors/). In Proceedings of NeurIPS.
32. Tianxiao Shen, Myle Ott, Michael Auli, and Marc’Aurelio Ranzato. 2019. Mixture models for diverse machine translation: Tricks of the trade. In Proceedings of ICML. [find](https://scholar.google.com/scholar?q=Tianxiao+Shen%2C+Myle+Ott%2C+Michael+Auli%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2019%2E+Mixture+models+for+diverse+machine+translation%3A+Tricks+of+the+trade%2E+In+Proceedings+of+ICML%2E)
33. Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2018a. An analysis of neural language modeling at multiple scales. CoRR, abs/1803.08240. [find](https://scholar.google.com/scholar?q=Stephen+Merity%2C+Nitish+Shirish+Keskar%2C+and+Richard+Socher%2E+2018a%2E+An+analysis+of+neural+language+modeling+at+multiple+scales%2E+CoRR%2C+abs%2F1803%2E08240%2E)
34. Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2018b. [Regularizing and optimizing LSTM language models](https://md.research.elamur.ai/free/en/papers/merity-2017-regularizing-optimizing-lstm-language-models/). In Proceedings of ICLR.
35. Matthew Snover, Bonnie Dorr, Richard Schwarz, Linnea Micciulla, and John Makhoul. 2006. [A study of translation edit rate with targeted human annotation](https://md.research.elamur.ai/free/en/papers/snover-2006-study-translation-edit-rate-targeted/). In Proceedings of AMTA.
36. Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. [Facebook fair’s WMT19 news translation task submission](https://md.research.elamur.ai/free/en/papers/ng-2019-facebook-fair-s-wmt19-news/). In Proceedings of WMT.
37. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2019. [MASS: Masked sequence to sequence pre-training for language generation](https://md.research.elamur.ai/free/en/papers/song-2019-mass-masked-sequence-sequence-pre/). In Proceedings of ICML.
38. Luke de Oliveira and Alfredo Láinez Rodrigo. 2019. Repurposing decoder-transformer language models for abstractive summarization. ArXiv, abs/1909.00325. — [arXiv:1909.00325](https://arxiv.org/abs/1909.00325) [find](https://scholar.google.com/scholar?q=Luke+de+Oliveira+and+Alfredo+La%CC%81inez+Rodrigo%2E+2019%2E+Repurposing+decoder%2Dtransformer+language+models+for+abstractive+summarization%2E+ArXiv%2C+abs%2F1909%2E00325%2E+%E2%80%94+%5BarXiv%3A1909%2E00325%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F190)
39. Meng Sun, Bojian Jiang, Hao Xiong, Zhongjun He, Hua Wu, and Haifeng Wang. 2019. Baidu neural machine translation systems for WMT19. In Proceedings of WMT. [find](https://scholar.google.com/scholar?q=Meng+Sun%2C+Bojian+Jiang%2C+Hao+Xiong%2C+Zhongjun+He%2C+Hua+Wu%2C+and+Haifeng+Wang%2E+2019%2E+Baidu+neural+machine+translation+systems+for+WMT19%2E+In+Proceedings+of+WMT%2E)
40. Jörg Tiedemann and Yves Scherrer. 2017. [Neural machine translation with extended context](https://md.research.elamur.ai/free/en/papers/tiedemann-2017-neural-machine-translation-extended-context/). In Proceedings of DiscoMT@EMNLP. 2019. Microsoft Research Asia’s systems for WMT19. In Proceedings of WMT.
41. Hao Xiong, Zhongjun He, Hua Wu, and Haifeng Wang. 2019. Modeling coherence for discourse neural machine translation. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Hao+Xiong%2C+Zhongjun+He%2C+Hua+Wu%2C+and+Haifeng+Wang%2E+2019%2E+Modeling+coherence+for+discourse+neural+machine+translation%2E+In+Proceedings+of+AAAI%2E)
42. Zhaopeng Tu, Yang Liu, Shuming Shi, and Tong Zhang. 2018. Learning to remember translation history with a continuous cache. Transactions of the Association for Computational Linguistics, 6:407–420. [find](https://scholar.google.com/scholar?q=Zhaopeng+Tu%2C+Yang+Liu%2C+Shuming+Shi%2C+and+Tong+Zhang%2E+2018%2E+Learning+to+remember+translation+history+with+a+continuous+cache%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+6%3A407%E2%80%93420%2E)
43. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. [Xlnet: Generalized autoregressive pretraining for language understanding](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/). CoRR, abs/1906.08237.
44. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Proceedings of NeurIPS.
45. Kyra Yee, Yann Dauphin, and Michael Auli. 2019. [Simple and effective noisy channel modeling for neural machine translation](https://md.research.elamur.ai/free/en/papers/yee-2019-simple-effective-noisy-channel-modeling/). In Proceedings of EMNLP.
46. Elena Voita, Rico Sennrich, and Ivan Titov. 2019a. [Context-aware monolingual repair for neural machine translation](https://md.research.elamur.ai/free/en/papers/voita-2019-context-aware-monolingual-repair-neural/). In Proceedings of EMNLP-IJCNLP.
47. Lei Yu, Phil Blunsom, Chris Dyer, Edward Grefenstette, and Tomás Kociský. 2017. The neural noisy channel. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Lei+Yu%2C+Phil+Blunsom%2C+Chris+Dyer%2C+Edward+Grefenstette%2C+and+Toma%CC%81s+Kocisky%CC%81%2E+2017%2E+The+neural+noisy+channel%2E+In+Proceedings+of+ICLR%2E)
48. Elena Voita, Rico Sennrich, and Ivan Titov. 2019b. [When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion](https://md.research.elamur.ai/free/en/papers/voita-2019-when-good-translation-is-wrong/). In Proceedings of ACL.
49. Lei Yu, Jan Buys, and Phil Blunsom. 2016. Online segment to segment neural transduction. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Lei+Yu%2C+Jan+Buys%2C+and+Phil+Blunsom%2E+2016%2E+Online+segment+to+segment+neural+transduction%2E+In+Proceedings+of+EMNLP%2E)
50. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. [Defending against neural fake news](https://md.research.elamur.ai/free/en/papers/zellers-2019-defending-against-neural-fake-news/). arXiv preprint arXiv:1905.12616.
51. Elena Voita, Pavel Serdyukov, Rico Sennrich, and Ivan Titov. 2018. [Context-aware neural machine translation learns anaphora resolution](https://md.research.elamur.ai/free/en/papers/voita-2018-context-aware-neural-machine-translation/). In Proceedings of ACL.
52. Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. 2017. [Exploiting cross-sentence context for neural machine translation](https://md.research.elamur.ai/free/en/papers/wang-2017-exploiting-cross-sentence-context-neural/). In Proceedings of EMNLP.
53. Lesly Miculicich Werlen, Dhananjay Ram, Nikolaos Pappas, and James Henderson. 2018. [Document-level neural machine translation with hierarchical attention networks](https://md.research.elamur.ai/free/en/papers/miculicich-2018-document-level-neural-machine-translation/). In Proceedings of EMNLP.
54. Haoyu Zhang, Yeyun Gong, Yu Yan, Nan Duan, Jianjun Xu, Ji Wang, Ming Gong, and Ming Zhou. 2019. Pretraining-based natural language generation for text summarization. CoRR, abs/1902.09243. [find](https://scholar.google.com/scholar?q=Haoyu+Zhang%2C+Yeyun+Gong%2C+Yu+Yan%2C+Nan+Duan%2C+Jianjun+Xu%2C+Ji+Wang%2C+Ming+Gong%2C+and+Ming+Zhou%2E+2019%2E+Pretraining%2Dbased+natural+language+generation+for+text+summarization%2E+CoRR%2C+abs%2F1902%2E09243%2E)
55. Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, and Yang Liu. 2018. [Improving the transformer translation model with document-level context](https://md.research.elamur.ai/free/en/papers/zhang-2018-improving-transformer-translation-model-document/). In Proceedings of EMNLP.
56. Zachary M. Ziegler, Luke Melas-Kyriazi, Sebastian Gehrmann, and Alexander M. Rush. 2019. Encoder-agnostic adaptation for conditional language generation. CoRR, abs/1908.06938. — [arXiv:1908.06938](https://arxiv.org/abs/1908.06938) [find](https://scholar.google.com/scholar?q=Zachary+M%2E+Ziegler%2C+Luke+Melas%2DKyriazi%2C+Sebastian+Gehrmann%2C+and+Alexander+M%2E+Rush%2E+2019%2E+Encoder%2Dagnostic+adaptation+for+conditional+language+generation%2E+CoRR%2C+abs%2F1908%2E06938%2E+%E2%80%94+%5BarXiv%3A1908%2E06938%5D%28htt)
57. Yingce Xia, Xu Tan, Fei Tian, Fei Gao, Di He, Weicong Chen, Yang Fan, Linyuan Gong, Yichong Leng, Renqian Luo, et al. [find](https://scholar.google.com/scholar?q=Yingce+Xia%2C+Xu+Tan%2C+Fei+Tian%2C+Fei+Gao%2C+Di+He%2C+Weicong+Chen%2C+Yang+Fan%2C+Linyuan+Gong%2C+Yichong+Leng%2C+Renqian+Luo%2C+et+al%2E)

*Updated 31 October 2026*