---
title: "G-Transformer for Document-Level Machine Translation (2021)"
description: "Bao et al. study why a standard Transformer often fails when an entire document is translated as one… 46 references, cited by 54 papers here."
authors: ["Guangsheng Bao", "Yue Zhang", "Zhiyang Teng", "Boxing Chen", "Weihua Luo"]
year: 2021
venue: "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)"
source: "https://aclanthology.org/2021.acl-long.267/"
doi: "10.18653/v1/2021.acl-long.267"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bao-2021-g-transformer-document-level-machine/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bao-2021-g-transformer-document-level-machine/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / G-Transformer for Document-Level Machine Translation

# G-Transformer for Document-Level Machine Translation

**Authors:** [Guangsheng Bao](https://md.research.elamur.ai/free/en/authors/guangsheng-bao/), [Yue Zhang](https://md.research.elamur.ai/free/en/authors/yue-zhang/), [Zhiyang Teng](https://md.research.elamur.ai/free/en/authors/zhiyang-teng/), [Boxing Chen](https://md.research.elamur.ai/free/en/authors/boxing-chen/) and [Weihua Luo](https://md.research.elamur.ai/free/en/authors/weihua-luo/) · **Year:** 2021 · **Venue:** [Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2021](https://md.research.elamur.ai/free/en/venues/acl/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.acl-long.267/) · [PDF](https://aclanthology.org/2021.acl-long.267.pdf) · [DOI 10.18653/v1/2021.acl-long.267](https://doi.org/10.18653/v1/2021.acl-long.267) · [OpenAlex](https://openalex.org/W3173691187)

**Citations:** 55 citations (OpenAlex, 2026-10-14) · [54](https://md.research.elamur.ai/free/en/papers/bao-2021-g-transformer-document-level-machine/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Bao et al. study why a standard Transformer often fails when an entire document is translated as one sequence. Under plain supervised training, long inputs, small datasets or large models can make BLEU collapse. The authors show this collapse is not overfitting. Instead, training gets trapped in local minima where decoder-to-encoder attention stays flat (high entropy), because longer inputs enlarge the space of possible source alignments.

Their remedy, G-Transformer, still encodes the whole document as one unit. It assigns each sentence a group tag and restricts self-attention and target-to-source attention to matching groups, which adds a locality inductive bias. In a few top layers, group attention is combined with global attention so wider document context is still used.

On three English-German benchmarks (TED, News, Europarl), the model converges faster and more stably than Transformer. It reports state-of-the-art s-BLEU and d-BLEU with both random initialization and pre-training. This makes true document-by-document translation trainable without heavy data augmentation. Code is public at github.com/baoguangsheng/g-transformer.

## Key points
- Failure reproduction on Europarl with the Base model: raising input length from 256 to 512 tokens dropped BLEU from 30.5 to 2.3. Training sets of 20K instances or fewer, and the Big and Large models, also failed with BLEU under 3.
- Data: the Maruf et al. (2019) En-De benchmarks. TED (IWSLT 2017, 0.21M training sentences), News (News Commentary v11, 0.24M) and Europarl (v7, 1.67M) were split into instances of up to 512 tokens, then processed with MOSES and BPE (30,000 merges).
- Ablation of locality biases: the translation (cross-attention) locality bias contributes about 14.68 d-BLEU on average, while the language-modeling locality bias adds about 1.78. Word-dropout adds about 0.37.
- Combined attention in the top 2 layers is enough (33.87 d-BLEU on Europarl). Using only group attention lowers d-BLEU by 0.22/0.09/0.75 on TED/News/Europarl, and using only global attention lowers it by 0.84/0.69/1.00.
- On contrastive discourse tests (OpenSubtitles2018), G-Transformer beats the concat Transformer baseline. It also beats LSTM-T on ellipsis but still trails it on deixis.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 64 works · 14,113 citations
- [Westlake University](https://md.research.elamur.ai/free/en/universities/westlake-university/) — University, Hangzhou, China · 30 works · 7,732 citations
- [Alibaba DAMO Academy](https://md.research.elamur.ai/free/en/companies/alibaba-damo-academy/) — Company, Hangzhou, China · 28 works · 6,115 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works

## Cite

Cite as: Guangsheng Bao, Yue Zhang, Zhiyang Teng, Boxing Chen and Weihua Luo. 2021. G-Transformer for Document-Level Machine Translation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{bao2021g,
  title = {{G-Transformer for Document-Level Machine Translation}},
  author = {Guangsheng Bao, Yue Zhang, Zhiyang Teng, Boxing Chen and Weihua Luo},
  booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.267},
  url = {https://doi.org/10.18653/v1/2021.acl-long.267}
}
```

- [← previous Coreference Resolution without Span Representations](https://md.research.elamur.ai/free/en/papers/kirstain-2021-coreference-resolution-without-span-representations/)
- [next → CDLM: Cross-Document Language Modeling](https://md.research.elamur.ai/free/en/papers/caciularu-2021-cdlm-cross-document-language-modeling/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/)

## References

46 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ruchit Rajeshkumar Agrawal, Marco Turchi, and Matteo Negri. 2018. Contextual handling in neural machine translation: Look behind, ahead and on both sides. In 21st Annual Conference of the European Association for Machine Translation, pages 11–20. [find](https://scholar.google.com/scholar?q=Ruchit+Rajeshkumar+Agrawal%2C+Marco+Turchi%2C+and+Matteo+Negri%2E+2018%2E+Contextual+handling+in+neural+machine+translation%3A+Look+behind%2C+ahead+and+on+both+sides%2E+In+21st+Annual+Conference+of+the+European+Ass)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). arXiv preprint arXiv:1409.0473.
3. Guangsheng Bao and Yue Zhang. 2021. Contextualized rewriting for text summarization. In The Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021. [find](https://scholar.google.com/scholar?q=Guangsheng+Bao+and+Yue+Zhang%2E+2021%2E+Contextualized+rewriting+for+text+summarization%2E+In+The+Thirty%2DFifth+AAAI+Conference+on+Artificial+Intelligence%2C+AAAI+2021%2E)
4. Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. [Longformer: The long-document transformer](https://md.research.elamur.ai/free/en/papers/beltagy-2020-longformer-long-document-transformer/). arXiv:2004.05150.
5. Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. 2016. [Generating sentences from a continuous space](https://md.research.elamur.ai/free/en/papers/bowman-2016-generating-sentences-continuous-space/). In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 10–21, Berlin, Germany. Association for Computational Lingui…
6. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263– 311. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Stephen+A%2E+Della+Pietra%2C+Vincent+J%2E+Della+Pietra%2C+and+Robert+L%2E+Mercer%2E+1993%2E+The+mathematics+of+statistical+machine+translation%3A+Parameter+estimation%2E+Computational+Linguistics%2C+19%282%29)
7. David Chiang. 2005. [A hierarchical phrase-based model for statistical machine translation](https://md.research.elamur.ai/free/en/papers/chiang-2005-hierarchical-phrase-based-model-statistical/). In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 263–270, Ann Arbor, Michigan. Association for Computational Linguistics.
8. David Chiang. 2007. [Hierarchical phrase-based translation](https://md.research.elamur.ai/free/en/papers/chiang-2007-hierarchical-phrase-based-translation/). Computational Linguistics, 33(2):201–228.
9. Michael Collins, Philipp Koehn, and Ivona Kučerová. 2005. [Clause restructuring for statistical machine translation](https://md.research.elamur.ai/free/en/papers/collins-2005-clause-restructuring-statistical-machine-translation/). In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL05), pages 531–540.
10. Eva Martı́nez Garcia, Cristina España-Bonet, and Lluı́s Màrquez. 2015. Document-level machine translation with word vector models. In Proceedings of the 18th Annual Conference of the European Association for Machine Translation, pages 59–66, Antalya, Turkey. [find](https://scholar.google.com/scholar?q=Eva+Mart%C4%B1%CC%81nez+Garcia%2C+Cristina+Espan%CC%83a%2DBonet%2C+and+Llu%C4%B1%CC%81s+Ma%CC%80rquez%2E+2015%2E+Document%2Dlevel+machine+translation+with+word+vector+models%2E+In+Proceedings+of+the+18th+Annual+Conference+of+the+European+Associ)
11. Zhengxian Gong, Min Zhang, and Guodong Zhou. 2011. Cache-based document-level statistical machine translation. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 909–919, Edinburgh, Scotland, UK. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhengxian+Gong%2C+Min+Zhang%2C+and+Guodong+Zhou%2E+2011%2E+Cache%2Dbased+document%2Dlevel+statistical+machine+translation%2E+In+Proceedings+of+the+2011+Conference+on+Empirical+Methods+in+Natural+Language+Processing)
12. Christian Hardmeier. 2014. Discourse in statistical machine translation. Ph.D. thesis, Acta Universitatis Upsaliensis. [find](https://scholar.google.com/scholar?q=Christian+Hardmeier%2E+2014%2E+Discourse+in+statistical+machine+translation%2E+Ph%2ED%2E+thesis%2C+Acta+Universitatis+Upsaliensis%2E)
13. Christian Hardmeier, Sara Stymne, Jörg Tiedemann, and Joakim Nivre. 2013. Docent: A document-level decoder for phrase-based statistical machine translation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 193–198, Sofia, Bulga… [find](https://scholar.google.com/scholar?q=Christian+Hardmeier%2C+Sara+Stymne%2C+Jo%CC%88rg+Tiedemann%2C+and+Joakim+Nivre%2E+2013%2E+Docent%3A+A+document%2Dlevel+decoder+for+phrase%2Dbased+statistical+machine+translation%2E+In+Proceedings+of+the+51st+Annual+Meeting+)
14. Ganesh Jawahar, Benoı̂t Sagot, and Djamé Seddah. 2019. [What does BERT learn about the structure of language?](https://md.research.elamur.ai/free/en/papers/jawahar-2019-what-does-bert-learn-about/) In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3651–3657, Florence, Italy. Association for Computational Linguistics.
15. Marcin Junczys-Dowmunt. 2019. [Microsoft translator at WMT 2019: Towards large-scale document-level neural machine translation](https://md.research.elamur.ai/free/en/papers/dowmunt-2019-microsoft-translator-wmt-2019-large/). In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 225–233, Florence, Italy. Association for Computational Linguisti…
16. Nal Kalchbrenner and Phil Blunsom. 2013. [Recurrent continuous translation models](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1700–1709, Seattle, Washington, USA. Association for Computational Linguistics.
17. Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. 2020. [Reformer: The efficient transformer](https://md.research.elamur.ai/free/en/papers/kitaev-2020-reformer-efficient-transformer/). In International Conference on Learning Representations.
18. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Pro… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondr%CC%8Cej+Bojar%2C+Alexandra+Const)
19. Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pages 28–39, Vancouver. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Philipp+Koehn+and+Rebecca+Knowles%2E+2017%2E+Six+challenges+for+neural+machine+translation%2E+In+Proceedings+of+the+First+Workshop+on+Neural+Machine+Translation%2C+pages+28%E2%80%9339%2C+Vancouver%2E+Association+for+Comp)
20. Philipp Koehn, Franz J. Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 127–133. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Franz+J%2E+Och%2C+and+Daniel+Marcu%2E+2003%2E+Statistical+phrase%2Dbased+translation%2E+In+Proceedings+of+the+2003+Human+Language+Technology+Conference+of+the+North+American+Chapter+of+the+Associat)
21. Samuel Läubli, Rico Sennrich, and Martin Volk. 2018. [Has machine translation achieved human parity? a case for document-level evaluation](https://md.research.elamur.ai/free/en/papers/laubli-2018-has-machine-translation-achieved-human/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4791–4796, Brussels, Belgium. Association for Computational Ling…
22. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. [BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/). In Proceedings of the 58th Annual Meeting of the…
23. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. [fairseq: A fast, extensible toolkit for sequence modeling](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstr…
24. Pierre Lison, Jörg Tiedemann, and Milen Kouylekov. 2018. [OpenSubtitles2018: Statistical rescoring of sentence alignments in large, noisy parallel corpora](https://md.research.elamur.ai/free/en/papers/lison-2018-opensubtitles2018-statistical-rescoring-sentence-alignments/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Res…
25. Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8:726–742. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Jiatao+Gu%2C+Naman+Goyal%2C+Xian+Li%2C+Sergey+Edunov%2C+Marjan+Ghazvininejad%2C+Mike+Lewis%2C+and+Luke+Zettlemoyer%2E+2020%2E+Multilingual+denoising+pre%2Dtraining+for+neural+machine+translation%2E+Transactio)
26. Shuming Ma, Dongdong Zhang, and Ming Zhou. 2020. [A simple and effective unified encoder for document-level machine translation](https://md.research.elamur.ai/free/en/papers/ma-2020-simple-effective-unified-encoder-document/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3505–3511, Online. Association for Computational Linguistics.
27. Sameen Maruf and Gholamreza Haffari. 2018. [Document context neural machine translation with memory networks](https://md.research.elamur.ai/free/en/papers/maruf-2018-document-context-neural-machine-translation/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1275– 1284, Melbourne, Australia. Association for Computational Lingu…
28. Sameen Maruf, André F. T. Martins, and Gholamreza Haffari. 2019. [Selective attention for context-aware neural machine translation](https://md.research.elamur.ai/free/en/papers/maruf-2019-selective-attention-context-aware-neural/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Sh…
29. Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, and James Henderson. 2018a. [Document-level neural machine translation with hierarchical attention networks](https://md.research.elamur.ai/free/en/papers/miculicich-2018-document-level-neural-machine-translation/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2947–2954, Brussels, Belgium. Association for…
30. Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, and James Henderson. 2018b. [Document-level neural machine translation with hierarchical attention networks](https://md.research.elamur.ai/free/en/papers/miculicich-2018-document-level-neural-machine-translation/). In Proceedings of the 2018 Conference Jean Pouget-Abadie, Dzmitry Bahdanau, Bart van Merriënboer, Kyunghyun Cho, and Yoshua Bengio. 2014. Ove…
31. Luigi Rizzi. 2013. Locality. Lingua, 130:169–186. [find](https://scholar.google.com/scholar?q=Luigi+Rizzi%2E+2013%2E+Locality%2E+Lingua%2C+130%3A169%E2%80%93186%2E)
32. Yves Scherrer, Jörg Tiedemann, and Sharid Loáiciga. 2019. Analysing concatenation approaches to document-level NMT in two different domains. In Proceedings of the Fourth Workshop on Discourse in Machine Translation (DiscoMT 2019), pages 51–61, Hong Kong, China. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Yves+Scherrer%2C+Jo%CC%88rg+Tiedemann%2C+and+Sharid+Loa%CC%81iciga%2E+2019%2E+Analysing+concatenation+approaches+to+document%2Dlevel+NMT+in+two+different+domains%2E+In+Proceedings+of+the+Fourth+Workshop+on+Discourse+in+Mac)
33. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Li…
34. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. arXiv preprint arXiv:1409.3215. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+arXiv+preprint+arXiv%3A1409%2E3215%2E)
35. Jörg Tiedemann and Yves Scherrer. 2017. [Neural machine translation with extended context](https://md.research.elamur.ai/free/en/papers/tiedemann-2017-neural-machine-translation-extended-context/). In Proceedings of the Third Workshop on Discourse in Machine Translation, pages 82–92, Copenhagen, Denmark. Association for Computational Linguistics.
36. Zhaopeng Tu, Yang Liu, Zhengdong Lu, Xiaohua Liu, and Hang Li. 2017. [Context gates for neural machine translation](https://md.research.elamur.ai/free/en/papers/tu-2017-context-gates-neural-machine-translation/). Transactions of the Association for Computational Linguistics, 5:87–99.
37. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 6000–6010.
38. Stephan Vogel, Hermann Ney, and Christoph Tillmann. 1996. HMM-based word alignment in statistical translation. In COLING 1996 Volume 2: The 16th International Conference on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Stephan+Vogel%2C+Hermann+Ney%2C+and+Christoph+Tillmann%2E+1996%2E+HMM%2Dbased+word+alignment+in+statistical+translation%2E+In+COLING+1996+Volume+2%3A+The+16th+International+Conference+on+Computational+Linguistics%2E)
39. Elena Voita, Rico Sennrich, and Ivan Titov. 2019a. [Context-aware monolingual repair for neural machine translation](https://md.research.elamur.ai/free/en/papers/voita-2019-context-aware-monolingual-repair-neural/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), page…
40. Elena Voita, Rico Sennrich, and Ivan Titov. 2019b. [When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion](https://md.research.elamur.ai/free/en/papers/voita-2019-when-good-translation-is-wrong/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1198–1212, Florence,…
41. Zhengxin Yang, Jinchao Zhang, Fandong Meng, Shuhao Gu, Yang Feng, and Jie Zhou. 2019. Enhancing context modeling with a query-guided capsule network for document-level translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joi… [find](https://scholar.google.com/scholar?q=Zhengxin+Yang%2C+Jinchao+Zhang%2C+Fandong+Meng%2C+Shuhao+Gu%2C+Yang+Feng%2C+and+Jie+Zhou%2E+2019%2E+Enhancing+context+modeling+with+a+query%2Dguided+capsule+network+for+document%2Dlevel+translation%2E+In+Proceedings+of+t)
42. Lei Yu, Laurent Sartran, Wojciech Stokowiec, Wang Ling, Lingpeng Kong, Phil Blunsom, and Chris Dyer. 2020. Better document-level machine translation with Bayes’ rule. Transactions of the Association for Computational Linguistics, 8:346–360. [find](https://scholar.google.com/scholar?q=Lei+Yu%2C+Laurent+Sartran%2C+Wojciech+Stokowiec%2C+Wang+Ling%2C+Lingpeng+Kong%2C+Phil+Blunsom%2C+and+Chris+Dyer%2E+2020%2E+Better+document%2Dlevel+machine+translation+with+Bayes%E2%80%99+rule%2E+Transactions+of+the+Association+f)
43. Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, and Yang Liu. 2018. [Improving the transformer translation model with document-level context](https://md.research.elamur.ai/free/en/papers/zhang-2018-improving-transformer-translation-model-document/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 533–542, Brussels, Belgium. Ass…
44. Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2016. Gated neural networks for targeted sentiment analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30. [find](https://scholar.google.com/scholar?q=Meishan+Zhang%2C+Yue+Zhang%2C+and+Duy%2DTin+Vo%2E+2016%2E+Gated+neural+networks+for+targeted+sentiment+analysis%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+30%2E)
45. Pei Zhang, Boxing Chen, Niyu Ge, and Kai Fan. 2020. Long-short term masking transformer: A simple but effective baseline for document-level neural machine translation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1081–1087, Online. Associat… [find](https://scholar.google.com/scholar?q=Pei+Zhang%2C+Boxing+Chen%2C+Niyu+Ge%2C+and+Kai+Fan%2E+2020%2E+Long%2Dshort+term+masking+transformer%3A+A+simple+but+effective+baseline+for+document%2Dlevel+neural+machine+translation%2E+In+Proceedings+of+the+2020+Confe)
46. Zaixiang Zheng, Xiang Yue, Shujian Huang, Jiajun Chen, and Alexandra Birch. 2020. Towards making the most of context in neural machine translation. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 3983–3989. [find](https://scholar.google.com/scholar?q=Zaixiang+Zheng%2C+Xiang+Yue%2C+Shujian+Huang%2C+Jiajun+Chen%2C+and+Alexandra+Birch%2E+2020%2E+Towards+making+the+most+of+context+in+neural+machine+translation%2E+In+Proceedings+of+the+Twenty%2DNinth+International+Joi)

*Updated 24 October 2026*