---
title: "Document-level Neural MT — Lopes 2020"
description: "Many context-aware neural MT methods exist, but they have been tested on different languages, datasets and model sizes, so their relative merits were…"
authors: ["António Lopes", "M. Amin Farajian", "Rachel Bawden", "Michael Zhang", "André F. T. Martins"]
year: 2020
venue: "Proceedings of the 22nd Annual Conference of the European Association for Machine Translation"
source: "https://aclanthology.org/2020.eamt-1.24/"
licence: "CC BY-ND 4.0 (EAMT)"
licence_url: "https://creativecommons.org/licenses/by-nd/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/lopes-2020-document-level-neural-mt-systematic/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/lopes-2020-document-level-neural-mt-systematic/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2020](https://md.research.elamur.ai/free/papers/2020/page/2/) / Document-level Neural MT: A Systematic Comparison

# Document-level Neural MT: A Systematic Comparison

**Authors:** António Lopes, [M. Amin Farajian](https://md.research.elamur.ai/free/authors/m-amin-farajian/), [Rachel Bawden](https://md.research.elamur.ai/free/authors/rachel-bawden/), Michael Zhang, [André F. T. Martins](https://md.research.elamur.ai/free/authors/andre-f-t-martins/) · **Year:** 2020 · **Venue:** Proceedings of the 22nd Annual Conference of the European Association for Machine Translation · **Licence:** [CC BY-ND 4.0 (EAMT)](https://creativecommons.org/licenses/by-nd/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.eamt-1.24/) · [PDF](https://aclanthology.org/2020.eamt-1.24.pdf) · [OpenAlex](https://openalex.org/W3046531489)

**Citations:** 39 citations (openalex-snapshot · 2026-10-14) · [75](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Many context-aware neural MT methods exist, but they have been tested on different languages, datasets and model sizes, so their relative merits were unclear. The authors re-implement representative approaches under a shared setup with strong Transformer baselines pre-trained on large sentence-level corpora: Concat2to1/Concat2to2, a multi-source context encoder, and a cache-based model. They also propose the Doc-Star-Transformer, a document-level adaptation of the Star Transformer. It compresses each sentence into a single embedding and lets words attend to these embeddings. Gradient-stopped caching allows it to scale to documents of any length. The systems were tested on two tasks: document translation for EN→FR and EN→DE using public data, and chat translation for EN→FR, EN→DE and EN→PT br using proprietary data. Evaluation combined BLEU, contrastive test sets for pronouns and coherence/cohesion, and human MQM scoring. With strong baselines, context brought only small BLEU gains. The simple Concat2to2 handled discourse phenomena best, and Doc-Star-Transformer received good human ratings. The work also shows that BLEU fails to separate strong systems.

## Key points
- Concat2to2, which passes translated context through the decoder, was the clear winner on contrastive sets. EN→DE pronoun accuracy rose from 45.0% to 70.8%, and one EN→FR anaphora set rose from 50.0% to 82.5%. The multi-source encoder fell below the baseline.
- In chat translation, Doc-Star-Transformer beat the baseline consistently, with the largest effect on the lowest-resource pair, EN→PT br. Feeding it predicted rather than gold target history lowered its scores.
- Human MQM scores for EN→PT br were 92.47 for the baseline, 93.20 for Cache, and 95.20 for Star. For EN→FR, Star scored slightly below the baseline (86.80 vs 87.46). BLEU differences were marginal throughout.
- The authors release a new large-scale EN→FR anaphoric pronoun contrastive test set (GitHub: rbawden/Large-contrastive-pronoun-testset-EN-FR).
- For chat, speaker tags were added so that models can use context from both speakers.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/)

Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/)

## Cited by

Cited by

75 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 71 not yet.

- [Findings of the WMT 2020 Shared Task on Chat Translation](https://md.research.elamur.ai/free/papers/farajian-2020-findings-wmt-2020-shared-task/)
- [Large Language Models Effectively Leverage Document-level Context for Literary Translation, but Critical Errors Persist](https://md.research.elamur.ai/free/papers/karpinska-2023-large-language-models-effectively-leverage/)
- [Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric into a Document-Level Metric](https://md.research.elamur.ai/free/papers/vernikos-2022-embarrassingly-easy-document-level-mt/)
- [DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence](https://md.research.elamur.ai/free/papers/zhao-2023-discoscore-evaluating-text-generation-bert/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lopes2020document,
  title = {{Document-level Neural MT: A Systematic Comparison}},
  author = {Lopes, António and Farajian, M. Amin and Bawden, Rachel and Zhang, Michael and Martins, André F. T.},
  booktitle = {Proceedings of the 22nd Annual Conference of the European Association for Machine Translation},
  year = {2020},
  url = {https://aclanthology.org/2020.eamt-1.24/}
}
```

- [← previous Distilling Neural Networks for Greener and Faster Dependency Parsing](https://md.research.elamur.ai/free/papers/anderson-2020-distilling-neural-networks-greener-faster/)
- [next → DocVQA: A Dataset for VQA on Document Images](https://md.research.elamur.ai/free/papers/mathew-2020-docvqa-dataset-vqa-document-images/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/papers/2020/page/2/)

## References

41 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bahdanau, D., K. Cho, and Y. Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. [find](https://scholar.google.com/scholar?q=Bahdanau%2C+D%2E%2C+K%2E+Cho%2C+and+Y%2E+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+arXiv+preprint+arXiv%3A1409%2E0473%2E) [our card](https://md.research.elamur.ai/free/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
2. Barrault, Loı̈c, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller, Santanu Pal, Matt Post, and Marcos Zampieri. 2019. *Figure 3: Examples of four challenges for MT of Ope… [find](https://scholar.google.com/scholar?q=Barrault%2C+Lo%C4%B1%CC%88c%2C+Ondr%CC%8Cej+Bojar%2C+Marta+R%2E+Costa%2Djussa%CC%80%2C+Christian+Federmann%2C+Mark+Fishel%2C+Yvette+Graham%2C+Barry+Haddow%2C+Matthias+Huck%2C+Philipp+Koehn%2C+Shervin+Malmasi%2C+Christof+Monz%2C+Mathias+Mu%CC%88ller%2C+San) [our card](https://md.research.elamur.ai/free/papers/barrault-2019-findings-2019-conference-machine-translation/)
3. Hardmeier, Christian. 2012. Discourse in Statistical Machine Translation. a survey and a case study. Discours, 11. [find](https://scholar.google.com/scholar?q=Hardmeier%2C+Christian%2E+2012%2E+Discourse+in+Statistical+Machine+Translation%2E+a+survey+and+a+case+study%2E+Discours%2C+11%2E)
4. Bawden, Rachel, Rico Sennrich, Alexandra Birch, and Barry Haddow. 2018. Evaluating Discourse Phenomena in Neural Machine Translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics; Human Language Technologies. [find](https://scholar.google.com/scholar?q=Bawden%2C+Rachel%2C+Rico+Sennrich%2C+Alexandra+Birch%2C+and+Barry+Haddow%2E+2018%2E+Evaluating+Discourse+Phenomena+in+Neural+Machine+Translation%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapte) [our card](https://md.research.elamur.ai/free/papers/bawden-2017-evaluating-discourse-phenomena-neural-machine/)
5. Hassan, Hany, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, et al. 2018. Achieving human parity on automatic Chinese to English news translation. arXiv preprint arXiv:1803.05567. [find](https://scholar.google.com/scholar?q=Hassan%2C+Hany%2C+Anthony+Aue%2C+Chang+Chen%2C+Vishal+Chowdhary%2C+Jonathan+Clark%2C+Christian+Federmann%2C+Xuedong+Huang%2C+Marcin+Junczys%2DDowmunt%2C+William+Lewis%2C+Mu+Li%2C+et+al%2E+2018%2E+Achieving+human+parity+on+automa) [our card](https://md.research.elamur.ai/free/papers/hassan-2018-achieving-human-parity-automatic-chinese/)
6. Bawden, Rachel, Sophie Rosset, Thomas Lavergne, and Eric Bilinski. 2019. DiaBLa: A Corpus of Bilingual Spontaneous Written Dialogues for Machine Translation. [find](https://scholar.google.com/scholar?q=Bawden%2C+Rachel%2C+Sophie+Rosset%2C+Thomas+Lavergne%2C+and+Eric+Bilinski%2E+2019%2E+DiaBLa%3A+A+Corpus+of+Bilingual+Spontaneous+Written+Dialogues+for+Machine+Translation%2E)
7. Junczys-Dowmunt, Marcin. 2019. Microsoft translator at wmt 2019: Towards large-scale document-level neural machine translation. In Proceedings of the Fourth Conference on Machine Translation. [find](https://scholar.google.com/scholar?q=Junczys%2DDowmunt%2C+Marcin%2E+2019%2E+Microsoft+translator+at+wmt+2019%3A+Towards+large%2Dscale+document%2Dlevel+neural+machine+translation%2E+In+Proceedings+of+the+Fourth+Conference+on+Machine+Translation%2E) [our card](https://md.research.elamur.ai/free/papers/dowmunt-2019-microsoft-translator-wmt-2019-large/)
8. Bawden, Rachel. 2018. Going beyond the sentence: Contextual Machine Translation of Dialogue. Ph.D. thesis, LIMSI, CNRS, Université Paris-Sud, Université Paris-Saclay, Orsay, France. [find](https://scholar.google.com/scholar?q=Bawden%2C+Rachel%2E+2018%2E+Going+beyond+the+sentence%3A+Contextual+Machine+Translation+of+Dialogue%2E+Ph%2ED%2E+thesis%2C+LIMSI%2C+CNRS%2C+Universite%CC%81+Paris%2DSud%2C+Universite%CC%81+Paris%2DSaclay%2C+Orsay%2C+France%2E)
9. Klein, Guillaume, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. Open-NMT: Open-source toolkit for neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Klein%2C+Guillaume%2C+Yoon+Kim%2C+Yuntian+Deng%2C+Jean+Senellart%2C+and+Alexander+M%2E+Rush%2E+2017%2E+Open%2DNMT%3A+Open%2Dsource+toolkit+for+neural+machine+translation%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+As)
10. Cettolo, Mauro, Christian Girardi, and Marcello Federico. 2012. WIT 3 : Web Inventory of Transcribed and Translated Talks. In Proceedings of the 16th Conference of the European Association for Machine Translation. [find](https://scholar.google.com/scholar?q=Cettolo%2C+Mauro%2C+Christian+Girardi%2C+and+Marcello+Federico%2E+2012%2E+WIT+3+%3A+Web+Inventory+of+Transcribed+and+Translated+Talks%2E+In+Proceedings+of+the+16th+Conference+of+the+European+Association+for+Machine)
11. Dai, Zihang, Zhilin Yang, Yiming Yang, Jaime G. Carbonell, Quoc V. Le, and Ruslan Salakhutdinov. 2019. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. In Proceedings of the 57th annual meeting on association for computational linguistics. [find](https://scholar.google.com/scholar?q=Dai%2C+Zihang%2C+Zhilin+Yang%2C+Yiming+Yang%2C+Jaime+G%2E+Carbonell%2C+Quoc+V%2E+Le%2C+and+Ruslan+Salakhutdinov%2E+2019%2E+Transformer%2DXL%3A+Attentive+Language+Models+Beyond+a+Fixed%2DLength+Context%2E+In+Proceedings+of+the+57)
12. Dyer, Chris, Victor Chahuneau, and Noah A. Smith. 2013. A simple, fast, and effective reparameterization of IBM model 2. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. [find](https://scholar.google.com/scholar?q=Dyer%2C+Chris%2C+Victor+Chahuneau%2C+and+Noah+A%2E+Smith%2E+2013%2E+A+simple%2C+fast%2C+and+effective+reparameterization+of+IBM+model+2%2E+In+Proceedings+of+the+2013+Conference+of+the+North+American+Chapter+of+the+Asso)
13. Grave, Edouard, Armand Joulin, and Nicolas Usunier. 2016. Improving neural language models with a continuous cache. arXiv preprint arXiv:1612.04426. [find](https://scholar.google.com/scholar?q=Grave%2C+Edouard%2C+Armand+Joulin%2C+and+Nicolas+Usunier%2E+2016%2E+Improving+neural+language+models+with+a+continuous+cache%2E+arXiv+preprint+arXiv%3A1612%2E04426%2E)
14. Guo, Qipeng, Xipeng Qiu, Pengfei Liu, Yunfan Shao, Xiangyang Xue, and Zheng Zhang. 2019. Star-transformer. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. [find](https://scholar.google.com/scholar?q=Guo%2C+Qipeng%2C+Xipeng+Qiu%2C+Pengfei+Liu%2C+Yunfan+Shao%2C+Xiangyang+Xue%2C+and+Zheng+Zhang%2E+2019%2E+Star%2Dtransformer%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Association+for+Co)
15. Koehn, Philipp, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Pr… [find](https://scholar.google.com/scholar?q=Koehn%2C+Philipp%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondr%CC%8Cej+Bojar%2C+Alexandra+Cons)
16. Läubli, Samuel, Rico Sennrich, and Martin Volk. 2018. Has machine translation achieved human parity? a case for document-level evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=La%CC%88ubli%2C+Samuel%2C+Rico+Sennrich%2C+and+Martin+Volk%2E+2018%2E+Has+machine+translation+achieved+human+parity%3F+a+case+for+document%2Dlevel+evaluation%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+i) [our card](https://md.research.elamur.ai/free/papers/laubli-2018-has-machine-translation-achieved-human/)
17. Laubli, Samuel, Sheila Castilho, Graham Neubig, Rico Sennrich, Qinlan Shen, and Antonio Toral. 2020. A set of recommendations for assessing human–machine parity in language translation. Journal of Artificial Intelligence Research (JAIR), 67. [find](https://scholar.google.com/scholar?q=Laubli%2C+Samuel%2C+Sheila+Castilho%2C+Graham+Neubig%2C+Rico+Sennrich%2C+Qinlan+Shen%2C+and+Antonio+Toral%2E+2020%2E+A+set+of+recommendations+for+assessing+human%E2%80%93machine+parity+in+language+translation%2E+Journal+of+Art)
18. Lison, Pierre, Jörg Tiedemann, and Milen Kouylekov. 2018. OpenSubtitles2018: Statistical rescoring of sentence alignments in large, noisy parallel corpora. In Proceedings of the 11th International Conference on Language Resources and Evaluation. [find](https://scholar.google.com/scholar?q=Lison%2C+Pierre%2C+Jo%CC%88rg+Tiedemann%2C+and+Milen+Kouylekov%2E+2018%2E+OpenSubtitles2018%3A+Statistical+rescoring+of+sentence+alignments+in+large%2C+noisy+parallel+corpora%2E+In+Proceedings+of+the+11th+International+Co) [our card](https://md.research.elamur.ai/free/papers/lison-2018-opensubtitles2018-statistical-rescoring-sentence-alignments/)
19. Lommel, Arle Richard. 2013. Multidimensional quality metrics: a flexible system for assessing translation quality Maruf, Sameen, André F. T. Martins, and Gholamreza Haffari. 2018. Contextual neural model for translating bilingual multi-speaker conversations. In Proc. of the 3rd Conference on Machin… [find](https://scholar.google.com/scholar?q=Lommel%2C+Arle+Richard%2E+2013%2E+Multidimensional+quality+metrics%3A+a+flexible+system+for+assessing+translation+quality+Maruf%2C+Sameen%2C+Andre%CC%81+F%2E+T%2E+Martins%2C+and+Gholamreza+Haffari%2E+2018%2E+Contextual+neural+m) [our card](https://md.research.elamur.ai/free/papers/lommel-2013-multidimensional-quality-metrics-flexible-system/)
20. Sennrich, Rico. 2017. How Grammatical is Character-level Neural Machine Translation? In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sennrich%2C+Rico%2E+2017%2E+How+Grammatical+is+Character%2Dlevel+Neural+Machine+Translation%3F+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics%2E)
21. Maruf, Sameen, André F. T. Martins, and Gholamreza Haffari. 2019. Selective Attention for Context-aware Neural Machine Translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. [find](https://scholar.google.com/scholar?q=Maruf%2C+Sameen%2C+Andre%CC%81+F%2E+T%2E+Martins%2C+and+Gholamreza+Haffari%2E+2019%2E+Selective+Attention+for+Context%2Daware+Neural+Machine+Translation%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter)
22. Smith, Karin Sim. 2017. On integrating discourse in machine translation. In Proceedings of the Third Workshop on Discourse in Machine Translation. [find](https://scholar.google.com/scholar?q=Smith%2C+Karin+Sim%2E+2017%2E+On+integrating+discourse+in+machine+translation%2E+In+Proceedings+of+the+Third+Workshop+on+Discourse+in+Machine+Translation%2E)
23. Miculicich, Lesly, Dhananjay Ram, Nikolaos Pappas, and James Henderson. 2018. Document-Level Neural Machine Translation with Hierarchical Attention Networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Miculicich%2C+Lesly%2C+Dhananjay+Ram%2C+Nikolaos+Pappas%2C+and+James+Henderson%2E+2018%2E+Document%2DLevel+Neural+Machine+Translation+with+Hierarchical+Attention+Networks%2E+In+Proceedings+of+the+2018+Conference+on+E) [our card](https://md.research.elamur.ai/free/papers/miculicich-2018-document-level-neural-machine-translation/)
24. Tiedemann, Jörg and Yves Scherrer. 2017. Neural machine translation with extended context. In Proceedings of the 3rd Workshop on Discourse in Machine Translation. [find](https://scholar.google.com/scholar?q=Tiedemann%2C+Jo%CC%88rg+and+Yves+Scherrer%2E+2017%2E+Neural+machine+translation+with+extended+context%2E+In+Proceedings+of+the+3rd+Workshop+on+Discourse+in+Machine+Translation%2E) [our card](https://md.research.elamur.ai/free/papers/tiedemann-2017-neural-machine-translation-extended-context/)
25. Toral, Antonio, Sheila Castilho, Ke Hu, and Andy Way. 2018. Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation. In Proceedings of the Third Conference on Machine Translation. [find](https://scholar.google.com/scholar?q=Toral%2C+Antonio%2C+Sheila+Castilho%2C+Ke+Hu%2C+and+Andy+Way%2E+2018%2E+Attaining+the+Unattainable%3F+Reassessing+Claims+of+Human+Parity+in+Neural+Machine+Translation%2E+In+Proceedings+of+the+Third+Conference+on+Mach) [our card](https://md.research.elamur.ai/free/papers/toral-2018-attaining-unattainable-reassessing-claims-human/)
26. Müller, Mathias, Annette Rios, Elena Voita, and Rico Sennrich. 2018. A large-scale test set for the evaluation of context-aware pronoun translation in neural machine translation. In Proceedings of the Third Conference on Machine Translation. [find](https://scholar.google.com/scholar?q=Mu%CC%88ller%2C+Mathias%2C+Annette+Rios%2C+Elena+Voita%2C+and+Rico+Sennrich%2E+2018%2E+A+large%2Dscale+test+set+for+the+evaluation+of+context%2Daware+pronoun+translation+in+neural+machine+translation%2E+In+Proceedings+of+th) [our card](https://md.research.elamur.ai/free/papers/muller-2018-large-scale-test-set-evaluation/)
27. Tu, Zhaopeng, Yang Liu, Shuming Shi, and Tong Zhang. 2018. Learning to remember translation history with a continuous cache. Transactions of the Association for Computational Linguistics, 6. [find](https://scholar.google.com/scholar?q=Tu%2C+Zhaopeng%2C+Yang+Liu%2C+Shuming+Shi%2C+and+Tong+Zhang%2E+2018%2E+Learning+to+remember+translation+history+with+a+continuous+cache%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+6%2E)
28. Ng, Nathan, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook fair’s wmt19 news translation task submission. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1). [find](https://scholar.google.com/scholar?q=Ng%2C+Nathan%2C+Kyra+Yee%2C+Alexei+Baevski%2C+Myle+Ott%2C+Michael+Auli%2C+and+Sergey+Edunov%2E+2019%2E+Facebook+fair%E2%80%99s+wmt19+news+translation+task+submission%2E+In+Proceedings+of+the+Fourth+Conference+on+Machine+Transl) [our card](https://md.research.elamur.ai/free/papers/ng-2019-facebook-fair-s-wmt19-news/)
29. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. [find](https://scholar.google.com/scholar?q=Vaswani%2C+Ashish%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+process)
30. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Papineni%2C+Kishore%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+BLEU%3A+a+method+for+Automatic+Evaluation+of+Machine+Translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computa) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
31. Voita, Elena, Rico Sennrich, and Ivan Titov. 2019a. Context-aware monolingual repair for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. [find](https://scholar.google.com/scholar?q=Voita%2C+Elena%2C+Rico+Sennrich%2C+and+Ivan+Titov%2E+2019a%2E+Context%2Daware+monolingual+repair+for+neural+machine+translation%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Proc)
32. Pereyra, Gabriel, George Tucker, Jan Chorowski, Łukasz Kaiser, and Geoffrey Hinton. 2017. Regularizing neural networks by penalizing confident output distributions. In Proceedings of the 5th International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Pereyra%2C+Gabriel%2C+George+Tucker%2C+Jan+Chorowski%2C+%C5%81ukasz+Kaiser%2C+and+Geoffrey+Hinton%2E+2017%2E+Regularizing+neural+networks+by+penalizing+confident+output+distributions%2E+In+Proceedings+of+the+5th+Internati) [our card](https://md.research.elamur.ai/free/papers/pereyra-2017-regularizing-neural-networks-penalizing-confident/)
33. Popel, Martin and Ondřej Bojar. 2018. Training tips for the transformer model. The Prague Bulletin of Mathematical Linguistics, 110(1). [find](https://scholar.google.com/scholar?q=Popel%2C+Martin+and+Ondr%CC%8Cej+Bojar%2E+2018%2E+Training+tips+for+the+transformer+model%2E+The+Prague+Bulletin+of+Mathematical+Linguistics%2C+110%281%29%2E)
34. Press, Ofir and Lior Wolf. 2017. Using the output embedding to improve language models. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Press%2C+Ofir+and+Lior+Wolf%2E+2017%2E+Using+the+output+embedding+to+improve+language+models%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/papers/press-2017-output-embedding-improve-language-models/)
35. Rios Gonzales, Annette, Laura Mascarell, and Rico Sennrich. 2017. Improving word sense disambiguation in neural machine translation with sense embeddings. In Proceedings of the 2nd Conference on Machine Translation. [find](https://scholar.google.com/scholar?q=Rios+Gonzales%2C+Annette%2C+Laura+Mascarell%2C+and+Rico+Sennrich%2E+2017%2E+Improving+word+sense+disambiguation+in+neural+machine+translation+with+sense+embeddings%2E+In+Proceedings+of+the+2nd+Conference+on+Machi) [our card](https://md.research.elamur.ai/free/papers/gonzales-2017-improving-word-sense-disambiguation-neural/)
36. Sagot, Benoı̂t. 2010. The lefff, a freely available and large-coverage morphological and syntactic lexicon for French. In Proceedings of the 7th International Conference on Language Resources and Evaluation. [find](https://scholar.google.com/scholar?q=Sagot%2C+Beno%C4%B1%CC%82t%2E+2010%2E+The+lefff%2C+a+freely+available+and+large%2Dcoverage+morphological+and+syntactic+lexicon+for+French%2E+In+Proceedings+of+the+7th+International+Conference+on+Language+Resources+and+Eval) [our card](https://md.research.elamur.ai/free/papers/sagot-2010-lefff-freely-available-large-coverage/)
37. Sennrich, Rico, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proc. of the 54th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sennrich%2C+Rico%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+Proc%2E+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguisti)
38. Voita, Elena, Rico Sennrich, and Ivan Titov. 2019b. When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Voita%2C+Elena%2C+Rico+Sennrich%2C+and+Ivan+Titov%2E+2019b%2E+When+a+good+translation+is+wrong+in+context%3A+Context%2Daware+machine+translation+improves+on+deixis%2C+ellipsis%2C+and+lexical+cohesion%2E+In+Proceedings+of) [our card](https://md.research.elamur.ai/free/papers/voita-2019-when-good-translation-is-wrong/)
39. Wang, Longyue. 2019. Discourse-Aware Neural Machine Translation. Ph.D. thesis, Dublin City University, Dublin, Ireland. [find](https://scholar.google.com/scholar?q=Wang%2C+Longyue%2E+2019%2E+Discourse%2DAware+Neural+Machine+Translation%2E+Ph%2ED%2E+thesis%2C+Dublin+City+University%2C+Dublin%2C+Ireland%2E)
40. Yu, Lei, Laurent Sartran, Wojciech Stokowiec, Wang Ling, Lingpeng Kong, Phil Blunsom, and Chris Dyer. 2019. Putting machine translation in context with the noisy channel model. [find](https://scholar.google.com/scholar?q=Yu%2C+Lei%2C+Laurent+Sartran%2C+Wojciech+Stokowiec%2C+Wang+Ling%2C+Lingpeng+Kong%2C+Phil+Blunsom%2C+and+Chris+Dyer%2E+2019%2E+Putting+machine+translation+in+context+with+the+noisy+channel+model%2E)
41. Zhang, Jiacheng, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, and Yang Liu. 2018. Improving the Transformer Translation Model with Document-Level Context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Zhang%2C+Jiacheng%2C+Huanbo+Luan%2C+Maosong+Sun%2C+Feifei+Zhai%2C+Jingfang+Xu%2C+Min+Zhang%2C+and+Yang+Liu%2E+2018%2E+Improving+the+Transformer+Translation+Model+with+Document%2DLevel+Context%2E+In+Proceedings+of+the+2018+) [our card](https://md.research.elamur.ai/free/papers/zhang-2018-improving-transformer-translation-model-document/)