---
title: "Findings of the WMT 2020 Shared Task on Chat — Farajian 2020"
description: "This report presents the inaugural WMT competition devoted to translating chat."
authors: ["M. Amin Farajian", "António V. Lopes", "André F. T. Martins", "Sameen Maruf", "Gholamreza Haffari"]
year: 2020
venue: "Proceedings of the Fifth Conference on Machine Translation"
source: "https://aclanthology.org/2020.wmt-1.3/"
doi: "10.18653/v1/2020.wmt-1.3"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/farajian-2020-findings-wmt-2020-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/farajian-2020-findings-wmt-2020-shared-task/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2020](https://md.research.elamur.ai/free/papers/2020/page/2/) / Findings of the WMT 2020 Shared Task on Chat Translation

# Findings of the WMT 2020 Shared Task on Chat Translation

**Authors:** [M. Amin Farajian](https://md.research.elamur.ai/free/authors/m-amin-farajian/), António V. Lopes, [André F. T. Martins](https://md.research.elamur.ai/free/authors/andre-f-t-martins/), Sameen Maruf, [Gholamreza Haffari](https://md.research.elamur.ai/free/authors/gholamreza-haffari/) · **Year:** 2020 · **Venue:** Proceedings of the Fifth Conference on Machine Translation · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.wmt-1.3/) · [PDF](https://aclanthology.org/2020.wmt-1.3.pdf) · [DOI 10.18653/v1/2020.wmt-1.3](https://doi.org/10.18653/v1/2020.wmt-1.3) · [OpenAlex](https://openalex.org/W7126368594)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14) · [29](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
This report presents the inaugural WMT competition devoted to translating chat. Its focus is bilingual customer-support conversations in which a German-speaking customer and an English-speaking agent each write in their own language. Such chat text is short, informal, noisy and heavy in pronouns, and public data for it has been scarce. To address this, the organizers built BConTrasT. They took English task-oriented dialogues from Taskmaster-1, machine-translated them into German, and had native German editors at Unbabel post-edit the output. Only dialogues containing the anaphoric pronoun "it" were kept, identified with NEURALCOREF.

Six teams sent 14 runs, all covering both directions. Systems were scored with BLEU and TER. Agent-side output (En→De) was also judged through document-level Direct Assessment, carried out by professional translators rather than crowd workers. NaverLabs ranked highest overall, but every system still fell well short of human translations on sentences containing "it". The task matters because it supplies a shared benchmark for context-aware, two-speaker MT in a commercially relevant setting.

## Key points
- **Dataset:** BConTrasT contains about 18k English–German sentence pairs drawn from Taskmaster-1, which covers six domains such as pizza ordering, ride booking and restaurant reservations.
  - Customer lines: 6,216 / 862 / 967 (train / dev / test).
  - Agent lines: 7,629 / 1,040 / 1,133 (train / dev / test).
- **Participation:** 6 teams (NaverLabs, UEdinUppsala, IndTaoWang, Tencent, UMaryland, UJordan) submitted 14 runs. The baseline was Facebook's WMT19 document-level system (Ng et al., 2019), which was scored only with automatic metrics.
- **Data rules:** Extra Taskmaster-1 monolingual data was banned because its sentences are highly repetitive. WMT News parallel data and pre-trained models such as BERT were permitted.
- **Human evaluation:** It covered only En→De, since the German→English side suffers from translationese.
  - On sentences containing "it", systems scored 10–50 DA points below the human reference.
  - NaverLabs, however, beat the human reference on 16+ word sentences, by +5.0 DA points.
- **Use of context:** Three primary systems were purely sentence-level. Context-aware systems appeared to do comparatively better under human judgment than under automatic metrics.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/)

## Cited by

Cited by

29 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 27 not yet.

- [Findings of the 2020 Conference on Machine Translation (WMT20)](https://md.research.elamur.ai/free/papers/barrault-2020-findings-2020-conference-machine-translation/)
- [文档级神经机器翻译综述](https://md.research.elamur.ai/free/papers/lv-2024-document-level-neural/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{farajian2020findings,
  title = {{Findings of the WMT 2020 Shared Task on Chat Translation}},
  author = {Farajian, M. Amin and Lopes, António V. and Martins, André F. T. and Maruf, Sameen and Haffari, Gholamreza},
  booktitle = {Proceedings of the Fifth Conference on Machine Translation},
  year = {2020},
  doi = {10.18653/v1/2020.wmt-1.3},
  url = {https://doi.org/10.18653/v1/2020.wmt-1.3}
}
```

- [← previous Findings of the 2020 Conference on Machine Translation (WMT20)](https://md.research.elamur.ai/free/papers/barrault-2020-findings-2020-conference-machine-translation/)
- [next → Findings of the WMT 2020 Shared Task on Parallel Corpus Filtering and Alignment](https://md.research.elamur.ai/free/papers/koehn-2020-findings-wmt-2020-shared-task/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/papers/2020/page/2/)

## References

36 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Calvin Bao, Yow-Ting Shiue, Chujun Song, Jie S. Li, and Marine Carpuat. 2020. The university of maryland’s submissions to the wmt20 chat translation task: Searching for more data to adapt discourse-aware neural machine translation. In Proceedings of the Fifth Conference on Machine Translation. [find](https://scholar.google.com/scholar?q=Calvin+Bao%2C+Yow%2DTing+Shiue%2C+Chujun+Song%2C+Jie+S%2E+Li%2C+and+Marine+Carpuat%2E+2020%2E+The+university+of+maryland%E2%80%99s+submissions+to+the+wmt20+chat+translation+task%3A+Searching+for+more+data+to+adapt+discourse%2Daw)
2. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+Computation%2C+9%288%29%3A1735%E2%80%931780%2E)
3. Sebastien Jean, Stanislas Lauly, Orhan Firat, and Kyunghyun Cho. 2017. Does neural machine translation benefit from larger context? arXiv preprint arXiv:1704.05135. [find](https://scholar.google.com/scholar?q=Sebastien+Jean%2C+Stanislas+Lauly%2C+Orhan+Firat%2C+and+Kyunghyun+Cho%2E+2017%2E+Does+neural+machine+translation+benefit+from+larger+context%3F+arXiv+preprint+arXiv%3A1704%2E05135%2E)
4. Loı̈c Barrault, Ondřej Bojar, Marta R Costa-Jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, et al. 2019. Findings of the 2019 conference on machine translation (wmt19). In Proceedings of the Fourth Conference on Machine Translati… [find](https://scholar.google.com/scholar?q=Lo%C4%B1%CC%88c+Barrault%2C+Ondr%CC%8Cej+Bojar%2C+Marta+R+Costa%2DJussa%CC%80%2C+Christian+Federmann%2C+Mark+Fishel%2C+Yvette+Graham%2C+Barry+Haddow%2C+Matthias+Huck%2C+Philipp+Koehn%2C+Shervin+Malmasi%2C+et+al%2E+2019%2E+Findings+of+the+2019+con) [our card](https://md.research.elamur.ai/free/papers/barrault-2019-findings-2019-conference-machine-translation/)
5. Marcin Junczys-Dowmunt. 2019. Microsoft translator at wmt 2019: Towards large-scale document-level neural machine translation. In Proceedings of the Fourth Conference on Machine Translation. [find](https://scholar.google.com/scholar?q=Marcin+Junczys%2DDowmunt%2E+2019%2E+Microsoft+translator+at+wmt+2019%3A+Towards+large%2Dscale+document%2Dlevel+neural+machine+translation%2E+In+Proceedings+of+the+Fourth+Conference+on+Machine+Translation%2E) [our card](https://md.research.elamur.ai/free/papers/dowmunt-2019-microsoft-translator-wmt-2019-large/)
6. Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. 2018. Marian: Fast neural machine translation in C++. In Proceedings of ACL 2018, Syste… [find](https://scholar.google.com/scholar?q=Marcin+Junczys%2DDowmunt%2C+Roman+Grundkiewicz%2C+Tomasz+Dwojak%2C+Hieu+Hoang%2C+Kenneth+Heafield%2C+Tom+Neckermann%2C+Frank+Seide%2C+Ulrich+Germann%2C+Alham+Fikri+Aji%2C+Nikolay+Bogoychev%2C+Andre%CC%81+F%2E+T%2E+Martins%2C+and+Alex) [our card](https://md.research.elamur.ai/free/papers/junczys-2018-dowmunt-marian-fast-neural-machine-translation/)
7. Alexandre Bérard, Vassilina Nikoulina, Ioan Calapodescu, and Jerin Philip. 2020. Naver labs europe’s participation in the robustness, chat, and biomedical tasks at wmt 2020. In Proceedings of the Fifth Conference on Machine Translation. [find](https://scholar.google.com/scholar?q=Alexandre+Be%CC%81rard%2C+Vassilina+Nikoulina%2C+Ioan+Calapodescu%2C+and+Jerin+Philip%2E+2020%2E+Naver+labs+europe%E2%80%99s+participation+in+the+robustness%2C+chat%2C+and+biomedical+tasks+at+wmt+2020%2E+In+Proceedings+of+the+Fif)
8. Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar, Arvind Neelakantan, Ben Goodrich, Daniel Duckworth, Semih Yavuz, Amit Dubey, Kyu-Young Kim, and Andy Cedilnik. 2019. Taskmaster-1: Toward a realistic and diverse dialog dataset. In Proceedings of the 2019 Conference on Empirical Methods in Nat… [find](https://scholar.google.com/scholar?q=Bill+Byrne%2C+Karthik+Krishnamoorthi%2C+Chinnadhurai+Sankar%2C+Arvind+Neelakantan%2C+Ben+Goodrich%2C+Daniel+Duckworth%2C+Semih+Yavuz%2C+Amit+Dubey%2C+Kyu%2DYoung+Kim%2C+and+Andy+Cedilnik%2E+2019%2E+Taskmaster%2D1%3A+Toward+a+rea)
9. Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. 2020. Reformer: The efficient transformer. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Nikita+Kitaev%2C+Lukasz+Kaiser%2C+and+Anselm+Levskaya%2E+2020%2E+Reformer%3A+The+efficient+transformer%2E+In+International+Conference+on+Learning+Representations%2E) [our card](https://md.research.elamur.ai/free/papers/kitaev-2020-reformer-efficient-transformer/)
10. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In MT Summit. Citeseer. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2005%2E+Europarl%3A+A+parallel+corpus+for+statistical+machine+translation%2E+In+MT+Summit%2E+Citeseer%2E)
11. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluation the role of bleu in machine translation research. In 11th Conference of the Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. Advances in Neural Information Processing Systems (NeurIPS). [find](https://scholar.google.com/scholar?q=Chris+Callison%2DBurch%2C+Miles+Osborne%2C+and+Philipp+Koehn%2E+2006%2E+Re%2Devaluation+the+role+of+bleu+in+machine+translation+research%2E+In+11th+Conference+of+the+Guillaume+Lample+and+Alexis+Conneau%2E+2019%2E+Cross)
12. Samuel Läubli, Sheila Castilho, Graham Neubig, Rico Sennrich, Qinlan Shen, and Antonio Toral. 2020. A set of recommendations for assessing human– machine parity in language translation. Journal of Artificial Intelligence Research, 67:653–672. [find](https://scholar.google.com/scholar?q=Samuel+La%CC%88ubli%2C+Sheila+Castilho%2C+Graham+Neubig%2C+Rico+Sennrich%2C+Qinlan+Shen%2C+and+Antonio+Toral%2E+2020%2E+A+set+of+recommendations+for+assessing+human%E2%80%93+machine+parity+in+language+translation%2E+Journal+of+Ar)
13. Nikita Moghe, Christian Hardmeier, and Rachel Bawden. 2020. The university of edinburgh-uppsala university’s submission to the wmt 2020 chat translation task. In Proceedings of the Fifth Conference on Machine Translation. [find](https://scholar.google.com/scholar?q=Nikita+Moghe%2C+Christian+Hardmeier%2C+and+Rachel+Bawden%2E+2020%2E+The+university+of+edinburgh%2Duppsala+university%E2%80%99s+submission+to+the+wmt+2020+chat+translation+task%2E+In+Proceedings+of+the+Fifth+Conference+on)
14. Samuel Läubli, Rico Sennrich, and Martin Volk. 2018. Has machine translation achieved human parity? a case for document-level evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4791–4796, Brussels, Belgium. Association for Computational Ling… [find](https://scholar.google.com/scholar?q=Samuel+La%CC%88ubli%2C+Rico+Sennrich%2C+and+Martin+Volk%2E+2018%2E+Has+machine+translation+achieved+human+parity%3F+a+case+for+document%2Dlevel+evaluation%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in) [our card](https://md.research.elamur.ai/free/papers/laubli-2018-has-machine-translation-achieved-human/)
15. Roweida Mohammed, Mahmoud Al-Ayyoub, and Malak Abdullah. 2020. Just system for wmt20 chat translation task. In Proceedings of the Fifth Conference on Machine Translation. [find](https://scholar.google.com/scholar?q=Roweida+Mohammed%2C+Mahmoud+Al%2DAyyoub%2C+and+Malak+Abdullah%2E+2020%2E+Just+system+for+wmt20+chat+translation+task%2E+In+Proceedings+of+the+Fifth+Conference+on+Machine+Translation%2E)
16. Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook fair’s wmt19 news translation task submission. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1). [find](https://scholar.google.com/scholar?q=Nathan+Ng%2C+Kyra+Yee%2C+Alexei+Baevski%2C+Myle+Ott%2C+Michael+Auli%2C+and+Sergey+Edunov%2E+2019%2E+Facebook+fair%E2%80%99s+wmt19+news+translation+task+submission%2E+In+Proceedings+of+the+Fourth+Conference+on+Machine+Transla) [our card](https://md.research.elamur.ai/free/papers/ng-2019-facebook-fair-s-wmt19-news/)
17. Pierre Lison and Jörg Tiedemann. 2016. Opensubtitles2016: Extracting large parallel corpora from movie and tv subtitles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 923–929. [find](https://scholar.google.com/scholar?q=Pierre+Lison+and+Jo%CC%88rg+Tiedemann%2E+2016%2E+Opensubtitles2016%3A+Extracting+large+parallel+corpora+from+movie+and+tv+subtitles%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resources+and) [our card](https://md.research.elamur.ai/free/papers/lison-2016-opensubtitles2016-extracting-large-parallel-corpora/)
18. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstr… [find](https://scholar.google.com/scholar?q=Myle+Ott%2C+Sergey+Edunov%2C+Alexei+Baevski%2C+Angela+Fan%2C+Sam+Gross%2C+Nathan+Ng%2C+David+Grangier%2C+and+Michael+Auli%2E+2019%2E+fairseq%3A+A+fast%2C+extensible+toolkit+for+sequence+modeling%2E+In+Proceedings+of+the+2019) [our card](https://md.research.elamur.ai/free/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
19. António V Lopes, M Amin Farajian, Rachel Bawden, Michael Zhang, and André F T Martins. 2020. Document-level Neural MT: A Systematic Comparison. In 22nd Annual Conference of the European Association for Machine Translation, pages 225–234, Lisboa, Portugal. [find](https://scholar.google.com/scholar?q=Anto%CC%81nio+V+Lopes%2C+M+Amin+Farajian%2C+Rachel+Bawden%2C+Michael+Zhang%2C+and+Andre%CC%81+F+T+Martins%2E+2020%2E+Document%2Dlevel+Neural+MT%3A+A+Systematic+Comparison%2E+In+22nd+Annual+Conference+of+the+European+Association+) [our card](https://md.research.elamur.ai/free/papers/lopes-2020-document-level-neural-mt-systematic/)
20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Comp) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
21. Ryan Lowe, Nissan Pow, Iulian Vlad Serban, and Joelle Pineau. 2015. The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 285–294. [find](https://scholar.google.com/scholar?q=Ryan+Lowe%2C+Nissan+Pow%2C+Iulian+Vlad+Serban%2C+and+Joelle+Pineau%2E+2015%2E+The+ubuntu+dialogue+corpus%3A+A+large+dataset+for+research+in+unstructured+multi%2Dturn+dialogue+systems%2E+In+Proceedings+of+the+16th+Ann) [our card](https://md.research.elamur.ai/free/papers/lowe-2015-ubuntu-dialogue-corpus-large-dataset/)
22. Qingsong Ma, Johnny Wei, Ondřej Bojar, and Yvette Graham. 2019. Results of the WMT19 metrics shared task: Segment-level and strong MT systems pose big challenges. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 62–90, Florence, Italy. Asso… [find](https://scholar.google.com/scholar?q=Qingsong+Ma%2C+Johnny+Wei%2C+Ondr%CC%8Cej+Bojar%2C+and+Yvette+Graham%2E+2019%2E+Results+of+the+WMT19+metrics+shared+task%3A+Segment%2Dlevel+and+strong+MT+systems+pose+big+challenges%2E+In+Proceedings+of+the+Fourth+Confere) [our card](https://md.research.elamur.ai/free/papers/ma-2019-results-wmt19-metrics-shared-task/)
23. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Matt+Post%2E+2018%2E+A+call+for+clarity+in+reporting+BLEU+scores%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%3A+Research+Papers%2C+pages+186%E2%80%93+191%2C+Brussels%2C+Belgium%2E+Association+for+Computa) [our card](https://md.research.elamur.ai/free/papers/post-2018-call-clarity-reporting-bleu-scores/)
24. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and Ralph Weischedel. 2006. A study of translation error rate with targeted human annotation. In In Proceedings of the Association for Machine Transaltion in the Americas (AMTA 2006. [find](https://scholar.google.com/scholar?q=Matthew+Snover%2C+Bonnie+Dorr%2C+Richard+Schwartz%2C+Linnea+Micciulla%2C+and+Ralph+Weischedel%2E+2006%2E+A+study+of+translation+error+rate+with+targeted+human+annotation%2E+In+In+Proceedings+of+the+Association+for+)
25. Sameen Maruf, André F. T. Martins, and Gholamreza Haffari. 2019. Selective attention for context-aware neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Sh… [find](https://scholar.google.com/scholar?q=Sameen+Maruf%2C+Andre%CC%81+F%2E+T%2E+Martins%2C+and+Gholamreza+Haffari%2E+2019%2E+Selective+attention+for+context%2Daware+neural+machine+translation%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+)
26. Jörg Tiedemann and Yves Scherrer. 2017. Neural machine translation with extended context. In Proceedings of the Third Workshop on Discourse in Machine Translation, pages 82–92, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann+and+Yves+Scherrer%2E+2017%2E+Neural+machine+translation+with+extended+context%2E+In+Proceedings+of+the+Third+Workshop+on+Discourse+in+Machine+Translation%2C+pages+82%E2%80%9392%2C+Copenhagen%2C+Denmark%2E+A) [our card](https://md.research.elamur.ai/free/papers/tiedemann-2017-neural-machine-translation-extended-context/)
27. Antonio Toral, Sheila Castilho, Ke Hu, and Andy Way. 2018. Attaining the unattainable? reassessing claims of human parity in neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, WMT 2018, Belgium, Brussels, October 31 - November 1, 2018, pages 1… [find](https://scholar.google.com/scholar?q=Antonio+Toral%2C+Sheila+Castilho%2C+Ke+Hu%2C+and+Andy+Way%2E+2018%2E+Attaining+the+unattainable%3F+reassessing+claims+of+human+parity+in+neural+machine+translation%2E+In+Proceedings+of+the+Third+Conference+on+Machi) [our card](https://md.research.elamur.ai/free/papers/toral-2018-attaining-unattainable-reassessing-claims-human/)
28. Sameen Maruf, André FT Martins, and Gholamreza Haffari. 2018. Contextual neural model for translating bilingual multi-speaker conversations. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 101–112. [find](https://scholar.google.com/scholar?q=Sameen+Maruf%2C+Andre%CC%81+FT+Martins%2C+and+Gholamreza+Haffari%2E+2018%2E+Contextual+neural+model+for+translating+bilingual+multi%2Dspeaker+conversations%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translat)
29. Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, and James Henderson. 2018. Document-level neural machine translation with hierarchical attention networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2947–2954, Brussels, Belgium. Association for… [find](https://scholar.google.com/scholar?q=Lesly+Miculicich%2C+Dhananjay+Ram%2C+Nikolaos+Pappas%2C+and+James+Henderson%2E+2018%2E+Document%2Dlevel+neural+machine+translation+with+hierarchical+attention+networks%2E+In+Proceedings+of+the+2018+Conference+on+Em) [our card](https://md.research.elamur.ai/free/papers/miculicich-2018-document-level-neural-machine-translation/)
30. Zhaopeng Tu, Yang Liu, Shuming Shi, and Tong Zhang. 2018. Learning to remember translation history with a continuous cache. Transactions of the Association for Computational Linguistics, 6:407–420. [find](https://scholar.google.com/scholar?q=Zhaopeng+Tu%2C+Yang+Liu%2C+Shuming+Shi%2C+and+Tong+Zhang%2E+2018%2E+Learning+to+remember+translation+history+with+a+continuous+cache%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+6%3A407%E2%80%93420%2E)
31. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces… [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81+ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+I%2E+Guyon%2C+U%2E+V%2E+Luxburg%2C+S%2E+Bengio%2C+H%2E)
32. Elena Voita, Rico Sennrich, and Ivan Titov. 2019a. Context-aware monolingual repair for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), page… [find](https://scholar.google.com/scholar?q=Elena+Voita%2C+Rico+Sennrich%2C+and+Ivan+Titov%2E+2019a%2E+Context%2Daware+monolingual+repair+for+neural+machine+translation%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Proce)
33. Elena Voita, Rico Sennrich, and Ivan Titov. 2019b. When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1198–1212, Florence,… [find](https://scholar.google.com/scholar?q=Elena+Voita%2C+Rico+Sennrich%2C+and+Ivan+Titov%2E+2019b%2E+When+a+good+translation+is+wrong+in+context%3A+Context%2Daware+machine+translation+improves+on+deixis%2C+ellipsis%2C+and+lexical+cohesion%2E+In+Proceedings+of+) [our card](https://md.research.elamur.ai/free/papers/voita-2019-when-good-translation-is-wrong/)
34. Elena Voita, Pavel Serdyukov, Rico Sennrich, and Ivan Titov. 2018. Context-aware neural machine translation learns anaphora resolution. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Elena+Voita%2C+Pavel+Serdyukov%2C+Rico+Sennrich%2C+and+Ivan+Titov%2E+2018%2E+Context%2Daware+neural+machine+translation+learns+anaphora+resolution%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for) [our card](https://md.research.elamur.ai/free/papers/voita-2018-context-aware-neural-machine-translation/)
35. Longyue Wang, Zhaopeng Tu, Xing Wang, Li Ding, Liang Ding, and Shuming Shi. 2020. Tencent AI Lab machine translation systems for the WMT20 chat translation task. In Proceedings of the Fifth Conference on Machine Translation. [find](https://scholar.google.com/scholar?q=Longyue+Wang%2C+Zhaopeng+Tu%2C+Xing+Wang%2C+Li+Ding%2C+Liang+Ding%2C+and+Shuming+Shi%2E+2020%2E+Tencent+AI+Lab+machine+translation+systems+for+the+WMT20+chat+translation+task%2E+In+Proceedings+of+the+Fifth+Conference)
36. Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, and Yang Liu. 2018. Improving the transformer translation model with document-level context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 533–542, Brussels, Belgium. Ass… [find](https://scholar.google.com/scholar?q=Jiacheng+Zhang%2C+Huanbo+Luan%2C+Maosong+Sun%2C+Feifei+Zhai%2C+Jingfang+Xu%2C+Min+Zhang%2C+and+Yang+Liu%2E+2018%2E+Improving+the+transformer+translation+model+with+document%2Dlevel+context%2E+In+Proceedings+of+the+2018+C) [our card](https://md.research.elamur.ai/free/papers/zhang-2018-improving-transformer-translation-model-document/)