---
title: "Facebook AI's WMT21 News Translation Task — Tran 2021"
description: "The paper asks whether one multilingual system can beat strong, carefully built bilingual systems on competitive news translation."
authors: ["Chau Tran", "Shruti Bhosale", "James Cross", "Philipp Koehn", "Sergey Edunov", "Angela Fan"]
year: 2021
venue: "Proceedings of the Sixth Conference on Machine Translation"
source: "https://aclanthology.org/2021.wmt-1.19/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/tran-2021-facebook-ai-s-wmt21-news/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/tran-2021-facebook-ai-s-wmt21-news/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2021](https://md.research.elamur.ai/free/papers/2021/page/2/) / Facebook AI's WMT21 News Translation Task Submission

# Facebook AI's WMT21 News Translation Task Submission

**Authors:** [Chau Tran](https://md.research.elamur.ai/free/authors/chau-tran/), [Shruti Bhosale](https://md.research.elamur.ai/free/authors/shruti-bhosale/), [James Cross](https://md.research.elamur.ai/free/authors/james-cross/), [Philipp Koehn](https://md.research.elamur.ai/free/authors/philipp-koehn/), [Sergey Edunov](https://md.research.elamur.ai/free/authors/sergey-edunov/), [Angela Fan](https://md.research.elamur.ai/free/authors/angela-fan/) · **Year:** 2021 · **Venue:** Proceedings of the Sixth Conference on Machine Translation · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.wmt-1.19/) · [PDF](https://aclanthology.org/2021.wmt-1.19.pdf) · [OpenAlex](https://openalex.org/W6929343747)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14) · [42](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper asks whether one multilingual system can beat strong, carefully built bilingual systems on competitive news translation. Earlier comparisons often used weak baselines or nonstandard data. Facebook AI entered WMT2021 in 14 directions, covering translation into and out of English for seven languages: Japanese, Chinese, Russian, Czech, German, Icelandic, and Hausa. The team built an unconstrained training set from WMT bitext, ccMatrix, ccAligned and OPUS, plus newly mined English–Hausa pairs. They also added large-scale backtranslation of news-like monolingual text selected from Newscrawl and Commoncrawl/CC100. Bilingual and multilingual models received the same techniques. The multilingual side used separate English-to-Many and Many-to-English models in two forms: dense Transformers of up to 4.7B parameters and Sparsely Gated Mixture-of-Experts models. The final system ensembled dense and MoE models, finetuned them on in-domain news, and reranked outputs with a noisy channel model. It improved on every WMT2020 winner by 2.0 BLEU on average and ranked first in 10 directions on automatic metrics. This is evidence that multilingual models can reach state-of-the-art quality for both high- and low-resource languages.

## Key points
- **Hausa data mining:** about 1M extra English–Hausa sentence pairs were mined from monolingual data, using LaBSE embeddings and KNN margin-based scoring.
- **Monolingual data selection:** in-domain text was chosen with Moore–Lewis cross-entropy filtering, which kept roughly 5% of Commoncrawl.
- **Vocabulary:** SentencePiece was trained with temperature upsampling (T=5). Bilingual models used 32,000 units and multilingual models used 128,000.
- **Model architectures:**
  - Dense configurations ranged from 12/12 (480M parameters) to 24/24 Wide (4.7B parameters).
  - MoE layers with top-2 gating replaced the FFN in every other layer.
  - MoE-128 24/24 performed best.
- **Ablation findings:**
  - Backtranslation helped 10 of 14 bilingual directions. In-domain finetuning reversed the small losses seen in high-resource directions.
  - Finetuning added about 3 BLEU into English and 0.7 BLEU out of English.
  - Reranking added about 0.3–0.5 BLEU.
  - Normalizing Chinese punctuation added nearly 5 BLEU.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/)

Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/)

## Cited by

Cited by

42 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 39 not yet.

- [Findings of the 2021 Conference on Machine Translation (WMT21)](https://md.research.elamur.ai/free/papers/akhbardeh-2021-findings-2021-conference-machine-translation/)
- [Iterative Translation Refinement with Large Language Models](https://md.research.elamur.ai/free/papers/chen-2024-iterative-translation-refinement-large-language/)
- [Survey of Low-Resource Machine Translation](https://md.research.elamur.ai/free/papers/haddow-2022-survey-low-resource-machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tran2021facebook,
  title = {{Facebook AI's WMT21 News Translation Task Submission}},
  author = {Tran, Chau and Bhosale, Shruti and Cross, James and Koehn, Philipp and Edunov, Sergey and Fan, Angela},
  booktitle = {Proceedings of the Sixth Conference on Machine Translation},
  year = {2021},
  doi = {10.48448/mqya-vg74},
  url = {https://doi.org/10.48448/mqya-vg74}
}
```

- [← previous Extractive Summarization Considering Discourse and Coreference Relations based on Heterogeneous Graph](https://md.research.elamur.ai/free/papers/huang-2021-extractive-summarization-considering-discourse-coreference/)
- [next → Factual Probing Is MASK: Learning vs. Learning to Recall](https://md.research.elamur.ai/free/papers/zhong-2021-factual-probing-is-mask-learning/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/papers/2021/page/2/)

## References

40 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. Massively multilingual neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3874– 388… [find](https://scholar.google.com/scholar?q=Roee+Aharoni%2C+Melvin+Johnson%2C+and+Orhan+Firat%2E+2019%2E+Massively+multilingual+neural+machine+translation%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Association+for+Compu) [our card](https://md.research.elamur.ai/free/papers/aharoni-2019-massively-multilingual-neural-machine-translation/)
2. Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George Foster, Colin Cherry, et al. 2019. Massively multilingual neural machine translation in the wild: Findings and challenges. arXiv preprint arXiv:1907.05019. [find](https://scholar.google.com/scholar?q=Naveen+Arivazhagan%2C+Ankur+Bapna%2C+Orhan+Firat%2C+Dmitry+Lepikhin%2C+Melvin+Johnson%2C+Maxim+Krikun%2C+Mia+Xu+Chen%2C+Yuan+Cao%2C+George+Foster%2C+Colin+Cherry%2C+et+al%2E+2019%2E+Massively+multilingual+neural+machine+tran) [our card](https://md.research.elamur.ai/free/papers/arivazhagan-2019-massively-multilingual-neural-machine-translation/)
3. Mikel Artetxe and Holger Schwenk. 2019. Margin-based parallel corpus mining with multilingual sentence embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3197–3203. [find](https://scholar.google.com/scholar?q=Mikel+Artetxe+and+Holger+Schwenk%2E+2019%2E+Margin%2Dbased+parallel+corpus+mining+with+multilingual+sentence+embeddings%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Lingui) [our card](https://md.research.elamur.ai/free/papers/artetxe-2019-margin-based-parallel-corpus-mining/)
4. Loïc Barrault, Magdalena Biesialska, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Matthias Huck, Eric Joanis, Tom Kocmi, Philipp Koehn, Chi-kiu Lo, Nikola Ljubešić, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Sant… [find](https://scholar.google.com/scholar?q=Lo%C3%AFc+Barrault%2C+Magdalena+Biesialska%2C+Ondr%CC%8Cej+Bojar%2C+Marta+R%2E+Costa%2Djuss%C3%A0%2C+Christian+Federmann%2C+Yvette+Graham%2C+Roman+Grundkiewicz%2C+Barry+Haddow%2C+Matthias+Huck%2C+Eric+Joanis%2C+Tom+Kocmi%2C+Philipp+Koehn%2C+Ch) [our card](https://md.research.elamur.ai/free/papers/barrault-2020-findings-2020-conference-machine-translation/)
5. Shruti Bhosale, Kyra Yee, Sergey Edunov, and Michael Auli. 2020. Language models not just for pre-training: Fast online neural noisy channel modeling. arXiv preprint arXiv:2011.07164. [find](https://scholar.google.com/scholar?q=Shruti+Bhosale%2C+Kyra+Yee%2C+Sergey+Edunov%2C+and+Michael+Auli%2E+2020%2E+Language+models+not+just+for+pre%2Dtraining%3A+Fast+online+neural+noisy+channel+modeling%2E+arXiv+preprint+arXiv%3A2011%2E07164%2E)
6. Nikolay Bogoychev and Rico Sennrich. 2019. Domain, translationese and noise in synthetic data for neural machine translation. arXiv preprint arXiv:1911.03362. — [arXiv:1911.03362](https://arxiv.org/abs/1911.03362) [find](https://scholar.google.com/scholar?q=Nikolay+Bogoychev+and+Rico+Sennrich%2E+2019%2E+Domain%2C+translationese+and+noise+in+synthetic+data+for+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1911%2E03362%2E+%E2%80%94+%5BarXiv%3A1911%2E03362%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
7. Peng-Jen Chen, Jiajun Shen, Matt Le, Vishrav Chaudhary, Ahmed El-Kishky, Guillaume Wenzek, Myle Ott, and Marc’Aurelio Ranzato. 2019. Facebook ai’s wat19 myanmar-english translation task submission. arXiv preprint arXiv:1910.06848. [find](https://scholar.google.com/scholar?q=Peng%2DJen+Chen%2C+Jiajun+Shen%2C+Matt+Le%2C+Vishrav+Chaudhary%2C+Ahmed+El%2DKishky%2C+Guillaume+Wenzek%2C+Myle+Ott%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2019%2E+Facebook+ai%E2%80%99s+wat19+myanmar%2Denglish+translation+task+submission%2E+ar)
8. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Édouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+%C3%89douard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlingu)
9. Raj Dabre, Atsushi Fujita, and Chenhui Chu. 2019. Exploiting multilingualism through multistage fine-tuning for low-resource neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan… [find](https://scholar.google.com/scholar?q=Raj+Dabre%2C+Atsushi+Fujita%2C+and+Chenhui+Chu%2E+2019%2E+Exploiting+multilingualism+through+multistage+fine%2Dtuning+for+low%2Dresource+neural+machine+translation%2E+In+Proceedings+of+the+2019+Conference+on+Empiri)
10. Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. arXiv preprint arXiv:1808.09381. [find](https://scholar.google.com/scholar?q=Sergey+Edunov%2C+Myle+Ott%2C+Michael+Auli%2C+and+David+Grangier%2E+2018%2E+Understanding+back%2Dtranslation+at+scale%2E+arXiv+preprint+arXiv%3A1808%2E09381%2E)
11. Sergey Edunov, Myle Ott, Marc’Aurelio Ranzato, and Michael Auli. 2019. On the evaluation of machine translation systems trained with back-translation. arXiv preprint arXiv:1908.05204. [find](https://scholar.google.com/scholar?q=Sergey+Edunov%2C+Myle+Ott%2C+Marc%E2%80%99Aurelio+Ranzato%2C+and+Michael+Auli%2E+2019%2E+On+the+evaluation+of+machine+translation+systems+trained+with+back%2Dtranslation%2E+arXiv+preprint+arXiv%3A1908%2E05204%2E) [our card](https://md.research.elamur.ai/free/papers/edunov-2020-evaluation-machine-translation-systems-trained/)
12. Ahmed El-Kishky, Vishrav Chaudhary, Francisco Guzman, and Philipp Koehn. 2020. CCAligned: A massive collection of cross-lingual web-document pairs. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Ahmed+El%2DKishky%2C+Vishrav+Chaudhary%2C+Francisco+Guzman%2C+and+Philipp+Koehn%2E+2020%2E+CCAligned%3A+A+massive+collection+of+cross%2Dlingual+web%2Ddocument+pairs%2E+In+Proc%2E+of+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/kishky-2020-ccaligned-massive-collection-cross-lingual/)
13. Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. [find](https://scholar.google.com/scholar?q=Angela+Fan%2C+Shruti+Bhosale%2C+Holger+Schwenk%2C+Zhiyi+Ma%2C+Ahmed+El%2DKishky%2C+Siddharth+Goyal%2C+Mandeep+Baines%2C+Onur+Celebi%2C+Guillaume+Wenzek%2C+Vishrav+Chaudhary%2C+et+al%2E+2021%2E+Beyond+english%2Dcentric+multilingu)
14. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020. Language-agnostic bert sentence embedding. arXiv preprint arXiv:2007.01852. [find](https://scholar.google.com/scholar?q=Fangxiaoyu+Feng%2C+Yinfei+Yang%2C+Daniel+Cer%2C+Naveen+Arivazhagan%2C+and+Wei+Wang%2E+2020%2E+Language%2Dagnostic+bert+sentence+embedding%2E+arXiv+preprint+arXiv%3A2007%2E01852%2E) [our card](https://md.research.elamur.ai/free/papers/feng-2020-language-agnostic-bert-sentence-embedding/)
15. Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. 2016. Multi-way, multilingual neural machine translation with a shared attention mechanism. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 866–875. [find](https://scholar.google.com/scholar?q=Orhan+Firat%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2016%2E+Multi%2Dway%2C+multilingual+neural+machine+translation+with+a+shared+attention+mechanism%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+C) [our card](https://md.research.elamur.ai/free/papers/firat-2016-multi-way-multilingual-neural-machine/)
16. Yvette Graham, Barry Haddow, and Philipp Koehn. 2020. Translationese in machine translation evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 72–81. [find](https://scholar.google.com/scholar?q=Yvette+Graham%2C+Barry+Haddow%2C+and+Philipp+Koehn%2E+2020%2E+Translationese+in+machine+translation+evaluation%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP)
17. Kenneth Heafield. 2011. Kenlm: faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187–197. [find](https://scholar.google.com/scholar?q=Kenneth+Heafield%2E+2011%2E+Kenlm%3A+faster+and+smaller+language+model+queries%2E+In+Proceedings+of+the+Sixth+Workshop+on+Statistical+Machine+Translation%2C+pages+187%E2%80%93197%2E) [our card](https://md.research.elamur.ai/free/papers/heafield-2011-kenlm-faster-smaller-language-model/)
18. Vu Cong Duy Hoang, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. 2018. Iterative back-translation for neural machine translation. In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, pages 18–24. [find](https://scholar.google.com/scholar?q=Vu+Cong+Duy+Hoang%2C+Philipp+Koehn%2C+Gholamreza+Haffari%2C+and+Trevor+Cohn%2E+2018%2E+Iterative+back%2Dtranslation+for+neural+machine+translation%2E+In+Proceedings+of+the+2nd+Workshop+on+Neural+Machine+Translation) [our card](https://md.research.elamur.ai/free/papers/hoang-2018-iterative-back-translation-neural-machine/)
19. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431. Association for Computation… [find](https://scholar.google.com/scholar?q=Armand+Joulin%2C+Edouard+Grave%2C+Piotr+Bojanowski%2C+and+Tomas+Mikolov%2E+2017%2E+Bag+of+tricks+for+efficient+text+classification%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Associati)
20. Taku Kudo and John Richardson. 2018. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71. [find](https://scholar.google.com/scholar?q=Taku+Kudo+and+John+Richardson%2E+2018%2E+Sentencepiece%3A+A+simple+and+language+independent+subword+tokenizer+and+detokenizer+for+neural+text+processing%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+M) [our card](https://md.research.elamur.ai/free/papers/kudo-2018-sentencepiece-simple-language-independent-subword/)
21. Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. 2020. Gshard: Scaling giant models with conditional computation and automatic sharding. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Dmitry+Lepikhin%2C+HyoukJoong+Lee%2C+Yuanzhong+Xu%2C+Dehao+Chen%2C+Orhan+Firat%2C+Yanping+Huang%2C+Maxim+Krikun%2C+Noam+Shazeer%2C+and+Zhifeng+Chen%2E+2020%2E+Gshard%3A+Scaling+giant+models+with+conditional+computation+and)
22. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
23. Benjamin Marie, Raphael Rubino, and Atsushi Fujita. 2020. Tagged back-translation revisited: Why does it really work? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5990–5997. [find](https://scholar.google.com/scholar?q=Benjamin+Marie%2C+Raphael+Rubino%2C+and+Atsushi+Fujita%2E+2020%2E+Tagged+back%2Dtranslation+revisited%3A+Why+does+it+really+work%3F+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Lin)
24. Robert Moore and William Lewis. 2010. Intelligent selection of language model training data. In Proceedings of the ACL 2010 Conference Short Papers, pages 220–224. [find](https://scholar.google.com/scholar?q=Robert+Moore+and+William+Lewis%2E+2010%2E+Intelligent+selection+of+language+model+training+data%2E+In+Proceedings+of+the+ACL+2010+Conference+Short+Papers%2C+pages+220%E2%80%93224%2E) [our card](https://md.research.elamur.ai/free/papers/moore-2010-intelligent-selection-language-model-training/)
25. Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook FAIR’s WMT19 news translation task submission. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1). [find](https://scholar.google.com/scholar?q=Nathan+Ng%2C+Kyra+Yee%2C+Alexei+Baevski%2C+Myle+Ott%2C+Michael+Auli%2C+and+Sergey+Edunov%2E+2019%2E+Facebook+FAIR%E2%80%99s+WMT19+news+translation+task+submission%2E+In+Proceedings+of+the+Fourth+Conference+on+Machine+Transla) [our card](https://md.research.elamur.ai/free/papers/ng-2019-facebook-fair-s-wmt19-news/)
26. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations. [find](https://scholar.google.com/scholar?q=Myle+Ott%2C+Sergey+Edunov%2C+Alexei+Baevski%2C+Angela+Fan%2C+Sam+Gross%2C+Nathan+Ng%2C+David+Grangier%2C+and+Michael+Auli%2E+2019%2E+fairseq%3A+A+fast%2C+extensible+toolkit+for+sequence+modeling%2E+In+Proceedings+of+NAACL%2DHL) [our card](https://md.research.elamur.ai/free/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
27. Matt Post. 2018. A call for clarity in reporting BLEU scores. CoRR, abs/1804.08771. [find](https://scholar.google.com/scholar?q=Matt+Post%2E+2018%2E+A+call+for+clarity+in+reporting+BLEU+scores%2E+CoRR%2C+abs%2F1804%2E08771%2E) [our card](https://md.research.elamur.ai/free/papers/post-2018-call-clarity-reporting-bleu-scores/)
28. Translation (Volume 1: Research Papers), pages 73– 81. [find](https://scholar.google.com/scholar?q=Translation+%28Volume+1%3A+Research+Papers%29%2C+pages+73%E2%80%93+81%2E)
29. Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Edouard Grave, Armand Joulin, and Angela Fan. 2019. CCMatrix: Mining billions of high-quality parallel sentences on the web. arXiv preprint arXiv:1911.04944. [find](https://scholar.google.com/scholar?q=Holger+Schwenk%2C+Guillaume+Wenzek%2C+Sergey+Edunov%2C+Edouard+Grave%2C+Armand+Joulin%2C+and+Angela+Fan%2E+2019%2E+CCMatrix%3A+Mining+billions+of+high%2Dquality+parallel+sentences+on+the+web%2E+arXiv+preprint+arXiv%3A1911%2E) [our card](https://md.research.elamur.ai/free/papers/schwenk-2019-ccmatrix-mining-billions-high-quality/)
30. Michał Ziemski, Marcin Junczys-Dowmunt, and Bruno Pouliquen. 2016. The united nations parallel corpus v1. 0. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 3530–3534. [find](https://scholar.google.com/scholar?q=Micha%C5%82+Ziemski%2C+Marcin+Junczys%2DDowmunt%2C+and+Bruno+Pouliquen%2E+2016%2E+The+united+nations+parallel+corpus+v1%2E+0%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resources+and+Evaluation+%28) [our card](https://md.research.elamur.ai/free/papers/ziemski-2016-united-nations-parallel-corpus-v1/)
31. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 86–96. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2015%2E+Improving+neural+machine+translation+models+with+monolingual+data%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computationa)
32. Xu Tan, Jiale Chen, Di He, Yingce Xia, QIN Tao, and Tie-Yan Liu. 2019. Multilingual neural machine translation with language clustering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing… [find](https://scholar.google.com/scholar?q=Xu+Tan%2C+Jiale+Chen%2C+Di+He%2C+Yingce+Xia%2C+QIN+Tao%2C+and+Tie%2DYan+Liu%2E+2019%2E+Multilingual+neural+machine+translation+with+language+clustering%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+N)
33. Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020. Multilingual translation with extensible multilingual pretraining and finetuning. arXiv preprint arXiv:2008.00401. [find](https://scholar.google.com/scholar?q=Yuqing+Tang%2C+Chau+Tran%2C+Xian+Li%2C+Peng%2DJen+Chen%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Jiatao+Gu%2C+and+Angela+Fan%2E+2020%2E+Multilingual+translation+with+extensible+multilingual+pretraining+and+finetuning%2E+arXiv)
34. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in OPUS. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 2214–2218. [find](https://scholar.google.com/scholar?q=J%C3%B6rg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+OPUS%2E+In+Proceedings+of+the+Eighth+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9912%29%2C+pages+2214%E2%80%932218%2E)
35. Chau Tran, Yuqing Tang, Xian Li, and Jiatao Gu. 2020. Cross-lingual retrieval for iterative self-supervised training. Advances in Neural Information Processing Systems, 33:2207–2219. [find](https://scholar.google.com/scholar?q=Chau+Tran%2C+Yuqing+Tang%2C+Xian+Li%2C+and+Jiatao+Gu%2E+2020%2E+Cross%2Dlingual+retrieval+for+iterative+self%2Dsupervised+training%2E+Advances+in+Neural+Information+Processing+Systems%2C+33%3A2207%E2%80%932219%2E)
36. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processi)
37. Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. 2019. Ccnet: Extracting high quality monolingual datasets from web crawl data. arXiv preprint arXiv:1911.00359. [find](https://scholar.google.com/scholar?q=Guillaume+Wenzek%2C+Marie%2DAnne+Lachaux%2C+Alexis+Conneau%2C+Vishrav+Chaudhary%2C+Francisco+Guzm%C3%A1n%2C+Armand+Joulin%2C+and+Edouard+Grave%2E+2019%2E+Ccnet%3A+Extracting+high+quality+monolingual+datasets+from+web+crawl+da) [our card](https://md.research.elamur.ai/free/papers/wenzek-2020-ccnet-extracting-high-quality-monolingual/)
38. Kyra Yee, Yann Dauphin, and Michael Auli. 2019. Simple and effective noisy channel modeling for neural machine translation. In Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Kyra+Yee%2C+Yann+Dauphin%2C+and+Michael+Auli%2E+2019%2E+Simple+and+effective+noisy+channel+modeling+for+neural+machine+translation%2E+In+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E)
39. Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020. Improving massively multilingual neural machine translation and zero-shot translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1628– 1639. [find](https://scholar.google.com/scholar?q=Biao+Zhang%2C+Philip+Williams%2C+Ivan+Titov%2C+and+Rico+Sennrich%2E+2020%2E+Improving+massively+multilingual+neural+machine+translation+and+zero%2Dshot+translation%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+th) [our card](https://md.research.elamur.ai/free/papers/zhang-2020-improving-massively-multilingual-neural-machine/)
40. Mike Zhang and Antonio Toral. 2019. The effect of translationese in machine translation test sets. In Proceedings of the Fourth Conference on Machine [find](https://scholar.google.com/scholar?q=Mike+Zhang+and+Antonio+Toral%2E+2019%2E+The+effect+of+translationese+in+machine+translation+test+sets%2E+In+Proceedings+of+the+Fourth+Conference+on+Machine) [our card](https://md.research.elamur.ai/free/papers/zhang-2019-effect-translationese-machine-translation-test/)