---
title: "Exploiting Monolingual Data at Scale for Neural Machine"
description: "Wu et al. propose a three-stage recipe that uses both kinds of data. 39 references, cited by 48 papers here."
authors: ["Lijun Wu", "Yiren Wang", "Yingce Xia", "Tao Qin", "Jianhuang Lai", "Tie-Yan Liu"]
year: 2019
venue: "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)"
source: "https://aclanthology.org/D19-1430/"
doi: "10.18653/v1/d19-1430"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2019-exploiting-monolingual-data-scale-neural/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2019-exploiting-monolingual-data-scale-neural/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/) / Exploiting Monolingual Data at Scale for Neural Machine Translation

# Exploiting Monolingual Data at Scale for Neural Machine Translation

**Authors:** [Lijun Wu](https://md.research.elamur.ai/free/en/authors/lijun-wu/), [Yiren Wang](https://md.research.elamur.ai/free/en/authors/yiren-wang/), [Yingce Xia](https://md.research.elamur.ai/free/en/authors/yingce-xia/), [Tao Qin](https://md.research.elamur.ai/free/en/authors/tao-qin/), [Jianhuang Lai](https://md.research.elamur.ai/free/en/authors/jianhuang-lai/) and [Tie-Yan Liu](https://md.research.elamur.ai/free/en/authors/tie-yan-liu/) · **Year:** 2019 · **Venue:** [Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2019](https://md.research.elamur.ai/free/en/venues/emnlp/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D19-1430/) · [PDF](https://aclanthology.org/D19-1430.pdf) · [DOI 10.18653/v1/d19-1430](https://doi.org/10.18653/v1/d19-1430) · [OpenAlex](https://openalex.org/W2970311224)

**Citations:** 54 citations (OpenAlex, 2026-10-14) · 48 in our library, of the 141,840 papers whose bibliographies we read

## Summary
Back translation (BT) is a standard way to use target-language monolingual text in neural machine translation. How source-language monolingual text can help has received much less study, especially at large scale. Wu et al. propose a three-stage recipe that uses both kinds of data. First, forward and backward NMT models trained on genuine bitext translate source-side and target-side monolingual sentences, producing two synthetic parallel sets. Second, a model is trained on the genuine bitext plus all synthetic pairs, with every synthetic source sentence corrupted by word masking, deletion and local shuffling. Third, this model is fine-tuned on genuine bitext together with a smaller, uncorrupted batch of synthetic data. The authors report state-of-the-art results on WMT English↔German (newstest 2016–2018) and German→French (newstest2019). Ablations show that the noised combined data keeps improving results as it grows to 120M sentences. Plain BT stops helping beyond about 20M sentences, and source-side data alone brings no gain. The method offers a scalable way to turn abundant monolingual text in both languages into better translation quality.

## Key points
- **Noise operator** (following Edunov et al., 2018): each word is replaced with "\<UNK>" with probability 0.1 and dropped with probability 0.1. Words are also shuffled locally, moving no more than three positions.
- **Scaling behaviour:**
  - Clean BT data improves results at 20M sentences, about the size of the bitext, but hurts as it grows to 60M and 120M.
  - Clean forward-translated source-side data never beats the bilingual baseline.
  - Noised combined data (equal source and target shares, 30M → 120M) improves steadily on all test sets.
- **Noising matters:** noised synthetic data outperforms both clean data and randomly sampled generation. Clean pretraining followed by fine-tuning also helps, but it remains below noised pretraining.
- **Fine-tuning data:** fine-tuning on a separately generated synthetic set works better than reusing a subset of the noised-training data. On En→De newstest2018, the scores were 49.2 vs. 48.0 detokenized SacreBLEU.
- **Benchmarks:** state-of-the-art results are reported on WMT16/17/18 English↔German and WMT19 German→French, using Transformer-based models and up to 120M monolingual sentences.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 240 works · 123,537 citations
- [University of Illinois Urbana-Champaign](https://md.research.elamur.ai/free/en/universities/university-of-illinois-urbana-champaign/) — University, Urbana, United States · 108 works · 67,432 citations
- [Microsoft Research Asia](https://md.research.elamur.ai/free/en/companies/microsoft-research-asia/) — Company, Beijing, China · 54 works · 59,921 citations
- [Sun Yat-sen University](https://md.research.elamur.ai/free/en/universities/sun-yat-sen-university/) — University, Guangzhou, China · 23 works · 8,980 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works

## Cite

Cite as: Lijun Wu, Yiren Wang, Yingce Xia, Tao Qin, Jianhuang Lai and Tie-Yan Liu. 2019. Exploiting Monolingual Data at Scale for Neural Machine Translation. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wu2019exploiting,
  title = {{Exploiting Monolingual Data at Scale for Neural Machine Translation}},
  author = {Lijun Wu, Yiren Wang, Yingce Xia, Tao Qin, Jianhuang Lai and Tie-Yan Liu},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1430},
  url = {https://doi.org/10.18653/v1/d19-1430}
}
```

- [← previous Exact Hard Monotonic Attention for Character-Level Transduction](https://md.research.elamur.ai/free/en/papers/wu-2019-exact-hard-monotonic-attention-character/)
- [next → Findings of the WMT 2019 Shared Task on Automatic Post-Editing](https://md.research.elamur.ai/free/en/papers/chatterjee-2019-findings-wmt-2019-shared-task/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/)

## References

39 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2017. [Unsupervised neural machine translation](https://md.research.elamur.ai/free/en/papers/artetxe-2017-unsupervised-neural-machine-translation/). arXiv preprint arXiv:1710.11041.
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). arXiv preprint arXiv:1409.0473.
3. Franck Burlot and François Yvon. 2019. Using monolingual data in neural machine translation: a systematic study. arXiv preprint arXiv:1903.11437. [find](https://scholar.google.com/scholar?q=Franck+Burlot+and+Franc%CC%A7ois+Yvon%2E+2019%2E+Using+monolingual+data+in+neural+machine+translation%3A+a+systematic+study%2E+arXiv+preprint+arXiv%3A1903%2E11437%2E)
4. Yong Cheng, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. [Semi-supervised learning for neural machine translation](https://md.research.elamur.ai/free/en/papers/cheng-2016-semi-supervised-learning-neural-machine/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1965–1974.
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). NAACL.
6. Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 489–500. [find](https://scholar.google.com/scholar?q=Sergey+Edunov%2C+Myle+Ott%2C+Michael+Auli%2C+and+David+Grangier%2E+2018%2E+Understanding+back%2Dtranslation+at+scale%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pag)
7. Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. [Convolutional sequence to sequence learning](https://md.research.elamur.ai/free/en/papers/gehring-2017-convolutional-sequence-sequence-learning/). In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1243–1252. JMLR. org.
8. Miguel Graça, Yunsu Kim, Julian Schamper, Jiahui Geng, and Hermann Ney. 2018. The rwth aachen university english-german and german-english [unsupervised neural machine translation](https://md.research.elamur.ai/free/en/papers/artetxe-2017-unsupervised-neural-machine-translation/) systems for wmt 2018. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 377–385.
9. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor OK Li. 2016. [Incorporating copying mechanism in sequence-to-sequence learning](https://md.research.elamur.ai/free/en/papers/gu-2016-incorporating-copying-mechanism-sequence-sequence/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1631–1640.
10. Caglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loic Barrault, Huei-Chi Lin, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2015. [On using monolingual corpora in neural machine translation](https://md.research.elamur.ai/free/en/papers/gulcehre-2015-using-monolingual-corpora-neural-machine/). arXiv preprint arXiv:1503.03535.
11. Caglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, and Yoshua Bengio. 2017. On integrating a language model into neural machine translation. Computer Speech & Language, 45:137–148. [find](https://scholar.google.com/scholar?q=Caglar+Gulcehre%2C+Orhan+Firat%2C+Kelvin+Xu%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2017%2E+On+integrating+a+language+model+into+neural+machine+translation%2E+Computer+Speech+%26+Language%2C+45%3A137%E2%80%93148%2E)
12. Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, et al. 2018. [Achieving human parity on automatic chinese to english news translation](https://md.research.elamur.ai/free/en/papers/hassan-2018-achieving-human-parity-automatic-chinese/). arXiv preprint arXiv:1803.05567.
13. Di He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, and Wei-Ying Ma. 2016. Dual learning for machine translation. In Advances in Neural Information Processing Systems, pages 820–828. [find](https://scholar.google.com/scholar?q=Di+He%2C+Yingce+Xia%2C+Tao+Qin%2C+Liwei+Wang%2C+Nenghai+Yu%2C+Tie%2DYan+Liu%2C+and+Wei%2DYing+Ma%2E+2016%2E+Dual+learning+for+machine+translation%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+820%E2%80%93828%2E)
14. Mara Chinea-Rios, Alvaro Peris, and Francisco Casacuberta. 2017. Adapting neural machine translation with parallel synthetic data. In Proceedings of the Second Conference on Machine Translation, pages 138–147. [find](https://scholar.google.com/scholar?q=Mara+Chinea%2DRios%2C+Alvaro+Peris%2C+and+Francisco+Casacuberta%2E+2017%2E+Adapting+neural+machine+translation+with+parallel+synthetic+data%2E+In+Proceedings+of+the+Second+Conference+on+Machine+Translation%2C+pages)
15. Marcin Junczys-Dowmunt. 2018. Microsoft’s submission to the wmt2018 news translation task: How i learned to stop worrying and love the data. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 425–430. [find](https://scholar.google.com/scholar?q=Marcin+Junczys%2DDowmunt%2E+2018%2E+Microsoft%E2%80%99s+submission+to+the+wmt2018+news+translation+task%3A+How+i+learned+to+stop+worrying+and+love+the+data%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translati)
16. Ryan Cotterell and Julia Kreutzer. 2018. Explaining and generalizing back-translation through wake-sleep. arXiv preprint arXiv:1806.04402. — [arXiv:1806.04402](https://arxiv.org/abs/1806.04402) [find](https://scholar.google.com/scholar?q=Ryan+Cotterell+and+Julia+Kreutzer%2E+2018%2E+Explaining+and+generalizing+back%2Dtranslation+through+wake%2Dsleep%2E+arXiv+preprint+arXiv%3A1806%2E04402%2E+%E2%80%94+%5BarXiv%3A1806%2E04402%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1806%2E04402%29)
17. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2E)
18. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th annual meeting of the association for… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+et+al%2E+2007%2E+Moses%3A+Open+source+toolkit+fo)
19. Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2017. [Unsupervised machine translation using monolingual corpora only](https://md.research.elamur.ai/free/en/papers/lample-2017-unsupervised-machine-translation-using-monolingual/). arXiv preprint arXiv:1711.00043.
20. [Marco Lui and Timothy Baldwin. 2012. langid. py: An off-the-shelf language identification tool. In Proceedings of the ACL 2012 system demonstrations, pages 25–30. Association for Computational Linguistics.](https://md.research.elamur.ai/free/en/papers/lui-2012-langid-py-off-shelf-language/)
21. Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. [Effective approaches to attention-based neural machine translation](https://md.research.elamur.ai/free/en/papers/luong-2015-effective-approaches-attention-based-neural/). arXiv preprint arXiv:1508.04025.
22. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. [fairseq: A fast, extensible toolkit for sequence modeling](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/). In Proceedings of NAACL-HLT 2019: Demonstrations.
23. Alberto Poncelas, Dimitar Shterionov, Andy Way, Gideon Maillette de Buy Wenniger, and Peyman Passban. 2018. [Investigating backtranslation in neural machine translation](https://md.research.elamur.ai/free/en/papers/poncelas-2018-investigating-backtranslation-neural-machine-translation/). In Proceedings of the 21st Annual Conference of the European Association for Machine Translation: 28-30 May 2018, Universitat d’Ala…
24. Matt Post. 2018. [A call for clarity in reporting bleu scores](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/). In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191.
25. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. Improving neural machine translation models with monolingual data. In ACL. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016a%2E+Improving+neural+machine+translation+models+with+monolingual+data%2E+In+ACL%2E)
26. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1715–1725.
27. Felix Stahlberg, Adrià de Gispert, and Bill Byrne. 2018. The university of cambridges machine translation systems for wmt18. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 504–512. [find](https://scholar.google.com/scholar?q=Felix+Stahlberg%2C+Adria%CC%80+de+Gispert%2C+and+Bill+Byrne%2E+2018%2E+The+university+of+cambridges+machine+translation+systems+for+wmt18%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%3A+Shared+Task)
28. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+neural+information+processing+systems%2C+pages+3104%E2%80%933112%2E)
29. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826. [find](https://scholar.google.com/scholar?q=Christian+Szegedy%2C+Vincent+Vanhoucke%2C+Sergey+Ioffe%2C+Jon+Shlens%2C+and+Zbigniew+Wojna%2E+2016%2E+Rethinking+the+inception+architecture+for+computer+vision%2E+In+Proceedings+of+the+IEEE+conference+on+computer+v)
30. Nicola Ueffing, Gholamreza Haffari, and Anoop Sarkar. 2007. Transductive learning for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 25–32. [find](https://scholar.google.com/scholar?q=Nicola+Ueffing%2C+Gholamreza+Haffari%2C+and+Anoop+Sarkar%2E+2007%2E+Transductive+learning+for+statistical+machine+translation%2E+In+Proceedings+of+the+45th+Annual+Meeting+of+the+Association+of+Computational+Lin)
31. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in neural information processing systems, pages 5998–6008.
32. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1096–1103. ACM. [find](https://scholar.google.com/scholar?q=Pascal+Vincent%2C+Hugo+Larochelle%2C+Yoshua+Bengio%2C+and+Pierre%2DAntoine+Manzagol%2E+2008%2E+Extracting+and+composing+robust+features+with+denoising+autoencoders%2E+In+Proceedings+of+the+25th+international+confer)
33. Yijun Wang, Yingce Xia, Li Zhao, Jiang Bian, Tao Qin, Guiquan Liu, and Tie-Yan Liu. 2018. Dual transfer learning for neural machine translation with marginal distribution regularization. In Thirty-Second AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Yijun+Wang%2C+Yingce+Xia%2C+Li+Zhao%2C+Jiang+Bian%2C+Tao+Qin%2C+Guiquan+Liu%2C+and+Tie%2DYan+Liu%2E+2018%2E+Dual+transfer+learning+for+neural+machine+translation+with+marginal+distribution+regularization%2E+In+Thirty%2DSec)
34. Yiren Wang, Yingce Xia, Tianyu He, Fei Tian, Tao Qin, ChengXiang Zhai, and Tie-Yan Liu. 2019. Multi-agent dual learning. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yiren+Wang%2C+Yingce+Xia%2C+Tianyu+He%2C+Fei+Tian%2C+Tao+Qin%2C+ChengXiang+Zhai%2C+and+Tie%2DYan+Liu%2E+2019%2E+Multi%2Dagent+dual+learning%2E+In+International+Conference+on+Learning+Representations%2E)
35. Lijun Wu, Fei Tian, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2018. [A study of reinforcement learning for neural machine translation](https://md.research.elamur.ai/free/en/papers/wu-2018-study-reinforcement-learning-neural-machine/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3612–3621.
36. Lijun Wu, Li Zhao, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2017. Sequence prediction with unlabeled data by reward function learning. In IJCAI, pages 3098–3104. [find](https://scholar.google.com/scholar?q=Lijun+Wu%2C+Li+Zhao%2C+Tao+Qin%2C+Jianhuang+Lai%2C+and+Tie%2DYan+Liu%2E+2017%2E+Sequence+prediction+with+unlabeled+data+by+reward+function+learning%2E+In+IJCAI%2C+pages+3098%E2%80%933104%2E)
37. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+et+al%2E+2016%2E+Google%E2%80%99s+neural+machine+translation+system%3A+Bridg)
38. Jiajun Zhang and Chengqing Zong. 2016. [Exploiting source-side monolingual data in neural machine translation](https://md.research.elamur.ai/free/en/papers/zhang-2016-exploiting-source-side-monolingual-data/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1535–1545.
39. Zhirui Zhang, Shujie Liu, Mu Li, Ming Zhou, and Enhong Chen. 2018. Joint training for neural machine translation models with monolingual data. In Thirty-Second AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Zhirui+Zhang%2C+Shujie+Liu%2C+Mu+Li%2C+Ming+Zhou%2C+and+Enhong+Chen%2E+2018%2E+Joint+training+for+neural+machine+translation+models+with+monolingual+data%2E+In+Thirty%2DSecond+AAAI+Conference+on+Artificial+Intelligen)

*Updated 24 October 2026*