--- title: "Rapid Adaptation of Neural Machine Translation — Neubig 2018" description: "Neubig and Hu study how to build neural MT from a newly needed low-resource source language (LRL) into English that is both accurate and quick to train." authors: ["Graham Neubig", "Junjie Hu"] year: 2018 venue: "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/D18-1103/" doi: "10.18653/v1/d18-1103" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/neubig-2018-rapid-adaptation-neural-machine-translation/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/neubig-2018-rapid-adaptation-neural-machine-translation/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / Rapid Adaptation of Neural Machine Translation to New Languages # Rapid Adaptation of Neural Machine Translation to New Languages **Authors:** [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/), [Junjie Hu](https://md.research.elamur.ai/free/en/authors/junjie-hu/) · **Year:** 2018 · **Venue:** Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/D18-1103/) · [PDF](https://aclanthology.org/D18-1103.pdf) · [DOI 10.18653/v1/d18-1103](https://doi.org/10.18653/v1/d18-1103) · [OpenAlex](https://openalex.org/W2887920589) **Citations:** 185 citations (openalex-snapshot · 2026-10-14) · [135](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Neubig and Hu study how to build neural MT from a newly needed low-resource source language (LRL) into English that is both accurate and quick to train. The motivation is disaster response, where waiting days for a system is not acceptable. Their approach combines multilingual training with cross-lingual transfer: a seed model is trained in advance on many languages and then fine-tuned on the LRL. They compare single-source, bi-source and all-source seed models. They also contrast warm-start, where the LRL is included in seed training, with cold-start, where LRL data arrives only later. Their new technique, similar-language regularization (SLR), adds data from a closely related high-resource language during fine-tuning to limit over-fitting. This is done either by simple corpus concatenation or by balanced mini-batch sampling. Experiments use the 58-language TED corpus and an attentional LSTM model built in xnmt. A universal model already gives usable output with no LRL data at all, and fine-tuning it with SLR works best in both start settings. Pre-trained massively multilingual models are therefore a practical basis for rapid emergency MT. ## Key points - **Language pairs (LRL/helper):** Azerbaijani/Turkish (aze/tur), Belarusian/Russian (bel/rus), Galician/Portuguese (glg/por) and Slovak/Czech (slk/ces). - LRL training sets range from about 4.5k sentences (bel) to 61.5k (slk). - Helper sets range from about 103k to 208k sentences. - **Multilingual seed models:** Bi-source or all-source training beats single-source training by 7–13 BLEU. All multilingual variants clearly outperform the Moses phrase-based baseline. - **Cold-start results:** With zero LRL training data, the all-languages-but-LRL model (All−) reaches up to 15.5 BLEU on glg-eng and beats the Bi− model. Unsupervised NMT (undreamt) scores near 0 BLEU. - **SLR gains:** Similar-language regularization improves on fine-tuning with only LRL data by 1.7 BLEU on average across the 4 settings. More regularization data trades training speed for accuracy. - **Setup:** - Subword vocabularies are learned separately per language with sentencepiece (8,000 units each) and then merged. - This allows LRL subwords to be built only when that language's data arrives. - Code is released at github.com/neubig/rapid-adaptation. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) ## Cited by Cited by 135 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 125 not yet. - [Massively Multilingual Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/aharoni-2019-massively-multilingual-neural-machine-translation/) - [Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges](https://md.research.elamur.ai/free/en/papers/arivazhagan-2019-massively-multilingual-neural-machine-translation/) - [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://md.research.elamur.ai/free/en/papers/artetxe-2018-massively-multilingual-sentence-embeddings-zero/) - [Simple, Scalable Adaptation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/bapna-2019-simple-scalable-adaptation-neural-machine/) - [The FLORES Evaluation Datasets for Low-Resource Machine Translation: Nepali–English and Sinhala–English](https://md.research.elamur.ai/free/en/papers/guzman-2019-flores-evaluation-datasets-low-resource/) - [Choosing Transfer Languages for Cross-Lingual Learning](https://md.research.elamur.ai/free/en/papers/lin-2019-choosing-transfer-languages-cross-lingual/) - [Masked Language Model Scoring](https://md.research.elamur.ai/free/en/papers/salazar-2020-masked-language-model-scoring/) - [Cross-Cultural Similarity Features for Cross-Lingual Transfer Learning of Pragmatically Motivated Tasks](https://md.research.elamur.ai/free/en/papers/sun-2021-cross-cultural-similarity-features-cross/) - [Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing](https://md.research.elamur.ai/free/en/papers/thompson-2020-automatic-machine-translation-evaluation-many/) - [Handling Syntactic Divergence in Low-resource Machine Translation](https://md.research.elamur.ai/free/en/papers/zhou-2019-handling-syntactic-divergence-low-resource/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{neubig2018rapid, title = {{Rapid Adaptation of Neural Machine Translation to New Languages}}, author = {Neubig, Graham and Hu, Junjie}, booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing}, year = {2018}, doi = {10.18653/v1/d18-1103}, url = {https://doi.org/10.18653/v1/d18-1103} } ``` - [← previous QUD-Based Annotation of Discourse Structure and Information Structure: Tool and Evaluation](https://md.research.elamur.ai/free/en/papers/kuthy-2018-qud-based-annotation-discourse-structure/) - [next → Reasoning with Sarcasm by Reading In-Between](https://md.research.elamur.ai/free/en/papers/tay-2018-reasoning-sarcasm-reading-between/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) ## References 11 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Graham Neubig, Yuichiroh Matsubayashi, Masato Hagiwara, and Koji Murakami. 2011. Safety information mining - what can NLP do in a disaster -. In Proc. IJCNLP, pages 965–973. [find](https://scholar.google.com/scholar?q=Graham+Neubig%2C+Yuichiroh+Matsubayashi%2C+Masato+Hagiwara%2C+and+Koji+Murakami%2E+2011%2E+Safety+information+mining+%2D+what+can+NLP+do+in+a+disaster+%2D%2E+In+Proc%2E+IJCNLP%2C+pages+965%E2%80%93973%2E) 2. Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2017. Unsupervised neural machine translation. arXiv preprint arXiv:1710.11041. [find](https://scholar.google.com/scholar?q=Mikel+Artetxe%2C+Gorka+Labaka%2C+Eneko+Agirre%2C+and+Kyunghyun+Cho%2E+2017%2E+Unsupervised+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1710%2E11041%2E) [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2017-unsupervised-neural-machine-translation/) 3. Graham Neubig, Matthias Sperber, Xinyi Wang, Matthieu Felix, Austin Matthews, Sarguna Padmanabhan, Ye Qi, Devendra Singh Sachan, Philip Arthur, Pierre Godard, John Hewitt, Rachid Riad, and Liming Wang. 2018. XNMT: The extensible neural machine translation toolkit. In Proc. AMTA, Boston. [find](https://scholar.google.com/scholar?q=Graham+Neubig%2C+Matthias+Sperber%2C+Xinyi+Wang%2C+Matthieu+Felix%2C+Austin+Matthews%2C+Sarguna+Padmanabhan%2C+Ye+Qi%2C+Devendra+Singh+Sachan%2C+Philip+Arthur%2C+Pierre+Godard%2C+John+Hewitt%2C+Rachid+Riad%2C+and+Liming+Wang) 4. Toan Q. Nguyen and David Chiang. 2017. Transfer learning across low-resource, related languages for neural machine translation. pages 296–301. [find](https://scholar.google.com/scholar?q=Toan+Q%2E+Nguyen+and+David+Chiang%2E+2017%2E+Transfer+learning+across+low%2Dresource%2C+related+languages+for+neural+machine+translation%2E+pages+296%E2%80%93301%2E) [our card](https://md.research.elamur.ai/free/en/papers/nguyen-2017-transfer-learning-across-low-resource/) 5. Ye Qi, Devendra Sachan, Matthieu Felix, Sarguna Padmanabhan, and Graham Neubig. 2018. When and why are pre-trained word embeddings useful for neural machine translation? In Proc. NAACL, New Orleans, USA. [find](https://scholar.google.com/scholar?q=Ye+Qi%2C+Devendra+Sachan%2C+Matthieu+Felix%2C+Sarguna+Padmanabhan%2C+and+Graham+Neubig%2E+2018%2E+When+and+why+are+pre%2Dtrained+word+embeddings+useful+for+neural+machine+translation%3F+In+Proc%2E+NAACL%2C+New+Orleans%2C+U) [our card](https://md.research.elamur.ai/free/en/papers/qi-2018-when-why-are-pre-trained/) 6. Tanja Schultz and Alan W Black. 2006. Challenges with rapid adaptation of speech translation systems to new language pairs. In Proc. ICASSP, volume 5. IEEE. [find](https://scholar.google.com/scholar?q=Tanja+Schultz+and+Alan+W+Black%2E+2006%2E+Challenges+with+rapid+adaptation+of+speech+translation+systems+to+new+language+pairs%2E+In+Proc%2E+ICASSP%2C+volume+5%2E+IEEE%2E) 7. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proc. ACL, pages 1715–1725. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+Proc%2E+ACL%2C+pages+1715%E2%80%931725%2E) 8. Kate Starbird, Grace Muzny, and Leysia Palen. 2012. Learning from the crowd: Collaborative filtering techniques for identifying on-the-ground Twitterers during mass disruptions. In Proc. ISCRAM. [find](https://scholar.google.com/scholar?q=Kate+Starbird%2C+Grace+Muzny%2C+and+Leysia+Palen%2E+2012%2E+Learning+from+the+crowd%3A+Collaborative+filtering+techniques+for+identifying+on%2Dthe%2Dground+Twitterers+during+mass+disruptions%2E+In+Proc%2E+ISCRAM%2E) 9. Jörg Tiedemann. 2018. Emerging language spaces learned from massively multilingual corpora. arXiv preprint arXiv:1802.00273. — [arXiv:1802.00273](https://arxiv.org/abs/1802.00273) [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann%2E+2018%2E+Emerging+language+spaces+learned+from+massively+multilingual+corpora%2E+arXiv+preprint+arXiv%3A1802%2E00273%2E+%E2%80%94+%5BarXiv%3A1802%2E00273%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1802%2E00273%29) 10. Sarah Vieweg, Amanda L Hughes, Kate Starbird, and Leysia Palen. 2010. Microblogging during two natural hazards events: what Twitter may contribute to situational awareness. In Proc. CHI, pages 1079– 1088. [find](https://scholar.google.com/scholar?q=Sarah+Vieweg%2C+Amanda+L+Hughes%2C+Kate+Starbird%2C+and+Leysia+Palen%2E+2010%2E+Microblogging+during+two+natural+hazards+events%3A+what+Twitter+may+contribute+to+situational+awareness%2E+In+Proc%2E+CHI%2C+pages+1079%E2%80%93+1) 11. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proc. EMNLP, pages 1568–1575. [find](https://scholar.google.com/scholar?q=Barret+Zoph%2C+Deniz+Yuret%2C+Jonathan+May%2C+and+Kevin+Knight%2E+2016%2E+Transfer+learning+for+low%2Dresource+neural+machine+translation%2E+In+Proc%2E+EMNLP%2C+pages+1568%E2%80%931575%2E)