---
title: "An Empirical Comparison of Domain Adaptation — Chu 2017"
description: "The paper addresses neural machine translation (NMT) for domains with little parallel data."
authors: ["Chenhui Chu", "Raj Dabre", "Sadao Kurohashi"]
year: 2017
venue: "Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)"
source: "https://aclanthology.org/P17-2061/"
doi: "10.18653/v1/p17-2061"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chu-2017-empirical-comparison-domain-adaptation-methods/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chu-2017-empirical-comparison-domain-adaptation-methods/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/) / An Empirical Comparison of Domain Adaptation Methods for Neural Machine Translation

# An Empirical Comparison of Domain Adaptation Methods for Neural Machine Translation

**Authors:** [Chenhui Chu](https://md.research.elamur.ai/free/en/authors/chenhui-chu/), [Raj Dabre](https://md.research.elamur.ai/free/en/authors/raj-dabre/), [Sadao Kurohashi](https://md.research.elamur.ai/free/en/authors/sadao-kurohashi/) · **Year:** 2017 · **Venue:** Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P17-2061/) · [PDF](https://aclanthology.org/P17-2061.pdf) · [DOI 10.18653/v1/p17-2061](https://doi.org/10.18653/v1/p17-2061) · [OpenAlex](https://openalex.org/W2744813330)

**Citations:** 212 citations (openalex-snapshot · 2026-10-14) · [104](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper addresses neural machine translation (NMT) for domains with little parallel data. In these settings plain NMT often loses to phrase-based SMT, and standard fine tuning on a small in-domain corpus quickly overfits. The authors propose "mixed fine tuning", which combines fine tuning with tag-based multi domain NMT. A model is first trained on out-of-domain data. Training then continues on a blend of out-of-domain data and oversampled in-domain data, with every source sentence marked by a domain tag.

They compare this method against fine tuning, multi domain NMT, and stacked variants in two settings:
- **Chinese→English:** NTCIR patents adapted to IWSLT TED talks, where the in-domain corpus is clean and manually built.
- **Chinese→Japanese:** ASPEC scientific text adapted to WIKI-CJ, where the in-domain corpus is noisy and automatically extracted.

Mixed fine tuning gives the best in-domain BLEU on IWSLT and ties with fine tuning on WIKI-CJ. Unlike fine tuning, it preserves out-of-domain quality. The work matters because it is a simple, architecture-free recipe, and it is an early systematic comparison of NMT domain adaptation methods.

## Key points
- **Data:**
  - NTCIR-CE: about 1M training pairs.
  - IWSLT-CE: 209,491 pairs; tested on the 2010–2013 test sets.
  - ASPEC-CJ: 672,315 pairs.
  - WIKI-CJ: 136,013 pairs, automatically mined from Wikipedia using ASPEC-CJ as a seed.
- **Systems:** KyotoNMT with 2-layer LSTMs and a 32k vocabulary. BPE used 30k merges, with a joint BPE model for Chinese–Japanese. The baseline was Moses PBSMT.
- **IWSLT-CE results (average BLEU):**
  - Mixed fine tuning scored 18.01.
  - Fine tuning scored 16.41 and multi domain 16.34.
  - Plain NMT scored 7.87, below SMT at 14.31.
- **WIKI-CJ results:**
  - Fine tuning (37.66), mixed fine tuning (37.57), and mixed fine tuning + fine tuning (37.77) were close.
  - Multi domain trailed at 35.79.
- **Out-of-domain retention:**
  - Fine tuning collapsed out-of-domain quality, with NTCIR-CE falling from 37.11 to 17.37 and ASPEC-CJ from 42.92 to 22.10.
  - Mixed fine tuning kept out-of-domain quality at 37.01 and 42.56 respectively.
  - Adding a final in-domain fine-tuning stage gave no in-domain gain.

## Interesting ideas
- **Keep the source data during adaptation.** Continuing training on in-domain data mixed with the original corpus acts as a regularizer against overfitting and forgetting. This transfers to any low-resource adaptation, such as adapting a general Ukrainian model to a niche domain.
- **Use domain tags as cheap conditioning.** A pseudo-token like "\<2domain>" on the source side lets one model serve several domains without architecture changes. Tags helped both multi domain and mixed fine tuning in these experiments.
- **Oversample the small corpus.** Oversampling the minority domain forces training to give each domain comparable weight. The authors also note a trade-off: without tags, the larger corpus can benefit more from the shared data.
- **Measure forgetting explicitly.** The authors evaluate on the out-of-domain test set as well as the target domain. This makes catastrophic forgetting visible, and it is a useful default check for any adaptation experiment.
- **Treat in-domain data quality as a variable.** Using one clean and one noisy, automatically extracted in-domain corpus showed that the benefit of mixing depends on data quality. This is relevant whenever parallel data is scraped or mined rather than curated.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cited by

Cited by

104 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 99 not yet.

- [Domain Adaptation for Arabic Machine Translation: The Case of Financial Texts](https://md.research.elamur.ai/free/en/papers/alghamdi-2023-domain-adaptation-arabic-machine-translation/)
- [A Survey of Domain Adaptation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/chu-2018-survey-domain-adaptation-neural-machine/)
- [Multiword expressions as discourse markers in Hebrew and Lithuanian](https://md.research.elamur.ai/free/en/papers/disc-2021-oleskeviciene-multiword-expressions-as-discourse-markers/)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/)
- [SemiAdapt: Semi-Supervised and Efficient LoRA-Based Domain Adaptation for Low-Resource Irish Machine Translation with Transformers](https://md.research.elamur.ai/free/en/papers/mcgiff-2026-semiadapt-semi-supervised-efficient-lora/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{chu2017empirical,
  title = {{An Empirical Comparison of Domain Adaptation Methods for Neural Machine Translation}},
  author = {Chu, Chenhui and Dabre, Raj and Kurohashi, Sadao},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2017},
  doi = {10.18653/v1/p17-2061},
  url = {https://doi.org/10.18653/v1/p17-2061}
}
```

- [← previous AMR-to-text Generation with Synchronous Node Replacement Grammar](https://md.research.elamur.ai/free/en/papers/song-2017-amr-text-generation-synchronous-node/)
- [next → An Incremental Parser for Abstract Meaning Representation](https://md.research.elamur.ai/free/en/papers/damonte-2017-incremental-parser-abstract-meaning-representation/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/)

## References

26 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M Cettolo, J Niehues, S Stüker, L Bentivogli, R Cattoni, and M Federico. 2015. The iwslt 2015 evaluation campaign. In Proceedings of the Twelfth International Workshop on Spoken Language Translation (IWSLT). [find](https://scholar.google.com/scholar?q=M+Cettolo%2C+J+Niehues%2C+S+Stu%CC%88ker%2C+L+Bentivogli%2C+R+Cattoni%2C+and+M+Federico%2E+2015%2E+The+iwslt+2015+evaluation+campaign%2E+In+Proceedings+of+the+Twelfth+International+Workshop+on+Spoken+Language+Translation+)
2. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of EMNLP 2004. Association for Computational Linguistics, Barcelona, Spain, pages 388–395. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2004%2E+Statistical+significance+tests+for+machine+translation+evaluation%2E+In+Proceedings+of+EMNLP+2004%2E+Association+for+Computational+Linguistics%2C+Barcelona%2C+Spain%2C+pages+388%E2%80%93395%2E)
3. Kyunghyun Cho, Bart van Merriënboer, Çalar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Lan… [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+van+Merrie%CC%88nboer%2C+C%CC%A7alar+Gu%CC%88lc%CC%A7ehre%2C+Dzmitry+Bahdanau%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014%2E+Learning+phrase+representations+using+rnn+encoder%E2%80%93decoder+for+statis)
4. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proc… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondrej+Bojar%2C+Alexandra+Consta)
5. Chenhui Chu, Raj Dabre, and Sadao Kurohashi. 2016a. Parallel sentence extraction from comparable corpora with neural network features. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). European Language Resources Association (ELRA), Paris, France. [find](https://scholar.google.com/scholar?q=Chenhui+Chu%2C+Raj+Dabre%2C+and+Sadao+Kurohashi%2E+2016a%2E+Parallel+sentence+extraction+from+comparable+corpora+with+neural+network+features%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+)
6. Sadao Kurohashi, Toshihisa Nakamura, Yuji Matsumoto, and Makoto Nagao. 1994. Improvements of Japanese morphological analyzer JUMAN. In Proceedings of the International Workshop on Sharable Natural Language. pages 22–28. [find](https://scholar.google.com/scholar?q=Sadao+Kurohashi%2C+Toshihisa+Nakamura%2C+Yuji+Matsumoto%2C+and+Makoto+Nagao%2E+1994%2E+Improvements+of+Japanese+morphological+analyzer+JUMAN%2E+In+Proceedings+of+the+International+Workshop+on+Sharable+Natural+Lan)
7. Chenhui Chu, Toshiaki Nakazawa, Daisuke Kawahara, and Sadao Kurohashi. 2016b. SCTB: A Chinese treebank in scientific domain. In Proceedings of the 12th Workshop on Asian Language Resources (ALR12). [find](https://scholar.google.com/scholar?q=Chenhui+Chu%2C+Toshiaki+Nakazawa%2C+Daisuke+Kawahara%2C+and+Sadao+Kurohashi%2E+2016b%2E+SCTB%3A+A+Chinese+treebank+in+scientific+domain%2E+In+Proceedings+of+the+12th+Workshop+on+Asian+Language+Resources+%28ALR12%29%2E)
8. The COLING 2016 Organizing Committee, Osaka, Japan, pages 59–67. [http://aclweb.org/anthology/W16-5407](http://aclweb.org/anthology/W16-5407). [↗](http://aclweb.org/anthology/W16-5407)
9. Minh-Thang Luong and Christopher D Manning. 2015. Stanford neural machine translation systems for spoken language domains. In Proceedings of the 12th International Workshop on Spoken Language Translation. Da Nang, Vietnam, pages 76–79. [find](https://scholar.google.com/scholar?q=Minh%2DThang+Luong+and+Christopher+D+Manning%2E+2015%2E+Stanford+neural+machine+translation+systems+for+spoken+language+domains%2E+In+Proceedings+of+the+12th+International+Workshop+on+Spoken+Language+Translat)
10. Fabien Cromieres, Chenhui Chu, Toshiaki Nakazawa, and Sadao Kurohashi. 2016. Kyoto university participation to wat 2016. In Proceedings of the 3rd Workshop on Asian Translation (WAT2016). [find](https://scholar.google.com/scholar?q=Fabien+Cromieres%2C+Chenhui+Chu%2C+Toshiaki+Nakazawa%2C+and+Sadao+Kurohashi%2E+2016%2E+Kyoto+university+participation+to+wat+2016%2E+In+Proceedings+of+the+3rd+Workshop+on+Asian+Translation+%28WAT2016%29%2E)
11. The COLING 2016 Organizing Committee, Osaka, Japan, pages 166–174. [http://aclweb.org/anthology/W16-4616](http://aclweb.org/anthology/W16-4616). [↗](http://aclweb.org/anthology/W16-4616)
12. Lili Mou, Zhao Meng, Rui Yan, Ge Li, Yan Xu, Lu Zhang, and Zhi Jin. 2016. How transferable are neural networks in nlp applications? In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 479–489. http… [find](https://scholar.google.com/scholar?q=Lili+Mou%2C+Zhao+Meng%2C+Rui+Yan%2C+Ge+Li%2C+Yan+Xu%2C+Lu+Zhang%2C+and+Zhi+Jin%2E+2016%2E+How+transferable+are+neural+networks+in+nlp+applications%3F+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natura)
13. Markus Freitag and Yaser Al-Onaizan. 2016. Fast domain adaptation for neural machine translation. arXiv preprint arXiv:1612.06897 . — [arXiv:1612.06897](https://arxiv.org/abs/1612.06897) [find](https://scholar.google.com/scholar?q=Markus+Freitag+and+Yaser+Al%2DOnaizan%2E+2016%2E+Fast+domain+adaptation+for+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1612%2E06897+%2E+%E2%80%94+%5BarXiv%3A1612%2E06897%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1612%2E06897%29)
14. Toshiaki Nakazawa, Hideya Mino, Isao Goto, Graham Neubig, Sadao Kurohashi, and Eiichiro Sumita. 2015. Overview of the 2nd Workshop on Asian Translation. In Proceedings of the 2nd Workshop on Asian Translation (WAT2015). Kyoto, Japan, pages 1–28. [find](https://scholar.google.com/scholar?q=Toshiaki+Nakazawa%2C+Hideya+Mino%2C+Isao+Goto%2C+Graham+Neubig%2C+Sadao+Kurohashi%2C+and+Eiichiro+Sumita%2E+2015%2E+Overview+of+the+2nd+Workshop+on+Asian+Translation%2E+In+Proceedings+of+the+2nd+Workshop+on+Asian+Tra)
15. Isao Goto, Ka-Po Chow, Bin Lu, Eiichiro Sumita, and Benjamin K. Tsou. 2013. Overview of the patent machine translation task at the ntcir-10 workshop. In Proceedings of the 10th NTCIR Conference. National Institute of Informatics (NII), Tokyo, Japan, pages 260–286. [find](https://scholar.google.com/scholar?q=Isao+Goto%2C+Ka%2DPo+Chow%2C+Bin+Lu%2C+Eiichiro+Sumita%2C+and+Benjamin+K%2E+Tsou%2E+2013%2E+Overview+of+the+patent+machine+translation+task+at+the+ntcir%2D10+workshop%2E+In+Proceedings+of+the+10th+NTCIR+Conference%2E+Natio)
16. Toshiaki Nakazawa, Manabu Yaguchi, Kiyotaka Uchimoto, Masao Utiyama, Eiichiro Sumita, Sadao Kurohashi, and Hitoshi Isahara. 2016. Aspec: Asian scientific paper excerpt corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). European Language Res… [find](https://scholar.google.com/scholar?q=Toshiaki+Nakazawa%2C+Manabu+Yaguchi%2C+Kiyotaka+Uchimoto%2C+Masao+Utiyama%2C+Eiichiro+Sumita%2C+Sadao+Kurohashi%2C+and+Hitoshi+Isahara%2E+2016%2E+Aspec%3A+Asian+scientific+paper+excerpt+corpus%2E+In+Proceedings+of+the+Te) [our card](https://md.research.elamur.ai/free/en/papers/nakazawa-2016-aspec-asian-scientific-paper-excerpt/)
17. Çaglar Gülçehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loı̈c Barrault, Huei-Chi Lin, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2015. On using monolingual corpora in neural machine translation. CoRR abs/1503.03535. http://arxiv.org/abs/1503.03535. [↗](http://arxiv.org/abs/1503.03535) [our card](https://md.research.elamur.ai/free/en/papers/gulcehre-2015-using-monolingual-corpora-neural-machine/)
18. Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda B. Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016. Google’s multilingual neural machine translation system: Enabling zero-shot translation. CoRR abs/1611.04558… [find](https://scholar.google.com/scholar?q=Melvin+Johnson%2C+Mike+Schuster%2C+Quoc+V%2E+Le%2C+Maxim+Krikun%2C+Yonghui+Wu%2C+Zhifeng+Chen%2C+Nikhil+Thorat%2C+Fernanda+B%2E+Vie%CC%81gas%2C+Martin+Wattenberg%2C+Greg+Corrado%2C+Macduff+Hughes%2C+and+Jeffrey+Dean%2E+2016%2E+Google%E2%80%99s)
19. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Sapporo, Japan, pages 160– 167. [http://www.aclweb.org/anthology/P03-1021](http://www.aclweb.org/anthology/P03-1021). [↗](http://www.aclweb.org/anthology/P03-1021)
20. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. Controlling politeness in neural machine translation via side constraints. In Proceedings of the 2016 Conference of the Catherine Kobus, Josep Crego, and Jean Senellart. 2016. Domain control for neural machine translation. arXiv preprint arXiv… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016a%2E+Controlling+politeness+in+neural+machine+translation+via+side+constraints%2E+In+Proceedings+of+the+2016+Conference+of+the+Catherine+Kobus%2C+Josep+) [our card](https://md.research.elamur.ai/free/en/papers/sennrich-2016-controlling-politeness-neural-machine-translation/)
21. North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, San Diego, California, pages 35–40. [http://www.aclweb.org/anthology/N16-1005](http://www.aclweb.org/anthology/N16-1005). [↗](http://www.aclweb.org/anthology/N16-1005) [our card](https://md.research.elamur.ai/free/en/papers/sennrich-2016-controlling-politeness-neural-machine-translation/)
22. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany,… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016b%2E+Improving+neural+machine+translation+models+with+monolingual+data%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computation)
23. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016c. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, pages… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016c%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Lin)
24. Christophe Servan, Josep Crego, and Jean Senellart. 2016. Domain specialization: a post-training domain adaptation for neural machine translation. arXiv preprint arXiv:1612.06141 . — [arXiv:1612.06141](https://arxiv.org/abs/1612.06141) [find](https://scholar.google.com/scholar?q=Christophe+Servan%2C+Josep+Crego%2C+and+Jean+Senellart%2E+2016%2E+Domain+specialization%3A+a+post%2Dtraining+domain+adaptation+for+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1612%2E06141+%2E+%E2%80%94+%5BarXiv%3A1612%2E06141)
25. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems. MIT Press, Cambridge, MA, USA, NIPS’14, pages 3104–3112. [http://dl.acm.org/citation.cfm?id=2969033.29…](http://dl.acm.org/citation.cfm?id=2969033.29…) [↗](http://dl.acm.org/citation.cfm?id=2969033.29…)
26. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016. pages 1568–1575. http://aclweb… [find](https://scholar.google.com/scholar?q=Barret+Zoph%2C+Deniz+Yuret%2C+Jonathan+May%2C+and+Kevin+Knight%2E+2016%2E+Transfer+learning+for+low%2Dresource+neural+machine+translation%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Lan)