---
title: "Multi-Task Learning for Multiple Language — Dong 2015"
description: "Translating one source language into several target languages usually means training a separate system for each pair."
authors: ["Daxiang Dong", "Hua Wu", "Wei He", "Dianhai Yu", "Haifeng Wang"]
year: 2015
venue: "Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)"
source: "https://aclanthology.org/P15-1166/"
doi: "10.3115/v1/p15-1166"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/dong-2015-multi-task-learning-multiple-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dong-2015-multi-task-learning-multiple-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/) / Multi-Task Learning for Multiple Language Translation

# Multi-Task Learning for Multiple Language Translation

**Authors:** [Daxiang Dong](https://md.research.elamur.ai/free/en/authors/daxiang-dong/), [Hua Wu](https://md.research.elamur.ai/free/en/authors/hua-wu/), [Wei He](https://md.research.elamur.ai/free/en/authors/wei-he/), [Dianhai Yu](https://md.research.elamur.ai/free/en/authors/dianhai-yu/), [Haifeng Wang](https://md.research.elamur.ai/free/en/authors/haifeng-wang/) · **Year:** 2015 · **Venue:** Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P15-1166/) · [PDF](https://aclanthology.org/P15-1166.pdf) · [DOI 10.3115/v1/p15-1166](https://doi.org/10.3115/v1/p15-1166) · [OpenAlex](https://openalex.org/W2251743902)

**Citations:** 565 citations (openalex-snapshot · 2026-10-14) · [226](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Translating one source language into several target languages usually means training a separate system for each pair. Statistical phrase-table systems cannot easily share knowledge across pairs, and quality drops sharply for pairs with little parallel data. Dong et al. (Baidu) recast this as multi-task learning on top of the attention-based RNN encoder-decoder of Bahdanau et al. (2014). A single bidirectional-RNN encoder for English is shared across all translation directions, while each target language gets its own decoder with its own attention. Training maximizes the sum of per-pair log-likelihoods, each normalized by the size of that pair's corpus.

The authors tested the approach with English as the source and French, Spanish, Dutch and Portuguese as targets. They used Europarl and WMT 2013 data, in both a large-data setting and a setting where some pairs were artificially reduced. The joint model beat separately trained NMT models in every configuration and also outperformed Moses on WMT 2013. It converged faster and produced better source word embeddings. These results suggest that sharing the encoder lets low-resource pairs benefit from data in high-resource pairs without a pivot language.

## Key points
- **Architecture:** one shared English encoder plus a separate decoder (with its own attention) for each target language. The authors describe this as the first end-to-end NMT system trained in a multi-task framework.
- **WMT 2013 test set (BLEU):**
  - En-Fr rose from 23.89 (single NMT) to 26.02 with the fully trained multi-task model.
  - En-Es rose from 23.28 to 25.31, and reached 25.83 in the partial-data variant.
  - Moses scored 23.83 and 23.58, roughly the level of the single NMT models.
- **Low-resource setting:** En-Nl and En-Pt were sub-sampled to 15% of their bilingual data. Multi-task gains over single models were:
  - En-Es: +1.64
  - En-Fr: +0.67
  - En-Nl: +1.26
  - En-Pt: +1.06
- **Convergence:** on the Europarl common test set, learning curves for the resource-poor pairs improved more quickly under multi-task training. The joint model overtook the single models within three training iterations.
- **Embedding quality:** nearest neighbours of English words from the shared encoder were far more sensible than those from the low-resource single English-Portuguese model. For example, "provide" was closest to "deliver" in the shared encoder but to "though" in the single model.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cited by

Cited by

226 papers of our library cite it (of the 140673 whose bibliographies we read); 27 of them have a page here, 199 not yet.

- [Few-Shot and Zero-Shot Learning for Historical Text Normalization](https://md.research.elamur.ai/free/en/papers/aclbulk-2019-bollmann-few-shot-zero-shot-learning/)
- [Massively Multilingual Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/aharoni-2019-massively-multilingual-neural-machine-translation/)
- [Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges](https://md.research.elamur.ai/free/en/papers/arivazhagan-2019-massively-multilingual-neural-machine-translation/)
- [Simple, Scalable Adaptation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/bapna-2019-simple-scalable-adaptation-neural-machine/)
- [Evaluating prose style transfer with the Bible](https://md.research.elamur.ai/free/en/papers/carlson-2017-evaluating-prose-style-transfer-bible/)
- [Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism](https://md.research.elamur.ai/free/en/papers/firat-2016-multi-way-multilingual-neural-machine/)
- [Zero-Resource Translation with Multi-Lingual Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/firat-2016-zero-resource-translation-multi-lingual/)
- [Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder](https://md.research.elamur.ai/free/en/papers/ha-2016-multilingual-neural-machine-translation-universal/)
- [Survey of Low-Resource Machine Translation](https://md.research.elamur.ai/free/en/papers/haddow-2022-survey-low-resource-machine-translation/)
- [Choosing Transfer Languages for Cross-Lingual Learning](https://md.research.elamur.ai/free/en/papers/lin-2019-choosing-transfer-languages-cross-lingual/)
- [Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information](https://md.research.elamur.ai/free/en/papers/lin-2020-pre-training-multilingual-neural-machine/)
- [A neural interlingua for multilingual machine translation](https://md.research.elamur.ai/free/en/papers/lu-2018-neural-interlingua-multilingual-machine-translation/)
- [Multi-task Sequence to Sequence Learning](https://md.research.elamur.ai/free/en/papers/luong-2015-multi-task-sequence-sequence-learning/)
- [Paraphrasing Revisited with Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/mallinson-2017-paraphrasing-revisited-neural-machine-translation/)
- [Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages](https://md.research.elamur.ai/free/en/papers/nekoto-2020-participatory-research-low-resourced-machine/)
- [Tigrinya Neural Machine Translation with Transfer Learning for Humanitarian Response](https://md.research.elamur.ai/free/en/papers/oktem-2020-tigrinya-neural-machine-translation-transfer/)
- [Interlinear Glosses as a Multilingual Pivot for Machine Translation: An Updated Study on Turkish with Restricted Resources](https://md.research.elamur.ai/free/en/papers/ozer-2026-interlinear-glosses-as-multilingual-pivot/)
- [Contrastive Learning for Many-to-many Multilingual Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/pan-2021-contrastive-learning-many-many-multilingual/)
- [Learning Joint Multilingual Sentence Representations with Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/schwenk-2017-learning-joint-multilingual-sentence-representations/)
- [Multilingual Translation from Denoising Pre-Training](https://md.research.elamur.ai/free/en/papers/tang-2021-multilingual-translation-denoising-pre-training/)
- [Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing](https://md.research.elamur.ai/free/en/papers/thompson-2020-automatic-machine-translation-evaluation-many/)
- [Paraphrase Generation as Zero-Shot Multilingual Translation: Disentangling Semantic Similarity from Lexical and Syntactic Diversity](https://md.research.elamur.ai/free/en/papers/thompson-2020-paraphrase-generation-as-zero-shot/)
- [Exploiting Cross-Sentence Context for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/wang-2017-exploiting-cross-sentence-context-neural/)
- [Gradient Surgery for Multi-Task Learning](https://md.research.elamur.ai/free/en/papers/yu-2020-gradient-surgery-multi-task-learning/)
- [Exploiting Source-side Monolingual Data in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/zhang-2016-exploiting-source-side-monolingual-data/)
- [Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation](https://md.research.elamur.ai/free/en/papers/zhang-2020-improving-massively-multilingual-neural-machine/)
- [Legal Judgment Prediction via Topological Learning](https://md.research.elamur.ai/free/en/papers/zhong-2018-legal-judgment-prediction-topological-learning/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{dong2015multi,
  title = {{Multi-Task Learning for Multiple Language Translation}},
  author = {Dong, Daxiang and Wu, Hua and He, Wei and Yu, Dianhai and Wang, Haifeng},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-1166},
  url = {https://doi.org/10.3115/v1/p15-1166}
}
```

- [← previous Multi-level Translation Quality Prediction with QuEst++](https://md.research.elamur.ai/free/en/papers/specia-2015-multi-level-translation-quality-prediction/)
- [next → Multi-task Sequence to Sequence Learning](https://md.research.elamur.ai/free/en/papers/luong-2015-multi-task-sequence-sequence-learning/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/)

## References

22 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rie Kubota Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6:1817–1853. [find](https://scholar.google.com/scholar?q=Rie+Kubota+Ando+and+Tong+Zhang%2E+2005%2E+A+framework+for+learning+predictive+structures+from+multiple+tasks+and+unlabeled+data%2E+Journal+of+Machine+Learning+Research%2C+6%3A1817%E2%80%931853%2E)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+CoRR%2C+abs%2F1409%2E0473%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, James Bergstra, Ian J. Goodfellow, Arnaud Bergeron, Nicolas Bouchard, David Warde-Farley, and Yoshua Bengio. 2012. Theano: new features and speed improvements. CoRR, abs/1211.5590. [find](https://scholar.google.com/scholar?q=Fre%CC%81de%CC%81ric+Bastien%2C+Pascal+Lamblin%2C+Razvan+Pascanu%2C+James+Bergstra%2C+Ian+J%2E+Goodfellow%2C+Arnaud+Bergeron%2C+Nicolas+Bouchard%2C+David+Warde%2DFarley%2C+and+Yoshua+Bengio%2E+2012%2E+Theano%3A+new+features+and+speed+im)
4. Léon Bottou. 1991. Stochastic gradient learning in neural networks. In Proceedings of Neuro-Nı̂mes 91, Nimes, France. EC2. [find](https://scholar.google.com/scholar?q=Le%CC%81on+Bottou%2E+1991%2E+Stochastic+gradient+learning+in+neural+networks%2E+In+Proceedings+of+Neuro%2DN%C4%B1%CC%82mes+91%2C+Nimes%2C+France%2E+EC2%2E)
5. KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. CoRR, abs/1409.1259. [find](https://scholar.google.com/scholar?q=KyungHyun+Cho%2C+Bart+van+Merrienboer%2C+Dzmitry+Bahdanau%2C+and+Yoshua+Bengio%2E+2014%2E+On+the+properties+of+neural+machine+translation%3A+Encoder%2Ddecoder+approaches%2E+CoRR%2C+abs%2F1409%2E1259%2E) [our card](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/)
6. Trevor Cohn and Mirella Lapata. 2007. Machine translation by triangulation: Making effective use of multi-parallel corpora. In Proc. ACL, pages 728– 735. [find](https://scholar.google.com/scholar?q=Trevor+Cohn+and+Mirella+Lapata%2E+2007%2E+Machine+translation+by+triangulation%3A+Making+effective+use+of+multi%2Dparallel+corpora%2E+In+Proc%2E+ACL%2C+pages+728%E2%80%93+735%2E)
7. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel P. Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493–2537. [find](https://scholar.google.com/scholar?q=Ronan+Collobert%2C+Jason+Weston%2C+Le%CC%81on+Bottou%2C+Michael+Karlen%2C+Koray+Kavukcuoglu%2C+and+Pavel+P%2E+Kuksa%2E+2011%2E+Natural+language+processing+%28almost%29+from+scratch%2E+Journal+of+Machine+Learning+Research%2C+12%3A24) [our card](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/)
8. Lei Cui, Xilun Chen, Dongdong Zhang, Shujie Liu, Mu Li, and Ming Zhou. 2013. Multi-domain adaptation for SMT using multi-task learning. In Proc. EMNLP, pages 1055–1065. [find](https://scholar.google.com/scholar?q=Lei+Cui%2C+Xilun+Chen%2C+Dongdong+Zhang%2C+Shujie+Liu%2C+Mu+Li%2C+and+Ming+Zhou%2E+2013%2E+Multi%2Ddomain+adaptation+for+SMT+using+multi%2Dtask+learning%2E+In+Proc%2E+EMNLP%2C+pages+1055%E2%80%931065%2E)
9. Jacob Devlin, Rabih Zbib, Zhongqiang Huang, Thomas Lamar, Richard M. Schwartz, and John Makhoul. 2014. Fast and robust neural network joint models for statistical machine translation. In Proc. ACL, pages 1370–1380. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Rabih+Zbib%2C+Zhongqiang+Huang%2C+Thomas+Lamar%2C+Richard+M%2E+Schwartz%2C+and+John+Makhoul%2E+2014%2E+Fast+and+robust+neural+network+joint+models+for+statistical+machine+translation%2E+In+Proc%2E+ACL%2C+pa) [our card](https://md.research.elamur.ai/free/en/papers/devlin-2014-fast-robust-neural-network-joint/)
10. Jianfeng Gao, Xiaodong He, Wen-tau Yih, and Li Deng. 2014. Learning continuous phrase representations for translation modeling. In Proc. ACL, pages 699–709. [find](https://scholar.google.com/scholar?q=Jianfeng+Gao%2C+Xiaodong+He%2C+Wen%2Dtau+Yih%2C+and+Li+Deng%2E+2014%2E+Learning+continuous+phrase+representations+for+translation+modeling%2E+In+Proc%2E+ACL%2C+pages+699%E2%80%93709%2E)
11. Jun Hatori, Takuya Matsuzaki, Yusuke Miyao, and Jun’ichi Tsujii. 2012. Incremental joint approach to word segmentation, POS tagging, and dependency parsing in chinese. In Proc. ACL, pages 1045–1053. [find](https://scholar.google.com/scholar?q=Jun+Hatori%2C+Takuya+Matsuzaki%2C+Yusuke+Miyao%2C+and+Jun%E2%80%99ichi+Tsujii%2E+2012%2E+Incremental+joint+approach+to+word+segmentation%2C+POS+tagging%2C+and+dependency+parsing+in+chinese%2E+In+Proc%2E+ACL%2C+pages+1045%E2%80%931053%2E)
12. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In Proc. EMNLP, pages 1700–1709. [find](https://scholar.google.com/scholar?q=Nal+Kalchbrenner+and+Phil+Blunsom%2E+2013%2E+Recurrent+continuous+translation+models%2E+In+Proc%2E+EMNLP%2C+pages+1700%E2%80%931709%2E) [our card](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/)
13. Philipp Koehn. 2004. Pharaoh: A beam search decoder for phrase-based statistical machine translation models. In Machine Translation: From Real Users to Research, 6th Conference of the Association for Machine Translation in the Americas, AMTA 2004, Washington, DC, USA, September 28-October 2, 2004, P… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2004%2E+Pharaoh%3A+A+beam+search+decoder+for+phrase%2Dbased+statistical+machine+translation+models%2E+In+Machine+Translation%3A+From+Real+Users+to+Research%2C+6th+Conference+of+the+Association+for+)
14. Zhenghua Li, Min Zhang, Wanxiang Che, Ting Liu, and Wenliang Chen. 2014. Joint optimization for chinese POS tagging and dependency parsing. IEEE/ACM Transactions on Audio, Speech & Language Processing, 22(1):274–286. [find](https://scholar.google.com/scholar?q=Zhenghua+Li%2C+Min+Zhang%2C+Wanxiang+Che%2C+Ting+Liu%2C+and+Wenliang+Chen%2E+2014%2E+Joint+optimization+for+chinese+POS+tagging+and+dependency+parsing%2E+IEEE%2FACM+Transactions+on+Audio%2C+Speech+%26+Language+Processing)
15. Shujie Liu, Nan Yang, Mu Li, and Ming Zhou. 2014. A recursive recurrent neural network for statistical machine translation. In Proc. ACL, pages 1491– 1500. [find](https://scholar.google.com/scholar?q=Shujie+Liu%2C+Nan+Yang%2C+Mu+Li%2C+and+Ming+Zhou%2E+2014%2E+A+recursive+recurrent+neural+network+for+statistical+machine+translation%2E+In+Proc%2E+ACL%2C+pages+1491%E2%80%93+1500%2E)
16. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proc. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+A+method+for+automatic+evaluation+of+machine+translation%2E+In+Proc%2E) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
17. ACL, ACL 2002, pages 311–318, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=ACL%2C+ACL+2002%2C+pages+311%E2%80%93318%2C+Stroudsburg%2C+PA%2C+USA%2E+Association+for+Computational+Linguistics%2E)
18. Rico Sennrich, Holger Schwenk, and Walid Aransa. 2013. A multi-domain translation model framework for statistical machine translation. In Proc. ACL, pages 832–840. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Holger+Schwenk%2C+and+Walid+Aransa%2E+2013%2E+A+multi%2Ddomain+translation+model+framework+for+statistical+machine+translation%2E+In+Proc%2E+ACL%2C+pages+832%E2%80%93840%2E)
19. Martin Sundermeyer, Tamer Alkhouli, Joern Wuebker, and Hermann Ney. 2014. Translation modeling with bidirectional recurrent neural networks. In Proc. EMNLP, pages 14–25. [find](https://scholar.google.com/scholar?q=Martin+Sundermeyer%2C+Tamer+Alkhouli%2C+Joern+Wuebker%2C+and+Hermann+Ney%2E+2014%2E+Translation+modeling+with+bidirectional+recurrent+neural+networks%2E+In+Proc%2E+EMNLP%2C+pages+14%E2%80%9325%2E)
20. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V%2E+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+Neural+Information+Processing+Systems+27%3A+Annual+Conference+on+Neural+Informatio)
21. Hua Wu and Haifeng Wang. 2007. Pivot language approach for phrase-based statistical machine translation. In Proc. ACL, pages 165–181. [find](https://scholar.google.com/scholar?q=Hua+Wu+and+Haifeng+Wang%2E+2007%2E+Pivot+language+approach+for+phrase%2Dbased+statistical+machine+translation%2E+In+Proc%2E+ACL%2C+pages+165%E2%80%93181%2E) [our card](https://md.research.elamur.ai/free/en/papers/wu-2007-pivot-language-approach-phrase-based/)
22. Matthew D. Zeiler. 2012. ADADELTA: an adaptive learning rate method. CoRR, abs/1212.5701. [find](https://scholar.google.com/scholar?q=Matthew+D%2E+Zeiler%2E+2012%2E+ADADELTA%3A+an+adaptive+learning+rate+method%2E+CoRR%2C+abs%2F1212%2E5701%2E) [our card](https://md.research.elamur.ai/free/en/papers/zeiler-2012-adadelta-adaptive-learning-rate-method/)