---
title: "Multi-task Sequence to Sequence Learning — Luong 2015"
description: "Sequence-to-sequence (seq2seq) models had mostly been trained for one task at a time."
authors: ["Minh-Thang Luong", "Quoc V. Le", "Ilya Sutskever", "Oriol Vinyals", "Lukasz Kaiser"]
year: 2015
venue: "arXiv:1511.06114"
source: "https://arxiv.org/abs/1511.06114"
doi: "10.48550/arxiv.1511.06114"
arxiv: "1511.06114"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/luong-2015-multi-task-sequence-sequence-learning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/luong-2015-multi-task-sequence-sequence-learning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/) / Multi-task Sequence to Sequence Learning

# Multi-task Sequence to Sequence Learning

**Authors:** [Minh-Thang Luong](https://md.research.elamur.ai/free/en/authors/minh-thang-luong/), [Quoc V. Le](https://md.research.elamur.ai/free/en/authors/quoc-v-le/), [Ilya Sutskever](https://md.research.elamur.ai/free/en/authors/ilya-sutskever/), [Oriol Vinyals](https://md.research.elamur.ai/free/en/authors/oriol-vinyals/), [Lukasz Kaiser](https://md.research.elamur.ai/free/en/authors/lukasz-kaiser/) · **Year:** 2015 · **Venue:** arXiv:1511.06114 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1511.06114) · [PDF](https://arxiv.org/pdf/1511.06114) · [DOI 10.48550/arxiv.1511.06114](https://doi.org/10.48550/arxiv.1511.06114) · [arXiv 1511.06114](https://arxiv.org/abs/1511.06114) · [OpenAlex](https://openalex.org/W2172589779)

**Citations:** 56 citations (openalex-snapshot · 2026-10-14) · [257](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Sequence-to-sequence (seq2seq) models had mostly been trained for one task at a time. The authors ask whether training them jointly on several related tasks helps. They define three ways to share parameters. In one-to-many, a single encoder feeds several task-specific decoders, such as translation and constituency parsing. In many-to-one, several encoders share one decoder, such as translation and image captioning. In many-to-many, several encoders and decoders are shared, which lets translation be combined with unsupervised objectives on monolingual text.

Training switches between tasks, choosing each mini-batch's task at random according to mixing ratios α. The experiments focus on WMT'15 English⇆German and use LSTM models without attention. Adding fairly small parsing and captioning datasets improves translation by up to 1.5 BLEU over single-task baselines. Translation data also helps parsing, and an ensemble reaches a new best constituent parsing score of 93.0 F1.

The paper matters as an early systematic look at when parameter sharing pays off across different seq2seq tasks. It also compares how two unsupervised objectives behave in this setting.

## Key points
- **Data:** about 4.5M WMT'15 sentence pairs per translation direction, with 50K-word vocabularies shared across tasks. The parsing output vocabulary has 104 tags. Testing uses newstest2014 (English→German) and newstest2015 (German→English).
- **Auxiliary corpora:** Penn Tree Bank parsing (40K), a High-Confidence Corpus for parsing (11.0M), image captioning data (596K), and monolingual English and German text (12.1M and 13.8M).
- **Unsupervised objectives:** sequence autoencoders and skip-thought. Because sentences are unordered, skip-thought is adapted by predicting one half of a sentence from the other half. Compared with skip-thought, the autoencoder improves perplexity less but improves BLEU more.
- **Limitation:** none of the models use attention, and the authors leave multi-task learning with attention to future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

257 papers of our library cite it (of the 140673 whose bibliographies we read); 34 of them have a page here, 223 not yet.

- [Improving historical spelling normalization with bi-directional LSTMs and multi-task learning](https://md.research.elamur.ai/free/en/papers/aclbulk-2016-bollmann-improving-historical-spelling-normalization-bi/)
- [Few-Shot and Zero-Shot Learning for Historical Text Normalization](https://md.research.elamur.ai/free/en/papers/aclbulk-2019-bollmann-few-shot-zero-shot-learning/)
- [Towards String-To-Tree Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/aharoni-2017-string-tree-neural-machine-translation/)
- [Tied Multitask Learning for Neural Speech Translation](https://md.research.elamur.ai/free/en/papers/anastasopoulos-2018-tied-multitask-learning-neural-speech/)
- [Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges](https://md.research.elamur.ai/free/en/papers/arivazhagan-2019-massively-multilingual-neural-machine-translation/)
- [Graph Convolutional Encoders for Syntax-aware Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/bastings-2017-graph-convolutional-encoders-syntax-aware/)
- [Identifying beneficial task relations for multi-task learning in deep neural networks](https://md.research.elamur.ai/free/en/papers/bingel-2017-identifying-beneficial-task-relations-multi/)
- [Semi-Supervised Sequence Modeling with Cross-View Training](https://md.research.elamur.ai/free/en/papers/clark-2018-semi-supervised-sequence-modeling-cross/)
- [BAM! Born-Again Multi-Task Networks for Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/clark-2019-bam-born-again-multi-task/)
- [Copied Monolingual Data Improves Low-Resource Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/currey-2017-copied-monolingual-data-improves-low/)
- [Controllable Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/fan-2018-controllable-abstractive-summarization/)
- [Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism](https://md.research.elamur.ai/free/en/papers/firat-2016-multi-way-multilingual-neural-machine/)
- [Zero-Resource Translation with Multi-Lingual Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/firat-2016-zero-resource-translation-multi-lingual/)
- [Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder](https://md.research.elamur.ai/free/en/papers/ha-2016-multilingual-neural-machine-translation-universal/)
- [A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks](https://md.research.elamur.ai/free/en/papers/hashimoto-2016-joint-many-task-model-growing/)
- [An Interactive Multi-Task Learning Network for End-to-End Aspect-Based Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/he-2019-interactive-multi-task-learning-network/)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/)
- [Scheduled Multi-Task Learning: From Syntax to Translation](https://md.research.elamur.ai/free/en/papers/kiperwasser-2018-scheduled-multi-task-learning-syntax/)
- [Adversarial Multi-task Learning for Text Classification](https://md.research.elamur.ai/free/en/papers/liu-2017-adversarial-multi-task-learning-text/)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/liu-2019-multi-task-deep-neural-networks/)
- [A neural interlingua for multilingual machine translation](https://md.research.elamur.ai/free/en/papers/lu-2018-neural-interlingua-multilingual-machine-translation/)
- [The Natural Language Decathlon: Multitask Learning as Question Answering](https://md.research.elamur.ai/free/en/papers/mccann-2018-natural-language-decathlon-multitask-learning/)
- [Multilingual and Multi-Aspect Hate Speech Analysis](https://md.research.elamur.ai/free/en/papers/ousidhoum-2019-multilingual-multi-aspect-hate-speech/)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/)
- [Unsupervised Pretraining for Sequence to Sequence Learning](https://md.research.elamur.ai/free/en/papers/ramachandran-2017-unsupervised-pretraining-sequence-sequence-learning/)
- [Learning Joint Multilingual Sentence Representations with Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/schwenk-2017-learning-joint-multilingual-sentence-representations/)
- [Deep multi-task learning with low level tasks supervised at lower layers](https://md.research.elamur.ai/free/en/papers/sogaard-2016-deep-multi-task-learning-low/)
- [Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold](https://md.research.elamur.ai/free/en/papers/swayamdipta-2017-frame-semantic-parsing-softmax-margin/)
- [Exploring and Predicting Transferability across NLP Tasks](https://md.research.elamur.ai/free/en/papers/vu-2020-exploring-predicting-transferability-across-nlp/)
- [Bilingually-constrained Synthetic Data for Implicit Discourse Relation Recognition](https://md.research.elamur.ai/free/en/papers/wu-2016-bilingually-constrained-synthetic-data-implicit/)
- [Multitask Learning-Based Neural Bridging Reference Resolution](https://md.research.elamur.ai/free/en/papers/yu-2020-multitask-learning-based-neural-bridging/)
- [Exploiting Source-side Monolingual Data in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/zhang-2016-exploiting-source-side-monolingual-data/)
- [Legal Judgment Prediction via Topological Learning](https://md.research.elamur.ai/free/en/papers/zhong-2018-legal-judgment-prediction-topological-learning/)
- [Multi-Source Neural Translation](https://md.research.elamur.ai/free/en/papers/zoph-2016-multi-source-neural-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{luong2015multi,
  title = {{Multi-task Sequence to Sequence Learning}},
  author = {Luong, Minh-Thang and Le, Quoc V. and Sutskever, Ilya and Vinyals, Oriol and Kaiser, Lukasz},
  year = {2015},
  doi = {10.48550/arxiv.1511.06114},
  eprint = {1511.06114},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1511.06114}
}
```

- [← previous Multi-Task Learning for Multiple Language Translation](https://md.research.elamur.ai/free/en/papers/dong-2015-multi-task-learning-multiple-language/)
- [next → MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems](https://md.research.elamur.ai/free/en/papers/chen-2015-mxnet-flexible-efficient-machine-learning/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/)

## References

32 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ando, Rie Kubota and Zhang, Tong. A framework for learning predictive structures from multiple tasks and unlabeled data. JMLR, 6:1817–1853, 2005. [find](https://scholar.google.com/scholar?q=Ando%2C+Rie+Kubota+and+Zhang%2C+Tong%2E+A+framework+for+learning+predictive+structures+from+multiple+tasks+and+unlabeled+data%2E+JMLR%2C+6%3A1817%E2%80%931853%2C+2005%2E)
2. Argyriou, Andreas, Evgeniou, Theodoros, and Pontil, Massimiliano. Multi-task feature learning. In NIPS, 2007. [find](https://scholar.google.com/scholar?q=Argyriou%2C+Andreas%2C+Evgeniou%2C+Theodoros%2C+and+Pontil%2C+Massimiliano%2E+Multi%2Dtask+feature+learning%2E+In+NIPS%2C+2007%2E)
3. Bahdanau, Dzmitry, Cho, Kyunghyun, and Bengio, Yoshua. Neural machine translation by jointly learning to align and translate. In ICLR, 2015. [find](https://scholar.google.com/scholar?q=Bahdanau%2C+Dzmitry%2C+Cho%2C+Kyunghyun%2C+and+Bengio%2C+Yoshua%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+ICLR%2C+2015%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
4. Bojar, Ondřej, Chatterjee, Rajen, Federmann, Christian, Haddow, Barry, Huck, Matthias, Hokamp, Chris, Koehn, Philipp, Logacheva, Varvara, Monz, Christof, Negri, Matteo, Post, Matt, Scarton, Carolina, Specia, Lucia, and Turchi, Marco. Findings of the 2015 workshop on statistical machine translation.… [find](https://scholar.google.com/scholar?q=Bojar%2C+Ondr%CC%8Cej%2C+Chatterjee%2C+Rajen%2C+Federmann%2C+Christian%2C+Haddow%2C+Barry%2C+Huck%2C+Matthias%2C+Hokamp%2C+Chris%2C+Koehn%2C+Philipp%2C+Logacheva%2C+Varvara%2C+Monz%2C+Christof%2C+Negri%2C+Matteo%2C+Post%2C+Matt%2C+Scarton%2C+Carolina%2C) [our card](https://md.research.elamur.ai/free/en/papers/bojar-2015-findings-2015-workshop-statistical-machine/)
5. Caruana, Rich. Multitask learning. Machine Learning, 28(1):41–75, 1997. [find](https://scholar.google.com/scholar?q=Caruana%2C+Rich%2E+Multitask+learning%2E+Machine+Learning%2C+28%281%29%3A41%E2%80%9375%2C+1997%2E)
6. Cho, Kyunghyun, van Merrienboer, Bart, Gulcehre, Caglar, Bougares, Fethi, Schwenk, Holger, and Bengio, Yoshua. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP, 2014. [find](https://scholar.google.com/scholar?q=Cho%2C+Kyunghyun%2C+van+Merrienboer%2C+Bart%2C+Gulcehre%2C+Caglar%2C+Bougares%2C+Fethi%2C+Schwenk%2C+Holger%2C+and+Bengio%2C+Yoshua%2E+Learning+phrase+representations+using+RNN+encoder%2Ddecoder+for+statistical+machine+transla)
7. Dai, Andrew M. and Le, Quoc V. Semi-supervised sequence learning. In NIPS, 2015. [find](https://scholar.google.com/scholar?q=Dai%2C+Andrew+M%2E+and+Le%2C+Quoc+V%2E+Semi%2Dsupervised+sequence+learning%2E+In+NIPS%2C+2015%2E)
8. Donahue, Jeff, Jia, Yangqing, Vinyals, Oriol, Hoffman, Judy, Zhang, Ning, Tzeng, Eric, and Darrell, Trevor. DeCAF: A deep convolutional activation feature for generic visual recognition, 2014. [find](https://scholar.google.com/scholar?q=Donahue%2C+Jeff%2C+Jia%2C+Yangqing%2C+Vinyals%2C+Oriol%2C+Hoffman%2C+Judy%2C+Zhang%2C+Ning%2C+Tzeng%2C+Eric%2C+and+Darrell%2C+Trevor%2E+DeCAF%3A+A+deep+convolutional+activation+feature+for+generic+visual+recognition%2C+2014%2E)
9. Dong, Daxiang, Wu, Hua, He, Wei, Yu, Dianhai, and Wang, Haifeng. Multi-task learning for multiple language translation. In ACL, 2015. [find](https://scholar.google.com/scholar?q=Dong%2C+Daxiang%2C+Wu%2C+Hua%2C+He%2C+Wei%2C+Yu%2C+Dianhai%2C+and+Wang%2C+Haifeng%2E+Multi%2Dtask+learning+for+multiple+language+translation%2E+In+ACL%2C+2015%2E) [our card](https://md.research.elamur.ai/free/en/papers/dong-2015-multi-task-learning-multiple-language/)
10. Evgeniou, Theodoros and Pontil, Massimiliano. Regularized multi–task learning. In SIGKDD, 2004. [find](https://scholar.google.com/scholar?q=Evgeniou%2C+Theodoros+and+Pontil%2C+Massimiliano%2E+Regularized+multi%E2%80%93task+learning%2E+In+SIGKDD%2C+2004%2E)
11. Gulcehre, Caglar, Firat, Orhan, Xu, Kelvin, Cho, Kyunghyun, Barrault, Loic, Lin, Huei-Chi, Bougares, Fethi, Schwenk, Holger, and Bengio, Yoshua. On using monolingual corpora in neural machine translation. arXiv preprint arXiv:1503.03535, 2015. [find](https://scholar.google.com/scholar?q=Gulcehre%2C+Caglar%2C+Firat%2C+Orhan%2C+Xu%2C+Kelvin%2C+Cho%2C+Kyunghyun%2C+Barrault%2C+Loic%2C+Lin%2C+Huei%2DChi%2C+Bougares%2C+Fethi%2C+Schwenk%2C+Holger%2C+and+Bengio%2C+Yoshua%2E+On+using+monolingual+corpora+in+neural+machine+translat) [our card](https://md.research.elamur.ai/free/en/papers/gulcehre-2015-using-monolingual-corpora-neural-machine/)
12. Heigold, Georg, Vanhoucke, Vincent, Senior, Alan, Nguyen, Patrick, Ranzato, Marc’Aurelio, Devin, Matthieu, and Dean, Jeffrey. Multilingual acoustic models using distributed deep neural networks. In ICASSP, 2013. [find](https://scholar.google.com/scholar?q=Heigold%2C+Georg%2C+Vanhoucke%2C+Vincent%2C+Senior%2C+Alan%2C+Nguyen%2C+Patrick%2C+Ranzato%2C+Marc%E2%80%99Aurelio%2C+Devin%2C+Matthieu%2C+and+Dean%2C+Jeffrey%2E+Multilingual+acoustic+models+using+distributed+deep+neural+networks%2E+In+IC)
13. Hochreiter, Sepp and Schmidhuber, Jürgen. Long short-term memory. Neural Computation, 9(8): 1735–1780, 1997. [find](https://scholar.google.com/scholar?q=Hochreiter%2C+Sepp+and+Schmidhuber%2C+Ju%CC%88rgen%2E+Long+short%2Dterm+memory%2E+Neural+Computation%2C+9%288%29%3A+1735%E2%80%931780%2C+1997%2E)
14. Huang, Jui-Ting, Li, Jinyu, Yu, Dong, Deng, Li, and Gong, Yifan. Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers. In ICASSP, 2013. [find](https://scholar.google.com/scholar?q=Huang%2C+Jui%2DTing%2C+Li%2C+Jinyu%2C+Yu%2C+Dong%2C+Deng%2C+Li%2C+and+Gong%2C+Yifan%2E+Cross%2Dlanguage+knowledge+transfer+using+multilingual+deep+neural+network+with+shared+hidden+layers%2E+In+ICASSP%2C+2013%2E)
15. Jean, Sébastien, Cho, Kyunghyun, Memisevic, Roland, and Bengio, Yoshua. On using very large target vocabulary for neural machine translation. In ACL, 2015a. [find](https://scholar.google.com/scholar?q=Jean%2C+Se%CC%81bastien%2C+Cho%2C+Kyunghyun%2C+Memisevic%2C+Roland%2C+and+Bengio%2C+Yoshua%2E+On+using+very+large+target+vocabulary+for+neural+machine+translation%2E+In+ACL%2C+2015a%2E) [our card](https://md.research.elamur.ai/free/en/papers/jean-2015-very-large-target-vocabulary-neural/)
16. Jean, Sébastien, Firat, Orhan, Cho, Kyunghyun, Memisevic, Roland, and Bengio, Yoshua. Montreal neural machine translation systems for WMT’15. In WMT, 2015b. [find](https://scholar.google.com/scholar?q=Jean%2C+Se%CC%81bastien%2C+Firat%2C+Orhan%2C+Cho%2C+Kyunghyun%2C+Memisevic%2C+Roland%2C+and+Bengio%2C+Yoshua%2E+Montreal+neural+machine+translation+systems+for+WMT%E2%80%9915%2E+In+WMT%2C+2015b%2E)
17. Jozefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., and Wu, Y. Exploring the limits of language modeling. arXiv preprint arXiv:1602.02410, 2016. [find](https://scholar.google.com/scholar?q=Jozefowicz%2C+R%2E%2C+Vinyals%2C+O%2E%2C+Schuster%2C+M%2E%2C+Shazeer%2C+N%2E%2C+and+Wu%2C+Y%2E+Exploring+the+limits+of+language+modeling%2E+arXiv+preprint+arXiv%3A1602%2E02410%2C+2016%2E)
18. Kalchbrenner, Nal and Blunsom, Phil. Recurrent continuous translation models. In EMNLP, 2013. [find](https://scholar.google.com/scholar?q=Kalchbrenner%2C+Nal+and+Blunsom%2C+Phil%2E+Recurrent+continuous+translation+models%2E+In+EMNLP%2C+2013%2E) [our card](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/)
19. Kiros, Ryan, Zhu, Yukun, Salakhutdinov, Ruslan, Zemel, Richard S., Torralba, Antonio, Urtasun, Raquel, and Fidler, Sanja. Skip-thought vectors. In NIPS, 2015. [find](https://scholar.google.com/scholar?q=Kiros%2C+Ryan%2C+Zhu%2C+Yukun%2C+Salakhutdinov%2C+Ruslan%2C+Zemel%2C+Richard+S%2E%2C+Torralba%2C+Antonio%2C+Urtasun%2C+Raquel%2C+and+Fidler%2C+Sanja%2E+Skip%2Dthought+vectors%2E+In+NIPS%2C+2015%2E)
20. Kumar, Abhishek and III, Hal Daumé. Learning task grouping and overlap in multi-task learning. In ICML, 2012. [find](https://scholar.google.com/scholar?q=Kumar%2C+Abhishek+and+III%2C+Hal+Daume%CC%81%2E+Learning+task+grouping+and+overlap+in+multi%2Dtask+learning%2E+In+ICML%2C+2012%2E)
21. Liu, Xiaodong, Gao, Jianfeng, He, Xiaodong, Deng, Li, Duh, Kevin, and Wang, Ye-Yi. Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In NAACL, 2015. [find](https://scholar.google.com/scholar?q=Liu%2C+Xiaodong%2C+Gao%2C+Jianfeng%2C+He%2C+Xiaodong%2C+Deng%2C+Li%2C+Duh%2C+Kevin%2C+and+Wang%2C+Ye%2DYi%2E+Representation+learning+using+multi%2Dtask+deep+neural+networks+for+semantic+classification+and+information+retrieval%2E+) [our card](https://md.research.elamur.ai/free/en/papers/liu-2015-representation-learning-multi-task-deep/)
22. Luong, Minh-Thang and Manning, Christopher D. Stanford neural machine translation systems for spoken language domain. In IWSLT, 2015. [find](https://scholar.google.com/scholar?q=Luong%2C+Minh%2DThang+and+Manning%2C+Christopher+D%2E+Stanford+neural+machine+translation+systems+for+spoken+language+domain%2E+In+IWSLT%2C+2015%2E)
23. Luong, Minh-Thang, Pham, Hieu, and Manning, Christopher D. Effective approaches to attention-based neural machine translation. In EMNLP, 2015a. [find](https://scholar.google.com/scholar?q=Luong%2C+Minh%2DThang%2C+Pham%2C+Hieu%2C+and+Manning%2C+Christopher+D%2E+Effective+approaches+to+attention%2Dbased+neural+machine+translation%2E+In+EMNLP%2C+2015a%2E)
24. Luong, Minh-Thang, Sutskever, Ilya, Le, Quoc V., Vinyals, Oriol, and Zaremba, Wojciech. Addressing the rare word problem in neural machine translation. In ACL, 2015b. [find](https://scholar.google.com/scholar?q=Luong%2C+Minh%2DThang%2C+Sutskever%2C+Ilya%2C+Le%2C+Quoc+V%2E%2C+Vinyals%2C+Oriol%2C+and+Zaremba%2C+Wojciech%2E+Addressing+the+rare+word+problem+in+neural+machine+translation%2E+In+ACL%2C+2015b%2E) [our card](https://md.research.elamur.ai/free/en/papers/luong-2015-addressing-rare-word-problem-neural/)
25. Marcus, Mitchell P., Marcinkiewicz, Mary Ann, and Santorini, Beatrice. Building a large annotated corpus of english: The penn treebank. Computational Linguistics, 19(2):313–330, 1993. [find](https://scholar.google.com/scholar?q=Marcus%2C+Mitchell+P%2E%2C+Marcinkiewicz%2C+Mary+Ann%2C+and+Santorini%2C+Beatrice%2E+Building+a+large+annotated+corpus+of+english%3A+The+penn+treebank%2E+Computational+Linguistics%2C+19%282%29%3A313%E2%80%93330%2C+1993%2E) [our card](https://md.research.elamur.ai/free/en/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
26. Papineni, Kishore, Roukos, Salim, Ward, Todd, and jing Zhu, Wei. Bleu: a method for automatic evaluation of machine translation. In ACL, 2002. [find](https://scholar.google.com/scholar?q=Papineni%2C+Kishore%2C+Roukos%2C+Salim%2C+Ward%2C+Todd%2C+and+jing+Zhu%2C+Wei%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+ACL%2C+2002%2E) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
27. Pham, Vu, Bluche, Théodore, Kermorvant, Christopher, and Louradour, Jérôme. Dropout improves recurrent neural networks for handwriting recognition. In Frontiers in Handwriting Recognition (ICFHR), 2014 14th International Conference on, pp. 285–290. [find](https://scholar.google.com/scholar?q=Pham%2C+Vu%2C+Bluche%2C+The%CC%81odore%2C+Kermorvant%2C+Christopher%2C+and+Louradour%2C+Je%CC%81ro%CC%82me%2E+Dropout+improves+recurrent+neural+networks+for+handwriting+recognition%2E+In+Frontiers+in+Handwriting+Recognition+%28ICFHR%29%2C+)
28. Sutskever, Ilya, Vinyals, Oriol, and Le, Quoc V. Sequence to sequence learning with neural networks. In NIPS, 2014. [find](https://scholar.google.com/scholar?q=Sutskever%2C+Ilya%2C+Vinyals%2C+Oriol%2C+and+Le%2C+Quoc+V%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+NIPS%2C+2014%2E)
29. Thrun, Sebastian. Is learning the n-th thing any easier than learning the first? In NIPS, 1996. [find](https://scholar.google.com/scholar?q=Thrun%2C+Sebastian%2E+Is+learning+the+n%2Dth+thing+any+easier+than+learning+the+first%3F+In+NIPS%2C+1996%2E)
30. Vinyals, Oriol, Kaiser, Lukasz, Koo, Terry, Petrov, Slav, Sutskever, Ilya, and Hinton, Geoffrey. Grammar as a foreign language. In NIPS, 2015a. [find](https://scholar.google.com/scholar?q=Vinyals%2C+Oriol%2C+Kaiser%2C+Lukasz%2C+Koo%2C+Terry%2C+Petrov%2C+Slav%2C+Sutskever%2C+Ilya%2C+and+Hinton%2C+Geoffrey%2E+Grammar+as+a+foreign+language%2E+In+NIPS%2C+2015a%2E)
31. Vinyals, Oriol, Toshev, Alexander, Bengio, Samy, and Erhan, Dumitru. Show and tell: A neural image caption generator. In CVPR, 2015b. [find](https://scholar.google.com/scholar?q=Vinyals%2C+Oriol%2C+Toshev%2C+Alexander%2C+Bengio%2C+Samy%2C+and+Erhan%2C+Dumitru%2E+Show+and+tell%3A+A+neural+image+caption+generator%2E+In+CVPR%2C+2015b%2E) [our card](https://md.research.elamur.ai/free/en/papers/vinyals-2014-show-tell-neural-image-caption/)
32. Xu, Kelvin, Ba, Jimmy, Kiros, Ryan, Cho, Kyunghyun, Courville, Aaron C., Salakhutdinov, Ruslan, Zemel, Richard S., and Bengio, Yoshua. Show, attend and tell: Neural image caption generation with visual attention. In ICML, 2015. [find](https://scholar.google.com/scholar?q=Xu%2C+Kelvin%2C+Ba%2C+Jimmy%2C+Kiros%2C+Ryan%2C+Cho%2C+Kyunghyun%2C+Courville%2C+Aaron+C%2E%2C+Salakhutdinov%2C+Ruslan%2C+Zemel%2C+Richard+S%2E%2C+and+Bengio%2C+Yoshua%2E+Show%2C+attend+and+tell%3A+Neural+image+caption+generation+with+visua) [our card](https://md.research.elamur.ai/free/en/papers/xu-2015-show-attend-tell-neural-image/)