---
title: "Massive Exploration of Neural Machine — Britz 2017"
description: "NMT models are costly to train, so researchers tend to reuse familiar architectures without knowing which design decisions actually drive translation…"
authors: ["Denny Britz", "Anna Goldie", "Minh-Thang Luong", "Quoc Le"]
year: 2017
venue: "Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D17-1151/"
doi: "10.18653/v1/d17-1151"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/britz-2017-massive-exploration-neural-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/britz-2017-massive-exploration-neural-machine-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/) / Massive Exploration of Neural Machine Translation Architectures

# Massive Exploration of Neural Machine Translation Architectures

**Authors:** [Denny Britz](https://md.research.elamur.ai/free/en/authors/denny-britz/), [Anna Goldie](https://md.research.elamur.ai/free/en/authors/anna-goldie/), [Minh-Thang Luong](https://md.research.elamur.ai/free/en/authors/minh-thang-luong/), [Quoc Le](https://md.research.elamur.ai/free/en/authors/quoc-v-le/) · **Year:** 2017 · **Venue:** Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D17-1151/) · [PDF](https://aclanthology.org/D17-1151.pdf) · [DOI 10.18653/v1/d17-1151](https://doi.org/10.18653/v1/d17-1151) · [OpenAlex](https://openalex.org/W2594990650)

**Citations:** 482 citations (OpenAlex, 2026-10-14) · [80](https://md.research.elamur.ai/free/en/papers/britz-2017-massive-exploration-neural-machine-translation/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
NMT models are costly to train, so researchers tend to reuse familiar architectures without knowing which design decisions actually drive translation quality. Google Brain researchers ran a systematic hyperparameter sweep to answer this. They used several hundred training runs, totalling more than 250,000 GPU hours, on WMT'15 English→German (4.5M sentence pairs, BPE subwords). Starting from a simple attention-based encoder-decoder baseline, they changed one factor at a time: embedding size, cell type, encoder and decoder depth, residual connections, attention variant, and beam search settings. Every configuration was trained four times with different initializations, so they could report means and standard deviations alongside the best scores.

The study found that LSTM beats GRU, that additive attention works best, that deep encoders are hard to optimize, and that a carefully tuned beam search adds more than a BLEU point. Combining the individually best settings gave results competitive with the state of the art, behind only the far more complex GNMT. The authors released an open-source TensorFlow framework together with all configurations and preprocessing scripts, which makes the study a reproducible reference for baseline choices.

## Key points
- **Setup:**
  - Data: WMT'15 En→De drawn from three sources (the Common Crawl corpus, version 10 of News Commentary, and version 7 of Europarl), with newstest2013 for validation and newstest2014/2015 for testing.
  - Preprocessing: Moses tokenization, and shared BPE with 32k merges giving a vocabulary of about 37k.
  - Scoring: BLEU computed with multi-bleu.perl.
- **Baseline:** 512-unit GRU cells, a bidirectional encoder, a 2-layer decoder, multiplicative attention, Adam with a learning rate of 0.0001, dropout of 0.2, 512-dimensional embeddings, and beam width 10. Each run lasted up to 2.5M steps and was repeated 4 times.
- **Embeddings and cells:**
  - 2048-dimensional embeddings scored highest (21.86 BLEU on newstest2013), only slightly above 128-dimensional ones (21.50), and took roughly twice as long to converge.
  - LSTM cells (22.22) significantly outperformed GRU cells (21.78; p \< 0.00001).
  - A vanilla RNN decoder collapsed to 15.38.
- **Architecture and decoding:**
  - Bidirectional encoders with 2–4 layers worked best.
  - 4-layer decoders were slightly better than shallower ones, and 8-layer decoders could only be trained with residual connections.
  - Beam widths of 5–10 with a length penalty of 1.0 worked well, while very large beams hurt quality.
  - A coverage penalty gave no gain beyond a sufficiently large length penalty.
- **Generalization:** The same configuration reached 38.87 BLEU on ASPEC Japanese→English, which the authors report as state-of-the-art.

## Interesting ideas
- **Report variance across seeds.** Running each configuration with several random initializations shows how much BLEU moves from noise alone. This makes it possible to judge whether a small improvement in any NLP benchmark is real.
- **Vary one factor at a time from a plain baseline.** Interactions are tested only where they are likely, such as depth combined with residual connections. This keeps a large search interpretable and is a cheap template for ablations on limited compute.
- **Tune decoding as seriously as the model.** Beam width and length penalty shifted scores about as much as architectural changes did. Any generation system should therefore sweep its inference settings before anyone concludes that one model beats another.
- **Ship preprocessing with the model.** The authors observed that preprocessing strongly affects final scores, so they released the scripts and configuration files alongside the code. This is a good practice for any shared corpus or model library.
- **Check whether extra capacity is actually used.** Training perplexity, gradient norms and signs of overfitting showed that large embeddings added parameters without meaningful benefit. The same diagnostic can reveal wasted model size in other architectures.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{britz2017massive,
  title = {{Massive Exploration of Neural Machine Translation Architectures}},
  author = {Britz, Denny and Goldie, Anna and Luong, Minh-Thang and Le, Quoc},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1151},
  url = {https://doi.org/10.18653/v1/d17-1151}
}
```

- [← previous Sentence Simplification with Deep Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/zhang-2017-sentence-simplification-deep-reinforcement-learning/)
- [next → Simple and Effective Multi-Paragraph Reading Comprehension](https://md.research.elamur.ai/free/en/papers/clark-2017-simple-effective-multi-paragraph-reading/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/)

## References

32 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yua… [find](https://scholar.google.com/scholar?q=Martin+Abadi%2C+Paul+Barham%2C+Jianmin+Chen%2C+Zhifeng+Chen%2C+Andy+Davis%2C+Jeffrey+Dean%2C+Matthieu+Devin%2C+Sanjay+Ghemawat%2C+Geoffrey+Irving%2C+Michael+Isard%2C+Manjunath+Kudlur%2C+Josh+Levenberg%2C+Rajat+Monga%2C+Sherry+)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+van+Merrienboer%2C+C%CC%A7aglar+Gu%CC%88lc%CC%A7ehre%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014%2E+Learning+phrase+representations+using+RNN+encoder%2Ddecoder+for+statistical+machine+tran)
4. Junyoung Chung, Kyunghyun Cho, and Yoshua Bengio. 2016. A character-level decoder without explicit segmentation for neural machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Junyoung+Chung%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2016%2E+A+character%2Dlevel+decoder+without+explicit+segmentation+for+neural+machine+translation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/chung-2016-character-level-decoder-without-explicit/)
5. Jonas Gehring, Michael Auli, David Grangier, and Yann N. Dauphin. 2016. A convolutional encoder model for neural machine translation. CoRR abs/1611.02344. [find](https://scholar.google.com/scholar?q=Jonas+Gehring%2C+Michael+Auli%2C+David+Grangier%2C+and+Yann+N%2E+Dauphin%2E+2016%2E+A+convolutional+encoder+model+for+neural+machine+translation%2E+CoRR+abs%2F1611%2E02344%2E) [our card](https://md.research.elamur.ai/free/en/papers/gehring-2017-convolutional-encoder-model-neural-machine/)
6. Klaus Greff, Rupesh Kumar Srivastava, Jan Koutnı́k, Bas R. Steunebrink, and Jürgen Schmidhuber. 2016. LSTM: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems PP(99):1–11. [find](https://scholar.google.com/scholar?q=Klaus+Greff%2C+Rupesh+Kumar+Srivastava%2C+Jan+Koutn%C4%B1%CC%81k%2C+Bas+R%2E+Steunebrink%2C+and+Ju%CC%88rgen+Schmidhuber%2E+2016%2E+LSTM%3A+A+search+space+odyssey%2E+IEEE+Transactions+on+Neural+Networks+and+Learning+Systems+PP%2899%29%3A1%E2%80%93)
7. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. [find](https://scholar.google.com/scholar?q=Kaiming+He%2C+Xiangyu+Zhang%2C+Shaoqing+Ren%2C+and+Jian+Sun%2E+2016%2E+Deep+residual+learning+for+image+recognition%2E+In+CVPR%2E)
8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+Computation+9%288%29%3A1735%E2%80%931780%2E)
9. Gao Huang, Zhuang Liu, and Kilian Q. Weinberger. 2016a. Densely connected convolutional networks. CoRR abs/1608.06993. — [arXiv:1608.06993](https://arxiv.org/abs/1608.06993) [find](https://scholar.google.com/scholar?q=Gao+Huang%2C+Zhuang+Liu%2C+and+Kilian+Q%2E+Weinberger%2E+2016a%2E+Densely+connected+convolutional+networks%2E+CoRR+abs%2F1608%2E06993%2E+%E2%80%94+%5BarXiv%3A1608%2E06993%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1608%2E06993%29)
10. Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, and Kevin Murphy. 2016b. Speed/accuracy trade-offs for modern convolutional object detectors. CoRR abs/1611.10012. — [arXiv:1611.10012](https://arxiv.org/abs/1611.10012) [find](https://scholar.google.com/scholar?q=Jonathan+Huang%2C+Vivek+Rathod%2C+Chen+Sun%2C+Menglong+Zhu%2C+Anoop+Korattikara%2C+Alireza+Fathi%2C+Ian+Fischer%2C+Zbigniew+Wojna%2C+Yang+Song%2C+Sergio+Guadarrama%2C+and+Kevin+Murphy%2E+2016b%2E+Speed%2Faccuracy+trade%2Doffs+fo)
11. Sébastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. On using very large target vocabulary for neural machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Se%CC%81bastien+Jean%2C+Kyunghyun+Cho%2C+Roland+Memisevic%2C+and+Yoshua+Bengio%2E+2015%2E+On+using+very+large+target+vocabulary+for+neural+machine+translation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/jean-2015-very-large-target-vocabulary-neural/)
12. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In EMNLP. [find](https://scholar.google.com/scholar?q=Nal+Kalchbrenner+and+Phil+Blunsom%2E+2013%2E+Recurrent+continuous+translation+models%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/)
13. Andrej Karpathy and Li Fei-Fei. 2015. Deep visual-semantic alignments for generating image descriptions. In CVPR. [find](https://scholar.google.com/scholar?q=Andrej+Karpathy+and+Li+Fei%2DFei%2E+2015%2E+Deep+visual%2Dsemantic+alignments+for+generating+image+descriptions%2E+In+CVPR%2E) [our card](https://md.research.elamur.ai/free/en/papers/karpathy-2014-deep-visual-semantic-alignments-generating/)
14. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. Open-NMT: Open-source toolkit for neural machine translation. CoRR abs/1701.02810. — [arXiv:1701.02810](https://arxiv.org/abs/1701.02810) [find](https://scholar.google.com/scholar?q=Guillaume+Klein%2C+Yoon+Kim%2C+Yuntian+Deng%2C+Jean+Senellart%2C+and+Alexander+M%2E+Rush%2E+2017%2E+Open%2DNMT%3A+Open%2Dsource+toolkit+for+neural+machine+translation%2E+CoRR+abs%2F1701%2E02810%2E+%E2%80%94+%5BarXiv%3A1701%2E02810%5D%28https%3A%2F%2Far)
15. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055 . [find](https://scholar.google.com/scholar?q=Jiwei+Li%2C+Michel+Galley%2C+Chris+Brockett%2C+Jianfeng+Gao%2C+and+Bill+Dolan%2E+2015%2E+A+diversity%2Dpromoting+objective+function+for+neural+conversation+models%2E+arXiv+preprint+arXiv%3A1510%2E03055+%2E) [our card](https://md.research.elamur.ai/free/en/papers/li-2016-diversity-promoting-objective-function-neural/)
16. Minh-Thang Luong and Christopher D. Manning. 2016. Achieving open vocabulary neural machine translation with hybrid word-character models. In ACL. [find](https://scholar.google.com/scholar?q=Minh%2DThang+Luong+and+Christopher+D%2E+Manning%2E+2016%2E+Achieving+open+vocabulary+neural+machine+translation+with+hybrid+word%2Dcharacter+models%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/luong-2016-achieving-open-vocabulary-neural-machine/)
17. Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015a. Effective approaches to attention-based neural machine translation. In EMNLP. [find](https://scholar.google.com/scholar?q=Minh%2DThang+Luong%2C+Hieu+Pham%2C+and+Christopher+D%2E+Manning%2E+2015a%2E+Effective+approaches+to+attention%2Dbased+neural+machine+translation%2E+In+EMNLP%2E)
18. Minh-Thang Luong, Ilya Sutskever, Quoc V. Le, Oriol Vinyals, and Wojciech Zaremba. 2015b. Addressing the rare word problem in neural machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Minh%2DThang+Luong%2C+Ilya+Sutskever%2C+Quoc+V%2E+Le%2C+Oriol+Vinyals%2C+and+Wojciech+Zaremba%2E+2015b%2E+Addressing+the+rare+word+problem+in+neural+machine+translation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/luong-2015-addressing-rare-word-problem-neural/)
19. Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023 . [find](https://scholar.google.com/scholar?q=Ramesh+Nallapati%2C+Bowen+Zhou%2C+Caglar+Gulcehre%2C+Bing+Xiang%2C+et+al%2E+2016%2E+Abstractive+text+summarization+using+sequence%2Dto%2Dsequence+rnns+and+beyond%2E+arXiv+preprint+arXiv%3A1602%2E06023+%2E)
20. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. Edinburgh neural machine translation systems for wmt 16. In ACL. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016a%2E+Edinburgh+neural+machine+translation+systems+for+wmt+16%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/sennrich-2016-edinburgh-neural-machine-translation-systems/)
21. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. Neural machine translation of rare words with subword units. In ACL. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016b%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+ACL%2E)
22. Lifeng Shang, Zhengdong Lu, and Hang Li. 2015. Neural responding machine for short-text conversation. arXiv preprint arXiv:1503.02364 . [find](https://scholar.google.com/scholar?q=Lifeng+Shang%2C+Zhengdong+Lu%2C+and+Hang+Li%2E+2015%2E+Neural+responding+machine+for+short%2Dtext+conversation%2E+arXiv+preprint+arXiv%3A1503%2E02364+%2E) [our card](https://md.research.elamur.ai/free/en/papers/shang-2015-neural-responding-machine-short-text/)
23. Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. arXiv preprint arXiv:1506.06714 . [find](https://scholar.google.com/scholar?q=Alessandro+Sordoni%2C+Michel+Galley%2C+Michael+Auli%2C+Chris+Brockett%2C+Yangfeng+Ji%2C+Margaret+Mitchell%2C+Jian%2DYun+Nie%2C+Jianfeng+Gao%2C+and+Bill+Dolan%2E+2015%2E+A+neural+network+approach+to+context%2Dsensitive+genera) [our card](https://md.research.elamur.ai/free/en/papers/sordoni-2015-neural-network-approach-context-sensitive/)
24. Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015. Highway networks. arXiv preprint arXiv:1505.00387 . [find](https://scholar.google.com/scholar?q=Rupesh+Kumar+Srivastava%2C+Klaus+Greff%2C+and+Ju%CC%88rgen+Schmidhuber%2E+2015%2E+Highway+networks%2E+arXiv+preprint+arXiv%3A1505%2E00387+%2E) [our card](https://md.research.elamur.ai/free/en/papers/srivastava-2015-highway-networks/)
25. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In NIPS. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V%2E+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+NIPS%2E)
26. Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. Neural machine translation with reconstruction. In AAAI. [find](https://scholar.google.com/scholar?q=Zhaopeng+Tu%2C+Yang+Liu%2C+Lifeng+Shang%2C+Xiaohua+Liu%2C+and+Hang+Li%2E+2017%2E+Neural+machine+translation+with+reconstruction%2E+In+AAAI%2E)
27. Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling coverage for neural machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Zhaopeng+Tu%2C+Zhengdong+Lu%2C+Yang+Liu%2C+Xiaohua+Liu%2C+and+Hang+Li%2E+2016%2E+Modeling+coverage+for+neural+machine+translation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/tu-2016-modeling-coverage-neural-machine-translation/)
28. Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869 . [find](https://scholar.google.com/scholar?q=Oriol+Vinyals+and+Quoc+Le%2E+2015%2E+A+neural+conversational+model%2E+arXiv+preprint+arXiv%3A1506%2E05869+%2E) [our card](https://md.research.elamur.ai/free/en/papers/vinyals-2015-neural-conversational-model/)
29. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. [find](https://scholar.google.com/scholar?q=Oriol+Vinyals%2C+Alexander+Toshev%2C+Samy+Bengio%2C+and+Dumitru+Erhan%2E+2015%2E+Show+and+tell%3A+A+neural+image+caption+generator%2E+In+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition) [our card](https://md.research.elamur.ai/free/en/papers/vinyals-2014-show-tell-neural-image-caption/)
30. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, N… [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V%2E+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+Jeff+Klingner%2C+Apurva+Shah%2C+Melvin+Johnson%2C+Xiaobing+Liu%2C+Luk)
31. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron C Courville, Ruslan Salakhutdinov, Richard S Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In ICML. [find](https://scholar.google.com/scholar?q=Kelvin+Xu%2C+Jimmy+Ba%2C+Ryan+Kiros%2C+Kyunghyun+Cho%2C+Aaron+C+Courville%2C+Ruslan+Salakhutdinov%2C+Richard+S+Zemel%2C+and+Yoshua+Bengio%2E+2015%2E+Show%2C+attend+and+tell%3A+Neural+image+caption+generation+with+visual+at) [our card](https://md.research.elamur.ai/free/en/papers/xu-2015-show-attend-tell-neural-image/)
32. Jie Zhou, Ying Cao, Xuguang Wang, Peng Li, and Wei Xu. 2016. Deep recurrent models with fast-forward connections for neural machine translation. TACL . [find](https://scholar.google.com/scholar?q=Jie+Zhou%2C+Ying+Cao%2C+Xuguang+Wang%2C+Peng+Li%2C+and+Wei+Xu%2E+2016%2E+Deep+recurrent+models+with+fast%2Dforward+connections+for+neural+machine+translation%2E+TACL+%2E)