---
title: "Show, Attend and Tell — Xu 2015"
description: "The paper looks at image captioning: generating a natural-language sentence that describes a photo."
authors: ["Kelvin Xu", "Jimmy Ba", "Ryan Kiros", "Kyunghyun Cho", "Aaron Courville", "Ruslan Salakhutdinov", "Richard Zemel", "Yoshua Bengio"]
year: 2015
venue: "arXiv:1502.03044"
source: "https://arxiv.org/abs/1502.03044"
doi: "10.48550/arxiv.1502.03044"
arxiv: "1502.03044"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xu-2015-show-attend-tell-neural-image/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xu-2015-show-attend-tell-neural-image/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/) / Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

# Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

**Authors:** [Kelvin Xu](https://md.research.elamur.ai/free/en/authors/kelvin-xu/), [Jimmy Ba](https://md.research.elamur.ai/free/en/authors/jimmy-ba/), [Ryan Kiros](https://md.research.elamur.ai/free/en/authors/ryan-kiros/), [Kyunghyun Cho](https://md.research.elamur.ai/free/en/authors/kyunghyun-cho/), [Aaron Courville](https://md.research.elamur.ai/free/en/authors/aaron-courville/), [Ruslan Salakhutdinov](https://md.research.elamur.ai/free/en/authors/ruslan-salakhutdinov/), [Richard Zemel](https://md.research.elamur.ai/free/en/authors/richard-zemel/), [Yoshua Bengio](https://md.research.elamur.ai/free/en/authors/yoshua-bengio/) · **Year:** 2015 · **Venue:** arXiv:1502.03044 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1502.03044) · [PDF](https://arxiv.org/pdf/1502.03044) · [DOI 10.48550/arxiv.1502.03044](https://doi.org/10.48550/arxiv.1502.03044) · [arXiv 1502.03044](https://arxiv.org/abs/1502.03044) · [OpenAlex](https://openalex.org/W1514535095)

**Citations:** 6862 citations (openalex-snapshot · 2026-10-14) · [621](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper looks at image captioning: generating a natural-language sentence that describes a photo. Earlier neural captioners squeezed the whole image into one vector taken from the top of a convnet, which can throw away detail. The authors keep a spatial grid of lower-layer convolutional features from the 19-layer Oxford VGG network. An LSTM decoder then learns to weight different image regions as it produces each word. They propose two versions of this attention. The "soft" version is deterministic and trained with ordinary backpropagation. The "hard" version picks one location stochastically and is trained by maximizing a variational lower bound, which is equivalent to REINFORCE. They evaluate on Flickr8k, Flickr30k and MS COCO using BLEU and METEOR and report state-of-the-art results from a single model without ensembling. Plotting the attention weights shows where the model looks for each word, and these alignments mostly match human intuition. The maps also help explain errors. This work became a foundational reference for visual attention in encoder–decoder models.

## Key points
- The encoder uses a 14×14 feature map from a lower convolutional layer of VGG, not a fully connected layer. Attention is computed over these regions by an MLP conditioned on the previous LSTM hidden state.
- Hard attention scored best on BLEU-4 on all three datasets: 21.3 on Flickr8k, 19.9 on Flickr30k and 25.0 on MS COCO. Soft attention scored 19.5, 19.1 and 24.3.
- On MS COCO, soft attention reached a METEOR of 23.90, a large improvement over earlier reported scores.
- Setup: vocabulary fixed at 10,000 words, the public Karpathy & Li splits for Flickr30k and COCO, and a single model compared only against systems using GoogLeNet or VGG features.
- The model attends to regions without using object detectors, so it can also focus on salient areas that aren't objects.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

621 papers of our library cite it (of the 140673 whose bibliographies we read); 67 of them have a page here, 554 not yet.

- [Quantifying Attention Flow in Transformers](https://md.research.elamur.ai/free/en/papers/abnar-2020-quantifying-attention-flow-transformers/)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://md.research.elamur.ai/free/en/papers/anderson-2016-spice-semantic-propositional-image-caption/)
- [Neural Module Networks](https://md.research.elamur.ai/free/en/papers/andreas-2015-neural-module-networks/)
- [Learning to Compose Neural Networks for Question Answering](https://md.research.elamur.ai/free/en/papers/andreas-2016-learning-compose-neural-networks-question/)
- [An Actor-Critic Algorithm for Sequence Prediction](https://md.research.elamur.ai/free/en/papers/bahdanau-2016-actor-critic-algorithm-sequence-prediction/)
- [Massive Exploration of Neural Machine Translation Architectures](https://md.research.elamur.ai/free/en/papers/britz-2017-massive-exploration-neural-machine-translation/)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://md.research.elamur.ai/free/en/papers/camburu-2018-e-snli-natural-language-inference/)
- [Neural Legal Judgment Prediction in English](https://md.research.elamur.ai/free/en/papers/chalkidis-2019-neural-legal-judgment-prediction-english/)
- [Attention-Based Models for Speech Recognition](https://md.research.elamur.ai/free/en/papers/chorowski-2015-attention-based-models-speech-recognition/)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://md.research.elamur.ai/free/en/papers/donahue-2014-long-term-recurrent-convolutional-networks/)
- [Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring](https://md.research.elamur.ai/free/en/papers/dong-2017-attention-based-recurrent-convolutional-neural/)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://md.research.elamur.ai/free/en/papers/du-2017-learning-ask-neural-question-generation/)
- [Multi30K: Multilingual English-German Image Descriptions](https://md.research.elamur.ai/free/en/papers/elliott-2016-multi30k-multilingual-english-german-image/)
- [Linguistic realisation as machine translation: Comparing different MT models for AMR-to-text generation](https://md.research.elamur.ai/free/en/papers/ferreira-2017-linguistic-realisation-as-machine-translation/)
- [Sentence Compression by Deletion with LSTMs](https://md.research.elamur.ai/free/en/papers/filippova-2015-sentence-compression-deletion-lstms/)
- [Sarcasm Analysis Using Conversation Context](https://md.research.elamur.ai/free/en/papers/ghosh-2018-sarcasm-analysis-conversation-context/)
- [Attention Guided Graph Convolutional Networks for Relation Extraction](https://md.research.elamur.ai/free/en/papers/guo-2019-attention-guided-graph-convolutional-networks/)
- [Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions](https://md.research.elamur.ai/free/en/papers/han-2020-explaining-black-box-predictions-unveiling/)
- [The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations](https://md.research.elamur.ai/free/en/papers/hill-2015-goldilocks-principle-reading-children-s/)
- [Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search](https://md.research.elamur.ai/free/en/papers/hokamp-2017-lexically-constrained-decoding-sequence-generation/)
- [Visual Storytelling](https://md.research.elamur.ai/free/en/papers/huang-2016-visual-storytelling/)
- [Attention is not Explanation](https://md.research.elamur.ai/free/en/papers/jain-2019-attention-is-not-explanation/)
- [Categorical Reparameterization with Gumbel-Softmax](https://md.research.elamur.ai/free/en/papers/jang-2016-categorical-reparameterization-gumbel-softmax/)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/)
- [Globally Coherent Text Generation with Neural Checklist Models](https://md.research.elamur.ai/free/en/papers/kiddon-2016-globally-coherent-text-generation-neural/)
- [Controlling Output Length in Neural Encoder-Decoders](https://md.research.elamur.ai/free/en/papers/kikuchi-2016-controlling-output-length-neural-encoder/)
- [OpenNMT: Open-Source Toolkit for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/klein-2017-opennmt-open-source-toolkit-neural/)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://md.research.elamur.ai/free/en/papers/krishna-2016-visual-genome-connecting-language-vision/)
- [Ask Me Anything: Dynamic Memory Networks for Natural Language Processing](https://md.research.elamur.ai/free/en/papers/kumar-2015-ask-me-anything-dynamic-memory/)
- [Neural Text Generation from Structured Data with Application to the Biography Domain](https://md.research.elamur.ai/free/en/papers/lebret-2016-neural-text-generation-structured-data/)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://md.research.elamur.ai/free/en/papers/li-2016-diversity-promoting-objective-function-neural/)
- [Transformation Networks for Target-Oriented Sentiment Classification](https://md.research.elamur.ai/free/en/papers/li-2018-transformation-networks-target-oriented-sentiment/)
- [Multi-task Sequence to Sequence Learning](https://md.research.elamur.ai/free/en/papers/luong-2015-multi-task-sequence-sequence-learning/)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://md.research.elamur.ai/free/en/papers/maddison-2016-concrete-distribution-continuous-relaxation-discrete/)
- [Encode, Tag, Realize: High-Precision Text Editing](https://md.research.elamur.ai/free/en/papers/malmi-2019-encode-tag-realize-high-precision/)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://md.research.elamur.ai/free/en/papers/mao-2015-generation-comprehension-unambiguous-object-descriptions/)
- [What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment](https://md.research.elamur.ai/free/en/papers/mei-2016-what-talk-about-how-selective/)
- [Ranking Sentences for Extractive Summarization with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/narayan-2018-ranking-sentences-extractive-summarization-reinforcement/)
- [Style Transfer Through Back-Translation](https://md.research.elamur.ai/free/en/papers/prabhumoye-2018-style-transfer-through-back-translation/)
- [Sequence Level Training with Recurrent Neural Networks](https://md.research.elamur.ai/free/en/papers/ranzato-2015-sequence-level-training-recurrent-neural/)
- [Self-critical Sequence Training for Image Captioning](https://md.research.elamur.ai/free/en/papers/rennie-2016-self-critical-sequence-training-image/)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://md.research.elamur.ai/free/en/papers/ribeiro-2016-why-should-i-trust-explaining/)
- [Evaluating Historical Text Normalization Systems: How Well Do They Generalize?](https://md.research.elamur.ai/free/en/papers/robertson-2018-evaluating-historical-text-normalization-systems/)
- [Reasoning about Entailment with Neural Attention](https://md.research.elamur.ai/free/en/papers/rocktaschel-2015-reasoning-about-entailment-neural-attention/)
- [Object Hallucination in Image Captioning](https://md.research.elamur.ai/free/en/papers/rohrbach-2018-object-hallucination-image-captioning/)
- [Efficient Content-Based Sparse Attention with Routing Transformers](https://md.research.elamur.ai/free/en/papers/roy-2020-efficient-content-based-sparse-attention/)
- [A simple neural network module for relational reasoning](https://md.research.elamur.ai/free/en/papers/santoro-2017-simple-neural-network-module-relational/)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://md.research.elamur.ai/free/en/papers/schuhmann-2022-laion-5b-open-large-scale/)
- [Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning](https://md.research.elamur.ai/free/en/papers/sharma-2018-conceptual-captions-cleaned-hypernymed-image/)
- [ViperGPT: Visual Inference via Python Execution for Reasoning](https://md.research.elamur.ai/free/en/papers/suris-2023-vipergpt-visual-inference-python-execution/)
- [Document Modeling with Gated Recurrent Neural Network for Sentiment Classification](https://md.research.elamur.ai/free/en/papers/tang-2015-document-modeling-gated-recurrent-neural/)
- [Modeling Coverage for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/tu-2016-modeling-coverage-neural-machine-translation/)
- [Maintaining Common Ground in Dynamic Environments](https://md.research.elamur.ai/free/en/papers/udagawa-2021-maintaining-common-ground-dynamic-environments/)
- [Attention is not not Explanation](https://md.research.elamur.ai/free/en/papers/wiegreffe-2019-attention-is-not-not-explanation/)
- [Hierarchical Attention Networks for Document Classification](https://md.research.elamur.ai/free/en/papers/yang-2016-hierarchical-attention-networks-document-classification/)
- [ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs](https://md.research.elamur.ai/free/en/papers/yin-2016-abcnn-attention-based-convolutional-neural/)
- [Modeling Context in Referring Expressions](https://md.research.elamur.ai/free/en/papers/yu-2016-modeling-context-referring-expressions/)
- [ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering](https://md.research.elamur.ai/free/en/papers/yu-2019-activitynet-qa-dataset-understanding-complex/)
- [What You See is What You Get: Visual Pronoun Coreference Resolution in Dialogues](https://md.research.elamur.ai/free/en/papers/yu-2019-what-you-see-is-what/)
- [Sentence Simplification with Deep Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/zhang-2017-sentence-simplification-deep-reinforcement-learning/)
- [Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks](https://md.research.elamur.ai/free/en/papers/zhang-2019-aspect-based-sentiment-classification-aspect/)
- [Integrating Transformer and Paraphrase Rules for Sentence Simplification](https://md.research.elamur.ai/free/en/papers/zhao-2018-integrating-transformer-paraphrase-rules-sentence/)
- [Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification](https://md.research.elamur.ai/free/en/papers/zhou-2016-attention-based-bidirectional-long-short/)
- [Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network](https://md.research.elamur.ai/free/en/papers/zhou-2018-multi-turn-response-selection-chatbots/)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://md.research.elamur.ai/free/en/papers/zhu-2015-aligning-books-movies-towards-story/)
- [Visual7W: Grounded Question Answering in Images](https://md.research.elamur.ai/free/en/papers/zhu-2015-visual7w-grounded-question-answering-images/)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://md.research.elamur.ai/free/en/papers/zhu-2018-texygen-benchmarking-platform-text-generation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{xu2015show,
  title = {{Show, Attend and Tell: Neural Image Caption Generation with Visual Attention}},
  author = {Xu, Kelvin and Ba, Jimmy and Kiros, Ryan and Cho, Kyunghyun and Courville, Aaron and Salakhutdinov, Ruslan and Zemel, Richard and Bengio, Yoshua},
  year = {2015},
  doi = {10.48550/arxiv.1502.03044},
  eprint = {1502.03044},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1502.03044}
}
```

- [← previous Show Me Your Evidence - an Automatic Method for Context Dependent Evidence Detection](https://md.research.elamur.ai/free/en/papers/rinott-2015-show-me-your-evidence-automatic/)
- [next → SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation](https://md.research.elamur.ai/free/en/papers/hill-2015-simlex-999-evaluating-semantic-models/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/)

## References

46 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ba, Jimmy Lei, Mnih, Volodymyr, and Kavukcuoglu, Koray. Multiple object recognition with visual attention. arXiv:1412.7755, December 2014. — [arXiv:1412.7755](https://arxiv.org/abs/1412.7755) [find](https://scholar.google.com/scholar?q=Ba%2C+Jimmy+Lei%2C+Mnih%2C+Volodymyr%2C+and+Kavukcuoglu%2C+Koray%2E+Multiple+object+recognition+with+visual+attention%2E+arXiv%3A1412%2E7755%2C+December+2014%2E+%E2%80%94+%5BarXiv%3A1412%2E7755%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1412%2E7755%29)
2. Bahdanau, Dzmitry, Cho, Kyunghyun, and Bengio, Yoshua. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473, September 2014. [find](https://scholar.google.com/scholar?q=Bahdanau%2C+Dzmitry%2C+Cho%2C+Kyunghyun%2C+and+Bengio%2C+Yoshua%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+arXiv%3A1409%2E0473%2C+September+2014%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Baldi, Pierre and Sadowski, Peter. The dropout learning algorithm. Artificial intelligence, 210:78–122, 2014. [find](https://scholar.google.com/scholar?q=Baldi%2C+Pierre+and+Sadowski%2C+Peter%2E+The+dropout+learning+algorithm%2E+Artificial+intelligence%2C+210%3A78%E2%80%93122%2C+2014%2E)
4. Bastien, Frederic, Lamblin, Pascal, Pascanu, Razvan, Bergstra, James, Goodfellow, Ian, Bergeron, Arnaud, Bouchard, Nicolas, Warde-Farley, David, and Bengio, Yoshua. Theano: new features and speed improvements. Submited to the Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop, 2012. [find](https://scholar.google.com/scholar?q=Bastien%2C+Frederic%2C+Lamblin%2C+Pascal%2C+Pascanu%2C+Razvan%2C+Bergstra%2C+James%2C+Goodfellow%2C+Ian%2C+Bergeron%2C+Arnaud%2C+Bouchard%2C+Nicolas%2C+Warde%2DFarley%2C+David%2C+and+Bengio%2C+Yoshua%2E+Theano%3A+new+features+and+speed+impr)
5. Bergstra, James, Breuleux, Olivier, Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), 2010. [find](https://scholar.google.com/scholar?q=Bergstra%2C+James%2C+Breuleux%2C+Olivier%2C+Bastien%2C+Fre%CC%81de%CC%81ric%2C+Lamblin%2C+Pascal%2C+Pascanu%2C+Razvan%2C+Desjardins%2C+Guillaume%2C+Turian%2C+Joseph%2C+Warde%2DFarley%2C+David%2C+and+Bengio%2C+Yoshua%2E+Theano%3A+a+CPU+and+GPU+math+ex)
6. Chen, Xinlei and Zitnick, C Lawrence. Learning a recurrent visual representation for image caption generation. arXiv preprint arXiv:1411.5654, 2014. — [arXiv:1411.5654](https://arxiv.org/abs/1411.5654) [find](https://scholar.google.com/scholar?q=Chen%2C+Xinlei+and+Zitnick%2C+C+Lawrence%2E+Learning+a+recurrent+visual+representation+for+image+caption+generation%2E+arXiv+preprint+arXiv%3A1411%2E5654%2C+2014%2E+%E2%80%94+%5BarXiv%3A1411%2E5654%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1411%2E5654)
7. Cho, Kyunghyun, van Merrienboer, Bart, Gulcehre, Caglar, Bougares, Fethi, Schwenk, Holger, and Bengio, Yoshua. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP, October 2014. [find](https://scholar.google.com/scholar?q=Cho%2C+Kyunghyun%2C+van+Merrienboer%2C+Bart%2C+Gulcehre%2C+Caglar%2C+Bougares%2C+Fethi%2C+Schwenk%2C+Holger%2C+and+Bengio%2C+Yoshua%2E+Learning+phrase+representations+using+RNN+encoder%2Ddecoder+for+statistical+machine+transla)
8. Corbetta, Maurizio and Shulman, Gordon L. Control of goal-directed and stimulus-driven attention in the brain. Nature reviews neuroscience, 3(3):201–215, 2002. [find](https://scholar.google.com/scholar?q=Corbetta%2C+Maurizio+and+Shulman%2C+Gordon+L%2E+Control+of+goal%2Ddirected+and+stimulus%2Ddriven+attention+in+the+brain%2E+Nature+reviews+neuroscience%2C+3%283%29%3A201%E2%80%93215%2C+2002%2E)
9. Denil, Misha, Bazzani, Loris, Larochelle, Hugo, and de Freitas, Nando. Learning where to attend with deep architectures for image tracking. Neural Computation, 2012. [find](https://scholar.google.com/scholar?q=Denil%2C+Misha%2C+Bazzani%2C+Loris%2C+Larochelle%2C+Hugo%2C+and+de+Freitas%2C+Nando%2E+Learning+where+to+attend+with+deep+architectures+for+image+tracking%2E+Neural+Computation%2C+2012%2E)
10. Denkowski, Michael and Lavie, Alon. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the EACL 2014 Workshop on Statistical Machine Translation, 2014. [find](https://scholar.google.com/scholar?q=Denkowski%2C+Michael+and+Lavie%2C+Alon%2E+Meteor+universal%3A+Language+specific+translation+evaluation+for+any+target+language%2E+In+Proceedings+of+the+EACL+2014+Workshop+on+Statistical+Machine+Translation%2C+201) [our card](https://md.research.elamur.ai/free/en/papers/denkowski-2014-meteor-universal-language-specific-translation/)
11. Donahue, Jeff, Hendrikcs, Lisa Anne, Guadarrama, Se-gio, Rohrbach, Marcus, Venugopalan, Subhashini, Saenko, Kate, and Darrell, Trevor. Long-term recurrent convolutional networks for visual recognition and description. arXiv:1411.4389v2, November 2014. [find](https://scholar.google.com/scholar?q=Donahue%2C+Jeff%2C+Hendrikcs%2C+Lisa+Anne%2C+Guadarrama%2C+Se%2Dgio%2C+Rohrbach%2C+Marcus%2C+Venugopalan%2C+Subhashini%2C+Saenko%2C+Kate%2C+and+Darrell%2C+Trevor%2E+Long%2Dterm+recurrent+convolutional+networks+for+visual+recognition) [our card](https://md.research.elamur.ai/free/en/papers/donahue-2014-long-term-recurrent-convolutional-networks/)
12. Elliott, Desmond and Keller, Frank. Image description using visual dependency representations. In EMNLP, 2013. [find](https://scholar.google.com/scholar?q=Elliott%2C+Desmond+and+Keller%2C+Frank%2E+Image+description+using+visual+dependency+representations%2E+In+EMNLP%2C+2013%2E)
13. Fang, Hao, Gupta, Saurabh, Iandola, Forrest, Srivastava, Rupesh, Deng, Li, Dollár, Piotr, Gao, Jianfeng, He, Xiaodong, Mitchell, Margaret, Platt, John, et al. From captions to visual concepts and back. arXiv:1411.4952, November 2014. [find](https://scholar.google.com/scholar?q=Fang%2C+Hao%2C+Gupta%2C+Saurabh%2C+Iandola%2C+Forrest%2C+Srivastava%2C+Rupesh%2C+Deng%2C+Li%2C+Dolla%CC%81r%2C+Piotr%2C+Gao%2C+Jianfeng%2C+He%2C+Xiaodong%2C+Mitchell%2C+Margaret%2C+Platt%2C+John%2C+et+al%2E+From+captions+to+visual+concepts+and+bac)
14. Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997. [find](https://scholar.google.com/scholar?q=Hochreiter%2C+S%2E+and+Schmidhuber%2C+J%2E+Long+short%2Dterm+memory%2E+Neural+Computation%2C+9%288%29%3A1735%E2%80%931780%2C+1997%2E)
15. Hodosh, Micah, Young, Peter, and Hockenmaier, Julia. Framing image description as a ranking task: Data, models and evaluation metrics. Journal of Artificial Intelligence Research, pp. 853–899, 2013. [find](https://scholar.google.com/scholar?q=Hodosh%2C+Micah%2C+Young%2C+Peter%2C+and+Hockenmaier%2C+Julia%2E+Framing+image+description+as+a+ranking+task%3A+Data%2C+models+and+evaluation+metrics%2E+Journal+of+Artificial+Intelligence+Research%2C+pp%2E+853%E2%80%93899%2C+2013%2E)
16. Karpathy, Andrej and Li, Fei-Fei. Deep visual-semantic alignments for generating image descriptions. arXiv:1412.2306, December 2014. [find](https://scholar.google.com/scholar?q=Karpathy%2C+Andrej+and+Li%2C+Fei%2DFei%2E+Deep+visual%2Dsemantic+alignments+for+generating+image+descriptions%2E+arXiv%3A1412%2E2306%2C+December+2014%2E) [our card](https://md.research.elamur.ai/free/en/papers/karpathy-2014-deep-visual-semantic-alignments-generating/)
17. Kingma, Diederik P. and Ba, Jimmy. Adam: A Method for Stochastic Optimization. arXiv:1412.6980, December 2014. [find](https://scholar.google.com/scholar?q=Kingma%2C+Diederik+P%2E+and+Ba%2C+Jimmy%2E+Adam%3A+A+Method+for+Stochastic+Optimization%2E+arXiv%3A1412%2E6980%2C+December+2014%2E)
18. Kiros, Ryan, Salahutdinov, Ruslan, and Zemel, Richard. Multimodal neural language models. In International Conference on Machine Learning, pp. 595–603, 2014a. [find](https://scholar.google.com/scholar?q=Kiros%2C+Ryan%2C+Salahutdinov%2C+Ruslan%2C+and+Zemel%2C+Richard%2E+Multimodal+neural+language+models%2E+In+International+Conference+on+Machine+Learning%2C+pp%2E+595%E2%80%93603%2C+2014a%2E)
19. Kiros, Ryan, Salakhutdinov, Ruslan, and Zemel, Richard. Unifying visual-semantic embeddings with multimodal neural language models. arXiv:1411.2539, November 2014b. [find](https://scholar.google.com/scholar?q=Kiros%2C+Ryan%2C+Salakhutdinov%2C+Ruslan%2C+and+Zemel%2C+Richard%2E+Unifying+visual%2Dsemantic+embeddings+with+multimodal+neural+language+models%2E+arXiv%3A1411%2E2539%2C+November+2014b%2E) [our card](https://md.research.elamur.ai/free/en/papers/kiros-2014-unifying-visual-semantic-embeddings-multimodal/)
20. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey. ImageNet classification with deep convolutional neural networks. In NIPS. 2012. [find](https://scholar.google.com/scholar?q=Krizhevsky%2C+Alex%2C+Sutskever%2C+Ilya%2C+and+Hinton%2C+Geoffrey%2E+ImageNet+classification+with+deep+convolutional+neural+networks%2E+In+NIPS%2E+2012%2E)
21. Kulkarni, Girish, Premraj, Visruth, Ordonez, Vicente, Dhar, Sagnik, Li, Siming, Choi, Yejin, Berg, Alexander C, and Berg, Tamara L. Babytalk: Understanding and generating simple image descriptions. PAMI, IEEE Transactions on, 35(12):2891– 2903, 2013. [find](https://scholar.google.com/scholar?q=Kulkarni%2C+Girish%2C+Premraj%2C+Visruth%2C+Ordonez%2C+Vicente%2C+Dhar%2C+Sagnik%2C+Li%2C+Siming%2C+Choi%2C+Yejin%2C+Berg%2C+Alexander+C%2C+and+Berg%2C+Tamara+L%2E+Babytalk%3A+Understanding+and+generating+simple+image+descriptions%2E+PA)
22. Kuznetsova, Polina, Ordonez, Vicente, Berg, Alexander C, Berg, Tamara L, and Choi, Yejin. Collective generation of natural image descriptions. In Association for Computational Linguistics. ACL, 2012. [find](https://scholar.google.com/scholar?q=Kuznetsova%2C+Polina%2C+Ordonez%2C+Vicente%2C+Berg%2C+Alexander+C%2C+Berg%2C+Tamara+L%2C+and+Choi%2C+Yejin%2E+Collective+generation+of+natural+image+descriptions%2E+In+Association+for+Computational+Linguistics%2E+ACL%2C+2012%2E)
23. Kuznetsova, Polina, Ordonez, Vicente, Berg, Tamara L, and Choi, Yejin. Treetalk: Composition and compression of trees for image descriptions. TACL, 2(10):351–362, 2014. [find](https://scholar.google.com/scholar?q=Kuznetsova%2C+Polina%2C+Ordonez%2C+Vicente%2C+Berg%2C+Tamara+L%2C+and+Choi%2C+Yejin%2E+Treetalk%3A+Composition+and+compression+of+trees+for+image+descriptions%2E+TACL%2C+2%2810%29%3A351%E2%80%93362%2C+2014%2E)
24. Larochelle, Hugo and Hinton, Geoffrey E. Learning to combine foveal glimpses with a third-order boltzmann machine. In NIPS, pp. 1243–1251, 2010. [find](https://scholar.google.com/scholar?q=Larochelle%2C+Hugo+and+Hinton%2C+Geoffrey+E%2E+Learning+to+combine+foveal+glimpses+with+a+third%2Dorder+boltzmann+machine%2E+In+NIPS%2C+pp%2E+1243%E2%80%931251%2C+2010%2E)
25. Li, Siming, Kulkarni, Girish, Berg, Tamara L, Berg, Alexander C, and Choi, Yejin. Composing simple image descriptions using web-scale n-grams. In Computational Natural Language Learning. ACL, 2011. [find](https://scholar.google.com/scholar?q=Li%2C+Siming%2C+Kulkarni%2C+Girish%2C+Berg%2C+Tamara+L%2C+Berg%2C+Alexander+C%2C+and+Choi%2C+Yejin%2E+Composing+simple+image+descriptions+using+web%2Dscale+n%2Dgrams%2E+In+Computational+Natural+Language+Learning%2E+ACL%2C+2011%2E)
26. Lin, Tsung-Yi, Maire, Michael, Belongie, Serge, Hays, James, Perona, Pietro, Ramanan, Deva, Dollár, Piotr, and Zitnick, C Lawrence. Microsoft coco: Common objects in context. In ECCV, pp. 740–755. 2014. [find](https://scholar.google.com/scholar?q=Lin%2C+Tsung%2DYi%2C+Maire%2C+Michael%2C+Belongie%2C+Serge%2C+Hays%2C+James%2C+Perona%2C+Pietro%2C+Ramanan%2C+Deva%2C+Dolla%CC%81r%2C+Piotr%2C+and+Zitnick%2C+C+Lawrence%2E+Microsoft+coco%3A+Common+objects+in+context%2E+In+ECCV%2C+pp%2E+740%E2%80%93755%2E+20) [our card](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/)
27. Mao, Junhua, Xu, Wei, Yang, Yi, Wang, Jiang, and Yuille, Alan. Deep captioning with multimodal recurrent neural networks (m-rnn). arXiv:1412.6632, December 2014. [find](https://scholar.google.com/scholar?q=Mao%2C+Junhua%2C+Xu%2C+Wei%2C+Yang%2C+Yi%2C+Wang%2C+Jiang%2C+and+Yuille%2C+Alan%2E+Deep+captioning+with+multimodal+recurrent+neural+networks+%28m%2Drnn%29%2E+arXiv%3A1412%2E6632%2C+December+2014%2E)
28. Mitchell, Margaret, Han, Xufeng, Dodge, Jesse, Mensch, Alyssa, Goyal, Amit, Berg, Alex, Yamaguchi, Kota, Berg, Tamara, Stratos, Karl, and Daumé III, Hal. Midge: Generating image descriptions from computer vision detections. In European Chapter of the Association for Computational Linguistics, pp. 7… [find](https://scholar.google.com/scholar?q=Mitchell%2C+Margaret%2C+Han%2C+Xufeng%2C+Dodge%2C+Jesse%2C+Mensch%2C+Alyssa%2C+Goyal%2C+Amit%2C+Berg%2C+Alex%2C+Yamaguchi%2C+Kota%2C+Berg%2C+Tamara%2C+Stratos%2C+Karl%2C+and+Daume%CC%81+III%2C+Hal%2E+Midge%3A+Generating+image+descriptions+from+com)
29. Mnih, Volodymyr, Hees, Nicolas, Graves, Alex, and Kavukcuoglu, Koray. Recurrent models of visual attention. In NIPS, 2014. [find](https://scholar.google.com/scholar?q=Mnih%2C+Volodymyr%2C+Hees%2C+Nicolas%2C+Graves%2C+Alex%2C+and+Kavukcuoglu%2C+Koray%2E+Recurrent+models+of+visual+attention%2E+In+NIPS%2C+2014%2E)
30. Pascanu, Razvan, Gulcehre, Caglar, Cho, Kyunghyun, and Bengio, Yoshua. How to construct deep recurrent neural networks. In ICLR, 2014. [find](https://scholar.google.com/scholar?q=Pascanu%2C+Razvan%2C+Gulcehre%2C+Caglar%2C+Cho%2C+Kyunghyun%2C+and+Bengio%2C+Yoshua%2E+How+to+construct+deep+recurrent+neural+networks%2E+In+ICLR%2C+2014%2E)
31. Rensink, Ronald A. The dynamic representation of scenes. Visual cognition, 7(1-3):17–42, 2000. [find](https://scholar.google.com/scholar?q=Rensink%2C+Ronald+A%2E+The+dynamic+representation+of+scenes%2E+Visual+cognition%2C+7%281%2D3%29%3A17%E2%80%9342%2C+2000%2E)
32. Russakovsky, Olga, Deng, Jia, Su, Hao, Krause, Jonathan, Satheesh, Sanjeev, Ma, Sean, Huang, Zhiheng, Karpathy, Andrej, Khosla, Aditya, Bernstein, Michael, Berg, Alexander C., and Fei-Fei, Li. ImageNet Large Scale Visual Recognition Challenge, 2014. [find](https://scholar.google.com/scholar?q=Russakovsky%2C+Olga%2C+Deng%2C+Jia%2C+Su%2C+Hao%2C+Krause%2C+Jonathan%2C+Satheesh%2C+Sanjeev%2C+Ma%2C+Sean%2C+Huang%2C+Zhiheng%2C+Karpathy%2C+Andrej%2C+Khosla%2C+Aditya%2C+Bernstein%2C+Michael%2C+Berg%2C+Alexander+C%2E%2C+and+Fei%2DFei%2C+Li%2E+ImageNe) [our card](https://md.research.elamur.ai/free/en/papers/russakovsky-2014-imagenet-large-scale-visual-recognition/)
33. Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014. [find](https://scholar.google.com/scholar?q=Simonyan%2C+K%2E+and+Zisserman%2C+A%2E+Very+deep+convolutional+networks+for+large%2Dscale+image+recognition%2E+CoRR%2C+abs%2F1409%2E1556%2C+2014%2E) [our card](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/)
34. Snoek, Jasper, Larochelle, Hugo, and Adams, Ryan P. Practical bayesian optimization of machine learning algorithms. In NIPS, pp. 2951–2959, 2012. [find](https://scholar.google.com/scholar?q=Snoek%2C+Jasper%2C+Larochelle%2C+Hugo%2C+and+Adams%2C+Ryan+P%2E+Practical+bayesian+optimization+of+machine+learning+algorithms%2E+In+NIPS%2C+pp%2E+2951%E2%80%932959%2C+2012%2E)
35. Snoek, Jasper, Swersky, Kevin, Zemel, Richard S, and Adams, Ryan P. Input warping for bayesian optimization of non-stationary functions. arXiv preprint arXiv:1402.0929, 2014. — [arXiv:1402.0929](https://arxiv.org/abs/1402.0929) [find](https://scholar.google.com/scholar?q=Snoek%2C+Jasper%2C+Swersky%2C+Kevin%2C+Zemel%2C+Richard+S%2C+and+Adams%2C+Ryan+P%2E+Input+warping+for+bayesian+optimization+of+non%2Dstationary+functions%2E+arXiv+preprint+arXiv%3A1402%2E0929%2C+2014%2E+%E2%80%94+%5BarXiv%3A1402%2E0929%5D%28https)
36. Srivastava, Nitish, Hinton, Geoffrey, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan. Dropout: A simple way to prevent neural networks from overfitting. JMLR, 15, 2014. [find](https://scholar.google.com/scholar?q=Srivastava%2C+Nitish%2C+Hinton%2C+Geoffrey%2C+Krizhevsky%2C+Alex%2C+Sutskever%2C+Ilya%2C+and+Salakhutdinov%2C+Ruslan%2E+Dropout%3A+A+simple+way+to+prevent+neural+networks+from+overfitting%2E+JMLR%2C+15%2C+2014%2E)
37. Sutskever, Ilya, Vinyals, Oriol, and Le, Quoc VV. Sequence to sequence learning with neural networks. In NIPS, pp. 3104– 3112, 2014. [find](https://scholar.google.com/scholar?q=Sutskever%2C+Ilya%2C+Vinyals%2C+Oriol%2C+and+Le%2C+Quoc+VV%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+NIPS%2C+pp%2E+3104%E2%80%93+3112%2C+2014%2E)
38. Szegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet, Pierre, Reed, Scott, Anguelov, Dragomir, Erhan, Dumitru, Vanhoucke, Vincent, and Rabinovich, Andrew. Going deeper with convolutions. arXiv preprint arXiv:1409.4842, 2014. — [arXiv:1409.4842](https://arxiv.org/abs/1409.4842) [find](https://scholar.google.com/scholar?q=Szegedy%2C+Christian%2C+Liu%2C+Wei%2C+Jia%2C+Yangqing%2C+Sermanet%2C+Pierre%2C+Reed%2C+Scott%2C+Anguelov%2C+Dragomir%2C+Erhan%2C+Dumitru%2C+Vanhoucke%2C+Vincent%2C+and+Rabinovich%2C+Andrew%2E+Going+deeper+with+convolutions%2E+arXiv+prepri)
39. Tang, Yichuan, Srivastava, Nitish, and Salakhutdinov, Ruslan R. Learning generative models with visual attention. In NIPS, pp. 1808–1816, 2014. [find](https://scholar.google.com/scholar?q=Tang%2C+Yichuan%2C+Srivastava%2C+Nitish%2C+and+Salakhutdinov%2C+Ruslan+R%2E+Learning+generative+models+with+visual+attention%2E+In+NIPS%2C+pp%2E+1808%E2%80%931816%2C+2014%2E)
40. Tieleman, Tijmen and Hinton, Geoffrey. Lecture 6.5 - rmsprop. Technical report, 2012. [find](https://scholar.google.com/scholar?q=Tieleman%2C+Tijmen+and+Hinton%2C+Geoffrey%2E+Lecture+6%2E5+%2D+rmsprop%2E+Technical+report%2C+2012%2E)
41. Vinyals, Oriol, Toshev, Alexander, Bengio, Samy, and Erhan, Dumitru. Show and tell: A neural image caption generator. arXiv:1411.4555, November 2014. [find](https://scholar.google.com/scholar?q=Vinyals%2C+Oriol%2C+Toshev%2C+Alexander%2C+Bengio%2C+Samy%2C+and+Erhan%2C+Dumitru%2E+Show+and+tell%3A+A+neural+image+caption+generator%2E+arXiv%3A1411%2E4555%2C+November+2014%2E) [our card](https://md.research.elamur.ai/free/en/papers/vinyals-2014-show-tell-neural-image-caption/)
42. Weaver, Lex and Tao, Nigel. The optimal reward baseline for gradient-based reinforcement learning. In Proc. UAI’2001, pp. 538–545, 2001. [find](https://scholar.google.com/scholar?q=Weaver%2C+Lex+and+Tao%2C+Nigel%2E+The+optimal+reward+baseline+for+gradient%2Dbased+reinforcement+learning%2E+In+Proc%2E+UAI%E2%80%992001%2C+pp%2E+538%E2%80%93545%2C+2001%2E)
43. Williams, Ronald J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229–256, 1992. [find](https://scholar.google.com/scholar?q=Williams%2C+Ronald+J%2E+Simple+statistical+gradient%2Dfollowing+algorithms+for+connectionist+reinforcement+learning%2E+Machine+learning%2C+8%283%2D4%29%3A229%E2%80%93256%2C+1992%2E)
44. Yang, Yezhou, Teo, Ching Lik, Daumé III, Hal, and Aloimonos, Yiannis. Corpus-guided sentence generation of natural images. In EMNLP, pp. 444–454. ACL, 2011. [find](https://scholar.google.com/scholar?q=Yang%2C+Yezhou%2C+Teo%2C+Ching+Lik%2C+Daume%CC%81+III%2C+Hal%2C+and+Aloimonos%2C+Yiannis%2E+Corpus%2Dguided+sentence+generation+of+natural+images%2E+In+EMNLP%2C+pp%2E+444%E2%80%93454%2E+ACL%2C+2011%2E)
45. Young, Peter, Lai, Alice, Hodosh, Micah, and Hockenmaier, Julia. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. TACL, 2:67–78, 2014. [find](https://scholar.google.com/scholar?q=Young%2C+Peter%2C+Lai%2C+Alice%2C+Hodosh%2C+Micah%2C+and+Hockenmaier%2C+Julia%2E+From+image+descriptions+to+visual+denotations%3A+New+similarity+metrics+for+semantic+inference+over+event+descriptions%2E+TACL%2C+2%3A67%E2%80%9378%2C+20)
46. Zaremba, Wojciech, Sutskever, Ilya, and Vinyals, Oriol. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329, September 2014. [find](https://scholar.google.com/scholar?q=Zaremba%2C+Wojciech%2C+Sutskever%2C+Ilya%2C+and+Vinyals%2C+Oriol%2E+Recurrent+neural+network+regularization%2E+arXiv+preprint+arXiv%3A1409%2E2329%2C+September+2014%2E) [our card](https://md.research.elamur.ai/free/en/papers/zaremba-2014-recurrent-neural-network-regularization/)