---
title: "From Captions to Visual Concepts and Back — Fang 2014"
description: "The paper tackles automatic image captioning. 25.7% BLEU. 51 references, cited by 163 papers here."
authors: ["Hao Fang", "Saurabh Gupta", "Forrest Iandola", "Rupesh Srivastava", "Li Deng", "Piotr Dollar", "Jianfeng Gao", "Xiaodong He", "Margaret Mitchell", "John C. Platt", "C. Lawrence Zitnick", "Geoffrey Zweig"]
year: 2014
venue: "arXiv:1411.4952"
source: "https://arxiv.org/abs/1411.4952"
doi: "10.48550/arxiv.1411.4952"
arxiv: "1411.4952"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/fang-2014-captions-visual-concepts-back/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/fang-2014-captions-visual-concepts-back/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/) / From Captions to Visual Concepts and Back

# From Captions to Visual Concepts and Back

**Authors:** [Hao Fang](https://md.research.elamur.ai/free/en/authors/hao-fang/), [Saurabh Gupta](https://md.research.elamur.ai/free/en/authors/saurabh-gupta/), Forrest Iandola, Rupesh Srivastava, [Li Deng](https://md.research.elamur.ai/free/en/authors/li-deng/), [Piotr Dollar](https://md.research.elamur.ai/free/en/authors/piotr-dollar/), [Jianfeng Gao](https://md.research.elamur.ai/free/en/authors/jianfeng-gao/), [Xiaodong He](https://md.research.elamur.ai/free/en/authors/xiaodong-he/), [Margaret Mitchell](https://md.research.elamur.ai/free/en/authors/margaret-mitchell/), [John C. Platt](https://md.research.elamur.ai/free/en/authors/john-c-platt/), [C. Lawrence Zitnick](https://md.research.elamur.ai/free/en/authors/c-lawrence-zitnick/) and [Geoffrey Zweig](https://md.research.elamur.ai/free/en/authors/geoffrey-zweig/) · **Year:** 2014 · **Venue:** [arXiv:1411.4952](https://md.research.elamur.ai/free/en/venues/ieee-computer-society-conference-on-computer-vision-and-pattern-recognition/) ([IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2014](https://md.research.elamur.ai/free/en/venues/ieee-computer-society-conference-on-computer-vision-and-pattern-recognition/2014/)) · **Publisher:** [Institute of Electrical and Electronics Engineers](https://md.research.elamur.ai/free/en/publishers/institute-of-electrical-and-electronics-engineers/) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1411.4952) · [PDF](https://arxiv.org/pdf/1411.4952) · [DOI 10.48550/arxiv.1411.4952](https://doi.org/10.48550/arxiv.1411.4952) · [arXiv 1411.4952](https://arxiv.org/abs/1411.4952) · [OpenAlex](https://openalex.org/W1931639407)

**Citations:** 1,287 citations (OpenAlex, 2026-10-14) · [163](https://md.research.elamur.ai/free/en/papers/fang-2014-captions-visual-concepts-back/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles automatic image captioning. Its central idea is that every component of the system can be learned from caption-annotated images, with no separately hand-labeled object, attribute or relation data. The pipeline has three stages:

- **Word detection.** Detectors for frequent caption words (nouns, verbs and adjectives alike) are trained with weakly supervised Multiple Instance Learning (MIL) on CNN features of image regions.
- **Sentence generation.** The detector scores condition a maximum-entropy language model trained on more than 400,000 captions. A search then looks for high-likelihood sentences that use the detected words.
- **Re-ranking.** Candidate sentences are re-ranked with MERT-tuned sentence features, including a new Deep Multimodal Similarity Model (DMSM) that embeds images and text in a shared vector space.

On Microsoft COCO, the system was state-of-the-art on all 14 official metrics when the paper was written. It matched or beat human captions on 12 of them. Human judges rated its captions as good as or better than human-written ones about a third of the time. The work shows that captions alone are enough supervision for detectors, language statistics and global image–text similarity.

## Key points
- The vocabulary is the 1000 most frequent caption words, covering over 92% of word occurrences. Detectors use a noisy-OR MIL layer on top of a fully convolutional AlexNet or VGG, which produces a 12×12 response map per image.
- Training uses the Microsoft COCO train split: 82,783 images with 5 captions each.
- On the COCO evaluation server (40,775 test images), BLEU-4 reaches 29.1% versus 21.7% for humans. CIDEr is 0.912 versus 0.854 for humans, and this was the only system to beat humans on CIDEr.
- On the authors' held-out set, the best variant (VGG+Score+DMSM+ft) gets 25.7% BLEU (humans 19.3%) and is judged same-or-better than humans 34% of the time. Replacing MIL with whole-image classification lowers BLEU to 21.9%.
- On the PASCAL sentence dataset (847 images), the system strongly beats Midge: 17.6% vs 2.0% BLEU and 19.2% vs 9.2% METEOR.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 492 works · 440,463 citations
- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 378 works · 161,630 citations
- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 402 works · 212,763 citations
- [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 214 works · 241,492 citations
- [Dalle Molle Institute for Artificial Intelligence Research](https://md.research.elamur.ai/free/en/labs/dalle-molle-institute-for-artificial-intelligence-research/) — Research institute, Lugano, Switzerland · 10 works · 8,825 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

## Cite

Cite as: Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh Srivastava, Li Deng, Piotr Dollar, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John C. Platt, C. Lawrence Zitnick and Geoffrey Zweig. 2014. From Captions to Visual Concepts and Back. arXiv:1411.4952.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{fang2014captions,
  title = {{From Captions to Visual Concepts and Back}},
  author = {Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh Srivastava, Li Deng, Piotr Dollar, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John C. Platt, C. Lawrence Zitnick and Geoffrey Zweig},
  year = {2014},
  doi = {10.48550/arxiv.1411.4952},
  eprint = {1411.4952},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1411.4952}
}
```

- [← previous Deep Speech: Scaling up end-to-end speech recognition](https://md.research.elamur.ai/free/en/papers/hannun-2014-deep-speech-scaling-up-end/)
- [next → Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models](https://md.research.elamur.ai/free/en/papers/kiros-2014-unifying-visual-semantic-embeddings-multimodal/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/)

## References

51 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. S. Banerjee and A. Lavie. [METEOR: An automatic metric for MT evaluation with improved correlation with human judgments](https://md.research.elamur.ai/free/en/papers/banerjee-2005-meteor-automatic-metric-mt-evaluation/). In ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 2005. 2, 6
2. A. L. Berger, S. A. D. Pietra, and V. J. D. Pietra. [A maximum entropy approach to natural language processing](https://md.research.elamur.ai/free/en/papers/berger-1996-maximum-entropy-approach-natural-language/). Computational Linguistics, 1996. 2, 4
3. A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E. R. Hruschka Jr, and T. M. Mitchell. Toward an architecture for never-ending language learning. In AAAI, 2010. 2 [find](https://scholar.google.com/scholar?q=A%2E+Carlson%2C+J%2E+Betteridge%2C+B%2E+Kisiel%2C+B%2E+Settles%2C+E%2E+R%2E+Hruschka+Jr%2C+and+T%2E+M%2E+Mitchell%2E+Toward+an+architecture+for+never%2Dending+language+learning%2E+In+AAAI%2C+2010%2E+2)
4. X. Chen, H. Fang, T. Lin, R. Vedantam, S. Gupta, P. Dollár, and C. L. Zitnick. [Microsoft coco captions: Data collection and evaluation server](https://md.research.elamur.ai/free/en/papers/chen-2015-microsoft-coco-captions-data-collection/). arXiv preprint arXiv:1504.00325, 2015. 2, 6, 7
5. X. Chen, A. Shrivastava, and A. Gupta. Neil: Extracting visual knowledge from web data. In ICCV, 2013. 1 [find](https://scholar.google.com/scholar?q=X%2E+Chen%2C+A%2E+Shrivastava%2C+and+A%2E+Gupta%2E+Neil%3A+Extracting+visual+knowledge+from+web+data%2E+In+ICCV%2C+2013%2E+1)
6. X. Chen and C. L. Zitnick. Mind’s eye: A recurrent visual representation for image caption generation. CVPR, 2015. 2 [find](https://scholar.google.com/scholar?q=X%2E+Chen+and+C%2E+L%2E+Zitnick%2E+Mind%E2%80%99s+eye%3A+A+recurrent+visual+representation+for+image+caption+generation%2E+CVPR%2C+2015%2E+2)
7. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A large-scale hierarchical image database. In CVPR, 2009. 2, 3 [find](https://scholar.google.com/scholar?q=J%2E+Deng%2C+W%2E+Dong%2C+R%2E+Socher%2C+L%2E%2DJ%2E+Li%2C+K%2E+Li%2C+and+L%2E+Fei%2DFei%2E+ImageNet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+CVPR%2C+2009%2E+2%2C+3)
8. S. Divvala, A. Farhadi, and C. Guestrin. Learning everything about anything: Webly-supervised visual concept learning. In CVPR, 2014. 1 [find](https://scholar.google.com/scholar?q=S%2E+Divvala%2C+A%2E+Farhadi%2C+and+C%2E+Guestrin%2E+Learning+everything+about+anything%3A+Webly%2Dsupervised+visual+concept+learning%2E+In+CVPR%2C+2014%2E+1)
9. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. [Long-term recurrent convolutional networks for visual recognition and description](https://md.research.elamur.ai/free/en/papers/donahue-2014-long-term-recurrent-convolutional-networks/). CVPR, 2015. 2, 3
10. D. Elliott and F. Keller. Comparing automatic evaluation measures for image description. In ACL, 2014. 6 [find](https://scholar.google.com/scholar?q=D%2E+Elliott+and+F%2E+Keller%2E+Comparing+automatic+evaluation+measures+for+image+description%2E+In+ACL%2C+2014%2E+6)
11. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL visual object classes (VOC) challenge. IJCV, 88(2):303–338, June 2010. 2, 6 [find](https://scholar.google.com/scholar?q=M%2E+Everingham%2C+L%2E+Van+Gool%2C+C%2E+K%2E+I%2E+Williams%2C+J%2E+Winn%2C+and+A%2E+Zisserman%2E+The+PASCAL+visual+object+classes+%28VOC%29+challenge%2E+IJCV%2C+88%282%29%3A303%E2%80%93338%2C+June+2010%2E+2%2C+6)
12. A. Farhadi, M. Hejrati, M. A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth. Every picture tells a story: Generating sentences from images. In ECCV, 2010. 2 [find](https://scholar.google.com/scholar?q=A%2E+Farhadi%2C+M%2E+Hejrati%2C+M%2E+A%2E+Sadeghi%2C+P%2E+Young%2C+C%2E+Rashtchian%2C+J%2E+Hockenmaier%2C+and+D%2E+Forsyth%2E+Every+picture+tells+a+story%3A+Generating+sentences+from+images%2E+In+ECCV%2C+2010%2E+2)
13. R. Girshick, J. Donahue, T. Darrell, and J. Malik. [Rich feature hierarchies for accurate object detection and semantic segmentation](https://md.research.elamur.ai/free/en/papers/girshick-2013-rich-feature-hierarchies-accurate-object/). In CVPR, 2014. 3
14. G. Gkioxari, B. Hariharan, R. Girshick, and J. Malik. Using k-poselets for detecting people and localizing their keypoints. In CVPR, 2014. 4 [find](https://scholar.google.com/scholar?q=G%2E+Gkioxari%2C+B%2E+Hariharan%2C+R%2E+Girshick%2C+and+J%2E+Malik%2E+Using+k%2Dposelets+for+detecting+people+and+localizing+their+keypoints%2E+In+CVPR%2C+2014%2E+4)
15. M. Hodosh, P. Young, and J. Hockenmaier. Framing image description as a ranking task: Data, models and evaluation metrics. JAIR, 47:853–899, 2013. 2 [find](https://scholar.google.com/scholar?q=M%2E+Hodosh%2C+P%2E+Young%2C+and+J%2E+Hockenmaier%2E+Framing+image+description+as+a+ranking+task%3A+Data%2C+models+and+evaluation+metrics%2E+JAIR%2C+47%3A853%E2%80%93899%2C+2013%2E+2)
16. P. Huang, X. He, J. Gao, L. Deng, A. Acero, and L. Heck. Learning deep structured semantic models for web search using clickthrough data. In CIKM, 2013. 5 [find](https://scholar.google.com/scholar?q=P%2E+Huang%2C+X%2E+He%2C+J%2E+Gao%2C+L%2E+Deng%2C+A%2E+Acero%2C+and+L%2E+Heck%2E+Learning+deep+structured+semantic+models+for+web+search+using+clickthrough+data%2E+In+CIKM%2C+2013%2E+5)
17. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. [Caffe: Convolutional architecture for fast feature embedding](https://md.research.elamur.ai/free/en/papers/jia-2014-caffe-convolutional-architecture-fast-feature/). arXiv preprint arXiv:1408.5093, 2014. 3
18. A. Karpathy and L. Fei-Fei. [Deep visual-semantic alignments for generating image descriptions](https://md.research.elamur.ai/free/en/papers/karpathy-2014-deep-visual-semantic-alignments-generating/). CVPR, 2015. 2
19. A. Karpathy, A. Joulin, and L. Fei-Fei. Deep fragment embeddings for bidirectional image sentence mapping. arXiv preprint arXiv:1406.5679, 2014. 2 [find](https://scholar.google.com/scholar?q=A%2E+Karpathy%2C+A%2E+Joulin%2C+and+L%2E+Fei%2DFei%2E+Deep+fragment+embeddings+for+bidirectional+image+sentence+mapping%2E+arXiv+preprint+arXiv%3A1406%2E5679%2C+2014%2E+2)
20. R. Kiros, R. Zemel, and R. Salakhutdinov. Multimodal neural language models. In NIPS Deep Learning Workshop, 2013. 2, 3 [find](https://scholar.google.com/scholar?q=R%2E+Kiros%2C+R%2E+Zemel%2C+and+R%2E+Salakhutdinov%2E+Multimodal+neural+language+models%2E+In+NIPS+Deep+Learning+Workshop%2C+2013%2E+2%2C+3)
21. A. Krizhevsky, I. Sutskever, and G. E. Hinton. [ImageNet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/). In NIPS, 2012. 2, 3, 6, 7
22. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. Baby talk: Understanding and generating simple image descriptions. In CVPR, 2011. 1, 2, 6, 8 [find](https://scholar.google.com/scholar?q=G%2E+Kulkarni%2C+V%2E+Premraj%2C+S%2E+Dhar%2C+S%2E+Li%2C+Y%2E+Choi%2C+A%2E+C%2E+Berg%2C+and+T%2E+L%2E+Berg%2E+Baby+talk%3A+Understanding+and+generating+simple+image+descriptions%2E+In+CVPR%2C+2011%2E+1%2C+2%2C+6%2C+8)
23. P. Kuznetsova, V. Ordonez, A. C. Berg, T. L. Berg, and Y. Choi. Collective generation of natural image descriptions. In ACL, 2012. 2 [find](https://scholar.google.com/scholar?q=P%2E+Kuznetsova%2C+V%2E+Ordonez%2C+A%2E+C%2E+Berg%2C+T%2E+L%2E+Berg%2C+and+Y%2E+Choi%2E+Collective+generation+of+natural+image+descriptions%2E+In+ACL%2C+2012%2E+2)
24. R. Lau, R. Rosenfeld, and S. Roukos. Trigger-based language models: A maximum entropy approach. In ICASSP, 1993. 4 [find](https://scholar.google.com/scholar?q=R%2E+Lau%2C+R%2E+Rosenfeld%2C+and+S%2E+Roukos%2E+Trigger%2Dbased+language+models%3A+A+maximum+entropy+approach%2E+In+ICASSP%2C+1993%2E+4)
25. R. Lebret, P. O. Pinheiro, and R. Collobert. Phrase-based image captioning. arXiv preprint arXiv:1502.03671, 2015. 2, 3 — [arXiv:1502.03671](https://arxiv.org/abs/1502.03671) [find](https://scholar.google.com/scholar?q=R%2E+Lebret%2C+P%2E+O%2E+Pinheiro%2C+and+R%2E+Collobert%2E+Phrase%2Dbased+image+captioning%2E+arXiv+preprint+arXiv%3A1502%2E03671%2C+2015%2E+2%2C+3+%E2%80%94+%5BarXiv%3A1502%2E03671%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1502%2E03671%29)
26. S. Li, G. Kulkarni, T. L. Berg, A. C. Berg, and Y. Choi. Composing simple image descriptions using web-scale n-grams. In CoNLL, 2011. 2 [find](https://scholar.google.com/scholar?q=S%2E+Li%2C+G%2E+Kulkarni%2C+T%2E+L%2E+Berg%2C+A%2E+C%2E+Berg%2C+and+Y%2E+Choi%2E+Composing+simple+image+descriptions+using+web%2Dscale+n%2Dgrams%2E+In+CoNLL%2C+2011%2E+2)
27. C.-Y. Lin and F. J. Och. [Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics](https://md.research.elamur.ai/free/en/papers/lin-2004-automatic-evaluation-machine-translation-quality/). In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, ACL ’04, Stroudsburg, PA, USA, 2004. Association for Computational Linguis…
28. T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. [Microsoft COCO: Common objects in context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/). In ECCV, 2014. 2, 6
29. J. Mao, W. Xu, Y. Yang, J. Wang, and A. L. Yuille. Explain images with multimodal recurrent neural networks. arXiv preprint arXiv:1410.1090, 2014. 2, 3 [find](https://scholar.google.com/scholar?q=J%2E+Mao%2C+W%2E+Xu%2C+Y%2E+Yang%2C+J%2E+Wang%2C+and+A%2E+L%2E+Yuille%2E+Explain+images+with+multimodal+recurrent+neural+networks%2E+arXiv+preprint+arXiv%3A1410%2E1090%2C+2014%2E+2%2C+3)
30. O. Maron and T. Lozano-Pérez. A framework for multiple-instance learning. NIPS, 1998. 2, 3 [find](https://scholar.google.com/scholar?q=O%2E+Maron+and+T%2E+Lozano%2DPe%CC%81rez%2E+A+framework+for+multiple%2Dinstance+learning%2E+NIPS%2C+1998%2E+2%2C+3)
31. T. Mikolov, A. Deoras, D. Povey, L. Burget, and J. Cernocky. Strategies for training large scale neural network language models. In ASRU, 2011. 4 [find](https://scholar.google.com/scholar?q=T%2E+Mikolov%2C+A%2E+Deoras%2C+D%2E+Povey%2C+L%2E+Burget%2C+and+J%2E+Cernocky%2E+Strategies+for+training+large+scale+neural+network+language+models%2E+In+ASRU%2C+2011%2E+4)
32. M. Mitchell, X. Han, J. Dodge, A. Mensch, A. Goyal, A. Berg, K. Yamaguchi, T. Berg, K. Stratos, and H. Daumé III. Midge: Generating image descriptions from computer vision detections. In EACL, 2012. 2, 6, 8 [find](https://scholar.google.com/scholar?q=M%2E+Mitchell%2C+X%2E+Han%2C+J%2E+Dodge%2C+A%2E+Mensch%2C+A%2E+Goyal%2C+A%2E+Berg%2C+K%2E+Yamaguchi%2C+T%2E+Berg%2C+K%2E+Stratos%2C+and+H%2E+Daume%CC%81+III%2E+Midge%3A+Generating+image+descriptions+from+computer+vision+detections%2E+In+EACL%2C+2012%2E+)
33. A. Mnih and G. Hinton. Three new graphical models for statistical language modelling. In ICML, 2007. 4 [find](https://scholar.google.com/scholar?q=A%2E+Mnih+and+G%2E+Hinton%2E+Three+new+graphical+models+for+statistical+language+modelling%2E+In+ICML%2C+2007%2E+4)
34. A. Mnih and Y. W. Teh. [A fast and simple algorithm for training neural probabilistic language models](https://md.research.elamur.ai/free/en/papers/mnih-2012-fast-simple-algorithm-training-neural/). In ICML, 2012. 4
35. F. J. Och. Minimum error rate training in statistical machine translation. In ACL, 2003. 2, 5 [find](https://scholar.google.com/scholar?q=F%2E+J%2E+Och%2E+Minimum+error+rate+training+in+statistical+machine+translation%2E+In+ACL%2C+2003%2E+2%2C+5)
36. V. Ordonez, G. Kulkarni, and T. L. Berg. Im2text: Describing images using 1 million captioned photographs. In NIPS, 2011. 2 [find](https://scholar.google.com/scholar?q=V%2E+Ordonez%2C+G%2E+Kulkarni%2C+and+T%2E+L%2E+Berg%2E+Im2text%3A+Describing+images+using+1+million+captioned+photographs%2E+In+NIPS%2C+2011%2E+2)
37. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In ACL, 2002. 2, 6
38. C. Rashtchian, P. Young, M. Hodosh, and J. Hockenmaier. Collecting image annotations using Amazon’s mechanical turk. In NAACL HLT Workshop Creating Speech and Language Data with Amazon’s Mechanical Turk, 2010. 2, 6, 8 — [Collecting Image Annotations Using Amazon's Mechanical Turk](https://md.research.elamur.ai/free/en/papers/rashtchian-2010-collecting-image-annotations-amazon-s/)
39. A. Ratnaparkhi. Trainable methods for surface natural language generation. In NAACL, 2000. 4 [find](https://scholar.google.com/scholar?q=A%2E+Ratnaparkhi%2E+Trainable+methods+for+surface+natural+language+generation%2E+In+NAACL%2C+2000%2E+4)
40. A. Ratnaparkhi. Trainable approaches to surface natural language generation and their application to conversational dialog systems. Computer Speech & Language, 16(3):435–455, 2002. 2, 3 [find](https://scholar.google.com/scholar?q=A%2E+Ratnaparkhi%2E+Trainable+approaches+to+surface+natural+language+generation+and+their+application+to+conversational+dialog+systems%2E+Computer+Speech+%26+Language%2C+16%283%29%3A435%E2%80%93455%2C+2002%2E+2%2C+3)
41. Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil. A latent semantic model with convolutional-pooling structure for information retrieval. In CIKM, 2014. 5 [find](https://scholar.google.com/scholar?q=Y%2E+Shen%2C+X%2E+He%2C+J%2E+Gao%2C+L%2E+Deng%2C+and+G%2E+Mesnil%2E+A+latent+semantic+model+with+convolutional%2Dpooling+structure+for+information+retrieval%2E+In+CIKM%2C+2014%2E+5)
42. K. Simonyan and A. Zisserman. [Very deep convolutional networks for large-scale image recognition](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/). CoRR, abs/1409.1556, 2014. 2, 3, 6, 7
43. R. Socher, Q. Le, C. Manning, and A. Ng. [Grounded compositional semantics for finding and describing images with sentences](https://md.research.elamur.ai/free/en/papers/socher-2014-grounded-compositional-semantics-finding-describing/). In NIPS Deep Learning Workshop, 2013. 2
44. R. Vedantam, C. L. Zitnick, and D. Parikh. [Cider: Consensus-based image description evaluation](https://md.research.elamur.ai/free/en/papers/vedantam-2014-cider-consensus-based-image-description/). CoRR, abs/1411.5726, 2014. 6, 8
45. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. [Show and tell: A neural image caption generator](https://md.research.elamur.ai/free/en/papers/vinyals-2014-show-tell-neural-image-caption/). CVPR, 2015. 2
46. K. Xu, J. Ba, R. Kiros, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio. [Show, attend and tell: Neural image caption generation with visual attention](https://md.research.elamur.ai/free/en/papers/xu-2015-show-attend-tell-neural-image/). arXiv preprint arXiv:1502.03044, 2015. 2, 3
47. Y. Yang, C. L. Teo, H. Daumé III, and Y. Aloimonos. Corpus-guided sentence generation of natural images. In EMNLP, 2011. 1, 2 [find](https://scholar.google.com/scholar?q=Y%2E+Yang%2C+C%2E+L%2E+Teo%2C+H%2E+Daume%CC%81+III%2C+and+Y%2E+Aloimonos%2E+Corpus%2Dguided+sentence+generation+of+natural+images%2E+In+EMNLP%2C+2011%2E+1%2C+2)
48. B. Z. Yao, X. Yang, L. Lin, M. W. Lee, and S.-C. Zhu. I2T: Image parsing to text description. Proceedings of the IEEE, 98(8):1485–1508, 2010. 2 [find](https://scholar.google.com/scholar?q=B%2E+Z%2E+Yao%2C+X%2E+Yang%2C+L%2E+Lin%2C+M%2E+W%2E+Lee%2C+and+S%2E%2DC%2E+Zhu%2E+I2T%3A+Image+parsing+to+text+description%2E+Proceedings+of+the+IEEE%2C+98%288%29%3A1485%E2%80%931508%2C+2010%2E+2)
49. C. Zhang, J. C. Platt, and P. A. Viola. Multiple instance boosting for object detection. In NIPS, 2005. 2, 3 [find](https://scholar.google.com/scholar?q=C%2E+Zhang%2C+J%2E+C%2E+Platt%2C+and+P%2E+A%2E+Viola%2E+Multiple+instance+boosting+for+object+detection%2E+In+NIPS%2C+2005%2E+2%2C+3)
50. C. L. Zitnick and P. Dollár. Edge boxes: Locating object proposals from edges. In ECCV, 2014. 8 [find](https://scholar.google.com/scholar?q=C%2E+L%2E+Zitnick+and+P%2E+Dolla%CC%81r%2E+Edge+boxes%3A+Locating+object+proposals+from+edges%2E+In+ECCV%2C+2014%2E+8)
51. C. L. Zitnick and D. Parikh. Bringing semantics into focus using visual abstraction. In CVPR, 2013. 1 [find](https://scholar.google.com/scholar?q=C%2E+L%2E+Zitnick+and+D%2E+Parikh%2E+Bringing+semantics+into+focus+using+visual+abstraction%2E+In+CVPR%2C+2013%2E+1)

*Updated 2 November 2026*