---
title: "Generating Natural Questions About an — Mostafazadeh 2016"
description: "Most vision-and-language work, such as image captioning and VQA, deals with literal content: which objects appear and what colour or how many they are."
authors: ["Nasrin Mostafazadeh", "Ishan Misra", "Jacob Devlin", "Margaret Mitchell", "Xiaodong He", "Lucy Vanderwende"]
year: 2016
venue: "Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P16-1170/"
doi: "10.18653/v1/p16-1170"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mostafazadeh-2016-generating-natural-questions-about-image/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mostafazadeh-2016-generating-natural-questions-about-image/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/) / Generating Natural Questions About an Image

# Generating Natural Questions About an Image

**Authors:** [Nasrin Mostafazadeh](https://md.research.elamur.ai/free/en/authors/nasrin-mostafazadeh/), [Ishan Misra](https://md.research.elamur.ai/free/en/authors/ishan-misra/), [Jacob Devlin](https://md.research.elamur.ai/free/en/authors/jacob-devlin/), [Margaret Mitchell](https://md.research.elamur.ai/free/en/authors/margaret-mitchell/), [Xiaodong He](https://md.research.elamur.ai/free/en/authors/xiaodong-he/), [Lucy Vanderwende](https://md.research.elamur.ai/free/en/authors/lucy-vanderwende/) · **Year:** 2016 · **Venue:** Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P16-1170/) · [PDF](https://aclanthology.org/P16-1170.pdf) · [DOI 10.18653/v1/p16-1170](https://doi.org/10.18653/v1/p16-1170) · [OpenAlex](https://openalex.org/W2306876680)

**Citations:** 265 citations (openalex-snapshot · 2026-10-14) · [74](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary

Most vision-and-language work, such as image captioning and VQA, deals with literal content: which objects appear and what colour or how many they are. This paper introduces Visual Question Generation (VQG). Given an image, a system should ask the kind of engaging, natural question a person would ask when starting a conversation. Such questions usually concern inferred events, causes or states rather than things directly visible. The authors crowdsourced three datasets on Amazon Mechanical Turk, each with 5,000 images and 5 questions per image, drawn from MS COCO, Flickr and Bing image search. The image sets range from object-centric to event-centric. They also gathered captions for the Bing set, and these showed that its images are hard for a state-of-the-art captioner. They compared generative models (GRNN) with nearest-neighbour retrieval models. The end-to-end neural model is strongest on the most difficult Bing data, but all systems remain clearly below human questions. The work matters because it reframes question asking as a sign of understanding and a building block for conversational agents.

## Key points

- Three datasets, VQG_COCO-5000, VQG_Flickr-5000 and VQG_Bing-5000, contain 75,000 questions in total. They are complemented by 25,000 gold captions for the event-centric images.
- The COCO subset was chosen to overlap with images already annotated in VQA and COCO-QA (CQA), which allows direct comparison. VQG questions turn out to be far less literal and less object-focused than those in the other two datasets.
- The models compared are GRNN (generative), 1-NN, and K-NN variants that select candidates via min-distance plus BLEU or gensim similarity. Each model was trained either on all data or on the matching dataset only.
- ∆BLEU correlates with human judgments at Pearson r = 0.915, Spearman ρ = 0.702 and Kendall τ = 0.557. This is somewhat better than BLEU or METEOR on the rank-based measures.
- In human evaluation, GRNN_all wins in 2 of 3 settings. No retrieval model surpasses GRNN on Bing, which makes Bing the recommended benchmark.

## Interesting ideas

- **Exclude the trivially answerable.** The task explicitly rules out questions answerable just by looking at the image. Defining the scope by what to exclude can steer crowd annotations away from shallow outputs in other generation tasks.
- **Graded data sources.** The same annotation protocol is applied to image pools ranging from object-centric to event-centric. This yields a built-in difficulty gradient, a design that could be reused for any corpus where domain abstraction varies.
- **Consensus vs. random human baselines.** Both a "consensus" human answer and a randomly chosen human answer are scored, and the consensus one scores consistently higher. This checks whether annotators share an intuition about the "best" output, which is useful for validating any multi-reference dataset.
- **Choosing a metric by correlation with human judgments.** The authors pick the automatic metric that correlates best with human ratings and recommend it as the standard. This practice is applicable to any open-ended generation benchmark.
- **Strong retrieval baselines.** K-NN retrieval with n-gram overlap reranking proved competitive on easier data. It is a cheap baseline to report before training generative models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

74 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 69 not yet.

- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://md.research.elamur.ai/free/en/papers/du-2017-learning-ask-neural-question-generation/)
- [GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering](https://md.research.elamur.ai/free/en/papers/hudson-2019-gqa-new-dataset-real-world/)
- [Interpretation of Natural Language Rules in Conversational Machine Reading](https://md.research.elamur.ai/free/en/papers/saeidi-2018-interpretation-natural-language-rules-conversational/)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://md.research.elamur.ai/free/en/papers/vijayakumar-2016-diverse-beam-search-decoding-diverse/)
- [Generative Data Augmentation for Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/yang-2020-generative-data-augmentation-commonsense-reasoning/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mostafazadeh2016generating,
  title = {{Generating Natural Questions About an Image}},
  author = {Mostafazadeh, Nasrin and Misra, Ishan and Devlin, Jacob and Mitchell, Margaret and He, Xiaodong and Vanderwende, Lucy},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1170},
  url = {https://doi.org/10.18653/v1/p16-1170}
}
```

- [← previous Generating English from Abstract Meaning Representations](https://md.research.elamur.ai/free/en/papers/pourdamghani-2016-generating-english-abstract-meaning-representations/)
- [next → Generating Sentences from a Continuous Space](https://md.research.elamur.ai/free/en/papers/bowman-2016-generating-sentences-continuous-space/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/)

## References

49 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2014. 2d human pose estimation: New benchmark and state of the art analysis. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June. [find](https://scholar.google.com/scholar?q=Mykhaylo+Andriluka%2C+Leonid+Pishchulin%2C+Peter+Gehler%2C+and+Bernt+Schiele%2E+2014%2E+2d+human+pose+estimation%3A+New+benchmark+and+state+of+the+art+analysis%2E+In+IEEE+Conference+on+Computer+Vision+and+Pattern+R)
2. Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. VQA: Visual question answering. In International Conference on Computer Vision (ICCV). [find](https://scholar.google.com/scholar?q=Stanislaw+Antol%2C+Aishwarya+Agrawal%2C+Jiasen+Lu%2C+Margaret+Mitchell%2C+Dhruv+Batra%2C+C%2E+Lawrence+Zitnick%2C+and+Devi+Parikh%2E+2015%2E+VQA%3A+Visual+question+answering%2E+In+International+Conference+on+Computer+Visio)
3. Lee Becker, Sumit Basu, and Lucy Vanderwende. 2012. Mind the gap: Learning to choose gaps for question generation. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 742–751, Montréal, Canada, June… [find](https://scholar.google.com/scholar?q=Lee+Becker%2C+Sumit+Basu%2C+and+Lucy+Vanderwende%2E+2012%2E+Mind+the+gap%3A+Learning+to+choose+gaps+for+question+generation%2E+In+Proceedings+of+the+2012+Conference+of+the+North+American+Chapter+of+the+Associatio)
4. Yu-Wei Chao, Zhan Wang, Yugeng He, Jiaxuan Wang, and Jia Deng. 2015. HICO: A benchmark for recognizing human-object interactions in images. In Proceedings of the IEEE International Conference on Computer Vision. [find](https://scholar.google.com/scholar?q=Yu%2DWei+Chao%2C+Zhan+Wang%2C+Yugeng+He%2C+Jiaxuan+Wang%2C+and+Jia+Deng%2E+2015%2E+HICO%3A+A+benchmark+for+recognizing+human%2Dobject+interactions+in+images%2E+In+Proceedings+of+the+IEEE+International+Conference+on+Compu)
5. Jianfu Chen, Polina Kuznetsova, David Warren, and Yejin Choi. 2015. Déjà image-captions: A corpus of expressive descriptions in repetition. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 504–5… [find](https://scholar.google.com/scholar?q=Jianfu+Chen%2C+Polina+Kuznetsova%2C+David+Warren%2C+and+Yejin+Choi%2E+2015%2E+De%CC%81ja%CC%80+image%2Dcaptions%3A+A+corpus+of+expressive+descriptions+in+repetition%2E+In+Proceedings+of+the+2015+Conference+of+the+North+America)
6. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+Van+Merrie%CC%88nboer%2C+Caglar+Gulcehre%2C+Dzmitry+Bahdanau%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014%2E+Learning+phrase+representations+using+rnn+encoder%2Ddecoder+for+statisti)
7. Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the EACL 2014 Workshop on Statistical Machine Translation. [find](https://scholar.google.com/scholar?q=Michael+Denkowski+and+Alon+Lavie%2E+2014%2E+Meteor+universal%3A+Language+specific+translation+evaluation+for+any+target+language%2E+In+Proceedings+of+the+EACL+2014+Workshop+on+Statistical+Machine+Translation%2E) [our card](https://md.research.elamur.ai/free/en/papers/denkowski-2014-meteor-universal-language-specific-translation/)
8. Jacob Devlin, Hao Cheng, Hao Fang, Saurabh Gupta, Li Deng, Xiaodong He, Geoffrey Zweig, and Margaret Mitchell. 2015. Language models for image captioning: The quirks and what works. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International J… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Hao+Cheng%2C+Hao+Fang%2C+Saurabh+Gupta%2C+Li+Deng%2C+Xiaodong+He%2C+Geoffrey+Zweig%2C+and+Margaret+Mitchell%2E+2015%2E+Language+models+for+image+captioning%3A+The+quirks+and+what+works%2E+In+Proceedings+of+)
9. Jeff Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, and Trevor Darrell. 2014. Long-term recurrent convolutional networks for visual recognition and description. CoRR, abs/1411.4389. [find](https://scholar.google.com/scholar?q=Jeff+Donahue%2C+Lisa+Anne+Hendricks%2C+Sergio+Guadarrama%2C+Marcus+Rohrbach%2C+Subhashini+Venugopalan%2C+Kate+Saenko%2C+and+Trevor+Darrell%2E+2014%2E+Long%2Dterm+recurrent+convolutional+networks+for+visual+recognition+) [our card](https://md.research.elamur.ai/free/en/papers/donahue-2014-long-term-recurrent-convolutional-networks/)
10. Mark Everingham, Luc Gool, Christopher K. Williams, John Winn, and Andrew Zisserman. 2010. The pascal visual object classes (voc) challenge. Int. J. Comput. Vision, 88(2):303–338, June. [find](https://scholar.google.com/scholar?q=Mark+Everingham%2C+Luc+Gool%2C+Christopher+K%2E+Williams%2C+John+Winn%2C+and+Andrew+Zisserman%2E+2010%2E+The+pascal+visual+object+classes+%28voc%29+challenge%2E+Int%2E+J%2E+Comput%2E+Vision%2C+88%282%29%3A303%E2%80%93338%2C+June%2E)
11. Hao Fang, Saurabh Gupta, Forrest N. Iandola, Rupesh Srivastava, Li Deng, Piotr Dollár, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John C. Platt, C. Lawrence Zitnick, and Geoffrey Zweig. 2014. From captions to visual concepts and back. CoRR, abs/1411.4952. [find](https://scholar.google.com/scholar?q=Hao+Fang%2C+Saurabh+Gupta%2C+Forrest+N%2E+Iandola%2C+Rupesh+Srivastava%2C+Li+Deng%2C+Piotr+Dolla%CC%81r%2C+Jianfeng+Gao%2C+Xiaodong+He%2C+Margaret+Mitchell%2C+John+C%2E+Platt%2C+C%2E+Lawrence+Zitnick%2C+and+Geoffrey+Zweig%2E+2014%2E+From)
12. Ali Farhadi, Mohsen Hejrati, Mohammad Amin Sadeghi, Peter Young, Cyrus Rashtchian, Julia Hockenmaier, and David Forsyth. 2010. Every picture tells a story: Generating sentences from images. In Proceedings of the 11th European Conference on Computer Vision: Part IV, ECCV’10, pages 15–29, Berlin, Heid… [find](https://scholar.google.com/scholar?q=Ali+Farhadi%2C+Mohsen+Hejrati%2C+Mohammad+Amin+Sadeghi%2C+Peter+Young%2C+Cyrus+Rashtchian%2C+Julia+Hockenmaier%2C+and+David+Forsyth%2E+2010%2E+Every+picture+tells+a+story%3A+Generating+sentences+from+images%2E+In+Proceed)
13. Francis Ferraro, Nasrin Mostafazadeh, Ting-Hao Huang, Lucy Vanderwende, Jacob Devlin, Michel Galley, and Margaret Mitchell. 2015. A survey of current datasets for vision and language research. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 207–213, L… [find](https://scholar.google.com/scholar?q=Francis+Ferraro%2C+Nasrin+Mostafazadeh%2C+Ting%2DHao+Huang%2C+Lucy+Vanderwende%2C+Jacob+Devlin%2C+Michel+Galley%2C+and+Margaret+Mitchell%2E+2015%2E+A+survey+of+current+datasets+for+vision+and+language+research%2E+In+Proc)
14. Michel Galley, Chris Brockett, Alessandro Sordoni, Yangfeng Ji, Michael Auli, Chris Quirk, Margaret Mitchell, Jianfeng Gao, and Bill Dolan. 2015. deltaBLEU: A discriminative metric for generation tasks with intrinsically diverse targets. In Proceedings of the 53rd Annual Meeting of the Association f… [find](https://scholar.google.com/scholar?q=Michel+Galley%2C+Chris+Brockett%2C+Alessandro+Sordoni%2C+Yangfeng+Ji%2C+Michael+Auli%2C+Chris+Quirk%2C+Margaret+Mitchell%2C+Jianfeng+Gao%2C+and+Bill+Dolan%2E+2015%2E+deltaBLEU%3A+A+discriminative+metric+for+generation+task)
15. Haoyuan Gao, Junhua Mao, Jie Zhou, Zhiheng Huang, Lei Wang, and Wei Xu. 2015. Are you talking to a machine? dataset and methods for multilingual image question answering. CoRR, abs/1505.05612. — [arXiv:1505.05612](https://arxiv.org/abs/1505.05612) [find](https://scholar.google.com/scholar?q=Haoyuan+Gao%2C+Junhua+Mao%2C+Jie+Zhou%2C+Zhiheng+Huang%2C+Lei+Wang%2C+and+Wei+Xu%2E+2015%2E+Are+you+talking+to+a+machine%3F+dataset+and+methods+for+multilingual+image+question+answering%2E+CoRR%2C+abs%2F1505%2E05612%2E+%E2%80%94+%5BarXi)
16. Michael Heilman and Noah A. Smith. 2010. Good question! statistical ranking for question generation. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 609–617, Los Angeles, California, June. Association fo… [find](https://scholar.google.com/scholar?q=Michael+Heilman+and+Noah+A%2E+Smith%2E+2010%2E+Good+question%21+statistical+ranking+for+question+generation%2E+In+Human+Language+Technologies%3A+The+2010+Annual+Conference+of+the+North+American+Chapter+of+the+Ass)
17. Micah Hodosh, Peter Young, and Julia Hockenmaier. 2013. Framing image description as a ranking task: Data, models and evaluation metrics. J. Artif. Int. Res., 47(1):853–899, May. [find](https://scholar.google.com/scholar?q=Micah+Hodosh%2C+Peter+Young%2C+and+Julia+Hockenmaier%2E+2013%2E+Framing+image+description+as+a+ranking+task%3A+Data%2C+models+and+evaluation+metrics%2E+J%2E+Artif%2E+Int%2E+Res%2E%2C+47%281%29%3A853%E2%80%93899%2C+May%2E)
18. Ting-Hao Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Aishwarya Agrawal, Jacob Devlin, Ross B. Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley, and Margaret Mitchell. 2016. Visual storytelling. [find](https://scholar.google.com/scholar?q=Ting%2DHao+Huang%2C+Francis+Ferraro%2C+Nasrin+Mostafazadeh%2C+Ishan+Misra%2C+Aishwarya+Agrawal%2C+Jacob+Devlin%2C+Ross+B%2E+Girshick%2C+Xiaodong+He%2C+Pushmeet+Kohli%2C+Dhruv+Batra%2C+C%2E+Lawrence+Zitnick%2C+Devi+Parikh%2C+Lucy+V) [our card](https://md.research.elamur.ai/free/en/papers/huang-2016-visual-storytelling/)
19. In Proceedings of NAACL 2016. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+NAACL+2016%2E+Association+for+Computational+Linguistics%2E)
20. Joseph Jordania. 2006. Who Asked the First Question? The Origins of Human Choral Singing, Intelligence, Language and Speech. Logos. [find](https://scholar.google.com/scholar?q=Joseph+Jordania%2E+2006%2E+Who+Asked+the+First+Question%3F+The+Origins+of+Human+Choral+Singing%2C+Intelligence%2C+Language+and+Speech%2E+Logos%2E)
21. Igor Labutov, Sumit Basu, and Lucy Vanderwende. 2015. Deep questions without deep understanding. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). [find](https://scholar.google.com/scholar?q=Igor+Labutov%2C+Sumit+Basu%2C+and+Lucy+Vanderwende%2E+2015%2E+Deep+questions+without+deep+understanding%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+7th+)
22. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A persona-based neural conversation model. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Lin… [find](https://scholar.google.com/scholar?q=Jiwei+Li%2C+Michel+Galley%2C+Chris+Brockett%2C+Jianfeng+Gao%2C+and+Bill+Dolan%2E+2016%2E+A+persona%2Dbased+neural+conversation+model%2E+In+Proceedings+of+the+2015+Conference+of+the+North+American+Chapter+of+the+Assoc) [our card](https://md.research.elamur.ai/free/en/papers/li-2016-persona-based-neural-conversation-model/)
23. Chin-Yew Lin and Franz Josef Och. 2004. Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In Proceedings of the 42Nd Annual Meeting on Association for Computational Linguistics, ACL ’04, Stroudsburg, PA, USA. Association for Computationa… [find](https://scholar.google.com/scholar?q=Chin%2DYew+Lin+and+Franz+Josef+Och%2E+2004%2E+Automatic+evaluation+of+machine+translation+quality+using+longest+common+subsequence+and+skip%2Dbigram+statistics%2E+In+Proceedings+of+the+42Nd+Annual+Meeting+on+As) [our card](https://md.research.elamur.ai/free/en/papers/lin-2004-automatic-evaluation-machine-translation-quality/)
24. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and C. Lawrence Zitnick. 2014. Microsoft COCO: common objects in context. CoRR, abs/1405.0312. [find](https://scholar.google.com/scholar?q=Tsung%2DYi+Lin%2C+Michael+Maire%2C+Serge+Belongie%2C+James+Hays%2C+Pietro+Perona%2C+Deva+Ramanan%2C+Piotr+Dollar%2C+and+C%2E+Lawrence+Zitnick%2E+2014%2E+Microsoft+COCO%3A+common+objects+in+context%2E+CoRR%2C+abs%2F1405%2E0312%2E) [our card](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/)
25. David Lindberg, Fred Popowich, John Nesbit, and Phil Winne. 2013. Generating natural language questions to support learning on-line. In Proceedings of the 14th European Workshop on Natural Language Generation, pages 105–114, Sofia, Bulgaria, August. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=David+Lindberg%2C+Fred+Popowich%2C+John+Nesbit%2C+and+Phil+Winne%2E+2013%2E+Generating+natural+language+questions+to+support+learning+on%2Dline%2E+In+Proceedings+of+the+14th+European+Workshop+on+Natural+Language+Ge)
26. George A. Miller. 1995. Wordnet: A lexical database for english. Commun. ACM, 38(11):39–41, November. [find](https://scholar.google.com/scholar?q=George+A%2E+Miller%2E+1995%2E+Wordnet%3A+A+lexical+database+for+english%2E+Commun%2E+ACM%2C+38%2811%29%3A39%E2%80%9341%2C+November%2E) [our card](https://md.research.elamur.ai/free/en/papers/lex-1994-miller-wordnet/)
27. Ruslan Mitkov and Le An Ha. 2003. Computer-aided generation of multiple-choice tests. In Jill Burstein and Claudia Leacock, editors, Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural Language Processing, pages 17–22. [find](https://scholar.google.com/scholar?q=Ruslan+Mitkov+and+Le+An+Ha%2E+2003%2E+Computer%2Daided+generation+of+multiple%2Dchoice+tests%2E+In+Jill+Burstein+and+Claudia+Leacock%2C+editors%2C+Proceedings+of+the+HLT%2DNAACL+03+Workshop+on+Building+Educational+Ap)
28. Vicente Ordonez, Girish Kulkarni, and Tamara L. Berg. 2011. Im2text: Describing images using 1 million captioned photographs. In Neural Information Processing Systems (NIPS). [find](https://scholar.google.com/scholar?q=Vicente+Ordonez%2C+Girish+Kulkarni%2C+and+Tamara+L%2E+Berg%2E+2011%2E+Im2text%3A+Describing+images+using+1+million+captioned+photographs%2E+In+Neural+Information+Processing+Systems+%28NIPS%29%2E)
29. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 311–318, Stroudsburg, PA, USA. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+BLEU%3A+A+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computat) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
30. J. Pustejovsky, P. Hanks, R. Sauri, A. See, R. Gaizauskas, A. Setzer, D. Radev, B. Sundheim, D. Day, L. Ferro, and M. Lazo. 2003. The TIMEBANK corpus. In Proceedings of Corpus Linguistics 2003, pages 647–656, Lancaster, March. [find](https://scholar.google.com/scholar?q=J%2E+Pustejovsky%2C+P%2E+Hanks%2C+R%2E+Sauri%2C+A%2E+See%2C+R%2E+Gaizauskas%2C+A%2E+Setzer%2C+D%2E+Radev%2C+B%2E+Sundheim%2C+D%2E+Day%2C+L%2E+Ferro%2C+and+M%2E+Lazo%2E+2003%2E+The+TIMEBANK+corpus%2E+In+Proceedings+of+Corpus+Linguistics+2003%2C+pages+)
31. Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45– 50, Valletta, Malta, May. ELRA. [http://is.muni.cz/publication/884893/en](http://is.muni.cz/publication/884893/en). [↗](http://is.muni.cz/publication/884893/en)
32. Mengye Ren, Ryan Kiros, and Richard Zemel. 2015. Question answering about images using visual semantic embeddings. In Deep Learning Workshop, ICML 2015. [find](https://scholar.google.com/scholar?q=Mengye+Ren%2C+Ryan+Kiros%2C+and+Richard+Zemel%2E+2015%2E+Question+answering+about+images+using+visual+semantic+embeddings%2E+In+Deep+Learning+Workshop%2C+ICML+2015%2E)
33. Mateusz Malinowski and Mario Fritz. 2014. A multi-world approach to question answering about real-world scenes based on uncertain input. In Advances in Neural Information Processing Systems 27, pages 1682–1690. [find](https://scholar.google.com/scholar?q=Mateusz+Malinowski+and+Mario+Fritz%2E+2014%2E+A+multi%2Dworld+approach+to+question+answering+about+real%2Dworld+scenes+based+on+uncertain+input%2E+In+Advances+in+Neural+Information+Processing+Systems+27%2C+pages+)
34. Marcus Rohrbach, Sikandar Amin, Mykhaylo Andriluka, and Bernt Schiele. 2012. A database for fine grained activity detection of cooking activities. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, IEEE, June. [find](https://scholar.google.com/scholar?q=Marcus+Rohrbach%2C+Sikandar+Amin%2C+Mykhaylo+Andriluka%2C+and+Bernt+Schiele%2E+2012%2E+A+database+for+fine+grained+activity+detection+of+cooking+activities%2E+In+IEEE+Conference+on+Computer+Vision+and+Pattern+Rec)
35. Karen Mazidi and Rodney D. Nielsen. 2014. Linguistic considerations in automatic question generation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 321–326, Baltimore, Maryland, June. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Karen+Mazidi+and+Rodney+D%2E+Nielsen%2E+2014%2E+Linguistic+considerations+in+automatic+question+generation%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume)
36. K. Simonyan and A. Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556. [find](https://scholar.google.com/scholar?q=K%2E+Simonyan+and+A%2E+Zisserman%2E+2014%2E+Very+deep+convolutional+networks+for+large%2Dscale+image+recognition%2E+CoRR%2C+abs%2F1409%2E1556%2E) [our card](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/)
37. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings… [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Gregory+S%2E+Corrado%2C+and+Jeffrey+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Advances+in+Neural+Information+Proc)
38. Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. In Proceedings of the 2015 Conference of the North American Chapte… [find](https://scholar.google.com/scholar?q=Alessandro+Sordoni%2C+Michel+Galley%2C+Michael+Auli%2C+Chris+Brockett%2C+Yangfeng+Ji%2C+Margaret+Mitchell%2C+Jian%2DYun+Nie%2C+Jianfeng+Gao%2C+and+Bill+Dolan%2E+2015%2E+A+neural+network+approach+to+context%2Dsensitive+genera) [our card](https://md.research.elamur.ai/free/en/papers/sordoni-2015-neural-network-approach-context-sensitive/)
39. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 3104– 3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V%2E+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+Neural+Information+Processing+Systems+27%3A+Annual+Conference+on+Neural+Informatio)
40. Yuke Zhu, Oliver Groth, Michael S. Bernstein, and Li Fei-Fei. 2016. Visual7w: Grounded question answering in images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, IEEE. [find](https://scholar.google.com/scholar?q=Yuke+Zhu%2C+Oliver+Groth%2C+Michael+S%2E+Bernstein%2C+and+Li+Fei%2DFei%2E+2016%2E+Visual7w%3A+Grounded+question+answering+in+images%2E+In+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2E+IEEE%2C+IEEE%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhu-2015-visual7w-grounded-question-answering-images/)
41. Kenneth Tran, Xiaodong He, Lei Zhang, Jian Sun, Cornelia Carapcea, Chris Thrasher, Chris Buehler, and Chris Sienkiewicz. 2016. Rich image captioning in the wild. In Proceedings of Deep Vision Workshop at CVPR 2016. IEEE, June. [find](https://scholar.google.com/scholar?q=Kenneth+Tran%2C+Xiaodong+He%2C+Lei+Zhang%2C+Jian+Sun%2C+Cornelia+Carapcea%2C+Chris+Thrasher%2C+Chris+Buehler%2C+and+Chris+Sienkiewicz%2E+2016%2E+Rich+image+captioning+in+the+wild%2E+In+Proceedings+of+Deep+Vision+Workshop)
42. Lucy Vanderwende, Arul Menezes, and Chris Quirk. 2015. An amr parser for english, french, german, spanish and japanese and a new amr-annotated corpus. Proceedings of NAACL 2015, June. [find](https://scholar.google.com/scholar?q=Lucy+Vanderwende%2C+Arul+Menezes%2C+and+Chris+Quirk%2E+2015%2E+An+amr+parser+for+english%2C+french%2C+german%2C+spanish+and+japanese+and+a+new+amr%2Dannotated+corpus%2E+Proceedings+of+NAACL+2015%2C+June%2E)
43. Lucy Vanderwende. 2008. The importance of being important: Question generation. In In Workshop on the Question Generation Shared Task and Evaluation Challenge, Arlington, VA. [find](https://scholar.google.com/scholar?q=Lucy+Vanderwende%2E+2008%2E+The+importance+of+being+important%3A+Question+generation%2E+In+In+Workshop+on+the+Question+Generation+Shared+Task+and+Evaluation+Challenge%2C+Arlington%2C+VA%2E)
44. Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, and Kate Saenko. 2015. Translating videos to natural language using deep recurrent neural networks. In Proceedings the 2015 Conference of the North American Chapter of the Association for Computational Linguistics – H… [find](https://scholar.google.com/scholar?q=Subhashini+Venugopalan%2C+Huijuan+Xu%2C+Jeff+Donahue%2C+Marcus+Rohrbach%2C+Raymond+Mooney%2C+and+Kate+Saenko%2E+2015%2E+Translating+videos+to+natural+language+using+deep+recurrent+neural+networks%2E+In+Proceedings+th)
45. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In Computer Vision and Pattern Recognition. [find](https://scholar.google.com/scholar?q=Oriol+Vinyals%2C+Alexander+Toshev%2C+Samy+Bengio%2C+and+Dumitru+Erhan%2E+2015%2E+Show+and+tell%3A+A+neural+image+caption+generator%2E+In+Computer+Vision+and+Pattern+Recognition%2E) [our card](https://md.research.elamur.ai/free/en/papers/vinyals-2014-show-tell-neural-image-caption/)
46. John H Wolfe. 1976. Automatic question generation from text-an aid to independent study. In ACM SIGCUE Outlook, volume 10, pages 104–112. ACM. [find](https://scholar.google.com/scholar?q=John+H+Wolfe%2E+1976%2E+Automatic+question+generation+from+text%2Dan+aid+to+independent+study%2E+In+ACM+SIGCUE+Outlook%2C+volume+10%2C+pages+104%E2%80%93112%2E+ACM%2E)
47. Yuanjun Xiong, Kai Zhu, Dahua Lin, and Xiaoou Tang. 2015. Recognize complex events from static images by fusing deep channels. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June. [find](https://scholar.google.com/scholar?q=Yuanjun+Xiong%2C+Kai+Zhu%2C+Dahua+Lin%2C+and+Xiaoou+Tang%2E+2015%2E+Recognize+complex+events+from+static+images+by+fusing+deep+channels%2E+In+The+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2C)
48. Bangpeng Yao, Xiaoye Jiang, Aditya Khosla, Andy Lai Lin, Leonidas J. Guibas, and Li Fei-Fei. 2011a. Action recognition by learning bases of action attributes and parts. In International Conference on Computer Vision (ICCV), Barcelona, Spain, November. [find](https://scholar.google.com/scholar?q=Bangpeng+Yao%2C+Xiaoye+Jiang%2C+Aditya+Khosla%2C+Andy+Lai+Lin%2C+Leonidas+J%2E+Guibas%2C+and+Li+Fei%2DFei%2E+2011a%2E+Action+recognition+by+learning+bases+of+action+attributes+and+parts%2E+In+International+Conference+on+)
49. Bangpeng Yao, Xiaoye Jiang, Aditya Khosla, Andy Lai Lin, Leonidas J. Guibas, and Li Fei-Fei. 2011b. Human action recognition by learning bases of action attributes and parts. In International Conference on Computer Vision (ICCV), Barcelona, Spain, November. [find](https://scholar.google.com/scholar?q=Bangpeng+Yao%2C+Xiaoye+Jiang%2C+Aditya+Khosla%2C+Andy+Lai+Lin%2C+Leonidas+J%2E+Guibas%2C+and+Li+Fei%2DFei%2E+2011b%2E+Human+action+recognition+by+learning+bases+of+action+attributes+and+parts%2E+In+International+Conferen)