---
title: "Inferring and Executing Programs for Visual Reasoning (2017)"
description: "The authors argue that compositional reasoning should be built into the model's structure, and they… 51 references, cited by 90 papers here."
authors: ["Justin Johnson", "Bharath Hariharan", "Laurens van der Maaten", "Judy Hoffman", "Li Fei-Fei", "C. Lawrence Zitnick", "Ross Girshick"]
year: 2017
venue: "arXiv:1705.03633"
source: "https://arxiv.org/abs/1705.03633"
doi: "10.48550/arxiv.1705.03633"
arxiv: "1705.03633"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/johnson-2017-inferring-executing-programs-visual-reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/johnson-2017-inferring-executing-programs-visual-reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/) / Inferring and Executing Programs for Visual Reasoning

# Inferring and Executing Programs for Visual Reasoning

**Authors:** [Justin Johnson](https://md.research.elamur.ai/free/en/authors/justin-m-johnson/), [Bharath Hariharan](https://md.research.elamur.ai/free/en/authors/bharath-hariharan/), [Laurens van der Maaten](https://md.research.elamur.ai/free/en/authors/laurens-van-der-maaten/), [Judy Hoffman](https://md.research.elamur.ai/free/en/authors/judy-hoffman/), [Li Fei-Fei](https://md.research.elamur.ai/free/en/authors/li-fei-fei/), [C. Lawrence Zitnick](https://md.research.elamur.ai/free/en/authors/c-lawrence-zitnick/) and [Ross Girshick](https://md.research.elamur.ai/free/en/authors/ross-girshick/) · **Year:** 2017 · **Venue:** [arXiv:1705.03633](https://md.research.elamur.ai/free/en/venues/ieee-international-conference-on-computer-vision-iccv/) ([ICCV 2017](https://md.research.elamur.ai/free/en/venues/ieee-international-conference-on-computer-vision-iccv/2017/)) · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1705.03633) · [PDF](https://arxiv.org/pdf/1705.03633) · [DOI 10.48550/arxiv.1705.03633](https://doi.org/10.48550/arxiv.1705.03633) · [arXiv 1705.03633](https://arxiv.org/abs/1705.03633) · [OpenAlex](https://openalex.org/W2613404084)

**Citations:** 467 citations (OpenAlex, 2026-10-14) · [90](https://md.research.elamur.ai/free/en/papers/johnson-2017-inferring-executing-programs-visual-reasoning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Visual question answering systems usually learn a direct mapping from image and question to answer. Such black-box models tend to exploit dataset biases instead of actually reasoning. The authors argue that compositional reasoning should be built into the model's structure, and they split the task into two learned parts. A program generator, an LSTM sequence-to-sequence model, turns the question into a program: a sequence of functions that encodes a syntax tree in prefix order. An execution engine then assembles a neural module network that mirrors this program and runs it on CNN image features to predict an answer. Unlike earlier module networks, neither part relies on hand-built syntactic parsers or hand-designed modules. Only the function vocabulary and a generic module design are fixed. Training uses backpropagation, with REINFORCE for joint end-to-end tuning. On CLEVR, the model beats strong non-compositional baselines by about 20 points using little program supervision. It also adapts to free-form questions written by humans. This shows that explicit, interpretable reasoning structures generalize better than direct mappings.

## Key points
- Evaluated on the CLEVR benchmark. Only about 9,000 ground-truth programs (roughly 2% of those available) were needed to outperform state-of-the-art VQA models by around 20 percentage points.
- Programs are built from a fixed set of functions of arity 1 or 2, plus a special Scene constant for image features. Malformed predicted sequences are repaired by padding with Scene or by dropping unused functions.
- The authors collected a new dataset of human-written free-form questions about CLEVR images. After finetuning without extra program supervision, the model gained about 9 accuracy points over the best competing models.
- Reimplemented baselines include LSTM, CNN+LSTM, CNN+LSTM+SA and CNN+LSTM+SA+MLP. Adding an MLP head to the stacked-attention model raised CLEVR accuracy from 69.8 to 73.2, and shape-comparison accuracy from 50.9 to 69.7.
- The dependency-parser layouts from Andreas et al.'s dynamic module networks often failed on long CLEVR questions. In those cases they fell back to the default fragment "(what thing)".

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 506 works · 330,053 citations
- [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 214 works · 241,492 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C. Lawrence Zitnick and Ross Girshick. 2017. Inferring and Executing Programs for Visual Reasoning. arXiv:1705.03633.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{johnson2017inferring,
  title = {{Inferring and Executing Programs for Visual Reasoning}},
  author = {Justin Johnson, Bharath Hariharan, Laurens {van der Maaten}, Judy Hoffman, Li Fei-Fei, C. Lawrence Zitnick and Ross Girshick},
  year = {2017},
  doi = {10.48550/arxiv.1705.03633},
  eprint = {1705.03633},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1705.03633}
}
```

- [← previous Cross-Sentence N-ary Relation Extraction with Graph LSTMs](https://md.research.elamur.ai/free/en/papers/peng-2017-cross-sentence-n-ary-relation/)
- [next → SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine](https://md.research.elamur.ai/free/en/papers/dunn-2017-searchqa-new-q-dataset-augmented/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/)

## References

51 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. J. Andreas, M. Rohrbach, T. Darrell, and D. Klein. [Learning to compose neural networks for question answering](https://md.research.elamur.ai/free/en/papers/andreas-2016-learning-compose-neural-networks-question/). In NAACL, 2016. 1, 2, 3, 5, 8, 9, 10, 11
2. J. Andreas, M. Rohrbach, T. Darrell, and D. Klein. [Neural module networks](https://md.research.elamur.ai/free/en/papers/andreas-2015-neural-module-networks/). In CVPR, 2016. 1, 2, 3, 4, 5, 8, 9
3. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. Zitnick, and D. Parikh. VQA: Visual question answering. In ICCV, 2015. 1, 2, 4, 7, 11 [find](https://scholar.google.com/scholar?q=S%2E+Antol%2C+A%2E+Agrawal%2C+J%2E+Lu%2C+M%2E+Mitchell%2C+D%2E+Batra%2C+C%2E+Zitnick%2C+and+D%2E+Parikh%2E+VQA%3A+Visual+question+answering%2E+In+ICCV%2C+2015%2E+1%2C+2%2C+4%2C+7%2C+11)
4. M. Balog, A. Gaunt, M. Brockschmidt, S. Nowozin, and D. Tarlow. Deepcoder: Learning to write programs. In ICLR, 2017. 2, 8 [find](https://scholar.google.com/scholar?q=M%2E+Balog%2C+A%2E+Gaunt%2C+M%2E+Brockschmidt%2C+S%2E+Nowozin%2C+and+D%2E+Tarlow%2E+Deepcoder%3A+Learning+to+write+programs%2E+In+ICLR%2C+2017%2E+2%2C+8)
5. J. Cai, R. Shin, and D. Song. Making neural programming architectures generalize via recursion. In ICLR, 2017. 8 [find](https://scholar.google.com/scholar?q=J%2E+Cai%2C+R%2E+Shin%2C+and+D%2E+Song%2E+Making+neural+programming+architectures+generalize+via+recursion%2E+In+ICLR%2C+2017%2E+8)
6. A. Das, S. Kottur, K. Gupta, A. Singh, D. Yadav, J. Moura, D. Parikh, and D. Batra. Visual dialog. In CVPR, 2017. 2 [find](https://scholar.google.com/scholar?q=A%2E+Das%2C+S%2E+Kottur%2C+K%2E+Gupta%2C+A%2E+Singh%2C+D%2E+Yadav%2C+J%2E+Moura%2C+D%2E+Parikh%2C+and+D%2E+Batra%2E+Visual+dialog%2E+In+CVPR%2C+2017%2E+2)
7. J. Devlin, S. Gupta, R. Girshick, M. Mitchell, and C. L. Zitnick. Exploring nearest neighbor approaches for image captioning. arXiv preprint arXiv:1505.04467, 2015. 1 [find](https://scholar.google.com/scholar?q=J%2E+Devlin%2C+S%2E+Gupta%2C+R%2E+Girshick%2C+M%2E+Mitchell%2C+and+C%2E+L%2E+Zitnick%2E+Exploring+nearest+neighbor+approaches+for+image+captioning%2E+arXiv+preprint+arXiv%3A1505%2E04467%2C+2015%2E+1)
8. A. Fukui, D. H. Park, D. Yang, A. Rohrbach, T. Darrell, and M. Rohrbach. [Multimodal compact bilinear pooling for visual question answering and visual grounding](https://md.research.elamur.ai/free/en/papers/fukui-2016-multimodal-compact-bilinear-pooling-visual/). In arXiv:1606.01847, 2016. 2
9. Y. Gao, O. Beijbom, N. Zhang, and T. Darrell. Compact bilinear pooling. In CVPR, 2016. 2 [find](https://scholar.google.com/scholar?q=Y%2E+Gao%2C+O%2E+Beijbom%2C+N%2E+Zhang%2C+and+T%2E+Darrell%2E+Compact+bilinear+pooling%2E+In+CVPR%2C+2016%2E+2)
10. R. Girshick. Fast R-CNN. In ICCV, 2015. 1 [find](https://scholar.google.com/scholar?q=R%2E+Girshick%2E+Fast+R%2DCNN%2E+In+ICCV%2C+2015%2E+1)
11. Y. Goyal, T. Khot, D. Summers-Stay, D. Batra, and D. Parikh. [Making the V in VQA matter: Elevating the role of image understanding in visual question answering](https://md.research.elamur.ai/free/en/papers/goyal-2016-making-v-vqa-matter-elevating/). In CVPR, 2017. 2
12. A. Graves, G. Wayne, and I. Danihelka. Neural turing machines. arXiv preprint arXiv:1410.5401, 2014. 2 [find](https://scholar.google.com/scholar?q=A%2E+Graves%2C+G%2E+Wayne%2C+and+I%2E+Danihelka%2E+Neural+turing+machines%2E+arXiv+preprint+arXiv%3A1410%2E5401%2C+2014%2E+2)
13. A. Graves, G. Wayne, M. Reynolds, T. Harley, I. Danihelka, A. Grabska-Barwinska, S. Colmenarejo, E. Grefenstette, T. Ramalho, J. Agapiou, A. Badia, K. Hermann, Y. Zwols, G. Ostrovski, A. Cain, H. King, C. Summerfield, P. Blunsom, K. . Kavukcuoglu, and D. Hassabis. Hybrid computing using a neural net… [find](https://scholar.google.com/scholar?q=A%2E+Graves%2C+G%2E+Wayne%2C+M%2E+Reynolds%2C+T%2E+Harley%2C+I%2E+Danihelka%2C+A%2E+Grabska%2DBarwinska%2C+S%2E+Colmenarejo%2C+E%2E+Grefenstette%2C+T%2E+Ramalho%2C+J%2E+Agapiou%2C+A%2E+Badia%2C+K%2E+Hermann%2C+Y%2E+Zwols%2C+G%2E+Ostrovski%2C+A%2E+Cain%2C+H%2E+King)
14. K. He, X. Zhang, S. Ren, and J. Sun. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). In CVPR, 2016. 3, 9
15. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997. 4 [find](https://scholar.google.com/scholar?q=S%2E+Hochreiter+and+J%2E+Schmidhuber%2E+Long+short%2Dterm+memory%2E+Neural+Computation%2C+9%288%29%3A1735%E2%80%931780%2C+1997%2E+4)
16. R. Hu, M. Rohrbach, J. Andreas, T. Darrell, and K. Saenko. Modeling relationships in referential expressions with compositional modular networks. In CVPR, 2017. 2 [find](https://scholar.google.com/scholar?q=R%2E+Hu%2C+M%2E+Rohrbach%2C+J%2E+Andreas%2C+T%2E+Darrell%2C+and+K%2E+Saenko%2E+Modeling+relationships+in+referential+expressions+with+compositional+modular+networks%2E+In+CVPR%2C+2017%2E+2)
17. S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015. 9 [find](https://scholar.google.com/scholar?q=S%2E+Ioffe+and+C%2E+Szegedy%2E+Batch+normalization%3A+Accelerating+deep+network+training+by+reducing+internal+covariate+shift%2E+In+ICML%2C+2015%2E+9)
18. A. Jabri, A. Joulin, and L. van der Maaten. Revisiting visual question answering baselines. In ECCV, 2016. 1, 2 [find](https://scholar.google.com/scholar?q=A%2E+Jabri%2C+A%2E+Joulin%2C+and+L%2E+van+der+Maaten%2E+Revisiting+visual+question+answering+baselines%2E+In+ECCV%2C+2016%2E+1%2C+2)
19. J. Johnson, B. Hariharan, L. van der Maaten, L. Fei-Fei, C. L. Zitnick, and R. Girshick. [CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning](https://md.research.elamur.ai/free/en/papers/johnson-2016-clevr-diagnostic-dataset-compositional-language/). In CVPR, 2017. 1, 2, 4, 5, 6, 7, 9, 10
20. A. Joulin and T. Mikolov. Inferring algorithmic patterns with stack-augmented recurrent nets. In NIPS, 2015. 2 [find](https://scholar.google.com/scholar?q=A%2E+Joulin+and+T%2E+Mikolov%2E+Inferring+algorithmic+patterns+with+stack%2Daugmented+recurrent+nets%2E+In+NIPS%2C+2015%2E+2)
21. K. Kafle and C. Kanan. Visual question answering: Datasets, algorithms, and future challenges. In arXiv preprint arXiv:1610.01465, 2016. 2 [find](https://scholar.google.com/scholar?q=K%2E+Kafle+and+C%2E+Kanan%2E+Visual+question+answering%3A+Datasets%2C+algorithms%2C+and+future+challenges%2E+In+arXiv+preprint+arXiv%3A1610%2E01465%2C+2016%2E+2)
22. Ł. Kaiser and I. Sutskever. Neural GPUs learn algorithms. In ICLR, 2016. 2 [find](https://scholar.google.com/scholar?q=%C5%81%2E+Kaiser+and+I%2E+Sutskever%2E+Neural+GPUs+learn+algorithms%2E+In+ICLR%2C+2016%2E+2)
23. D. Kingma and J. Ba. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In ICLR, 2015. 9
24. D. Klein and C. D. Manning. [Accurate unlexicalized parsing](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/). In ACL, pages 423–430, 2003. 2
25. R. Krishna, Y. Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y. Kalantidis, L.-J. Li, D. A. Shamma, et al. [Visual genome: Connecting language and vision using crowdsourced dense image annotations](https://md.research.elamur.ai/free/en/papers/krishna-2016-visual-genome-connecting-language-vision/). IJCV, 2017. 2
26. A. Krizhevsky, I. Sutskever, and G. E. Hinton. [ImageNet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/). In NIPS. 2012. 1
27. K. Kurach, M. Andrychowicz, and I. Sutskever. Neural random-access machines. In ICLR, 2016. 2 [find](https://scholar.google.com/scholar?q=K%2E+Kurach%2C+M%2E+Andrychowicz%2C+and+I%2E+Sutskever%2E+Neural+random%2Daccess+machines%2E+In+ICLR%2C+2016%2E+2)
28. B. M. Lake, T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman. Building machines that learn and think like people. Behavioral and Brain Sciences, 2016. 1 [find](https://scholar.google.com/scholar?q=B%2E+M%2E+Lake%2C+T%2E+D%2E+Ullman%2C+J%2E+B%2E+Tenenbaum%2C+and+S%2E+J%2E+Gershman%2E+Building+machines+that+learn+and+think+like+people%2E+Behavioral+and+Brain+Sciences%2C+2016%2E+1)
29. C. Liang, J. Berant, Q. Le, K. D. Forbus, and N. Lao. Neural symbolic machines: Learning semantic parsers on freebase with weak supervision. arXiv preprint arXiv:1611.00020, 2016. 2, 8 [find](https://scholar.google.com/scholar?q=C%2E+Liang%2C+J%2E+Berant%2C+Q%2E+Le%2C+K%2E+D%2E+Forbus%2C+and+N%2E+Lao%2E+Neural+symbolic+machines%3A+Learning+semantic+parsers+on+freebase+with+weak+supervision%2E+arXiv+preprint+arXiv%3A1611%2E00020%2C+2016%2E+2%2C+8)
30. P. Liang, M. I. Jordan, and D. Klein. Learning dependency-based compositional semantics. In ACL, 2011. 2 [find](https://scholar.google.com/scholar?q=P%2E+Liang%2C+M%2E+I%2E+Jordan%2C+and+D%2E+Klein%2E+Learning+dependency%2Dbased+compositional+semantics%2E+In+ACL%2C+2011%2E+2)
31. J. Lu, J. Yang, D. Batra, and D. Parikh. Hierarchical question-image co-attention for visual question answering. In NIPS, 2016. 2 [find](https://scholar.google.com/scholar?q=J%2E+Lu%2C+J%2E+Yang%2C+D%2E+Batra%2C+and+D%2E+Parikh%2E+Hierarchical+question%2Dimage+co%2Dattention+for+visual+question+answering%2E+In+NIPS%2C+2016%2E+2)
32. M. Malinowski and M. Fritz. A multi-world approach to question answering about real-world scenes based on uncertain input. In NIPS, 2014. 2 [find](https://scholar.google.com/scholar?q=M%2E+Malinowski+and+M%2E+Fritz%2E+A+multi%2Dworld+approach+to+question+answering+about+real%2Dworld+scenes+based+on+uncertain+input%2E+In+NIPS%2C+2014%2E+2)
33. M. Malinowski, M. Rohrbach, and M. Fritz. Ask your neurons: A neural-based approach to answering questions about images. In ICCV, 2015. 2, 4 [find](https://scholar.google.com/scholar?q=M%2E+Malinowski%2C+M%2E+Rohrbach%2C+and+M%2E+Fritz%2E+Ask+your+neurons%3A+A+neural%2Dbased+approach+to+answering+questions+about+images%2E+In+ICCV%2C+2015%2E+2%2C+4)
34. A. Mallya and S. Lazebnik. Learning models for actions and person-object interactions with transfer to question answering. In ECCV, 2016. 2 [find](https://scholar.google.com/scholar?q=A%2E+Mallya+and+S%2E+Lazebnik%2E+Learning+models+for+actions+and+person%2Dobject+interactions+with+transfer+to+question+answering%2E+In+ECCV%2C+2016%2E+2)
35. A. Neelakantan, Q. V. Le, and I. Sutskever. Neural programmer: Inducing latent programs with gradient descent. In ICLR, 2016. 2 [find](https://scholar.google.com/scholar?q=A%2E+Neelakantan%2C+Q%2E+V%2E+Le%2C+and+I%2E+Sutskever%2E+Neural+programmer%3A+Inducing+latent+programs+with+gradient+descent%2E+In+ICLR%2C+2016%2E+2)
36. S. Reed and N. De Freitas. Neural programmer-interpreters. In ICLR, 2016. 2, 8 [find](https://scholar.google.com/scholar?q=S%2E+Reed+and+N%2E+De+Freitas%2E+Neural+programmer%2Dinterpreters%2E+In+ICLR%2C+2016%2E+2%2C+8)
37. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. [ImageNet large scale visual recognition challenge](https://md.research.elamur.ai/free/en/papers/russakovsky-2014-imagenet-large-scale-visual-recognition/). IJCV, 2015. 3, 9
38. S. Sukhbaatar, A. Szlam, J. Weston, and R. Fergus. End-to-end memory networks. In NIPS, 2015. 2 [find](https://scholar.google.com/scholar?q=S%2E+Sukhbaatar%2C+A%2E+Szlam%2C+J%2E+Weston%2C+and+R%2E+Fergus%2E+End%2Dto%2Dend+memory+networks%2E+In+NIPS%2C+2015%2E+2)
39. I. Sutskever, O. Vinyals, and Q. V. Le. [Sequence to sequence learning with neural networks](https://md.research.elamur.ai/free/en/papers/sutskever-2014-sequence-sequence-learning-neural-networks/). In NIPS, 2014. 3, 9
40. M. Tapaswi, Y. Zhu, R. Stiefelhagen, A. Torralba, R. Urtasun, and S. Fidler. Movieqa: Understanding stories in movies through question-answering. In CVPR, 2016. 2 [find](https://scholar.google.com/scholar?q=M%2E+Tapaswi%2C+Y%2E+Zhu%2C+R%2E+Stiefelhagen%2C+A%2E+Torralba%2C+R%2E+Urtasun%2C+and+S%2E+Fidler%2E+Movieqa%3A+Understanding+stories+in+movies+through+question%2Danswering%2E+In+CVPR%2C+2016%2E+2)
41. J. Weston, S. Chopra, and A. Bordes. Memory networks. In ICLR, 2015. 2 [find](https://scholar.google.com/scholar?q=J%2E+Weston%2C+S%2E+Chopra%2C+and+A%2E+Bordes%2E+Memory+networks%2E+In+ICLR%2C+2015%2E+2)
42. R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(23), 1992. 4 [find](https://scholar.google.com/scholar?q=R%2E+J%2E+Williams%2E+Simple+statistical+gradient%2Dfollowing+algorithms+for+connectionist+reinforcement+learning%2E+Machine+Learning%2C+8%2823%29%2C+1992%2E+4)
43. T. Winograd. Understanding Natural Language. Academic Press, 1972. 2 [find](https://scholar.google.com/scholar?q=T%2E+Winograd%2E+Understanding+Natural+Language%2E+Academic+Press%2C+1972%2E+2)
44. Q. Wu, D. Teney, P. Wang, C. Shen, A. Dick, and A. van den Hengel. Visual question answering: A survey of methods and datasets. In arXiv preprint arXiv:1607.05910, 2016. 2 — [arXiv:1607.05910](https://arxiv.org/abs/1607.05910) [find](https://scholar.google.com/scholar?q=Q%2E+Wu%2C+D%2E+Teney%2C+P%2E+Wang%2C+C%2E+Shen%2C+A%2E+Dick%2C+and+A%2E+van+den+Hengel%2E+Visual+question+answering%3A+A+survey+of+methods+and+datasets%2E+In+arXiv+preprint+arXiv%3A1607%2E05910%2C+2016%2E+2+%E2%80%94+%5BarXiv%3A1607%2E05910%5D%28https%3A%2F)
45. C. Xiong, S. Merity, and R. Socher. Dynamic memory networks for visual and textual question answering. ICML, 2016. 2 [find](https://scholar.google.com/scholar?q=C%2E+Xiong%2C+S%2E+Merity%2C+and+R%2E+Socher%2E+Dynamic+memory+networks+for+visual+and+textual+question+answering%2E+ICML%2C+2016%2E+2)
46. Z. Yang, X. He, J. Gao, L. Deng, and A. Smola. [Stacked attention networks for image question answering](https://md.research.elamur.ai/free/en/papers/yang-2015-stacked-attention-networks-image-question/). In CVPR, 2016. 4, 5, 10
47. W. Zaremba, T. Mikolov, A. Joulin, and R. Fergus. Learning simple algorithms from examples. In ICML, 2016. 2 [find](https://scholar.google.com/scholar?q=W%2E+Zaremba%2C+T%2E+Mikolov%2C+A%2E+Joulin%2C+and+R%2E+Fergus%2E+Learning+simple+algorithms+from+examples%2E+In+ICML%2C+2016%2E+2)
48. W. Zaremba and I. Sutskever. Learning to execute. arXiv preprint arXiv:1410.4615, 2014. 2 [find](https://scholar.google.com/scholar?q=W%2E+Zaremba+and+I%2E+Sutskever%2E+Learning+to+execute%2E+arXiv+preprint+arXiv%3A1410%2E4615%2C+2014%2E+2)
49. W. Zaremba and I. Sutskever. Reinforcement learning neural turing machines. arXiv preprint arXiv:1505.[00521, 2015](https://md.research.elamur.ai/free/en/papers/zaremba-2015-reinforcement-learning-neural-turing-machines/). 2
50. P. Zhang, Y. Goyal, D. Summers-Stay, D. Batra, and D. Parikh. Yin and yang: Balancing and answering binary visual questions. In CVPR, 2016. 2 [find](https://scholar.google.com/scholar?q=P%2E+Zhang%2C+Y%2E+Goyal%2C+D%2E+Summers%2DStay%2C+D%2E+Batra%2C+and+D%2E+Parikh%2E+Yin+and+yang%3A+Balancing+and+answering+binary+visual+questions%2E+In+CVPR%2C+2016%2E+2)
51. Y. Zhu, O. Groth, M. Bernstein, and L. Fei-Fei. [Visual7W: Grounded question answering in images](https://md.research.elamur.ai/free/en/papers/zhu-2015-visual7w-grounded-question-answering-images/). In CVPR, 2016. 1, 2

*Updated 2 November 2026*