---
title: "Neural-Symbolic VQA — paper by Yi et al. (2018)"
description: "Yi et al. propose NS-VQA, which separates perception and language understanding… 99.8% accuracy. 50 references, cited by 87 papers here."
authors: ["Kexin Yi", "Jiajun Wu", "Chuang Gan", "Antonio Torralba", "Pushmeet Kohli", "Joshua B. Tenenbaum"]
year: 2018
venue: "arXiv:1810.02338"
source: "https://arxiv.org/abs/1810.02338"
doi: "10.48550/arxiv.1810.02338"
arxiv: "1810.02338"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yi-2018-neural-symbolic-vqa-disentangling-reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yi-2018-neural-symbolic-vqa-disentangling-reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding

# Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding

**Authors:** Kexin Yi, [Jiajun Wu](https://md.research.elamur.ai/free/en/authors/jiajun-wu/), [Chuang Gan](https://md.research.elamur.ai/free/en/authors/chuang-gan/), [Antonio Torralba](https://md.research.elamur.ai/free/en/authors/antonio-b-torralba/), [Pushmeet Kohli](https://md.research.elamur.ai/free/en/authors/pushmeet-kohli/) and [Joshua B. Tenenbaum](https://md.research.elamur.ai/free/en/authors/joshua-b-tenenbaum/) · **Year:** 2018 · **Venue:** arXiv:1810.02338 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1810.02338) · [PDF](https://arxiv.org/pdf/1810.02338) · [DOI 10.48550/arxiv.1810.02338](https://doi.org/10.48550/arxiv.1810.02338) · [arXiv 1810.02338](https://arxiv.org/abs/1810.02338) · [OpenAlex](https://openalex.org/W2891021031)

**Citations:** 320 citations (OpenAlex, 2026-10-14) · [87](https://md.research.elamur.ai/free/en/papers/yi-2018-neural-symbolic-vqa-disentangling-reasoning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
End-to-end visual question answering (VQA) models often struggle with compositional, multi-step reasoning. They also reveal little about how they arrive at an answer. Yi et al. propose NS-VQA, which separates perception and language understanding from reasoning. A neural scene parser turns an image into a compact table of objects and their attributes. It uses Mask R-CNN proposals plus a ResNet-34 that estimates 3D position and pose. An attention-based seq2seq LSTM translates the question into a hierarchical program of functional modules. This parser is first trained on a few annotated programs and then refined with REINFORCE. A deterministic symbolic executor runs the program over the object table to produce the answer. On CLEVR, the system reaches 99.8% accuracy with only 270 annotated programs and 4K images. It beats the IEP baseline on CLEVR-Humans under limited program supervision and transfers to a new Minecraft-based dataset. The work shows that building symbolic structure into a neural pipeline can raise accuracy, reduce annotation and memory needs, and make every reasoning step inspectable.

## Key points
- Pipeline: Mask R-CNN segments objects and predicts color, material, size and shape, dropping proposals scored below 0.9. A ResNet-34 receives each segment together with the full image at 224×224 and predicts spatial attributes.
- CLEVR: 99.8% accuracy using 270 program annotations and 4K training images. The structural scene representation cuts memory cost for offline QA by about 99% compared with state-of-the-art methods.
- CLEVR-Humans: the question parser is pretrained on CLEVR programs, with fixed GloVe embeddings. It is then fine-tuned with REINFORCE for up to 1M iterations with early stopping, and it clearly outperforms IEP when few programs are annotated.
- Minecraft extension: 10,000 scenes were rendered via Malmo, each with 3–6 objects drawn from 12 entity types and encoded as 18-dim vectors. The split is 9,000 images / 88,109 questions for training and 1,000 / 9,761 for testing. Accuracy is 71.1%, 72.4%, 86.9% and 87.3% with 50, 100, 200 and 500 annotated programs.
- Most Minecraft errors come from heavily occluded objects rather than from question parsing. The authors name generalizing structured representations to truly novel situations as an open challenge.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 243 works · 76,506 citations
- [IBM](https://md.research.elamur.ai/free/en/companies/ibm/) — Company, Armonk, United States · 154 works · 72,606 citations
- [Harvard University](https://md.research.elamur.ai/free/en/universities/harvard-university/) — University, Cambridge, United States · 126 works · 45,130 citations
- [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) — Research institute, Cambridge, United States · 76 works · 32,207 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works

## Cite

Cite as: Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli and Joshua B. Tenenbaum. 2018. Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding. arXiv:1810.02338.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{yi2018neural,
  title = {{Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding}},
  author = {Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli and Joshua B. Tenenbaum},
  year = {2018},
  doi = {10.48550/arxiv.1810.02338},
  eprint = {1810.02338},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1810.02338}
}
```

- [← previous Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network](https://md.research.elamur.ai/free/en/papers/zhou-2018-multi-turn-response-selection-chatbots/)
- [next → Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems](https://md.research.elamur.ai/free/en/papers/madotto-2018-mem2seq-effectively-incorporating-knowledge-bases/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/)

## References

50 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Somak Aditya, Yezhou Yang, and Chitta Baral. Explicit reasoning over end-to-end neural architectures for visual question answering. In AAAI, 2018. 3 [find](https://scholar.google.com/scholar?q=Somak+Aditya%2C+Yezhou+Yang%2C+and+Chitta+Baral%2E+Explicit+reasoning+over+end%2Dto%2Dend+neural+architectures+for+visual+question+answering%2E+In+AAAI%2C+2018%2E+3)
2. Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. [Learning to compose neural networks for question answering](https://md.research.elamur.ai/free/en/papers/andreas-2016-learning-compose-neural-networks-question/). In NAACL-HLT, 2016. 3
3. Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In ICCV, 2015. 1, 3 [find](https://scholar.google.com/scholar?q=Stanislaw+Antol%2C+Aishwarya+Agrawal%2C+Jiasen+Lu%2C+Margaret+Mitchell%2C+Dhruv+Batra%2C+C+Lawrence+Zitnick%2C+and+Devi+Parikh%2E+Vqa%3A+Visual+question+answering%2E+In+ICCV%2C+2015%2E+1%2C+3)
4. Jimmy Ba, Volodymyr Mnih, and Koray Kavukcuoglu. Multiple object recognition with visual attention. In ICLR, 2015. 2 [find](https://scholar.google.com/scholar?q=Jimmy+Ba%2C+Volodymyr+Mnih%2C+and+Koray+Kavukcuoglu%2E+Multiple+object+recognition+with+visual+attention%2E+In+ICLR%2C+2015%2E+2)
5. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). In ICLR, 2015. 4
6. Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. Deepcoder: Learning to write programs. In ICLR, 2017. 2 [find](https://scholar.google.com/scholar?q=Matej+Balog%2C+Alexander+L+Gaunt%2C+Marc+Brockschmidt%2C+Sebastian+Nowozin%2C+and+Daniel+Tarlow%2E+Deepcoder%3A+Learning+to+write+programs%2E+In+ICLR%2C+2017%2E+2)
7. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. [Semantic parsing on freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In EMNLP, 2013. 2
8. Yonatan Bisk, Kevin J Shih, Yejin Choi, and Daniel Marcu. Learning interpretable spatial operations in a rich 3d blocks world. In AAAI, 2018. 3 [find](https://scholar.google.com/scholar?q=Yonatan+Bisk%2C+Kevin+J+Shih%2C+Yejin+Choi%2C+and+Daniel+Marcu%2E+Learning+interpretable+spatial+operations+in+a+rich+3d+blocks+world%2E+In+AAAI%2C+2018%2E+3)
9. Qingxing Cao, Xiaodan Liang, Bailing Li, Guanbin Li, and Liang Lin. Visual question reasoning on general dependency tree. In CVPR, 2018. 3, 5 [find](https://scholar.google.com/scholar?q=Qingxing+Cao%2C+Xiaodan+Liang%2C+Bailing+Li%2C+Guanbin+Li%2C+and+Liang+Lin%2E+Visual+question+reasoning+on+general+dependency+tree%2E+In+CVPR%2C+2018%2E+3%2C+5)
10. SM Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, Koray Kavukcuoglu, and Geoffrey E Hinton. Attend, infer, repeat: Fast scene understanding with generative models. In NIPS, 2016. 2 [find](https://scholar.google.com/scholar?q=SM+Eslami%2C+Nicolas+Heess%2C+Theophane+Weber%2C+Yuval+Tassa%2C+Koray+Kavukcuoglu%2C+and+Geoffrey+E+Hinton%2E+Attend%2C+infer%2C+repeat%3A+Fast+scene+understanding+with+generative+models%2E+In+NIPS%2C+2016%2E+2)
11. Chuang Gan, Yandong Li, Haoxiang Li, Chen Sun, and Boqing Gong. Vqs: Linking segmentations to questions and answers for supervised attention in vqa and question-focused semantic segmentation. In ICCV, 2017. 3 [find](https://scholar.google.com/scholar?q=Chuang+Gan%2C+Yandong+Li%2C+Haoxiang+Li%2C+Chen+Sun%2C+and+Boqing+Gong%2E+Vqs%3A+Linking+segmentations+to+questions+and+answers+for+supervised+attention+in+vqa+and+question%2Dfocused+semantic+segmentation%2E+In+ICCV%2C)
12. Ross Girshick, Ilija Radosavovic, Georgia Gkioxari, Piotr Dollár, and Kaiming He. Detectron. [https://github.com/facebookresearch/detectron](https://github.com/facebookresearch/detectron), 2018. 5 [↗](https://github.com/facebookresearch/detectron)
13. Omer Goldman, Veronica Latcinnik, Udi Naveh, Amir Globerson, and Jonathan Berant. Weakly-supervised semantic parsing with abstract examples. In ACL, 2018. 3 [find](https://scholar.google.com/scholar?q=Omer+Goldman%2C+Veronica+Latcinnik%2C+Udi+Naveh%2C+Amir+Globerson%2C+and+Jonathan+Berant%2E+Weakly%2Dsupervised+semantic+parsing+with+abstract+examples%2E+In+ACL%2C+2018%2E+3)
14. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. [Making the v in vqa matter: Elevating the role of image understanding in visual question answering](https://md.research.elamur.ai/free/en/papers/goyal-2016-making-v-vqa-matter-elevating/). In CVPR, 2017. 3
15. Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, and Percy Liang. [From language to programs: Bridging reinforcement learning and maximum marginal likelihood](https://md.research.elamur.ai/free/en/papers/guu-2017-language-programs-bridging-reinforcement-learning/). In ACL, 2017. 3
16. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). In CVPR, 2015. 4
17. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, 2017. 4, 5 [find](https://scholar.google.com/scholar?q=Kaiming+He%2C+Georgia+Gkioxari%2C+Piotr+Doll%C3%A1r%2C+and+Ross+Girshick%2E+Mask+r%2Dcnn%2E+In+ICCV%2C+2017%2E+4%2C+5)
18. Irina Higgins, Nicolas Sonnerat, Loic Matthey, Arka Pal, Christopher P Burgess, Matthew Botvinick, Demis Hassabis, and Alexander Lerchner. Scan: learning abstract hierarchical compositional visual concepts. In ICLR, 2018. 2 [find](https://scholar.google.com/scholar?q=Irina+Higgins%2C+Nicolas+Sonnerat%2C+Loic+Matthey%2C+Arka+Pal%2C+Christopher+P+Burgess%2C+Matthew+Botvinick%2C+Demis+Hassabis%2C+and+Alexander+Lerchner%2E+Scan%3A+learning+abstract+hierarchical+compositional+visual+con)
19. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Comput., 9(8):1735–1780, 1997. 4 [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+J%C3%BCrgen+Schmidhuber%2E+Long+short%2Dterm+memory%2E+Neural+Comput%2E%2C+9%288%29%3A1735%E2%80%931780%2C+1997%2E+4)
20. Ronghang Hu, Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Kate Saenko. Learning to reason: End-to-end module networks for visual question answering. In CVPR, 2017. 3, 5 [find](https://scholar.google.com/scholar?q=Ronghang+Hu%2C+Jacob+Andreas%2C+Marcus+Rohrbach%2C+Trevor+Darrell%2C+and+Kate+Saenko%2E+Learning+to+reason%3A+End%2Dto%2Dend+module+networks+for+visual+question+answering%2E+In+CVPR%2C+2017%2E+3%2C+5)
21. Drew A Hudson and Christopher D Manning. [Compositional attention networks for machine reasoning](https://md.research.elamur.ai/free/en/papers/hudson-2018-compositional-attention-networks-machine-reasoning/). In ICLR, 2018. 3, 5, 7
22. Allan Jabri, Armand Joulin, and Laurens van der Maaten. Revisiting visual question answering baselines. In ECCV, 2016. 3 [find](https://scholar.google.com/scholar?q=Allan+Jabri%2C+Armand+Joulin%2C+and+Laurens+van+der+Maaten%2E+Revisiting+visual+question+answering+baselines%2E+In+ECCV%2C+2016%2E+3)
23. Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. [Clevr: A diagnostic dataset for compositional language and elementary visual reasoning](https://md.research.elamur.ai/free/en/papers/johnson-2016-clevr-diagnostic-dataset-compositional-language/). In CVPR, 2017a. 1, 3, 6, 7
24. Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. [Inferring and executing programs for visual reasoning](https://md.research.elamur.ai/free/en/papers/johnson-2017-inferring-executing-programs-visual-reasoning/). In ICCV, 2017b. 1, 3, 4, 5, 6, 7, 8
25. Matthew Johnson, Katja Hofmann, Tim Hutton, and David Bignell. The malmo platform for artificial intelligence experimentation. In IJCAI, 2016. 8 [find](https://scholar.google.com/scholar?q=Matthew+Johnson%2C+Katja+Hofmann%2C+Tim+Hutton%2C+and+David+Bignell%2E+The+malmo+platform+for+artificial+intelligence+experimentation%2E+In+IJCAI%2C+2016%2E+8)
26. Tejas D Kulkarni, William F Whitney, Pushmeet Kohli, and Joshua B Tenenbaum. Deep convolutional inverse graphics network. In NIPS, 2015. 2 [find](https://scholar.google.com/scholar?q=Tejas+D+Kulkarni%2C+William+F+Whitney%2C+Pushmeet+Kohli%2C+and+Joshua+B+Tenenbaum%2E+Deep+convolutional+inverse+graphics+network%2E+In+NIPS%2C+2015%2E+2)
27. Percy Liang, Michael I Jordan, and Dan Klein. Learning dependency-based compositional semantics. Computational Linguistics, 39(2):389–446, 2013. 2 [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Michael+I+Jordan%2C+and+Dan+Klein%2E+Learning+dependency%2Dbased+compositional+semantics%2E+Computational+Linguistics%2C+39%282%29%3A389%E2%80%93446%2C+2013%2E+2)
28. Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. [Feature pyramid networks for object detection](https://md.research.elamur.ai/free/en/papers/lin-2016-feature-pyramid-networks-object-detection/). In CVPR, 2017. 5
29. Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. Hierarchical question-image co-attention for visual question answering. In NIPS, 2016. 3 [find](https://scholar.google.com/scholar?q=Jiasen+Lu%2C+Jianwei+Yang%2C+Dhruv+Batra%2C+and+Devi+Parikh%2E+Hierarchical+question%2Dimage+co%2Dattention+for+visual+question+answering%2E+In+NIPS%2C+2016%2E+3)
30. Minh-Thang Luong, Hieu Pham, and Christopher D Manning. [Effective approaches to attention-based neural machine translation](https://md.research.elamur.ai/free/en/papers/luong-2015-effective-approaches-attention-based-neural/). In EMNLP, 2015. 4
31. M Malinowski and M Fritz. A multi-world approach to question answering about real-world scenes based on uncertain input. In NIPS, 2014. 1, 3 [find](https://scholar.google.com/scholar?q=M+Malinowski+and+M+Fritz%2E+A+multi%2Dworld+approach+to+question+answering+about+real%2Dworld+scenes+based+on+uncertain+input%2E+In+NIPS%2C+2014%2E+1%2C+3)
32. David Mascharka, Philip Tran, Ryan Soklaski, and Arjun Majumdar. Transparency by design: Closing the gap between performance and interpretability in visual reasoning. In CVPR, 2018. 3, 5, 7 [find](https://scholar.google.com/scholar?q=David+Mascharka%2C+Philip+Tran%2C+Ryan+Soklaski%2C+and+Arjun+Majumdar%2E+Transparency+by+design%3A+Closing+the+gap+between+performance+and+interpretability+in+visual+reasoning%2E+In+CVPR%2C+2018%2E+3%2C+5%2C+7)
33. Ishan Misra, Ross Girshick, Rob Fergus, Martial Hebert, Abhinav Gupta, and Laurens van der Maaten. Learning by asking questions. In CVPR, 2018. 3 [find](https://scholar.google.com/scholar?q=Ishan+Misra%2C+Ross+Girshick%2C+Rob+Fergus%2C+Martial+Hebert%2C+Abhinav+Gupta%2C+and+Laurens+van+der+Maaten%2E+Learning+by+asking+questions%2E+In+CVPR%2C+2018%2E+3)
34. Arvind Neelakantan, Quoc V Le, and Ilya Sutskever. Neural programmer: Inducing latent programs with gradient descent. In ICLR, 2016. 2 [find](https://scholar.google.com/scholar?q=Arvind+Neelakantan%2C+Quoc+V+Le%2C+and+Ilya+Sutskever%2E+Neural+programmer%3A+Inducing+latent+programs+with+gradient+descent%2E+In+ICLR%2C+2016%2E+2)
35. Emilio Parisotto, Abdel-rahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, and Pushmeet Kohli. [Neuro-symbolic program synthesis](https://md.research.elamur.ai/free/en/papers/parisotto-2016-neuro-symbolic-program-synthesis/). In ICLR, 2017. 2
36. Jeffrey Pennington, Richard Socher, and Christopher Manning. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In EMNLP, 2014. 8
37. Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. [Film: Visual reasoning with a general conditioning layer](https://md.research.elamur.ai/free/en/papers/perez-2017-film-visual-reasoning-general-conditioning/). In AAAI, 2018. 3, 5, 7
38. Anselm Rothe, Brenden M Lake, and Todd Gureckis. Question asking as program generation. In NIPS, 2017. 3 [find](https://scholar.google.com/scholar?q=Anselm+Rothe%2C+Brenden+M+Lake%2C+and+Todd+Gureckis%2E+Question+asking+as+program+generation%2E+In+NIPS%2C+2017%2E+3)
39. Adam Santoro, David Raposo, David GT Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. [A simple neural network module for relational reasoning](https://md.research.elamur.ai/free/en/papers/santoro-2017-simple-neural-network-module-relational/). In NIPS, 2017. 3, 5
40. N Siddharth, T. B. Paige, J.W. Meent, A. Desmaison, N. Goodman, P. Kohli, F. Wood, and P. Torr. Learning disentangled representations with semi-supervised deep generative models. In NIPS, 2017. 2 [find](https://scholar.google.com/scholar?q=N+Siddharth%2C+T%2E+B%2E+Paige%2C+J%2EW%2E+Meent%2C+A%2E+Desmaison%2C+N%2E+Goodman%2C+P%2E+Kohli%2C+F%2E+Wood%2C+and+P%2E+Torr%2E+Learning+disentangled+representations+with+semi%2Dsupervised+deep+generative+models%2E+In+NIPS%2C+2017%2E+2)
41. Joseph Suarez, Justin Johnson, and Fei-Fei Li. Ddrprog: A clevr differentiable dynamic reasoning programmer. arXiv:1803.11361, 2018. 3, 5 — [arXiv:1803.11361](https://arxiv.org/abs/1803.11361) [find](https://scholar.google.com/scholar?q=Joseph+Suarez%2C+Justin+Johnson%2C+and+Fei%2DFei+Li%2E+Ddrprog%3A+A+clevr+differentiable+dynamic+reasoning+programmer%2E+arXiv%3A1803%2E11361%2C+2018%2E+3%2C+5+%E2%80%94+%5BarXiv%3A1803%2E11361%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1803%2E11361%29)
42. Ramakrishna Vedantam, Ian Fischer, Jonathan Huang, and Kevin Murphy. Generative models of visually grounded imagination. In ICLR, 2018. 2 [find](https://scholar.google.com/scholar?q=Ramakrishna+Vedantam%2C+Ian+Fischer%2C+Jonathan+Huang%2C+and+Kevin+Murphy%2E+Generative+models+of+visually+grounded+imagination%2E+In+ICLR%2C+2018%2E+2)
43. Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. Grammar as a foreign language. In NIPS, 2015. 2 [find](https://scholar.google.com/scholar?q=Oriol+Vinyals%2C+%C5%81ukasz+Kaiser%2C+Terry+Koo%2C+Slav+Petrov%2C+Ilya+Sutskever%2C+and+Geoffrey+Hinton%2E+Grammar+as+a+foreign+language%2E+In+NIPS%2C+2015%2E+2)
44. Peng Wang, Qi Wu, Chunhua Shen, Anton van den Hengel, and Anthony Dick. Explicit knowledge-based reasoning for visual question answering. In IJCAI, 2017. 3 [find](https://scholar.google.com/scholar?q=Peng+Wang%2C+Qi+Wu%2C+Chunhua+Shen%2C+Anton+van+den+Hengel%2C+and+Anthony+Dick%2E+Explicit+knowledge%2Dbased+reasoning+for+visual+question+answering%2E+In+IJCAI%2C+2017%2E+3)
45. Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. MLJ, 8(3-4):229–256, 1992. 5 [find](https://scholar.google.com/scholar?q=Ronald+J+Williams%2E+Simple+statistical+gradient%2Dfollowing+algorithms+for+connectionist+reinforcement+learning%2E+MLJ%2C+8%283%2D4%29%3A229%E2%80%93256%2C+1992%2E+5)
46. Jiajun Wu, Joshua B Tenenbaum, and Pushmeet Kohli. Neural scene de-rendering. In CVPR, 2017. 2, 8 [find](https://scholar.google.com/scholar?q=Jiajun+Wu%2C+Joshua+B+Tenenbaum%2C+and+Pushmeet+Kohli%2E+Neural+scene+de%2Drendering%2E+In+CVPR%2C+2017%2E+2%2C+8)
47. Jimei Yang, Scott E Reed, Ming-Hsuan Yang, and Honglak Lee. Weakly-supervised disentangling with recurrent transformations for 3d view synthesis. In NIPS, 2015. 2 [find](https://scholar.google.com/scholar?q=Jimei+Yang%2C+Scott+E+Reed%2C+Ming%2DHsuan+Yang%2C+and+Honglak+Lee%2E+Weakly%2Dsupervised+disentangling+with+recurrent+transformations+for+3d+view+synthesis%2E+In+NIPS%2C+2015%2E+2)
48. Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alex Smola. [Stacked attention networks for image question answering](https://md.research.elamur.ai/free/en/papers/yang-2015-stacked-attention-networks-image-question/). In CVPR, 2016. 1, 3
49. Alan Yuille and Daniel Kersten. Vision as bayesian inference: analysis by synthesis? Trends Cogn. Sci., 10(7): 301–308, 2006. 2 [find](https://scholar.google.com/scholar?q=Alan+Yuille+and+Daniel+Kersten%2E+Vision+as+bayesian+inference%3A+analysis+by+synthesis%3F+Trends+Cogn%2E+Sci%2E%2C+10%287%29%3A+301%E2%80%93308%2C+2006%2E+2)
50. Chen Zhu, Yanpeng Zhao, Shuaiyi Huang, Kewei Tu, and Yi Ma. Structured attentions for visual question answering. In ICCV, 2017. 3 [find](https://scholar.google.com/scholar?q=Chen+Zhu%2C+Yanpeng+Zhao%2C+Shuaiyi+Huang%2C+Kewei+Tu%2C+and+Yi+Ma%2E+Structured+attentions+for+visual+question+answering%2E+In+ICCV%2C+2017%2E+3)

*Updated 2 November 2026*