---
title: "Mapping Instructions and Visual Observations to Actions with"
description: "They build an interactive simulator on the Blocks data of Bisk et al. (2016) and recast the task as… 62 references, cited by 70 papers here."
authors: ["Dipendra Misra", "John Langford", "Yoav Artzi"]
year: 2017
venue: "arXiv:1704.08795"
source: "https://arxiv.org/abs/1704.08795"
doi: "10.48550/arxiv.1704.08795"
arxiv: "1704.08795"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/misra-2017-mapping-instructions-visual-observations-actions/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/misra-2017-mapping-instructions-visual-observations-actions/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/) / Mapping Instructions and Visual Observations to Actions with Reinforcement Learning

# Mapping Instructions and Visual Observations to Actions with Reinforcement Learning

**Authors:** [Dipendra Misra](https://md.research.elamur.ai/free/en/authors/dipendra-misra/), [John Langford](https://md.research.elamur.ai/free/en/authors/john-c-langford/) and [Yoav Artzi](https://md.research.elamur.ai/free/en/authors/yoav-artzi/) · **Year:** 2017 · **Venue:** [arXiv:1704.08795](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2017](https://md.research.elamur.ai/free/en/venues/emnlp/2017/)) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1704.08795) · [PDF](https://arxiv.org/pdf/1704.08795) · [DOI 10.48550/arxiv.1704.08795](https://doi.org/10.48550/arxiv.1704.08795) · [arXiv 1704.08795](https://arxiv.org/abs/1704.08795) · [OpenAlex](https://openalex.org/W2611884151)

**Citations:** 185 citations (OpenAlex, 2026-10-14) · [70](https://md.research.elamur.ai/free/en/papers/misra-2017-mapping-instructions-visual-observations-actions/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

The paper tackles natural-language instruction following when the agent sees only raw camera images, not a structured symbolic description of the world. Earlier systems either assumed such structured input or chained separately trained components for vision, language and planning. The authors instead train one neural network policy that reads the instruction, current and previous RGB frames, and the last action, then outputs the next move. They build an interactive simulator on the Blocks data of Bisk et al. (2016) and recast the task as generating full action sequences. Each step offers 81 actions, and tasks need about 15 steps on average.

Training uses policy gradient in a contextual bandit formulation. Reward shaping injects supervision from full demonstrations or from goal states alone. The approach cuts execution error by 24% compared with supervised learning and by 34–39% compared with standard RL variants. This shows that a single end-to-end model can learn grounded instruction execution from limited data.

## Key points

- **Environment:** Blocks environment with 20 blocks. The 81 actions are a block moved one step in one of four compass directions, plus STOP. Reward is computed from the true state, which is available only during training.
- **Test results (ensembles, mean distance error):**
  - Our approach: 3.78
  - SUPERVISED: 4.95
  - REINFORCE: 5.69
  - DQN: 6.15
  - Demonstrations upper bound: 0.37
- **Limited demonstrations:** Having complete demonstrations for only 12.5% of training examples recovers most of the gain. The remaining examples supply just the goal state.
- **Main failure mode:** The agent often gets near the goal but does not stop. It averages 28.7 steps versus 15.2 in development demonstrations and hits the 40-step limit 55.2% of the time.
- **Unknown words and resources:** Error rises with unknown words: 3.49 with none, 3.97 with one, 4.19 with two. The simulator, code and models are released at github.com/clic-lab/blocks.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 378 works · 161,630 citations
- [Cornell University](https://md.research.elamur.ai/free/en/universities/cornell-university/) — University, Ithaca, United States · 134 works · 65,159 citations
- [Cornell Tech](https://md.research.elamur.ai/free/en/universities/cornell-tech/) — University, New York, United States · 9 works · 2,342 citations
- [Microsoft Research New York City](https://md.research.elamur.ai/free/en/companies/microsoft-research-new-york-city/) — Company, New York, United States · 2 works · 469 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [⚙️ Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/) · 188 works · [⚙️ Control and Systems Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/control-and-systems-engineering/) · 62 works

## Cite

Cite as: Dipendra Misra, John Langford and Yoav Artzi. 2017. Mapping Instructions and Visual Observations to Actions with Reinforcement Learning. arXiv:1704.08795.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{misra2017mapping,
  title = {{Mapping Instructions and Visual Observations to Actions with Reinforcement Learning}},
  author = {Dipendra Misra, John Langford and Yoav Artzi},
  year = {2017},
  doi = {10.48550/arxiv.1704.08795},
  eprint = {1704.08795},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1704.08795}
}
```

- [← previous A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification](https://md.research.elamur.ai/free/en/papers/zhang-2017-sensitivity-analysis-practitioners-guide-convolutional/)
- [next → Evaluating Discourse Phenomena in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/bawden-2017-evaluating-discourse-phenomena-neural-machine/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/)

## References

62 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alekh Agarwal, Daniel J. Hsu, Satyen Kale, John Langford, Lihong Li, and Robert E. Schapire. 2014. Taming the monster: A fast and simple algorithm for contextual bandits. In Proceedings of the International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Alekh+Agarwal%2C+Daniel+J%2E+Hsu%2C+Satyen+Kale%2C+John+Langford%2C+Lihong+Li%2C+and+Robert+E%2E+Schapire%2E+2014%2E+Taming+the+monster%3A+A+fast+and+simple+algorithm+for+contextual+bandits%2E+In+Proceedings+of+the+Interna)
2. Jacob Andreas and Dan Klein. 2015. Alignment-based compositional semantics for instruction following. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/D15-1138. [↗](https://doi.org/10.18653/v1/D15-1138)
3. Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. 2016a. [Learning to compose neural networks for question answering](https://md.research.elamur.ai/free/en/papers/andreas-2016-learning-compose-neural-networks-question/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. https://doi.org/10.1865…
4. Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. 2016b. [Neural module networks](https://md.research.elamur.ai/free/en/papers/andreas-2015-neural-module-networks/). In Conference on Computer Vision and Pattern Recognition.
5. Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. VQA: Visual question answering. In International Journal of Computer Vision. [find](https://scholar.google.com/scholar?q=Stanislaw+Antol%2C+Aishwarya+Agrawal%2C+Jiasen+Lu%2C+Margaret+Mitchell%2C+Dhruv+Batra%2C+C%2E+Lawrence+Zitnick%2C+and+Devi+Parikh%2E+2015%2E+VQA%3A+Visual+question+answering%2E+In+International+Journal+of+Computer+Vision%2E)
6. Yoav Artzi, Dipanjan Das, and Slav Petrov. 2014a. Learning compact lexicons for CCG semantic parsing. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.3115/v1/D14-1134. [↗](https://doi.org/10.3115/v1/D14-1134)
7. Yoav Artzi, Maxwell Forbes, Kenton Lee, and Maya Cakmak. 2014b. Programming by demonstration with situated semantic parsing. In AAAI Fall Symposium Series. [find](https://scholar.google.com/scholar?q=Yoav+Artzi%2C+Maxwell+Forbes%2C+Kenton+Lee%2C+and+Maya+Cakmak%2E+2014b%2E+Programming+by+demonstration+with+situated+semantic+parsing%2E+In+AAAI+Fall+Symposium+Series%2E)
8. Yoav Artzi and Luke Zettlemoyer. 2013. [Weakly supervised learning of semantic parsers for mapping instructions to actions](https://md.research.elamur.ai/free/en/papers/artzi-2013-weakly-supervised-learning-semantic-parsers/). Transactions of the Association of Computational Linguistics 1:49–62. [http://aclweb.org/anthology/Q13-1005](http://aclweb.org/anthology/Q13-1005). [↗](http://aclweb.org/anthology/Q13-1005)
9. Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. 2002. The nonstochastic multi-armed bandit problem. SIAM J. Comput. 32(1):48– 77. [find](https://scholar.google.com/scholar?q=Peter+Auer%2C+Nicol%C3%B2+Cesa%2DBianchi%2C+Yoav+Freund%2C+and+Robert+E%2E+Schapire%2E+2002%2E+The+nonstochastic+multi%2Darmed+bandit+problem%2E+SIAM+J%2E+Comput%2E+32%281%29%3A48%E2%80%93+77%2E)
10. Yonatan Bisk, Deniz Yuret, and Daniel Marcu. 2016. Natural language communication with robots. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. https://doi.org/10.18653/v1/N16-1089. [↗](https://doi.org/10.18653/v1/N16-1089)
11. S.R.K. Branavan, Harr Chen, Luke Zettlemoyer, and Regina Barzilay. 2009. Reinforcement learning for mapping instructions to actions. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. ht… [find](https://scholar.google.com/scholar?q=S%2ER%2EK%2E+Branavan%2C+Harr+Chen%2C+Luke+Zettlemoyer%2C+and+Regina+Barzilay%2E+2009%2E+Reinforcement+learning+for+mapping+instructions+to+actions%2E+In+Proceedings+of+the+Joint+Conference+of+the+47th+Annual+Meeting+o)
12. S.R.K. Branavan, Luke Zettlemoyer, and Regina Barzilay. 2010. Reading between the lines: Learning to map high-level instructions to commands. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. [http://aclweb.org/anthology/P10-1129](http://aclweb.org/anthology/P10-1129). [↗](http://aclweb.org/anthology/P10-1129)
13. Tim Brys, Anna Harutyunyan, Halit Bener Suay, Sonia Chernova, Matthew E. Taylor, and Ann Nowé. 2015. Reinforcement learning from demonstration through shaping. In Proceedings of the International Joint Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Tim+Brys%2C+Anna+Harutyunyan%2C+Halit+Bener+Suay%2C+Sonia+Chernova%2C+Matthew+E%2E+Taylor%2C+and+Ann+Now%C3%A9%2E+2015%2E+Reinforcement+learning+from+demonstration+through+shaping%2E+In+Proceedings+of+the+International+Join)
14. David L. Chen and Raymond J. Mooney. 2011. Learning to interpret natural language navigation instructions from observations. In Proceedings of the National Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=David+L%2E+Chen+and+Raymond+J%2E+Mooney%2E+2011%2E+Learning+to+interpret+natural+language+navigation+instructions+from+observations%2E+In+Proceedings+of+the+National+Conference+on+Artificial+Intelligence%2E)
15. Wenhu Chen, Aurélien Lucchi, and Thomas Hofmann. 2016. Bootstrap, review, decode: Using out-of-domain textual data to improve image captioning. CoRR abs/1611.05321. — [arXiv:1611.05321](https://arxiv.org/abs/1611.05321) [find](https://scholar.google.com/scholar?q=Wenhu+Chen%2C+Aur%C3%A9lien+Lucchi%2C+and+Thomas+Hofmann%2E+2016%2E+Bootstrap%2C+review%2C+decode%3A+Using+out%2Dof%2Ddomain+textual+data+to+improve+image+captioning%2E+CoRR+abs%2F1611%2E05321%2E+%E2%80%94+%5BarXiv%3A1611%2E05321%5D%28https%3A%2F%2Farxiv%2E)
16. Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C. Lawrence Zitnick. 2015. [Microsoft COCO captions: Data collection and evaluation server](https://md.research.elamur.ai/free/en/papers/chen-2015-microsoft-coco-captions-data-collection/). CoRR abs/1504.00325.
17. Kevin Clark and D. Christopher Manning. 2016. [Deep reinforcement learning for mention-ranking coreference models](https://md.research.elamur.ai/free/en/papers/clark-2016-deep-reinforcement-learning-mention-ranking/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. [http://aclweb.org/anthology/D16-1245](http://aclweb.org/anthology/D16-1245). [↗](http://aclweb.org/anthology/D16-1245)
18. Jeffrey L. Elman. 1990. Finding structure in time. Cognitive Science 14:179–211. [find](https://scholar.google.com/scholar?q=Jeffrey+L%2E+Elman%2E+1990%2E+Finding+structure+in+time%2E+Cognitive+Science+14%3A179%E2%80%93211%2E)
19. Ji He, Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng, and Mari Ostendorf. 2016. Deep reinforcement learning with a natural language action space. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653… [find](https://scholar.google.com/scholar?q=Ji+He%2C+Jianshu+Chen%2C+Xiaodong+He%2C+Jianfeng+Gao%2C+Lihong+Li%2C+Li+Deng%2C+and+Mari+Ostendorf%2E+2016%2E+Deep+reinforcement+learning+with+a+natural+language+action+space%2E+In+Proceedings+of+the+54th+Annual+Meetin)
20. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+J%C3%BCrgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation+9%2E)
21. Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross B. Girshick. 2016. [CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning](https://md.research.elamur.ai/free/en/papers/johnson-2016-clevr-diagnostic-dataset-compositional-language/). CoRR abs/1612.06890.
22. Sham Kakade and John Langford. 2002. Approximately optimal approximate reinforcement learning. In Machine Learning, Proceedings of the Nineteenth International Conference (ICML 2002), University of New South Wales, Sydney, Australia, July 8-12, 2002. [find](https://scholar.google.com/scholar?q=Sham+Kakade+and+John+Langford%2E+2002%2E+Approximately+optimal+approximate+reinforcement+learning%2E+In+Machine+Learning%2C+Proceedings+of+the+Nineteenth+International+Conference+%28ICML+2002%29%2C+University+of+Ne)
23. Michael Kearns, Yishay Mansour, and Andrew Y. Ng. 1999. A sparse sampling algorithm for near-optimal planning in large markov decision processes. In Proeceediings of the International Joint Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Michael+Kearns%2C+Yishay+Mansour%2C+and+Andrew+Y%2E+Ng%2E+1999%2E+A+sparse+sampling+algorithm+for+near%2Doptimal+planning+in+large+markov+decision+processes%2E+In+Proeceediings+of+the+International+Joint+Conference)
24. Joohyun Kim and Raymond Mooney. 2012. Unsupervised PCFG induction for grounded language learning with highly ambiguous supervision. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. [http://aclweb.org/anthology…](http://aclweb.org/anthology…) [↗](http://aclweb.org/anthology…)
25. Joohyun Kim and Raymond Mooney. 2013. Adapting discriminative reranking to grounded language learning. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). [http://aclweb.org/anthology/P13-1022](http://aclweb.org/anthology/P13-1022). [↗](http://aclweb.org/anthology/P13-1022)
26. Diederik Kingma and Jimmy Ba. 2014. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In Proceedings of the International Conference on Learning Representations.
27. Jens Kober, J. Andrew Bagnell, and Jan Peters. 2013. Reinforcement learning in robotics: A survey. International Journal of Robotics Research 32:1238– 1274. [find](https://scholar.google.com/scholar?q=Jens+Kober%2C+J%2E+Andrew+Bagnell%2C+and+Jan+Peters%2E+2013%2E+Reinforcement+learning+in+robotics%3A+A+survey%2E+International+Journal+of+Robotics+Research+32%3A1238%E2%80%93+1274%2E)
28. Akshay Krishnamurthy, Alekh Agarwal, and John Langford. 2016. PAC reinforcement learning with rich observations. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Akshay+Krishnamurthy%2C+Alekh+Agarwal%2C+and+John+Langford%2E+2016%2E+PAC+reinforcement+learning+with+rich+observations%2E+In+Advances+in+Neural+Information+Processing+Systems%2E)
29. John Langford and Tong Zhang. 2007. The epoch-greedy algorithm for multi-armed bandits with side information. In Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, Dece… [find](https://scholar.google.com/scholar?q=John+Langford+and+Tong+Zhang%2E+2007%2E+The+epoch%2Dgreedy+algorithm+for+multi%2Darmed+bandits+with+side+information%2E+In+Advances+in+Neural+Information+Processing+Systems+20%2C+Proceedings+of+the+Twenty%2DFirst+A)
30. Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. 2016. End-to-end training of deep visuomotor policies. Journal of Machine Learning Research 17. [find](https://scholar.google.com/scholar?q=Sergey+Levine%2C+Chelsea+Finn%2C+Trevor+Darrell%2C+and+Pieter+Abbeel%2E+2016%2E+End%2Dto%2Dend+training+of+deep+visuomotor+policies%2E+Journal+of+Machine+Learning+Research+17%2E)
31. Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016. [Deep reinforcement learning for dialogue generation](https://md.research.elamur.ai/free/en/papers/li-2016-deep-reinforcement-learning-dialogue-generation/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. [http://aclweb.org/anthology/D16-1127](http://aclweb.org/anthology/D16-1127). [↗](http://aclweb.org/anthology/D16-1127)
32. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. [Effective approaches to attention-based neural machine translation](https://md.research.elamur.ai/free/en/papers/luong-2015-effective-approaches-attention-based-neural/). In Proceedings of the Conference on Empirical Methods in Natural Language Processing. [http://aclweb.org/anthology/D15-1166](http://aclweb.org/anthology/D15-1166). [↗](http://aclweb.org/anthology/D15-1166)
33. Matthew MacMahon, Brian Stankiewics, and Benjamin Kuipers. 2006. Walk the talk: Connecting language, knowledge, action in route instructions. In Proceedings of the National Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Matthew+MacMahon%2C+Brian+Stankiewics%2C+and+Benjamin+Kuipers%2E+2006%2E+Walk+the+talk%3A+Connecting+language%2C+knowledge%2C+action+in+route+instructions%2E+In+Proceedings+of+the+National+Conference+on+Artificial+In)
34. Cynthia Matuszek, Dieter Fox, and Karl Koscher. 2010. Following directions using statistical machine translation. In Proceedings of the international conference on Human-robot interaction. [find](https://scholar.google.com/scholar?q=Cynthia+Matuszek%2C+Dieter+Fox%2C+and+Karl+Koscher%2E+2010%2E+Following+directions+using+statistical+machine+translation%2E+In+Proceedings+of+the+international+conference+on+Human%2Drobot+interaction%2E)
35. Cynthia Matuszek, Evan Herbst, Luke S. Zettlemoyer, and Dieter Fox. 2012. Learning to parse natural language commands to a robot control system. In Proceedings of the International Symposium on Experimental Robotics. [find](https://scholar.google.com/scholar?q=Cynthia+Matuszek%2C+Evan+Herbst%2C+Luke+S%2E+Zettlemoyer%2C+and+Dieter+Fox%2E+2012%2E+Learning+to+parse+natural+language+commands+to+a+robot+control+system%2E+In+Proceedings+of+the+International+Symposium+on+Experi)
36. Hongyuan Mei, Mohit Bansal, and R. Matthew Walter. 2016. [What to talk about and how? selective generation using lstms with coarse-to-fine alignment](https://md.research.elamur.ai/free/en/papers/mei-2016-what-talk-about-how-selective/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. http…
37. Dipendra K. Misra, Jaeyong Sung, Kevin Lee, and Ashutosh Saxena. 2016. Tell me dave: Context-sensitive grounding of natural language to manipulation instructions. The International Journal of Robotics Research 35(1-3):281–300. [find](https://scholar.google.com/scholar?q=Dipendra+K%2E+Misra%2C+Jaeyong+Sung%2C+Kevin+Lee%2C+and+Ashutosh+Saxena%2E+2016%2E+Tell+me+dave%3A+Context%2Dsensitive+grounding+of+natural+language+to+manipulation+instructions%2E+The+International+Journal+of+Robotics)
38. Kumar Dipendra Misra, Kejia Tao, Percy Liang, and Ashutosh Saxena. 2015. Environment-driven lexicon induction for high-level instructions. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Process… [find](https://scholar.google.com/scholar?q=Kumar+Dipendra+Misra%2C+Kejia+Tao%2C+Percy+Liang%2C+and+Ashutosh+Saxena%2E+2015%2E+Environment%2Ddriven+lexicon+induction+for+high%2Dlevel+instructions%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+the+Association+)
39. Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In Proceedings of the International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Adri%C3%A0+Puigdom%C3%A8nech+Badia%2C+Mehdi+Mirza%2C+Alex+Graves%2C+Timothy+P%2E+Lillicrap%2C+Tim+Harley%2C+David+Silver%2C+and+Koray+Kavukcuoglu%2E+2016%2E+Asynchronous+methods+for+deep+reinforcement+learning%2E+I)
40. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin A. Riedmiller. 2013. Playing atari with deep reinforcement learning. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Koray+Kavukcuoglu%2C+David+Silver%2C+Alex+Graves%2C+Ioannis+Antonoglou%2C+Daan+Wierstra%2C+and+Martin+A%2E+Riedmiller%2E+2013%2E+Playing+atari+with+deep+reinforcement+learning%2E+In+Advances+in+Neural+I)
41. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, and Georg Ostrovski. 2015. Human-level control through deep reinforcement learning. Nature 518(7540). [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Koray+Kavukcuoglu%2C+David+Silver%2C+Andrei+A+Rusu%2C+Joel+Veness%2C+Marc+G+Bellemare%2C+Alex+Graves%2C+Martin+Riedmiller%2C+Andreas+K+Fidjeland%2C+and+Georg+Ostrovski%2E+2015%2E+Human%2Dlevel+control+throu)
42. Karthik Narasimhan, Tejas Kulkarni, and Regina Barzilay. 2015. [Language understanding for text-based games using deep reinforcement learning](https://md.research.elamur.ai/free/en/papers/narasimhan-2015-language-understanding-text-based-games/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/D15-1001. [↗](https://doi.org/10.18653/v1/D15-1001)
43. Karthik Narasimhan, Adam Yala, and Regina Barzilay. 2016. Improving information extraction by acquiring external evidence with reinforcement learning. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. [http://aclweb.org/anthology/D16-1261](http://aclweb.org/anthology/D16-1261). [↗](http://aclweb.org/anthology/D16-1261)
44. Andrew Y. Ng, Daishi Harada, and Stuart J. Russell. 1999. Policy invariance under reward transformations: Theory and application to reward shaping. In Proceedings of the International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Andrew+Y%2E+Ng%2C+Daishi+Harada%2C+and+Stuart+J%2E+Russell%2E+1999%2E+Policy+invariance+under+reward+transformations%3A+Theory+and+application+to+reward+shaping%2E+In+Proceedings+of+the+International+Conference+on+Ma)
45. Junhyuk Oh, Valliappa Chockalingam, Satinder P. Singh, and Honglak Lee. 2016. Control of memory, active perception, and action in minecraft. In Proceedings of the International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Junhyuk+Oh%2C+Valliappa+Chockalingam%2C+Satinder+P%2E+Singh%2C+and+Honglak+Lee%2E+2016%2E+Control+of+memory%2C+active+perception%2C+and+action+in+minecraft%2E+In+Proceedings+of+the+International+Conference+on+Machine+L)
46. Rohan Paul, Jacob Arkin, Nicholas Roy, and Thomas M. Howard. 2016. Efficient grounding of abstract spatial concepts for natural language interaction with robot manipulators. In Robotics: Science and Systems. [find](https://scholar.google.com/scholar?q=Rohan+Paul%2C+Jacob+Arkin%2C+Nicholas+Roy%2C+and+Thomas+M%2E+Howard%2E+2016%2E+Efficient+grounding+of+abstract+spatial+concepts+for+natural+language+interaction+with+robot+manipulators%2E+In+Robotics%3A+Science+and+S)
47. Andrei A. Rusu, Matej Vecerik, Thomas Rothörl, Nicolas Heess, Razvan Pascanu, and Raia Hadsell. 2016. Sim-to-real robot learning from pixels with progressive nets. CoRR . [find](https://scholar.google.com/scholar?q=Andrei+A%2E+Rusu%2C+Matej+Vecerik%2C+Thomas+Roth%C3%B6rl%2C+Nicolas+Heess%2C+Razvan+Pascanu%2C+and+Raia+Hadsell%2E+2016%2E+Sim%2Dto%2Dreal+robot+learning+from+pixels+with+progressive+nets%2E+CoRR+%2E)
48. John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, and Pieter Abbeel. 2015. Trust region policy optimization . [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Sergey+Levine%2C+Philipp+Moritz%2C+Michael+I%2E+Jordan%2C+and+Pieter+Abbeel%2E+2015%2E+Trust+region+policy+optimization+%2E)
49. David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcu… [find](https://scholar.google.com/scholar?q=David+Silver%2C+Aja+Huang%2C+Chris+J+Maddison%2C+Arthur+Guez%2C+Laurent+Sifre%2C+George+van+den+Driessche%2C+Julian+Schrittwieser%2C+Ioannis+Antonoglou%2C+Veda+Panneershelvam%2C+Marc+Lanctot%2C+Sander+Dieleman%2C+Dominik+G)
50. Alane Suhr, Mike Lewis, James Yeh, and Yoav Artzi. 2017. A corpus of compositional language for visual reasoning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alane+Suhr%2C+Mike+Lewis%2C+James+Yeh%2C+and+Yoav+Artzi%2E+2017%2E+A+corpus+of+compositional+language+for+visual+reasoning%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computational+Linguis)
51. Jaeyong Sung, Seok Hyun Jin, and Ashutosh Saxena. 2015. Robobarista: Object part based transfer of manipulation trajectories from crowd-sourcing in 3d pointclouds. In International Symposium on Robotics Research. [find](https://scholar.google.com/scholar?q=Jaeyong+Sung%2C+Seok+Hyun+Jin%2C+and+Ashutosh+Saxena%2E+2015%2E+Robobarista%3A+Object+part+based+transfer+of+manipulation+trajectories+from+crowd%2Dsourcing+in+3d+pointclouds%2E+In+International+Symposium+on+Roboti)
52. Richard S. Sutton and Andrew G. Barto. 1998. Reinforcement learning: An introduction. IEEE Trans. Neural Networks 9:1054–1054. [find](https://scholar.google.com/scholar?q=Richard+S%2E+Sutton+and+Andrew+G%2E+Barto%2E+1998%2E+Reinforcement+learning%3A+An+introduction%2E+IEEE+Trans%2E+Neural+Networks+9%3A1054%E2%80%931054%2E)
53. Richard S. Sutton, David A. McAllester, Satinder P. Singh, and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Richard+S%2E+Sutton%2C+David+A%2E+McAllester%2C+Satinder+P%2E+Singh%2C+and+Yishay+Mansour%2E+1999%2E+Policy+gradient+methods+for+reinforcement+learning+with+function+approximation%2E+In+Advances+in+Neural+Information+P)
54. Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew Walter, Ashis G. Banerjee, Seth Teller, and Nicholas Roy. 2011. Understanding natural language commands for robotic navigation and mobile manipulation. In Proceedings of the National Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Stefanie+Tellex%2C+Thomas+Kollar%2C+Steven+Dickerson%2C+Matthew+Walter%2C+Ashis+G%2E+Banerjee%2C+Seth+Teller%2C+and+Nicholas+Roy%2E+2011%2E+Understanding+natural+language+commands+for+robotic+navigation+and+mobile+mani)
55. Adam Vogel and Daniel Jurafsky. 2010. Learning to follow navigational directions. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. [http://aclweb.org/anthology/P10-1083](http://aclweb.org/anthology/P10-1083). [↗](http://aclweb.org/anthology/P10-1083)
56. Bonnie Webber, Norman Badler, Barbara Di Eugenio, Christopher Geib, Libby Levison, and Michael Moore. 1995. Instructions, intentions and expectations. Artificial Intelligence 73(1):253–269. [find](https://scholar.google.com/scholar?q=Bonnie+Webber%2C+Norman+Badler%2C+Barbara+Di+Eugenio%2C+Christopher+Geib%2C+Libby+Levison%2C+and+Michael+Moore%2E+1995%2E+Instructions%2C+intentions+and+expectations%2E+Artificial+Intelligence+73%281%29%3A253%E2%80%93269%2E)
57. Eric Wiewiora, Garrison W. Cottrell, and Charles Elkan. 2003. Principled methods for advising reinforcement learning agents. In Proceedings of the International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Eric+Wiewiora%2C+Garrison+W%2E+Cottrell%2C+and+Charles+Elkan%2E+2003%2E+Principled+methods+for+advising+reinforcement+learning+agents%2E+In+Proceedings+of+the+International+Conference+on+Machine+Learning%2E)
58. Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8. [find](https://scholar.google.com/scholar?q=Ronald+J%2E+Williams%2E+1992%2E+Simple+statistical+gradient%2Dfollowing+algorithms+for+connectionist+reinforcement+learning%2E+Machine+Learning+8%2E)
59. Ronald J Williams and Jing Peng. 1991. Function optimization using connectionist reinforcement learning algorithms. Connection Science 3(3):241–268. [find](https://scholar.google.com/scholar?q=Ronald+J+Williams+and+Jing+Peng%2E+1991%2E+Function+optimization+using+connectionist+reinforcement+learning+algorithms%2E+Connection+Science+3%283%29%3A241%E2%80%93268%2E)
60. Terry Winograd. 1972. Understanding natural language. Cognitive Psychology 3(1):1–191. [find](https://scholar.google.com/scholar?q=Terry+Winograd%2E+1972%2E+Understanding+natural+language%2E+Cognitive+Psychology+3%281%29%3A1%E2%80%93191%2E)
61. Kelvin Xu, Jimmy Ba, Jamie Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. 2015. [Show, attend and tell: Neural image caption generation with visual attention](https://md.research.elamur.ai/free/en/papers/xu-2015-show-attend-tell-neural-image/). In Proceedings of the International Conference on Machine Learning.
62. Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J. Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi. 2017. Target-driven visual navigation in indoor scenes using deep reinforcement learning. [find](https://scholar.google.com/scholar?q=Yuke+Zhu%2C+Roozbeh+Mottaghi%2C+Eric+Kolve%2C+Joseph+J%2E+Lim%2C+Abhinav+Gupta%2C+Li+Fei%2DFei%2C+and+Ali+Farhadi%2E+2017%2E+Target%2Ddriven+visual+navigation+in+indoor+scenes+using+deep+reinforcement+learning%2E)

*Updated 2 November 2026*