---
title: "A Survey of Reinforcement Learning Informed by Natural Language"
description: "Standard reinforcement learning (RL) agents learn from scratch. 85 references, cited by 43 papers here."
authors: ["Jelena Luketina", "Nantas Nardelli", "Gregory Farquhar", "Jakob Foerster", "Jacob Andreas", "Edward Grefenstette", "Shimon Whiteson", "Tim Rocktäschel"]
year: 2019
venue: "arXiv:1906.03926"
source: "https://arxiv.org/abs/1906.03926"
doi: "10.48550/arxiv.1906.03926"
arxiv: "1906.03926"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/luketina-2019-survey-reinforcement-learning-informed-natural/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/luketina-2019-survey-reinforcement-learning-informed-natural/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / A Survey of Reinforcement Learning Informed by Natural Language

# A Survey of Reinforcement Learning Informed by Natural Language

**Authors:** [Jelena Luketina](https://md.research.elamur.ai/free/en/authors/jelena-luketina/), Nantas Nardelli, Gregory Farquhar, [Jakob Foerster](https://md.research.elamur.ai/free/en/authors/jakob-foerster/), [Jacob Andreas](https://md.research.elamur.ai/free/en/authors/jacob-andreas/), [Edward Grefenstette](https://md.research.elamur.ai/free/en/authors/edward-grefenstette/), Shimon Whiteson and [Tim Rocktäschel](https://md.research.elamur.ai/free/en/authors/tim-rocktaschel/) · **Year:** 2019 · **Venue:** [arXiv:1906.03926](https://md.research.elamur.ai/free/en/venues/international-joint-conference-on-artificial-intelligence/) ([International Joint Conference on Artificial Intelligence 2019](https://md.research.elamur.ai/free/en/venues/international-joint-conference-on-artificial-intelligence/2019/)) · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1906.03926) · [PDF](https://arxiv.org/pdf/1906.03926) · [DOI 10.48550/arxiv.1906.03926](https://doi.org/10.48550/arxiv.1906.03926) · [arXiv 1906.03926](https://arxiv.org/abs/1906.03926) · [OpenAlex](https://openalex.org/W2948380112)

**Citations:** 188 citations (OpenAlex, 2026-10-14) · [43](https://md.research.elamur.ai/free/en/papers/luketina-2019-survey-reinforcement-learning-informed-natural/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Standard reinforcement learning (RL) agents learn from scratch. They need huge numbers of environment interactions and transfer poorly to even slightly altered tasks. This survey, by authors from Oxford, Facebook AI Research, MIT and UCL, asks whether natural language can supply the priors and structure these agents lack. It reviews existing work and organizes it along two axes. The first is whether language is built into the task itself (language-conditional RL: instruction following, rewards induced from text, text games) or only optional support for learning (language-assisted RL: transfer from domain texts, language as a policy representation). The second is whether the linguistic information is task-dependent, such as manuals and instructions, or task-independent, such as pre-trained representations. The authors find that most studies rely on small corpora or synthetic, template-generated language in simple closed worlds. They argue that recent advances in pre-trained language representations make a more ambitious agenda worthwhile. Concretely, they call for new environments with richer real-world semantics and for wider use of NLP tools in RL.

## Key points
- **Taxonomy:** the survey sorts work into language-conditional versus language-assisted RL, and task-dependent versus task-independent language. The categories can overlap, for example when extra corpora help an instruction-following agent.
- **Motivation:** deep RL systems often need millions to billions of interactions. Text sources such as wikis or game manuals (cf. Branavan et al., 2012; Narasimhan et al., 2018) could convey dynamics, rewards and strategies more cheaply.
- **Main limitation of current work:** language is mostly synthetic, with small vocabularies and simple templates. As a result, it lacks real-language properties like the Zipfian power-law word distribution and its long tail of rare words.
- **Proposed directions:**
  - encode task text using pre-trained language models, such as those of Radford et al. (2019), Devlin et al. (2018) and Peters et al. (2018a);
  - couple agents with machine reading and retrieval systems;
  - let a language model complete plan-like prompts.
- **Suggested testbeds:** extended 3D house simulators with templated instructions, and open-world games such as Minecraft. Long-term applications include household robots, virtual assistants and autonomous vehicles.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 214 works · 241,492 citations
- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 243 works · 76,506 citations
- [University of Oxford](https://md.research.elamur.ai/free/en/universities/university-of-oxford/) — University, Oxford, United Kingdom · 122 works · 160,277 citations
- [University College London](https://md.research.elamur.ai/free/en/universities/university-college-london/) — University, London, United Kingdom · 107 works · 22,924 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson and Tim Rocktäschel. 2019. A Survey of Reinforcement Learning Informed by Natural Language. arXiv:1906.03926.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{luketina2019survey,
  title = {{A Survey of Reinforcement Learning Informed by Natural Language}},
  author = {Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson and Tim Rocktäschel},
  year = {2019},
  doi = {10.48550/arxiv.1906.03926},
  eprint = {1906.03926},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1906.03926}
}
```

- [← previous AlBERTo: Italian BERT Language Understanding Model for NLP Challenging Tasks Based on Tweets](https://md.research.elamur.ai/free/en/papers/polignano-2019-alberto-italian-bert-language-understanding/)
- [next → Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation](https://md.research.elamur.ai/free/en/papers/moryossef-2019-step-step-separating-planning-realization/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

85 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rishabh Agarwal, Chen Liang, Dale Schuurmans, and Mohammad Norouzi. Learning to Generalize from Sparse and Underspecified Rewards. arXiv:1902.07198 [cs, stat], 2019. — [arXiv:1902.07198](https://arxiv.org/abs/1902.07198) [find](https://scholar.google.com/scholar?q=Rishabh+Agarwal%2C+Chen+Liang%2C+Dale+Schuurmans%2C+and+Mohammad+Norouzi%2E+Learning+to+Generalize+from+Sparse+and+Underspecified+Rewards%2E+arXiv%3A1902%2E07198+%5Bcs%2C+stat%5D%2C+2019%2E+%E2%80%94+%5BarXiv%3A1902%2E07198%5D%28https%3A%2F%2Farxiv)
2. Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. [Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments](https://md.research.elamur.ai/free/en/papers/anderson-2017-vision-language-navigation-interpreting-visually/). In CVPR, 2018.
3. Jacob Andreas and Dan Klein. Alignment-based compositional semantics for instruction following. ACL, 2015. [find](https://scholar.google.com/scholar?q=Jacob+Andreas+and+Dan+Klein%2E+Alignment%2Dbased+compositional+semantics+for+instruction+following%2E+ACL%2C+2015%2E)
4. Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. [Neural module networks](https://md.research.elamur.ai/free/en/papers/andreas-2015-neural-module-networks/). In CVPR, 2016.
5. Jacob Andreas, Dan Klein, and Sergey Levine. Modular Multitask Reinforcement Learning with Policy Sketches. In ICML, 2017. [find](https://scholar.google.com/scholar?q=Jacob+Andreas%2C+Dan+Klein%2C+and+Sergey+Levine%2E+Modular+Multitask+Reinforcement+Learning+with+Policy+Sketches%2E+In+ICML%2C+2017%2E)
6. Jacob Andreas, Dan Klein, and Sergey Levine. Learning with latent language. In NAACL-HLT, 2018. [find](https://scholar.google.com/scholar?q=Jacob+Andreas%2C+Dan+Klein%2C+and+Sergey+Levine%2E+Learning+with+latent+language%2E+In+NAACL%2DHLT%2C+2018%2E)
7. Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. VQA: Visual question answering. In ICCV, 2015. [find](https://scholar.google.com/scholar?q=Stanislaw+Antol%2C+Aishwarya+Agrawal%2C+Jiasen+Lu%2C+Margaret+Mitchell%2C+Dhruv+Batra%2C+C+Lawrence+Zitnick%2C+and+Devi+Parikh%2E+VQA%3A+Visual+question+answering%2E+In+ICCV%2C+2015%2E)
8. Yoav Artzi and Luke Zettlemoyer. [Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions](https://md.research.elamur.ai/free/en/papers/artzi-2013-weakly-supervised-learning-semantic-parsers/). In ACL, 2013.
9. Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. A Brief Survey of Deep Reinforcement Learning. IEEE Signal Proc. Magazine, 2017. [find](https://scholar.google.com/scholar?q=Kai+Arulkumaran%2C+Marc+Peter+Deisenroth%2C+Miles+Brundage%2C+and+Anil+Anthony+Bharath%2E+A+Brief+Survey+of+Deep+Reinforcement+Learning%2E+IEEE+Signal+Proc%2E+Magazine%2C+2017%2E)
10. Monica Babes, Vukosi Marivate, Kaushik Subramanian, and Michael L Littman. Apprenticeship learning about multiple intentions. In ICML, 2011. [find](https://scholar.google.com/scholar?q=Monica+Babes%2C+Vukosi+Marivate%2C+Kaushik+Subramanian%2C+and+Michael+L+Littman%2E+Apprenticeship+learning+about+multiple+intentions%2E+In+ICML%2C+2011%2E)
11. Dzmitry Bahdanau, Felix Hill, Jan Leike, Edward Hughes, Arian Hosseini, Pushmeet Kohli, and Edward Grefenstette. Learning to Understand Goal Specifications by Modelling Reward. In ICLR, 2019. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Felix+Hill%2C+Jan+Leike%2C+Edward+Hughes%2C+Arian+Hosseini%2C+Pushmeet+Kohli%2C+and+Edward+Grefenstette%2E+Learning+to+Understand+Goal+Specifications+by+Modelling+Reward%2E+In+ICLR%2C+2019%2E)
12. Michele Banko, Michael J Cafarella, Stephen Soderland, Matthew Broadhead, and Oren Etzioni. [Open information extraction from the web](https://md.research.elamur.ai/free/en/papers/banko-2007-open-information-extraction-web/). In IJCAI, 2007.
13. Andrew G Barto and Sridhar Mahadevan. Recent advances in hierarchical reinforcement learning. Discrete event dynamic systems, 13(1-2):41–77, 2003. [find](https://scholar.google.com/scholar?q=Andrew+G+Barto+and+Sridhar+Mahadevan%2E+Recent+advances+in+hierarchical+reinforcement+learning%2E+Discrete+event+dynamic+systems%2C+13%281%2D2%29%3A41%E2%80%9377%2C+2003%2E)
14. Yonatan Bisk, Deniz Yuret, and Daniel Marcu. Natural language communication with robots. In ACL, 2016. [find](https://scholar.google.com/scholar?q=Yonatan+Bisk%2C+Deniz+Yuret%2C+and+Daniel+Marcu%2E+Natural+language+communication+with+robots%2E+In+ACL%2C+2016%2E)
15. Abdelghani Bouziane, Djelloul Bouchiha, Noureddine Doumi, and Mimoun Malki. Question answering systems: survey and trends. Procedia Computer Science, 73:366–375, 2015. [find](https://scholar.google.com/scholar?q=Abdelghani+Bouziane%2C+Djelloul+Bouchiha%2C+Noureddine+Doumi%2C+and+Mimoun+Malki%2E+Question+answering+systems%3A+survey+and+trends%2E+Procedia+Computer+Science%2C+73%3A366%E2%80%93375%2C+2015%2E)
16. S. R. K. Branavan, Luke S Zettlemoyer, and Regina Barzilay. Reading between the lines: Learning to map high-level instructions to commands. In ACL, 2010. [find](https://scholar.google.com/scholar?q=S%2E+R%2E+K%2E+Branavan%2C+Luke+S+Zettlemoyer%2C+and+Regina+Barzilay%2E+Reading+between+the+lines%3A+Learning+to+map+high%2Dlevel+instructions+to+commands%2E+In+ACL%2C+2010%2E)
17. S. R. K. Branavan, David Silver, and Regina Barzilay. Learning to Win by Reading Manuals in a Monte-Carlo Framework. JAIR, 2012. [find](https://scholar.google.com/scholar?q=S%2E+R%2E+K%2E+Branavan%2C+David+Silver%2C+and+Regina+Barzilay%2E+Learning+to+Win+by+Reading+Manuals+in+a+Monte%2DCarlo+Framework%2E+JAIR%2C+2012%2E)
18. Devendra Singh Chaplot, Kanthashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, and Ruslan Salakhutdinov. Gated-Attention Architectures for Task-Oriented Language Grounding. In AAAI, 2018. [find](https://scholar.google.com/scholar?q=Devendra+Singh+Chaplot%2C+Kanthashree+Mysore+Sathyendra%2C+Rama+Kumar+Pasumarthi%2C+Dheeraj+Rajagopal%2C+and+Ruslan+Salakhutdinov%2E+Gated%2DAttention+Architectures+for+Task%2DOriented+Language+Grounding%2E+In+AAAI%2C+)
19. David L Chen and Raymond J Mooney. Learning to interpret natural language navigation instructions from observations. In AAAI, 2011. [find](https://scholar.google.com/scholar?q=David+L+Chen+and+Raymond+J+Mooney%2E+Learning+to+interpret+natural+language+navigation+instructions+from+observations%2E+In+AAAI%2C+2011%2E)
20. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. [Reading wikipedia to answer open-domain questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/). In ACL, 2017.
21. Hongshen Chen, Xiaorui Liu, Dawei Yin, and Jiliang Tang. A survey on dialogue systems: Recent advances and new frontiers. ACM SIGKDD Explorations Newsletter, 2017. [find](https://scholar.google.com/scholar?q=Hongshen+Chen%2C+Xiaorui+Liu%2C+Dawei+Yin%2C+and+Jiliang+Tang%2E+A+survey+on+dialogue+systems%3A+Recent+advances+and+new+frontiers%2E+ACM+SIGKDD+Explorations+Newsletter%2C+2017%2E)
22. Howard Chen, Alane Shur, Dipendra Misra, Noah Snavely, and Yoav Artzi. Touchdown: Natural language navigation and spatial reasoning in visual street environments. arXiv preprint arXiv:1811.12354, 2018. [find](https://scholar.google.com/scholar?q=Howard+Chen%2C+Alane+Shur%2C+Dipendra+Misra%2C+Noah+Snavely%2C+and+Yoav+Artzi%2E+Touchdown%3A+Natural+language+navigation+and+spatial+reasoning+in+visual+street+environments%2E+arXiv+preprint+arXiv%3A1811%2E12354%2C+2018)
23. Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. [BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning](https://md.research.elamur.ai/free/en/papers/chevalier-boisvert-2018-babyai-platform-study-sample-efficiency/). In ICLR, 2019.
24. Marc-Alexandre Côté, Ákos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Matthew Hausknecht, Layla El Asri, Mahmoud Adada, Wendy Tay, and Adam Trischler. [TextWorld: A Learning Environment for Text-based Games](https://md.research.elamur.ai/free/en/papers/cote-2018-textworld-learning-environment-text-based/). arXiv:1806.11532 [cs, stat], 2018.
25. Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Embodied Question Answering. In CVPR, 2018. [find](https://scholar.google.com/scholar?q=Abhishek+Das%2C+Samyak+Datta%2C+Georgia+Gkioxari%2C+Stefan+Lee%2C+Devi+Parikh%2C+and+Dhruv+Batra%2E+Embodied+Question+Answering%2E+In+CVPR%2C+2018%2E)
26. Abhishek Das, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Neural Modular Control for Embodied Question Answering. CoRL, 2018. [find](https://scholar.google.com/scholar?q=Abhishek+Das%2C+Georgia+Gkioxari%2C+Stefan+Lee%2C+Devi+Parikh%2C+and+Dhruv+Batra%2E+Neural+Modular+Control+for+Embodied+Question+Answering%2E+CoRL%2C+2018%2E)
27. Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, and Richard Harshman. Indexing by latent semantic analysis. Journal of the American society for information science, 1990. [find](https://scholar.google.com/scholar?q=Scott+Deerwester%2C+Susan+T+Dumais%2C+George+W+Furnas%2C+Thomas+K+Landauer%2C+and+Richard+Harshman%2E+Indexing+by+latent+semantic+analysis%2E+Journal+of+the+American+society+for+information+science%2C+1990%2E)
28. Mark A DePristo and Robert Zubek. being-in-the-world. In AAAI, 2001. [find](https://scholar.google.com/scholar?q=Mark+A+DePristo+and+Robert+Zubek%2E+being%2Din%2Dthe%2Dworld%2E+In+AAAI%2C+2001%2E)
29. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv:1810.04805 [cs], 2018.
30. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. [Wizard of wikipedia: Knowledge-powered conversational agents](https://md.research.elamur.ai/free/en/papers/dinan-2018-wizard-wikipedia-knowledge-powered-conversational/). CoRR, abs/1811.01241, 2018.
31. Jacob Eisenstein, James Clarke, Dan Goldwasser, and Dan Roth. Reading to learn: constructing features from semantic abstracts. In ACL, 2009. [find](https://scholar.google.com/scholar?q=Jacob+Eisenstein%2C+James+Clarke%2C+Dan+Goldwasser%2C+and+Dan+Roth%2E+Reading+to+learn%3A+constructing+features+from+semantic+abstracts%2E+In+ACL%2C+2009%2E)
32. John R Firth. A synopsis of linguistic theory, 1957. [find](https://scholar.google.com/scholar?q=John+R+Firth%2E+A+synopsis+of+linguistic+theory%2C+1957%2E)
33. Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc Aurelio Ranzato, and Tomas Mikolov. DeViSE: A Deep Visual-Semantic Embedding Model. In NIPS, 2013. [find](https://scholar.google.com/scholar?q=Andrea+Frome%2C+Greg+S+Corrado%2C+Jon+Shlens%2C+Samy+Bengio%2C+Jeff+Dean%2C+Marc+Aurelio+Ranzato%2C+and+Tomas+Mikolov%2E+DeViSE%3A+A+Deep+Visual%2DSemantic+Embedding+Model%2E+In+NIPS%2C+2013%2E)
34. Justin Fu, Anoop Korattikara, Sergey Levine, and Sergio Guadarrama. From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following. In ICLR, 2019. [find](https://scholar.google.com/scholar?q=Justin+Fu%2C+Anoop+Korattikara%2C+Sergey+Levine%2C+and+Sergio+Guadarrama%2E+From+Language+to+Goals%3A+Inverse+Reinforcement+Learning+for+Vision%2DBased+Instruction+Following%2E+In+ICLR%2C+2019%2E)
35. Nancy Fulda, Daniel Ricks, Ben Murdoch, and David Wingate. What can you do with a rock? affordance extraction via word embeddings. arXiv preprint arXiv:1703.03429, 2017. [find](https://scholar.google.com/scholar?q=Nancy+Fulda%2C+Daniel+Ricks%2C+Ben+Murdoch%2C+and+David+Wingate%2E+What+can+you+do+with+a+rock%3F+affordance+extraction+via+word+embeddings%2E+arXiv+preprint+arXiv%3A1703%2E03429%2C+2017%2E)
36. Yoav Goldberg. Assessing BERT’s Syntactic Abilities. CoRR, abs/1901.[05287, 2019](https://md.research.elamur.ai/free/en/papers/goldberg-2019-assessing-bert-s-syntactic-abilities/).
37. Alison Gopnik and Andrew Meltzoff. The development of categorization in the second year and its relation to other cognitive and linguistic developments. Child development, 1987. [find](https://scholar.google.com/scholar?q=Alison+Gopnik+and+Andrew+Meltzoff%2E+The+development+of+categorization+in+the+second+year+and+its+relation+to+other+cognitive+and+linguistic+developments%2E+Child+development%2C+1987%2E)
38. Daniel Gordon, Aniruddha Kembhavi, Mohammad Rastegari, Joseph Redmon, Dieter Fox, and Ali Farhadi. Iqa: Visual question answering in interactive environments. In CVPR, 2018. [find](https://scholar.google.com/scholar?q=Daniel+Gordon%2C+Aniruddha+Kembhavi%2C+Mohammad+Rastegari%2C+Joseph+Redmon%2C+Dieter+Fox%2C+and+Ali+Farhadi%2E+Iqa%3A+Visual+question+answering+in+interactive+environments%2E+In+CVPR%2C+2018%2E)
39. Prasoon Goyal, Scott Niekum, and Raymond J. Mooney. Using Natural Language for Reward Shaping in Reinforcement Learning. IJCAI, 2019. [find](https://scholar.google.com/scholar?q=Prasoon+Goyal%2C+Scott+Niekum%2C+and+Raymond+J%2E+Mooney%2E+Using+Natural+Language+for+Reward+Shaping+in+Reinforcement+Learning%2E+IJCAI%2C+2019%2E)
40. Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis, and Phil Blunsom. [Grounded Language Learning in a Simulated 3d World](https://md.research.elamur.ai/free/en/papers/hermann-2017-grounded-language-learning-simulated-3d/). arXiv:1706.06551 […
41. Jonathan Ho and Stefano Ermon. Generative Adversarial Imitation Learning. In NIPS, 2016. [find](https://scholar.google.com/scholar?q=Jonathan+Ho+and+Stefano+Ermon%2E+Generative+Adversarial+Imitation+Learning%2E+In+NIPS%2C+2016%2E)
42. Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. In ACL, 2018. [find](https://scholar.google.com/scholar?q=Jeremy+Howard+and+Sebastian+Ruder%2E+Universal+language+model+fine%2Dtuning+for+text+classification%2E+In+ACL%2C+2018%2E)
43. Hengyuan Hu, Denis Yarats, Qucheng Gong, Yuandong Tian, and Mike Lewis. Hierarchical decision making by generating and following natural language instructions. arXiv preprint arXiv:1906.00744, 2019. — [arXiv:1906.00744](https://arxiv.org/abs/1906.00744) [find](https://scholar.google.com/scholar?q=Hengyuan+Hu%2C+Denis+Yarats%2C+Qucheng+Gong%2C+Yuandong+Tian%2C+and+Mike+Lewis%2E+Hierarchical+decision+making+by+generating+and+following+natural+language+instructions%2E+arXiv+preprint+arXiv%3A1906%2E00744%2C+2019%2E+%E2%80%94)
44. Michael Janner, Karthik Narasimhan, and Regina Barzilay. [Representation learning for grounded spatial reasoning](https://md.research.elamur.ai/free/en/papers/janner-2018-representation-learning-grounded-spatial-reasoning/). TACL, 2018.
45. Matthew Johnson, Katja Hofmann, Tim Hutton, and David Bignell. The malmo platform for artificial intelligence experimentation. In IJCAI, pages 4246–4247, 2016. [find](https://scholar.google.com/scholar?q=Matthew+Johnson%2C+Katja+Hofmann%2C+Tim+Hutton%2C+and+David+Bignell%2E+The+malmo+platform+for+artificial+intelligence+experimentation%2E+In+IJCAI%2C+pages+4246%E2%80%934247%2C+2016%2E)
46. Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. [Clevr: A diagnostic dataset for compositional language and elementary visual reasoning](https://md.research.elamur.ai/free/en/papers/johnson-2016-clevr-diagnostic-dataset-compositional-language/). In CVPR, 2017.
47. Thomas Kollar, Stefanie Tellex, Deb Roy, and Nicholas Roy. Toward understanding natural language directions. In HRI, 2010. [find](https://scholar.google.com/scholar?q=Thomas+Kollar%2C+Stefanie+Tellex%2C+Deb+Roy%2C+and+Nicholas+Roy%2E+Toward+understanding+natural+language+directions%2E+In+HRI%2C+2010%2E)
48. B. Kostka, J. Kwiecieli, J. Kowalski, and P. Rychlikowski. Text-based adventures of the golovin AI agent. In Conference on Computational Intelligence and Games (CIG), 2017. [find](https://scholar.google.com/scholar?q=B%2E+Kostka%2C+J%2E+Kwiecieli%2C+J%2E+Kowalski%2C+and+P%2E+Rychlikowski%2E+Text%2Dbased+adventures+of+the+golovin+AI+agent%2E+In+Conference+on+Computational+Intelligence+and+Games+%28CIG%29%2C+2017%2E)
49. Gregory Kuhlmann, Peter Stone, Raymond Mooney, and Jude Shavlik. Guiding a Reinforcement Learner with Natural Language Advice: Initial Results in RoboCup Soccer. 2004. [find](https://scholar.google.com/scholar?q=Gregory+Kuhlmann%2C+Peter+Stone%2C+Raymond+Mooney%2C+and+Jude+Shavlik%2E+Guiding+a+Reinforcement+Learner+with+Natural+Language+Advice%3A+Initial+Results+in+RoboCup+Soccer%2E+2004%2E)
50. James MacGlashan, Monica Babes-Vroman, Marie desJardins, Michael L. Littman, Smaranda Muresan, Shawn Squire, Stefanie Tellex, Dilip Arumugam, and Lei Yang. Grounding english commands to reward functions. In Robotics: Science and Systems XI, 2015. [find](https://scholar.google.com/scholar?q=James+MacGlashan%2C+Monica+Babes%2DVroman%2C+Marie+desJardins%2C+Michael+L%2E+Littman%2C+Smaranda+Muresan%2C+Shawn+Squire%2C+Stefanie+Tellex%2C+Dilip+Arumugam%2C+and+Lei+Yang%2E+Grounding+english+commands+to+reward+functio)
51. Matt MacMahon, Brian Stankiewicz, and Benjamin Kuipers. Walk the talk: Connecting language, knowledge, and action in route instructions. In AAAI, 2006. [find](https://scholar.google.com/scholar?q=Matt+MacMahon%2C+Brian+Stankiewicz%2C+and+Benjamin+Kuipers%2E+Walk+the+talk%3A+Connecting+language%2C+knowledge%2C+and+action+in+route+instructions%2E+In+AAAI%2C+2006%2E)
52. Daniela Massiceti, N Siddharth, Puneet K Dokania, and Philip HS Torr. Flipdial: A generative model for two-way visual dialogue. In CVPR, 2018. [find](https://scholar.google.com/scholar?q=Daniela+Massiceti%2C+N+Siddharth%2C+Puneet+K+Dokania%2C+and+Philip+HS+Torr%2E+Flipdial%3A+A+generative+model+for+two%2Dway+visual+dialogue%2E+In+CVPR%2C+2018%2E)
53. Hongyuan Mei, Mohit Bansal, and Matthew R. Walter. Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences. AAAI, 2016. [find](https://scholar.google.com/scholar?q=Hongyuan+Mei%2C+Mohit+Bansal%2C+and+Matthew+R%2E+Walter%2E+Listen%2C+Attend%2C+and+Walk%3A+Neural+Mapping+of+Navigational+Instructions+to+Action+Sequences%2E+AAAI%2C+2016%2E)
54. J. Mey. Pragmatics: An Introduction. Blackwell, 1993. [find](https://scholar.google.com/scholar?q=J%2E+Mey%2E+Pragmatics%3A+An+Introduction%2E+Blackwell%2C+1993%2E)
55. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. [Distributed Representations of Words and Phrases and their Compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). NIPS, 2013.
56. Dipendra Misra, John Langford, and Yoav Artzi. [Mapping Instructions and Visual Observations to Actions with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/misra-2017-mapping-instructions-visual-observations-actions/). EMNLP, 2017.
57. Karthik Narasimhan, Tejas D. Kulkarni, and Regina Barzilay. [Language understanding for text-based games using deep reinforcement learning](https://md.research.elamur.ai/free/en/papers/narasimhan-2015-language-understanding-text-based-games/). In EMNLP, 2015.
58. Karthik Narasimhan, Regina Barzilay, and Tommi Jaakkola. Grounding Language for Transfer in Deep Reinforcement Learning. JAIR, 2018. [find](https://scholar.google.com/scholar?q=Karthik+Narasimhan%2C+Regina+Barzilay%2C+and+Tommi+Jaakkola%2E+Grounding+Language+for+Transfer+in+Deep+Reinforcement+Learning%2E+JAIR%2C+2018%2E)
59. Junhyuk Oh, Satinder P. Singh, Honglak Lee, and Pushmeet Kohli. Zero-shot task generalization with multi-task deep reinforcement learning. In ICML, 2017. [find](https://scholar.google.com/scholar?q=Junhyuk+Oh%2C+Satinder+P%2E+Singh%2C+Honglak+Lee%2C+and+Pushmeet+Kohli%2E+Zero%2Dshot+task+generalization+with+multi%2Dtask+deep+reinforcement+learning%2E+In+ICML%2C+2017%2E)
60. Takayuki Osa, Joni Pajarinen, Gerhard Neumann, J Andrew Bagnell, Pieter Abbeel, Jan Peters, et al. An algorithmic perspective on imitation learning. Foundations and Trends® in Robotics, 7(1-2):1–179, 2018. [find](https://scholar.google.com/scholar?q=Takayuki+Osa%2C+Joni+Pajarinen%2C+Gerhard+Neumann%2C+J+Andrew+Bagnell%2C+Pieter+Abbeel%2C+Jan+Peters%2C+et+al%2E+An+algorithmic+perspective+on+imitation+learning%2E+Foundations+and+Trends%C2%AE+in+Robotics%2C+7%281%2D2%29%3A1%E2%80%93179%2C+)
61. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In NAACL, 2018.
62. Matthew E. Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. [Dissecting contextual word embeddings: Architecture and representation](https://md.research.elamur.ai/free/en/papers/peters-2018-dissecting-contextual-word-embeddings-architecture/). In EMNLP, 2018.
63. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+Language+models+are+unsupervised+multitask+learners%2E+2019%2E)
64. Tianmin Shu, Caiming Xiong, and Richard Socher. Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning. ICLR, 2018. [find](https://scholar.google.com/scholar?q=Tianmin+Shu%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+Hierarchical+and+Interpretable+Skill+Acquisition+in+Multi%2Dtask+Reinforcement+Learning%2E+ICLR%2C+2018%2E)
65. Anna Shusterman, Sang Ah Lee, and Elizabeth Spelke. Cognitive effects of language on human navigation. Cognition, 2011. [find](https://scholar.google.com/scholar?q=Anna+Shusterman%2C+Sang+Ah+Lee%2C+and+Elizabeth+Spelke%2E+Cognitive+effects+of+language+on+human+navigation%2E+Cognition%2C+2011%2E)
66. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. Mastering the game of Go withou… [find](https://scholar.google.com/scholar?q=David+Silver%2C+Julian+Schrittwieser%2C+Karen+Simonyan%2C+Ioannis+Antonoglou%2C+Aja+Huang%2C+Arthur+Guez%2C+Thomas+Hubert%2C+Lucas+Baker%2C+Matthew+Lai%2C+Adrian+Bolton%2C+Yutian+Chen%2C+Timothy+Lillicrap%2C+Fan+Hui%2C+Laurent)
67. Satinder Singh, Diane Litman, Michael Kearns, and Marilyn Walker. Optimizing dialogue management with reinforcement learning: Experiments with the njfun system. JAIR, 2002. [find](https://scholar.google.com/scholar?q=Satinder+Singh%2C+Diane+Litman%2C+Michael+Kearns%2C+and+Marilyn+Walker%2E+Optimizing+dialogue+management+with+reinforcement+learning%3A+Experiments+with+the+njfun+system%2E+JAIR%2C+2002%2E)
68. Richard Socher, Milind Ganjoo, Christopher D. Manning, and Andrew Y. Ng. Zero-shot Learning Through Cross-modal Transfer. In NIPS, 2013. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Milind+Ganjoo%2C+Christopher+D%2E+Manning%2C+and+Andrew+Y%2E+Ng%2E+Zero%2Dshot+Learning+Through+Cross%2Dmodal+Transfer%2E+In+NIPS%2C+2013%2E)
69. Elizabeth Spelke and Katherine D Kinzler. Core knowledge. Developmental science, 2007. [find](https://scholar.google.com/scholar?q=Elizabeth+Spelke+and+Katherine+D+Kinzler%2E+Core+knowledge%2E+Developmental+science%2C+2007%2E)
70. Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018. [find](https://scholar.google.com/scholar?q=Richard+S+Sutton+and+Andrew+G+Barto%2E+Reinforcement+learning%3A+An+introduction%2E+MIT+press%2C+2018%2E)
71. Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew R Walter, Ashis Gopal Banerjee, Seth Teller, and Nicholas Roy. Understanding natural language commands for robotic navigation and mobile manipulation. In AAAI, 2011. [find](https://scholar.google.com/scholar?q=Stefanie+Tellex%2C+Thomas+Kollar%2C+Steven+Dickerson%2C+Matthew+R+Walter%2C+Ashis+Gopal+Banerjee%2C+Seth+Teller%2C+and+Nicholas+Roy%2E+Understanding+natural+language+commands+for+robotic+navigation+and+mobile+manip)
72. Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick. [What do you learn from context? probing for sentence structure in contextualized word representations](https://md.research.elamur.ai/free/en/papers/tenney-2019-learn-context-probing-sentence-structure/). In ICLR, 2019.
73. Gerald Tesauro. Temporal Difference Learning and TD-Gammon. Communications of the ACM, 1995. [find](https://scholar.google.com/scholar?q=Gerald+Tesauro%2E+Temporal+Difference+Learning+and+TD%2DGammon%2E+Communications+of+the+ACM%2C+1995%2E)
74. Yuandong Tian, Qucheng Gong, Wenling Shang, Yuxin Wu, and C. Lawrence Zitnick. ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games. NIPS, 2017. [find](https://scholar.google.com/scholar?q=Yuandong+Tian%2C+Qucheng+Gong%2C+Wenling+Shang%2C+Yuxin+Wu%2C+and+C%2E+Lawrence+Zitnick%2E+ELF%3A+An+Extensive%2C+Lightweight+and+Flexible+Research+Platform+for+Real%2Dtime+Strategy+Games%2E+NIPS%2C+2017%2E)
75. Ruben Rodriguez Torrado, Philip Bontrager, Julian Togelius, Jialin Liu, and Diego Perez-Liebana. Deep reinforcement learning for general video game ai. In CIG. IEEE, 2018. [find](https://scholar.google.com/scholar?q=Ruben+Rodriguez+Torrado%2C+Philip+Bontrager%2C+Julian+Togelius%2C+Jialin+Liu%2C+and+Diego+Perez%2DLiebana%2E+Deep+reinforcement+learning+for+general+video+game+ai%2E+In+CIG%2E+IEEE%2C+2018%2E)
76. Pedro Tsividis, Thomas Pouncy, Jaqueline L. Xu, Joshua B. Tenenbaum, and Samuel J. Gershman. Human learning in atari. In AAAI, 2017. [find](https://scholar.google.com/scholar?q=Pedro+Tsividis%2C+Thomas+Pouncy%2C+Jaqueline+L%2E+Xu%2C+Joshua+B%2E+Tenenbaum%2C+and+Samuel+J%2E+Gershman%2E+Human+learning+in+atari%2E+In+AAAI%2C+2017%2E)
77. Sida I Wang, Percy Liang, and Christopher D Manning. Learning Language Games through Interaction. In ACL, 2016. [find](https://scholar.google.com/scholar?q=Sida+I+Wang%2C+Percy+Liang%2C+and+Christopher+D+Manning%2E+Learning+Language+Games+through+Interaction%2E+In+ACL%2C+2016%2E)
78. Xin Wang, Qiuyuan Huang, Asli Çelikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang, and Lei Zhang. Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation. In CVPR, 2019. [find](https://scholar.google.com/scholar?q=Xin+Wang%2C+Qiuyuan+Huang%2C+Asli+%C3%87elikyilmaz%2C+Jianfeng+Gao%2C+Dinghan+Shen%2C+Yuan%2DFang+Wang%2C+William+Yang+Wang%2C+and+Lei+Zhang%2E+Reinforced+cross%2Dmodal+matching+and+self%2Dsupervised+imitation+learning+for+visi)
79. David Ashley White and Donald A Sofge. Handbook of Intelligent Control: Neural, Fuzzy, and Adaptative Approaches. Van Nostrand Reinhold Company, 1992. [find](https://scholar.google.com/scholar?q=David+Ashley+White+and+Donald+A+Sofge%2E+Handbook+of+Intelligent+Control%3A+Neural%2C+Fuzzy%2C+and+Adaptative+Approaches%2E+Van+Nostrand+Reinhold+Company%2C+1992%2E)
80. Claudia Yan, Dipendra Misra, Andrew Bennnett, Aaron Walsman, Yonatan Bisk, and Yoav Artzi. Chalet: Cornell house agent learning environment. arXiv preprint arXiv:1801.07357, 2018. — [arXiv:1801.07357](https://arxiv.org/abs/1801.07357) [find](https://scholar.google.com/scholar?q=Claudia+Yan%2C+Dipendra+Misra%2C+Andrew+Bennnett%2C+Aaron+Walsman%2C+Yonatan+Bisk%2C+and+Yoav+Artzi%2E+Chalet%3A+Cornell+house+agent+learning+environment%2E+arXiv+preprint+arXiv%3A1801%2E07357%2C+2018%2E+%E2%80%94+%5BarXiv%3A1801%2E07357%5D)
81. Haonan Yu, Haichao Zhang, and Wei Xu. Interactive Grounded Language Acquisition and Generalization in a 2d World. ICLR, 2018. [find](https://scholar.google.com/scholar?q=Haonan+Yu%2C+Haichao+Zhang%2C+and+Wei+Xu%2E+Interactive+Grounded+Language+Acquisition+and+Generalization+in+a+2d+World%2E+ICLR%2C+2018%2E)
82. Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, and Adam Trischler. Counting to Explore and Generalize in Text-based Games. arXiv:1806.11525 [cs], 2018. — [arXiv:1806.11525](https://arxiv.org/abs/1806.11525) [find](https://scholar.google.com/scholar?q=Xingdi+Yuan%2C+Marc%2DAlexandre+C%C3%B4t%C3%A9%2C+Alessandro+Sordoni%2C+Romain+Laroche%2C+Remi+Tachet+des+Combes%2C+Matthew+Hausknecht%2C+and+Adam+Trischler%2E+Counting+to+Explore+and+Generalize+in+Text%2Dbased+Games%2E+arXiv%3A1806)
83. Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. SWAG: A large-scale adversarial dataset for grounded commonsense inference. In EMNLP, 2018. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Yonatan+Bisk%2C+Roy+Schwartz%2C+and+Yejin+Choi%2E+SWAG%3A+A+large%2Dscale+adversarial+dataset+for+grounded+commonsense+inference%2E+In+EMNLP%2C+2018%2E)
84. Brian D Ziebart, Andrew Maas, J Andrew Bagnell, and Anind K Dey. Maximum Entropy Inverse Reinforcement Learning. In AAAI, 2008. [find](https://scholar.google.com/scholar?q=Brian+D+Ziebart%2C+Andrew+Maas%2C+J+Andrew+Bagnell%2C+and+Anind+K+Dey%2E+Maximum+Entropy+Inverse+Reinforcement+Learning%2E+In+AAAI%2C+2008%2E)
85. George Kingsley Zipf. Human behavior and the principle of least effort. 1949. [find](https://scholar.google.com/scholar?q=George+Kingsley+Zipf%2E+Human+behavior+and+the+principle+of+least+effort%2E+1949%2E)

*Updated 2 November 2026*