---
title: "BabyAI — paper by Chevalier-Boisvert et al. (2018)"
description: "The authors release BabyAI, a platform for measuring this cost. 47 references, cited by 60 papers here."
authors: ["Maxime Chevalier-Boisvert", "Dzmitry Bahdanau", "Salem Lahlou", "Lucas Willems", "Chitwan Saharia", "Thien Huu Nguyen", "Yoshua Bengio"]
year: 2018
venue: "arXiv:1810.08272"
source: "https://arxiv.org/abs/1810.08272"
doi: "10.48550/arxiv.1810.08272"
arxiv: "1810.08272"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chevalier-boisvert-2018-babyai-platform-study-sample-efficiency/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chevalier-boisvert-2018-babyai-platform-study-sample-efficiency/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/) / BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning

# BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning

**Authors:** Maxime Chevalier-Boisvert, [Dzmitry Bahdanau](https://md.research.elamur.ai/free/en/authors/dzmitry-bahdanau/), [Salem Lahlou](https://md.research.elamur.ai/free/en/authors/salem-lahlou/), Lucas Willems, [Chitwan Saharia](https://md.research.elamur.ai/free/en/authors/chitwan-saharia/), [Thien Huu Nguyen](https://md.research.elamur.ai/free/en/authors/thien-huu-nguyen/) and [Yoshua Bengio](https://md.research.elamur.ai/free/en/authors/yoshua-bengio/) · **Year:** 2018 · **Venue:** arXiv:1810.08272 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1810.08272) · [PDF](https://arxiv.org/pdf/1810.08272) · [DOI 10.48550/arxiv.1810.08272](https://doi.org/10.48550/arxiv.1810.08272) · [arXiv 1810.08272](https://arxiv.org/abs/1810.08272) · [OpenAlex](https://openalex.org/W2907923173)

**Citations:** 63 citations (OpenAlex, 2026-10-14) · [60](https://md.research.elamur.ai/free/en/papers/chevalier-boisvert-2018-babyai-platform-study-sample-efficiency/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Training agents to follow language instructions with a human in the loop could make AI assistants easier to customize. It would also connect machine learning research with work on how children acquire language. The obstacle is cost: deep imitation and reinforcement learning methods need far more interaction than a person could reasonably supply.

The authors release BabyAI, a platform for measuring this cost. It is built on MiniGrid, a fast, partially observable 2D gridworld with keys, doors, balls and boxes. Tasks are written in the Baby Language, a synthetic subset of English defined by a grammar. A curriculum of 19 levels raises both environment and language difficulty, and a scripted bot plays the role of a human teacher.

Using imitation and reinforcement learning baselines, the authors estimate how many demonstrations or episodes are needed to reach near-perfect performance on several levels. They also test pretraining and interactive imitation learning. Their central finding is that current deep learning approaches are not sample-efficient enough to learn a compositional language. BabyAI provides a controlled benchmark for tracking progress toward practical human-in-the-loop training.

## Key points
- BabyAI contains 19 levels of increasing difficulty, ranging from simple tasks such as GoToObj up to the BossLevel. The code and pretrained models are released at github.com/mila-iqia/babyai.
- The Baby Language is defined by a BNF context-free grammar and allows about 2.48 × 10^19 distinct instructions. These cover going to objects, picking them up, opening doors and placing objects next to others, and can be chained with "then" or "after you".
- In MiniGrid, the agent sees an egocentric 7x7 window encoded as a 7x7x3 tensor and chooses among seven discrete actions. Rewards are sparse, with a bonus for finishing quickly. The simulator exceeds 3000 frames per second on a laptop and integrates with OpenAI Gym.
- The bot breaks each instruction into a stack of subgoals (GoNextTo, Open, Explore, Pickup, Drop). It can produce demonstrations on demand and advise the learner, which supports interactive methods like DAGGER.
- Unlike earlier 3D or template-based environments, BabyAI combines world-state manipulation, partial observability, systematic language semantics, and a simulated teacher.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Université de Montréal](https://md.research.elamur.ai/free/en/universities/universite-de-montreal/) — University, Montreal, Canada · 87 works · 91,597 citations
- [University of Oregon](https://md.research.elamur.ai/free/en/universities/university-of-oregon/) — University, Eugene, United States · 26 works · 3,555 citations
- [Indian Institute of Technology Bombay](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-bombay/) — University, Mumbai, India · 32 works · 2,535 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen and Yoshua Bengio. 2018. BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning. arXiv:1810.08272.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{chevalierboisvert2018babyai,
  title = {{BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning}},
  author = {Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen and Yoshua Bengio},
  year = {2018},
  doi = {10.48550/arxiv.1810.08272},
  eprint = {1810.08272},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1810.08272}
}
```

- [← previous A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/muller-2018-large-scale-test-set-evaluation/)
- [next → Online normalizer calculation for softmax](https://md.research.elamur.ai/free/en/papers/milakov-2018-online-normalizer-calculation-softmax/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/)

## References

47 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sünderhauf, N., Reid, I., Gould, S., and Hengel, A. v. d. (2018). Vision-and-Language Navigation: Interpreting visually-grounded naviga-IL from RL Expert 1.53 - 2.11 36.6 - 44.5 74.2 - 81.8 [find](https://scholar.google.com/scholar?q=Anderson%2C+P%2E%2C+Wu%2C+Q%2E%2C+Teney%2C+D%2E%2C+Bruce%2C+J%2E%2C+Johnson%2C+M%2E%2C+S%C3%BCnderhauf%2C+N%2E%2C+Reid%2C+I%2E%2C+Gould%2C+S%2E%2C+and+Hengel%2C+A%2E+v%2E+d%2E+%282018%29%2E+Vision%2Dand%2DLanguage+Navigation%3A+Interpreting+visually%2Dgrounded+naviga%2DIL+from)
2. Interactive IL from Bot 1.71 - 1.88 31.8 - 36 93 - 107 tion instructions in real environments. In Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). [find](https://scholar.google.com/scholar?q=Interactive+IL+from+Bot+1%2E71+%2D+1%2E88+31%2E8+%2D+36+93+%2D+107+tion+instructions+in+real+environments%2E+In+Proceedings+of+the+2018+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2E)
3. Andreas, J., Rohrbach, M., Darrell, T., and Klein, D. (2016). [Neural Module Networks](https://md.research.elamur.ai/free/en/papers/andreas-2015-neural-module-networks/). In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
4. Artzi, Y. and Zettlemoyer, L. (2013). [Weakly supervised learning of semantic parsers for mapping instructions to actions](https://md.research.elamur.ai/free/en/papers/artzi-2013-weakly-supervised-learning-semantic-parsers/). Transactions of the Association for Computational Linguistics, 1:49–62.
5. Bahdanau, D., Cho, K., and Bengio, Y. (2015). [Neural Machine Translation by Jointly Learning to Align and Translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). In Proceedings of the 2015 International Conference on Learning Representations.
6. Bahdanau, D., Hill, F., Leike, J., Hughes, E., Hosseini, A., Kohli, P., and Grefenstette, E. (2018). Learning to Understand Goal Specifications by Modelling Reward. In ICLR. [find](https://scholar.google.com/scholar?q=Bahdanau%2C+D%2E%2C+Hill%2C+F%2E%2C+Leike%2C+J%2E%2C+Hughes%2C+E%2E%2C+Hosseini%2C+A%2E%2C+Kohli%2C+P%2E%2C+and+Grefenstette%2C+E%2E+%282018%29%2E+Learning+to+Understand+Goal+Specifications+by+Modelling+Reward%2E+In+ICLR%2E)
7. Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M. (2013). [The Arcade Learning Environment: An Evaluation Platform for General Agents](https://md.research.elamur.ai/free/en/papers/bellemare-2012-arcade-learning-environment-evaluation-platform/). Journal of Artificial Intelligence Research, 47:253–279. arXiv: 1207.4708.
8. Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009). [Curriculum learning](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/). In Proceedings of the 26th annual international conference on machine learning, pages 41–48.
9. Chaplot, D. S., Sathyendra, K. M., Pasumarthi, R. K., Rajagopal, D., and Salakhutdinov, R. (2018). Gated-Attention Architectures for Task-Oriented Language Grounding. In Proceedings of 32nd AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Chaplot%2C+D%2E+S%2E%2C+Sathyendra%2C+K%2E+M%2E%2C+Pasumarthi%2C+R%2E+K%2E%2C+Rajagopal%2C+D%2E%2C+and+Salakhutdinov%2C+R%2E+%282018%29%2E+Gated%2DAttention+Architectures+for+Task%2DOriented+Language+Grounding%2E+In+Proceedings+of+32nd+AAAI+Confe)
10. Chen, D. L. and Mooney, R. J. (2011). Learning to Interpret Natural Language Navigation Instructions from Observations. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, pages 859–865. [find](https://scholar.google.com/scholar?q=Chen%2C+D%2E+L%2E+and+Mooney%2C+R%2E+J%2E+%282011%29%2E+Learning+to+Interpret+Natural+Language+Navigation+Instructions+from+Observations%2E+In+Proceedings+of+the+Twenty%2DFifth+AAAI+Conference+on+Artificial+Intelligence%2C+p)
11. Cho, K., van Merrienboer, B., Gulcehre, C., Bougares, F., Schwenk, H., and Bengio, Y. (2014). [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/cho-2014-learning-phrase-representations-rnn-encoder/). In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).
12. Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., and Amodei, D. (2017). Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems 30. arXiv: 1706.03741. [find](https://scholar.google.com/scholar?q=Christiano%2C+P%2E%2C+Leike%2C+J%2E%2C+Brown%2C+T%2E+B%2E%2C+Martic%2C+M%2E%2C+Legg%2C+S%2E%2C+and+Amodei%2C+D%2E+%282017%29%2E+Deep+reinforcement+learning+from+human+preferences%2E+In+Advances+in+Neural+Information+Processing+Systems+30%2E+arXiv)
13. Daumé Iii, H., Langford, J., and Marcu, D. (2009). Search-based structured prediction. Machine learning, 75(3):297–325. [find](https://scholar.google.com/scholar?q=Daum%C3%A9+Iii%2C+H%2E%2C+Langford%2C+J%2E%2C+and+Marcu%2C+D%2E+%282009%29%2E+Search%2Dbased+structured+prediction%2E+Machine+learning%2C+75%283%29%3A297%E2%80%93325%2E)
14. DeepMind (2017). PycoLab. [find](https://scholar.google.com/scholar?q=DeepMind+%282017%29%2E+PycoLab%2E)
15. Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., Legg, S., and Kavukcuoglu, K. (2018). IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures. In Proceedings of the 22nd international conference o… [find](https://scholar.google.com/scholar?q=Espeholt%2C+L%2E%2C+Soyer%2C+H%2E%2C+Munos%2C+R%2E%2C+Simonyan%2C+K%2E%2C+Mnih%2C+V%2E%2C+Ward%2C+T%2E%2C+Doron%2C+Y%2E%2C+Firoiu%2C+V%2E%2C+Harley%2C+T%2E%2C+Dunning%2C+I%2E%2C+Legg%2C+S%2E%2C+and+Kavukcuoglu%2C+K%2E+%282018%29%2E+IMPALA%3A+Scalable+Distributed+Deep%2DRL+with+Im)
16. Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., Badia, A. P., Hermann, K. M., Zwols, Y., Ostrovski, G., Cain, A., King, H., Summerfield, C., Blunsom, P., Kavukcuoglu, K., and Hassabis, D. (2016).… [find](https://scholar.google.com/scholar?q=Graves%2C+A%2E%2C+Wayne%2C+G%2E%2C+Reynolds%2C+M%2E%2C+Harley%2C+T%2E%2C+Danihelka%2C+I%2E%2C+Grabska%2DBarwin%CC%81ska%2C+A%2E%2C+Colmenarejo%2C+S%2E+G%2E%2C+Grefenstette%2C+E%2E%2C+Ramalho%2C+T%2E%2C+Agapiou%2C+J%2E%2C+Badia%2C+A%2E+P%2E%2C+Hermann%2C+K%2E+M%2E%2C+Zwols%2C+Y%2E%2C+Ostrovs)
17. Hermann, K. M., Hill, F., Green, S., Wang, F., Faulkner, R., Soyer, H., Szepesvari, D., Czarnecki, W. M., Jaderberg, M., Teplyashin, D., Wainwright, M., Apps, C., Hassabis, D., and Blunsom, P. (2017). [Grounded Language Learning in a Simulated 3d World](https://md.research.elamur.ai/free/en/papers/hermann-2017-grounded-language-learning-simulated-3d/). arXiv:1706.06551 [cs, stat].
18. Hochreiter, S. and Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Hochreiter%2C+S%2E+and+Schmidhuber%2C+J%2E+%281997%29%2E+Long+Short%2DTerm+Memory%2E+Neural+Computation%2C+9%288%29%3A1735%E2%80%931780%2E)
19. Ioffe, S. and Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, pages 448–456. [find](https://scholar.google.com/scholar?q=Ioffe%2C+S%2E+and+Szegedy%2C+C%2E+%282015%29%2E+Batch+Normalization%3A+Accelerating+Deep+Network+Training+by+Reducing+Internal+Covariate+Shift%2E+In+Proceedings+of+the+32nd+International+Conference+on+Machine+Learning%2C)
20. Kempka, M., Wydmuch, M., Runc, G., Toczek, J., and Jaskowski, W. (2016). ViZDoom: A Doom-based AI research platform for visual reinforcement learning. In CIG, pages 1–8. IEEE. [find](https://scholar.google.com/scholar?q=Kempka%2C+M%2E%2C+Wydmuch%2C+M%2E%2C+Runc%2C+G%2E%2C+Toczek%2C+J%2E%2C+and+Jaskowski%2C+W%2E+%282016%29%2E+ViZDoom%3A+A+Doom%2Dbased+AI+research+platform+for+visual+reinforcement+learning%2E+In+CIG%2C+pages+1%E2%80%938%2E+IEEE%2E)
21. Kingma, D. P. and Ba, J. (2015). [Adam: A Method for Stochastic Optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In Proceedings of the 2015 International Conference on Learning Representations. arXiv: 1412.6980.
22. Koenig, N. and Howard, A. (2004). Design and Use Paradigms for Gazebo, An Open-Source Multi-Robot Simulator. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2149–2154, Sendai, Japan. [find](https://scholar.google.com/scholar?q=Koenig%2C+N%2E+and+Howard%2C+A%2E+%282004%29%2E+Design+and+Use+Paradigms+for+Gazebo%2C+An+Open%2DSource+Multi%2DRobot+Simulator%2E+In+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems%2C+pages+2149%E2%80%932154%2C+Se)
23. Kolve, E., Mottaghi, R., Gordon, D., Zhu, Y., Gupta, A., and Farhadi, A. (2017). [AI2-THOR: An Interactive 3d Environment for Visual AI](https://md.research.elamur.ai/free/en/papers/kolve-2017-ai2-thor-interactive-3d-environment/). CoRR, abs/1712.05474.
24. Kumar, M. P., Packer, B., and Koller, D. (2010). Self-Paced Learning for Latent Variable Models. In Advances in Neural Information Processing Systems 23, pages 1189–1197. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Kumar%2C+M%2E+P%2E%2C+Packer%2C+B%2E%2C+and+Koller%2C+D%2E+%282010%29%2E+Self%2DPaced+Learning+for+Latent+Variable+Models%2E+In+Advances+in+Neural+Information+Processing+Systems+23%2C+pages+1189%E2%80%931197%2E+Curran+Associates%2C+Inc%2E)
25. Macmahon, M., Stankiewicz, B., and Kuipers, B. (2006). Walk the Talk: Connecting Language, Knowledge, Action in Route Instructions. In In Proc. of the Nat. Conf. on Artificial Intelligence (AAAI, pages 1475–1482. [find](https://scholar.google.com/scholar?q=Macmahon%2C+M%2E%2C+Stankiewicz%2C+B%2E%2C+and+Kuipers%2C+B%2E+%282006%29%2E+Walk+the+Talk%3A+Connecting+Language%2C+Knowledge%2C+Action+in+Route+Instructions%2E+In+In+Proc%2E+of+the+Nat%2E+Conf%2E+on+Artificial+Intelligence+%28AAAI%2C+page)
26. Mei, H., Bansal, M., and Walter, M. R. (2016). Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences. In Proceedings of the 2016 AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Mei%2C+H%2E%2C+Bansal%2C+M%2E%2C+and+Walter%2C+M%2E+R%2E+%282016%29%2E+Listen%2C+Attend%2C+and+Walk%3A+Neural+Mapping+of+Navigational+Instructions+to+Action+Sequences%2E+In+Proceedings+of+the+2016+AAAI+Conference+on+Artificial+Intel)
27. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: machine learning in Python. Journal of Machine Learning… [find](https://scholar.google.com/scholar?q=Pedregosa%2C+F%2E%2C+Varoquaux%2C+G%2E%2C+Gramfort%2C+A%2E%2C+Michel%2C+V%2E%2C+Thirion%2C+B%2E%2C+Grisel%2C+O%2E%2C+Blondel%2C+M%2E%2C+Prettenhofer%2C+P%2E%2C+Weiss%2C+R%2E%2C+Dubourg%2C+V%2E%2C+Vanderplas%2C+J%2E%2C+Passos%2C+A%2E%2C+Cournapeau%2C+D%2E%2C+Brucher%2C+M%2E%2C+Perrot%2C)
28. Perez, E., Strub, F., de Vries, H., Dumoulin, V., and Courville, A. (2017). [FiLM: Visual Reasoning with a General Conditioning Layer](https://md.research.elamur.ai/free/en/papers/perez-2017-film-visual-reasoning-general-conditioning/). In In Proceedings of the 2017 AAAI Conference on Artificial Intelligence.
29. Rasmussen, C. E. and Williams, C. K. I. (2005). Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). [find](https://scholar.google.com/scholar?q=Rasmussen%2C+C%2E+E%2E+and+Williams%2C+C%2E+K%2E+I%2E+%282005%29%2E+Gaussian+Processes+for+Machine+Learning+%28Adaptive+Computation+and+Machine+Learning%29%2E)
30. Reed, S. and de Freitas, N. (2015). [find](https://scholar.google.com/scholar?q=Reed%2C+S%2E+and+de+Freitas%2C+N%2E+%282015%29%2E)
31. Neural Programmer-Interpreters. In 2016 International Conference on Learning Representations. arXiv: 1511.06279. [find](https://scholar.google.com/scholar?q=Neural+Programmer%2DInterpreters%2E+In+2016+International+Conference+on+Learning+Representations%2E+arXiv%3A+1511%2E06279%2E)
32. Ross, S., Gordon, G., and Bagnell, D. (2011). A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. In PMLR, pages 627–635. [find](https://scholar.google.com/scholar?q=Ross%2C+S%2E%2C+Gordon%2C+G%2E%2C+and+Bagnell%2C+D%2E+%282011%29%2E+A+Reduction+of+Imitation+Learning+and+Structured+Prediction+to+No%2DRegret+Online+Learning%2E+In+PMLR%2C+pages+627%E2%80%93635%2E)
33. Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. (2015). High-Dimensional Continuous Control Using Generalized Advantage Estimation. In Advances in Neural Information Processing Systems 30. [find](https://scholar.google.com/scholar?q=Schulman%2C+J%2E%2C+Moritz%2C+P%2E%2C+Levine%2C+S%2E%2C+Jordan%2C+M%2E%2C+and+Abbeel%2C+P%2E+%282015%29%2E+High%2DDimensional+Continuous+Control+Using+Generalized+Advantage+Estimation%2E+In+Advances+in+Neural+Information+Processing+System)
34. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv:1707.06347 [cs]. arXiv: 1707.06347. [find](https://scholar.google.com/scholar?q=Schulman%2C+J%2E%2C+Wolski%2C+F%2E%2C+Dhariwal%2C+P%2E%2C+Radford%2C+A%2E%2C+and+Klimov%2C+O%2E+%282017%29%2E+Proximal+Policy+Optimization+Algorithms%2E+arXiv%3A1707%2E06347+%5Bcs%5D%2E+arXiv%3A+1707%2E06347%2E)
35. Sukhbaatar, S., Szlam, A., Synnaeve, G., Chintala, S., and Fergus, R. (2015). MazeBase: A Sandbox for Learning from Games. arXiv:1511.07401 [cs]. arXiv: 1511.07401. [find](https://scholar.google.com/scholar?q=Sukhbaatar%2C+S%2E%2C+Szlam%2C+A%2E%2C+Synnaeve%2C+G%2E%2C+Chintala%2C+S%2E%2C+and+Fergus%2C+R%2E+%282015%29%2E+MazeBase%3A+A+Sandbox+for+Learning+from+Games%2E+arXiv%3A1511%2E07401+%5Bcs%5D%2E+arXiv%3A+1511%2E07401%2E)
36. Sutskever, I., Vinyals, O., and Le, Q. V. (2014). [Sequence to Sequence Learning with Neural Networks](https://md.research.elamur.ai/free/en/papers/sutskever-2014-sequence-sequence-learning-neural-networks/). In Advances in Neural Information Processing Systems 27, pages 3104–3112.
37. Tellex, S., Kollar, T., Dickerson, S., Walter, M. R., Banerjee, A. G., Teller, S., and Roy, N. (2011). Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation. In Twenty-Fifth AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Tellex%2C+S%2E%2C+Kollar%2C+T%2E%2C+Dickerson%2C+S%2E%2C+Walter%2C+M%2E+R%2E%2C+Banerjee%2C+A%2E+G%2E%2C+Teller%2C+S%2E%2C+and+Roy%2C+N%2E+%282011%29%2E+Understanding+Natural+Language+Commands+for+Robotic+Navigation+and+Mobile+Manipulation%2E+In+Twenty)
38. Wang, S. I., Liang, P., and Manning, C. D. (2016). Learning Language Games through Interaction. In Proceedings Of the 54th Annual Meeting of the Association for Computational Linguistics. arXiv: 1606.02447. [find](https://scholar.google.com/scholar?q=Wang%2C+S%2E+I%2E%2C+Liang%2C+P%2E%2C+and+Manning%2C+C%2E+D%2E+%282016%29%2E+Learning+Language+Games+through+Interaction%2E+In+Proceedings+Of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2E+arXiv%3A+1606%2E)
39. Warnell, G., Waytowich, N., Lawhern, V., and Stone, P. (2017). Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces. In Proceedings of 32nd AAAI Conference on Artificial Intelligence. arXiv: 1709.10163. — [arXiv:1709.10163](https://arxiv.org/abs/1709.10163) [find](https://scholar.google.com/scholar?q=Warnell%2C+G%2E%2C+Waytowich%2C+N%2E%2C+Lawhern%2C+V%2E%2C+and+Stone%2C+P%2E+%282017%29%2E+Deep+TAMER%3A+Interactive+Agent+Shaping+in+High%2DDimensional+State+Spaces%2E+In+Proceedings+of+32nd+AAAI+Conference+on+Artificial+Intelligence)
40. Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A., and Mikolov, T. (2016). [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://md.research.elamur.ai/free/en/papers/weston-2015-towards-ai-complete-question-answering/).
41. Williams, E. C., Gopalan, N., Rhee, M., and Tellex, S. (2018). Learning to Parse Natural Language to Grounded Reward Functions with Weak Supervision. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, May 21-25, 2018, pages 1–7. [find](https://scholar.google.com/scholar?q=Williams%2C+E%2E+C%2E%2C+Gopalan%2C+N%2E%2C+Rhee%2C+M%2E%2C+and+Tellex%2C+S%2E+%282018%29%2E+Learning+to+Parse+Natural+Language+to+Grounded+Reward+Functions+with+Weak+Supervision%2E+In+2018+IEEE+International+Conference+on+Robotics+)
42. Wilson, A., Fern, A., and Tadepalli, P. (2012). A Bayesian Approach for Policy Learning from Trajectory Preference Queries. In Pereira, F., Burges, C. J. C., Bottou, L., and Weinberger, K. Q., editors, Advances in Neural Information Processing Systems 25, pages 1133–1141. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Wilson%2C+A%2E%2C+Fern%2C+A%2E%2C+and+Tadepalli%2C+P%2E+%282012%29%2E+A+Bayesian+Approach+for+Policy+Learning+from+Trajectory+Preference+Queries%2E+In+Pereira%2C+F%2E%2C+Burges%2C+C%2E+J%2E+C%2E%2C+Bottou%2C+L%2E%2C+and+Weinberger%2C+K%2E+Q%2E%2C+editors)
43. Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., and others (2016). Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv preprint arXiv:1609.08144. [find](https://scholar.google.com/scholar?q=Wu%2C+Y%2E%2C+Schuster%2C+M%2E%2C+Chen%2C+Z%2E%2C+Le%2C+Q%2E+V%2E%2C+Norouzi%2C+M%2E%2C+Macherey%2C+W%2E%2C+Krikun%2C+M%2E%2C+Cao%2C+Y%2E%2C+Gao%2C+Q%2E%2C+Macherey%2C+K%2E%2C+and+others+%282016%29%2E+Google%E2%80%99s+Neural+Machine+Translation+System%3A+Bridging+the+Gap+betwee)
44. Wu, Y., Wu, Y., Gkioxari, G., and Tian, Y. (2018). Building Generalizable Agents with a Realistic and Rich 3d Environment. arXiv:1801.02209 [cs]. arXiv: 1801.02209. — [arXiv:1801.02209](https://arxiv.org/abs/1801.02209) [find](https://scholar.google.com/scholar?q=Wu%2C+Y%2E%2C+Wu%2C+Y%2E%2C+Gkioxari%2C+G%2E%2C+and+Tian%2C+Y%2E+%282018%29%2E+Building+Generalizable+Agents+with+a+Realistic+and+Rich+3d+Environment%2E+arXiv%3A1801%2E02209+%5Bcs%5D%2E+arXiv%3A+1801%2E02209%2E+%E2%80%94+%5BarXiv%3A1801%2E02209%5D%28https%3A%2F%2Farxiv%2E)
45. Yu, H., Zhang, H., and Xu, W. (2018). Interactive Grounded Language Acquisition and Generalization in 2d Environment. In ICLR. [find](https://scholar.google.com/scholar?q=Yu%2C+H%2E%2C+Zhang%2C+H%2E%2C+and+Xu%2C+W%2E+%282018%29%2E+Interactive+Grounded+Language+Acquisition+and+Generalization+in+2d+Environment%2E+In+ICLR%2E)
46. Zaremba, W. and Sutskever, I. (2015). Learning to Execute. In 2015 International Conference on Learning Representations. arXiv: 1410.4615. [find](https://scholar.google.com/scholar?q=Zaremba%2C+W%2E+and+Sutskever%2C+I%2E+%282015%29%2E+Learning+to+Execute%2E+In+2015+International+Conference+on+Learning+Representations%2E+arXiv%3A+1410%2E4615%2E)
47. Ziebart, B. D., Maas, A., Bagnell, J. A., and Dey, A. K. (2008). Maximum Entropy Inverse Reinforcement Learning. In Proc. AAAI, pages 1433–1438. [find](https://scholar.google.com/scholar?q=Ziebart%2C+B%2E+D%2E%2C+Maas%2C+A%2E%2C+Bagnell%2C+J%2E+A%2E%2C+and+Dey%2C+A%2E+K%2E+%282008%29%2E+Maximum+Entropy+Inverse+Reinforcement+Learning%2E+In+Proc%2E+AAAI%2C+pages+1433%E2%80%931438%2E)

*Updated 2 November 2026*