---
title: "TextWorld: A Learning Environment for Text-based — Cote 2018"
description: "Text adventure games such as Zork are a hard testbed for reinforcement learning."
authors: ["Marc-Alexandre Cote", "Akos Kadar", "Xingdi Yuan", "Ben Kybartas", "Tavian Barnes", "Emery Fine", "James Moore", "Ruo Yu Tao", "Matthew Hausknecht", "Layla El Asri", "Mahmoud Adada", "Wendy Tay", "Adam Trischler"]
year: 2018
venue: "arXiv:1806.11532"
source: "https://arxiv.org/abs/1806.11532"
doi: "10.48550/arxiv.1806.11532"
arxiv: "1806.11532"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/cote-2018-textworld-learning-environment-text-based/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/cote-2018-textworld-learning-environment-text-based/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/) / TextWorld: A Learning Environment for Text-based Games

# TextWorld: A Learning Environment for Text-based Games

**Authors:** [Marc-Alexandre Cote](https://md.research.elamur.ai/free/en/authors/marc-alexandre-cote/), Akos Kadar, [Xingdi Yuan](https://md.research.elamur.ai/free/en/authors/xingdi-yuan/), Ben Kybartas, Tavian Barnes, [Emery Fine](https://md.research.elamur.ai/free/en/authors/emery-fine/), James Moore, Ruo Yu Tao, [Matthew Hausknecht](https://md.research.elamur.ai/free/en/authors/matthew-hausknecht/), [Layla El Asri](https://md.research.elamur.ai/free/en/authors/layla-el-asri/), Mahmoud Adada, Wendy Tay et al. (13 authors) · **Year:** 2018 · **Venue:** arXiv:1806.11532 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1806.11532) · [PDF](https://arxiv.org/pdf/1806.11532) · [DOI 10.48550/arxiv.1806.11532](https://doi.org/10.48550/arxiv.1806.11532) · [arXiv 1806.11532](https://arxiv.org/abs/1806.11532) · [OpenAlex](https://openalex.org/W2810346659)

**Citations:** 209 citations (openalex-snapshot · 2026-10-14) · [122](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Text adventure games such as Zork are a hard testbed for reinforcement learning. Both the observations and the actions are free-form, compositional language, the world is only partly visible, and rewards are rare. Hand-written games of this kind remain out of reach for current agents. The authors present TextWorld, a Python sandbox that runs text games and handles state tracking and reward assignment. Beyond replaying a curated, analyzed set of existing games, it can build new ones. A logic engine assembles maps, objects, obstacles and quests, and a context-free grammar with synonym-rich vocabulary renders the state as prose. Because the full game state is known, the framework supports exact tracking, optional intermediate rewards and tunable difficulty. These controls enable studies of curriculum learning, generalization and transfer. The paper frames text games as POMDPs and releases a generated "treasure hunter" benchmark. Two existing agents, BYU and Golovin, nearly fail on unseen generated games, which shows that fresh, controllable games are needed to measure real progress.

## Key points
- Commands are restricted to at most L words from a fixed vocabulary V, with the pattern verb [noun phrase [adverb phrase]] and no determiners. Generation is word by word and ends with a `<return>` token.
- The treasure hunter benchmark has 30 levels in three tiers:
  - easy: 5 rooms, quest length 1–5;
  - medium: 10 rooms, quest length 2–10, closed doors and containers;
  - hard: 20 rooms, quest length 3–20, locked doors and containers.
- The one-life evaluation used 100 new games per level, each capped at 1000 steps. Picking up the correct object scored +1, the wrong one −1, and finding nothing 0.
- BYU and Golovin scored about 0.75–0.78 on level 1 but stayed near 0 on harder levels. A random agent choosing from admissible commands scored roughly 0.2–0.4, but the authors caution that its results cannot be fairly set side by side with those of BYU and Golovin.
- Stated limitations:
  - quests are linear chains;
  - there are no time-triggered events, NPCs or multi-user play;
  - CFG-generated text has weak continuity between sentences.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

122 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 112 not yet.

- [Experience Grounds Language](https://md.research.elamur.ai/free/en/papers/bisk-2020-experience-grounds-language/)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/)
- [AgentBench: Evaluating LLMs as Agents](https://md.research.elamur.ai/free/en/papers/liu-2023-agentbench-evaluating-llms-agents/)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://md.research.elamur.ai/free/en/papers/reynolds-2021-prompt-programming-large-language-models/)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://md.research.elamur.ai/free/en/papers/shridhar-2020-alfworld-aligning-text-embodied-environments/)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://md.research.elamur.ai/free/en/papers/wang-2023-decodingtrust-comprehensive-assessment-trustworthiness-gpt/)
- [MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback](https://md.research.elamur.ai/free/en/papers/wang-2023-mint-evaluating-llms-multi-turn/)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://md.research.elamur.ai/free/en/papers/xi-2023-rise-potential-large-language/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{cote2018textworld,
  title = {{TextWorld: A Learning Environment for Text-based Games}},
  author = {Cote, Marc-Alexandre and Kadar, Akos and Yuan, Xingdi and Kybartas, Ben and Barnes, Tavian and Fine, Emery and Moore, James and Tao, Ruo Yu and Hausknecht, Matthew and Asri, Layla El and Adada, Mahmoud and Tay, Wendy and Trischler, Adam},
  year = {2018},
  doi = {10.48550/arxiv.1806.11532},
  eprint = {1806.11532},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1806.11532}
}
```

- [← previous Textual Commentary on the Ethiopic Text of Ezekiel 1–11](https://md.research.elamur.ai/free/en/papers/knibb-2018-textual-commentary-ethiopic-text-ezekiel/)
- [next → Texygen: A Benchmarking Platform for Text Generation Models](https://md.research.elamur.ai/free/en/papers/zhu-2018-texygen-benchmarking-platform-text-generation/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/)

## References

40 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Douglas Adams and Steve Meretzky. The hitchhiker’s guide to the galaxy, 1984. URL [http://ifdb.tads.org/viewgame?id=ouv80gvsl32xlion](http://ifdb.tads.org/viewgame?id=ouv80gvsl32xlion). [↗](http://ifdb.tads.org/viewgame?id=ouv80gvsl32xlion)
2. Timothy Atkinson, Hendrik Baier, Tara Copplestone, Sam Devlin, and Jerry Swan. The text-based adventure ai competition. arXiv preprint arXiv:1808.01262, 2018. — [arXiv:1808.01262](https://arxiv.org/abs/1808.01262) [find](https://scholar.google.com/scholar?q=Timothy+Atkinson%2C+Hendrik+Baier%2C+Tara+Copplestone%2C+Sam+Devlin%2C+and+Jerry+Swan%2E+The+text%2Dbased+adventure+ai+competition%2E+arXiv+preprint+arXiv%3A1808%2E01262%2C+2018%2E+%E2%80%94+%5BarXiv%3A1808%2E01262%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
3. Marco Baroni, Armand Joulin, Allan Jabri, Germàn Kruszewski, Angeliki Lazaridou, Klemen Simonic, and Tomas Mikolov. Commai: Evaluating the first steps towards a useful general ai. arXiv preprint arXiv:1701.08954, 2017. [find](https://scholar.google.com/scholar?q=Marco+Baroni%2C+Armand+Joulin%2C+Allan+Jabri%2C+Germ%C3%A0n+Kruszewski%2C+Angeliki+Lazaridou%2C+Klemen+Simonic%2C+and+Tomas+Mikolov%2E+Commai%3A+Evaluating+the+first+steps+towards+a+useful+general+ai%2E+arXiv+preprint+arXiv)
4. Bob Bates. Sherlock: The riddle of the crown jewels, 1987. URL [http://ifdb.tads.org/viewgame?id=j8lmspy4iz73mx26](http://ifdb.tads.org/viewgame?id=j8lmspy4iz73mx26). [↗](http://ifdb.tads.org/viewgame?id=j8lmspy4iz73mx26)
5. Charles Beattie, Joel Z Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Víctor Valdés, Amir Sadik, et al. Deepmind lab. arXiv preprint arXiv:1612.03801, 2016. [find](https://scholar.google.com/scholar?q=Charles+Beattie%2C+Joel+Z+Leibo%2C+Denis+Teplyashin%2C+Tom+Ward%2C+Marcus+Wainwright%2C+Heinrich+K%C3%BCttler%2C+Andrew+Lefrancq%2C+Simon+Green%2C+V%C3%ADctor+Vald%C3%A9s%2C+Amir+Sadik%2C+et+al%2E+Deepmind+lab%2E+arXiv+preprint+arXiv%3A1612%2E)
6. Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. J. Artif. Intell. Res.(JAIR), 47:253–279, 2013. [find](https://scholar.google.com/scholar?q=Marc+G+Bellemare%2C+Yavar+Naddaf%2C+Joel+Veness%2C+and+Michael+Bowling%2E+The+arcade+learning+environment%3A+An+evaluation+platform+for+general+agents%2E+J%2E+Artif%2E+Intell%2E+Res%2E%28JAIR%29%2C+47%3A253%E2%80%93279%2C+2013%2E)
7. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pages 41–48. ACM, 2009. [find](https://scholar.google.com/scholar?q=Yoshua+Bengio%2C+J%C3%A9r%C3%B4me+Louradour%2C+Ronan+Collobert%2C+and+Jason+Weston%2E+Curriculum+learning%2E+In+Proceedings+of+the+26th+annual+international+conference+on+machine+learning%2C+pages+41%E2%80%9348%2E+ACM%2C+2009%2E) [our card](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/)
8. Michael Berlyn. Infidel, anu79a4n1jedg5mm. Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016. [find](https://scholar.google.com/scholar?q=Michael+Berlyn%2E+Infidel%2C+anu79a4n1jedg5mm%2E+Greg+Brockman%2C+Vicki+Cheung%2C+Ludwig+Pettersson%2C+Jonas+Schneider%2C+John+Schulman%2C+Jie+Tang%2C+and+Wojciech+Zaremba%2E+Openai+gym%2E+arXiv+preprint+arXiv%3A1606%2E01540%2C+)
9. Devendra Singh Chaplot, Kanthashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, and Ruslan Salakhutdinov. Gated-attention architectures for task-oriented language grounding. arXiv preprint arXiv:1706.07230, 2017. — [arXiv:1706.07230](https://arxiv.org/abs/1706.07230) [find](https://scholar.google.com/scholar?q=Devendra+Singh+Chaplot%2C+Kanthashree+Mysore+Sathyendra%2C+Rama+Kumar+Pasumarthi%2C+Dheeraj+Rajagopal%2C+and+Ruslan+Salakhutdinov%2E+Gated%2Dattention+architectures+for+task%2Doriented+language+grounding%2E+arXiv+pre)
10. N. Chomsky. Three models for the description of language. IEEE Transactions on Information Theory, 2(3):113–124, sep 1956. doi: 10.1109/tit.1956.1056813. [find](https://scholar.google.com/scholar?q=N%2E+Chomsky%2E+Three+models+for+the+description+of+language%2E+IEEE+Transactions+on+Information+Theory%2C+2%283%29%3A113%E2%80%93124%2C+sep+1956%2E+doi%3A+10%2E1109%2Ftit%2E1956%2E1056813%2E)
11. William Crowther and Donald Woods. viewgame?id=fft6pu91j85y4acv. Brian Desilets. The enterprise incidents, 2002. URL [http://ifdb.tads.org/viewgame?id=ld1f3t5epeagilfz](http://ifdb.tads.org/viewgame?id=ld1f3t5epeagilfz). [↗](http://ifdb.tads.org/viewgame?id=ld1f3t5epeagilfz)
12. Richard E Fikes and Nils J Nilsson. Strips: A new approach to the application of theorem proving to problem solving. Artificial intelligence, 2(3-4):189–208, 1971. [find](https://scholar.google.com/scholar?q=Richard+E+Fikes+and+Nils+J+Nilsson%2E+Strips%3A+A+new+approach+to+the+application+of+theorem+proving+to+problem+solving%2E+Artificial+intelligence%2C+2%283%2D4%29%3A189%E2%80%93208%2C+1971%2E)
13. Nancy Fulda, Daniel Ricks, Ben Murdoch, and David Wingate. What can you do with a rock? affordance extraction via word embeddings. arXiv preprint arXiv:1703.03429, 2017. [find](https://scholar.google.com/scholar?q=Nancy+Fulda%2C+Daniel+Ricks%2C+Ben+Murdoch%2C+and+David+Wingate%2E+What+can+you+do+with+a+rock%3F+affordance+extraction+via+word+embeddings%2E+arXiv+preprint+arXiv%3A1703%2E03429%2C+2017%2E)
14. Stu Galley and Jim Lawrence. Seastalker, 1984. URL [http://ifdb.tads.org/viewgame?id=56wb8hflec2isvzm](http://ifdb.tads.org/viewgame?id=56wb8hflec2isvzm). [↗](http://ifdb.tads.org/viewgame?id=56wb8hflec2isvzm)
15. Michael Genesereth, Nathaniel Love, and Barney Pell. General game playing: Overview of the aaai competition. AI magazine, 26(2):62, 2005. [find](https://scholar.google.com/scholar?q=Michael+Genesereth%2C+Nathaniel+Love%2C+and+Barney+Pell%2E+General+game+playing%3A+Overview+of+the+aaai+competition%2E+AI+magazine%2C+26%282%29%3A62%2C+2005%2E)
16. Russell Glasser. Reverberations, 1996. dop7nbjl90r5zmf9. [find](https://scholar.google.com/scholar?q=Russell+Glasser%2E+Reverberations%2C+1996%2E+dop7nbjl90r5zmf9%2E)
17. J.J. Guest. Goldilocks is a fox!, 2002. 59ztsy9p01avd6wp. 1983. URL [http://ifdb.tads.org/viewgame?id=](http://ifdb.tads.org/viewgame?id=) [↗](http://ifdb.tads.org/viewgame?id=)
18. URL [http://ifdb.tads.org/1998](http://ifdb.tads.org/1998). URL http://ifdb.tads.org/viewgame?id=URL http://ifdb.tads.org/viewgame?id=URL http://ifdb.tads.org/viewgame?id=Matan Haroush, Tom Zahavy, Daniel J Mankowitz, and Shie Mannor. Learning how not to act in text-based games. International Conference on Learning Representati… [↗](http://ifdb.tads.org/1998)
19. Ji He, Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng, and Mari Ostendorf. Deep reinforcement learning with a natural language action space. arXiv preprint arXiv:1511.04636, 2015. [find](https://scholar.google.com/scholar?q=Ji+He%2C+Jianshu+Chen%2C+Xiaodong+He%2C+Jianfeng+Gao%2C+Lihong+Li%2C+Li+Deng%2C+and+Mari+Ostendorf%2E+Deep+reinforcement+learning+with+a+natural+language+action+space%2E+arXiv+preprint+arXiv%3A1511%2E04636%2C+2015%2E)
20. Matthew Henderson, Blaise Thomson, and Jason D Williams. The second dialog state tracking challenge. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 263–272, 2014. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Jason+D+Williams%2E+The+second+dialog+state+tracking+challenge%2E+In+Proceedings+of+the+15th+Annual+Meeting+of+the+Special+Interest+Group+on+Discourse+and+Dialogue+%28)
21. Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojtek Czarnecki, Max Jaderberg, Denis Teplyashin, et al. Grounded language learning in a simulated 3d world. arXiv preprint arXiv:1706.06551, 2017. [find](https://scholar.google.com/scholar?q=Karl+Moritz+Hermann%2C+Felix+Hill%2C+Simon+Green%2C+Fumin+Wang%2C+Ryan+Faulkner%2C+Hubert+Soyer%2C+David+Szepesvari%2C+Wojtek+Czarnecki%2C+Max+Jaderberg%2C+Denis+Teplyashin%2C+et+al%2E+Grounded+language+learning+in+a+simul)
22. Infocom. Zork I, 1980. URL [http://ifdb.tads.org/viewgame?id=0dbnusxunq7fw5ro](http://ifdb.tads.org/viewgame?id=0dbnusxunq7fw5ro). [↗](http://ifdb.tads.org/viewgame?id=0dbnusxunq7fw5ro)
23. Infocom. Zork III, 1982. URL [http://ifdb.tads.org/viewgame?id=vrsot1zgy1wfcdru](http://ifdb.tads.org/viewgame?id=vrsot1zgy1wfcdru). [↗](http://ifdb.tads.org/viewgame?id=vrsot1zgy1wfcdru)
24. Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2):99–134, 1998. [find](https://scholar.google.com/scholar?q=Leslie+Pack+Kaelbling%2C+Michael+L+Littman%2C+and+Anthony+R+Cassandra%2E+Planning+and+acting+in+partially+observable+stochastic+domains%2E+Artificial+intelligence%2C+101%281%2D2%29%3A99%E2%80%93134%2C+1998%2E)
25. Michał Kempka, Marek Wydmuch, Grzegorz Runc, Jakub Toczek, and Wojciech Jaśkowski. Vizdoom: A doom-based ai research platform for visual reinforcement learning. In Computational Intelligence and Games (CIG), 2016 IEEE Conference on, pages 1–8. [find](https://scholar.google.com/scholar?q=Micha%C5%82+Kempka%2C+Marek+Wydmuch%2C+Grzegorz+Runc%2C+Jakub+Toczek%2C+and+Wojciech+Ja%C5%9Bkowski%2E+Vizdoom%3A+A+doom%2Dbased+ai+research+platform+for+visual+reinforcement+learning%2E+In+Computational+Intelligence+and+Games)
26. Bartosz Kostka, Jaroslaw Kwiecieli, Jakub Kowalski, and Pawel Rychlikowski. Text-based adventures of the golovin ai agent. In Computational Intelligence and Games (CIG), 2017 IEEE Conference on, pages 181–188. [find](https://scholar.google.com/scholar?q=Bartosz+Kostka%2C+Jaroslaw+Kwiecieli%2C+Jakub+Kowalski%2C+and+Pawel+Rychlikowski%2E+Text%2Dbased+adventures+of+the+golovin+ai+agent%2E+In+Computational+Intelligence+and+Games+%28CIG%29%2C+2017+IEEE+Conference+on%2C+pages)
27. Marlos C Machado, Marc G Bellemare, Erik Talvitie, Joel Veness, Matthew Hausknecht, and Michael Bowling. Revisiting the arcade learning environment: Evaluation protocols and open problems for general agents. arXiv preprint arXiv:1709.06009, 2017. [find](https://scholar.google.com/scholar?q=Marlos+C+Machado%2C+Marc+G+Bellemare%2C+Erik+Talvitie%2C+Joel+Veness%2C+Matthew+Hausknecht%2C+and+Michael+Bowling%2E+Revisiting+the+arcade+learning+environment%3A+Evaluation+protocols+and+open+problems+for+general+)
28. Chris Martens. Ceptre: A language for modeling generative interactive systems. In Eleventh Artificial Intelligence and Interactive Digital Entertainment Conference, 2015. [find](https://scholar.google.com/scholar?q=Chris+Martens%2E+Ceptre%3A+A+language+for+modeling+generative+interactive+systems%2E+In+Eleventh+Artificial+Intelligence+and+Interactive+Digital+Entertainment+Conference%2C+2015%2E)
29. Roger McFarlane. A survey of exploration strategies in reinforcement learning. 2003. URL http://www.cs.mcgill.ca/\\begingroup\\let\\relax\\relax\\endgroup[Pleaseinsert\\ PrerenderUnicode{âĹij}intopreamble]cs526/roger.pdf. [find](https://scholar.google.com/scholar?q=Roger+McFarlane%2E+A+survey+of+exploration+strategies+in+reinforcement+learning%2E+2003%2E+URL+http%3A%2F%2Fwww%2Ecs%2Emcgill%2Eca%2F%5C%5Cbegingroup%5C%5Clet%5C%5Crelax%5C%5Crelax%5C%5Cendgroup%5BPleaseinsert%5C%5C+PrerenderUnicode%7B%C3%A2L%CC%81ij%7Dintopre)
30. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529–533, 2015. [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Koray+Kavukcuoglu%2C+David+Silver%2C+Andrei+A+Rusu%2C+Joel+Veness%2C+Marc+G+Bellemare%2C+Alex+Graves%2C+Martin+Riedmiller%2C+Andreas+K+Fidjeland%2C+Georg+Ostrovski%2C+et+al%2E+Human%2Dlevel+control+through+)
31. Brian Moriarty. Wishbringer, 1985. z02joykzh66wfhcl. [find](https://scholar.google.com/scholar?q=Brian+Moriarty%2E+Wishbringer%2C+1985%2E+z02joykzh66wfhcl%2E)
32. Karthik Narasimhan, Tejas Kulkarni, and Regina Barzilay. Language understanding for text-based games using deep reinforcement learning. arXiv preprint arXiv:1506.08941, 2015. [find](https://scholar.google.com/scholar?q=Karthik+Narasimhan%2C+Tejas+Kulkarni%2C+and+Regina+Barzilay%2E+Language+understanding+for+text%2Dbased+games+using+deep+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1506%2E08941%2C+2015%2E)
33. Emilio Parisotto and Ruslan Salakhutdinov. Neural map: Structured memory for deep reinforcement learning. arXiv preprint arXiv:1702.08360, 2017. — [arXiv:1702.08360](https://arxiv.org/abs/1702.08360) [find](https://scholar.google.com/scholar?q=Emilio+Parisotto+and+Ruslan+Salakhutdinov%2E+Neural+map%3A+Structured+memory+for+deep+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1702%2E08360%2C+2017%2E+%E2%80%94+%5BarXiv%3A1702%2E08360%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1702%2E08360%29)
34. Karl Pearson. The problem of the random walk. Nature, 72(1867):342, 1905. [find](https://scholar.google.com/scholar?q=Karl+Pearson%2E+The+problem+of+the+random+walk%2E+Nature%2C+72%281867%29%3A342%2C+1905%2E)
35. Andrew Plotkin. Inhumane, wvs2vmbigm9unlpd. Martin L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., New York, NY, USA, 1st edition, 1994. ISBN 0471619779. [find](https://scholar.google.com/scholar?q=Andrew+Plotkin%2E+Inhumane%2C+wvs2vmbigm9unlpd%2E+Martin+L%2E+Puterman%2E+Markov+Decision+Processes%3A+Discrete+Stochastic+Dynamic+Programming%2E+John+Wiley+%26+Sons%2C+Inc%2E%2C+New+York%2C+NY%2C+USA%2C+1st+edition%2C+1994%2E+ISBN+)
36. Stuart J Russell and Peter Norvig. Artificial intelligence: a modern approach. Malaysia; Pearson Education Limited„ 2016. [find](https://scholar.google.com/scholar?q=Stuart+J+Russell+and+Peter+Norvig%2E+Artificial+intelligence%3A+a+modern+approach%2E+Malaysia%3B+Pearson+Education+Limited%E2%80%9E+2016%2E)
37. URL [http://ifdb.tads.org/viewgame?id=1985](http://ifdb.tads.org/viewgame?id=1985). URL http://ifdb.tads.org/viewgame?id=James Ryan, Ethan Seither, Michael Mateas, and Noah Wardrip-Fruin. Expressionist: An Authoring Tool for In-Game Text Generation, pages 221–233. Springer International Publishing, Cham, 2016. ISBN 978-3-319-48279-8. [↗](http://ifdb.tads.org/viewgame?id=1985)
38. Sainbayar Sukhbaatar, Arthur Szlam, Gabriel Synnaeve, Soumith Chintala, and Rob Fergus. Mazebase: A sandbox for learning from games. arXiv preprint arXiv:1511.07401, 2015. — [arXiv:1511.07401](https://arxiv.org/abs/1511.07401) [find](https://scholar.google.com/scholar?q=Sainbayar+Sukhbaatar%2C+Arthur+Szlam%2C+Gabriel+Synnaeve%2C+Soumith+Chintala%2C+and+Rob+Fergus%2E+Mazebase%3A+A+sandbox+for+learning+from+games%2E+arXiv+preprint+arXiv%3A1511%2E07401%2C+2015%2E+%E2%80%94+%5BarXiv%3A1511%2E07401%5D%28https%3A%2F)
39. Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press Cambridge, 2018. [find](https://scholar.google.com/scholar?q=Richard+S+Sutton+and+Andrew+G+Barto%2E+Reinforcement+learning%3A+An+introduction%2E+MIT+press+Cambridge%2C+2018%2E)
40. Will Wright. SimCity. Erbe, 1996. [find](https://scholar.google.com/scholar?q=Will+Wright%2E+SimCity%2E+Erbe%2C+1996%2E)