---
title: "Language Understanding for Text-based Games Using Deep"
description: "The paper tackles automatic play of text-based adventure games (Multi-User Dungeons). 29 references, cited by 95 papers here."
authors: ["Karthik Narasimhan", "Tejas Kulkarni", "Regina Barzilay"]
year: 2015
venue: "arXiv:1506.08941"
source: "https://arxiv.org/abs/1506.08941"
doi: "10.48550/arxiv.1506.08941"
arxiv: "1506.08941"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/narasimhan-2015-language-understanding-text-based-games/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/narasimhan-2015-language-understanding-text-based-games/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/) / Language Understanding for Text-based Games Using Deep Reinforcement Learning

# Language Understanding for Text-based Games Using Deep Reinforcement Learning

**Authors:** [Karthik Narasimhan](https://md.research.elamur.ai/free/en/authors/karthik-narasimhan/), Tejas Kulkarni and [Regina Barzilay](https://md.research.elamur.ai/free/en/authors/regina-barzilay/) · **Year:** 2015 · **Venue:** [arXiv:1506.08941](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2015](https://md.research.elamur.ai/free/en/venues/emnlp/2015/)) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1506.08941) · [PDF](https://arxiv.org/pdf/1506.08941) · [DOI 10.48550/arxiv.1506.08941](https://doi.org/10.48550/arxiv.1506.08941) · [arXiv 1506.08941](https://arxiv.org/abs/1506.08941) · [OpenAlex](https://openalex.org/W1934909785)

**Citations:** 285 citations (OpenAlex, 2026-10-14) · [95](https://md.research.elamur.ai/free/en/papers/narasimhan-2015-language-understanding-text-based-games/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles automatic play of text-based adventure games (Multi-User Dungeons). In these games the true world state is hidden, and the agent sees only natural-language descriptions, so it has to interpret text before it can act well. Narasimhan, Kulkarni and Barzilay treat gameplay as a Markov Decision Process. They train a deep Q-network, LSTM-DQN, using only in-game rewards, with no annotated state labels. An LSTM with mean pooling turns each description into a state vector, and a multi-layer action scorer then rates action–object commands. Both parts are learned jointly. Experiments use two worlds built on the Evennia framework: a controlled Home world and a Fantasy world whose descriptions are written by humans and vary more. LSTM-DQN beats a random agent and bag-of-words or bag-of-bigrams DQNs in both quest completion and accumulated reward. Its learned representations also transfer to a re-mapped world. The work shows that rich text representations can be learned from reward alone, which links grounded language understanding with reinforcement learning.

## Key points
- On the Fantasy world, LSTM-DQN completes about 96% of quests with an average reward of −11.33. BOW-DQN reaches about 82% (reward −12.68), and the random agent solves about 5%. BI-DQN completes 97% of quests but earns a worse reward (−26.68) because its solutions are sub-optimal.
- Transfer test: the LSTM parameters (θ_R) were copied from the original Home world into a Home world with a new map. This agent reached the optimal policy roughly 20 epochs sooner than one trained from scratch.
- Prioritized minibatch sampling reached the optimal policy about 50 epochs earlier than uniform sampling.
- In t-SNE plots, words learned from random initialization form coherent groups for each room, its objects and its quest words (e.g., "hungry" sits near kitchen and pizza). Nearest neighbors by cosine similarity tend to be descriptions of the same or nearby states.
- Code is released at people.csail.mit.edu/karthikn/mud-play (arXiv:1506.08941, EMNLP 2015).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 243 works · 76,506 citations
- [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) — Research institute, Cambridge, United States · 76 works · 32,207 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Karthik Narasimhan, Tejas Kulkarni and Regina Barzilay. 2015. Language Understanding for Text-based Games Using Deep Reinforcement Learning. arXiv:1506.08941.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{narasimhan2015language,
  title = {{Language Understanding for Text-based Games Using Deep Reinforcement Learning}},
  author = {Karthik Narasimhan, Tejas Kulkarni and Regina Barzilay},
  year = {2015},
  doi = {10.48550/arxiv.1506.08941},
  eprint = {1506.08941},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1506.08941}
}
```

- [← previous Truth Is a Lie: Crowd Truth and the Seven Myths of Human Annotation](https://md.research.elamur.ai/free/en/papers/aroyo-2015-truth-lie-crowd-truth-seven/)
- [next → From Paraphrase Database to Compositional Paraphrase Model and Back](https://md.research.elamur.ai/free/en/papers/wieting-2015-paraphrase-database-compositional-paraphrase-model/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/)

## References

29 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Christopher Amato and Guy Shani. 2010. High-level reinforcement learning in strategy games. In Proceedings of the 9th International Conference on Autonomous Agents and Multi-agent Systems: Volume 1, pages 75–82. International Foundation for Autonomous Agents and Multiagent Systems. [find](https://scholar.google.com/scholar?q=Christopher+Amato+and+Guy+Shani%2E+2010%2E+High%2Dlevel+reinforcement+learning+in+strategy+games%2E+In+Proceedings+of+the+9th+International+Conference+on+Autonomous+Agents+and+Multi%2Dagent+Systems%3A+Volume+1%2C+p)
2. Eyal Amir and Patrick Doyle. 2002. Adventure games: A challenge for cognitive robotics. In Proc. Int. Cognitive Robotics Workshop, pages 148–155. [find](https://scholar.google.com/scholar?q=Eyal+Amir+and+Patrick+Doyle%2E+2002%2E+Adventure+games%3A+A+challenge+for+cognitive+robotics%2E+In+Proc%2E+Int%2E+Cognitive+Robotics+Workshop%2C+pages+148%E2%80%93155%2E)
3. Jacob Andreas and Dan Klein. 2015. Alignment-based compositional semantics for instruction following. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Jacob+Andreas+and+Dan+Klein%2E+2015%2E+Alignment%2Dbased+compositional+semantics+for+instruction+following%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E)
4. Yoav Artzi and Luke Zettlemoyer. 2013. [Weakly supervised learning of semantic parsers for mapping instructions to actions](https://md.research.elamur.ai/free/en/papers/artzi-2013-weakly-supervised-learning-semantic-parsers/). Transactions of the Association for Computational Linguistics, 1(1):49–62.
5. SRK Branavan, Luke S Zettlemoyer, and Regina Barzilay. 2010. Reading between the lines: Learning to map high-level instructions to commands. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1268–1277. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=SRK+Branavan%2C+Luke+S+Zettlemoyer%2C+and+Regina+Barzilay%2E+2010%2E+Reading+between+the+lines%3A+Learning+to+map+high%2Dlevel+instructions+to+commands%2E+In+Proceedings+of+the+48th+Annual+Meeting+of+the+Associatio)
6. SRK Branavan, David Silver, and Regina Barzilay. 2011a. Learning to win by reading manuals in a monte-carlo framework. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 268–277. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=SRK+Branavan%2C+David+Silver%2C+and+Regina+Barzilay%2E+2011a%2E+Learning+to+win+by+reading+manuals+in+a+monte%2Dcarlo+framework%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Association+for+Computational+Li)
7. SRK Branavan, David Silver, and Regina Barzilay. 2011b. Non-linear monte-carlo search in Civilization II. AAAI Press/International Joint Conferences on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=SRK+Branavan%2C+David+Silver%2C+and+Regina+Barzilay%2E+2011b%2E+Non%2Dlinear+monte%2Dcarlo+search+in+Civilization+II%2E+AAAI+Press%2FInternational+Joint+Conferences+on+Artificial+Intelligence%2E)
8. Pavel Curtis. 1992. Mudding: Social phenomena in text-based virtual realities. High noon on the electronic frontier: Conceptual issues in cyberspace, pages 347–374. [find](https://scholar.google.com/scholar?q=Pavel+Curtis%2E+1992%2E+Mudding%3A+Social+phenomena+in+text%2Dbased+virtual+realities%2E+High+noon+on+the+electronic+frontier%3A+Conceptual+issues+in+cyberspace%2C+pages+347%E2%80%93374%2E)
9. Mark A DePristo and Robert Zubek. 2001. being-in-the-world. In Proceedings of the 2001 AAAI Spring Symposium on Artificial Intelligence and Interactive Entertainment, pages 31–34. [find](https://scholar.google.com/scholar?q=Mark+A+DePristo+and+Robert+Zubek%2E+2001%2E+being%2Din%2Dthe%2Dworld%2E+In+Proceedings+of+the+2001+AAAI+Spring+Symposium+on+Artificial+Intelligence+and+Interactive+Entertainment%2C+pages+31%E2%80%9334%2E)
10. Jacob Eisenstein, James Clarke, Dan Goldwasser, and Dan Roth. 2009. Reading to learn: Constructing features from semantic abstracts. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 958–967, Singapore, August. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jacob+Eisenstein%2C+James+Clarke%2C+Dan+Goldwasser%2C+and+Dan+Roth%2E+2009%2E+Reading+to+learn%3A+Constructing+features+from+semantic+abstracts%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+La)
11. Peter Gorniak and Deb Roy. 2005. Speaking with your sidekick: Understanding situated speech in computer role playing games. In R. Michael Young and John E. Laird, editors, Proceedings of the First Artificial Intelligence and Interactive Digital Entertainment Conference, June 1-5, 2005, Marina del Re… [find](https://scholar.google.com/scholar?q=Peter+Gorniak+and+Deb+Roy%2E+2005%2E+Speaking+with+your+sidekick%3A+Understanding+situated+speech+in+computer+role+playing+games%2E+In+R%2E+Michael+Young+and+John+E%2E+Laird%2C+editors%2C+Proceedings+of+the+First+Art)
12. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%931780%2E)
13. Thomas Kollar, Stefanie Tellex, Deb Roy, and Nicholas Roy. 2010. Toward understanding natural language directions. In Human-Robot Interaction (HRI), 2010 5th ACM/IEEE International Conference on, pages 259–266. IEEE. [find](https://scholar.google.com/scholar?q=Thomas+Kollar%2C+Stefanie+Tellex%2C+Deb+Roy%2C+and+Nicholas+Roy%2E+2010%2E+Toward+understanding+natural+language+directions%2E+In+Human%2DRobot+Interaction+%28HRI%29%2C+2010+5th+ACM%2FIEEE+International+Conference+on%2C+page)
14. Jan Koutnı́k, Giuseppe Cuccu, Jürgen Schmidhuber, and Faustino Gomez. 2013. Evolving large-scale neural networks for vision-based reinforcement learning. In Proceedings of the 15th annual conference on Genetic and evolutionary computation, pages 1061–1068. ACM. [find](https://scholar.google.com/scholar?q=Jan+Koutn%C4%B1%CC%81k%2C+Giuseppe+Cuccu%2C+Ju%CC%88rgen+Schmidhuber%2C+and+Faustino+Gomez%2E+2013%2E+Evolving+large%2Dscale+neural+networks+for+vision%2Dbased+reinforcement+learning%2E+In+Proceedings+of+the+15th+annual+conference+)
15. Nate Kushman, Yoav Artzi, Luke Zettlemoyer, and Regina Barzilay. 2014. Learning to automatically solve algebra word problems. ACL (1), pages 271–281. [find](https://scholar.google.com/scholar?q=Nate+Kushman%2C+Yoav+Artzi%2C+Luke+Zettlemoyer%2C+and+Regina+Barzilay%2E+2014%2E+Learning+to+automatically+solve+algebra+word+problems%2E+ACL+%281%29%2C+pages+271%E2%80%93281%2E)
16. Cynthia Matuszek, Evan Herbst, Luke Zettlemoyer, and Dieter Fox. 2013. Learning to parse natural language commands to a robot control system. In Experimental Robotics, pages 403– 415. Springer. [find](https://scholar.google.com/scholar?q=Cynthia+Matuszek%2C+Evan+Herbst%2C+Luke+Zettlemoyer%2C+and+Dieter+Fox%2E+2013%2E+Learning+to+parse+natural+language+commands+to+a+robot+control+system%2E+In+Experimental+Robotics%2C+pages+403%E2%80%93+415%2E+Springer%2E)
17. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). arXiv preprint arXiv:1301.3781.
18. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis H… [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Koray+Kavukcuoglu%2C+David+Silver%2C+Andrei+A%2E+Rusu%2C+Joel+Veness%2C+Marc+G%2E+Bellemare%2C+Alex+Graves%2C+Martin+Riedmiller%2C+Andreas+K%2E+Fidjeland%2C+Georg+Ostrovski%2C+Stig+Petersen%2C+Charles+Beattie%2C+)
19. Andrew W Moore and Christopher G Atkeson. 1993. Prioritized sweeping: Reinforcement learning with less data and less time. Machine Learning, 13(1):103–130. [find](https://scholar.google.com/scholar?q=Andrew+W+Moore+and+Christopher+G+Atkeson%2E+1993%2E+Prioritized+sweeping%3A+Reinforcement+learning+with+less+data+and+less+time%2E+Machine+Learning%2C+13%281%29%3A103%E2%80%93130%2E)
20. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014), 12.
21. David Silver, Richard S Sutton, and Martin Müller. 2007. Reinforcement learning of local shape in the game of go. In IJCAI, volume 7, pages 1053–1058. [find](https://scholar.google.com/scholar?q=David+Silver%2C+Richard+S+Sutton%2C+and+Martin+Mu%CC%88ller%2E+2007%2E+Reinforcement+learning+of+local+shape+in+the+game+of+go%2E+In+IJCAI%2C+volume+7%2C+pages+1053%E2%80%931058%2E)
22. Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. 2014. [Sequence to sequence learning with neural networks](https://md.research.elamur.ai/free/en/papers/sutskever-2014-sequence-sequence-learning-neural-networks/). In Advances in Neural Information Processing Systems, pages 3104– 3112.
23. Richard S Sutton and Andrew G Barto. 1998. Introduction to reinforcement learning. MIT Press. [find](https://scholar.google.com/scholar?q=Richard+S+Sutton+and+Andrew+G+Barto%2E+1998%2E+Introduction+to+reinforcement+learning%2E+MIT+Press%2E)
24. István Szita. 2012. Reinforcement learning in games. In Reinforcement Learning, pages 539–577. Springer. [find](https://scholar.google.com/scholar?q=Istva%CC%81n+Szita%2E+2012%2E+Reinforcement+learning+in+games%2E+In+Reinforcement+Learning%2C+pages+539%E2%80%93577%2E+Springer%2E)
25. Kai Sheng Tai, Richard Socher, and Christopher D. Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural La… [find](https://scholar.google.com/scholar?q=Kai+Sheng+Tai%2C+Richard+Socher%2C+and+Christopher+D%2E+Manning%2E+2015%2E+Improved+semantic+representations+from+tree%2Dstructured+long+short%2Dterm+memory+networks%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+th)
26. Tijmen Tieleman and Geoffrey Hinton. 2012. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 4. [find](https://scholar.google.com/scholar?q=Tijmen+Tieleman+and+Geoffrey+Hinton%2E+2012%2E+Lecture+6%2E5%2Drmsprop%3A+Divide+the+gradient+by+a+running+average+of+its+recent+magnitude%2E+COURSERA%3A+Neural+Networks+for+Machine+Learning%2C+4%2E)
27. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of Machine Learning Research, 9(2579-2605):85. [find](https://scholar.google.com/scholar?q=Laurens+Van+der+Maaten+and+Geoffrey+Hinton%2E+2008%2E+Visualizing+data+using+t%2Dsne%2E+Journal+of+Machine+Learning+Research%2C+9%282579%2D2605%29%3A85%2E)
28. Adam Vogel and Dan Jurafsky. 2010. Learning to follow navigational directions. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 806–814. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Adam+Vogel+and+Dan+Jurafsky%2E+2010%2E+Learning+to+follow+navigational+directions%2E+In+Proceedings+of+the+48th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+806%E2%80%93814%2E+Association+fo)
29. Christopher JCH Watkins and Peter Dayan. 1992. Q-learning. Machine learning, 8(3-4):279–292. [find](https://scholar.google.com/scholar?q=Christopher+JCH+Watkins+and+Peter+Dayan%2E+1992%2E+Q%2Dlearning%2E+Machine+learning%2C+8%283%2D4%29%3A279%E2%80%93292%2E)

*Updated 2 November 2026*