---
title: "Grounded Language Learning in a Simulated 3D World (2017)"
description: "Teaching machines to connect words with what they see and do remains a hard problem. 40 references, cited by 58 papers here."
authors: ["Karl Moritz Hermann", "Felix Hill", "Simon Green", "Fumin Wang", "Ryan Faulkner", "Hubert Soyer", "David Szepesvari", "Wojciech Marian Czarnecki", "Max Jaderberg", "Denis Teplyashin", "Marcus Wainwright", "Chris Apps", "Demis Hassabis", "Phil Blunsom"]
year: 2017
venue: "arXiv:1706.06551"
source: "https://arxiv.org/abs/1706.06551"
doi: "10.48550/arxiv.1706.06551"
arxiv: "1706.06551"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hermann-2017-grounded-language-learning-simulated-3d/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hermann-2017-grounded-language-learning-simulated-3d/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/) / Grounded Language Learning in a Simulated 3D World

# Grounded Language Learning in a Simulated 3D World

**Authors:** [Karl Moritz Hermann](https://md.research.elamur.ai/free/en/authors/karl-moritz-hermann/), [Felix Hill](https://md.research.elamur.ai/free/en/authors/felix-hill/), [Simon Green](https://md.research.elamur.ai/free/en/authors/simon-green/), [Fumin Wang](https://md.research.elamur.ai/free/en/authors/fumin-wang/), [Ryan Faulkner](https://md.research.elamur.ai/free/en/authors/ryan-faulkner/), Hubert Soyer, David Szepesvari, [Wojciech Marian Czarnecki](https://md.research.elamur.ai/free/en/authors/wojciech-marian-czarnecki/), [Max Jaderberg](https://md.research.elamur.ai/free/en/authors/max-jaderberg/), [Denis Teplyashin](https://md.research.elamur.ai/free/en/authors/denis-teplyashin/), [Marcus Wainwright](https://md.research.elamur.ai/free/en/authors/marcus-wainwright/), Chris Apps, Demis Hassabis and [Phil Blunsom](https://md.research.elamur.ai/free/en/authors/phil-blunsom/) · **Year:** 2017 · **Venue:** arXiv:1706.06551 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1706.06551) · [PDF](https://arxiv.org/pdf/1706.06551) · [DOI 10.48550/arxiv.1706.06551](https://doi.org/10.48550/arxiv.1706.06551) · [arXiv 1706.06551](https://arxiv.org/abs/1706.06551) · [OpenAlex](https://openalex.org/W2627585944)

**Citations:** 132 citations (OpenAlex, 2026-10-14) · [58](https://md.research.elamur.ai/free/en/papers/hermann-2017-grounded-language-learning-simulated-3d/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Teaching machines to connect words with what they see and do remains a hard problem. Earlier systems such as SHRDLU depended on hand-written rules, and later work mostly used simplified, highly structured settings. This DeepMind paper adds a text-instruction channel to an extended version of DeepMind Lab, a 3D simulator. In this environment, an agent sees only raw pixels and receives reward when it picks up the object that a written command describes. The agent is a single neural network made of four parts: a convolutional vision module, an LSTM language encoder, a mixing module, and a two-layer LSTM action module. It is trained with an actor-critic method together with unsupervised auxiliary objectives, starting with almost no built-in knowledge. Language learning only succeeds when reward-based and unsupervised signals are combined. The trained agent understands new word combinations without extra training, and it picks up new words faster as its vocabulary grows. Curricula make harder relational and multi-task instructions learnable. The study suggests that grounded, embodied language can emerge end-to-end rather than being hand-coded.

## Key points
- **Environment:** DeepMind Lab is extended with a language channel. Layouts, objects, colours, patterns and positions are randomly sampled from billions of possible configurations.
- **Agent and training:** The vision module (CNN), language module (LSTM), mixing module and action module (2-layer LSTM) are optimised jointly. Training uses actor-critic RL (Mnih et al., 2016) plus unsupervised auxiliary losses.
- **Generalisation:** The agent shows zero-shot comprehension of unseen instructions and applies familiar modifiers and relations to unfamiliar objects. Its rate of word acquisition speeds up over training.
- **Curriculum:** Retrieving an object in a specified room required a four-lesson curriculum. Reported results average the best five of 16 random hyperparameter settings.
- **Multi-task learning:** One agent learned three tasks (Selection, Next to, In room) using a two-lesson curriculum. The tasks used six colour terms and 40 object shapes, so the agent had to be sensitive to word order.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) — Company, London, United Kingdom · 72 works · 84,326 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis and Phil Blunsom. 2017. Grounded Language Learning in a Simulated 3D World. arXiv:1706.06551.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{hermann2017grounded,
  title = {{Grounded Language Learning in a Simulated 3D World}},
  author = {Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis and Phil Blunsom},
  year = {2017},
  doi = {10.48550/arxiv.1706.06551},
  eprint = {1706.06551},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1706.06551}
}
```

- [← previous A Dataset for Multi-Target Stance Detection](https://md.research.elamur.ai/free/en/papers/sobhani-2017-dataset-multi-target-stance-detection/)
- [next → End-to-End Differentiable Proving](https://md.research.elamur.ai/free/en/papers/rocktaschel-2017-end-end-differentiable-proving/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/)

## References

40 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dilip Arumugam, Siddharth Karamcheti, Nakul Gopalan, Lawson L. S. Wong, and Stefanie Tellex. Accurately and efficiently interpreting human-robot instructions of varying granularities. CoRR, abs/1704.06616, 2017. URL http://arxiv.org/abs/1704.06616. — [arXiv:1704.06616](https://arxiv.org/abs/1704.06616) [↗](http://arxiv.org/abs/1704.06616)
2. Charles Beattie, Joel Z. Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Vı́ctor Valdés, Amir Sadik, Julian Schrittwieser, Keith Anderson, Sarah York, Max Cant, Adam Cain, Adrian Bolton, Stephen Gaffney, Helen King, Demis Hassabis, Shane Legg,… — [DeepMind Lab](https://md.research.elamur.ai/free/en/papers/beattie-2016-deepmind-lab/)
3. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. [Semantic parsing on freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In EMNLP, pages 1533–1544. ACL, 2013. ISBN 978-1-937284-97-8. URL [http://dblp.uni-trier.de/db/conf/emnlp/emnlp2013.html\#BerantCFL13](http://dblp.uni-trier.de/db/conf/emnlp/emnlp2013.html\#BerantCFL13). [↗](http://dblp.uni-trier.de/db/conf/emnlp/emnlp2013.html\#BerantCFL13)
4. David L. Chen and Raymond J. Mooney. Learning to sportscast: A test of grounded language acquisition. In Proceedings of the 25th International Conference on Machine Learning (ICML), Helsinki, Finland, July 2008. URL [http://www.cs.utexas.edu/users/ai-lab/?chen:icml08](http://www.cs.utexas.edu/users/ai-lab/?chen:icml08). [↗](http://www.cs.utexas.edu/users/ai-lab/?chen:icml08)
5. Noam Chomsky. A review of BF Skinner’s Verbal Behavior. Language, 35(1):26–58, 1959. [find](https://scholar.google.com/scholar?q=Noam+Chomsky%2E+A+review+of+BF+Skinner%E2%80%99s+Verbal+Behavior%2E+Language%2C+35%281%29%3A26%E2%80%9358%2C+1959%2E)
6. Leonidas AA Doumas, John E Hummel, and Catherine M Sandhofer. A theory of the discovery and predication of relational concepts. Psychological review, 115(1):1, 2008. [find](https://scholar.google.com/scholar?q=Leonidas+AA+Doumas%2C+John+E+Hummel%2C+and+Catherine+M+Sandhofer%2E+A+theory+of+the+discovery+and+predication+of+relational+concepts%2E+Psychological+review%2C+115%281%29%3A1%2C+2008%2E)
7. Sachithra Hemachandra, Matthew R. Walter, Stefanie Tellex, and Seth Teller. Learning spatially-semantic representations from natural language descriptions and scene classifications. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, May 2014. [find](https://scholar.google.com/scholar?q=Sachithra+Hemachandra%2C+Matthew+R%2E+Walter%2C+Stefanie+Tellex%2C+and+Seth+Teller%2E+Learning+spatially%2Dsemantic+representations+from+natural+language+descriptions+and+scene+classifications%2E+In+Proceedings+of+)
8. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%931780%2C+1997%2E)
9. Max Jaderberg, Volodymyr Mnih, Wojciech Marian Czarnecki, Tom Schaul, Joel Z Leibo, David Silver, and Koray Kavukcuoglu. [Reinforcement learning with unsupervised auxiliary tasks](https://md.research.elamur.ai/free/en/papers/jaderberg-2016-reinforcement-learning-unsupervised-auxiliary-tasks/). In International Conference on Learning Representations, 2016.
10. Samantha Krening, Brent Harrison, Karen M Feigh, Charles Isbell, Mark Riedl, and Andrea Thomaz. Learning from explanations using sentiment and advice in RL. In 2016 IEEE Transactions on Cognitive and Developmental Systems. [find](https://scholar.google.com/scholar?q=Samantha+Krening%2C+Brent+Harrison%2C+Karen+M+Feigh%2C+Charles+Isbell%2C+Mark+Riedl%2C+and+Andrea+Thomaz%2E+Learning+from+explanations+using+sentiment+and+advice+in+RL%2E+In+2016+IEEE+Transactions+on+Cognitive+and+)
11. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. [Imagenet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/). In Advances in neural information processing systems, pages 1097–1105, 2012.
12. Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541–551, 1989. [find](https://scholar.google.com/scholar?q=Yann+LeCun%2C+Bernhard+Boser%2C+John+S+Denker%2C+Donnie+Henderson%2C+Richard+E+Howard%2C+Wayne+Hubbard%2C+and+Lawrence+D+Jackel%2E+Backpropagation+applied+to+handwritten+zip+code+recognition%2E+Neural+computation%2C+1%28)
13. Tomas Mikolov, Armand Joulin, and Marco Baroni. A roadmap towards machine intelligence. arXiv preprint arXiv:1511.08130, 2015. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Armand+Joulin%2C+and+Marco+Baroni%2E+A+roadmap+towards+machine+intelligence%2E+arXiv+preprint+arXiv%3A1511%2E08130%2C+2015%2E)
14. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529–533, 2015. [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Koray+Kavukcuoglu%2C+David+Silver%2C+Andrei+A+Rusu%2C+Joel+Veness%2C+Marc+G+Bellemare%2C+Alex+Graves%2C+Martin+Riedmiller%2C+Andreas+K+Fidjeland%2C+Georg+Ostrovski%2C+et+al%2E+Human%2Dlevel+control+through+)
15. Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy P Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning, 2016. [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Adria+Puigdomenech+Badia%2C+Mehdi+Mirza%2C+Alex+Graves%2C+Timothy+P+Lillicrap%2C+Tim+Harley%2C+David+Silver%2C+and+Koray+Kavukcuoglu%2E+Asynchronous+methods+for+deep+reinforcement+learning%2E+In+Inter)
16. Karthik Narasimhan, Tejas Kulkarni, and Regina Barzilay. [Language understanding for text-based games using deep reinforcement learning](https://md.research.elamur.ai/free/en/papers/narasimhan-2015-language-understanding-text-based-games/). Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015.
17. Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard L Lewis, and Satinder Singh. [Action-conditional video prediction using deep networks in Atari games](https://md.research.elamur.ai/free/en/papers/oh-2015-action-conditional-video-prediction-using/). In Advances in Neural Information Processing Systems 28, 2015.
18. Steven Pinker. The bootstrapping problem in language acquisition. Mechanisms of language acquisition, pages 399–441, 1987. [find](https://scholar.google.com/scholar?q=Steven+Pinker%2E+The+bootstrapping+problem+in+language+acquisition%2E+Mechanisms+of+language+acquisition%2C+pages+399%E2%80%93441%2C+1987%2E)
19. Steven Pinker. Language learnability and language development, volume 7. Harvard University Press, 2009. [find](https://scholar.google.com/scholar?q=Steven+Pinker%2E+Language+learnability+and+language+development%2C+volume+7%2E+Harvard+University+Press%2C+2009%2E)
20. W. V. O. Quine. Word & Object. MIT Press, 1960. [find](https://scholar.google.com/scholar?q=W%2E+V%2E+O%2E+Quine%2E+Word+%26+Object%2E+MIT+Press%2C+1960%2E)
21. Paul C Quinn, Peter D Eimas, and Stacey L Rosenkrantz. Evidence for representations of perceptually similar natural categories by 3-month-old and 4-month-old infants. Perception, 22(4):463–475, 1993. [find](https://scholar.google.com/scholar?q=Paul+C+Quinn%2C+Peter+D+Eimas%2C+and+Stacey+L+Rosenkrantz%2E+Evidence+for+representations+of+perceptually+similar+natural+categories+by+3%2Dmonth%2Dold+and+4%2Dmonth%2Dold+infants%2E+Perception%2C+22%284%29%3A463%E2%80%93475%2C+1993%2E)
22. Timothy T Rogers and James L McClelland. Semantic cognition: A parallel distributed processing approach. MIT press, 2004. [find](https://scholar.google.com/scholar?q=Timothy+T+Rogers+and+James+L+McClelland%2E+Semantic+cognition%3A+A+parallel+distributed+processing+approach%2E+MIT+press%2C+2004%2E)
23. Deb K Roy and Alex P Pentland. Learning words from sights and sounds: A computational model. Cognitive science, 26(1):113–146, 2002. [find](https://scholar.google.com/scholar?q=Deb+K+Roy+and+Alex+P+Pentland%2E+Learning+words+from+sights+and+sounds%3A+A+computational+model%2E+Cognitive+science%2C+26%281%29%3A113%E2%80%93146%2C+2002%2E)
24. Carina Silberer and Mirella Lapata. Grounded models of semantic representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2012. [find](https://scholar.google.com/scholar?q=Carina+Silberer+and+Mirella+Lapata%2E+Grounded+models+of+semantic+representation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+2012%2E)
25. Jeffrey Mark Siskind. Grounding Language in Perception, pages 207–227. Springer Netherlands, Dordrecht, 1995. ISBN 978-94-011-0273-5. doi: 10.1007/978-94-011-0273-5 12. URL http://dx.doi.org/10.1007/978-94-011-0273-5_12. — [doi:10.1007/978-94-011-0273-5](https://doi.org/10.1007/978-94-011-0273-5) [↗](http://dx.doi.org/10.1007/978-94-011-0273-5_12)
26. Jeffrey Mark Siskind. Grounding the lexical semantics of verbs in visual perception using force dynamics and event logic. J. Artif. Intell. Res. (JAIR), 15:31–90, 2001. doi: 10.1613/jair.790. URL https://doi.org/10.1613/jair.790. — [doi:10.1613/jair.790](https://doi.org/10.1613/jair.790) [↗](https://doi.org/10.1613/jair.790)
27. Linda B Smith, Susan S Jones, and Barbara Landau. Naming in young children: A dumb attentional mechanism? Cognition, 60(2):143–171, 1996. [find](https://scholar.google.com/scholar?q=Linda+B+Smith%2C+Susan+S+Jones%2C+and+Barbara+Landau%2E+Naming+in+young+children%3A+A+dumb+attentional+mechanism%3F+Cognition%2C+60%282%29%3A143%E2%80%93171%2C+1996%2E)
28. Luc Steels. The symbol grounding problem has been solved. so what’s next. Symbols and embodiment: Debates on meaning and cognition, pages 223–244, 2008. [find](https://scholar.google.com/scholar?q=Luc+Steels%2E+The+symbol+grounding+problem+has+been+solved%2E+so+what%E2%80%99s+next%2E+Symbols+and+embodiment%3A+Debates+on+meaning+and+cognition%2C+pages+223%E2%80%93244%2C+2008%2E)
29. Jesse Thomason, Shiqi Zhang, Raymond Mooney, and Peter Stone. Learning to interpret natural language commands through human-robot dialog. In Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI’15, pages 1923–1929. [find](https://scholar.google.com/scholar?q=Jesse+Thomason%2C+Shiqi+Zhang%2C+Raymond+Mooney%2C+and+Peter+Stone%2E+Learning+to+interpret+natural+language+commands+through+human%2Drobot+dialog%2E+In+Proceedings+of+the+24th+International+Conference+on+Artific)
30. AAAI Press, 2015. ISBN 978-1-57735-738-4. URL [http://dl.acm.org/citation.cfm?id=2832415.2832516](http://dl.acm.org/citation.cfm?id=2832415.2832516). [↗](http://dl.acm.org/citation.cfm?id=2832415.2832516)
31. Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2), 2012. [find](https://scholar.google.com/scholar?q=Tijmen+Tieleman+and+Geoffrey+Hinton%2E+Lecture+6%2E5%2Drmsprop%3A+Divide+the+gradient+by+a+running+average+of+its+recent+magnitude%2E+COURSERA%3A+Neural+networks+for+machine+learning%2C+4%282%29%2C+2012%2E)
32. Ivan Vendrov, Ryan Kiros, Sanja Fidler, and Raquel Urtasun. [Order-embeddings of images and language](https://md.research.elamur.ai/free/en/papers/vendrov-2015-order-embeddings-images-language/). CoRR, abs/1511.06361, 2015. URL http://arxiv.org/abs/1511.06361. [↗](http://arxiv.org/abs/1511.06361)
33. Stella Vosniadou and William F Brewer. Mental models of the earth: A study of conceptual change in childhood. Cognitive psychology, 24(4):535–585, 1992. [find](https://scholar.google.com/scholar?q=Stella+Vosniadou+and+William+F+Brewer%2E+Mental+models+of+the+earth%3A+A+study+of+conceptual+change+in+childhood%2E+Cognitive+psychology%2C+24%284%29%3A535%E2%80%93585%2C+1992%2E)
34. Matthew R. Walter, Sachithra Hemachandra, Bianca Homberg, Stefanie Tellex, and Seth Teller. A framework for learning semantic maps from grounded natural language descriptions. The International Journal of Robotics Research, 33(9):1167–1190, 2014. doi: 10.1177/0278364914537359. URL http://dx.doi.org/… — [doi:10.1177/0278364914537359](https://doi.org/10.1177/0278364914537359) [find](https://scholar.google.com/scholar?q=Matthew+R%2E+Walter%2C+Sachithra+Hemachandra%2C+Bianca+Homberg%2C+Stefanie+Tellex%2C+and+Seth+Teller%2E+A+framework+for+learning+semantic+maps+from+grounded+natural+language+descriptions%2E+The+International+Journa)
35. S. I. Wang, P. Liang, and C. Manning. Learning language games through interaction. In Association for Computational Linguistics (ACL), 2016. [find](https://scholar.google.com/scholar?q=S%2E+I%2E+Wang%2C+P%2E+Liang%2C+and+C%2E+Manning%2E+Learning+language+games+through+interaction%2E+In+Association+for+Computational+Linguistics+%28ACL%29%2C+2016%2E)
36. Terry Winograd. Understanding natural language. Cognitive psychology, 3(1):1–191, 1972. [find](https://scholar.google.com/scholar?q=Terry+Winograd%2E+Understanding+natural+language%2E+Cognitive+psychology%2C+3%281%29%3A1%E2%80%93191%2C+1972%2E)
37. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard S Zemel, and Yoshua Bengio. [Show, attend and tell: Neural image caption generation with visual attention](https://md.research.elamur.ai/free/en/papers/xu-2015-show-attend-tell-neural-image/). International Conference of Machine Learning, 2(3):5, 2015.
38. Haonan Yu and Jeffrey Mark Siskind. Grounded language learning from video described with sentences. In ACL, pages 53–63. The Association for Computer Linguistics, 2013. ISBN 978-1-937284-50-3. [find](https://scholar.google.com/scholar?q=Haonan+Yu+and+Jeffrey+Mark+Siskind%2E+Grounded+language+learning+from+video+described+with+sentences%2E+In+ACL%2C+pages+53%E2%80%9363%2E+The+Association+for+Computer+Linguistics%2C+2013%2E+ISBN+978%2D1%2D937284%2D50%2D3%2E)
39. Haonan Yu, Haichao Zhang, and Wei Xu. A deep compositional framework for human-like language acquisition in virtual environment. CoRR, abs/1703.09831, 2017. URL https://arxiv.org/abs/1703.09831. — [arXiv:1703.09831](https://arxiv.org/abs/1703.09831) [↗](https://arxiv.org/abs/1703.09831)
40. Luke S. Zettlemoyer and Michael Collins. [Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars](https://md.research.elamur.ai/free/en/papers/zettlemoyer-2012-learning-map-sentences-logical-form/). In Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, UAI’05, pages 658– 666, Arlington, Virginia, United States, 2…

*Updated 2 November 2026*