---
title: "Augmenting Autotelic Agents with Large Language Models (2023)"
description: "Second, it proposes new higher-level goals built from subgoals the agent already masters. 56 references, cited by 7 papers here."
authors: ["Cedric Colas", "Laetitia Teodorescu", "Pierre-Yves Oudeyer", "Xingdi Yuan", "Marc-Alexandre Cote"]
year: 2023
venue: "arXiv:2305.12487"
source: "https://arxiv.org/abs/2305.12487"
doi: "10.48550/arxiv.2305.12487"
arxiv: "2305.12487"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/colas-2023-augmenting-autotelic-agents-large-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/colas-2023-augmenting-autotelic-agents-large-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/15/) / Augmenting Autotelic Agents with Large Language Models

# Augmenting Autotelic Agents with Large Language Models

**Authors:** Cedric Colas, Laetitia Teodorescu, [Pierre-Yves Oudeyer](https://md.research.elamur.ai/free/en/authors/pierre-yves-oudeyer/), [Xingdi Yuan](https://md.research.elamur.ai/free/en/authors/xingdi-yuan/) and [Marc-Alexandre Cote](https://md.research.elamur.ai/free/en/authors/marc-alexandre-cote/) · **Year:** 2023 · **Venue:** arXiv:2305.12487 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2305.12487) · [PDF](https://arxiv.org/pdf/2305.12487) · [DOI 10.48550/arxiv.2305.12487](https://doi.org/10.48550/arxiv.2305.12487) · [arXiv 2305.12487](https://arxiv.org/abs/2305.12487) · [OpenAlex](https://openalex.org/W4377864377)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/colas-2023-augmenting-autotelic-agents-large-language/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Autotelic agents learn by inventing and practicing their own goals. Most current systems, however, draw goals from a fixed space chosen in advance by designers. They cannot form new abstractions or propose goals that people would find meaningful. The authors note that humans pick up many goals through culture and language. They therefore use a pretrained language model as a rough proxy for that cultural input. Their Language Model Augmented Autotelic Agent (LMA3) hands three tasks to the LM through few-shot prompts. First, it relabels trajectories by naming the goals they accomplished. Second, it proposes new higher-level goals built from subgoals the agent already masters. Third, it acts as the reward function by judging whether each goal was reached. Experiments take place in an interactive, task-agnostic environment where observations, actions and goals are all text. LMA3 acquires a broad, varied skill repertoire with no hand-coded goals, reward functions or curriculum. This suggests pretrained LMs can replace manual goal engineering in open-ended agents.

## Key points
- LMA3 has three LM-driven components: a relabeler, a goal generator and a reward function. The relabeler lists the goals a trajectory achieved and the step where each was first completed. The goal generator outputs a high-level goal plus its subgoal decomposition. The reward function scores the main goal, its subgoals and the relabeled descriptions.
- The goal generator combines 2, 3 or 4 instructions into a new goal. These are picked from a random subsample of up to 60 already-mastered subgoals. Example prompt goals include "do the dishes" and "prepare a fire in the chimney".
- Prompts come in two versions: a chain-of-thought variant (LMA3) and an ablation without reasoning steps (LMA3 \ CoT).
- The approach extends the earlier Imagine agent (Colas et al., 2020). Its goal-based exploration resembles Go-Explore.
- LM API costs led the authors to keep the policy-learning algorithm deliberately simple, since the paper's focus is goal generation rather than policy generalization.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 403 works · 162,838 citations
- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 274 works · 82,013 citations
- [Institut national de recherche en sciences et technologies du numérique](https://md.research.elamur.ai/free/en/labs/institut-national-de-recherche-en-sciences-et-technologies-du-numerique/) — Research institute, Le Chesnay-Rocquencourt, France · 90 works · 26,097 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works

## Cite

Cite as: Cedric Colas, Laetitia Teodorescu, Pierre-Yves Oudeyer, Xingdi Yuan and Marc-Alexandre Cote. 2023. Augmenting Autotelic Agents with Large Language Models. arXiv:2305.12487.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{colas2023augmenting,
  title = {{Augmenting Autotelic Agents with Large Language Models}},
  author = {Cedric Colas, Laetitia Teodorescu, Pierre-Yves Oudeyer, Xingdi Yuan and Marc-Alexandre Cote},
  year = {2023},
  doi = {10.48550/arxiv.2305.12487},
  eprint = {2305.12487},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.12487}
}
```

- [← previous Are Frequent Phrases Directly Retrieved like Idioms? An Investigation with Self-Paced Reading and Language Models](https://md.research.elamur.ai/free/en/papers/rambelli-2023-are-frequent-phrases-directly-retrieved/)
- [next → Chain-of-Thought Reasoning is a Policy Improvement Operator](https://md.research.elamur.ai/free/en/papers/zhang-2023-chain-thought-reasoning-policy-improvement/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/15/)

## References

56 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. [Do as i can, not as i say: Grounding language in robotic affordances](https://md.research.elamur.ai/free/en/papers/ahn-2022-i-i-say-grounding-language/). arXiv preprint arXiv:2204.01691, 2022.
2. Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. [Hindsight experience replay](https://md.research.elamur.ai/free/en/papers/andrychowicz-2017-hindsight-experience-replay/). Advances in neural information processing systems, 30, 2017.
3. Saurabh Arora and Prashant Doshi. A survey of inverse reinforcement learning: Challenges, methods and progress. Artificial Intelligence, 297:103500, 2021. [find](https://scholar.google.com/scholar?q=Saurabh+Arora+and+Prashant+Doshi%2E+A+survey+of+inverse+reinforcement+learning%3A+Challenges%2C+methods+and+progress%2E+Artificial+Intelligence%2C+297%3A103500%2C+2021%2E)
4. Arthur Aubret, Laetitia Matignon, and Salima Hassas. A survey on intrinsic motivation in reinforcement learning. arXiv preprint arXiv:1908.06976, 2019. [find](https://scholar.google.com/scholar?q=Arthur+Aubret%2C+Laetitia+Matignon%2C+and+Salima+Hassas%2E+A+survey+on+intrinsic+motivation+in+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1908%2E06976%2C+2019%2E)
5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901, 2020.
6. Thomas Carta, Clément Romac, Thomas Wolf, Sylvain Lamprier, Olivier Sigaud, and Pierre-Yves Oudeyer. Grounding large language models in interactive environments with online reinforcement learning. arXiv preprint arXiv:2302.02662, 2023. — [arXiv:2302.02662](https://arxiv.org/abs/2302.02662) [find](https://scholar.google.com/scholar?q=Thomas+Carta%2C+Cle%CC%81ment+Romac%2C+Thomas+Wolf%2C+Sylvain+Lamprier%2C+Olivier+Sigaud%2C+and+Pierre%2DYves+Oudeyer%2E+Grounding+large+language+models+in+interactive+environments+with+online+reinforcement+learning%2E+ar)
7. Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in N… [find](https://scholar.google.com/scholar?q=Lili+Chen%2C+Kevin+Lu%2C+Aravind+Rajeswaran%2C+Kimin+Lee%2C+Aditya+Grover%2C+Misha+Laskin%2C+Pieter+Abbeel%2C+Aravind+Srinivas%2C+and+Igor+Mordatch%2E+Decision+transformer%3A+Reinforcement+learning+via+sequence+modeling%2E)
8. Noam Chomsky and David W Lightfoot. Syntactic structures. Walter de Gruyter, 2002. [find](https://scholar.google.com/scholar?q=Noam+Chomsky+and+David+W+Lightfoot%2E+Syntactic+structures%2E+Walter+de+Gruyter%2C+2002%2E)
9. Junyi Chu and Laura E Schulz. Play, curiosity, and cognition. Annual Review of Developmental Psychology, 2:317–343, 2020. [find](https://scholar.google.com/scholar?q=Junyi+Chu+and+Laura+E+Schulz%2E+Play%2C+curiosity%2C+and+cognition%2E+Annual+Review+of+Developmental+Psychology%2C+2%3A317%E2%80%93343%2C+2020%2E)
10. Cédric Colas, Pierre Fournier, Mohamed Chetouani, Olivier Sigaud, and Pierre-Yves Oudeyer. Curious: intrinsically motivated modular multi-goal reinforcement learning. In International conference on machine learning, pp. 1331– 1340. PMLR, 2019. [find](https://scholar.google.com/scholar?q=Ce%CC%81dric+Colas%2C+Pierre+Fournier%2C+Mohamed+Chetouani%2C+Olivier+Sigaud%2C+and+Pierre%2DYves+Oudeyer%2E+Curious%3A+intrinsically+motivated+modular+multi%2Dgoal+reinforcement+learning%2E+In+International+conference+on+m)
11. Cédric Colas, Tristan Karch, Nicolas Lair, Jean-Michel Dussoux, Clément Moulin-Frier, Peter Dominey, and Pierre-Yves Oudeyer. Language as a cognitive tool to imagine goals in curiosity driven exploration. Advances in Neural Information Processing Systems, 33:3761–3774, 2020. [find](https://scholar.google.com/scholar?q=Ce%CC%81dric+Colas%2C+Tristan+Karch%2C+Nicolas+Lair%2C+Jean%2DMichel+Dussoux%2C+Cle%CC%81ment+Moulin%2DFrier%2C+Peter+Dominey%2C+and+Pierre%2DYves+Oudeyer%2E+Language+as+a+cognitive+tool+to+imagine+goals+in+curiosity+driven+explor)
12. Cédric Colas, Tristan Karch, Clément Moulin-Frier, and Pierre-Yves Oudeyer. Language and culture internalization for human-like autotelic ai. Nature Machine Intelligence, 4(12):1068–1076, 2022a. [find](https://scholar.google.com/scholar?q=Ce%CC%81dric+Colas%2C+Tristan+Karch%2C+Cle%CC%81ment+Moulin%2DFrier%2C+and+Pierre%2DYves+Oudeyer%2E+Language+and+culture+internalization+for+human%2Dlike+autotelic+ai%2E+Nature+Machine+Intelligence%2C+4%2812%29%3A1068%E2%80%931076%2C+2022a%2E)
13. Cédric Colas, Tristan Karch, Olivier Sigaud, and Pierre-Yves Oudeyer. Autotelic agents with intrinsically motivated goal-conditioned reinforcement learning: a short survey. Journal of Artificial Intelligence Research, 74:1159–1199, 2022b. [find](https://scholar.google.com/scholar?q=Ce%CC%81dric+Colas%2C+Tristan+Karch%2C+Olivier+Sigaud%2C+and+Pierre%2DYves+Oudeyer%2E+Autotelic+agents+with+intrinsically+motivated+goal%2Dconditioned+reinforcement+learning%3A+a+short+survey%2E+Journal+of+Artificial+Inte)
14. Marc-Alexandre Côté, Akos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Matthew Hausknecht, Layla El Asri, Mahmoud Adada, et al. [Textworld: A learning environment for text-based games](https://md.research.elamur.ai/free/en/papers/cote-2018-textworld-learning-environment-text-based/). In Computer Games: 7th Workshop, CGW 2018, Held in Conjunction with the 27th Intern…
15. Ishita Dasgupta, Christine Kaeser-Chen, Kenneth Marino, Arun Ahuja, Sheila Babayan, Felix Hill, and Rob Fergus. [Collaborating with language models for embodied reasoning](https://md.research.elamur.ai/free/en/papers/dasgupta-2023-collaborating-language-models-embodied-reasoning/). arXiv preprint arXiv:2302.00763, 2023.
16. Guy Dove. Language as a disruptive technology: abstract concepts, embodiment and the flexible mind. Philosophical Transactions of the Royal Society B: Biological Sciences, 373(1752):20170135, 2018. [find](https://scholar.google.com/scholar?q=Guy+Dove%2E+Language+as+a+disruptive+technology%3A+abstract+concepts%2C+embodiment+and+the+flexible+mind%2E+Philosophical+Transactions+of+the+Royal+Society+B%3A+Biological+Sciences%2C+373%281752%29%3A20170135%2C+2018%2E)
17. Yuqing Du, Ksenia Konyushkova, Misha Denil, Akhil Raju, Jessica Landon, Felix Hill, Nando de Freitas, and Serkan Cabi. Vision-language models as success detectors. arXiv preprint arXiv:2303.07280, 2023a. — [arXiv:2303.07280](https://arxiv.org/abs/2303.07280) [find](https://scholar.google.com/scholar?q=Yuqing+Du%2C+Ksenia+Konyushkova%2C+Misha+Denil%2C+Akhil+Raju%2C+Jessica+Landon%2C+Felix+Hill%2C+Nando+de+Freitas%2C+and+Serkan+Cabi%2E+Vision%2Dlanguage+models+as+success+detectors%2E+arXiv+preprint+arXiv%3A2303%2E07280%2C+202)
18. Yuqing Du, Olivia Watkins, Zihan Wang, Cédric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. Guiding pretraining in reinforcement learning with large language models. arXiv preprint arXiv:2302.06692, 2023b. [find](https://scholar.google.com/scholar?q=Yuqing+Du%2C+Olivia+Watkins%2C+Zihan+Wang%2C+Ce%CC%81dric+Colas%2C+Trevor+Darrell%2C+Pieter+Abbeel%2C+Abhishek+Gupta%2C+and+Jacob+Andreas%2E+Guiding+pretraining+in+reinforcement+learning+with+large+language+models%2E+arXiv+)
19. Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O Stanley, and Jeff Clune. First return, then explore. Nature, 590(7847):580–586, 2021. [find](https://scholar.google.com/scholar?q=Adrien+Ecoffet%2C+Joost+Huizinga%2C+Joel+Lehman%2C+Kenneth+O+Stanley%2C+and+Jeff+Clune%2E+First+return%2C+then+explore%2E+Nature%2C+590%287847%29%3A580%E2%80%93586%2C+2021%2E)
20. Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. [Diversity is all you need: Learning skills without a reward function](https://md.research.elamur.ai/free/en/papers/eysenbach-2018-diversity-all-need-learning-skills/). In International Conference on Learning Representations, 2019. URL [https://openreview.net/forum?id=SJx63jRqFm](https://openreview.net/forum?id=SJx63jRqFm). [↗](https://openreview.net/forum?id=SJx63jRqFm)
21. Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. [Minedojo: Building open-ended embodied agents with internet-scale knowledge](https://md.research.elamur.ai/free/en/papers/fan-2022-minedojo-building-open-ended-embodied/). In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benc…
22. Dedre Gentner. Language as cognitive tool kit: How language supports relational thought. American psychologist, 71 (8):650, 2016. [find](https://scholar.google.com/scholar?q=Dedre+Gentner%2E+Language+as+cognitive+tool+kit%3A+How+language+supports+relational+thought%2E+American+psychologist%2C+71+%288%29%3A650%2C+2016%2E)
23. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023. [find](https://scholar.google.com/scholar?q=Danijar+Hafner%2C+Jurgis+Pasukonis%2C+Jimmy+Ba%2C+and+Timothy+Lillicrap%2E+Mastering+diverse+domains+through+world+models%2E+arXiv+preprint+arXiv%3A2301%2E04104%2C+2023%2E)
24. Ji He, Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng, and Mari Ostendorf. Deep reinforcement learning with a natural language action space. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1621–1630, Berlin, Germany… [find](https://scholar.google.com/scholar?q=Ji+He%2C+Jianshu+Chen%2C+Xiaodong+He%2C+Jianfeng+Gao%2C+Lihong+Li%2C+Li+Deng%2C+and+Mari+Ostendorf%2E+Deep+reinforcement+learning+with+a+natural+language+action+space%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+t)
25. Matteo Hessel, Joseph Modayil, Hado Van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, and David Silver. Rainbow: Combining improvements in deep reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018. [find](https://scholar.google.com/scholar?q=Matteo+Hessel%2C+Joseph+Modayil%2C+Hado+Van+Hasselt%2C+Tom+Schaul%2C+Georg+Ostrovski%2C+Will+Dabney%2C+Dan+Horgan%2C+Bilal+Piot%2C+Mohammad+Azar%2C+and+David+Silver%2E+Rainbow%3A+Combining+improvements+in+deep+reinforcemen)
26. Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. [Language models as zero-shot planners: Extracting actionable knowledge for embodied agents](https://md.research.elamur.ai/free/en/papers/huang-2022-language-models-zero-shot-planners/). In International Conference on Machine Learning, pp. 9118–9147. PMLR, 2022a.
27. Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, Pierre Sermanet, Tomas Jackson, Noah Brown, Linda Luu, Sergey Levine, Karol Hausman, and brian ichter. Inner monologue: Embodied reasoning through planning with lan… [find](https://scholar.google.com/scholar?q=Wenlong+Huang%2C+Fei+Xia%2C+Ted+Xiao%2C+Harris+Chan%2C+Jacky+Liang%2C+Pete+Florence%2C+Andy+Zeng%2C+Jonathan+Tompson%2C+Igor+Mordatch%2C+Yevgen+Chebotar%2C+Pierre+Sermanet%2C+Tomas+Jackson%2C+Noah+Brown%2C+Linda+Luu%2C+Sergey+Le)
28. Michael Janner, Qiyang Li, and Sergey Levine. Reinforcement learning as one big sequence modeling problem. In Neural Information Processing Systems, 2021. [find](https://scholar.google.com/scholar?q=Michael+Janner%2C+Qiyang+Li%2C+and+Sergey+Levine%2E+Reinforcement+learning+as+one+big+sequence+modeling+problem%2E+In+Neural+Information+Processing+Systems%2C+2021%2E)
29. Minqi Jiang, Edward Grefenstette, and Tim Rocktäschel. Prioritized level replay. In International Conference on Machine Learning, pp. 4940–4950. PMLR, 2021. [find](https://scholar.google.com/scholar?q=Minqi+Jiang%2C+Edward+Grefenstette%2C+and+Tim+Rockta%CC%88schel%2E+Prioritized+level+replay%2E+In+International+Conference+on+Machine+Learning%2C+pp%2E+4940%E2%80%934950%2E+PMLR%2C+2021%2E)
30. Lou Jost. Entropy and diversity. Oikos, 113(2):363–375, 2006. [find](https://scholar.google.com/scholar?q=Lou+Jost%2E+Entropy+and+diversity%2E+Oikos%2C+113%282%29%3A363%E2%80%93375%2C+2006%2E)
31. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL [https://openreview.net/forum?id=e…](https://openreview.net/forum?id=e…) [↗](https://openreview.net/forum?id=e…)
32. Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. arXiv preprint arXiv:2209.07753, 2022. [find](https://scholar.google.com/scholar?q=Jacky+Liang%2C+Wenlong+Huang%2C+Fei+Xia%2C+Peng+Xu%2C+Karol+Hausman%2C+Brian+Ichter%2C+Pete+Florence%2C+and+Andy+Zeng%2E+Code+as+policies%3A+Language+model+programs+for+embodied+control%2E+arXiv+preprint+arXiv%3A2209%2E07753)
33. Cam Linke, Nadia M Ady, Martha White, Thomas Degris, and Adam White. Adapting behavior via intrinsic reward: A survey and empirical study. Journal of artificial intelligence research, 69:1287–1332, 2020. [find](https://scholar.google.com/scholar?q=Cam+Linke%2C+Nadia+M+Ady%2C+Martha+White%2C+Thomas+Degris%2C+and+Adam+White%2E+Adapting+behavior+via+intrinsic+reward%3A+A+survey+and+empirical+study%2E+Journal+of+artificial+intelligence+research%2C+69%3A1287%E2%80%931332%2C+20)
34. Corey Lynch and Pierre Sermanet. [Language conditioned imitation learning over unstructured data](https://md.research.elamur.ai/free/en/papers/lynch-2020-language-conditioned-imitation-learning-over/). arXiv preprint arXiv:2005.07648, 2020.
35. Andrea Madotto, Mahdi Namazifar, Joost Huizinga, Piero Molino, Adrien Ecoffet, Huaixiu Zheng, Alexandros Papangelis, Dian Yu, Chandra Khatri, and Gokhan Tur. Exploration based language learning for text-based games. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Con… — [doi:10.24963/ijcai.2020/207](https://doi.org/10.24963/ijcai.2020/207) [find](https://scholar.google.com/scholar?q=Andrea+Madotto%2C+Mahdi+Namazifar%2C+Joost+Huizinga%2C+Piero+Molino%2C+Adrien+Ecoffet%2C+Huaixiu+Zheng%2C+Alexandros+Papangelis%2C+Dian+Yu%2C+Chandra+Khatri%2C+and+Gokhan+Tur%2E+Exploration+based+language+learning+for+te)
36. Sebastian Michelmann, Manoj Kumar, Kenneth A Norman, and Mariya Toneva. Large language models can segment narrative events similarly to humans. arXiv preprint arXiv:2301.10297, 2023. — [arXiv:2301.10297](https://arxiv.org/abs/2301.10297) [find](https://scholar.google.com/scholar?q=Sebastian+Michelmann%2C+Manoj+Kumar%2C+Kenneth+A+Norman%2C+and+Mariya+Toneva%2E+Large+language+models+can+segment+narrative+events+similarly+to+humans%2E+arXiv+preprint+arXiv%3A2301%2E10297%2C+2023%2E+%E2%80%94+%5BarXiv%3A2301%2E102)
37. Karthik Narasimhan, Tejas Kulkarni, and Regina Barzilay. [Language understanding for text-based games using deep reinforcement learning](https://md.research.elamur.ai/free/en/papers/narasimhan-2015-language-understanding-text-based-games/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1–11, Lisbon, Portugal, September 2015. Association for Computational…
38. Andrew Y Ng, Stuart Russell, et al. Algorithms for inverse reinforcement learning. In Icml, volume 1, pp. 2, 2000. [find](https://scholar.google.com/scholar?q=Andrew+Y+Ng%2C+Stuart+Russell%2C+et+al%2E+Algorithms+for+inverse+reinforcement+learning%2E+In+Icml%2C+volume+1%2C+pp%2E+2%2C+2000%2E)
39. Pierre-Yves Oudeyer and Frederic Kaplan. What is intrinsic motivation? a typology of computational approaches. Frontiers in neurorobotics, pp. 6, 2007. [find](https://scholar.google.com/scholar?q=Pierre%2DYves+Oudeyer+and+Frederic+Kaplan%2E+What+is+intrinsic+motivation%3F+a+typology+of+computational+approaches%2E+Frontiers+in+neurorobotics%2C+pp%2E+6%2C+2007%2E)
40. Vitchyr Pong, Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, and Sergey Levine. Skew-fit: State-covering self-supervised reinforcement learning. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine… [find](https://scholar.google.com/scholar?q=Vitchyr+Pong%2C+Murtaza+Dalal%2C+Steven+Lin%2C+Ashvin+Nair%2C+Shikhar+Bahl%2C+and+Sergey+Levine%2E+Skew%2Dfit%3A+State%2Dcovering+self%2Dsupervised+reinforcement+learning%2E+In+Hal+Daume%CC%81+III+and+Aarti+Singh+%28eds%2E%29%2C+Procee)
41. Rémy Portelas, Cédric Colas, Lilian Weng, Katja Hofmann, and Pierre-Yves Oudeyer. Automatic curriculum learning for deep rl: A short survey. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 4819–4825. Internationa… [find](https://scholar.google.com/scholar?q=Re%CC%81my+Portelas%2C+Ce%CC%81dric+Colas%2C+Lilian+Weng%2C+Katja+Hofmann%2C+and+Pierre%2DYves+Oudeyer%2E+Automatic+curriculum+learning+for+deep+rl%3A+A+short+survey%2E+In+Christian+Bessiere+%28ed%2E%29%2C+Proceedings+of+the+Twenty%2DNi)
42. Nils Reimers and Iryna Gurevych. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3982–3992, Hong Kon…
43. Tom Schaul, Daniel Horgan, Karol Gregor, and David Silver. Universal value function approximators. In International conference on machine learning, pp. 1312–1320. PMLR, 2015. [find](https://scholar.google.com/scholar?q=Tom+Schaul%2C+Daniel+Horgan%2C+Karol+Gregor%2C+and+David+Silver%2E+Universal+value+function+approximators%2E+In+International+conference+on+machine+learning%2C+pp%2E+1312%E2%80%931320%2E+PMLR%2C+2015%2E)
44. Jürgen Schmidhuber. Powerplay: Training an increasingly general problem solver by continually searching for the simplest still unsolvable problem. Frontiers in psychology, 4:313, 2013. [find](https://scholar.google.com/scholar?q=Ju%CC%88rgen+Schmidhuber%2E+Powerplay%3A+Training+an+increasingly+general+problem+solver+by+continually+searching+for+the+simplest+still+unsolvable+problem%2E+Frontiers+in+psychology%2C+4%3A313%2C+2013%2E)
45. Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. [Progprompt: Generating situated robot task plans using large language models](https://md.research.elamur.ai/free/en/papers/singh-2022-progprompt-generating-situated-robot-task/). arXiv preprint arXiv:2209.11302, 2022.
46. Satinder Singh, Richard L Lewis, Andrew G Barto, and Jonathan Sorg. Intrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Transactions on Autonomous Mental Development, 2(2):70–82, 2010. [find](https://scholar.google.com/scholar?q=Satinder+Singh%2C+Richard+L+Lewis%2C+Andrew+G+Barto%2C+and+Jonathan+Sorg%2E+Intrinsically+motivated+reinforcement+learning%3A+An+evolutionary+perspective%2E+IEEE+Transactions+on+Autonomous+Mental+Development%2C+2%282)
47. Richard S Sutton, Andrew G Barto, et al. Introduction to reinforcement learning, volume 135. MIT press Cambridge, 1998. [find](https://scholar.google.com/scholar?q=Richard+S+Sutton%2C+Andrew+G+Barto%2C+et+al%2E+Introduction+to+reinforcement+learning%2C+volume+135%2E+MIT+press+Cambridge%2C+1998%2E)
48. Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, et al. [Human-timescale adaptation in an open-ended task space](https://md.research.elamur.ai/free/en/papers/team-2023-human-timescale-adaptation-open-ended/). arXiv preprint arXiv:2301.07608, 2023.
49. Claudio Tennie, Josep Call, and Michael Tomasello. Ratcheting up the ratchet: on the evolution of cumulative culture. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1528):2405–2415, 2009. [find](https://scholar.google.com/scholar?q=Claudio+Tennie%2C+Josep+Call%2C+and+Michael+Tomasello%2E+Ratcheting+up+the+ratchet%3A+on+the+evolution+of+cumulative+culture%2E+Philosophical+Transactions+of+the+Royal+Society+B%3A+Biological+Sciences%2C+364%281528%29%3A)
50. Rui Wang, Joel Lehman, Aditya Rawal, Jiale Zhi, Yulun Li, Jeffrey Clune, and Kenneth Stanley. Enhanced poet: Open-ended reinforcement learning through unbounded invention of learning challenges and their solutions. In International Conference on Machine Learning, pp. 9940–9951. PMLR, 2020. [find](https://scholar.google.com/scholar?q=Rui+Wang%2C+Joel+Lehman%2C+Aditya+Rawal%2C+Jiale+Zhi%2C+Yulun+Li%2C+Jeffrey+Clune%2C+and+Kenneth+Stanley%2E+Enhanced+poet%3A+Open%2Dended+reinforcement+learning+through+unbounded+invention+of+learning+challenges+and+th)
51. Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma, and Yitao Liang. [Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents](https://md.research.elamur.ai/free/en/papers/wang-2023-describe-explain-plan-select-interactive/). arXiv preprint arXiv:2302.01560, 2023.
52. David Warde-Farley, Tom Van de Wiele, Tejas Kulkarni, Catalin Ionescu, Steven Hansen, and Volodymyr Mnih. Unsupervised control through non-parametric discriminative rewards. arXiv preprint arXiv:1811.11359, 2018. — [arXiv:1811.11359](https://arxiv.org/abs/1811.11359) [find](https://scholar.google.com/scholar?q=David+Warde%2DFarley%2C+Tom+Van+de+Wiele%2C+Tejas+Kulkarni%2C+Catalin+Ionescu%2C+Steven+Hansen%2C+and+Volodymyr+Mnih%2E+Unsupervised+control+through+non%2Dparametric+discriminative+rewards%2E+arXiv+preprint+arXiv%3A1811%2E)
53. Sandra R Waxman and Dana B Markow. Words as invitations to form categories: Evidence from 12-to 13-month-old infants. Cognitive psychology, 29(3):257–302, 1995. [find](https://scholar.google.com/scholar?q=Sandra+R+Waxman+and+Dana+B+Markow%2E+Words+as+invitations+to+form+categories%3A+Evidence+from+12%2Dto+13%2Dmonth%2Dold+infants%2E+Cognitive+psychology%2C+29%283%29%3A257%E2%80%93302%2C+1995%2E)
54. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Proce… — [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
55. Ted Xiao, Harris Chan, Pierre Sermanet, Ayzaan Wahid, Anthony Brohan, Karol Hausman, Sergey Levine, and Jonathan Tompson. Robotic skill acquisition via instruction augmentation with vision-language models. arXiv preprint arXiv:2211.11736, 2022. — [arXiv:2211.11736](https://arxiv.org/abs/2211.11736) [find](https://scholar.google.com/scholar?q=Ted+Xiao%2C+Harris+Chan%2C+Pierre+Sermanet%2C+Ayzaan+Wahid%2C+Anthony+Brohan%2C+Karol+Hausman%2C+Sergey+Levine%2C+and+Jonathan+Tompson%2E+Robotic+skill+acquisition+via+instruction+augmentation+with+vision%2Dlanguage+mo)
56. Shunyu Yao, Rohan Rao, Matthew Hausknecht, and Karthik Narasimhan. [Keep CALM and explore: Language models for action generation in text-based games](https://md.research.elamur.ai/free/en/papers/yao-2020-keep-calm-explore-language-models/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 8736–8754, Online, November 2020. Association fo…

*Updated 3 November 2026*