---
title: "Open-Endedness is Essential for Artificial Superhuman Intelligence"
description: "The authors propose a formal definition that depends on the observer. 141 references, cited by 11 papers here."
authors: ["Edward Hughes", "Michael Dennis", "Jack Parker-Holder", "Feryal Behbahani", "Aditi Mavalankar", "Yuge Shi", "Tom Schaul", "Tim Rocktaschel"]
year: 2024
venue: "arXiv:2406.04268"
source: "https://arxiv.org/abs/2406.04268"
doi: "10.48550/arxiv.2406.04268"
arxiv: "2406.04268"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hughes-2024-open-endedness-essential-artificial-superhuman/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hughes-2024-open-endedness-essential-artificial-superhuman/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/) / Open-Endedness is Essential for Artificial Superhuman Intelligence

# Open-Endedness is Essential for Artificial Superhuman Intelligence

**Authors:** [Edward Hughes](https://md.research.elamur.ai/free/en/authors/edward-hughes/), Michael Dennis, Jack Parker-Holder, [Feryal Behbahani](https://md.research.elamur.ai/free/en/authors/feryal-behbahani/), Aditi Mavalankar, Yuge Shi, [Tom Schaul](https://md.research.elamur.ai/free/en/authors/tom-schaul/) and [Tim Rocktaschel](https://md.research.elamur.ai/free/en/authors/tim-rocktaschel/) · **Year:** 2024 · **Venue:** arXiv:2406.04268 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2406.04268) · [PDF](https://arxiv.org/pdf/2406.04268) · [DOI 10.48550/arxiv.2406.04268](https://doi.org/10.48550/arxiv.2406.04268) · [arXiv 2406.04268](https://arxiv.org/abs/2406.04268) · [OpenAlex](https://openalex.org/W4399454695)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [11](https://md.research.elamur.ai/free/en/papers/hughes-2024-open-endedness-essential-artificial-superhuman/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This position paper from Google DeepMind (ICML 2024) asks what an AI system would need in order to keep generating new knowledge by itself. It argues that this capacity, open-endedness, is a prerequisite for artificial superhuman intelligence (ASI). No dataset, model or benchmark is released, so the contribution is conceptual. The authors propose a formal definition that depends on the observer. A system counts as open-ended for a given observer if its stream of artifacts keeps producing items that the observer's predictive model fails to anticipate (novelty), while a longer history still improves those predictions (learnability). The authors then argue that foundation models make practical open-ended systems feasible, because they encode human ideas of what is interesting and can serve as general-purpose generators of variations. They add that current foundation models are not yet open-ended. Since high-quality static web data is limited, they expect progress toward ASI to depend on systems that create and refine their own knowledge. The paper also discusses the safety and societal risks this raises.

## Key points
- **Formalism:** a system S emits artifacts X_t, and an observer O predicts future artifacts with a model X̂_t scored by a loss ℓ(t, T).
  - *Novelty:* for any fixed t, some later artifact always has higher expected loss.
  - *Learnability:* for a fixed target, expected loss falls as more history is observed.
- **Illustrations:**
  - A noisy TV with uniform noise is learnable but stops being novel once the model converges.
  - A TV whose channel is switched at random is novel but not learnable.
  - A figure shows that aircraft designs look different to a mouse, an aerospace student and a superintelligent alien.
- **Prior definitions:** the definition generalises the four necessary conditions of Soros and Stanley (2014) without requiring an evolutionary process. It overlaps substantially with Sigaud et al. (2023).
- **Motivation and directions:** the authors cite possible exhaustion of training data (Villalobos et al., 2022). Earlier open-ended successes they mention include Go, adaptation to 3D tasks (Bauer et al., 2023), the Minecraft tech tree (Wang et al., 2023a) and new mathematical results (Romera-Paredes et al., 2024). Proposed research directions pair foundation models with evolutionary algorithms and reinforcement learning.
- **Safety (Section 4):**
  - Humans must stay able to understand the generated artifacts, which makes explainability central.
  - Society needs to prepare for feedback-loop tipping points such as flash crashes.
  - Open-ended search should be steered toward artifacts that are useful, not merely safe.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 2,009 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 665 works

## Cite

Cite as: Edward Hughes, Michael Dennis, Jack Parker-Holder, Feryal Behbahani, Aditi Mavalankar, Yuge Shi, Tom Schaul and Tim Rocktaschel. 2024. Open-Endedness is Essential for Artificial Superhuman Intelligence. arXiv:2406.04268.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{hughes2024open,
  title = {{Open-Endedness is Essential for Artificial Superhuman Intelligence}},
  author = {Edward Hughes, Michael Dennis, Jack Parker-Holder, Feryal Behbahani, Aditi Mavalankar, Yuge Shi, Tom Schaul and Tim Rocktaschel},
  year = {2024},
  doi = {10.48550/arxiv.2406.04268},
  eprint = {2406.04268},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2406.04268}
}
```

- [← previous Neuro-Symbolic Data Generation for Math Reasoning](https://md.research.elamur.ai/free/en/papers/li-2024-neuro-symbolic-data-generation-math/)
- [next → Pantograph: A Machine-to-Machine Interaction Interface for Advanced Theorem Proving, High Level Reasoning, and Data Extraction in Lean 4](https://md.research.elamur.ai/free/en/papers/aniva-2024-pantograph-machine-machine-interaction-interface/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/)

## References

141 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. D. Abel, A. Barreto, B. Van Roy, D. Precup, H. van Hasselt, and S. Singh. A definition of continual reinforcement learning. ArXiv preprint, abs/2307.11046, 2023. URL https://arxiv.org/abs/2307.11046. — [arXiv:2307.11046](https://arxiv.org/abs/2307.11046) [↗](https://arxiv.org/abs/2307.11046)
2. M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman, A. Herzog, D. Ho, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, E. Jang, R. J. Ruano, K. Jeffrey, S. Jesmonth, N. J. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, K.-H. Lee, S. Levine, Y. Lu, L. Luu,… — [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://md.research.elamur.ai/free/en/papers/ahn-2022-i-i-say-grounding-language/)
3. K. Baumli, S. Baveja, F. Behbahani, H. Chan, G. Comanici, S. Flennerhag, M. Gazeau, K. Holsheimer, D. Horgan, M. Laskin, et al. Vision-language models as a source of rewards. ArXiv preprint, abs/2312.09187, 2023. URL https://arxiv.org/abs/2312.09187. — [arXiv:2312.09187](https://arxiv.org/abs/2312.09187) [↗](https://arxiv.org/abs/2312.09187)
4. M. Bedau. Measurement of evolutionary activity, teleology, and life. 1992. [find](https://scholar.google.com/scholar?q=M%2E+Bedau%2E+Measurement+of+evolutionary+activity%2C+teleology%2C+and+life%2E+1992%2E)
5. E. M. Aldrich, J. Grundfest, and G. Laughlin. The flash crash: A new deconstruction. Available at SSRN 2721922, 2017. [find](https://scholar.google.com/scholar?q=E%2E+M%2E+Aldrich%2C+J%2E+Grundfest%2C+and+G%2E+Laughlin%2E+The+flash+crash%3A+A+new+deconstruction%2E+Available+at+SSRN+2721922%2C+2017%2E)
6. M. A. Bedau, E. Snyder, C. T. Brown, N. H. Packard, et al. A comparison of evolutionary activity in artificial evolving D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schul-systems and in the biosphere. In Proceedings of the fourth European conference on artificial life, pages 125–134. MIt Pre… [find](https://scholar.google.com/scholar?q=M%2E+A%2E+Bedau%2C+E%2E+Snyder%2C+C%2E+T%2E+Brown%2C+N%2E+H%2E+Packard%2C+et+al%2E+A+comparison+of+evolutionary+activity+in+artificial+evolving+D%2E+Amodei%2C+C%2E+Olah%2C+J%2E+Steinhardt%2C+P%2E+Christiano%2C+J%2E+Schul%2Dsystems+and+in+the+bi)
7. N. Heess, L. Gonzalez, S. Osindero, S. Ozair, S. Reed, J. Zhang, K. Zolna, J. Clune, N. de Freitas, S. Singh, and T. Rocktäschel. Genie: Generative Interactive Environments, Feb. 2024. [find](https://scholar.google.com/scholar?q=N%2E+Heess%2C+L%2E+Gonzalez%2C+S%2E+Osindero%2C+S%2E+Ozair%2C+S%2E+Reed%2C+J%2E+Zhang%2C+K%2E+Zolna%2C+J%2E+Clune%2C+N%2E+de+Freitas%2C+S%2E+Singh%2C+and+T%2E+Rockta%CC%88schel%2E+Genie%3A+Generative+Interactive+Environments%2C+Feb%2E+2024%2E)
8. M. A. Bedau, E. Snyder, and N. H. Packard. A classification of long-term evolutionary dynamics. Artificial Life: The Proceedings..., page 228, 1998. [find](https://scholar.google.com/scholar?q=M%2E+A%2E+Bedau%2C+E%2E+Snyder%2C+and+N%2E+H%2E+Packard%2E+A+classification+of+long%2Dterm+evolutionary+dynamics%2E+Artificial+Life%3A+The+Proceedings%2E%2E%2E%2C+page+228%2C+1998%2E)
9. Y. Burda, H. Edwards, A. Storkey, and O. Klimov. [Exploration by Random Network Distillation](https://md.research.elamur.ai/free/en/papers/burda-2018-exploration-random-network-distillation/), Oct. 2018.
10. M. C. Campi and S. Garatti. Compression, generalization and learning. ArXiv preprint, abs/2301.12767, 2023. URL https://arxiv.org/abs/2301.12767. — [arXiv:2301.12767](https://arxiv.org/abs/2301.12767) [↗](https://arxiv.org/abs/2301.12767)
11. M. Bellemare, S. Srinivasan, G. Ostrovski, T. Schaul, D. Saxton, and R. Munos. Unifying count-based exploration and intrinsic motivation. Advances in neural information processing systems, 29, 2016. [find](https://scholar.google.com/scholar?q=M%2E+Bellemare%2C+S%2E+Srinivasan%2C+G%2E+Ostrovski%2C+T%2E+Schaul%2C+D%2E+Saxton%2C+and+R%2E+Munos%2E+Unifying+count%2Dbased+exploration+and+intrinsic+motivation%2E+Advances+in+neural+information+processing+systems%2C+29%2C+2016%2E)
12. R. Carey and T. Everitt. Human control: Definitions and algorithms. ArXiv preprint, abs/2305.19861, 2023. URL https://arxiv.org/abs/2305.19861. — [arXiv:2305.19861](https://arxiv.org/abs/2305.19861) [↗](https://arxiv.org/abs/2305.19861)
13. C. Berner, G. Brockman, B. Chan, V. Cheung, P. Debiak, C. Dennison, D. Farhi, Q. Fischer, S. Hashme, C. Hesse, et al. Dota 2 with large scale deep reinforcement learning. ArXiv preprint, abs/1912.06680, 2019. URL https://arxiv.org/abs/1912.06680. [↗](https://arxiv.org/abs/1912.06680)
14. H. Chan, V. Mnih, F. Behbahani, M. Laskin, L. Wang, F. Pardo, M. Gazeau, H. Sahni, D. Horgan, K. Baumli, Y. Schroecker, S. Spencer, R. Steigerwald, J. Quan, G. Comanici, S. Flennerhag, A. Neitz, L. M. Zhang, T. Schaul, S. Singh, C. Lyle, T. Rocktäschel, J. Parker-Holder, and K. Holsheimer. Vision-l… [find](https://scholar.google.com/scholar?q=H%2E+Chan%2C+V%2E+Mnih%2C+F%2E+Behbahani%2C+M%2E+Laskin%2C+L%2E+Wang%2C+F%2E+Pardo%2C+M%2E+Gazeau%2C+H%2E+Sahni%2C+D%2E+Horgan%2C+K%2E+Baumli%2C+Y%2E+Schroecker%2C+S%2E+Spencer%2C+R%2E+Steigerwald%2C+J%2E+Quan%2C+G%2E+Comanici%2C+S%2E+Flennerhag%2C+A%2E+Neitz%2C+L%2E+M%2E)
15. S. Bills, N. Cammarata, D. Mossing, H. Tillman, L. Gao, G. Goh, I. Sutskever, J. Leike, J. Wu, and W. Saunders. Language models can explain neurons in language models. URL [https://openaipublic.blob](https://openaipublic.blob). core. windows. net/neuron-explainer/paper/index. html.(Date accessed: 14.05. 2023), 2023. [↗](https://openaipublic.blob)
16. A. Chen, D. M. Dohan, and D. R. So. EvoPrompting: Language Models for Code-Level Neural Architecture Search, Feb. 2023a. [find](https://scholar.google.com/scholar?q=A%2E+Chen%2C+D%2E+M%2E+Dohan%2C+and+D%2E+R%2E+So%2E+EvoPrompting%3A+Language+Models+for+Code%2DLevel+Neural+Architecture+Search%2C+Feb%2E+2023a%2E)
17. R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al. [On the opportunities and risks of foundation models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/). arXiv preprint arXiv:2108.07258, 2021.
18. X. Chen, M. Lin, N. Schärli, and D. Zhou. Teaching Large Language Models to Self-Debug, Apr. 2023b. [find](https://scholar.google.com/scholar?q=X%2E+Chen%2C+M%2E+Lin%2C+N%2E+Scha%CC%88rli%2C+and+D%2E+Zhou%2E+Teaching+Large+Language+Models+to+Self%2DDebug%2C+Apr%2E+2023b%2E)
19. P. Christiano, A. Cotra, and M. Xu. Eliciting latent knowledge: How to tell if your eyes deceive you, 2021. [find](https://scholar.google.com/scholar?q=P%2E+Christiano%2C+A%2E+Cotra%2C+and+M%2E+Xu%2E+Eliciting+latent+knowledge%3A+How+to+tell+if+your+eyes+deceive+you%2C+2021%2E)
20. S. R. Bowman, J. Hyun, E. Perez, E. Chen, C. Pettit, S. Heiner, K. Lukošiūtė, A. Askell, A. Jones, A. Chen, et al. [Measuring progress on scalable oversight for large language models](https://md.research.elamur.ai/free/en/papers/bowman-2022-measuring-progress-scalable-oversight-large/). ArXiv preprint, abs/2211.03540, 2022. URL https://arxiv.org/abs/2211.03540. [↗](https://arxiv.org/abs/2211.03540)
21. J. Clark and D. Amodei. Faulty reward functions in the wild. Internet: [https://blog.openai](https://blog.openai). com/faulty-reward-functions, 2016. [↗](https://blog.openai)
22. J. Clune. AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence, Jan. 2020. [find](https://scholar.google.com/scholar?q=J%2E+Clune%2E+AI%2DGAs%3A+AI%2Dgenerating+algorithms%2C+an+alternate+paradigm+for+producing+general+artificial+intelligence%2C+Jan%2E+2020%2E)
23. H. Bradley, A. Dai, H. Teufel, J. Zhang, K. Oostermeijer, M. Bellagente, J. Clune, K. Stanley, G. Schott, and J. Lehman. [Quality-Diversity through AI Feedback](https://md.research.elamur.ai/free/en/papers/bradley-2023-quality-diversity-through-ai-feedback/), Oct. 2023.
24. J. Clune. Ai will go farterh if it stands on the shoulders of giant human data sets. Dec. 2022. [find](https://scholar.google.com/scholar?q=J%2E+Clune%2E+Ai+will+go+farterh+if+it+stands+on+the+shoulders+of+giant+human+data+sets%2E+Dec%2E+2022%2E)
25. J. C. Brant and K. O. Stanley. Minimal criterion coevolution: a new approach to open-ended search. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 67–74, 2017. [find](https://scholar.google.com/scholar?q=J%2E+C%2E+Brant+and+K%2E+O%2E+Stanley%2E+Minimal+criterion+coevolution%3A+a+new+approach+to+open%2Dended+search%2E+In+Proceedings+of+the+Genetic+and+Evolutionary+Computation+Conference%2C+pages+67%E2%80%9374%2C+2017%2E)
26. C. Colas, T. Karch, O. Sigaud, and P.-Y. Oudeyer. Autotelic agents with intrinsically motivated goal-conditioned reinforcement learning: a short survey. Journal of Artificial Intelligence Research, 74:1159–1199, 2022. [find](https://scholar.google.com/scholar?q=C%2E+Colas%2C+T%2E+Karch%2C+O%2E+Sigaud%2C+and+P%2E%2DY%2E+Oudeyer%2E+Autotelic+agents+with+intrinsically+motivated+goal%2Dconditioned+reinforcement+learning%3A+a+short+survey%2E+Journal+of+Artificial+Intelligence+Research%2C+74)
27. T. Brooks, B. Peebles, C. Holmes, W. DePue, Y. Guo, L. Jing, D. Schnurr, J. Taylor, T. Luhman, E. Luh-D. Collingridge. The Social Control of Technology. man, C. Ng, R. Wang, and A. Ramesh. Video St. Martin’s Press, 1980. ISBN 9780312731687. generation models as world simulators. 2024. URL https://bo… — [arXiv:2006.04948](https://arxiv.org/abs/2006.04948) [find](https://scholar.google.com/scholar?q=T%2E+Brooks%2C+B%2E+Peebles%2C+C%2E+Holmes%2C+W%2E+DePue%2C+Y%2E+Guo%2C+L%2E+Jing%2C+D%2E+Schnurr%2C+J%2E+Taylor%2C+T%2E+Luhman%2C+E%2E+Luh%2DD%2E+Collingridge%2E+The+Social+Control+of+Technology%2E+man%2C+C%2E+Ng%2C+R%2E+Wang%2C+and+A%2E+Ramesh%2E+Video+St%2E+M)
28. A. Dafoe, E. Hughes, Y. Bachrach, T. Collins, K. R. McKee, J. Z. Leibo, K. Larson, and T. Graepel. Open Problems in Cooperative AI, Dec. 2020. [find](https://scholar.google.com/scholar?q=A%2E+Dafoe%2C+E%2E+Hughes%2C+Y%2E+Bachrach%2C+T%2E+Collins%2C+K%2E+R%2E+McKee%2C+J%2E+Z%2E+Leibo%2C+K%2E+Larson%2C+and+T%2E+Graepel%2E+Open+Problems+in+Cooperative+AI%2C+Dec%2E+2020%2E)
29. Y. Du, S. Li, A. Torralba, J. B. Tenenbaum, and I. Mordatch. [Improving factuality and reasoning in language models through multiagent debate](https://md.research.elamur.ai/free/en/papers/du-2023-improving-factuality-reasoning-language-models/). arXiv preprint arXiv:2305.14325, 2023c.
30. O. David, S. Moran, and A. Yehudayoff. On statistical learning via the lens of compression. ArXiv preprint, abs/1610.03592, 2016. URL [https://arxiv.org/](https://arxiv.org/) abs/1610.03592. — [arXiv:1610.03592](https://arxiv.org/abs/1610.03592) [↗](https://arxiv.org/)
31. S. Earle, J. Togelius, and L. B. Soros. Video games as a testbed for open-ended phenomena. In 2021 IEEE Conference on Games (CoG), pages 1–9. [find](https://scholar.google.com/scholar?q=S%2E+Earle%2C+J%2E+Togelius%2C+and+L%2E+B%2E+Soros%2E+Video+games+as+a+testbed+for+open%2Dended+phenomena%2E+In+2021+IEEE+Conference+on+Games+%28CoG%29%2C+pages+1%E2%80%939%2E)
32. G. Delétang, A. Ruoss, P.-A. Duquenne, E. Catt, T. Genewein, C. Mattern, J. Grau-Moya, L. K. Wenliang, M. Aitchison, L. Orseau, M. Hutter, and J. Veness. [Language Modeling Is Compression](https://md.research.elamur.ai/free/en/papers/deletang-2023-language-modeling-compression/), Sept. 2023.
33. A. Ecoffet, J. Clune, and J. Lehman. Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity, June 2020. [find](https://scholar.google.com/scholar?q=A%2E+Ecoffet%2C+J%2E+Clune%2C+and+J%2E+Lehman%2E+Open+Questions+in+Creating+Safe+Open%2Dended+AI%3A+Tensions+Between+Control+and+Creativity%2C+June+2020%2E)
34. M. Faldor, J. Zhang, A. Cully, and J. Clune. [Omni-epic: Open-endedness via models of human notions of interestingness with environments programmed in code](https://md.research.elamur.ai/free/en/papers/faldor-2024-omni-epic-open-endedness-models/). arXiv preprint arXiv:2405.15568, 2024.
35. M. Dennis, N. Jaques, E. Vinitsky, A. M. Bayen, S. Russell, A. Critch, and S. Levine. Emergent complexity and zero-shot transfer via unsupervised environment design. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems 33: Annual… [find](https://scholar.google.com/scholar?q=M%2E+Dennis%2C+N%2E+Jaques%2C+E%2E+Vinitsky%2C+A%2E+M%2E+Bayen%2C+S%2E+Russell%2C+A%2E+Critch%2C+and+S%2E+Levine%2E+Emergent+complexity+and+zero%2Dshot+transfer+via+unsupervised+environment+design%2E+In+H%2E+Larochelle%2C+M%2E+Ranzato%2C+R%2E+H)
36. C. Fernando, D. Banarse, H. Michalewski, S. Osindero, and T. Rocktäschel. [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://md.research.elamur.ai/free/en/papers/fernando-2023-promptbreeder-self-referential-self-improvement/), Sept. 2023.
37. K. Gandhi, D. Sadigh, and N. D. Goodman. [Strategic Reasoning with Language Models](https://md.research.elamur.ai/free/en/papers/gandhi-2023-strategic-reasoning-language-models/), May 2023.
38. J. Derbyshire. Potential surprise theory as a theoretical foundation for scenario planning. Technological Forecasting and Social Change, 124:77–87, 2017. [find](https://scholar.google.com/scholar?q=J%2E+Derbyshire%2E+Potential+surprise+theory+as+a+theoretical+foundation+for+scenario+planning%2E+Technological+Forecasting+and+Social+Change%2C+124%3A77%E2%80%9387%2C+2017%2E)
39. J. Garcıa and F. Fernández. A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 16(1):1437–1480, 2015. [find](https://scholar.google.com/scholar?q=J%2E+Garc%C4%B1a+and+F%2E+Ferna%CC%81ndez%2E+A+comprehensive+survey+on+safe+reinforcement+learning%2E+Journal+of+Machine+Learning+Research%2C+16%281%29%3A1437%E2%80%931480%2C+2015%2E)
40. D. Deutsch. The beginning of infinity: Explanations that transform the world. Penguin UK, 2011. [find](https://scholar.google.com/scholar?q=D%2E+Deutsch%2E+The+beginning+of+infinity%3A+Explanations+that+transform+the+world%2E+Penguin+UK%2C+2011%2E)
41. L. L. di Langosco, J. Koch, L. D. Sharkey, J. Pfau, and D. Krueger. Goal misgeneralization in deep reinforcement learning. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvári, G. Niu, and S. Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryla… [find](https://scholar.google.com/scholar?q=L%2E+L%2E+di+Langosco%2C+J%2E+Koch%2C+L%2E+D%2E+Sharkey%2C+J%2E+Pfau%2C+and+D%2E+Krueger%2E+Goal+misgeneralization+in+deep+reinforcement+learning%2E+In+K%2E+Chaudhuri%2C+S%2E+Jegelka%2C+L%2E+Song%2C+C%2E+Szepesva%CC%81ri%2C+G%2E+Niu%2C+and+S%2E+Sabato%2C+)
42. PMLR, 2022. URL [https://proceedings.mlr.press/v162/langosco22a.html](https://proceedings.mlr.press/v162/langosco22a.html). [↗](https://proceedings.mlr.press/v162/langosco22a.html)
43. Z. Gou, Z. Shao, Y. Gong, Y. Shen, Y. Yang, N. Duan, and W. Chen. [Critic: Large language models can self-correct with tool-interactive critiquing](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/). ArXiv preprint, abs/2305.11738, 2023. URL [https://arxiv.org/](https://arxiv.org/) abs/2305.11738. [↗](https://arxiv.org/)
44. Q. Guo, R. Wang, J. Guo, B. Li, K. Song, X. Tan, G. Liu, J. Bian, and Y. Yang. Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers, Sept. 2023. [find](https://scholar.google.com/scholar?q=Q%2E+Guo%2C+R%2E+Wang%2C+J%2E+Guo%2C+B%2E+Li%2C+K%2E+Song%2C+X%2E+Tan%2C+G%2E+Liu%2C+J%2E+Bian%2C+and+Y%2E+Yang%2E+Connecting+Large+Language+Models+with+Evolutionary+Algorithms+Yields+Powerful+Prompt+Optimizers%2C+Sept%2E+2023%2E)
45. I. Gur, N. Jaques, Y. Miao, J. Choi, M. Tiwari, H. Lee, and A. Faust. Environment generation for zero-shot compositional reinforcement learning. Advances in Neural Information Processing Systems, 34:4157–4169, 2021. [find](https://scholar.google.com/scholar?q=I%2E+Gur%2C+N%2E+Jaques%2C+Y%2E+Miao%2C+J%2E+Choi%2C+M%2E+Tiwari%2C+H%2E+Lee%2C+and+A%2E+Faust%2E+Environment+generation+for+zero%2Dshot+compositional+reinforcement+learning%2E+Advances+in+Neural+Information+Processing+Systems%2C+34%3A4)
46. E. L. Dolson, A. E. Vostinar, M. J. Wiser, and C. Ofria. The modes toolbox: Measurements of open-ended dynamics in evolving systems. Artificial life, 25(1):50–73, 2019. [find](https://scholar.google.com/scholar?q=E%2E+L%2E+Dolson%2C+A%2E+E%2E+Vostinar%2C+M%2E+J%2E+Wiser%2C+and+C%2E+Ofria%2E+The+modes+toolbox%3A+Measurements+of+open%2Dended+dynamics+in+evolving+systems%2E+Artificial+life%2C+25%281%29%3A50%E2%80%9373%2C+2019%2E)
47. Y. Du, K. Konyushkova, M. Denil, A. Raju, J. Landon, F. Hill, N. de Freitas, and S. Cabi. Vision-language models as success detectors. In Proceedings of The 2nd Conference on Lifelong Learning Agents, pages 120–136, 2023a. [find](https://scholar.google.com/scholar?q=Y%2E+Du%2C+K%2E+Konyushkova%2C+M%2E+Denil%2C+A%2E+Raju%2C+J%2E+Landon%2C+F%2E+Hill%2C+N%2E+de+Freitas%2C+and+S%2E+Cabi%2E+Vision%2Dlanguage+models+as+success+detectors%2E+In+Proceedings+of+The+2nd+Conference+on+Lifelong+Learning+Agents%2C)
48. W. Gurnee and M. Tegmark. Language Models Represent Space and Time, Oct. 2023. [find](https://scholar.google.com/scholar?q=W%2E+Gurnee+and+M%2E+Tegmark%2E+Language+Models+Represent+Space+and+Time%2C+Oct%2E+2023%2E)
49. D. Hadfield-Menell, S. J. Russell, P. Abbeel, and A. D. Dragan. Cooperative inverse reinforcement learning. In D. D. Lee, M. Sugiyama, U. von Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2… [find](https://scholar.google.com/scholar?q=D%2E+Hadfield%2DMenell%2C+S%2E+J%2E+Russell%2C+P%2E+Abbeel%2C+and+A%2E+D%2E+Dragan%2E+Cooperative+inverse+reinforcement+learning%2E+In+D%2E+D%2E+Lee%2C+M%2E+Sugiyama%2C+U%2E+von+Luxburg%2C+I%2E+Guyon%2C+and+R%2E+Garnett%2C+editors%2C+Advances+in+Ne)
50. Y. Du, E. Kosoy, A. Dayan, M. Rufova, P. Abbeel, and A. Gopnik. What can ai learn from human exploration? intrinsically-motivated humans and agents in open-world exploration. In NeurIPS 2023 workshop: Information-Theoretic Principles in Cognitive Systems, 2023b. [find](https://scholar.google.com/scholar?q=Y%2E+Du%2C+E%2E+Kosoy%2C+A%2E+Dayan%2C+M%2E+Rufova%2C+P%2E+Abbeel%2C+and+A%2E+Gopnik%2E+What+can+ai+learn+from+human+exploration%3F+intrinsically%2Dmotivated+humans+and+agents+in+open%2Dworld+exploration%2E+In+NeurIPS+2023+workshop%3A)
51. D. Hadfield-Menell, A. D. Dragan, P. Abbeel, and S. J. Russell. The off-switch game. In C. Sierra, editor, Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, pages 220–227. ijcai.org, 2017. doi: 10.24963/ij… — [doi:10.24963/ijcai.2017/32](https://doi.org/10.24963/ijcai.2017/32) [find](https://scholar.google.com/scholar?q=D%2E+Hadfield%2DMenell%2C+A%2E+D%2E+Dragan%2C+P%2E+Abbeel%2C+and+S%2E+J%2E+Russell%2E+The+off%2Dswitch+game%2E+In+C%2E+Sierra%2C+editor%2C+Proceedings+of+the+Twenty%2DSixth+International+Joint+Conference+on+Artificial+Intelligence%2C+IJ)
52. M. Klissarov, P. D’Oro, S. Sodhani, R. Raileanu, P.-L. Bacon, P. Vincent, A. Zhang, and M. Henaff. Motif: Intrinsic Motivation from Artificial Intelligence Feedback, Sept. 2023. [find](https://scholar.google.com/scholar?q=M%2E+Klissarov%2C+P%2E+D%E2%80%99Oro%2C+S%2E+Sodhani%2C+R%2E+Raileanu%2C+P%2E%2DL%2E+Bacon%2C+P%2E+Vincent%2C+A%2E+Zhang%2C+and+M%2E+Henaff%2E+Motif%3A+Intrinsic+Motivation+from+Artificial+Intelligence+Feedback%2C+Sept%2E+2023%2E)
53. F. H. Knight. Risk, uncertainty and profit, volume 31. Houghton Mifflin, 1921. [find](https://scholar.google.com/scholar?q=F%2E+H%2E+Knight%2E+Risk%2C+uncertainty+and+profit%2C+volume+31%2E+Houghton+Mifflin%2C+1921%2E)
54. E. Hazan and S. Kale. Extracting certainty from uncertainty: Regret bounded by variation in costs. Machine learning, 80:165–188, 2010. [find](https://scholar.google.com/scholar?q=E%2E+Hazan+and+S%2E+Kale%2E+Extracting+certainty+from+uncertainty%3A+Regret+bounded+by+variation+in+costs%2E+Machine+learning%2C+80%3A165%E2%80%93188%2C+2010%2E)
55. V. Krakovna, L. Orseau, R. Kumar, M. Martic, and S. Legg. Penalizing side effects using stepwise relative reachability. ArXiv preprint, abs/1806.01186, 2018. URL https://arxiv.org/abs/1806.01186. — [arXiv:1806.01186](https://arxiv.org/abs/1806.01186) [↗](https://arxiv.org/abs/1806.01186)
56. M. Henaff, R. Raileanu, M. Jiang, and T. Rocktäschel. Exploration via Elliptical Episodic Bonuses, Jan. 2023. [find](https://scholar.google.com/scholar?q=M%2E+Henaff%2C+R%2E+Raileanu%2C+M%2E+Jiang%2C+and+T%2E+Rockta%CC%88schel%2E+Exploration+via+Elliptical+Episodic+Bonuses%2C+Jan%2E+2023%2E)
57. S. Legg and M. Hutter. Universal Intelligence: A Definition of Machine Intelligence, Dec. 2007. [find](https://scholar.google.com/scholar?q=S%2E+Legg+and+M%2E+Hutter%2E+Universal+Intelligence%3A+A+Definition+of+Machine+Intelligence%2C+Dec%2E+2007%2E)
58. J. H. Holland. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT press, 1992. [find](https://scholar.google.com/scholar?q=J%2E+H%2E+Holland%2E+Adaptation+in+natural+and+artificial+systems%3A+an+introductory+analysis+with+applications+to+biology%2C+control%2C+and+artificial+intelligence%2E+MIT+press%2C+1992%2E)
59. J. Lehman and K. O. Stanley. Abandoning Objectives: Evolution Through the Search for Novelty Alone. Evolutionary Computation, 19(2):189–223, June 2011. ISSN 1063-6560. doi: 10.1162/EVCO a 00025. — [doi:10.1162/evco](https://doi.org/10.1162/evco) [find](https://scholar.google.com/scholar?q=J%2E+Lehman+and+K%2E+O%2E+Stanley%2E+Abandoning+Objectives%3A+Evolution+Through+the+Search+for+Novelty+Alone%2E+Evolutionary+Computation%2C+19%282%29%3A189%E2%80%93223%2C+June+2011%2E+ISSN+1063%2D6560%2E+doi%3A+10%2E1162%2FEVCO+a+00025%2E+%E2%80%94+%5Bdo)
60. A. Hu, L. Russell, H. Yeo, Z. Murez, G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado. GAIA-1: A Generative World Model for Autonomous Driving, Sept. 2023. [find](https://scholar.google.com/scholar?q=A%2E+Hu%2C+L%2E+Russell%2C+H%2E+Yeo%2C+Z%2E+Murez%2C+G%2E+Fedoseev%2C+A%2E+Kendall%2C+J%2E+Shotton%2C+and+G%2E+Corrado%2E+GAIA%2D1%3A+A+Generative+World+Model+for+Autonomous+Driving%2C+Sept%2E+2023%2E)
61. J. Lehman, J. Gordon, S. Jain, K. Ndousse, C. Yeh, and K. O. Stanley. Evolution through Large Models, June 2022. [find](https://scholar.google.com/scholar?q=J%2E+Lehman%2C+J%2E+Gordon%2C+S%2E+Jain%2C+K%2E+Ndousse%2C+C%2E+Yeh%2C+and+K%2E+O%2E+Stanley%2E+Evolution+through+Large+Models%2C+June+2022%2E)
62. J. Huang, S. S. Gu, L. Hou, Y. Wu, X. Wang, H. Yu, and J. Han. [Large Language Models Can Self-Improve](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/), Oct. 2022.
63. J. Z. Leibo, E. Hughes, M. Lanctot, and T. Graepel. Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research, Mar. 2019. [find](https://scholar.google.com/scholar?q=J%2E+Z%2E+Leibo%2C+E%2E+Hughes%2C+M%2E+Lanctot%2C+and+T%2E+Graepel%2E+Autocurricula+and+the+Emergence+of+Innovation+from+Social+Interaction%3A+A+Manifesto+for+Multi%2DAgent+Intelligence+Research%2C+Mar%2E+2019%2E)
64. S. Lifshitz, K. Paster, H. Chan, J. Ba, and S. McIlraith. STEVE-1: A Generative Model for Text-to-Behavior in Minecraft, June 2023. [find](https://scholar.google.com/scholar?q=S%2E+Lifshitz%2C+K%2E+Paster%2C+H%2E+Chan%2C+J%2E+Ba%2C+and+S%2E+McIlraith%2E+STEVE%2D1%3A+A+Generative+Model+for+Text%2Dto%2DBehavior+in+Minecraft%2C+June+2023%2E)
65. P. Huang, X. Zhang, Z. Cao, S. Liu, M. Xu, W. Ding, J. Francis, B. Chen, and D. Zhao. What went wrong? closing the sim-to-real gap via differentiable causal discovery. In Conference on Robot Learning, pages 734–760. PMLR, 2023. [find](https://scholar.google.com/scholar?q=P%2E+Huang%2C+X%2E+Zhang%2C+Z%2E+Cao%2C+S%2E+Liu%2C+M%2E+Xu%2C+W%2E+Ding%2C+J%2E+Francis%2C+B%2E+Chen%2C+and+D%2E+Zhao%2E+What+went+wrong%3F+closing+the+sim%2Dto%2Dreal+gap+via+differentiable+causal+discovery%2E+In+Conference+on+Robot+Learning%2C)
66. S. Liu, C. Chen, X. Qu, K. Tang, and Y.-S. Ong. Large language models as evolutionary optimizers. arXiv preprint arXiv:2310.19046, 2023a. — [arXiv:2310.19046](https://arxiv.org/abs/2310.19046) [find](https://scholar.google.com/scholar?q=S%2E+Liu%2C+C%2E+Chen%2C+X%2E+Qu%2C+K%2E+Tang%2C+and+Y%2E%2DS%2E+Ong%2E+Large+language+models+as+evolutionary+optimizers%2E+arXiv+preprint+arXiv%3A2310%2E19046%2C+2023a%2E+%E2%80%94+%5BarXiv%3A2310%2E19046%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E19046%29)
67. M. Hutter. Universal artificial intelligence: Sequential decisions based on algorithmic probability. Springer Science & Business Media, 2004. [find](https://scholar.google.com/scholar?q=M%2E+Hutter%2E+Universal+artificial+intelligence%3A+Sequential+decisions+based+on+algorithmic+probability%2E+Springer+Science+%26+Business+Media%2C+2004%2E)
68. X. Liu, H. Yu, H. Zhang, Y. Xu, X. Lei, H. Lai, Y. Gu, H. Ding, K. Men, K. Yang, S. Zhang, X. Deng, A. Zeng, Z. Du, C. Zhang, S. Shen, T. Zhang, Y. Su, H. Sun, M. Huang, Y. Dong, and J. Tang. [AgentBench: Evaluating LLMs as Agents](https://md.research.elamur.ai/free/en/papers/liu-2023-agentbench-evaluating-llms-agents/), Aug. 2023b.
69. G. Irving, P. Christiano, and D. Amodei. [AI safety via debate](https://md.research.elamur.ai/free/en/papers/irving-2018-ai-safety-debate/), Oct. 2018.
70. M. Jiang, E. Grefenstette, and T. Rocktäschel. Prioritized Level Replay, June 2021. [find](https://scholar.google.com/scholar?q=M%2E+Jiang%2C+E%2E+Grefenstette%2C+and+T%2E+Rockta%CC%88schel%2E+Prioritized+Level+Replay%2C+June+2021%2E)
71. Y. J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayaraman, Y. Zhu, L. Fan, and A. Anandkumar. [Eureka: Human-Level Reward Design via Coding Large Language Models](https://md.research.elamur.ai/free/en/papers/ma-2023-eureka-human-level-reward-design/), Oct. 2023.
72. M. Jiang, T. Rocktäschel, and E. Grefenstette. General Intelligence Requires Rethinking Exploration, Nov. 2022. [find](https://scholar.google.com/scholar?q=M%2E+Jiang%2C+T%2E+Rockta%CC%88schel%2C+and+E%2E+Grefenstette%2E+General+Intelligence+Requires+Rethinking+Exploration%2C+Nov%2E+2022%2E)
73. A. N. Mavor-Parker, K. A. Young, C. Barry, and L. D. Griffin. How to stay curious while avoiding noisy tvs using aleatoric uncertainty estimation. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvári, G. Niu, and S. Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July… [find](https://scholar.google.com/scholar?q=A%2E+N%2E+Mavor%2DParker%2C+K%2E+A%2E+Young%2C+C%2E+Barry%2C+and+L%2E+D%2E+Griffin%2E+How+to+stay+curious+while+avoiding+noisy+tvs+using+aleatoric+uncertainty+estimation%2E+In+K%2E+Chaudhuri%2C+S%2E+Jegelka%2C+L%2E+Song%2C+C%2E+Szepesva%CC%81ri%2C)
74. PMLR, 2022. URL [https://proceedings.mlr.press/](https://proceedings.mlr.press/) v162/mavor-parker22a.html. [↗](https://proceedings.mlr.press/)
75. M. B. Johanson, E. Hughes, F. Timbers, and J. Z. Leibo. Emergent bartering behaviour in multi-agent reinforcement learning. ArXiv preprint, abs/2205.06760, 2022. URL https://arxiv.org/abs/2205.06760. — [arXiv:2205.06760](https://arxiv.org/abs/2205.06760) [↗](https://arxiv.org/abs/2205.06760)
76. N. Justesen, R. R. Torrado, P. Bontrager, A. Khalifa, J. Togelius, and S. Risi. Illuminating generalization in deep reinforcement learning through procedural level generation. arXiv preprint arXiv:1806.10729, 2018. — [arXiv:1806.10729](https://arxiv.org/abs/1806.10729) [find](https://scholar.google.com/scholar?q=N%2E+Justesen%2C+R%2E+R%2E+Torrado%2C+P%2E+Bontrager%2C+A%2E+Khalifa%2C+J%2E+Togelius%2C+and+S%2E+Risi%2E+Illuminating+generalization+in+deep+reinforcement+learning+through+procedural+level+generation%2E+arXiv+preprint+arXiv%3A180)
77. D. W. McShea. Perspective metazoan complexity and evolution: is there a trend? Evolution, 50(2):477–492, 1996. and S. Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 174… [find](https://scholar.google.com/scholar?q=D%2E+W%2E+McShea%2E+Perspective+metazoan+complexity+and+evolution%3A+is+there+a+trend%3F+Evolution%2C+50%282%29%3A477%E2%80%93492%2C+1996%2E+and+S%2E+Sabato%2C+editors%2C+International+Conference+on+Machine+Learning%2C+ICML+2022%2C+17%2D23+Ju)
78. PMLR, 2022. URL [https://proceedings.mlr.press/](https://proceedings.mlr.press/) v162/parker-holder22a.html. [↗](https://proceedings.mlr.press/)
79. E. Meyerson, M. J. Nelson, H. Bradley, A. Moradi, A. K. Hoover, and J. Lehman. Language Model Crossover: Variation through Few-Shot Prompting, Feb. 2023. [find](https://scholar.google.com/scholar?q=E%2E+Meyerson%2C+M%2E+J%2E+Nelson%2C+H%2E+Bradley%2C+A%2E+Moradi%2C+A%2E+K%2E+Hoover%2C+and+J%2E+Lehman%2E+Language+Model+Crossover%3A+Variation+through+Few%2DShot+Prompting%2C+Feb%2E+2023%2E)
80. S. Mirchandani, F. Xia, P. Florence, B. Ichter, D. Driess, M. G. Arenas, K. Rao, D. Sadigh, and A. Zeng. [Large Language Models as General Pattern Machines](https://md.research.elamur.ai/free/en/papers/mirchandani-2023-large-language-models-general-pattern/), July 2023.
81. D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell. Curiosity-driven exploration by self-supervised prediction. In D. Precup and Y. W. Teh, editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of… [find](https://scholar.google.com/scholar?q=D%2E+Pathak%2C+P%2E+Agrawal%2C+A%2E+A%2E+Efros%2C+and+T%2E+Darrell%2E+Curiosity%2Ddriven+exploration+by+self%2Dsupervised+prediction%2E+In+D%2E+Precup+and+Y%2E+W%2E+Teh%2C+editors%2C+Proceedings+of+the+34th+International+Conference+on)
82. PMLR, 2017. URL [http://proceedings.mlr.press/](http://proceedings.mlr.press/) v70/pathak17a.html. [↗](http://proceedings.mlr.press/)
83. I. Momennejad, H. Hasanbeig, F. Vieira Frujeri, H. Sharma, N. Jojic, H. Palangi, R. Ness, and J. Larson. Evaluating cognitive maps and planning in large language models with cogeval. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=I%2E+Momennejad%2C+H%2E+Hasanbeig%2C+F%2E+Vieira+Frujeri%2C+H%2E+Sharma%2C+N%2E+Jojic%2C+H%2E+Palangi%2C+R%2E+Ness%2C+and+J%2E+Larson%2E+Evaluating+cognitive+maps+and+planning+in+large+language+models+with+cogeval%2E+Advances+in+Neura)
84. J. Perolat, B. De Vylder, D. Hennes, E. Tarassov, F. Strub, V. de Boer, P. Muller, J. T. Connor, N. Burch, T. Anthony, et al. Mastering the game of stratego with model-free multiagent reinforcement learning. Science, 378(6623): 990–996, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Perolat%2C+B%2E+De+Vylder%2C+D%2E+Hennes%2C+E%2E+Tarassov%2C+F%2E+Strub%2C+V%2E+de+Boer%2C+P%2E+Muller%2C+J%2E+T%2E+Connor%2C+N%2E+Burch%2C+T%2E+Anthony%2C+et+al%2E+Mastering+the+game+of+stratego+with+model%2Dfree+multiagent+reinforcement+le)
85. M. R. Morris, J. Sohl-dickstein, N. Fiedel, T. Warkentin, A. Dafoe, A. Faust, C. Farabet, and S. Legg. Levels of AGI: Operationalizing Progress on the Path to AGI, Nov. 2023. [find](https://scholar.google.com/scholar?q=M%2E+R%2E+Morris%2C+J%2E+Sohl%2Ddickstein%2C+N%2E+Fiedel%2C+T%2E+Warkentin%2C+A%2E+Dafoe%2C+A%2E+Faust%2C+C%2E+Farabet%2C+and+S%2E+Legg%2E+Levels+of+AGI%3A+Operationalizing+Progress+on+the+Path+to+AGI%2C+Nov%2E+2023%2E)
86. J.-B. Mouret and J. Clune. Illuminating search spaces by mapping elites. ArXiv preprint, abs/1504.04909, 2015. URL https://arxiv.org/abs/1504.04909. — [arXiv:1504.04909](https://arxiv.org/abs/1504.04909) [↗](https://arxiv.org/abs/1504.04909)
87. J. K. Pugh, L. B. Soros, and K. O. Stanley. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, 3:40, 2016. ISSN 2296-9144. doi: 10.3389/frobt.2016.00040. — [doi:10.3389/frobt.2016.00040](https://doi.org/10.3389/frobt.2016.00040) [find](https://scholar.google.com/scholar?q=J%2E+K%2E+Pugh%2C+L%2E+B%2E+Soros%2C+and+K%2E+O%2E+Stanley%2E+Quality+diversity%3A+A+new+frontier+for+evolutionary+computation%2E+Frontiers+in+Robotics+and+AI%2C+3%3A40%2C+2016%2E+ISSN+2296%2D9144%2E+doi%3A+10%2E3389%2Ffrobt%2E2016%2E00040%2E+%E2%80%94+%5B)
88. OEL Team, A. Stooke, A. Mahajan, C. Barros, C. Deck, J. Bauer, J. Sygnowski, M. Trebacz, M. Jaderberg, M. Mathieu, N. McAleese, N. Bradley-Schmieg, N. Wong, N. Porcel, R. Raileanu, S. Hughes-Fitt, V. Dalibard, and W. M. Czarnecki. Open-ended learning leads to generally capable agents. ArXiv preprint… — [arXiv:2107.12808](https://arxiv.org/abs/2107.12808) [find](https://scholar.google.com/scholar?q=OEL+Team%2C+A%2E+Stooke%2C+A%2E+Mahajan%2C+C%2E+Barros%2C+C%2E+Deck%2C+J%2E+Bauer%2C+J%2E+Sygnowski%2C+M%2E+Trebacz%2C+M%2E+Jaderberg%2C+M%2E+Mathieu%2C+N%2E+McAleese%2C+N%2E+Bradley%2DSchmieg%2C+N%2E+Wong%2C+N%2E+Porcel%2C+R%2E+Raileanu%2C+S%2E+Hughes%2DFitt%2C+V%2E+)
89. R. Raileanu and T. Rocktäschel. RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments, Feb. 2020. [find](https://scholar.google.com/scholar?q=R%2E+Raileanu+and+T%2E+Rockta%CC%88schel%2E+RIDE%3A+Rewarding+Impact%2DDriven+Exploration+for+Procedurally%2DGenerated+Environments%2C+Feb%2E+2020%2E)
90. P. J. Richerson, R. Boyd, et al. Institutional evolution in the holocene: the rise of complex societies. In Proceedings-British Academy, volume 110, pages 197–234. Oxford University Press Inc., 2001. [find](https://scholar.google.com/scholar?q=P%2E+J%2E+Richerson%2C+R%2E+Boyd%2C+et+al%2E+Institutional+evolution+in+the+holocene%3A+the+rise+of+complex+societies%2E+In+Proceedings%2DBritish+Academy%2C+volume+110%2C+pages+197%E2%80%93234%2E+Oxford+University+Press+Inc%2E%2C+2001%2E)
91. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/), Ma…
92. B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. R. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi, P. Kohli, and A. Fawzi. [Mathematical discoveries from program search with large language models](https://md.research.elamur.ai/free/en/papers/paredes-2023-mathematical-discoveries-program-search-large/). Nature, 625(7995):468–475, Jan. 2024. ISSN 1476-4687. doi: 10.1038/s4158…
93. V. Pallagani, B. Muppasani, K. Murugesan, F. Rossi, B. Srivastava, L. Horesh, F. Fabiano, and A. Loreggia. Understanding the capabilities of large language models for automated planning. arXiv preprint arXiv:2305.16151, 2023. — [arXiv:2305.16151](https://arxiv.org/abs/2305.16151) [find](https://scholar.google.com/scholar?q=V%2E+Pallagani%2C+B%2E+Muppasani%2C+K%2E+Murugesan%2C+F%2E+Rossi%2C+B%2E+Srivastava%2C+L%2E+Horesh%2C+F%2E+Fabiano%2C+and+A%2E+Loreggia%2E+Understanding+the+capabilities+of+large+language+models+for+automated+planning%2E+arXiv+preprin)
94. A. L. Samuel. Some studies in machine learning using the game of checkers. IBM Journal of research and development, 3(3):210–229, 1959. [find](https://scholar.google.com/scholar?q=A%2E+L%2E+Samuel%2E+Some+studies+in+machine+learning+using+the+game+of+checkers%2E+IBM+Journal+of+research+and+development%2C+3%283%29%3A210%E2%80%93229%2C+1959%2E)
95. M. Samvelyan, A. Khan, M. Dennis, M. Jiang, J. Parker-Holder, J. Foerster, R. Raileanu, and T. Rocktäschel. MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning, Mar. 2023. [find](https://scholar.google.com/scholar?q=M%2E+Samvelyan%2C+A%2E+Khan%2C+M%2E+Dennis%2C+M%2E+Jiang%2C+J%2E+Parker%2DHolder%2C+J%2E+Foerster%2C+R%2E+Raileanu%2C+and+T%2E+Rockta%CC%88schel%2E+MAESTRO%3A+Open%2DEnded+Environment+Design+for+Multi%2DAgent+Reinforcement+Learning%2C+Mar%2E+2023%2E)
96. J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein. Generative Agents: Interactive Simulacra of Human Behavior, Apr. 2023. [find](https://scholar.google.com/scholar?q=J%2E+S%2E+Park%2C+J%2E+C%2E+O%E2%80%99Brien%2C+C%2E+J%2E+Cai%2C+M%2E+R%2E+Morris%2C+P%2E+Liang%2C+and+M%2E+S%2E+Bernstein%2E+Generative+Agents%3A+Interactive+Simulacra+of+Human+Behavior%2C+Apr%2E+2023%2E)
97. M. Samvelyan, S. C. Raparthy, A. Lupu, E. Hambro, A. H. Markosyan, M. Bhatt, Y. Mao, M. Jiang, J. Parker-Holder, J. Foerster, T. Rocktäschel, and R. Raileanu. Rainbow teaming: Open-ended generation of diverse adversarial prompts, 2024. [find](https://scholar.google.com/scholar?q=M%2E+Samvelyan%2C+S%2E+C%2E+Raparthy%2C+A%2E+Lupu%2C+E%2E+Hambro%2C+A%2E+H%2E+Markosyan%2C+M%2E+Bhatt%2C+Y%2E+Mao%2C+M%2E+Jiang%2C+J%2E+Parker%2DHolder%2C+J%2E+Foerster%2C+T%2E+Rockta%CC%88schel%2C+and+R%2E+Raileanu%2E+Rainbow+teaming%3A+Open%2Dended+generation+o)
98. J. Parker-Holder, M. Jiang, M. Dennis, M. Samvelyan, J. N. Foerster, E. Grefenstette, and T. Rocktäschel. Evolving curricula with regret-based environment design. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvári, G. Niu, W. Saunders, C. Yeh, J. Wu, S. Bills, L. Ouyang, J. Ward, and J. Leike. Se… — [Self-critiquing models for assisting human evaluators](https://md.research.elamur.ai/free/en/papers/saunders-2022-self-critiquing-models-assisting-human/)
99. I. Shumailov, Z. Shumaylov, Y. Zhao, Y. Gal, N. Papernot, and R. Anderson. Model dementia: Generated data makes models forget. arXiv e-prints, pages arXiv–2305, 2023. [find](https://scholar.google.com/scholar?q=I%2E+Shumailov%2C+Z%2E+Shumaylov%2C+Y%2E+Zhao%2C+Y%2E+Gal%2C+N%2E+Papernot%2C+and+R%2E+Anderson%2E+Model+dementia%3A+Generated+data+makes+models+forget%2E+arXiv+e%2Dprints%2C+pages+arXiv%E2%80%932305%2C+2023%2E)
100. P. Shyam, W. Jaskowski, and F. Gomez. Model-based active exploration. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pa… [find](https://scholar.google.com/scholar?q=P%2E+Shyam%2C+W%2E+Jaskowski%2C+and+F%2E+Gomez%2E+Model%2Dbased+active+exploration%2E+In+K%2E+Chaudhuri+and+R%2E+Salakhutdinov%2C+editors%2C+Proceedings+of+the+36th+International+Conference+on+Machine+Learning%2C+ICML+2019%2C+9%2D)
101. PMLR, 2019. URL [http://proceedings.mlr.press/](http://proceedings.mlr.press/) v97/shyam19a.html. [↗](http://proceedings.mlr.press/)
102. T. Schaul, D. Borsa, D. Ding, D. Szepesvari, G. Ostrovski, W. Dabney, and S. Osindero. Adapting behaviour for learning progress. arXiv preprint arXiv:1912.06910, 2019. — [arXiv:1912.06910](https://arxiv.org/abs/1912.06910) [find](https://scholar.google.com/scholar?q=T%2E+Schaul%2C+D%2E+Borsa%2C+D%2E+Ding%2C+D%2E+Szepesvari%2C+G%2E+Ostrovski%2C+W%2E+Dabney%2C+and+S%2E+Osindero%2E+Adapting+behaviour+for+learning+progress%2E+arXiv+preprint+arXiv%3A1912%2E06910%2C+2019%2E+%E2%80%94+%5BarXiv%3A1912%2E06910%5D%28https%3A%2F%2Farx)
103. T. Schick, J. Dwivedi-Yu, R. Dessı̀, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom. [Toolformer: Language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). Advances in Neural Information Processing Systems, 36, 2024.
104. O. Sigaud, G. Baldassarre, C. Colas, S. Doncieux, R. Duro, N. Perrin-Gilbert, and V.-G. Santucci. A definition of open-ended learning problems for goal-conditioned agents. ArXiv preprint, abs/2311.00344, 2023. URL https://arxiv.org/abs/2311.00344. — [arXiv:2311.00344](https://arxiv.org/abs/2311.00344) [↗](https://arxiv.org/abs/2311.00344)
105. J. Schmidhuber. Adaptive confidence and adaptive curiosity. Inst. für Informatik, 1991a. [find](https://scholar.google.com/scholar?q=J%2E+Schmidhuber%2E+Adaptive+confidence+and+adaptive+curiosity%2E+Inst%2E+fu%CC%88r+Informatik%2C+1991a%2E)
106. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis. Mastering the game of Go w… [find](https://scholar.google.com/scholar?q=D%2E+Silver%2C+A%2E+Huang%2C+C%2E+J%2E+Maddison%2C+A%2E+Guez%2C+L%2E+Sifre%2C+G%2E+van+den+Driessche%2C+J%2E+Schrittwieser%2C+I%2E+Antonoglou%2C+V%2E+Panneershelvam%2C+M%2E+Lanctot%2C+S%2E+Dieleman%2C+D%2E+Grewe%2C+J%2E+Nham%2C+N%2E+Kalchbrenner%2C+I%2E+Sutske)
107. J. Schmidhuber. A possibility for implementing curiosity and boredom in model-building neural controllers. In Proc. of the international conference on simulation of adaptive behavior: From animals to animats, pages 222– 227, 1991b. [find](https://scholar.google.com/scholar?q=J%2E+Schmidhuber%2E+A+possibility+for+implementing+curiosity+and+boredom+in+model%2Dbuilding+neural+controllers%2E+In+Proc%2E+of+the+international+conference+on+simulation+of+adaptive+behavior%3A+From+animals+to+)
108. J. Secretan, N. Beato, D. B. D Ambrosio, A. Rodriguez, A. Campbell, and K. O. Stanley. Picbreeder: Evolving pictures collaboratively online. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’08, pages 1759–1768, New York, NY, USA, Apr. 2008. Association for Computin… — [doi:10.1145/1357054.1357328](https://doi.org/10.1145/1357054.1357328) [find](https://scholar.google.com/scholar?q=J%2E+Secretan%2C+N%2E+Beato%2C+D%2E+B%2E+D+Ambrosio%2C+A%2E+Rodriguez%2C+A%2E+Campbell%2C+and+K%2E+O%2E+Stanley%2E+Picbreeder%3A+Evolving+pictures+collaboratively+online%2E+In+Proceedings+of+the+SIGCHI+Conference+on+Human+Factors+in)
109. D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel, T. Lillicrap, K. Simonyan, and D. Hassabis. [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://md.research.elamur.ai/free/en/papers/al-2017-mastering-chess-shogi-self-play/), Dec. 2017.
110. G. Shackle. Expectation in Economics. Cambridge University Press, 1949. ISBN 9781107629141. URL [https://books.google.co.uk/books?](https://books.google.co.uk/books?) id=zEb47udAsOcC. [↗](https://books.google.co.uk/books?)
111. R. J. Solomonoff. A preliminary report on a general theory of inductive inference. Citeseer, 1960. [find](https://scholar.google.com/scholar?q=R%2E+J%2E+Solomonoff%2E+A+preliminary+report+on+a+general+theory+of+inductive+inference%2E+Citeseer%2C+1960%2E)
112. L. Soros and K. Stanley. Identifying necessary conditions for open-ended evolution through the artificial life world of chromaria. In ALIFE 14: The Fourteenth International Conference on the Synthesis and Simulation of Living Systems, ALIFE 2023: Ghost in the Machine: Proceedings of the 2023 Artific… — [doi:10.1162/978-0-262-32621-6-ch128](https://doi.org/10.1162/978-0-262-32621-6-ch128) [find](https://scholar.google.com/scholar?q=L%2E+Soros+and+K%2E+Stanley%2E+Identifying+necessary+conditions+for+open%2Dended+evolution+through+the+artificial+life+world+of+chromaria%2E+In+ALIFE+14%3A+The+Fourteenth+International+Conference+on+the+Synthesis)
113. R. Shah, V. Varma, R. Kumar, M. Phuong, V. Krakovna, J. Uesato, and Z. Kenton. Goal misgeneralization: Why correct specifications aren’t enough for correct goals. ArXiv preprint, abs/2210.01790, 2022. URL https://arxiv.org/abs/2210.01790. — [arXiv:2210.01790](https://arxiv.org/abs/2210.01790) [↗](https://arxiv.org/abs/2210.01790)
114. A. Sharma, D. Czégel, M. Lachmann, C. P. Kempes, S. I. Walker, and L. Cronin. Assembly theory explains and quantifies selection and evolution. Nature, 622(7982): 321–328, Oct. 2023. ISSN 1476-4687. doi: 10.1038/ s41586-023-06600-9. [find](https://scholar.google.com/scholar?q=A%2E+Sharma%2C+D%2E+Cze%CC%81gel%2C+M%2E+Lachmann%2C+C%2E+P%2E+Kempes%2C+S%2E+I%2E+Walker%2C+and+L%2E+Cronin%2E+Assembly+theory+explains+and+quantifies+selection+and+evolution%2E+Nature%2C+622%287982%29%3A+321%E2%80%93328%2C+Oct%2E+2023%2E+ISSN+1476%2D4687%2E+d)
115. K. Stanley and J. Lehman. Why Greatness Cannot Be Planned: The Myth of the Objective. Springer International Publishing, 2015. ISBN 9783319155241. URL [https://books.google.co.uk/books?](https://books.google.co.uk/books?) id=Llb1CAAAQBAJ. [↗](https://books.google.co.uk/books?)
116. M. Shin, J. Kim, B. van Opheusden, and T. L. Griffiths. Superhuman Artificial Intelligence Can Improve Human Decision Making by Increasing Novelty. Proceedings of the National Academy of Sciences, 120(12):e2214840120, Mar. 2023. ISSN 0027-8424, 1091-6490. doi: 10.1073/ pnas.2214840120. [find](https://scholar.google.com/scholar?q=M%2E+Shin%2C+J%2E+Kim%2C+B%2E+van+Opheusden%2C+and+T%2E+L%2E+Griffiths%2E+Superhuman+Artificial+Intelligence+Can+Improve+Human+Decision+Making+by+Increasing+Novelty%2E+Proceedings+of+the+National+Academy+of+Sciences%2C+120)
117. K. O. Stanley. Why open-endedness matters. Artificial Life, 25(3):232–235, 2019. ISSN 1064-5462. doi: 10.1162/ artl a 00294. URL https://doi.org/10.1162/ artl_a_00294. [find](https://scholar.google.com/scholar?q=K%2E+O%2E+Stanley%2E+Why+open%2Dendedness+matters%2E+Artificial+Life%2C+25%283%29%3A232%E2%80%93235%2C+2019%2E+ISSN+1064%2D5462%2E+doi%3A+10%2E1162%2F+artl+a+00294%2E+URL+https%3A%2F%2Fdoi%2Eorg%2F10%2E1162%2F+artl%5Fa%5F00294%2E)
118. K. O. Stanley and L. Soros. The role of subjectivity in the evaluation of open-endedness. In Presentation delivered in OEE2: The Second Workshop on Open-Ended Evolution, at ALIFE 2016, 2016. [find](https://scholar.google.com/scholar?q=K%2E+O%2E+Stanley+and+L%2E+Soros%2E+The+role+of+subjectivity+in+the+evaluation+of+open%2Dendedness%2E+In+Presentation+delivered+in+OEE2%3A+The+Second+Workshop+on+Open%2DEnded+Evolution%2C+at+ALIFE+2016%2C+2016%2E)
119. T. Lillicrap, K. Kavukcuoglu, D. Hassabis, C. Apps, and D. Silver. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575 (7782):350–354, Nov. 2019. ISSN 1476-4687. doi: 10.1038/s41586-019-1724-z. [find](https://scholar.google.com/scholar?q=T%2E+Lillicrap%2C+K%2E+Kavukcuoglu%2C+D%2E+Hassabis%2C+C%2E+Apps%2C+and+D%2E+Silver%2E+Grandmaster+level+in+StarCraft+II+using+multi%2Dagent+reinforcement+learning%2E+Nature%2C+575+%287782%29%3A350%E2%80%93354%2C+Nov%2E+2019%2E+ISSN+1476%2D4687%2E+do)
120. K. O. Stanley, J. Lehman, and L. Soros. Open-endedness: The last grand challenge you’ve never heard of. While open-endedness could be a force for discovering intelligence, it could also be a component of AI itself, 2017. [find](https://scholar.google.com/scholar?q=K%2E+O%2E+Stanley%2C+J%2E+Lehman%2C+and+L%2E+Soros%2E+Open%2Dendedness%3A+The+last+grand+challenge+you%E2%80%99ve+never+heard+of%2E+While+open%2Dendedness+could+be+a+force+for+discovering+intelligence%2C+it+could+also+be+a+component)
121. L. S. Vygotsky and M. Cole. Mind in society: Development of higher psychological processes. Harvard university press, 1978. [find](https://scholar.google.com/scholar?q=L%2E+S%2E+Vygotsky+and+M%2E+Cole%2E+Mind+in+society%3A+Development+of+higher+psychological+processes%2E+Harvard+university+press%2C+1978%2E)
122. S. Stepney and S. Hickinbotham. On the open-endedness of detecting open-endedness. Artificial Life, pages 1–26, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Stepney+and+S%2E+Hickinbotham%2E+On+the+open%2Dendedness+of+detecting+open%2Dendedness%2E+Artificial+Life%2C+pages+1%E2%80%9326%2C+2023%2E)
123. C. H. Waddington. Paradigm for an evolutionary process. Biological Theory, 3:258–266, 2008. [find](https://scholar.google.com/scholar?q=C%2E+H%2E+Waddington%2E+Paradigm+for+an+evolutionary+process%2E+Biological+Theory%2C+3%3A258%E2%80%93266%2C+2008%2E)
124. R. J. Sternberg and J. E. Davidson. The nature of insight. The MIT Press, 1995. [find](https://scholar.google.com/scholar?q=R%2E+J%2E+Sternberg+and+J%2E+E%2E+Davidson%2E+The+nature+of+insight%2E+The+MIT+Press%2C+1995%2E)
125. G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar. Voyager: An Open-Ended Embodied Agent with Large Language Models, May 2023a. [find](https://scholar.google.com/scholar?q=G%2E+Wang%2C+Y%2E+Xie%2C+Y%2E+Jiang%2C+A%2E+Mandlekar%2C+C%2E+Xiao%2C+Y%2E+Zhu%2C+L%2E+Fan%2C+and+A%2E+Anandkumar%2E+Voyager%3A+An+Open%2DEnded+Embodied+Agent+with+Large+Language+Models%2C+May+2023a%2E)
126. N. N. Taleb. Antifragile: Things that gain from disorder, volume 3. Random House Trade Paperbacks, 2014. [find](https://scholar.google.com/scholar?q=N%2E+N%2E+Taleb%2E+Antifragile%3A+Things+that+gain+from+disorder%2C+volume+3%2E+Random+House+Trade+Paperbacks%2C+2014%2E)
127. R. Wang, J. Lehman, J. Clune, and K. O. Stanley. [Paired open-ended trailblazer (POET): endlessly generating increasingly complex and diverse learning environments and their solutions](https://md.research.elamur.ai/free/en/papers/wang-2019-paired-open-ended-trailblazer-poet/). ArXiv preprint, abs/1901.01753, 2019. URL https://arxiv.org/abs/1901.01753. [↗](https://arxiv.org/abs/1901.01753)
128. X. Tang, A. Zou, Z. Zhang, Y. Zhao, X. Zhang, A. Cohan, and M. Gerstein. [Medagents: Large language models as collaborators for zero-shot medical reasoning](https://md.research.elamur.ai/free/en/papers/tang-2024-medagents-large-language-models-as/). arXiv preprint arXiv:2311.10537, 2023.
129. T. Taylor. Requirements for open-ended evolution in natural and artificial systems. arXiv preprint arXiv:1507.07403, 2015. — [arXiv:1507.07403](https://arxiv.org/abs/1507.07403) [find](https://scholar.google.com/scholar?q=T%2E+Taylor%2E+Requirements+for+open%2Dended+evolution+in+natural+and+artificial+systems%2E+arXiv+preprint+arXiv%3A1507%2E07403%2C+2015%2E+%E2%80%94+%5BarXiv%3A1507%2E07403%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1507%2E07403%29)
130. R. Wang, J. Lehman, A. Rawal, J. Zhi, Y. Li, J. Clune, and K. O. Stanley. Enhanced POET: open-ended reinforcement learning through unbounded invention of learning challenges and their solutions. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virt… [find](https://scholar.google.com/scholar?q=R%2E+Wang%2C+J%2E+Lehman%2C+A%2E+Rawal%2C+J%2E+Zhi%2C+Y%2E+Li%2C+J%2E+Clune%2C+and+K%2E+O%2E+Stanley%2E+Enhanced+POET%3A+open%2Dended+reinforcement+learning+through+unbounded+invention+of+learning+challenges+and+their+solutions%2E+In+Pr)
131. PMLR, 2020. URL [http://proceedings.mlr.press/v119/wang20l.html](http://proceedings.mlr.press/v119/wang20l.html). [↗](http://proceedings.mlr.press/v119/wang20l.html)
132. T. Taylor. Routes to open-endedness in evolutionary systems. arXiv preprint arXiv:1806.01883, 2018. — [arXiv:1806.01883](https://arxiv.org/abs/1806.01883) [find](https://scholar.google.com/scholar?q=T%2E+Taylor%2E+Routes+to+open%2Dendedness+in+evolutionary+systems%2E+arXiv+preprint+arXiv%3A1806%2E01883%2C+2018%2E+%E2%80%94+%5BarXiv%3A1806%2E01883%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1806%2E01883%29)
133. A. M. Turner, D. Hadfield-Menell, and P. Tadepalli. Conservative agency via attainable utility preservation. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 385–391, 2020. [find](https://scholar.google.com/scholar?q=A%2E+M%2E+Turner%2C+D%2E+Hadfield%2DMenell%2C+and+P%2E+Tadepalli%2E+Conservative+agency+via+attainable+utility+preservation%2E+In+Proceedings+of+the+AAAI%2FACM+Conference+on+AI%2C+Ethics%2C+and+Society%2C+pages+385%E2%80%93391%2C+2020%2E)
134. T. T. Wang, A. Gleave, T. Tseng, K. Pelrine, N. Belrose, J. Miller, M. D. Dennis, Y. Duan, V. Pogrebniak, S. Levine, et al. Adversarial policies beat superhuman go ais. In International Conference on Machine Learning, pages 35655–35739. PMLR, 2023b. [find](https://scholar.google.com/scholar?q=T%2E+T%2E+Wang%2C+A%2E+Gleave%2C+T%2E+Tseng%2C+K%2E+Pelrine%2C+N%2E+Belrose%2C+J%2E+Miller%2C+M%2E+D%2E+Dennis%2C+Y%2E+Duan%2C+V%2E+Pogrebniak%2C+S%2E+Levine%2C+et+al%2E+Adversarial+policies+beat+superhuman+go+ais%2E+In+International+Conference+on+)
135. F. Urbina, F. Lentzos, C. Invernizzi, and S. Ekins. Dual use of artificial-intelligence-powered drug discovery. Nature Machine Intelligence, 4(3):189–191, 2022. [find](https://scholar.google.com/scholar?q=F%2E+Urbina%2C+F%2E+Lentzos%2C+C%2E+Invernizzi%2C+and+S%2E+Ekins%2E+Dual+use+of+artificial%2Dintelligence%2Dpowered+drug+discovery%2E+Nature+Machine+Intelligence%2C+4%283%29%3A189%E2%80%93191%2C+2022%2E)
136. K. Valmeekam, M. Marquez, S. Sreedharan, and S. Kambhampati. On the planning abilities of large language models-a critical investigation. Advances in Neural Information Processing Systems, 36:75993–76005, 2023. — [On the Planning Abilities of Large Language Models : A Critical Investigation](https://md.research.elamur.ai/free/en/papers/valmeekam-2023-planning-abilities-large-language-models/)
137. Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi. Self-instruct: Aligning language model with self generated instructions. ArXiv preprint, abs/2212.10560, 2022. URL https://arxiv.org/abs/2212.10560. [↗](https://arxiv.org/abs/2212.10560)
138. P. Villalobos, J. Sevilla, L. Heim, T. Besiroglu, M. Hobbhahn, and A. Ho. Will we run out of data? An analysis of the limits of scaling datasets in Machine Learning, Oct. 2022. [find](https://scholar.google.com/scholar?q=P%2E+Villalobos%2C+J%2E+Sevilla%2C+L%2E+Heim%2C+T%2E+Besiroglu%2C+M%2E+Hobbhahn%2C+and+A%2E+Ho%2E+Will+we+run+out+of+data%3F+An+analysis+of+the+limits+of+scaling+datasets+in+Machine+Learning%2C+Oct%2E+2022%2E)
139. Z. Wang, S. Cai, A. Liu, Y. Jin, J. Hou, B. Zhang, H. Lin, Z. He, Z. Zheng, Y. Yang, X. Ma, and Y. Liang. [JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models](https://md.research.elamur.ai/free/en/papers/wang-2023-jarvis-1-open-world-multi/), Nov. 2023c.
140. O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, J. Oh, D. Horgan, M. Kroiss, I. Danihelka, A. Huang, L. Sifre, T. Cai, J. P. Agapiou, M. Jaderberg, A. S. Vezhnevets, R. Leblond, T. Pohlen, V. Dalibard, D. Budden, Y. Sulsky,… [find](https://scholar.google.com/scholar?q=O%2E+Vinyals%2C+I%2E+Babuschkin%2C+W%2E+M%2E+Czarnecki%2C+M%2E+Mathieu%2C+A%2E+Dudzik%2C+J%2E+Chung%2C+D%2E+H%2E+Choi%2C+R%2E+Powell%2C+T%2E+Ewalds%2C+P%2E+Georgiev%2C+J%2E+Oh%2C+D%2E+Horgan%2C+M%2E+Kroiss%2C+I%2E+Danihelka%2C+A%2E+Huang%2C+L%2E+Sifre%2C+T%2E+Cai%2C+J%2E+P%2E)
141. A. Illustrating Open-Endedness M. L. Wong, C. E. Cleland, D. Arend Jr, S. Bartlett, H. J. Cleaves, H. Demarest, A. Prabhu, J. I. Lunine, and R. M. Hazen. On the roles of function and selection in evolving systems. Proceedings of the National Academy of Sciences, 120(43):e2310223120, 2023b. [find](https://scholar.google.com/scholar?q=A%2E+Illustrating+Open%2DEndedness+M%2E+L%2E+Wong%2C+C%2E+E%2E+Cleland%2C+D%2E+Arend+Jr%2C+S%2E+Bartlett%2C+H%2E+J%2E+Cleaves%2C+H%2E+Demarest%2C+A%2E+Prabhu%2C+J%2E+I%2E+Lunine%2C+and+R%2E+M%2E+Hazen%2E+On+the+roles+of+function+and+selection+in+evol)

*Updated 3 November 2026*