---
title: "Quality-Diversity through AI Feedback — Bradley 2023"
description: "The authors propose QDAIF, an evolutionary method in which language models handle all three parts… 91 references, cited by 13 papers here."
authors: ["Herbie Bradley", "Andrew Dai", "Hannah Teufel", "Jenny Zhang", "Koen Oostermeijer", "Marco Bellagente", "Jeff Clune", "Kenneth Stanley", "Gregory Schott", "Joel Lehman"]
year: 2023
venue: "arXiv:2310.13032"
source: "https://arxiv.org/abs/2310.13032"
doi: "10.48550/arxiv.2310.13032"
arxiv: "2310.13032"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bradley-2023-quality-diversity-through-ai-feedback/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bradley-2023-quality-diversity-through-ai-feedback/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) / Quality-Diversity through AI Feedback

# Quality-Diversity through AI Feedback

**Authors:** [Herbie Bradley](https://md.research.elamur.ai/free/en/authors/herbie-bradley/), [Andrew Dai](https://md.research.elamur.ai/free/en/authors/andrew-m-dai/), [Hannah Teufel](https://md.research.elamur.ai/free/en/authors/hannah-teufel/), Jenny Zhang, Koen Oostermeijer, [Marco Bellagente](https://md.research.elamur.ai/free/en/authors/marco-bellagente/), [Jeff Clune](https://md.research.elamur.ai/free/en/authors/jeff-clune/), Kenneth Stanley, Gregory Schott and Joel Lehman · **Year:** 2023 · **Venue:** arXiv:2310.13032 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2310.13032) · [PDF](https://arxiv.org/pdf/2310.13032) · [DOI 10.48550/arxiv.2310.13032](https://doi.org/10.48550/arxiv.2310.13032) · [arXiv 2310.13032](https://arxiv.org/abs/2310.13032) · [OpenAlex](https://openalex.org/W4387891162)

**Citations:** 6 citations (OpenAlex, 2026-10-14) · [13](https://md.research.elamur.ai/free/en/papers/bradley-2023-quality-diversity-through-ai-feedback/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Quality-diversity (QD) search maintains a population of solutions that are both good and varied. However, it has rarely been used for creative text because quality and diversity are hard to define in code. The authors propose QDAIF, an evolutionary method in which language models handle all three parts of QD search. LMs create new candidates through few-shot Language Model Crossover (LMX), and prompted LMs act as judges of both text quality and position along qualitative diversity axes, such as story genre or ending tone. No fine-tuning or hand-crafted features are needed. The implementation builds on the open-source OpenELM library. Across opinion writing, short stories and poetry, QDAIF covered more of the target search space with high-quality texts than non-QD baselines. Human raters broadly agreed with the AI judgments. This suggests that off-the-shelf LMs can steer open-ended search in subjective domains, which points toward systems that generate, assess and refine ideas on their own.

## Key points
- Paper: Bradley, Dai et al., arXiv:2310.13032 (2023). It is a collaboration involving CarperAI, EleutherAI, Aleph Alpha, the University of Cambridge, UBC and others. The project page is qdaif.github.io.
- QDAIF extends the Evolution through Large Models (ELM) framework. Variation comes from LMX, which concatenates several existing candidates into a prompt so the LM produces similar new ones.
- Diversity is measured by prompting an LM for qualitative attributes rather than low-level metrics. One example task is stories about a spy and a politician, binned by genre (romance to horror) and by ending (happy to tragic).
- The evaluation of QDAIF spanned three kinds of creative text: opinion pieces, short stories and poems. It was compared with baselines including Random-Search and LMX, Quality-Only.
- LMX, Quality-Only raised fitness over iterations but left many diversity bins empty. Random-Search kept every entry, which held back progress toward higher fitness.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 243 works · 121,396 citations
- [University of British Columbia](https://md.research.elamur.ai/free/en/universities/university-of-british-columbia/) — University, Vancouver, Canada · 98 works · 34,805 citations
- [Vector Institute](https://md.research.elamur.ai/free/en/labs/vector-institute/) — Research institute, Toronto, Canada · 33 works · 4,535 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🧬 Biochemistry, Genetics and Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/) · 234 works · [🧬 Genetics](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/genetics/) · 3 works

## Cite

Cite as: Herbie Bradley, Andrew Dai, Hannah Teufel, Jenny Zhang, Koen Oostermeijer, Marco Bellagente, Jeff Clune, Kenneth Stanley, Gregory Schott and Joel Lehman. 2023. Quality-Diversity through AI Feedback. arXiv:2310.13032.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{bradley2023quality,
  title = {{Quality-Diversity through AI Feedback}},
  author = {Herbie Bradley, Andrew Dai, Hannah Teufel, Jenny Zhang, Koen Oostermeijer, Marco Bellagente, Jeff Clune, Kenneth Stanley, Gregory Schott and Joel Lehman},
  year = {2023},
  doi = {10.48550/arxiv.2310.13032},
  eprint = {2310.13032},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2310.13032}
}
```

- [← previous Naturalistic Causal Probing for Morpho-Syntax](https://md.research.elamur.ai/free/en/papers/amini-2023-naturalistic-causal-probing-morpho-syntax/)
- [next → RECKONING: Reasoning through Dynamic Knowledge Encoding](https://md.research.elamur.ai/free/en/papers/chen-2023-reckoning-reasoning-through-dynamic-knowledge/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/)

## References

91 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. [Do as I can, not as I say: Grounding language in robotic affordances](https://md.research.elamur.ai/free/en/papers/ahn-2022-i-i-say-grounding-language/). arXiv preprint arXiv:2204.01691, 2022.
2. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. [Flamingo: A visual language model for few-shot learning](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/). Advances in Neural Information Processing Systems, 35:23716– 23736, 2022.
3. I. Elaine Allen and Christopher A. Seaman. Likert scales and data analyses. Quality progress, 40 (7):64–65, 2007. [find](https://scholar.google.com/scholar?q=I%2E+Elaine+Allen+and+Christopher+A%2E+Seaman%2E+Likert+scales+and+data+analyses%2E+Quality+progress%2C+40+%287%29%3A64%E2%80%9365%2C+2007%2E)
4. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. [Constitutional AI: Harmlessness from AI feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/). arXiv preprint arXiv:2212.08073, 2022.
5. Marco Bellagente, Manuel Brack, Hannah Teufel, Felix Friedrich, Björn Deiseroth, Constantin Eichenberg, Andrew Dai, Robert Baldock, Souradeep Nanda, Koen Oostermeijer, et al. Multifusion: Fusing pre-trained models for multi-lingual, multi-modal image generation. arXiv preprint arXiv:2305.15296, 2023… — [arXiv:2305.15296](https://arxiv.org/abs/2305.15296) [find](https://scholar.google.com/scholar?q=Marco+Bellagente%2C+Manuel+Brack%2C+Hannah+Teufel%2C+Felix+Friedrich%2C+Bj%C3%B6rn+Deiseroth%2C+Constantin+Eichenberg%2C+Andrew+Dai%2C+Robert+Baldock%2C+Souradeep+Nanda%2C+Koen+Oostermeijer%2C+et+al%2E+Multifusion%3A+Fusing+pre%2Dt)
6. Varun Bhatt, Bryon Tjanaka, Matthew Fontaine, and Stefanos Nikolaidis. Deep surrogate assisted generation of environments. Advances in Neural Information Processing Systems, 35:37762– 37777, 2022. [find](https://scholar.google.com/scholar?q=Varun+Bhatt%2C+Bryon+Tjanaka%2C+Matthew+Fontaine%2C+and+Stefanos+Nikolaidis%2E+Deep+surrogate+assisted+generation+of+environments%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A37762%E2%80%93+37777%2C+2022%2E)
7. Varun Bhatt, Heramb Nemlekar, Matthew Fontaine, Bryon Tjanaka, Hejia Zhang, Ya-Chuan Hsu, and Stefanos Nikolaidis. Surrogate assisted generation of human-robot interaction scenarios. arXiv preprint arXiv:2304.13787, 2023. — [arXiv:2304.13787](https://arxiv.org/abs/2304.13787) [find](https://scholar.google.com/scholar?q=Varun+Bhatt%2C+Heramb+Nemlekar%2C+Matthew+Fontaine%2C+Bryon+Tjanaka%2C+Hejia+Zhang%2C+Ya%2DChuan+Hsu%2C+and+Stefanos+Nikolaidis%2E+Surrogate+assisted+generation+of+human%2Drobot+interaction+scenarios%2E+arXiv+preprint+ar)
8. Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. [On the opportunities and risks of foundation models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/). arXiv preprint arXiv:2108.07258, 2021.
9. Herbie Bradley, Honglu Fan, Francisco Carvalho, Matthew Fisher, Louis Castricato, reciprocated, dmayhem93, Shivanshu Purohit, and Joel Lehman. OpenELM, January 2023a. URL [https://github.com/CarperAI/OpenELM](https://github.com/CarperAI/OpenELM). [↗](https://github.com/CarperAI/OpenELM)
10. Herbie Bradley, Honglu Fan, Harry Saini, Reshinth Adithyan, Shivanshu Purohit, and Joel Lehman. Diff models - A new way to edit code. CarperAI Blog, Jan 2023b. URL [https://carper.ai/diff-model/](https://carper.ai/diff-model/). [↗](https://carper.ai/diff-model/)
11. Jonathan C. Brant and Kenneth O. Stanley. Minimal criterion coevolution: A new approach to open-ended search. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 67–74, 2017. [find](https://scholar.google.com/scholar?q=Jonathan+C%2E+Brant+and+Kenneth+O%2E+Stanley%2E+Minimal+criterion+coevolution%3A+A+new+approach+to+open%2Dended+search%2E+In+Proceedings+of+the+Genetic+and+Evolutionary+Computation+Conference%2C+pp%2E+67%E2%80%9374%2C+2017%2E)
12. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901, 2020.
13. Angelica Chen, David M. Dohan, and David R. So. Evoprompting: Language models for code-level neural architecture search. arXiv preprint arXiv:2302.14838, 2023. — [arXiv:2302.14838](https://arxiv.org/abs/2302.14838) [find](https://scholar.google.com/scholar?q=Angelica+Chen%2C+David+M%2E+Dohan%2C+and+David+R%2E+So%2E+Evoprompting%3A+Language+models+for+code%2Dlevel+neural+architecture+search%2E+arXiv+preprint+arXiv%3A2302%2E14838%2C+2023%2E+%E2%80%94+%5BarXiv%3A2302%2E14838%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
14. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhai… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
15. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. [PaLM: Scaling language modeling with pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/). arXiv preprint arXiv:2204.02311, 2022.
16. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Eric+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+et+al%2E+Scaling+instruction%2Dfinetuned+language+models%2E+arXiv+prep)
17. Jeff Clune. AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence. arXiv preprint arXiv:1905.10985, 2019. — [arXiv:1905.10985](https://arxiv.org/abs/1905.10985) [find](https://scholar.google.com/scholar?q=Jeff+Clune%2E+AI%2DGAs%3A+AI%2Dgenerating+algorithms%2C+an+alternate+paradigm+for+producing+general+artificial+intelligence%2E+arXiv+preprint+arXiv%3A1905%2E10985%2C+2019%2E+%E2%80%94+%5BarXiv%3A1905%2E10985%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F190)
18. Cédric Colas, Laetitia Teodorescu, Pierre-Yves Oudeyer, Xingdi Yuan, and Marc-Alexandre Côté. Augmenting autotelic agents with large language models. arXiv preprint arXiv:2305.12487, 2023. [find](https://scholar.google.com/scholar?q=C%C3%A9dric+Colas%2C+Laetitia+Teodorescu%2C+Pierre%2DYves+Oudeyer%2C+Xingdi+Yuan%2C+and+Marc%2DAlexandre+C%C3%B4t%C3%A9%2E+Augmenting+autotelic+agents+with+large+language+models%2E+arXiv+preprint+arXiv%3A2305%2E12487%2C+2023%2E)
19. Antoine Cully. Autonomous skill discovery with quality-diversity and unsupervised descriptors. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 81–89, 2019. [find](https://scholar.google.com/scholar?q=Antoine+Cully%2E+Autonomous+skill+discovery+with+quality%2Ddiversity+and+unsupervised+descriptors%2E+In+Proceedings+of+the+Genetic+and+Evolutionary+Computation+Conference%2C+pp%2E+81%E2%80%9389%2C+2019%2E)
20. Antoine Cully and Yiannis Demiris. Hierarchical behavioral repertoires with unsupervised descriptors. Proceedings of the Genetic and Evolutionary Computation Conference, 2018. [find](https://scholar.google.com/scholar?q=Antoine+Cully+and+Yiannis+Demiris%2E+Hierarchical+behavioral+repertoires+with+unsupervised+descriptors%2E+Proceedings+of+the+Genetic+and+Evolutionary+Computation+Conference%2C+2018%2E)
21. Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret. Robots that can adapt like animals. Nature, 521(7553):503–507, 2015. [find](https://scholar.google.com/scholar?q=Antoine+Cully%2C+Jeff+Clune%2C+Danesh+Tarapore%2C+and+Jean%2DBaptiste+Mouret%2E+Robots+that+can+adapt+like+animals%2E+Nature%2C+521%287553%29%3A503%E2%80%93507%2C+2015%2E)
22. Li Ding, Jenny Zhang, Jeff Clune, Lee Spector, and Joel Lehman. Quality diversity through human feedback. arXiv preprint arXiv:2310.12103, 2023. — [arXiv:2310.12103](https://arxiv.org/abs/2310.12103) [find](https://scholar.google.com/scholar?q=Li+Ding%2C+Jenny+Zhang%2C+Jeff+Clune%2C+Lee+Spector%2C+and+Joel+Lehman%2E+Quality+diversity+through+human+feedback%2E+arXiv+preprint+arXiv%3A2310%2E12103%2C+2023%2E+%E2%80%94+%5BarXiv%3A2310%2E12103%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E12103%29)
23. Danny Driess, Fei Xia, Mehdi S.M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. PaLM-E: An embodied multimodal language model. arXiv preprint arXiv:2303.03378, 2023. [find](https://scholar.google.com/scholar?q=Danny+Driess%2C+Fei+Xia%2C+Mehdi+S%2EM%2E+Sajjadi%2C+Corey+Lynch%2C+Aakanksha+Chowdhery%2C+Brian+Ichter%2C+Ayzaan+Wahid%2C+Jonathan+Tompson%2C+Quan+Vuong%2C+Tianhe+Yu%2C+et+al%2E+PaLM%2DE%3A+An+embodied+multimodal+language+model%2E+)
24. Adrien Ecoffet, Jeff Clune, and Joel Lehman. Open questions in creating safe open-ended ai: tensions between control and creativity. In Artificial Life Conference Proceedings 32, pp. 27–35. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info . . . , 2020. [find](https://scholar.google.com/scholar?q=Adrien+Ecoffet%2C+Jeff+Clune%2C+and+Joel+Lehman%2E+Open+questions+in+creating+safe+open%2Dended+ai%3A+tensions+between+control+and+creativity%2E+In+Artificial+Life+Conference+Proceedings+32%2C+pp%2E+27%E2%80%9335%2E+MIT+Press+)
25. Constantin Eichenberg, Sidney Black, Samuel Weinbach, Letitia Parcalabescu, and Anette Frank. MAGMA - Multimodal augmentation of generative models through adapter-based finetuning. arXiv preprint arXiv:2112.05253, 2021. [find](https://scholar.google.com/scholar?q=Constantin+Eichenberg%2C+Sidney+Black%2C+Samuel+Weinbach%2C+Letitia+Parcalabescu%2C+and+Anette+Frank%2E+MAGMA+%2D+Multimodal+augmentation+of+generative+models+through+adapter%2Dbased+finetuning%2E+arXiv+preprint+arXi)
26. Manon Flageat and Antoine Cully. Fast and stable map-elites in noisy domains using deep grids. In Artificial Life Conference Proceedings 32, pp. 273–282. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info . . . , 2020. [find](https://scholar.google.com/scholar?q=Manon+Flageat+and+Antoine+Cully%2E+Fast+and+stable+map%2Delites+in+noisy+domains+using+deep+grids%2E+In+Artificial+Life+Conference+Proceedings+32%2C+pp%2E+273%E2%80%93282%2E+MIT+Press+One+Rogers+Street%2C+Cambridge%2C+MA+021)
27. Manon Flageat and Antoine Cully. Uncertain quality-diversity: Evaluation methodology and new methods for quality-diversity in uncertain domains. IEEE Transactions on Evolutionary Computation, 2023. [find](https://scholar.google.com/scholar?q=Manon+Flageat+and+Antoine+Cully%2E+Uncertain+quality%2Ddiversity%3A+Evaluation+methodology+and+new+methods+for+quality%2Ddiversity+in+uncertain+domains%2E+IEEE+Transactions+on+Evolutionary+Computation%2C+2023%2E)
28. Matthew Fontaine and Stefanos Nikolaidis. Differentiable quality diversity. Advances in Neural Information Processing Systems, 34:10040–10052, 2021. [find](https://scholar.google.com/scholar?q=Matthew+Fontaine+and+Stefanos+Nikolaidis%2E+Differentiable+quality+diversity%2E+Advances+in+Neural+Information+Processing+Systems%2C+34%3A10040%E2%80%9310052%2C+2021%2E)
29. Felix Friedrich, Patrick Schramowski, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Sasha Luccioni, and Kristian Kersting. Fair diffusion: Instructing text-to-image generation models on fairness. arXiv preprint arXiv:2302.10893, 2023. — [arXiv:2302.10893](https://arxiv.org/abs/2302.10893) [find](https://scholar.google.com/scholar?q=Felix+Friedrich%2C+Patrick+Schramowski%2C+Manuel+Brack%2C+Lukas+Struppek%2C+Dominik+Hintersdorf%2C+Sasha+Luccioni%2C+and+Kristian+Kersting%2E+Fair+diffusion%3A+Instructing+text%2Dto%2Dimage+generation+models+on+fairness%2E)
30. Adam Gaier. Accelerating Evolutionary Design Exploration with Predictive and Generative Models. PhD thesis, Université de Lorraine, 2020. [find](https://scholar.google.com/scholar?q=Adam+Gaier%2E+Accelerating+Evolutionary+Design+Exploration+with+Predictive+and+Generative+Models%2E+PhD+thesis%2C+Universit%C3%A9+de+Lorraine%2C+2020%2E)
31. Adam Gaier, Alexander Asteroth, and Jean-Baptiste Mouret. Data-efficient exploration, optimization, and modeling of diverse designs through surrogate-assisted illumination. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 99–106, 2017. [find](https://scholar.google.com/scholar?q=Adam+Gaier%2C+Alexander+Asteroth%2C+and+Jean%2DBaptiste+Mouret%2E+Data%2Defficient+exploration%2C+optimization%2C+and+modeling+of+diverse+designs+through+surrogate%2Dassisted+illumination%2E+In+Proceedings+of+the+Genet)
32. Adam Gaier, Alexander Asteroth, and Jean-Baptiste Mouret. Are quality diversity algorithms better at generating stepping stones than objective-based search? In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 115–116, 2019. [find](https://scholar.google.com/scholar?q=Adam+Gaier%2C+Alexander+Asteroth%2C+and+Jean%2DBaptiste+Mouret%2E+Are+quality+diversity+algorithms+better+at+generating+stepping+stones+than+objective%2Dbased+search%3F+In+Proceedings+of+the+Genetic+and+Evolution)
33. Luca Grillotti and Antoine Cully. Unsupervised behaviour discovery with quality-diversity optimisation. arXiv preprint arXiv:2106.05648, 2021. — [arXiv:2106.05648](https://arxiv.org/abs/2106.05648) [find](https://scholar.google.com/scholar?q=Luca+Grillotti+and+Antoine+Cully%2E+Unsupervised+behaviour+discovery+with+quality%2Ddiversity+optimisation%2E+arXiv+preprint+arXiv%3A2106%2E05648%2C+2021%2E+%E2%80%94+%5BarXiv%3A2106%2E05648%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2106%2E05648%29)
34. Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. [Towards a unified view of parameter-efficient transfer learning](https://md.research.elamur.ai/free/en/papers/he-2021-towards-unified-view-parameter-efficient/). arXiv preprint arXiv:2110.04366, 2021.
35. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning, pp. 2790–2799. PMLR, 2019. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+De+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+Parameter%2Defficient+transfer+learning+for+NLP%2E+In+Intern)
36. Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How can we know when language models know? On the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962–977, 2021. [find](https://scholar.google.com/scholar?q=Zhengbao+Jiang%2C+Jun+Araki%2C+Haibo+Ding%2C+and+Graham+Neubig%2E+How+can+we+know+when+language+models+know%3F+On+the+calibration+of+language+models+for+question+answering%2E+Transactions+of+the+Association+for+C)
37. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361, 2020.
38. Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401–4410, 2019. [find](https://scholar.google.com/scholar?q=Tero+Karras%2C+Samuli+Laine%2C+and+Timo+Aila%2E+A+style%2Dbased+generator+architecture+for+generative+adversarial+networks%2E+In+Proceedings+of+the+IEEE%2FCVF+conference+on+computer+vision+and+pattern+recognition)
39. Leon Keller, Daniel Tanneberg, Svenja Stark, and Jan Peters. Model-based quality-diversity search for efficient robot learning. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 9675–9680. [find](https://scholar.google.com/scholar?q=Leon+Keller%2C+Daniel+Tanneberg%2C+Svenja+Stark%2C+and+Jan+Peters%2E+Model%2Dbased+quality%2Ddiversity+search+for+efficient+robot+learning%2E+In+2020+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Syst)
40. Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. [Understanding the effects of rlhf on llm generalisation and diversity](https://md.research.elamur.ai/free/en/papers/kirk-2023-understanding-effects-rlhf-llm-generalisation/). arXiv preprint arXiv:2310.06452, 2023.
41. Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, and Abhinav Rastogi. RLAIF: Scaling reinforcement learning from human feedback with AI feedback. arXiv preprint arXiv:2309.00267, 2023. [find](https://scholar.google.com/scholar?q=Harrison+Lee%2C+Samrat+Phatale%2C+Hassan+Mansoor%2C+Kellie+Lu%2C+Thomas+Mesnard%2C+Colton+Bishop%2C+Victor+Carbune%2C+and+Abhinav+Rastogi%2E+RLAIF%3A+Scaling+reinforcement+learning+from+human+feedback+with+AI+feedback%2E)
42. Joel Lehman and Kenneth O. Stanley. Revising the evolutionary computation abstraction: Minimal criteria novelty search. In Proceedings of the 12th annual conference on Genetic and evolutionary computation, pp. 103–110, 2010. [find](https://scholar.google.com/scholar?q=Joel+Lehman+and+Kenneth+O%2E+Stanley%2E+Revising+the+evolutionary+computation+abstraction%3A+Minimal+criteria+novelty+search%2E+In+Proceedings+of+the+12th+annual+conference+on+Genetic+and+evolutionary+computa)
43. Joel Lehman and Kenneth O Stanley. Abandoning objectives: Evolution through the search for novelty alone. Evolutionary computation, 19(2):189–223, 2011a. [find](https://scholar.google.com/scholar?q=Joel+Lehman+and+Kenneth+O+Stanley%2E+Abandoning+objectives%3A+Evolution+through+the+search+for+novelty+alone%2E+Evolutionary+computation%2C+19%282%29%3A189%E2%80%93223%2C+2011a%2E)
44. Joel Lehman and Kenneth O. Stanley. Evolving a diversity of virtual creatures through novelty search and local competition. In Proceedings of the 13th annual conference on Genetic and evolutionary computation, pp. 211–218, 2011b. [find](https://scholar.google.com/scholar?q=Joel+Lehman+and+Kenneth+O%2E+Stanley%2E+Evolving+a+diversity+of+virtual+creatures+through+novelty+search+and+local+competition%2E+In+Proceedings+of+the+13th+annual+conference+on+Genetic+and+evolutionary+com)
45. Joel Lehman, Kenneth O Stanley, et al. Exploiting open-endedness to solve problems through the search for novelty. In ALIFE, pp. 329–336, 2008. [find](https://scholar.google.com/scholar?q=Joel+Lehman%2C+Kenneth+O+Stanley%2C+et+al%2E+Exploiting+open%2Dendedness+to+solve+problems+through+the+search+for+novelty%2E+In+ALIFE%2C+pp%2E+329%E2%80%93336%2C+2008%2E)
46. Joel Lehman, Bryan Wilder, and Kenneth O. Stanley. On the critical role of divergent selection in evolvability. Frontiers in Robotics and AI, 3:45, 2016. [find](https://scholar.google.com/scholar?q=Joel+Lehman%2C+Bryan+Wilder%2C+and+Kenneth+O%2E+Stanley%2E+On+the+critical+role+of+divergent+selection+in+evolvability%2E+Frontiers+in+Robotics+and+AI%2C+3%3A45%2C+2016%2E)
47. Joel Lehman, Jeff Clune, Dusan Misevic, Christoph Adami, Lee Altenberg, Julie Beaulieu, Peter J. Bentley, Samuel Bernard, Guillaume Beslon, David M. Bryson, Patryk Chrabaszcz, Nick Cheney, Antoine Cully, Stephane Doncieux, Fred C. Dyer, Kai Olav Ellefsen, Robert Feldt, Stephan Fischer, Stephanie For… [find](https://scholar.google.com/scholar?q=Joel+Lehman%2C+Jeff+Clune%2C+Dusan+Misevic%2C+Christoph+Adami%2C+Lee+Altenberg%2C+Julie+Beaulieu%2C+Peter+J%2E+Bentley%2C+Samuel+Bernard%2C+Guillaume+Beslon%2C+David+M%2E+Bryson%2C+Patryk+Chrabaszcz%2C+Nick+Cheney%2C+Antoine+Cul)
48. Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O. Stanley. Evolution through large models. arXiv preprint arXiv:2206.08896, 2022. — [arXiv:2206.08896](https://arxiv.org/abs/2206.08896) [find](https://scholar.google.com/scholar?q=Joel+Lehman%2C+Jonathan+Gordon%2C+Shawn+Jain%2C+Kamal+Ndousse%2C+Cathy+Yeh%2C+and+Kenneth+O%2E+Stanley%2E+Evolution+through+large+models%2E+arXiv+preprint+arXiv%3A2206%2E08896%2C+2022%2E+%E2%80%94+%5BarXiv%3A2206%2E08896%5D%28https%3A%2F%2Farxiv%2Eor)
49. Bryan Lim, Luca Grillotti, Lorenzo Bernasconi, and Antoine Cully. Dynamics-aware quality-diversity for efficient learning of skill repertoires. arXiv preprint arXiv:2109.08522, 2021. — [arXiv:2109.08522](https://arxiv.org/abs/2109.08522) [find](https://scholar.google.com/scholar?q=Bryan+Lim%2C+Luca+Grillotti%2C+Lorenzo+Bernasconi%2C+and+Antoine+Cully%2E+Dynamics%2Daware+quality%2Ddiversity+for+efficient+learning+of+skill+repertoires%2E+arXiv+preprint+arXiv%3A2109%2E08522%2C+2021%2E+%E2%80%94+%5BarXiv%3A2109%2E085)
50. Bryan Lim, Alexander Reichenbach, and Antoine Cully. Learning to walk autonomously via reset-free quality-diversity. arXiv preprint arXiv:2204.03655, 2022. — [arXiv:2204.03655](https://arxiv.org/abs/2204.03655) [find](https://scholar.google.com/scholar?q=Bryan+Lim%2C+Alexander+Reichenbach%2C+and+Antoine+Cully%2E+Learning+to+walk+autonomously+via+reset%2Dfree+quality%2Ddiversity%2E+arXiv+preprint+arXiv%3A2204%2E03655%2C+2022%2E+%E2%80%94+%5BarXiv%3A2204%2E03655%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
51. Chin-Yew Lin. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text summarization branches out, pp. 74–81, 2004.
52. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). arXiv preprint arXiv:2304.08485, 2023.
53. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. [Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/). arXiv preprint arXiv:2104.08786, 2021.
54. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651, 2023. [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+et+al%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Dfeedbac)
55. Christopher D Manning. An introduction to information retrieval. Cambridge university press, 2009. [find](https://scholar.google.com/scholar?q=Christopher+D+Manning%2E+An+introduction+to+information+retrieval%2E+Cambridge+university+press%2C+2009%2E)
56. Elliot Meyerson, Mark J. Nelson, Herbie Bradley, Arash Moradi, Amy K. Hoover, and Joel Lehman. Language model crossover: Variation through few-shot prompting. arXiv preprint arXiv:2302.12170, 2023. — [arXiv:2302.12170](https://arxiv.org/abs/2302.12170) [find](https://scholar.google.com/scholar?q=Elliot+Meyerson%2C+Mark+J%2E+Nelson%2C+Herbie+Bradley%2C+Arash+Moradi%2C+Amy+K%2E+Hoover%2C+and+Joel+Lehman%2E+Language+model+crossover%3A+Variation+through+few%2Dshot+prompting%2E+arXiv+preprint+arXiv%3A2302%2E12170%2C+2023%2E+%E2%80%94+)
57. Jean-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909, 2015. — [arXiv:1504.04909](https://arxiv.org/abs/1504.04909) [find](https://scholar.google.com/scholar?q=Jean%2DBaptiste+Mouret+and+Jeff+Clune%2E+Illuminating+search+spaces+by+mapping+elites%2E+arXiv+preprint+arXiv%3A1504%2E04909%2C+2015%2E+%E2%80%94+%5BarXiv%3A1504%2E04909%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1504%2E04909%29)
58. Niklas Muennighoff. [SGPT: GPT sentence embeddings for semantic search](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-sgpt-gpt-sentence-embeddings-semantic/). arXiv:2202.08904, 2022.
59. Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 427–436, 2015a. [find](https://scholar.google.com/scholar?q=Anh+Nguyen%2C+Jason+Yosinski%2C+and+Jeff+Clune%2E+Deep+neural+networks+are+easily+fooled%3A+High+confidence+predictions+for+unrecognizable+images%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+)
60. Anh Nguyen, Jason Yosinski, and Jeff Clune. Innovation engines: Automated creativity and improved stochastic optimization via deep learning. In Proceedings of the Genetic and Evolutionary Computation Conference, 2015b. [find](https://scholar.google.com/scholar?q=Anh+Nguyen%2C+Jason+Yosinski%2C+and+Jeff+Clune%2E+Innovation+engines%3A+Automated+creativity+and+improved+stochastic+optimization+via+deep+learning%2E+In+Proceedings+of+the+Genetic+and+Evolutionary+Computation+)
61. Anh Nguyen, Jason Yosinski, and Jeff Clune. Understanding innovation engines: Automated creativity and improved stochastic optimization via deep learning. Evolutionary computation, 24(3): 545–572, 2016. [find](https://scholar.google.com/scholar?q=Anh+Nguyen%2C+Jason+Yosinski%2C+and+Jeff+Clune%2E+Understanding+innovation+engines%3A+Automated+creativity+and+improved+stochastic+optimization+via+deep+learning%2E+Evolutionary+computation%2C+24%283%29%3A+545%E2%80%93572%2C+201)
62. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. [Adversarial NLI: A new benchmark for natural language understanding](https://md.research.elamur.ai/free/en/papers/nie-2020-adversarial-nli-new-benchmark-natural/). arXiv preprint arXiv:1910.14599, 2019.
63. Olle Nilsson and Antoine Cully. Policy gradient assisted MAP-Elites. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 866–875, 2021. [find](https://scholar.google.com/scholar?q=Olle+Nilsson+and+Antoine+Cully%2E+Policy+gradient+assisted+MAP%2DElites%2E+In+Proceedings+of+the+Genetic+and+Evolutionary+Computation+Conference%2C+pp%2E+866%E2%80%93875%2C+2021%2E)
64. OpenAI. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), 2023.
65. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in Neural Information Processing Systems, 35: 27730–27744, 2022.
66. Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. [Discovering language model behaviors with model-written evaluations](https://md.research.elamur.ai/free/en/papers/perez-2022-discovering-language-model-behaviors-model/). arXiv preprint arXiv:2212.09251, 2022.
67. Julien Pourcel, Cédric Colas, Pierre-Yves Oudeyer, and Laetitia Teodorescu. Aces: generating diverse programming puzzles with autotelic language models and semantic descriptors. arXiv preprint arXiv:2310.10692, 2023. — [arXiv:2310.10692](https://arxiv.org/abs/2310.10692) [find](https://scholar.google.com/scholar?q=Julien+Pourcel%2C+C%C3%A9dric+Colas%2C+Pierre%2DYves+Oudeyer%2C+and+Laetitia+Teodorescu%2E+Aces%3A+generating+diverse+programming+puzzles+with+autotelic+language+models+and+semantic+descriptors%2E+arXiv+preprint+arXiv%3A2)
68. Justin K. Pugh, Lisa B. Soros, and Kenneth O. Stanley. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, 3:40, 2016. [find](https://scholar.google.com/scholar?q=Justin+K%2E+Pugh%2C+Lisa+B%2E+Soros%2C+and+Kenneth+O%2E+Stanley%2E+Quality+diversity%3A+A+new+frontier+for+evolutionary+computation%2E+Frontiers+in+Robotics+and+AI%2C+3%3A40%2C+2016%2E)
69. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In International conference on machine learning, pp. 8748–8763. PMLR, 2021…
70. Nils Reimers and Iryna Gurevych. [Sentence-BERT: Sentence embeddings using siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). arXiv preprint arXiv:1908.10084, 2019.
71. Alex Renda, Aspen K. Hopkins, and Michael Carbin. Can LLMs generate random numbers? evaluating LLM sampling in controlled domains. In ICML 2023 Workshop: Sampling and Optimization in Discrete Space, 2023. URL [http://people.csail.mit.edu/renda/](http://people.csail.mit.edu/renda/) llm-sampling-paper. [↗](http://people.csail.mit.edu/renda/)
72. Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. [Multitask prompted training enables zero-shot task generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/). arXiv preprint arXiv:2110.08207, 2021.
73. William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. [Self-critiquing models for assisting human evaluators](https://md.research.elamur.ai/free/en/papers/saunders-2022-self-critiquing-models-assisting-human/). arXiv preprint arXiv:2206.05802, 2022.
74. Jimmy Secretan, Nicholas Beato, David B. D’Ambrosio, Adelein Rodriguez, Adam Campbell, Jeremiah T. Folsom-Kovarik, and Kenneth O. Stanley. Picbreeder: A case study in collaborative evolutionary exploration of design space. Evol. Comput., 19(3):373–403, 2011. doi: 10.1162/EVCO\\_a\\_00030. URL https://… — [doi:10.1162/evco\\_a\\_00030](https://doi.org/10.1162/evco_a_00030) [find](https://scholar.google.com/scholar?q=Jimmy+Secretan%2C+Nicholas+Beato%2C+David+B%2E+D%E2%80%99Ambrosio%2C+Adelein+Rodriguez%2C+Adam+Campbell%2C+Jeremiah+T%2E+Folsom%2DKovarik%2C+and+Kenneth+O%2E+Stanley%2E+Picbreeder%3A+A+case+study+in+collaborative+evolutionary+explor)
75. Noah Shinn, Beck Labash, and Ashwin Gopinath. Reflexion: An autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.[11366, 2023](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/).
76. Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward gaming. Advances in Neural Information Processing Systems, 35:9460–9471, 2022. [find](https://scholar.google.com/scholar?q=Joar+Skalse%2C+Nikolaus+Howe%2C+Dmitrii+Krasheninnikov%2C+and+David+Krueger%2E+Defining+and+characterizing+reward+gaming%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A9460%E2%80%939471%2C+2022%2E)
77. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. [Recursive deep models for semantic compositionality over a sentiment treebank](https://md.research.elamur.ai/free/en/papers/socher-2013-recursive-deep-models-semantic-compositionality/). In Proceedings of the 2013 conference on empirical methods in natural language processing, pp. 1631–1642,…
78. Kenneth O Stanley. Compositional pattern producing networks: A novel abstraction of development. Genetic programming and evolvable machines, 8:131–162, 2007. [find](https://scholar.google.com/scholar?q=Kenneth+O+Stanley%2E+Compositional+pattern+producing+networks%3A+A+novel+abstraction+of+development%2E+Genetic+programming+and+evolvable+machines%2C+8%3A131%E2%80%93162%2C+2007%2E)
79. Kenneth O. Stanley and Joel Lehman. Why greatness cannot be planned: The myth of the objective. Springer, 2015. [find](https://scholar.google.com/scholar?q=Kenneth+O%2E+Stanley+and+Joel+Lehman%2E+Why+greatness+cannot+be+planned%3A+The+myth+of+the+objective%2E+Springer%2C+2015%2E)
80. Kenneth O. Stanley, Joel Lehman, and Lisa Soros. Open-endedness: The last grand challenge you’ve never heard of. O’Reilly Radar, 2017. [find](https://scholar.google.com/scholar?q=Kenneth+O%2E+Stanley%2C+Joel+Lehman%2C+and+Lisa+Soros%2E+Open%2Dendedness%3A+The+last+grand+challenge+you%E2%80%99ve+never+heard+of%2E+O%E2%80%99Reilly+Radar%2C+2017%2E)
81. Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. [Roformer: Enhanced transformer with rotary position embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/). arXiv preprint arXiv:2104.09864, 2021.
82. Shyam Sudhakaran, Miguel González-Duque, Claire Glanois, Matthias Freiberger, Elias Najarro, and Sebastian Risi. Mariogpt: Open-ended text2level generation through large language models, 2023. [find](https://scholar.google.com/scholar?q=Shyam+Sudhakaran%2C+Miguel+Gonz%C3%A1lez%2DDuque%2C+Claire+Glanois%2C+Matthias+Freiberger%2C+Elias+Najarro%2C+and+Sebastian+Risi%2E+Mariogpt%3A+Open%2Dended+text2level+generation+through+large+language+models%2C+2023%2E)
83. Graham Todd, Sam Earle, Muhammad Umair Nasir, Michael Cerny Green, and Julian Togelius. Level generation through large language models. In Proceedings of the 18th International Conference on the Foundations of Digital Games, pp. 1–8, 2023. [find](https://scholar.google.com/scholar?q=Graham+Todd%2C+Sam+Earle%2C+Muhammad+Umair+Nasir%2C+Michael+Cerny+Green%2C+and+Julian+Togelius%2E+Level+generation+through+large+language+models%2E+In+Proceedings+of+the+18th+International+Conference+on+the+Found)
84. Vassilis Vassiliades, Konstantinos Chatzilygeroudis, and Jean-Baptiste Mouret. Scaling up MAP-Elites using centroidal Voronoi tessellations. arXiv preprint arXiv:1610.05729, 2016. — [arXiv:1610.05729](https://arxiv.org/abs/1610.05729) [find](https://scholar.google.com/scholar?q=Vassilis+Vassiliades%2C+Konstantinos+Chatzilygeroudis%2C+and+Jean%2DBaptiste+Mouret%2E+Scaling+up+MAP%2DElites+using+centroidal+Voronoi+tessellations%2E+arXiv+preprint+arXiv%3A1610%2E05729%2C+2016%2E+%E2%80%94+%5BarXiv%3A1610%2E05729%5D)
85. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023a. [find](https://scholar.google.com/scholar?q=Guanzhi+Wang%2C+Yuqi+Xie%2C+Yunfan+Jiang%2C+Ajay+Mandlekar%2C+Chaowei+Xiao%2C+Yuke+Zhu%2C+Linxi+Fan%2C+and+Anima+Anandkumar%2E+Voyager%3A+An+open%2Dended+embodied+agent+with+large+language+models%2E+arXiv+preprint+arXiv%3A23)
86. Ren-Jian Wang, Ke Xue, Yutong Wang, Peng Yang, Haobo Fu, Qiang Fu, and Chao Qian. Diversity from human feedback. arXiv preprint arXiv:2310.06648, 2023b. — [arXiv:2310.06648](https://arxiv.org/abs/2310.06648) [find](https://scholar.google.com/scholar?q=Ren%2DJian+Wang%2C+Ke+Xue%2C+Yutong+Wang%2C+Peng+Yang%2C+Haobo+Fu%2C+Qiang+Fu%2C+and+Chao+Qian%2E+Diversity+from+human+feedback%2E+arXiv+preprint+arXiv%3A2310%2E06648%2C+2023b%2E+%E2%80%94+%5BarXiv%3A2310%2E06648%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310)
87. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560, 2022a. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2Dinstruct%3A+Aligning+language+model+with+self+generated+instructions%2E+arXiv+preprint+)
88. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Super-NaturalInstructions: Generalization via declarative instructions on 1600+ NLP tasks. arXiv preprint arXiv:2204.07705,… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Swaroop+Mishra%2C+Pegah+Alipoormolabashi%2C+Yeganeh+Kordi%2C+Amirreza+Mirzaei%2C+Anjana+Arunkumar%2C+Arjun+Ashok%2C+Arut+Selvan+Dhanasekaran%2C+Atharva+Naik%2C+David+Stap%2C+et+al%2E+Super%2DNaturalInstructio)
89. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). arXiv preprint arXiv:2109.01652, 2021.
90. Jenny Zhang, Joel Lehman, Kenneth Stanley, and Jeff Clune. OMNI: Open-endedness via models of human notions of interestingness. arXiv preprint arXiv:2306.01711, 2023. [find](https://scholar.google.com/scholar?q=Jenny+Zhang%2C+Joel+Lehman%2C+Kenneth+Stanley%2C+and+Jeff+Clune%2E+OMNI%3A+Open%2Dendedness+via+models+of+human+notions+of+interestingness%2E+arXiv+preprint+arXiv%3A2306%2E01711%2C+2023%2E)
91. Yulun Zhang, Matthew C Fontaine, Amy K Hoover, and Stefanos Nikolaidis. Deep surrogate assisted MAP-Elites for automated hearthstone deckbuilding. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 158–167, 2022. [find](https://scholar.google.com/scholar?q=Yulun+Zhang%2C+Matthew+C+Fontaine%2C+Amy+K+Hoover%2C+and+Stefanos+Nikolaidis%2E+Deep+surrogate+assisted+MAP%2DElites+for+automated+hearthstone+deckbuilding%2E+In+Proceedings+of+the+Genetic+and+Evolutionary+Comput)

*Updated 30 October 2026*