---
title: "Surveying the Effects of Quality, Diversity, and Complexity in"
description: "The methods that create it differ in seed data, generator models, prompts, filters and task-specific… 248 references, cited by 12 papers here."
authors: ["Alex Havrilla", "Andrew Dai", "Laura O'Mahony", "Koen Oostermeijer", "Vera Zisler", "Alon Albalak", "Fabrizio Milo", "Sharath Chandra Raparthy", "Kanishk Gandhi", "Baber Abbasi", "Duy Phung", "Maia Iyer", "Dakota Mahan", "Chase Blagden", "Srishti Gureja", "Mohammed Hamdy", "Wen-Ding Li", "Giovanni Paolini", "Pawan Sasanka Ammanamanchi", "Elliot Meyerson"]
year: 2024
venue: "arXiv:2412.02980"
source: "https://arxiv.org/abs/2412.02980"
doi: "10.48550/arxiv.2412.02980"
arxiv: "2412.02980"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/havrilla-2024-surveying-effects-quality-diversity-complexity/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/havrilla-2024-surveying-effects-quality-diversity-complexity/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/12/) / Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models

# Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models

**Authors:** [Alex Havrilla](https://md.research.elamur.ai/free/en/authors/alex-havrilla/), [Andrew Dai](https://md.research.elamur.ai/free/en/authors/andrew-m-dai/), [Laura O'Mahony](https://md.research.elamur.ai/free/en/authors/laura-o-mahony/), [Koen Oostermeijer](https://md.research.elamur.ai/free/en/authors/koen-oostermeijer/), Vera Zisler, [Alon Albalak](https://md.research.elamur.ai/free/en/authors/alon-albalak/), Fabrizio Milo, [Sharath Chandra Raparthy](https://md.research.elamur.ai/free/en/authors/sharath-chandra-raparthy/), [Kanishk Gandhi](https://md.research.elamur.ai/free/en/authors/kanishk-gandhi/), Baber Abbasi, [Duy Phung](https://md.research.elamur.ai/free/en/authors/duy-phung/), Maia Iyer, [Dakota Mahan](https://md.research.elamur.ai/free/en/authors/dakota-mahan/), [Chase Blagden](https://md.research.elamur.ai/free/en/authors/chase-blagden/), Srishti Gureja, Mohammed Hamdy, [Wen-Ding Li](https://md.research.elamur.ai/free/en/authors/wen-ding-li/), Giovanni Paolini, Pawan Sasanka Ammanamanchi and Elliot Meyerson · **Year:** 2024 · **Venue:** arXiv:2412.02980 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2412.02980) · [PDF](https://arxiv.org/pdf/2412.02980) · [DOI 10.48550/arxiv.2412.02980](https://doi.org/10.48550/arxiv.2412.02980) · [arXiv 2412.02980](https://arxiv.org/abs/2412.02980) · [OpenAlex](https://openalex.org/W4405089011)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [12](https://md.research.elamur.ai/free/en/papers/havrilla-2024-surveying-effects-quality-diversity-complexity/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Synthetic data produced by LLMs is now a common way to expand training sets. The methods that create it differ in seed data, generator models, prompts, filters and task-specific tricks, so it is hard to see which design choices actually help. This survey suggests judging generation algorithms by the properties of the data they produce rather than by their recipes. Those properties are quality, diversity and complexity (QDC). The authors first collect the many practical metrics used for each property. They then review evidence on how each property shapes trained models and classify pipeline components by which property they strengthen. Their synthesis is that quality mainly supports in-distribution performance, diversity supports out-of-distribution generalization, and well-calibrated complexity helps both. Quality and diversity often conflict, both within training sets and in model outputs. Most models and benchmarks reward output quality alone, which narrows output diversity. The authors argue this limits RL and recursive self-improvement, and they point to quality-diversity search methods as a promising way to balance the two.

## Key points
- The survey is organized around three research questions: how to define and measure QDC, how QDC in training data affects generalization, and how existing generation algorithms promote each property.
- Typical pipelines secure quality by sampling from a large SOTA teacher model and diversity by starting from a big seed corpus, while complexity is often not addressed. Filter-heavy pipelines discard most of the data they generate (Zhou et al., 2023a).
- The appendix tables catalog the metrics used in the literature:
  - **Diversity:** n-gram statistics, MTLD, average pairwise cosine similarity of embeddings, SemDeDup, NLI diversity and MAP-Elites coverage.
  - **Complexity:** parse tree complexity, instruction length, difficulty level and semantic node count.
  - **Quality:** reward models, perplexity and high-quality-vs-unfiltered classifiers.
- The mechanisms are grouped into generation, filtration and distillation stages:
  - **Complexity:** evolving samples (Xu et al., 2023) and curriculum or weak-to-strong RL.
  - **Diversity:** decoding strategies, diverse seed datasets, deduplication, search and model ensembling.
- The open problems named include benchmarks that measure quality and diversity jointly, algorithms that explicitly manage the output quality-diversity trade-off, and a clearer understanding of how complexity interacts with the other two properties. The authors draw inspiration from classical QD search (Lehman & Stanley, 2011; Mouret & Clune, 2015).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 567 works · 335,190 citations
- [Cornell University](https://md.research.elamur.ai/free/en/universities/cornell-university/) — University, Ithaca, United States · 139 works · 65,295 citations
- [IBM](https://md.research.elamur.ai/free/en/companies/ibm/) — Company, Armonk, United States · 163 works · 73,139 citations
- [Georgia Institute of Technology](https://md.research.elamur.ai/free/en/universities/georgia-institute-of-technology/) — University, Atlanta, United States · 159 works · 28,096 citations
- [University of Bologna](https://md.research.elamur.ai/free/en/universities/university-of-bologna/) — University, Bologna, Italy · 22 works · 2,122 citations
- [University of Limerick](https://md.research.elamur.ai/free/en/universities/university-of-limerick/) — University, Limerick, Ireland · 2 works · 18 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works

## Cite

Cite as: Alex Havrilla, Andrew Dai, Laura O'Mahony, Koen Oostermeijer, Vera Zisler, Alon Albalak, Fabrizio Milo, Sharath Chandra Raparthy, Kanishk Gandhi, Baber Abbasi, Duy Phung, Maia Iyer, Dakota Mahan, Chase Blagden, Srishti Gureja, Mohammed Hamdy, Wen-Ding Li, Giovanni Paolini, Pawan Sasanka Ammanamanchi and Elliot Meyerson. 2024. Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models. arXiv:2412.02980.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{havrilla2024surveying,
  title = {{Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models}},
  author = {Alex Havrilla, Andrew Dai, Laura O'Mahony, Koen Oostermeijer, Vera Zisler, Alon Albalak, Fabrizio Milo, Sharath Chandra Raparthy, Kanishk Gandhi, Baber Abbasi, Duy Phung, Maia Iyer, Dakota Mahan, Chase Blagden, Srishti Gureja, Mohammed Hamdy, Wen-Ding Li, Giovanni Paolini, Pawan Sasanka Ammanamanchi and Elliot Meyerson},
  year = {2024},
  doi = {10.48550/arxiv.2412.02980},
  eprint = {2412.02980},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2412.02980}
}
```

- [← previous RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval](https://md.research.elamur.ai/free/en/papers/wen-2024-rnns-transformers-yet-key-bottleneck/)
- [next → SWE-Bench+: Enhanced Coding Benchmark for LLMs](https://md.research.elamur.ai/free/en/papers/aleithan-2024-swe-bench-enhanced-coding/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/12/)

## References

248 references, 125 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Amro Abbas, Kushal Tirumala, Dániel Simig, Surya Ganguli, and Ari S. Morcos. [Semdedup: Data-efficient learning at web-scale through semantic deduplication](https://md.research.elamur.ai/free/en/papers/abbas-2023-semdedup-data-efficient-learning-web/), 2023. URL [https://arxiv.org/](https://arxiv.org/) abs/2303.09540. [↗](https://arxiv.org/)
2. Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. [Phi-3 technical report: A highly capable language model locally on your phone](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/). arXiv preprint arXiv:2404.14219, 2024.
3. Oshin Agarwal, Heming Ge, Siamak Shakeri, and Rami Al-Rfou. [Knowledge graph based synthetic corpus generation for knowledge-enhanced language model pre-training](https://md.research.elamur.ai/free/en/papers/agarwal-2021-knowledge-graph-based-synthetic-corpus/). In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty…
4. Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms, 2024. URL https://arxiv.org/abs/2402.14740. [↗](https://arxiv.org/abs/2402.14740)
5. Alon Albalak, Liangming Pan, Colin Raffel, and William Yang Wang. [Efficient online data mixing for language model pre-training](https://md.research.elamur.ai/free/en/papers/albalak-2023-efficient-online-data-mixing-language/). In R0-FoMo:Robustness of Few-shot and Zero-shot Learning in Large Foundation Models, 2023a. URL [https://openreview.net/forum?id=9Tze4oy4lw](https://openreview.net/forum?id=9Tze4oy4lw). [↗](https://openreview.net/forum?id=9Tze4oy4lw)
6. Alon Albalak, Colin Raffel, and William Yang Wang. [Improving few-shot generalization by exploring and exploiting auxiliary data](https://md.research.elamur.ai/free/en/papers/albalak-2023-improving-few-shot-generalization-exploring/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023b. URL [https://openreview.net/forum?id=JDnLXc4NOn](https://openreview.net/forum?id=JDnLXc4NOn). [↗](https://openreview.net/forum?id=JDnLXc4NOn)
7. Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre, Nathan Lambert, Xinyi Wang, Niklas Muennighoff, Bairu Hou, Liangming Pan, Haewon Jeong, Colin Raffel, Shiyu Chang, Tatsunori Hashimoto, and William Yang Wang. [A survey on data selection for language models](https://md.research.elamur.ai/free/en/papers/albalak-2024-survey-data-selection-language-models/), 2024. URL [https://arxiv.org/abs…](https://arxiv.org/abs…) [↗](https://arxiv.org/abs…)
8. Zachary Ankner, Mansheej Paul, Brandon Cui, Jonathan D. Chang, and Prithviraj Ammanabrolu. [Critique-out-loud reward models](https://md.research.elamur.ai/free/en/papers/ankner-2024-critique-out-loud-reward-models/), 2024. URL https://arxiv.org/abs/2408.11791. [↗](https://arxiv.org/abs/2408.11791)
9. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
10. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared… — [Constitutional AI: Harmlessness from AI Feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
11. André Bauer, Simon Trapp, Michael Stenger, Robert Leppich, Samuel Kounev, Mark Leznik, Kyle Chard, and Ian Foster. Comprehensive exploration of synthetic data generation: A survey, 2024. URL https://arxiv.org/abs/2401.02524. — [arXiv:2401.02524](https://arxiv.org/abs/2401.02524) [↗](https://arxiv.org/abs/2401.02524)
12. Loubna Ben Allal, Anton Lozhkov, Guilherme Penedo, Thomas Wolf, and Leandro von Werra. Cosmopedia, February 2024. URL [https://huggingface.co/datasets/HuggingFaceTB/](https://huggingface.co/datasets/HuggingFaceTB/) cosmopedia. [↗](https://huggingface.co/datasets/HuggingFaceTB/)
13. Charles H. Bennett. Logical depth and physical complexity. semanticscholar.org/CorpusID:6365510. 1988. [find](https://scholar.google.com/scholar?q=Charles+H%2E+Bennett%2E+Logical+depth+and+physical+complexity%2E+semanticscholar%2Eorg%2FCorpusID%3A6365510%2E+1988%2E)
14. URL https://api. Varun Bhatt, Bryon Tjanaka, Matthew Fontaine, and Stefanos Nikolaidis. Deep surrogate assisted generation of environments. Advances in Neural Information Processing Systems, 35:37762–37777, 2022. [find](https://scholar.google.com/scholar?q=URL+https%3A%2F%2Fapi%2E+Varun+Bhatt%2C+Bryon+Tjanaka%2C+Matthew+Fontaine%2C+and+Stefanos+Nikolaidis%2E+Deep+surrogate+assisted+generation+of+environments%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A37762%E2%80%93)
15. Michael Boratko, Xiang Lorraine Li, Rajarshi Das, Tim O’Gorman, Dan Le, and Andrew McCallum. Protoqa: A question answering dataset for prototypical common-sense reasoning, 2020. URL https://arxiv.org/abs/2005.00771. [↗](https://arxiv.org/abs/2005.00771)
16. Herbie Bradley, Andrew Dai, Hannah Teufel, Jenny Zhang, Koen Oostermeijer, Marco Bellagente, Jeff Clune, Kenneth Stanley, Grégory Schott, and Joel Lehman. [Quality-diversity through ai feedback](https://md.research.elamur.ai/free/en/papers/bradley-2023-quality-diversity-through-ai-feedback/), 2023. URL https://arxiv.org/abs/2310.13032. [↗](https://arxiv.org/abs/2310.13032)
17. Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, and Azalia Mirhoseini. [Large language monkeys: Scaling inference compute with repeated sampling](https://md.research.elamur.ai/free/en/papers/brown-2024-large-language-monkeys-scaling-inference/), 2024. URL https://arxiv.org/abs/2407.21787. [↗](https://arxiv.org/abs/2407.21787)
18. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretchen)
19. Curran Associates, Inc., 2020a. URL [https://proceedings.neurips.cc/paper_files/paper/](https://proceedings.neurips.cc/paper_files/paper/) 2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf. [↗](https://proceedings.neurips.cc/paper_files/paper/)
20. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
21. Alexander Bukharin and Tuo Zhao. Data diversity matters for robust instruction tuning, 2024. URL https://arxiv.org/abs/2311.14736. [↗](https://arxiv.org/abs/2311.14736)
22. Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever, and Jeff Wu. [Weak-to-strong generalization: Eliciting strong capabilities with weak supervision](https://md.research.elamur.ai/free/en/papers/burns-2023-weak-strong-generalization-eliciting-strong/), 2023. URL [https://arxiv.org/](https://arxiv.org/) abs/2… [↗](https://arxiv.org/)
23. Yihan Cao, Yanbin Kang, Chi Wang, and Lichao Sun. [Instruction mining: Instruction data selection for tuning large language models](https://md.research.elamur.ai/free/en/papers/cao-2023-instruction-mining-instruction-data-selection/), 2024. URL https://arxiv.org/abs/2307.06290. [↗](https://arxiv.org/abs/2307.06290)
24. Navoneel Chakrabarty. A machine learning approach to comment toxicity classification, 2019. URL https://arxiv.org/abs/1903.06765. — [arXiv:1903.06765](https://arxiv.org/abs/1903.06765) [↗](https://arxiv.org/abs/1903.06765)
25. Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. [Scaling synthetic data creation with 1,000,000,000 personas](https://md.research.elamur.ai/free/en/papers/ge-2024-scaling-synthetic-data-creation-1/), 2024. URL https://arxiv.org/abs/2406.20094. [↗](https://arxiv.org/abs/2406.20094)
26. Allen Chang, Matthew C. Fontaine, Serena Booth, Maja J. Matarić, and Stefanos Nikolaidis. Quality-diversity generative sampling for learning with synthetic data, 2024. URL [https://arxiv.org/](https://arxiv.org/) abs/2312.14369. — [arXiv:2312.14369](https://arxiv.org/abs/2312.14369) [↗](https://arxiv.org/)
27. Wang Chao, Jiaxuan Zhao, Licheng Jiao, Lingling Li, Fang Liu, and Shuyuan Yang. A match made in consistency heaven: when large language models meet evolutionary algorithms. arXiv preprint arXiv:2401.10510, 2024. — [arXiv:2401.10510](https://arxiv.org/abs/2401.10510) [find](https://scholar.google.com/scholar?q=Wang+Chao%2C+Jiaxuan+Zhao%2C+Licheng+Jiao%2C+Lingling+Li%2C+Fang+Liu%2C+and+Shuyuan+Yang%2E+A+match+made+in+consistency+heaven%3A+when+large+language+models+meet+evolutionary+algorithms%2E+arXiv+preprint+arXiv%3A2401%2E1)
28. Konstantinos Chatzilygeroudis, Antoine Cully, Vassilis Vassiliades, and Jean-Baptiste Mouret. Quality-diversity optimization: a novel branch of stochastic optimization. In Black Box Optimization, Machine Learning, and No-Free Lunch Theorems, pp. 109–135. Springer, 2021. [find](https://scholar.google.com/scholar?q=Konstantinos+Chatzilygeroudis%2C+Antoine+Cully%2C+Vassilis+Vassiliades%2C+and+Jean%2DBaptiste+Mouret%2E+Quality%2Ddiversity+optimization%3A+a+novel+branch+of+stochastic+optimization%2E+In+Black+Box+Optimization%2C+Mach)
29. Angelica Chen, David M. Dohan, and David R. So. Evoprompting: Language models for code-level neural architecture search, 2023a. URL https://arxiv.org/abs/2302.14838. — [arXiv:2302.14838](https://arxiv.org/abs/2302.14838) [↗](https://arxiv.org/abs/2302.14838)
30. Hao Chen, Abdul Waheed, Xiang Li, Yidong Wang, Jindong Wang, Bhiksha Raj, and Marah I. Abdin. [On the diversity of synthetic data and its impact on training large language models](https://md.research.elamur.ai/free/en/papers/chen-2024-diversity-synthetic-data-impact-training/), 2024a. URL https://arxiv.org/abs/2410.15226. [↗](https://arxiv.org/abs/2410.15226)
31. Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. [Alpagasus: Training a better alpaca with fewer data](https://md.research.elamur.ai/free/en/papers/chen-2023-alpagasus-training-better-alpaca-fewer/), 2024b. URL https://arxiv.org/abs/2307.08701. [↗](https://arxiv.org/abs/2307.08701)
32. Mayee F. Chen, Nicholas Roberts, Kush Bhatia, Jue Wang, Ce Zhang, Frederic Sala, and Christopher Ré. Skill-it! a data-driven skills framework for understanding and training language models, 2023b. URL https://arxiv.org/abs/2307.14430. — [arXiv:2307.14430](https://arxiv.org/abs/2307.14430) [↗](https://arxiv.org/abs/2307.14430)
33. Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks, 2023c. URL https://arxiv.org/abs/2211.12588. [↗](https://arxiv.org/abs/2211.12588)
34. Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models, 2024c. URL https://arxiv.org/abs/2401. 01335. [↗](https://arxiv.org/abs/2401)
35. Zoey Chen, Sho Kiami, Abhishek Gupta, and Vikash Kumar. Genaug: Retargeting behaviors to unseen situations via generative augmentation. arXiv preprint arXiv:2302.06671, 2023d. — [arXiv:2302.06671](https://arxiv.org/abs/2302.06671) [find](https://scholar.google.com/scholar?q=Zoey+Chen%2C+Sho+Kiami%2C+Abhishek+Gupta%2C+and+Vikash+Kumar%2E+Genaug%3A+Retargeting+behaviors+to+unseen+situations+via+generative+augmentation%2E+arXiv+preprint+arXiv%3A2302%2E06671%2C+2023d%2E+%E2%80%94+%5BarXiv%3A2302%2E06671%5D%28htt)
36. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL [https://lmsys.org/](https://lmsys.org/) blog/2023-03-30-vicuna… [↗](https://lmsys.org/)
37. Geoffrey Cideron, Andrea Agostinelli, Johan Ferret, Sertan Girgin, Romuald Elie, Olivier Bachem, Sarah Perrin, and Alexandre Ramé. Diversity-rewarded cfg distillation, 2024. URL [https://arxiv.org/](https://arxiv.org/) abs/2410.06084. — [arXiv:2410.06084](https://arxiv.org/abs/2410.06084) [↗](https://arxiv.org/)
38. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/), 2021. URL https://arxiv.org/abs/2110.14168. [↗](https://arxiv.org/abs/2110.14168)
39. Antoine Cully and Yiannis Demiris. Quality and diversity optimization: A unifying modular framework. IEEE Transactions on Evolutionary Computation, 22(2):245–259, 2017. [find](https://scholar.google.com/scholar?q=Antoine+Cully+and+Yiannis+Demiris%2E+Quality+and+diversity+optimization%3A+A+unifying+modular+framework%2E+IEEE+Transactions+on+Evolutionary+Computation%2C+22%282%29%3A245%E2%80%93259%2C+2017%2E)
40. Christine E. DeMars. Item response theory. Assessing Measurement Invariance for Applied Research, 2010. URL [https://api.semanticscholar.org/CorpusID:15155431](https://api.semanticscholar.org/CorpusID:15155431). [↗](https://api.semanticscholar.org/CorpusID:15155431)
41. Bosheng Ding, Chengwei Qin, Linlin Liu, Yew Ken Chia, Shafiq Joty, Boyang Li, and Lidong Bing. [Is gpt-3 a good data annotator?](https://md.research.elamur.ai/free/en/papers/ding-2022-gpt-3-good-data-annotator/), 2023. URL https://arxiv.org/abs/2212.10450. [↗](https://arxiv.org/abs/2212.10450)
42. Li Ding, Jenny Zhang, Jeff Clune, Lee Spector, and Joel Lehman. Quality diversity through human feedback: Towards open-ended diversity-driven optimization, 2024. URL https://arxiv.org/abs/2310. 12103. [↗](https://arxiv.org/abs/2310)
43. Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. [How abilities in large language models are affected by supervised fine-tuning data composition](https://md.research.elamur.ai/free/en/papers/dong-2024-how-abilities-large-language-models/), 2024. URL https://arxiv.org/abs/2310.05492. [↗](https://arxiv.org/abs/2310.05492)
44. Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. [Raft: Reward ranked finetuning for generative foundation model alignment](https://md.research.elamur.ai/free/en/papers/dong-2023-raft-reward-ranked-finetuning-generative/), 2023. URL https://arxiv.org/abs/2304.06767. [↗](https://arxiv.org/abs/2304.06767)
45. Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen Meier-Hellstern, Toju Duke, Lucas… — [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://md.research.elamur.ai/free/en/papers/du-2021-glam-efficient-scaling-language-models/)
46. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez,… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
47. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783, 2024b.
48. Ronen Eldan and Yuanzhi Li. [Tinystories: How small can language models be and still speak coherent english?](https://md.research.elamur.ai/free/en/papers/eldan-2023-tinystories-small-language-models-still/), 2023. URL https://arxiv.org/abs/2305.07759. [↗](https://arxiv.org/abs/2305.07759)
49. Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding dataset difficulty with V-usable information. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of… [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2C+Yejin+Choi%2C+and+Swabha+Swayamdipta%2E+Understanding+dataset+difficulty+with+V%2Dusable+information%2E+In+Kamalika+Chaudhuri%2C+Stefanie+Jegelka%2C+Le+Song%2C+Csaba+Szepesvari%2C+Gang+Niu%2C+and+Siva)
50. Lijie Fan, Kaifeng Chen, Dilip Krishnan, Dina Katabi, Phillip Isola, and Yonglong Tian. Scaling laws of synthetic images for model training ... for now, 2023. URL https://arxiv.org/abs/2312. 04567. [↗](https://arxiv.org/abs/2312)
51. Simin Fan and Martin Jaggi. Irreducible curriculum for language model pretraining, 2023. URL https://arxiv.org/abs/2310.15389. — [arXiv:2310.15389](https://arxiv.org/abs/2310.15389) [↗](https://arxiv.org/abs/2310.15389)
52. Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. [Promptbreeder: Self-referential self-improvement via prompt evolution](https://md.research.elamur.ai/free/en/papers/fernando-2023-promptbreeder-self-referential-self-improvement/), 2023a.
53. Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. [Promptbreeder: Self-referential self-improvement via prompt evolution](https://md.research.elamur.ai/free/en/papers/fernando-2023-promptbreeder-self-referential-self-improvement/), 2023b. URL [https://arxiv.org/](https://arxiv.org/) abs/2309.16797. [↗](https://arxiv.org/)
54. Matthew Fontaine and Stefanos Nikolaidis. A quality diversity approach to automatically generating human-robot interaction scenarios in shared autonomy. Robotics: Science and Systems (RSS), 2021. [find](https://scholar.google.com/scholar?q=Matthew+Fontaine+and+Stefanos+Nikolaidis%2E+A+quality+diversity+approach+to+automatically+generating+human%2Drobot+interaction+scenarios+in+shared+autonomy%2E+Robotics%3A+Science+and+Systems+%28RSS%29%2C+2021%2E)
55. Matthew Fontaine, Sophie Hsu, Yulun Zhang, Bryon Tjanaka, and Stefanos Nikolaidis. On the importance of environments in human-robot coordination. Robotics: Science and Systems (RSS), 2021a. [find](https://scholar.google.com/scholar?q=Matthew+Fontaine%2C+Sophie+Hsu%2C+Yulun+Zhang%2C+Bryon+Tjanaka%2C+and+Stefanos+Nikolaidis%2E+On+the+importance+of+environments+in+human%2Drobot+coordination%2E+Robotics%3A+Science+and+Systems+%28RSS%29%2C+2021a%2E)
56. Matthew C. Fontaine, Ruilin Liu, Ahmed Khalifa, Jignesh Modi, Julian Togelius, Amy K. Hoover, and Stefanos Nikolaidis. Illuminating mario scenes in the latent space of a generative adversarial network, 2021b. URL https://arxiv.org/abs/2007.05674. — [arXiv:2007.05674](https://arxiv.org/abs/2007.05674) [↗](https://arxiv.org/abs/2007.05674)
57. Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah D. Goodman. [Stream of search (sos): Learning to search in language](https://md.research.elamur.ai/free/en/papers/gandhi-2024-stream-search-sos-learning-search/), 2024a. URL https://arxiv.org/abs/2404.03683. [↗](https://arxiv.org/abs/2404.03683)
58. Saumya Gandhi, Ritu Gala, Vijay Viswanathan, Tongshuang Wu, and Graham Neubig. Better synthetic data by retrieving and transforming existing datasets, 2024b. URL https://arxiv.org/abs/2404. 14361. [↗](https://arxiv.org/abs/2404)
59. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb dataset of diverse text for language modeling, 2020. URL https://arxiv.org/abs/2101.00027. [↗](https://arxiv.org/abs/2101.00027)
60. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. [Critic: Large language models can self-correct with tool-interactive critiquing](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/), 2024. URL https://arxiv.org/abs/2305.11738. [↗](https://arxiv.org/abs/2305.11738)
61. Alex Graves, Marc G. Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu. Automated curriculum learning for neural networks, 2017. URL https://arxiv.org/abs/1704.03003. [↗](https://arxiv.org/abs/1704.03003)
62. Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. [Reinforced self-training (rest) for language modeling](https://md.research.elamur.ai/free/en/papers/gulcehre-2023-reinforced-self-training-rest-language/), 2023. URL https:/…
63. Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. Textbooks are all you need. arXiv preprint arXiv:2306.11644, 2023. [find](https://scholar.google.com/scholar?q=Suriya+Gunasekar%2C+Yi+Zhang%2C+Jyoti+Aneja%2C+Caio+Ce%CC%81sar+Teodoro+Mendes%2C+Allie+Del+Giorno%2C+Sivakanth+Gopi%2C+Mojan+Javaheripi%2C+Piero+Kauffmann%2C+Gustavo+de+Rosa%2C+Olli+Saarikivi%2C+et+al%2E+Textbooks+are+all+you+)
64. Xu Guo and Yiqiang Chen. Generative ai for synthetic data generation: Methods, challenges and the future, 2024. URL https://arxiv.org/abs/2403.04190. — [arXiv:2403.04190](https://arxiv.org/abs/2403.04190) [↗](https://arxiv.org/abs/2403.04190)
65. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models, 2024. URL https://arxiv.org/abs/2301.04104. [↗](https://arxiv.org/abs/2301.04104)
66. Tatsunori B. Hashimoto, Hugh Zhang, and Percy Liang. [Unifying human and statistical evaluation for natural language generation](https://md.research.elamur.ai/free/en/papers/hashimoto-2019-unifying-human-statistical-evaluation-natural/), 2019. URL https://arxiv.org/abs/1904.02792. [↗](https://arxiv.org/abs/1904.02792)
67. Alex Havrilla and Wenjing Liao. Understanding scaling laws with statistical and approximation theory for transformer neural networks on intrinsically low-dimensional data, 2024. URL https://arxiv.org/abs/2411.06646. — [arXiv:2411.06646](https://arxiv.org/abs/2411.06646) [↗](https://arxiv.org/abs/2411.06646)
68. Alex Havrilla, Yuqing Du, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi-Yu, Maksym Zhuravinskyi, Eric Hambro, Sainbayar Sukhbaatar, and Roberta Raileanu. [Teaching large language models to reason with reinforcement learning](https://md.research.elamur.ai/free/en/papers/havrilla-2024-teaching-large-language-models-reason/), 2024a. URL https://arxiv.org/abs/2403.04642. [↗](https://arxiv.org/abs/2403.04642)
69. Alex Havrilla, Sharath Raparthy, Christoforus Nalmpantis, Jane Dwivedi-Yu, Maksym Zhuravinskyi, Eric Hambro, and Roberta Raileanu. [Glore: When, where, and how to improve llm reasoning via global and local refinements](https://md.research.elamur.ai/free/en/papers/havrilla-2024-glore-where-improve-llm-reasoning/), 2024b. URL https://arxiv.org/abs/2402.10963. [↗](https://arxiv.org/abs/2402.10963)
70. Alexander Havrilla, Maksym Zhuravinskyi, Duy Phung, Aman Tiwari, Jonathan Tow, Stella Biderman, Quentin Anthony, and Louis Castricato. trlX: A framework for large scale reinforcement learning from human feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processi… [find](https://scholar.google.com/scholar?q=Alexander+Havrilla%2C+Maksym+Zhuravinskyi%2C+Duy+Phung%2C+Aman+Tiwari%2C+Jonathan+Tow%2C+Stella+Biderman%2C+Quentin+Anthony%2C+and+Louis+Castricato%2E+trlX%3A+A+framework+for+large+scale+reinforcement+learning+from+hum)
71. Joy He-Yueya, Wanjing Anya Ma, Kanishk Gandhi, Benjamin W Domingue, Emma Brunskill, and Noah D Goodman. [Psychometric alignment: Capturing human knowledge distributions via language models](https://md.research.elamur.ai/free/en/papers/he-yueya-2024-psychometric-alignment-capturing-human-knowledge/). arXiv preprint arXiv:2407.15645, 2024.
72. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/), 2021a. URL [https://arxiv.org/](https://arxiv.org/) abs/2009.03300. [↗](https://arxiv.org/)
73. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/), 2021b. URL https://arxiv.org/abs/2103.03874. [↗](https://arxiv.org/abs/2103.03874)
74. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, E… — [Training Compute-Optimal Large Language Models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/)
75. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/), 2020. URL https://arxiv.org/abs/1904.09751. [↗](https://arxiv.org/abs/1904.09751)
76. Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. [Unnatural instructions: Tuning language models with (almost) no human labor](https://md.research.elamur.ai/free/en/papers/honovich-2023-unnatural-instructions-tuning-language-models/), 2022. URL https://arxiv.org/abs/2212.09689. [↗](https://arxiv.org/abs/2212.09689)
77. Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. [Large language models can self-improve](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/), 2022. URL https://arxiv.org/abs/2210.11610. [↗](https://arxiv.org/abs/2210.11610)
78. Edward Hughes, Michael Dennis, Jack Parker-Holder, Feryal Behbahani, Aditi Mavalankar, Yuge Shi, Tom Schaul, and Tim Rocktaschel. [Open-endedness is essential for artificial superhuman intelligence](https://md.research.elamur.ai/free/en/papers/hughes-2024-open-endedness-essential-artificial-superhuman/), 2024.
79. Mojan Javaheripi, Sébastien Bubeck, Marah Abdin, Jyoti Aneja, Sebastien Bubeck, Caio César Teodoro Mendes, Weizhu Chen, Allie Del Giorno, Ronen Eldan, Sivakanth Gopi, et al. Phi-2: The surprising power of small language models. Microsoft Research Blog, 2023. [find](https://scholar.google.com/scholar?q=Mojan+Javaheripi%2C+Se%CC%81bastien+Bubeck%2C+Marah+Abdin%2C+Jyoti+Aneja%2C+Sebastien+Bubeck%2C+Caio+Ce%CC%81sar+Teodoro+Mendes%2C+Weizhu+Chen%2C+Allie+Del+Giorno%2C+Ronen+Eldan%2C+Sivakanth+Gopi%2C+et+al%2E+Phi%2D2%3A+The+surprising+po)
80. Minqi Jiang, Tim Rocktäschel, and Edward Grefenstette. General intelligence requires rethinking exploration. Royal Society Open Science, 10(6):230539, 2023. [find](https://scholar.google.com/scholar?q=Minqi+Jiang%2C+Tim+Rockta%CC%88schel%2C+and+Edward+Grefenstette%2E+General+intelligence+requires+rethinking+exploration%2E+Royal+Society+Open+Science%2C+10%286%29%3A230539%2C+2023%2E)
81. Xue Jiang, Yihong Dong, Lecheng Wang, Zheng Fang, Qiwei Shang, Ge Li, Zhi Jin, and Wenpin Jiao. [Self-planning code generation with large language models](https://md.research.elamur.ai/free/en/papers/jiang-2023-self-planning-code-generation-large/), 2024a. URL https://arxiv.org/abs/ 2303.06689. [↗](https://arxiv.org/abs/)
82. Yiding Jiang, Allan Zhou, Zhili Feng, Sadhika Malladi, and J. Zico Kolter. Adaptive data optimization: Dynamic sample selection with scaling laws, 2024b. URL https://arxiv.org/abs/2410.11820. — [arXiv:2410.11820](https://arxiv.org/abs/2410.11820) [↗](https://arxiv.org/abs/2410.11820)
83. Julie Kallini, Isabel Papadimitriou, Richard Futrell, Kyle Mahowald, and Christopher Potts. [Mission: Impossible language models](https://md.research.elamur.ai/free/en/papers/kallini-2024-mission-impossible-language-models/), 2024. URL https://arxiv.org/abs/2401.06416. [↗](https://arxiv.org/abs/2401.06416)
84. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/), 2020. URL https://arxiv.org/abs/2001.08361. [↗](https://arxiv.org/abs/2001.08361)
85. Sungdong Kim, Sanghwan Bae, Jamin Shin, Soyoung Kang, Donghyun Kwak, Kang Min Yoo, and Minjoon Seo. Aligning large language models through synthetic feedback, 2023. URL [https://arxiv.org/](https://arxiv.org/) abs/2305.13735. [↗](https://arxiv.org/)
86. Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. [Understanding the effects of rlhf on llm generalisation and diversity](https://md.research.elamur.ai/free/en/papers/kirk-2023-understanding-effects-rlhf-llm-generalisation/), 2024. URL https://arxiv.org/abs/2310.06452. [↗](https://arxiv.org/abs/2310.06452)
87. Tomasz Korbak, Ethan Perez, and Christopher L Buckley. Rl with kl penalties is better viewed as bayesian inference, 2022. URL https://arxiv.org/abs/2205.11275. [↗](https://arxiv.org/abs/2205.11275)
88. Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, Lei M Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal Behbahani, and Aleksandra Faust. Training language models to… — [Training Language Models to Self-Correct via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/kumar-2024-training-language-models-self-correct/)
89. Jack Lanchantin, Shubham Toshniwal, Jason Weston, Arthur Szlam, and Sainbayar Sukhbaatar. [Learning to reason and memorize with self-notes](https://md.research.elamur.ai/free/en/papers/lanchantin-2023-learning-reason-memorize-self-notes/), 2023. URL https://arxiv.org/abs/2305.00833. [↗](https://arxiv.org/abs/2305.00833)
90. Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven C. H. Hoi. [Coderl: Mastering code generation through pretrained models and deep reinforcement learning](https://md.research.elamur.ai/free/en/papers/le-2022-coderl-mastering-code-generation-through/), 2022. URL https://arxiv.org/abs/2207.01780. [↗](https://arxiv.org/abs/2207.01780)
91. Alycia Lee, Brando Miranda, Sudharsan Sundar, and Sanmi Koyejo. Beyond scale: the diversity coefficient as a data quality metric demonstrates llms are pre-trained on formally diverse data, 2023. URL https://arxiv.org/abs/2306.13840. — [arXiv:2306.13840](https://arxiv.org/abs/2306.13840) [↗](https://arxiv.org/abs/2306.13840)
92. Bruce W. Lee, Hyunsoo Cho, and Kang Min Yoo. Instruction tuning with human curriculum, 2024. URL https://arxiv.org/abs/2310.09518. [↗](https://arxiv.org/abs/2310.09518)
93. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. Deduplicating training data makes language models better. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Associat… [find](https://scholar.google.com/scholar?q=Katherine+Lee%2C+Daphne+Ippolito%2C+Andrew+Nystrom%2C+Chiyuan+Zhang%2C+Douglas+Eck%2C+Chris+Callison%2DBurch%2C+and+Nicholas+Carlini%2E+Deduplicating+training+data+makes+language+models+better%2E+In+Smaranda+Muresan%2C+P)
94. Joel Lehman and Kenneth O Stanley. Abandoning objectives: Evolution through the search for novelty alone. Evolutionary computation, 19(2):189–223, 2011a. [find](https://scholar.google.com/scholar?q=Joel+Lehman+and+Kenneth+O+Stanley%2E+Abandoning+objectives%3A+Evolution+through+the+search+for+novelty+alone%2E+Evolutionary+computation%2C+19%282%29%3A189%E2%80%93223%2C+2011a%2E)
95. Joel Lehman and Kenneth O Stanley. Evolving a diversity of virtual creatures through novelty search and local competition. In Proceedings of the 13th annual conference on Genetic and evolutionary computation, pp. 211–218, 2011b. [find](https://scholar.google.com/scholar?q=Joel+Lehman+and+Kenneth+O+Stanley%2E+Evolving+a+diversity+of+virtual+creatures+through+novelty+search+and+local+competition%2E+In+Proceedings+of+the+13th+annual+conference+on+Genetic+and+evolutionary+comp)
96. Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O. Stanley. Evolution through large models, 2022. URL https://arxiv.org/abs/2206.08896. — [arXiv:2206.08896](https://arxiv.org/abs/2206.08896) [↗](https://arxiv.org/abs/2206.08896)
97. Chen Li, Weiqi Wang, Jingcheng Hu, Yixuan Wei, Nanning Zheng, Han Hu, Zheng Zhang, and Houwen Peng. [Common 7b language models already possess strong math capabilities](https://md.research.elamur.ai/free/en/papers/li-2024-common-7b-language-models-already/), 2024a. URL https://arxiv.org/abs/2403.04706. [↗](https://arxiv.org/abs/2403.04706)
98. Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel, Jean Mercat, Mayee Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan Bitton, Marianna Nezhur… — [DataComp-LM: In search of the next generation of training sets for language models](https://md.research.elamur.ai/free/en/papers/li-2024-datacomp-lm-search-next-generation/)
99. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. [A diversity-promoting objective function for neural conversation models](https://md.research.elamur.ai/free/en/papers/li-2016-diversity-promoting-objective-function-neural/). In Kevin Knight, Ani Nenkova, and Owen Rambow (eds.), Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational…
100. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. [A diversity-promoting objective function for neural conversation models](https://md.research.elamur.ai/free/en/papers/li-2016-diversity-promoting-objective-function-neural/), 2016b. URL https://arxiv.org/abs/1510.03055. [↗](https://arxiv.org/abs/1510.03055)
101. Ming Li and Paul M.B. Vitányi. An Introduction to Kolmogorov Complexity and Its Applications. Springer-Verlag, New York, 1997. [find](https://scholar.google.com/scholar?q=Ming+Li+and+Paul+M%2EB%2E+Vita%CC%81nyi%2E+An+Introduction+to+Kolmogorov+Complexity+and+Its+Applications%2E+Springer%2DVerlag%2C+New+York%2C+1997%2E)
102. Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. [From quantity to quality: Boosting llm performance with self-guided data selection for instruction tuning](https://md.research.elamur.ai/free/en/papers/li-2023-quantity-quality-boosting-llm-performance/), 2024c. URL https://arxiv.org/abs/2308.12032. [↗](https://arxiv.org/abs/2308.12032)
103. Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason Weston, and Mike Lewis. [Self-alignment with instruction backtranslation](https://md.research.elamur.ai/free/en/papers/li-2023-self-alignment-instruction-backtranslation/), 2024d. URL https://arxiv.org/abs/ 2308.06259. [↗](https://arxiv.org/abs/)
104. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. [Contrastive decoding: Open-ended text generation as optimization](https://md.research.elamur.ai/free/en/papers/li-2022-contrastive-decoding-open-ended-text/), 2023a. URL https://arxiv.org/abs/2210.15097. [↗](https://arxiv.org/abs/2210.15097)
105. Xiaomin Li, Mingye Gao, Zhiwei Zhang, Chang Yue, and Hong Hu. Rule-based data selection for large language models. arXiv preprint arXiv:2410.04715, 2024e. — [arXiv:2410.04715](https://arxiv.org/abs/2410.04715) [find](https://scholar.google.com/scholar?q=Xiaomin+Li%2C+Mingye+Gao%2C+Zhiwei+Zhang%2C+Chang+Yue%2C+and+Hong+Hu%2E+Rule%2Dbased+data+selection+for+large+language+models%2E+arXiv+preprint+arXiv%3A2410%2E04715%2C+2024e%2E+%E2%80%94+%5BarXiv%3A2410%2E04715%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F24)
106. Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. [Textbooks are all you need ii: phi-1.5 technical report](https://md.research.elamur.ai/free/en/papers/li-2023-textbooks-all-need-ii-phi/). arXiv preprint arXiv:2309.05463, 2023b.
107. Ziniu Li, Congliang Chen, Tian Xu, Zeyu Qin, Jiancong Xiao, Ruoyu Sun, and Zhi-Quan Luo. Entropic distribution matching in supervised fine-tuning of llms: Less overfitting and better diversity, 2024f. URL https://arxiv.org/abs/2408.16673. — [arXiv:2408.16673](https://arxiv.org/abs/2408.16673) [↗](https://arxiv.org/abs/2408.16673)
108. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step, 2023. [URL https://arxiv.org/abs/2305.20050](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). [↗](https://arxiv.org/abs/2305.20050)
109. Rensis Likert. A technique for the measurement of attitudes / by Rensis Likert. Archives of psychology ; no. 140. [s.n.], New York, 1932. [find](https://scholar.google.com/scholar?q=Rensis+Likert%2E+A+technique+for+the+measurement+of+attitudes+%2F+by+Rensis+Likert%2E+Archives+of+psychology+%3B+no%2E+140%2E+%5Bs%2En%2E%5D%2C+New+York%2C+1932%2E)
110. Chin-Yew Lin. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text Summarization Branches Out, pp. 74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013. [↗](https://aclanthology.org/W04-1013)
111. Bingbin Liu, Sebastien Bubeck, Ronen Eldan, Janardhan Kulkarni, Yuanzhi Li, Anh Nguyen, Rachel Ward, and Yi Zhang. Tinygsm: achieving ¿80 URL https://arxiv.org/abs/2312.09241. Hao Liu, Matei Zaharia, and Pieter Abbeel. Exploration with principles for diverse ai supervision, 2023b. URL https://arxiv.… — [TinyGSM: achieving >80% on GSM8k with small language models](https://md.research.elamur.ai/free/en/papers/liu-2023-tinygsm-achieving-80-gsm8k-small/) [↗](https://arxiv.org/abs/2312.09241)
112. Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, and Andrew M. Dai. [Best practices and lessons learned on synthetic data](https://md.research.elamur.ai/free/en/papers/liu-2024-best-practices-lessons-learned-synthetic/) for language models, 2024a. URL https://arxiv.org/abs/2404.07503. [↗](https://arxiv.org/abs/2404.07503)
113. Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. [What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-makes-good-data-alignment-comprehensive/), 2024b. URL https://arxiv.org/abs/2312.15685. [↗](https://arxiv.org/abs/2312.15685)
114. Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. [On llms-driven synthetic data generation, curation, and evaluation: A survey](https://md.research.elamur.ai/free/en/papers/long-2024-llms-driven-synthetic-data-generation/), 2024. URL [https://arxiv.org/](https://arxiv.org/) abs/2406.15126. [↗](https://arxiv.org/)
115. Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, and Daphne Ippolito. A pretrainer’s guide to training data: Measuring the effects of data age, domain coverage, quality, toxicity, 2023. [URL https://arxiv](https://md.research.elamur.ai/free/en/papers/longpre-2023-pretrainer-s-guide-training-data/). org/ab…
116. Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Qian Liu, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li,… [find](https://scholar.google.com/scholar?q=Anton+Lozhkov%2C+Raymond+Li%2C+Loubna+Ben+Allal%2C+Federico+Cassano%2C+Joel+Lamy%2DPoirier%2C+Nouamane+Tazi%2C+Ao+Tang%2C+Dmytro+Pykhtar%2C+Jiawei+Liu%2C+Yuxiang+Wei%2C+Tianyang+Liu%2C+Max+Tian%2C+Denis+Kocetkov%2C+Arthur+Zucker)
117. Cong Lu, Philip J. Ball, Yee Whye Teh, and Jack Parker-Holder. Synthetic experience replay, 2023a. URL https://arxiv.org/abs/2303.06614. — [arXiv:2303.06614](https://arxiv.org/abs/2303.06614) [↗](https://arxiv.org/abs/2303.06614)
118. Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Instag: Instruction tagging for analyzing supervised fine-tuning of large language models, 2023b. [URL https://arxiv.org/abs/2308.07074](https://md.research.elamur.ai/free/en/papers/lu-2023-instag-instruction-tagging-analyzing-supervised/). [↗](https://arxiv.org/abs/2308.07074)
119. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. [Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/), 2023a. URL https://arxiv.org/abs/2308.09583. [↗](https://arxiv.org/abs/2308.09583)
120. Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Wizardcoder: Empowering code large language models with evol-instruct, 2023b. URL https://arxiv.org/abs/2306.08568. [↗](https://arxiv.org/abs/2306.08568)
121. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-… [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+Shashank+Gupta%2C+Bodhisattwa+Prasad+Majumder%2C+Katherine+Her)
122. Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. [Generative reward models](https://md.research.elamur.ai/free/en/papers/mahan-2024-generative-reward-models/), 2024. URL https://arxiv.org/abs/2410.12832. [↗](https://arxiv.org/abs/2410.12832)
123. Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, and Navdeep Jaitly. [Rephrasing the web: A recipe for compute and data-efficient language modeling](https://md.research.elamur.ai/free/en/papers/maini-2024-rephrasing-web-recipe-compute-data/), 2024. URL [https://arxiv.org/](https://arxiv.org/) abs/2401.16380. [↗](https://arxiv.org/)
124. Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj S. Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations. In Proceedings of the Conference on Robot Learning, CoRL, 2023. URL [https://proceedings.…](https://proceedings.…) [↗](https://proceedings.…)
125. Philip M. McCarthy and Scott Jarvis. Mtld, vocd-d, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment, 2009. [find](https://scholar.google.com/scholar?q=Philip+M%2E+McCarthy+and+Scott+Jarvis%2E+Mtld%2C+vocd%2Dd%2C+and+hd%2Dd%3A+A+validation+study+of+sophisticated+approaches+to+lexical+diversity+assessment%2C+2009%2E)
126. Elliot Meyerson, Mark J Nelson, Herbie Bradley, Arash Moradi, Amy K Hoover, and Joel Lehman. Language model crossover: Variation through few-shot prompting. arXiv preprint arXiv:2302.12170, 2023. — [arXiv:2302.12170](https://arxiv.org/abs/2302.12170) [find](https://scholar.google.com/scholar?q=Elliot+Meyerson%2C+Mark+J+Nelson%2C+Herbie+Bradley%2C+Arash+Moradi%2C+Amy+K+Hoover%2C+and+Joel+Lehman%2E+Language+model+crossover%3A+Variation+through+few%2Dshot+prompting%2E+arXiv+preprint+arXiv%3A2302%2E12170%2C+2023%2E+%E2%80%94+%5Ba)
127. Ning Miao, Yee Whye Teh, and Tom Rainforth. [Selfcheck: Using llms to zero-shot check their own step-by-step reasoning](https://md.research.elamur.ai/free/en/papers/miao-2023-selfcheck-using-llms-zero-shot/), 2023. URL https://arxiv.org/abs/2308.00436. [↗](https://arxiv.org/abs/2308.00436)
128. John Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy on the line: On the strong correlation between out-of-distribution and in-distribution generalization, 2021. URL https://arxiv.org/abs/2107. 04649. [↗](https://arxiv.org/abs/2107)
129. Sören Mindermann, Jan Brauner, Muhammed Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt Höltgen, Aidan N. Gomez, Adrien Morisot, Sebastian Farquhar, and Yarin Gal. Prioritized training on points that are learnable, worth learning, and not yet learnt, 2022. URL https://arxiv.org/abs/22… — [arXiv:2206.07137](https://arxiv.org/abs/2206.07137) [↗](https://arxiv.org/abs/22…)
130. Melanie Mitchell. 0195124413. Complexity: A Guided Tour. [find](https://scholar.google.com/scholar?q=Melanie+Mitchell%2E+0195124413%2E+Complexity%3A+A+Guided+Tour%2E)
131. Oxford University Press, Inc., USA, 2009. [find](https://scholar.google.com/scholar?q=Oxford+University+Press%2C+Inc%2E%2C+USA%2C+2009%2E)
132. ISBN Jean-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites, 2015. URL https://arxiv.org/abs/1504.04909. — [arXiv:1504.04909](https://arxiv.org/abs/1504.04909) [↗](https://arxiv.org/abs/1504.04909)
133. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. [Orca: Progressive learning from complex explanation traces of gpt-4](https://md.research.elamur.ai/free/en/papers/mukherjee-2023-orca-progressive-learning-complex-explanation/), 2023. URL https://arxiv.org/abs/2306.02707. [↗](https://arxiv.org/abs/2306.02707)
134. Ranjita Naik, Varun Chandrasekaran, Mert Yuksekgonul, Hamid Palangi, and Besmira Nushi. Diversity of thought improves reasoning abilities of llms, 2024. URL https://arxiv.org/abs/2310.07088. — [arXiv:2310.07088](https://arxiv.org/abs/2310.07088) [↗](https://arxiv.org/abs/2310.07088)
135. Muhammad U Nasir, Sam Earle, Julian Togelius, Steven James, and Christopher Cleghorn. Llmatic: Neural architecture search via large language models and quality-diversity optimization. arXiv preprint arXiv:2306.01102, 2023. — [arXiv:2306.01102](https://arxiv.org/abs/2306.01102) [find](https://scholar.google.com/scholar?q=Muhammad+U+Nasir%2C+Sam+Earle%2C+Julian+Togelius%2C+Steven+James%2C+and+Christopher+Cleghorn%2E+Llmatic%3A+Neural+architecture+search+via+large+language+models+and+quality%2Ddiversity+optimization%2E+arXiv+preprint+a)
136. Ansong Ni, Miltiadis Allamanis, Arman Cohan, Yinlin Deng, Kensen Shi, Charles Sutton, and Pengcheng Yin. [Next: Teaching large language models to reason about code execution](https://md.research.elamur.ai/free/en/papers/ni-2024-next-teaching-large-language-models/), 2024. URL https://arxiv.org/abs/2404.14662. [↗](https://arxiv.org/abs/2404.14662)
137. Motoki Omura, Yasuhiro Fujita, and Toshiki Kataoka. Entropy controllable direct preference optimization, 2024. URL https://arxiv.org/abs/2411.07595. — [arXiv:2411.07595](https://arxiv.org/abs/2411.07595) [↗](https://arxiv.org/abs/2411.07595)
138. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Ja… — [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
139. Pedro Javier Ortiz Suárez, Benoı̂t Sagot, and Laurent Romary. Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures. In Piotr Bański, Adrien Barbaresi, Hanno Biber, Evelyn Breiteneder, Simon Clematide, Marc Kupietz, Harald Lüngen, and Caroline Iliadi (eds.),… — [doi:10.14618/ids-pub-9021](https://doi.org/10.14618/ids-pub-9021) [find](https://scholar.google.com/scholar?q=Pedro+Javier+Ortiz+Sua%CC%81rez%2C+Beno%C4%B1%CC%82t+Sagot%2C+and+Laurent+Romary%2E+Asynchronous+pipelines+for+processing+huge+corpora+on+medium+to+low+resource+infrastructures%2E+In+Piotr+Ban%CC%81ski%2C+Adrien+Barbaresi%2C+Hanno+B)
140. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training la… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
141. Laura O’Mahony, Leo Grinsztajn, Hailey Schoelkopf, and Stella Biderman. Attributing mode collapse in the fine-tuning of large language models. In ICLR 2024 Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) workshop, 2024. [find](https://scholar.google.com/scholar?q=Laura+O%E2%80%99Mahony%2C+Leo+Grinsztajn%2C+Hailey+Schoelkopf%2C+and+Stella+Biderman%2E+Attributing+mode+collapse+in+the+fine%2Dtuning+of+large+language+models%2E+In+ICLR+2024+Mathematical+and+Empirical+Understanding+of+)
142. Alizée Pace, Jonathan Mallinson, Eric Malmi, Sebastian Krause, and Aliaksei Severyn. West-of-n: Synthetic preference generation for improved reward modeling, 2024. URL https://arxiv.org/abs/ 2401.12086. [↗](https://arxiv.org/abs/)
143. Norman Packard, Mark A Bedau, Alastair Channon, Takashi Ikegami, Steen Rasmussen, Kenneth O Stanley, and Tim Taylor. An overview of open-ended evolution: Editorial introduction to the open-ended evolution ii special issue. Artificial life, 25(2):93–103, 2019. [find](https://scholar.google.com/scholar?q=Norman+Packard%2C+Mark+A+Bedau%2C+Alastair+Channon%2C+Takashi+Ikegami%2C+Steen+Rasmussen%2C+Kenneth+O+Stanley%2C+and+Tim+Taylor%2E+An+overview+of+open%2Dended+evolution%3A+Editorial+introduction+to+the+open%2Dended+evolu)
144. Rohan Pandey. gzip predicts data-dependent scaling laws, 2024. URL https://arxiv.org/abs/ 2405.16684. [↗](https://arxiv.org/abs/)
145. Richard Yuanzhe Pang, Weizhe Yuan, Kyunghyun Cho, He He, Sainbayar Sukhbaatar, and Jason Weston. [Iterative reasoning preference optimization](https://md.research.elamur.ai/free/en/papers/pang-2024-iterative-reasoning-preference-optimization/), 2024. URL https://arxiv.org/abs/2404.19733. [↗](https://arxiv.org/abs/2404.19733)
146. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Annual Meeting of the Association for Computational Linguistics, 2002a. URL [https://api.semanticscholar.org/CorpusID:11080756](https://api.semanticscholar.org/CorpusID:11080756). [↗](https://api.semanticscholar.org/CorpusID:11080756)
147. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311–318, 2002b.
148. Guilherme Penedo, Hynek Kydlı́ček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. [The fineweb datasets: Decanting the web for the finest text data at scale](https://md.research.elamur.ai/free/en/papers/penedo-2024-fineweb-datasets-decanting-web-finest/), 2024. URL https://arxiv.org/abs/2406.17557. [↗](https://arxiv.org/abs/2406.17557)
149. Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. [Mauve: Measuring the gap between neural text and human text using divergence frontiers](https://md.research.elamur.ai/free/en/papers/pillutla-2021-mauve-measuring-gap-between-neural/). Advances in Neural Information Processing Systems, 34:4816–4828, 2021.
150. Felipe Maia Polo, Lucas Weber, Leshem Choshen, Yuekai Sun, Gongjun Xu, and Mikhail Yurochkin. tinybenchmarks: evaluating llms with fewer examples. arXiv preprint arXiv:2402.14992, 2024. [find](https://scholar.google.com/scholar?q=Felipe+Maia+Polo%2C+Lucas+Weber%2C+Leshem+Choshen%2C+Yuekai+Sun%2C+Gongjun+Xu%2C+and+Mikhail+Yurochkin%2E+tinybenchmarks%3A+evaluating+llms+with+fewer+examples%2E+arXiv+preprint+arXiv%3A2402%2E14992%2C+2024%2E)
151. Julien Pourcel, Cédric Colas, Gaia Molinaro, Pierre-Yves Oudeyer, and Laetitia Teodorescu. Aces: Generating diverse programming puzzles with with autotelic generative models, 2024. URL https://arxiv.org/abs/2310.10692. — [arXiv:2310.10692](https://arxiv.org/abs/2310.10692) [↗](https://arxiv.org/abs/2310.10692)
152. Justin K Pugh, Lisa B Soros, and Kenneth O Stanley. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, 3:40, 2016. [find](https://scholar.google.com/scholar?q=Justin+K+Pugh%2C+Lisa+B+Soros%2C+and+Kenneth+O+Stanley%2E+Quality+diversity%3A+A+new+frontier+for+evolutionary+computation%2E+Frontiers+in+Robotics+and+AI%2C+3%3A40%2C+2016%2E)
153. Yulei Qin, Yuncheng Yang, Pengcheng Guo, Gang Li, Hang Shao, Yuchen Shi, Zihan Xu, Yun Gu, Ke Li, and Xing Sun. Unleashing the power of data tsunami: A comprehensive survey on data assessment and selection for instruction tuning of language models, 2024. URL https://arxiv.org/abs/2408. 02085. [↗](https://arxiv.org/abs/2408)
154. Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. Recursive introspection: Teaching LLM agents how to self-improve. In ICML 2024 Workshop on LLMs and Cognition, 2024. URL [https://openreview.net/forum?id=ze0nodITam](https://openreview.net/forum?id=ze0nodITam). [↗](https://openreview.net/forum?id=ze0nodITam)
155. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/), 2024. URL https://arxiv.org/abs/2305.18290. [↗](https://arxiv.org/abs/2305.18290)
156. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). J. Mach. Learn. Res., 21(1), jan 2020. ISSN 1532-4435. URL [https://jmlr.org/papers/](https://jmlr.org/papers/) volume2… [↗](https://jmlr.org/papers/)
157. Georg Rasch. Probabilistic models for some intelligence and attainment tests. ERIC, 1993. [find](https://scholar.google.com/scholar?q=Georg+Rasch%2E+Probabilistic+models+for+some+intelligence+and+attainment+tests%2E+ERIC%2C+1993%2E)
158. Nils Reimers and Iryna Gurevych. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/), 2019. URL https://arxiv.org/abs/1908.10084. [↗](https://arxiv.org/abs/1908.10084)
159. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof qa benchmark, 2023. [URL https://arxiv.org/abs/2311.12022](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). [↗](https://arxiv.org/abs/2311.12022)
160. Laria Reynolds and Kyle McDonell. [Prompt programming for large language models: Beyond the few-shot paradigm](https://md.research.elamur.ai/free/en/papers/reynolds-2021-prompt-programming-large-language-models/), 2021. URL https://arxiv.org/abs/2102.07350. [↗](https://arxiv.org/abs/2102.07350)
161. Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, 1 Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi. [Mathematical discoveries from program search with large language models](https://md.research.elamur.ai/free/en/papers/paredes-2023-mathematical-discoveries-program-search-large/). Na…
162. Yangjun Ruan, Chris J. Maddison, and Tatsunori Hashimoto. [Observational scaling laws and the predictability of language model performance](https://md.research.elamur.ai/free/en/papers/ruan-2024-observational-scaling-laws-predictability-language/), 2024. URL https://arxiv.org/abs/2405.10938. [↗](https://arxiv.org/abs/2405.10938)
163. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. [Imagenet large scale visual recognition challenge](https://md.research.elamur.ai/free/en/papers/russakovsky-2014-imagenet-large-scale-visual-recognition/), 2015. URL https://arxiv.org/abs/1409.0575. [↗](https://arxiv.org/abs/1409.0575)
164. Mikayel Samvelyan, Davide Paglieri, Minqi Jiang, Jack Parker-Holder, and Tim Rocktäschel. Multi-agent diagnostics for robustness via illuminated diversity. arXiv preprint arXiv:2401.13460, 2024a. — [arXiv:2401.13460](https://arxiv.org/abs/2401.13460) [find](https://scholar.google.com/scholar?q=Mikayel+Samvelyan%2C+Davide+Paglieri%2C+Minqi+Jiang%2C+Jack+Parker%2DHolder%2C+and+Tim+Rockta%CC%88schel%2E+Multi%2Dagent+diagnostics+for+robustness+via+illuminated+diversity%2E+arXiv+preprint+arXiv%3A2401%2E13460%2C+2024a%2E+%E2%80%94+%5B)
165. Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro, Aram H. Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Tim Rocktäschel, and Roberta Raileanu. Rainbow teaming: Open-ended generation of diverse adversarial prompts, 2024b. URL https://arxiv… — [arXiv:2402.16822](https://arxiv.org/abs/2402.16822) [find](https://scholar.google.com/scholar?q=Mikayel+Samvelyan%2C+Sharath+Chandra+Raparthy%2C+Andrei+Lupu%2C+Eric+Hambro%2C+Aram+H%2E+Markosyan%2C+Manish+Bhatt%2C+Yuning+Mao%2C+Minqi+Jiang%2C+Jack+Parker%2DHolder%2C+Jakob+Foerster%2C+Tim+Rockta%CC%88schel%2C+and+Roberta+Raile)
166. Tomohiro Sawada, Daniel Paleka, Alexander Havrilla, Pranav Tadepalli, Paula Vidas, Alexander Kranias, John J. Nay, Kshitij Gupta, and Aran Komatsuzaki. [Arb: Advanced reasoning benchmark for large language models](https://md.research.elamur.ai/free/en/papers/sawada-2023-arb-advanced-reasoning-benchmark-large/), 2023. URL https://arxiv.org/abs/2307.13692. [↗](https://arxiv.org/abs/2307.13692)
167. Timo Schick, Jane Dwivedi-Yu, Roberto Dessı̀, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. [Toolformer: Language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/), 2023. URL https://arxiv.org/abs/2302.04761. [↗](https://arxiv.org/abs/2302.04761)
168. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL https://arxiv.org/abs/1707.06347. [↗](https://arxiv.org/abs/1707.06347)
169. Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation models with monolingual data. In Katrin Erk and Noah A. Smith (eds.), Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 86–96, Berlin, Germany, Au… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+Improving+neural+machine+translation+models+with+monolingual+data%2E+In+Katrin+Erk+and+Noah+A%2E+Smith+%28eds%2E%29%2C+Proceedings+of+the+54th+Annual+Meeting+of+t)
170. Amrith Setlur, Saurabh Garg, Xinyang Geng, Naman Garg, Virginia Smith, and Aviral Kumar. [Rl on incorrect synthetic data scales the efficiency of llm math reasoning by eight-fold](https://md.research.elamur.ai/free/en/papers/setlur-2024-rl-incorrect-synthetic-data-scales/), 2024. URL https://arxiv.org/abs/2406.14532. [↗](https://arxiv.org/abs/2406.14532)
171. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/), 2024. URL https://arxiv.org/abs/2402.03300. [↗](https://arxiv.org/abs/2402.03300)
172. Utkarsh Sharma and Jared Kaplan. A neural scaling law from the dimension of the data manifold, 2020. URL https://arxiv.org/abs/2004.10802. [↗](https://arxiv.org/abs/2004.10802)
173. Vasu Sharma, Karthik Padthe, Newsha Ardalani, Kushal Tirumala, Russell Howes, Hu Xu, Po-Yao Huang, Shang-Wen Li, Armen Aghajanyan, Gargi Ghosh, and Luke Zettlemoyer. Text quality-based pruning for efficient training of language models, 2024. URL https://arxiv.org/abs/2405.01582. [↗](https://arxiv.org/abs/2405.01582)
174. Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/), 2023. URL [https://arxiv.org/](https://arxiv.org/) abs/2303.11366. [↗](https://arxiv.org/)
175. Olivier Sigaud, Gianluca Baldassarre, Cedric Colas, Stephane Doncieux, Richard Duro, Nicolas Perrin-Gilbert, and Vieri-Giuliano Santucci. A definition of open-ended learning problems for goal-conditioned agents. arXiv preprint arXiv:2311.00344, 2023. — [arXiv:2311.00344](https://arxiv.org/abs/2311.00344) [find](https://scholar.google.com/scholar?q=Olivier+Sigaud%2C+Gianluca+Baldassarre%2C+Cedric+Colas%2C+Stephane+Doncieux%2C+Richard+Duro%2C+Nicolas+Perrin%2DGilbert%2C+and+Vieri%2DGiuliano+Santucci%2E+A+definition+of+open%2Dended+learning+problems+for+goal%2Dconditio)
176. Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron Parisi, Abhishek Kumar, Alex Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Elsayed, Hanie Sedghi, Igor Mordatch, Isabelle Simp… — [Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models](https://md.research.elamur.ai/free/en/papers/singh-2023-human-data-scaling-self-training/)
177. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. [Scaling llm test-time compute optimally can be more effective than scaling model parameters](https://md.research.elamur.ai/free/en/papers/snell-2024-scaling-llm-test-time-compute/), 2024. URL https://arxiv.org/abs/2408. 03314. [↗](https://arxiv.org/abs/2408)
178. Asiiah Song. A little taxonomy of open-endedness. In ICLR Workshop on Agent Learning in Open-Endedness, 2022. [find](https://scholar.google.com/scholar?q=Asiiah+Song%2E+A+little+taxonomy+of+open%2Dendedness%2E+In+ICLR+Workshop+on+Agent+Learning+in+Open%2DEndedness%2C+2022%2E)
179. Lisa B Soros, Joel Lehman, and Kenneth O Stanley. Open-endedness: The last grand challenge you’ve never heard of, 2017. [find](https://scholar.google.com/scholar?q=Lisa+B+Soros%2C+Joel+Lehman%2C+and+Kenneth+O+Stanley%2E+Open%2Dendedness%3A+The+last+grand+challenge+you%E2%80%99ve+never+heard+of%2C+2017%2E)
180. Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human feedback, 2022. URL https://arxiv.org/abs/2009.01325. [↗](https://arxiv.org/abs/2009.01325)
181. Shyam Sudhakaran, Miguel González-Duque, Matthias Freiberger, Claire Glanois, Elias Najarro, and Sebastian Risi. Mariogpt: Open-ended text2level generation through large language models. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=Shyam+Sudhakaran%2C+Miguel+Gonza%CC%81lez%2DDuque%2C+Matthias+Freiberger%2C+Claire+Glanois%2C+Elias+Najarro%2C+and+Sebastian+Risi%2E+Mariogpt%3A+Open%2Dended+text2level+generation+through+large+language+models%2E+Advances+in+)
182. Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck, and Chuang Gan. [Easy-to-hard generalization: Scalable alignment beyond human supervision](https://md.research.elamur.ai/free/en/papers/sun-2024-easy-hard-generalization-scalable-alignment/), 2024. URL https://arxiv.org/abs/2403.09472. [↗](https://arxiv.org/abs/2403.09472)
183. Open Ended Learning Team, Adam Stooke, Anuj Mahajan, Catarina Barros, Charlie Deck, Jakob Bauer, Jakub Sygnowski, Maja Trebacz, Max Jaderberg, Michael Mathieu, Nat McAleese, Nathalie Bradley-Schmieg, Nathaniel Wong, Nicolas Porcel, Roberta Raileanu, Steph Hughes-Fitt, Valentin Dalibard, and Wojciech… — [arXiv:2107.12808](https://arxiv.org/abs/2107.12808) [find](https://scholar.google.com/scholar?q=Open+Ended+Learning+Team%2C+Adam+Stooke%2C+Anuj+Mahajan%2C+Catarina+Barros%2C+Charlie+Deck%2C+Jakob+Bauer%2C+Jakub+Sygnowski%2C+Maja+Trebacz%2C+Max+Jaderberg%2C+Michael+Mathieu%2C+Nat+McAleese%2C+Nathalie+Bradley%2DSchmieg%2C+)
184. David Thissen. Marginal maximum likelihood estimation for the one-parameter logistic model. Psychometrika, 47:175–186, 1982. [find](https://scholar.google.com/scholar?q=David+Thissen%2E+Marginal+maximum+likelihood+estimation+for+the+one%2Dparameter+logistic+model%2E+Psychometrika%2C+47%3A175%E2%80%93186%2C+1982%2E)
185. Ye Tian, Baolin Peng, Linfeng Song, Lifeng Jin, Dian Yu, Haitao Mi, and Dong Yu. [Toward self-improvement of llms via imagination, searching, and criticizing](https://md.research.elamur.ai/free/en/papers/tian-2024-toward-self-improvement-llms-imagination/), 2024. URL [https://arxiv.org/](https://arxiv.org/) abs/2404.12253. [↗](https://arxiv.org/)
186. Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari S. Morcos. [D4: Improving llm pretraining via document de-duplication and diversification](https://md.research.elamur.ai/free/en/papers/tirumala-2023-d4-improving-llm-pretraining-document/), 2023. URL https://arxiv.org/abs/2308.12284. [↗](https://arxiv.org/abs/2308.12284)
187. Bryon Tjanaka. Quantifying efficiency in quality diversity optimization. In Workshop on Benchmarks for Quality-Diversity Algorithms at GECCO, 2022. [find](https://scholar.google.com/scholar?q=Bryon+Tjanaka%2E+Quantifying+efficiency+in+quality+diversity+optimization%2E+In+Workshop+on+Benchmarks+for+Quality%2DDiversity+Algorithms+at+GECCO%2C+2022%2E)
188. Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Gitman. [Openmathinstruct-1: A 1.8 million math instruction tuning dataset](https://md.research.elamur.ai/free/en/papers/toshniwal-2024-openmathinstruct-1-1-8-million/), 2024. URL https://arxiv.org/abs/2402.10176. [↗](https://arxiv.org/abs/2402.10176)
189. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/), 2023a. URL h…
190. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
191. Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. [Solving math word problems with process- and outcome-based feedback](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/), 2022. URL https://arxiv.org/abs/2211.14275. [↗](https://arxiv.org/abs/2211.14275)
192. Vijay Viswanathan, Chenyang Zhao, Amanda Bertsch, Tongshuang Wu, and Graham Neubig. Prompt2model: Generating deployable models from natural language instructions, 2023. URL https://arxiv.org/abs/2308.12261. [↗](https://arxiv.org/abs/2308.12261)
193. Ben Wang and Aran Komatsuzaki. Gpt-j-6b: A 6 billion parameter autoregressive language model, 2021. [find](https://scholar.google.com/scholar?q=Ben+Wang+and+Aran+Komatsuzaki%2E+Gpt%2Dj%2D6b%3A+A+6+billion+parameter+autoregressive+language+model%2C+2021%2E)
194. Evan Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, Will Song, Vaskar Nath, Ziwen Han, Sean Hendryx, Summer Yue, and Hugh Zhang. Planning in natural language improves llm search for code generation, 2024a. URL https://arxiv.org/abs/2409.03733. [↗](https://arxiv.org/abs/2409.03733)
195. Fei Wang, Ninareh Mehrabi, Palash Goyal, Rahul Gupta, Kai-Wei Chang, and Aram Galstyan. Data advisor: Dynamic data curation for safety alignment of large language models, 2024b. URL https://arxiv.org/abs/2410.05269. [↗](https://arxiv.org/abs/2410.05269)
196. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models, 2023a. URL https://arxiv.org/abs/2305.16291. [↗](https://arxiv.org/abs/2305.16291)
197. Jiahao Wang, Bolin Zhang, Qianlong Du, Jiajun Zhang, and Dianhui Chu. A survey on data selection for llm instruction tuning, 2024c. URL https://arxiv.org/abs/2402.05123. — [arXiv:2402.05123](https://arxiv.org/abs/2402.05123) [↗](https://arxiv.org/abs/2402.05123)
198. Peiyi Wang, Lei Li, Zhihong Shao, R. X. Xu, Damai Dai, Yifei Li, Deli Chen, Y. Wu, and Zhifang Sui. [Math-shepherd: Verify and reinforce llms step-by-step without human annotations](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/), 2024d. URL https://arxiv.org/abs/2312.08935. [↗](https://arxiv.org/abs/2312.08935)
199. Tianlu Wang, Ping Yu, Xiaoqing Ellen Tan, Sean O’Brien, Ramakanth Pasunuru, Jane Dwivedi-Yu, Olga Golovneva, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. [Shepherd: A critic for language model generation](https://md.research.elamur.ai/free/en/papers/wang-2023-shepherd-critic-language-model-generation/), 2023b. URL https://arxiv.org/abs/2308.04592. [↗](https://arxiv.org/abs/2308.04592)
200. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Ma… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Swaroop+Mishra%2C+Pegah+Alipoormolabashi%2C+Yeganeh+Kordi%2C+Amirreza+Mirzaei%2C+Anjana+Arunkumar%2C+Arjun+Ashok%2C+Arut+Selvan+Dhanasekaran%2C+Atharva+Naik%2C+David+Stap%2C+Eshaan+Pathak%2C+Giannis+Karaman)
201. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions, 2023c. URL https://arxiv.org/abs/2212.10560. [↗](https://arxiv.org/abs/2212.10560)
202. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/), 2022. URL https://arxiv.org/abs/2109.01652. [↗](https://arxiv.org/abs/2109.01652)
203. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/), 2023. URL https://arxiv.org/abs/2201.11903. [↗](https://arxiv.org/abs/2201.11903)
204. Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V. Le. [Simple synthetic data reduces sycophancy in large language models](https://md.research.elamur.ai/free/en/papers/wei-2023-simple-synthetic-data-reduces-sycophancy/), 2024. URL https://arxiv.org/abs/2308.03958. [↗](https://arxiv.org/abs/2308.03958)
205. Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. [Neural text generation with unlikelihood training](https://md.research.elamur.ai/free/en/papers/welleck-2019-neural-text-generation-unlikelihood-training/). In International Conference on Learning Representations, 2020. URL [https://openreview.net/forum?id=SJeYe0NtvH](https://openreview.net/forum?id=SJeYe0NtvH). [↗](https://openreview.net/forum?id=SJeYe0NtvH)
206. Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. [Ccnet: Extracting high quality monolingual datasets from web crawl data](https://md.research.elamur.ai/free/en/papers/wenzek-2020-ccnet-extracting-high-quality-monolingual/). arXiv preprint arXiv:1911.00359, 2019.
207. Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. [CCNet: Extracting high quality monolingual datasets from web crawl data](https://md.research.elamur.ai/free/en/papers/wenzek-2020-ccnet-extracting-high-quality-monolingual/). In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thier…
208. Alexander Wettig, Aatmik Gupta, Saumya Malik, and Danqi Chen. [Qurating: Selecting high-quality data for training language models](https://md.research.elamur.ai/free/en/papers/wettig-2024-qurating-selecting-high-quality-data/), 2024. URL https://arxiv.org/abs/2402.09739. [↗](https://arxiv.org/abs/2402.09739)
209. Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3):229–256, may 1992. ISSN 1573-0565. doi: 10.1007/BF00992696. URL https://doi.org/10.1007/BF00992696. [↗](https://doi.org/10.1007/BF00992696)
210. Xingyu Wu, Sheng hao Wu, Jibin Wu, Liang Feng, and Kay Chen Tan. Evolutionary computation in the era of large language model: Survey and roadmap, 2024a. URL https://arxiv.org/abs/2401. 10034. [↗](https://arxiv.org/abs/2401)
211. Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. [Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models](https://md.research.elamur.ai/free/en/papers/wu-2024-inference-scaling-laws-empirical-analysis/), 2024b. URL https://arxiv.org/abs/2408.00724. [↗](https://arxiv.org/abs/2408.00724)
212. Sang Michael Xie, Shibani Santurkar, Tengyu Ma, and Percy Liang. Data selection for language models via importance resampling, 2023. URL https://arxiv.org/abs/2302.03169. [↗](https://arxiv.org/abs/2302.03169)
213. Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy S Liang, Quoc V Le, Tengyu Ma, and Adams Wei Yu. Doremi: Optimizing data mixtures speeds up language model pretraining. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=Sang+Michael+Xie%2C+Hieu+Pham%2C+Xuanyi+Dong%2C+Nan+Du%2C+Hanxiao+Liu%2C+Yifeng+Lu%2C+Percy+S+Liang%2C+Quoc+V+Le%2C+Tengyu+Ma%2C+and+Adams+Wei+Yu%2E+Doremi%3A+Optimizing+data+mixtures+speeds+up+language+model+pretraining%2E+)
214. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions, 2023. [URL https://arxiv.org/abs/2304.12244](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). [↗](https://arxiv.org/abs/2304.12244)
215. Joy Qiping Yang, Salman Salamatian, Ziteng Sun, Ananda Theertha Suresh, and Ahmad Beirami. Asymptotics of language model alignment, 2024a. URL https://arxiv.org/abs/2404.01730. — [arXiv:2404.01730](https://arxiv.org/abs/2404.01730) [↗](https://arxiv.org/abs/2404.01730)
216. Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, and Yuandong Tian. Rlcd: Reinforcement learning from contrastive distillation for language model alignment, 2024b. URL https://arxiv.org/abs/ 2307.12950. [↗](https://arxiv.org/abs/)
217. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models, 2023. URL https://arxiv.org/abs/2305.10601. [↗](https://arxiv.org/abs/2305.10601)
218. Jiacheng Ye, Jiahui Gao, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Lingpeng Kong. Progen: Progressive zero-shot dataset generation via in-context feedback, 2022a. URL https://arxiv.org/abs/ 2210.12329. [↗](https://arxiv.org/abs/)
219. Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Lingpeng Kong. [Zerogen: Efficient zero-shot learning via dataset generation](https://md.research.elamur.ai/free/en/papers/ye-2022-zerogen-efficient-zero-shot-learning/), 2022b. URL [https://arxiv.org/](https://arxiv.org/) abs/2202.07922. [↗](https://arxiv.org/)
220. Jiasheng Ye, Peiju Liu, Tianxiang Sun, Yunhua Zhou, Jun Zhan, and Xipeng Qiu. [Data mixing laws: Optimizing data mixtures by predicting language modeling performance](https://md.research.elamur.ai/free/en/papers/ye-2024-data-mixing-laws-optimizing-data/), 2024a. URL https://arxiv.org/abs/2403.16952. [↗](https://arxiv.org/abs/2403.16952)
221. Zihuiwen Ye, Fraser Greenlee-Scott, Max Bartolo, Phil Blunsom, Jon Ander Campos, and Matthias Gallé. Improving reward models with synthetic critiques, 2024b. URL https://arxiv.org/abs/2405. 20850. [↗](https://arxiv.org/abs/2405)
222. Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. [Metamath: Bootstrap your own mathematical questions for large language models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/), 2024. URL https://arxiv.org/abs/2309.12284. [↗](https://arxiv.org/abs/2309.12284)
223. Tianhe Yu, Ted Xiao, Austin Stone, Jonathan Tompson, Anthony Brohan, Su Wang, Jaspiar Singh, Clayton Tan, Jodilyn Peralta, Brian Ichter, et al. Scaling robot learning with semantically imagined experience. arXiv preprint arXiv:2302.11550, 2023a. — [arXiv:2302.11550](https://arxiv.org/abs/2302.11550) [find](https://scholar.google.com/scholar?q=Tianhe+Yu%2C+Ted+Xiao%2C+Austin+Stone%2C+Jonathan+Tompson%2C+Anthony+Brohan%2C+Su+Wang%2C+Jaspiar+Singh%2C+Clayton+Tan%2C+Jodilyn+Peralta%2C+Brian+Ichter%2C+et+al%2E+Scaling+robot+learning+with+semantically+imagined+experi)
224. Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. [Large language model as attributed training data generator: A tale of diversity and bias](https://md.research.elamur.ai/free/en/papers/yu-2023-large-language-model-attributed-training/), 2023b. URL https://arxiv.org/abs/2306.15895. [↗](https://arxiv.org/abs/2306.15895)
225. Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. [Scaling relationship on learning mathematical reasoning with large language models](https://md.research.elamur.ai/free/en/papers/yuan-2023-scaling-relationship-learning-mathematical-reasoning/), 2023. URL https://arxiv.org/abs/2308.01825. [↗](https://arxiv.org/abs/2308.01825)
226. Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. [Mammoth: Building math generalist models through hybrid instruction tuning](https://md.research.elamur.ai/free/en/papers/yue-2023-mammoth-building-math-generalist-models/), 2023. URL https://arxiv.org/abs/2309.05653. [↗](https://arxiv.org/abs/2309.05653)
227. Xiang Yue, Tuney Zheng, Ge Zhang, and Wenhu Chen. [Mammoth2: Scaling instructions from the web](https://md.research.elamur.ai/free/en/papers/yue-2024-mammoth2-scaling-instructions-web/), 2024. URL https://arxiv.org/abs/2405.03548. [↗](https://arxiv.org/abs/2405.03548)
228. Weihao Zeng, Can Xu, Yingxiu Zhao, Jian-Guang Lou, and Weizhu Chen. [Automatic instruction evolving for large language models](https://md.research.elamur.ai/free/en/papers/zeng-2024-automatic-instruction-evolving-large-language/), 2024. URL https://arxiv.org/abs/2406.00770. [↗](https://arxiv.org/abs/2406.00770)
229. Dylan Zhang, Justin Wang, and Francois Charton. Instruction diversity drives generalization to unseen tasks, 2024a. URL https://arxiv.org/abs/2402.10891. — [arXiv:2402.10891](https://arxiv.org/abs/2402.10891) [↗](https://arxiv.org/abs/2402.10891)
230. Dylan Zhang, Justin Wang, and Francois Charton. Only-IF:revealing the decisive effect of instruction diversity on generalization, 2024b. URL https://arxiv.org/abs/2410.04717. — [arXiv:2410.04717](https://arxiv.org/abs/2410.04717) [↗](https://arxiv.org/abs/2410.04717)
231. Jenny Zhang, Joel Lehman, Kenneth Stanley, and Jeff Clune. Omni: Open-endedness via models of human notions of interestingness. arXiv preprint arXiv:2306.01711, 2023. [find](https://scholar.google.com/scholar?q=Jenny+Zhang%2C+Joel+Lehman%2C+Kenneth+Stanley%2C+and+Jeff+Clune%2E+Omni%3A+Open%2Dendedness+via+models+of+human+notions+of+interestingness%2E+arXiv+preprint+arXiv%3A2306%2E01711%2C+2023%2E)
232. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. [Generative verifiers: Reward modeling as next-token prediction](https://md.research.elamur.ai/free/en/papers/zhang-2024-generative-verifiers-reward-modeling-as/), 2024c. URL https://arxiv.org/abs/ 2408.15240. [↗](https://arxiv.org/abs/)
233. Yifan Zhang, Yifan Luo, Yang Yuan, and Andrew C Yao. Autonomous data selection with language models for mathematical texts. In ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models, 2024d. URL [https://openreview.net/forum?id=bBF077z8LF](https://openreview.net/forum?id=bBF077z8LF). [↗](https://openreview.net/forum?id=bBF077z8LF)
234. Yiming Zhang, Avi Schwarzschild, Nicholas Carlini, Zico Kolter, and Daphne Ippolito. Forcing diffuse distributions out of language models, 2024e. URL https://arxiv.org/abs/2404.10859. — [arXiv:2404.10859](https://arxiv.org/abs/2404.10859) [↗](https://arxiv.org/abs/2404.10859)
235. Yulun Zhang, Matthew Fontaine, Varun Bhatt, Stefanos Nikolaidis, and Jiaoyang Li. Arbitrarily scalable environment generators via neural cellular automata. Advances in Neural Information Processing Systems, 36, 2024f. [find](https://scholar.google.com/scholar?q=Yulun+Zhang%2C+Matthew+Fontaine%2C+Varun+Bhatt%2C+Stefanos+Nikolaidis%2C+and+Jiaoyang+Li%2E+Arbitrarily+scalable+environment+generators+via+neural+cellular+automata%2E+Advances+in+Neural+Information+Processing+Sy)
236. Dora Zhao, Jerone T. A. Andrews, Orestis Papakyriakopoulos, and Alice Xiang. Position: Measure dataset diversity, don’t just claim it, 2024a. URL https://arxiv.org/abs/2407.08188. — [arXiv:2407.08188](https://arxiv.org/abs/2407.08188) [↗](https://arxiv.org/abs/2407.08188)
237. Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, and Steffen Eger. [MoverScore: Text generation evaluating with contextualized embeddings and earth mover distance](https://md.research.elamur.ai/free/en/papers/zhao-2019-moverscore-text-generation-evaluating-contextualized/). In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods…
238. Xinyu Zhao, Fangcong Yin, and Greg Durrett. Understanding synthetic context extension via retrieval heads, 2024b. URL https://arxiv.org/abs/2410.22316. — [arXiv:2410.22316](https://arxiv.org/abs/2410.22316) [↗](https://arxiv.org/abs/2410.22316)
239. Yingxiu Zhao, Bowen Yu, Binyuan Hui, Haiyang Yu, Fei Huang, Yongbin Li, and Nevin L. Zhang. A preliminary study of the intrinsic relationship between complexity and alignment, 2024c. URL https://arxiv.org/abs/2308.05696. — [arXiv:2308.05696](https://arxiv.org/abs/2308.05696) [↗](https://arxiv.org/abs/2308.05696)
240. Kunhao Zheng, Jesse Michael Han, and Stanislas Polu. [Minif2f: a cross-system benchmark for formal olympiad-level mathematics](https://md.research.elamur.ai/free/en/papers/zheng-2021-minif2f-cross-system-benchmark-formal/), 2022. URL https://arxiv.org/abs/2109.00110. [↗](https://arxiv.org/abs/2109.00110)
241. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-asa-judge with mt-bench and chatbot arena, 2023. [URL https://arxiv.org/abs/2306.05685](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). [↗](https://arxiv.org/abs/2306.05685)
242. Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. [Towards a unified multi-dimensional evaluator for text generation](https://md.research.elamur.ai/free/en/papers/zhong-2022-unified-multi-dimensional-evaluator-text/). arXiv preprint arXiv:2210.07197, 2022.
243. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. Lima: Less is more for alignment, 2023a. URL https://arxiv.org/abs/2305.11206. [↗](https://arxiv.org/abs/2305.11206)
244. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. [Large language models are human-level prompt engineers](https://md.research.elamur.ai/free/en/papers/zhou-2022-large-language-models-human-level/), 2023b. URL [https://arxiv.org/](https://arxiv.org/) abs/2211.01910. [↗](https://arxiv.org/)
245. Yuxuan Zhou, Margret Keuper, and Mario Fritz. [Balancing diversity and risk in llm sampling: How to select your method and parameter for open-ended text generation](https://md.research.elamur.ai/free/en/papers/zhou-2025-balancing-diversity-risk-llm-sampling/), 2024. URL https://arxiv.org/abs/ 2408.13586. [↗](https://arxiv.org/abs/)
246. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. [Texygen: A benchmarking platform for text generation models](https://md.research.elamur.ai/free/en/papers/zhu-2018-texygen-benchmarking-platform-text-generation/), 2018. URL https://arxiv.org/abs/1802. 01886. [↗](https://arxiv.org/abs/1802)
247. Yan Zhuang, Qi Liu, Yuting Ning, Weizhe Huang, Rui Lv, Zhenya Huang, Guanhao Zhao, Zheng Zhang, Qingyang Mao, Shijin Wang, et al. Efficiently measuring the cognitive ability of llms: An adaptive testing perspective. arXiv preprint arXiv:2306.10512, 2023. [find](https://scholar.google.com/scholar?q=Yan+Zhuang%2C+Qi+Liu%2C+Yuting+Ning%2C+Weizhe+Huang%2C+Rui+Lv%2C+Zhenya+Huang%2C+Guanhao+Zhao%2C+Zheng+Zhang%2C+Qingyang+Mao%2C+Shijin+Wang%2C+et+al%2E+Efficiently+measuring+the+cognitive+ability+of+llms%3A+An+adaptive+testi)
248. Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences, 2020. URL https://arxiv.org/abs/1909.08593. [↗](https://arxiv.org/abs/1909.08593)

*Updated 3 November 2026*