---
title: "Data-driven Discovery with Large Generative Models (2024)"
description: "This position paper, written by authors from the Allen Institute for AI and collaborators, urges the ML… 111 references, cited by 15 papers here."
authors: ["Bodhisattwa Prasad Majumder", "Harshit Surana", "Dhruv Agarwal", "Sanchaita Hazra", "Ashish Sabharwal", "Peter Clark"]
year: 2024
venue: "arXiv:2402.13610"
source: "https://arxiv.org/abs/2402.13610"
doi: "10.48550/arxiv.2402.13610"
arxiv: "2402.13610"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/majumder-2024-data-driven-discovery-large-generative/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/majumder-2024-data-driven-discovery-large-generative/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/) / Data-driven Discovery with Large Generative Models

# Data-driven Discovery with Large Generative Models

**Authors:** [Bodhisattwa Prasad Majumder](https://md.research.elamur.ai/free/en/authors/bodhisattwa-prasad-majumder/), Harshit Surana, Dhruv Agarwal, Sanchaita Hazra, [Ashish Sabharwal](https://md.research.elamur.ai/free/en/authors/ashish-sabharwal/) and [Peter Clark](https://md.research.elamur.ai/free/en/authors/peter-clark/) · **Year:** 2024 · **Venue:** arXiv:2402.13610 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2402.13610) · [PDF](https://arxiv.org/pdf/2402.13610) · [DOI 10.48550/arxiv.2402.13610](https://doi.org/10.48550/arxiv.2402.13610) · [arXiv 2402.13610](https://arxiv.org/abs/2402.13610) · [OpenAlex](https://openalex.org/W4392089965)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [15](https://md.research.elamur.ai/free/en/papers/majumder-2024-data-driven-discovery-large-generative/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This position paper, written by authors from the Allen Institute for AI and collaborators, urges the ML community to build systems that perform scientific discovery end to end using only datasets already at hand, with no new data collection or lab experiments. The authors treat discovery as a heuristic search for laws that explain how observed data came about. They divide the task into two parts: proposing hypotheses and verifying them. They also list traits an ideal system needs, including goal-driven and open-ended search, multi-step planning, statistical testing, knowledge integration, and learning from user feedback. To show that this is feasible, they present DataVoyager, a GPT-4-based multi-agent prototype. Given a dataset and a broad question, it repeatedly forms hypotheses, writes and runs analysis code, and interprets the outcomes. Their demonstrations suggest large generative models (LGMs) already provide several of the desired capabilities, but are not reliable enough on their own. The authors therefore recommend pairing LGMs with robust tools and active human moderation. This matters for data-heavy fields such as social science, medicine, climate science, and astronomy.

## Key points
- **DataVoyager** runs on GPT-4. Agents take roles such as planner, programmer, and data expert, and they call analysis functions such as `run_correlation()`, `run_ols()`, `add_interactions()`, and `run_glm()`.
- **Case study:** National Longitudinal Surveys (US BLS), examining how BMI relates to the time-preference variables DISSAVED and SAMESAVE.
  - The initial correlation was weak (−0.031).
  - An OLS interaction term of about 0.526 was significant.
  - Follow-up GLMs added SES, sex, race, and test scores.
- **Blueprint with three usage modes:**
  - an explicit hypothesis;
  - a vague high-level question;
  - ongoing feedback (e.g., "use the Gini coefficient for wealth disparities"), which a continual learner stores for future queries.
- **Critique of prior approaches:**
  - Early discovery programs (Langley) lacked computing power.
  - AlphaFold is domain-specific.
  - CoScientist depends on human wet-lab work.
  - AutoML and BI tools tune models without understanding what the data means.
- **Risks noted:** false discoveries, policy misuse, and legal consequences. The authors argue that LGMs alone cannot yet deliver accurate, robust discovery.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Massachusetts Amherst](https://md.research.elamur.ai/free/en/universities/university-of-massachusetts-amherst/) — University, Amherst Center, United States · 106 works · 66,692 citations
- [University of Utah](https://md.research.elamur.ai/free/en/universities/university-of-utah/) — University, Salt Lake City, United States · 38 works · 5,925 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🧱 Materials Science](https://md.research.elamur.ai/free/en/shelves/materials-science/) · 26 works · [🧱 Materials Chemistry](https://md.research.elamur.ai/free/en/shelves/materials-science/materials-chemistry/) · 26 works

## Cite

Cite as: Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Sanchaita Hazra, Ashish Sabharwal and Peter Clark. 2024. Data-driven Discovery with Large Generative Models. arXiv:2402.13610.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{majumder2024data,
  title = {{Data-driven Discovery with Large Generative Models}},
  author = {Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Sanchaita Hazra, Ashish Sabharwal and Peter Clark},
  year = {2024},
  doi = {10.48550/arxiv.2402.13610},
  eprint = {2402.13610},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2402.13610}
}
```

- [← previous DARES: Dataset for Arabic Readability Estimation of School Materials](https://md.research.elamur.ai/free/en/papers/haj-2024-dares-dataset-arabic-readability-estimation/)
- [next → Evaluating Large Language Models for Health-related Queries with Presuppositions](https://md.research.elamur.ai/free/en/papers/kaur-2024-evaluating-large-language-models-health/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/)

## References

111 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Achiam, O. J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., Bello, I., Berdine, J., Bernadett-Shapiro, G., Berner, C., Bogdonof… [find](https://scholar.google.com/scholar?q=Achiam%2C+O%2E+J%2E%2C+Adler%2C+S%2E%2C+Agarwal%2C+S%2E%2C+Ahmad%2C+L%2E%2C+Akkaya%2C+I%2E%2C+Aleman%2C+F%2E+L%2E%2C+Almeida%2C+D%2E%2C+Altenschmidt%2C+J%2E%2C+Altman%2C+S%2E%2C+Anadkat%2C+S%2E%2C+Avila%2C+R%2E%2C+Babuschkin%2C+I%2E%2C+Balaji%2C+S%2E%2C+Balcom%2C+V%2E%2C+Baltescu%2C+P%2E%2C+Ba)
2. Agrawal, A., McHale, J., and Oettl, A. Artificial intelligence and scientific discovery: A model of prioritized search. SSRN Electronic Journal, 2023. URL [https://api.semanticscholar.org/CorpusID:260906716](https://api.semanticscholar.org/CorpusID:260906716). [↗](https://api.semanticscholar.org/CorpusID:260906716)
3. Alexander, K. L., Riordan, C., Fennessey, J., and Pallas, A. M. Social background, academic resources, and college graduation: Recent evidence from the national longitudinal survey. American Journal of Education, 90(4): 315–333, 1982. [find](https://scholar.google.com/scholar?q=Alexander%2C+K%2E+L%2E%2C+Riordan%2C+C%2E%2C+Fennessey%2C+J%2E%2C+and+Pallas%2C+A%2E+M%2E+Social+background%2C+academic+resources%2C+and+college+graduation%3A+Recent+evidence+from+the+national+longitudinal+survey%2E+American+Journal+o)
4. Cai, T., Wang, X., Ma, T., Chen, X., and Zhou, D. Large language models as tool makers. ArXiv, abs/2305.17126, 2023. URL [https://api.semanticscholar.org/CorpusID:258947222](https://api.semanticscholar.org/CorpusID:258947222). [↗](https://api.semanticscholar.org/CorpusID:258947222)
5. Caliskan, A., Bryson, J. J., and Narayanan, A. Semantics derived automatically from language corpora contain human-like biases. Science, 356:183 – 186, 2016. URL [https://api.semanticscholar.org/CorpusID:23163324](https://api.semanticscholar.org/CorpusID:23163324). [↗](https://api.semanticscholar.org/CorpusID:23163324)
6. Callison-Burch, C. Understanding generative artificial intelligence and its relationship to copyright. Testimony before The U.S. House of Representatives Judiciary Committee, Subcommittee on Courts, Intellectual Property, and the Internet, May 2023. Hearing on Artificial Intelligence and Intellectua… [find](https://scholar.google.com/scholar?q=Callison%2DBurch%2C+C%2E+Understanding+generative+artificial+intelligence+and+its+relationship+to+copyright%2E+Testimony+before+The+U%2ES%2E+House+of+Representatives+Judiciary+Committee%2C+Subcommittee+on+Courts%2C+I)
7. Anderson, C. The end of theory: The data deluge makes the scientific method obsolete. Wired magazine, 16(7): 16–07, 2008. [find](https://scholar.google.com/scholar?q=Anderson%2C+C%2E+The+end+of+theory%3A+The+data+deluge+makes+the+scientific+method+obsolete%2E+Wired+magazine%2C+16%287%29%3A+16%E2%80%9307%2C+2008%2E)
8. Camerer, C., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., Altmejd, A., Buttrick, N., Chan, T., Chen, Y., Forsell, E., Gampa, A., Heikensten, E., Hummer, L., Imai, T., Isaksson, S., Manfredi, D., Rose, J., Wagenmakers, E., and… [find](https://scholar.google.com/scholar?q=Camerer%2C+C%2E%2C+Dreber%2C+A%2E%2C+Holzmeister%2C+F%2E%2C+Ho%2C+T%2E%2DH%2E%2C+Huber%2C+J%2E%2C+Johannesson%2C+M%2E%2C+Kirchler%2C+M%2E%2C+Nave%2C+G%2E%2C+Nosek%2C+B%2E+A%2E%2C+Pfeiffer%2C+T%2E%2C+Altmejd%2C+A%2E%2C+Buttrick%2C+N%2E%2C+Chan%2C+T%2E%2C+Chen%2C+Y%2E%2C+Forsell%2C+E%2E%2C+Gampa%2C+)
9. Bailis, P., Gan, E., Madden, S., Narayanan, D., Rong, K., and Suri, S. Macrobase: Prioritizing attention in fast data. In Proceedings of the 2017 ACM International Conference on Management of Data, pp. 541–556, 2017. [find](https://scholar.google.com/scholar?q=Bailis%2C+P%2E%2C+Gan%2C+E%2E%2C+Madden%2C+S%2E%2C+Narayanan%2C+D%2E%2C+Rong%2C+K%2E%2C+and+Suri%2C+S%2E+Macrobase%3A+Prioritizing+attention+in+fast+data%2E+In+Proceedings+of+the+2017+ACM+International+Conference+on+Management+of+Data%2C+pp)
10. Bekhuis, T. Conceptual biology, hypothesis discovery, and text mining: Swanson’s legacy. Biomedical digital libraries, 3:1–7, 2006. [find](https://scholar.google.com/scholar?q=Bekhuis%2C+T%2E+Conceptual+biology%2C+hypothesis+discovery%2C+and+text+mining%3A+Swanson%E2%80%99s+legacy%2E+Biomedical+digital+libraries%2C+3%3A1%E2%80%937%2C+2006%2E)
11. Bengio, Y., Louradour, J., Collobert, R., and Weston, J. [Curriculum learning](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/). In Proceedings of the 26th annual international conference on machine learning, pp. 41–48, 2009.
12. Cao, H., Dodge, J., Lo, K., McFarland, D. A., and Wang, L. L. The rise of open science: Tracking the evolution and perceived value of data and methods link-sharing practices. ArXiv, abs/2310.03193, 2023. URL [https://api.semanticscholar.org/CorpusID:263671521](https://api.semanticscholar.org/CorpusID:263671521). — [arXiv:2310.03193](https://arxiv.org/abs/2310.03193) [↗](https://api.semanticscholar.org/CorpusID:263671521)
13. Bianchini, S., Müller, M., and Pelletier, P. Artificial intelligence in science: An emerging general method of invention. Research Policy, 51(10):104604, 2022. [find](https://scholar.google.com/scholar?q=Bianchini%2C+S%2E%2C+M%C3%BCller%2C+M%2E%2C+and+Pelletier%2C+P%2E+Artificial+intelligence+in+science%3A+An+emerging+general+method+of+invention%2E+Research+Policy%2C+51%2810%29%3A104604%2C+2022%2E)
14. Boiko, D. A., MacKnight, R., Kline, B., and Gomes, G. [Autonomous chemical research with large language models](https://md.research.elamur.ai/free/en/papers/boiko-2023-autonomous-chemical-research-large-language/). Nature, 624:570 – 578, 2023. URL [https://api.semanticscholar.org/CorpusID:266432059](https://api.semanticscholar.org/CorpusID:266432059). [↗](https://api.semanticscholar.org/CorpusID:266432059)
15. Bowers, S. and Ludäscher, B. An ontology-driven framework for data transformation in scientific workflows. In International Workshop on Data Integration in the Life Sciences, pp. 1–16. Springer, 2004. [find](https://scholar.google.com/scholar?q=Bowers%2C+S%2E+and+Lud%C3%A4scher%2C+B%2E+An+ontology%2Ddriven+framework+for+data+transformation+in+scientific+workflows%2E+In+International+Workshop+on+Data+Integration+in+the+Life+Sciences%2C+pp%2E+1%E2%80%9316%2E+Springer%2C+2004%2E)
16. Brickley, D., Burgess, M., and Noy, N. Google dataset search: Building a search engine for datasets in an open web ecosystem. The World Wide Web Conference, 2019. URL [https://api.semanticscholar.org/CorpusID:86688027](https://api.semanticscholar.org/CorpusID:86688027). [↗](https://api.semanticscholar.org/CorpusID:86688027)
17. Burda, Y., Edwards, H., Storkey, A. J., and Klimov, O. Exploration by random network distillation. ArXiv, abs/1810.12894, 2018. URL [https://api.semanticscholar.org/CorpusID:53115163](https://api.semanticscholar.org/CorpusID:53115163). [↗](https://api.semanticscholar.org/CorpusID:53115163)
18. Chakraborty, A., Banerjee, A., Dasgupta, S., Raturi, V., Soni, A., Gupta, A., Harsola, S., and Subrahmaniam, V. T. Navigator: A gen-ai system for discovery of factual and predictive insights on domain-specific tabular datasets. Proceedings of the 7th Joint International Conference on Data Science &… [find](https://scholar.google.com/scholar?q=Chakraborty%2C+A%2E%2C+Banerjee%2C+A%2E%2C+Dasgupta%2C+S%2E%2C+Raturi%2C+V%2E%2C+Soni%2C+A%2E%2C+Gupta%2C+A%2E%2C+Harsola%2C+S%2E%2C+and+Subrahmaniam%2C+V%2E+T%2E+Navigator%3A+A+gen%2Dai+system+for+discovery+of+factual+and+predictive+insights+on+domain)
19. Chang, J. C., Zhang, A. X., Bragg, J., Head, A., Lo, K., Downey, D., and Weld, D. S. Citesee: Augmenting citations in scientific papers with persistent and personalized historical context. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023. URL [https://api.semanticsch…](https://api.semanticsch…) [↗](https://api.semanticsch…)
20. Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). ArXiv, abs/2110.14168, 2021. URL [https://api.semanticscholar.org/CorpusID:239998651](https://api.semanticscholar.org/CorpusID:239998651). [↗](https://api.semanticscholar.org/CorpusID:239998651)
21. Collaboration, O. S. Reproducibility project: Psychology, 2015. URL https://doi.org/10.17605/OSF. IO/EZCUJ. — [doi:10.17605/osf](https://doi.org/10.17605/osf) [↗](https://doi.org/10.17605/OSF)
22. Dalvi, B., Jansen, P. A., Tafjord, O., Xie, Z., Smith, H., Pipatanangkura, L., and Clark, P. [Explaining answers with entailment trees](https://md.research.elamur.ai/free/en/papers/dalvi-2021-explaining-answers-entailment-trees/). In Conference on Empirical Methods in Natural Language Processing, 2021. URL [https://api.semanticscholar.org/CorpusID:233297051](https://api.semanticscholar.org/CorpusID:233297051). [↗](https://api.semanticscholar.org/CorpusID:233297051)
23. Dunn, O. J. Multiple comparisons among means. Journal of the American Statistical Association, 56:52–64, 1961. URL [https://api.semanticscholar.org/CorpusID:122009246](https://api.semanticscholar.org/CorpusID:122009246). [↗](https://api.semanticscholar.org/CorpusID:122009246)
24. Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., Grosse, R., McCandlish, S., Kaplan, J., Amodei, D., Wattenberg, M., and Olah, C. Toy models of superposition. Transformer Circuits Thread, 2022. https://transformer-circuit… [find](https://scholar.google.com/scholar?q=Elhage%2C+N%2E%2C+Hume%2C+T%2E%2C+Olsson%2C+C%2E%2C+Schiefer%2C+N%2E%2C+Henighan%2C+T%2E%2C+Kravec%2C+S%2E%2C+Hatfield%2DDodds%2C+Z%2E%2C+Lasenby%2C+R%2E%2C+Drain%2C+D%2E%2C+Chen%2C+C%2E%2C+Grosse%2C+R%2E%2C+McCandlish%2C+S%2E%2C+Kaplan%2C+J%2E%2C+Amodei%2C+D%2E%2C+Wattenberg%2C+M%2E%2C+and+)
25. Intelligence and Copyright. US Copyright Office Docket No. 2023-6, 2023. Comment. [find](https://scholar.google.com/scholar?q=Intelligence+and+Copyright%2E+US+Copyright+Office+Docket+No%2E+2023%2D6%2C+2023%2E+Comment%2E)
26. Feng, S., Park, C. Y., Liu, Y., and Tsvetkov, Y. [From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair nlp models](https://md.research.elamur.ai/free/en/papers/feng-2023-pretraining-data-language-models-downstream/). ArXiv, abs/2305.08283, 2023. URL [https://api.semanticscholar.org/CorpusID:258686693](https://api.semanticscholar.org/CorpusID:258686693). [↗](https://api.semanticscholar.org/CorpusID:258686693)
27. Feurer, M., Klein, A., Eggensperger, K., Springenberg, J., Blum, M., and Hutter, F. Efficient and robust automated machine learning. Advances in neural information processing systems, 28, 2015. [find](https://scholar.google.com/scholar?q=Feurer%2C+M%2E%2C+Klein%2C+A%2E%2C+Eggensperger%2C+K%2E%2C+Springenberg%2C+J%2E%2C+Blum%2C+M%2E%2C+and+Hutter%2C+F%2E+Efficient+and+robust+automated+machine+learning%2E+Advances+in+neural+information+processing+systems%2C+28%2C+2015%2E)
28. Foster, A. and Ford, N. Serendipity and information seeking: an empirical study. Journal of documentation, 59(3):321– 340, 2003. [find](https://scholar.google.com/scholar?q=Foster%2C+A%2E+and+Ford%2C+N%2E+Serendipity+and+information+seeking%3A+an+empirical+study%2E+Journal+of+documentation%2C+59%283%29%3A321%E2%80%93+340%2C+2003%2E)
29. Gil, Y., McWeeney, S. K., and Mason, C. E. Using semantic workflows to disseminate best practices and accelerate discoveries in multi-omic data analysis. In AAAI Conference on Artificial Intelligence, 2013. URL [https://api.semanticscholar.org/CorpusID:15583030](https://api.semanticscholar.org/CorpusID:15583030). [↗](https://api.semanticscholar.org/CorpusID:15583030)
30. Gil, Y., Garijo, D., Ratnakar, V., Mayani, R., Adusumilli, R., Boyce, H., Srivastava, A., and Mallick, P. Towards continuous scientific data analysis and hypothesis evolution. In AAAI Conference on Artificial Intelligence, 2017. URL [https://api.semanticscholar.org/CorpusID:11269287](https://api.semanticscholar.org/CorpusID:11269287). [↗](https://api.semanticscholar.org/CorpusID:11269287)
31. Elliott, K. C., Cheruvelil, K. S., Montgomery, G. M., and Soranno, P. A. Conceptions of good science in our data-rich world. BioScience, 66(10):880–889, 2016. [find](https://scholar.google.com/scholar?q=Elliott%2C+K%2E+C%2E%2C+Cheruvelil%2C+K%2E+S%2E%2C+Montgomery%2C+G%2E+M%2E%2C+and+Soranno%2C+P%2E+A%2E+Conceptions+of+good+science+in+our+data%2Drich+world%2E+BioScience%2C+66%2810%29%3A880%E2%80%93889%2C+2016%2E)
32. Gil, Y., Khider, D., Osorio, M., Ratnakar, V., Vargas, H., and Garijo, D. Towards capturing scientific reasoning to automate data analysis. 2022. URL [https://api.semanticscholar.org/CorpusID:248914202](https://api.semanticscholar.org/CorpusID:248914202). [↗](https://api.semanticscholar.org/CorpusID:248914202)
33. Ellis, K., Wong, C., Nye, M., Sablé-Meyer, M., Cary, L., Morales, L., Hewitt, L., Solar-Lezama, A., and Tenenbaum, J. B. Dreamcoder: growing generalizable, interpretable knowledge with wake–sleep bayesian program learning. Philosophical Transactions of the Royal Society A, 381, 2020. URL https://ap… [find](https://scholar.google.com/scholar?q=Ellis%2C+K%2E%2C+Wong%2C+C%2E%2C+Nye%2C+M%2E%2C+Sabl%C3%A9%2DMeyer%2C+M%2E%2C+Cary%2C+L%2E%2C+Morales%2C+L%2E%2C+Hewitt%2C+L%2E%2C+Solar%2DLezama%2C+A%2E%2C+and+Tenenbaum%2C+J%2E+B%2E+Dreamcoder%3A+growing+generalizable%2C+interpretable+knowledge+with+wake%E2%80%93sleep+baye)
34. Google. Introducing duet ai for google workspace. [https://workspace.google.com/blog/](https://workspace.google.com/blog/) product-announcements/duet-ai, 2023. Accessed: 2024-02-18. [↗](https://workspace.google.com/blog/)
35. Eysenbach, B., Gupta, A., Ibarz, J., and Levine, S. Diversity is all you need: Learning skills without a reward function. In International Conference on Learning Representations, 2019. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=SJx63jRqFm. [↗](https://openreview.net/forum?)
36. Fanelli, D. Opinion: Is science really facing a reproducibility crisis, and do we need it to? Proceedings of the National Academy of Sciences, 115:2628 – 2631, 2018. URL [https://api.semanticscholar.org/CorpusID:4639856](https://api.semanticscholar.org/CorpusID:4639856). [↗](https://api.semanticscholar.org/CorpusID:4639856)
37. Farhadi, A., Atkinson, D., Callison-Burch, C., DeCario, N., Dumas, J., Lo, K., and Soldiani, L. AI2’s Response to the US Copyright Requence for Comments on Artificial Groh, M., Sankaranarayanan, A., Singh, N., Kim, D. Y., Lippman, A., and Picard, R. W. Human detection of political speech deepfakes a… [find](https://scholar.google.com/scholar?q=Farhadi%2C+A%2E%2C+Atkinson%2C+D%2E%2C+Callison%2DBurch%2C+C%2E%2C+DeCario%2C+N%2E%2C+Dumas%2C+J%2E%2C+Lo%2C+K%2E%2C+and+Soldiani%2C+L%2E+AI2%E2%80%99s+Response+to+the+US+Copyright+Requence+for+Comments+on+Artificial+Groh%2C+M%2E%2C+Sankaranarayanan%2C+A%2E%2C+S)
38. Groth, O., Wulfmeier, M., Vezzani, G., Dasagi, V., Hertweck, T., Hafner, R., Heess, N. M. O., and Riedmiller, M. A. Is curiosity all you need? on the utility of emergent behaviours from curious exploration. ArXiv, abs/2109.08603, 2021. URL [https://api.semanticscholar.org/CorpusID:237563118](https://api.semanticscholar.org/CorpusID:237563118). — [arXiv:2109.08603](https://arxiv.org/abs/2109.08603) [↗](https://api.semanticscholar.org/CorpusID:237563118)
39. Gu, K., Grunde-McLaughlin, M., McNutt, A. M., Heer, J., and Althoff, T. How do data analysts respond to ai assistance? a wizard-of-oz study. ArXiv, abs/2309.10108, 2023. URL [https://api.semanticscholar.org/CorpusID:262054482](https://api.semanticscholar.org/CorpusID:262054482). — [arXiv:2309.10108](https://arxiv.org/abs/2309.10108) [↗](https://api.semanticscholar.org/CorpusID:262054482)
40. Hassabis, D. Using ai to accelerate scientific discovery, 2002. URL [https://www.youtube.com/watch?v=jocWJiztxYA&ab_channel=](https://www.youtube.com/watch?v=jocWJiztxYA&ab_channel=) InstituteforEthicsinAIOxford. [↗](https://www.youtube.com/watch?v=jocWJiztxYA&ab_channel=)
41. Heaven, W. D. Why meta’s latest large language model survived only three days online. MIT Technology Review. Last accessed December, 15:2022, 2022. [find](https://scholar.google.com/scholar?q=Heaven%2C+W%2E+D%2E+Why+meta%E2%80%99s+latest+large+language+model+survived+only+three+days+online%2E+MIT+Technology+Review%2E+Last+accessed+December%2C+15%3A2022%2C+2022%2E)
42. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D. X., and Steinhardt, J. Measuring massive multi-task language understanding. ArXiv, abs/2009.[03300, 2020](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). URL [https://api.semanticscholar.org/CorpusID:221516475](https://api.semanticscholar.org/CorpusID:221516475). [↗](https://api.semanticscholar.org/CorpusID:221516475)
43. Hennig, P. and Schuler, C. J. Entropy search for information-efficient global optimization. Journal of Machine Learning Research, 13(6), 2012. [find](https://scholar.google.com/scholar?q=Hennig%2C+P%2E+and+Schuler%2C+C%2E+J%2E+Entropy+search+for+information%2Defficient+global+optimization%2E+Journal+of+Machine+Learning+Research%2C+13%286%29%2C+2012%2E)
44. Hoffmann, C. G., Kiladis, G. N., Gehne, M., and von Savigny, C. A python package to calculate the olr-based index of the madden-julian-oscillation (omi) in climate science and weather forecasting. Journal of Open Research Software, 2021. URL [https://api.semanticscholar.org/CorpusID:236586655](https://api.semanticscholar.org/CorpusID:236586655). [↗](https://api.semanticscholar.org/CorpusID:236586655)
45. Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J. C., and Wang, H. Large language models for software engineering: A systematic literature review. ArXiv, abs/2308.10620, 2023. URL [https://api.semanticscholar.org/CorpusID:261048648](https://api.semanticscholar.org/CorpusID:261048648). — [arXiv:2308.10620](https://arxiv.org/abs/2308.10620) [↗](https://api.semanticscholar.org/CorpusID:261048648)
46. Houthooft, R., Chen, X., Duan, Y., Schulman, J., De Turck, F., and Abbeel, P. Vime: Variational information maximizing exploration. Advances in neural information processing systems, 29, 2016. [find](https://scholar.google.com/scholar?q=Houthooft%2C+R%2E%2C+Chen%2C+X%2E%2C+Duan%2C+Y%2E%2C+Schulman%2C+J%2E%2C+De+Turck%2C+F%2E%2C+and+Abbeel%2C+P%2E+Vime%3A+Variational+information+maximizing+exploration%2E+Advances+in+neural+information+processing+systems%2C+29%2C+2016%2E)
47. Kambhampati, S., Valmeekam, K., Guan, L., Stechly, K., Verma, M., Bhambri, S., Saldyt, L., and Murthy, A. Llms can’t plan, but can help planning in llm-modulo frameworks. arXiv preprint arXiv:2402.01817, 2024. [find](https://scholar.google.com/scholar?q=Kambhampati%2C+S%2E%2C+Valmeekam%2C+K%2E%2C+Guan%2C+L%2E%2C+Stechly%2C+K%2E%2C+Verma%2C+M%2E%2C+Bhambri%2C+S%2E%2C+Saldyt%2C+L%2E%2C+and+Murthy%2C+A%2E+Llms+can%E2%80%99t+plan%2C+but+can+help+planning+in+llm%2Dmodulo+frameworks%2E+arXiv+preprint+arXiv%3A2402%2E018)
48. Kang, D., Emmons, J., Abuzaid, F., Bailis, P. D., and Zaharia, M. A. Noscope: Optimizing deep cnn-based queries over video streams at scale. Proc. VLDB Endow., 10:1586–1597, 2017. URL [https://api.semanticscholar.org/CorpusID:20732104](https://api.semanticscholar.org/CorpusID:20732104). [↗](https://api.semanticscholar.org/CorpusID:20732104)
49. Khot, T., Trivedi, H., Finlayson, M., Fu, Y., Richardson, K., Clark, P., and Sabharwal, A. Decomposed prompting: A modular approach for solving complex tasks. ArXiv, abs/2210.02406, 2022. URL [https://api.semanticscholar.org/CorpusID:252715485](https://api.semanticscholar.org/CorpusID:252715485). [↗](https://api.semanticscholar.org/CorpusID:252715485)
50. Korthauer, K., Kimes, P. K., Duvallet, C., Reyes, A., Subramanian, A., Teng, M., Shukla, C., Alm, E. J., and Hicks, S. C. A practical guide to methods controlling false discoveries in computational biology. Genome Biology, 20, 2019. doi: 10.1186/s13059-019-1716-1. URL [https://genomebiology.biomedcen…](https://genomebiology.biomedcen…) — [doi:10.1186/s13059-019-1716-1](https://doi.org/10.1186/s13059-019-1716-1) [↗](https://genomebiology.biomedcen…)
51. Korthauer, K. D., Kimes, P. K., Duvallet, C., Reyes, A., Subramanian, A., Teng, M., Shukla, C. J., Alm, E. J., and Hicks, S. C. A practical guide to methods controlling false discoveries in computational biology. Genome Biology, 20, 2018. URL [https://api.semanticscholar.org/CorpusID:91264977](https://api.semanticscholar.org/CorpusID:91264977). [↗](https://api.semanticscholar.org/CorpusID:91264977)
52. Lahiri, S. K., Naik, A., Sakkas, G., Choudhury, P., von Veh, C., Musuvathi, M., Inala, J. P., Wang, C., and Gao, J. Interactive code generation via test-driven user-intent formalization. ArXiv, abs/2208.05950, 2022. URL [https://api.semanticscholar.org/CorpusID:251492970](https://api.semanticscholar.org/CorpusID:251492970). — [arXiv:2208.05950](https://arxiv.org/abs/2208.05950) [↗](https://api.semanticscholar.org/CorpusID:251492970)
53. Langley, P. Bacon: A production system that discovers empirical laws. In International Joint Conference on Artificial Intelligence, 1977. URL [https://api.semanticscholar.org/CorpusID:2320342](https://api.semanticscholar.org/CorpusID:2320342). [↗](https://api.semanticscholar.org/CorpusID:2320342)
54. Huang, Q., Vora, J., Liang, P., and Leskovec, J. Benchmarking large language models as ai research agents. ArXiv, abs/2310.[03302, 2023](https://md.research.elamur.ai/free/en/papers/huang-2023-mlagentbench-evaluating-language-agents/). URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 263671541. [↗](https://api.semanticscholar.org/CorpusID:)
55. Langley, P. Data-driven discovery of physical laws. Cogn. Sci., 5:31–54, 1981. URL [https://api.semanticscholar.org/CorpusID:39694251](https://api.semanticscholar.org/CorpusID:39694251). [↗](https://api.semanticscholar.org/CorpusID:39694251)
56. Langley, P., Bradshaw, G. L., and Simon, H. A. Rediscovering chemistry with the bacon system. 1983. URL [https://api.semanticscholar.org/CorpusID:118714327](https://api.semanticscholar.org/CorpusID:118714327). [↗](https://api.semanticscholar.org/CorpusID:118714327)
57. Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. [Highly accurate protein structure prediction with alphafold](https://md.research.elamur.ai/free/en/papers/jumper-2021-highly-accurate-protein-structure-prediction/). Nature, 596(7873):583–589, 2021.
58. Langley, P., Zytkow, J. M., Simon, H. A., and Bradshaw, G. L. The search for regularity: Four aspects Kahneman, D. Thinking, fast and slow. macmillan, 2011. of scientific discovery. 1984. URL [https://api.semanticscholar.org/CorpusID:3155192](https://api.semanticscholar.org/CorpusID:3155192). [↗](https://api.semanticscholar.org/CorpusID:3155192)
59. Lau, J. W., Lehnert, E., Sethi, A., Malhotra, R., Kaushik, G., Onder, Z., Groves-Kirkby, N., Mihajlovic, A., DiGio-vanna, J., Srdic, M., et al. The cancer genomics cloud: collaborative, reproducible, and democratized—a new paradigm in large-scale computational research. Cancer research, 77(21):e3–e6… — [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/)
60. Lukasczyk, S. and Fraser, G. Pynguin: Automated unit test generation for python. 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 168–172, 2022. URL [https://api.semanticscholar.org/CorpusID:246706202](https://api.semanticscholar.org/CorpusID:246706202). [↗](https://api.semanticscholar.org/CorpusID:246706202)
61. LeCun, Y. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62(1), 2022. [find](https://scholar.google.com/scholar?q=LeCun%2C+Y%2E+A+path+towards+autonomous+machine+intelligence+version+0%2E9%2E+2%2C+2022%2D06%2D27%2E+Open+Review%2C+62%281%29%2C+2022%2E)
62. Leeman, J., Liu, Y., Stiles, J., Lee, S., Bhatt, P., Schoop, L., and Palgrave, R. Challenges in high-throughput inorganic material prediction and autonomous synthesis. 2024. [find](https://scholar.google.com/scholar?q=Leeman%2C+J%2E%2C+Liu%2C+Y%2E%2C+Stiles%2C+J%2E%2C+Lee%2C+S%2E%2C+Bhatt%2C+P%2E%2C+Schoop%2C+L%2E%2C+and+Palgrave%2C+R%2E+Challenges+in+high%2Dthroughput+inorganic+material+prediction+and+autonomous+synthesis%2E+2024%2E)
63. Li, Y., Choi, D. H., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Tom, Eccles, Keeling, J., Gimeno, F., Lago, A. D., Hubert, T., Choy, P., de, C., d’Autume, M., Babuschkin, I., Chen, X., Huang, P.-S., Welbl, J., Gowal, S., Alexey, Cherepanov, Molloy, J., Mankowitz, D. J., Robson, E. S., K… — [Competition-Level Code Generation with AlphaCode](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/)
64. Lin, B. Y., Wang, S. I., Lin, X. V., Jia, R., Xiao, L., Ren, X., and tau Yih, W. On continual model refinement in out-of-distribution data streams. In Annual Meeting of the Association for Computational Linguistics, 2022. URL [https://api.semanticscholar.org/CorpusID:248512744](https://api.semanticscholar.org/CorpusID:248512744). [↗](https://api.semanticscholar.org/CorpusID:248512744)
65. Liu, H., Li, C., Wu, Q., and Lee, Y. J. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). ArXiv, abs/2304.08485, 2023a. URL [https://api.semanticscholar.org/CorpusID:258179774](https://api.semanticscholar.org/CorpusID:258179774). [↗](https://api.semanticscholar.org/CorpusID:258179774)
66. Liu, J., Xia, C., Wang, Y., and Zhang, L. [Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/). ArXiv, abs/2305.01210, 2023b. URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 258437095. [↗](https://api.semanticscholar.org/CorpusID:)
67. Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Gu, Y., Ding, H., Men, K., Yang, K., Zhang, S., Deng, X., Zeng, A., Du, Z., Zhang, C., Shen, S., Zhang, T., Su, Y., Sun, H., Huang, M., Dong, Y., and Tang, J. [Agentbench: Evaluating llms as agents](https://md.research.elamur.ai/free/en/papers/liu-2023-agentbench-evaluating-llms-agents/). ArXiv, abs/2308.03688, 2023c. URL https:…
68. Lu, P., Bansal, H., Xia, T., Liu, J., yue Li, C., Hajishirzi, H., Cheng, H., Chang, K.-W., Galley, M., Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Welleck, S., Majumder, B. P., Gupta, S., Yazdanbakhsh, A., and Clark, P. Self… [find](https://scholar.google.com/scholar?q=Lu%2C+P%2E%2C+Bansal%2C+H%2E%2C+Xia%2C+T%2E%2C+Liu%2C+J%2E%2C+yue+Li%2C+C%2E%2C+Hajishirzi%2C+H%2E%2C+Cheng%2C+H%2E%2C+Chang%2C+K%2E%2DW%2E%2C+Galley%2C+M%2E%2C+Madaan%2C+A%2E%2C+Tandon%2C+N%2E%2C+Gupta%2C+P%2E%2C+Hallinan%2C+S%2E%2C+Gao%2C+L%2E%2C+Wiegreffe%2C+S%2E%2C+Alon%2C+U%2E%2C+Dziri%2C+N%2E%2C+Pra)
69. Magnusson, I. H., Smith, N. A., and Dodge, J. Reproducibility in nlp: What have we learned from the checklist? In Annual Meeting of the Association for Computational Linguistics, 2023. URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 259187997. [↗](https://api.semanticscholar.org/CorpusID:)
70. Majumder, B. P., Rao, S., Galley, M., and McAuley, J. Ask what’s missing and what’s useful: Improving clarification question generation using global knowledge. In North American Chapter of the Association for Computational Linguistics, 2021. URL [https://api.semanticscholar.org/](https://api.semanticscholar.org/) CorpusID:233231257. [↗](https://api.semanticscholar.org/)
71. Majumder, B. P., Jhamtani, H., Berg-Kirkpatrick, T., and McAuley, J. Achieving conversational goals with unsupervised post-hoc knowledge injection. ArXiv, abs/2203.11399, 2022. URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 247547046. [↗](https://api.semanticscholar.org/CorpusID:)
72. Majumder, B. P., Dalvi, B., Jansen, P., Tafjord, O., Tandon, N., Zhang, L., Callison-Burch, C., and Clark, P. [Clin: A continually learning language agent for rapid task adaptation and generalization](https://md.research.elamur.ai/free/en/papers/majumder-2023-clin-continually-learning-language-agent/). ArXiv, abs/2310.10134, 2023. URL [https://api.semanticscholar.org/CorpusID:264146262](https://api.semanticscholar.org/CorpusID:264146262). [↗](https://api.semanticscholar.org/CorpusID:264146262)
73. Monroy, D. Introducing copilot support for python in excel: Advanced data analysis using natural language. [https://techcommunity.microsoft.com/](https://techcommunity.microsoft.com/) t5/excel-blog/introducing-copilot-support-for-python-in-excel-advanced-data/ba-p/3928120, 2023. Accessed: 2024-02-18. [↗](https://techcommunity.microsoft.com/)
74. Newell, A. and Simon, H. A. Computer science as empirical inquiry: symbols and search. Commun. ACM, 19(3):113–126, mar 1976. ISSN 0001-0782. doi: 10.1145/360018.360022. URL https://doi.org/ 10.1145/360018.360022. [find](https://scholar.google.com/scholar?q=Newell%2C+A%2E+and+Simon%2C+H%2E+A%2E+Computer+science+as+empirical+inquiry%3A+symbols+and+search%2E+Commun%2E+ACM%2C+19%283%29%3A113%E2%80%93126%2C+mar+1976%2E+ISSN+0001%2D0782%2E+doi%3A+10%2E1145%2F360018%2E360022%2E+URL+https%3A%2F%2Fdoi%2Eorg%2F+10%2E1145%2F36)
75. Ristoski, P. and Paulheim, H. Semantic web in data mining and knowledge discovery: A comprehensive survey. J. Web Semant., 36:1–22, 2016. URL [https://api.semanticscholar.org/CorpusID:42846121](https://api.semanticscholar.org/CorpusID:42846121). [↗](https://api.semanticscholar.org/CorpusID:42846121)
76. Oudeyer, P.-Y. and Kaplan, F. How can we define intrinsic motivation? 2008. URL [https://api.semanticscholar.org/CorpusID:14217330](https://api.semanticscholar.org/CorpusID:14217330). [↗](https://api.semanticscholar.org/CorpusID:14217330)
77. Oudeyer, P.-Y., Kaplan, F., and Hafner, V. V. Intrinsic motivation systems for autonomous mental development. IEEE transactions on evolutionary computation, 11(2): 265–286, 2007. [find](https://scholar.google.com/scholar?q=Oudeyer%2C+P%2E%2DY%2E%2C+Kaplan%2C+F%2E%2C+and+Hafner%2C+V%2E+V%2E+Intrinsic+motivation+systems+for+autonomous+mental+development%2E+IEEE+transactions+on+evolutionary+computation%2C+11%282%29%3A+265%E2%80%93286%2C+2007%2E)
78. Palani, S., Naik, A., Downey, D., Zhang, A. X., Bragg, J., and Chang, J. C. Relatedly: Scaffolding literature reviews with existing related work sections. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023. URL [https://api.semanticscholar.org/CorpusID:256846632](https://api.semanticscholar.org/CorpusID:256846632). [↗](https://api.semanticscholar.org/CorpusID:256846632)
79. Pathak, D., Agrawal, P., Efros, A. A., and Darrell, T. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pp. 2778–2787. PMLR, 2017. [find](https://scholar.google.com/scholar?q=Pathak%2C+D%2E%2C+Agrawal%2C+P%2E%2C+Efros%2C+A%2E+A%2E%2C+and+Darrell%2C+T%2E+Curiosity%2Ddriven+exploration+by+self%2Dsupervised+prediction%2E+In+International+conference+on+machine+learning%2C+pp%2E+2778%E2%80%932787%2E+PMLR%2C+2017%2E)
80. Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J. R., Ellenberg, J. S., Wang, P., Fawzi, O., Kohli, P., Fawzi, A., Grochow, J., Lodi, A., Mouret, J.-B., Ringer, T., and Yu, T. [Mathematical discoveries from program search with large language models](https://md.research.elamur.ai/free/en/papers/paredes-2023-mathematical-discoveries-program-search-large/). Nat…
81. Rotolo, F., Paoletti, X., and Michiels, S. surrosurv: An r package for the evaluation of failure time surrogate endpoints in individual patient data meta-analyses of randomized clinical trials. Computer methods and programs in biomedicine, 155:189–198, 2018. URL https://api. semanticscholar.org/Corp… [find](https://scholar.google.com/scholar?q=Rotolo%2C+F%2E%2C+Paoletti%2C+X%2E%2C+and+Michiels%2C+S%2E+surrosurv%3A+An+r+package+for+the+evaluation+of+failure+time+surrogate+endpoints+in+individual+patient+data+meta%2Danalyses+of+randomized+clinical+trials%2E+Comput)
82. Santos, M., Clemente, F., and Abshire, C. Pandas-profiling now supports apache spark. [https://www.databricks.com/blog/2023/04/03/](https://www.databricks.com/blog/2023/04/03/) pandas-profiling-now-supports-apache-spark.html, 2023. Accessed: 2024-02-18. [↗](https://www.databricks.com/blog/2023/04/03/)
83. Schäfer, M., Nadi, S., Eghbali, A., and Tip, F. An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Software Engineering, 50:85–105, 2023. URL [https://api.semanticscholar.org/CorpusID:256827098](https://api.semanticscholar.org/CorpusID:256827098). [↗](https://api.semanticscholar.org/CorpusID:256827098)
84. Pelrine, K., Taufeeque, M., Zajkac, M., McLean, E., and Gleave, A. Exploiting novel gpt-4 apis. ArXiv, abs/2312.14302, 2023. URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 266521205. — [arXiv:2312.14302](https://arxiv.org/abs/2312.14302) [↗](https://api.semanticscholar.org/CorpusID:)
85. Perlitz, Y., Sheinwald, D., Slonim, N., and Shmueli-Scheuer, M. nbiig: A neural bi insights generation system for table reporting. In AAAI Conference on Artificial Intelligence, 2022. URL [https://api.semanticscholar.org/CorpusID:253397856](https://api.semanticscholar.org/CorpusID:253397856). [↗](https://api.semanticscholar.org/CorpusID:253397856)
86. Prasad, A., Koller, A., Hartmann, M., Clark, P., Sabharwal, A., Bansal, M., and Khot, T. [Adapt: As-needed decomposition and planning with language models](https://md.research.elamur.ai/free/en/papers/prasad-2024-adapt-as-needed-decomposition-planning/). ArXiv, abs/2311.05772, 2023. URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 265128575. [↗](https://api.semanticscholar.org/CorpusID:)
87. Qiu, L., Jiang, L., Lu, X., Sclar, M., Pyatkin, V., Bhagavatula, C., Wang, B., Kim, Y., Choi, Y., Dziri, N., and Ren, X. Phenomenal yet puzzling: Testing inductive reasoning capabilities of language models with hypothesis refinement. ArXiv, abs/2310.08559, 2023. URL [https://api.semanticscholar](https://api.semanticscholar). org/… [↗](https://api.semanticscholar)
88. Ramakrishnan, N. and Grama, A. Y. Data mining: From serendipity to science. Computer, 32(8):34–37, 1999. [find](https://scholar.google.com/scholar?q=Ramakrishnan%2C+N%2E+and+Grama%2C+A%2E+Y%2E+Data+mining%3A+From+serendipity+to+science%2E+Computer%2C+32%288%29%3A34%E2%80%9337%2C+1999%2E)
89. Schick, T., Dwivedi-Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., and Scialom, T. [Toolformer: Language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL [https://openreview.net/forum?](https://openreview.net/forum?) id… [↗](https://openreview.net/forum?)
90. Sharma, A., Li, X., Guan, H., Sun, G., Zhang, L., Wang, L., Wu, K., Cao, L., Zhu, E., Sim, A., Wu, T., and Zou, J. Automatic data transformation using large language model - an experimental study on building energy data. 2023 IEEE International Conference on Big Data (BigData), pp. 1824–1834, 2023.… [find](https://scholar.google.com/scholar?q=Sharma%2C+A%2E%2C+Li%2C+X%2E%2C+Guan%2C+H%2E%2C+Sun%2C+G%2E%2C+Zhang%2C+L%2E%2C+Wang%2C+L%2E%2C+Wu%2C+K%2E%2C+Cao%2C+L%2E%2C+Zhu%2C+E%2E%2C+Sim%2C+A%2E%2C+Wu%2C+T%2E%2C+and+Zou%2C+J%2E+Automatic+data+transformation+using+large+language+model+%2D+an+experimental+study+on+b)
91. Shinn, N., Cassano, F., Labash, B., Gopinath, A., Narasimhan, K., and Yao, S. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). 2023. URL [https://api.semanticscholar.org/CorpusID:258833055](https://api.semanticscholar.org/CorpusID:258833055). [↗](https://api.semanticscholar.org/CorpusID:258833055)
92. Smith, P. K., Bogin, B., and Bishai, D. Are time preference and body mass index associated?: Evidence from the national longitudinal survey of youth. Economics & Human Biology, 3(2):259–270, 2005. language models. CHI Conference on Human Factors in Computing Systems Extended Abstracts, 2022. URL htt… [find](https://scholar.google.com/scholar?q=Smith%2C+P%2E+K%2E%2C+Bogin%2C+B%2E%2C+and+Bishai%2C+D%2E+Are+time+preference+and+body+mass+index+associated%3F%3A+Evidence+from+the+national+longitudinal+survey+of+youth%2E+Economics+%26+Human+Biology%2C+3%282%29%3A259%E2%80%93270%2C+2005%2E+lan)
93. Stanley, K. O., Lehman, J., and Soros, L. Open-endedness: The last grand challenge you’ve never heard of. While open-endedness could be a force for discovering intelligence, it could also be a component of AI itself, 2017. [find](https://scholar.google.com/scholar?q=Stanley%2C+K%2E+O%2E%2C+Lehman%2C+J%2E%2C+and+Soros%2C+L%2E+Open%2Dendedness%3A+The+last+grand+challenge+you%E2%80%99ve+never+heard+of%2E+While+open%2Dendedness+could+be+a+force+for+discovering+intelligence%2C+it+could+also+be+a+compone)
94. Valmeekam, K., Olmo, A., Sreedharan, S., and Kambhampati, S. [Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change](https://md.research.elamur.ai/free/en/papers/valmeekam-2022-planbench-extensible-benchmark-evaluating-large/). 2022. URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 249889477. [↗](https://api.semanticscholar.org/CorpusID:)
95. Sun, R., Arik, S. Ö., Nakhost, H., Dai, H., Sinha, R., Yin, P., and Pfister, T. Sql-palm: Improved large language model adaptation for text-to-sql. ArXiv, abs/2306.00739, 2023. URL [https://api.semanticscholar.org/CorpusID:258999853](https://api.semanticscholar.org/CorpusID:258999853). — [arXiv:2306.00739](https://arxiv.org/abs/2306.00739) [↗](https://api.semanticscholar.org/CorpusID:258999853)
96. Swanson, D. R. Undiscovered public knowledge. The Library Quarterly, 56:103–118, 1986. URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 144270735. [↗](https://api.semanticscholar.org/CorpusID:)
97. Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L. J., and Anandkumar, A. Voyager: An open-ended embodied agent with large language models. ArXiv, abs/2305.16291, 2023a. URL [https://api.semanticscholar.org/CorpusID:258887849](https://api.semanticscholar.org/CorpusID:258887849). [↗](https://api.semanticscholar.org/CorpusID:258887849)
98. Wang, Q., Downey, D., Ji, H., and Hope, T. Learning to generate novel scientific directions with contextualized literature-based discovery. ArXiv, abs/2305.14259, 2023b. URL [https://api.semanticscholar.org/CorpusID:258841365](https://api.semanticscholar.org/CorpusID:258841365). [↗](https://api.semanticscholar.org/CorpusID:258841365)
99. Taleb, N. N. The Black Swan: The Impact of the Highly Improbable. Random House Group, 2007. ISBN 1400063515. [find](https://scholar.google.com/scholar?q=Taleb%2C+N%2E+N%2E+The+Black+Swan%3A+The+Impact+of+the+Highly+Improbable%2E+Random+House+Group%2C+2007%2E+ISBN+1400063515%2E)
100. Touvron, H., Martin, L., Stone, K. R., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D. M., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., Hartshorn, A. S… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
101. Wasserstein, R. and Lazar, N. A. The asa statement on p-values: Context, process, and purpose. The American Statistician, 70:129 – 133, 2016. URL [https://api.semanticscholar.org/CorpusID:124084622](https://api.semanticscholar.org/CorpusID:124084622). [↗](https://api.semanticscholar.org/CorpusID:124084622)
102. Wei, J., Bosma, M., Zhao, V., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations, 2022. URL [https://openreview.net/forum?id=gEZrGCozdqR](https://openreview.net/forum?id=gEZrGCozdqR). [↗](https://openreview.net/forum?id=gEZrGCozdqR)
103. Weiss, S. T. and Ware, J. H. Overview of issues in the longitudinal analysis of respiratory data. American journal of respiratory and critical care medicine, 154 6 Pt 2:S208– 11, 1996. URL [https://api.semanticscholar.org/CorpusID:45049299](https://api.semanticscholar.org/CorpusID:45049299). [↗](https://api.semanticscholar.org/CorpusID:45049299)
104. Trinh, T. H., Wu, Y., Le, Q. V., He, H., and Luong, T. [Solving olympiad geometry without human demonstrations](https://md.research.elamur.ai/free/en/papers/trinh-2024-solving-olympiad-geometry-without-human/). Nature, 625(7995):476–482, 2024.
105. Weitzman, M. Optimal search for the best alternative. Econometrica, 47:641–654, 1978. URL [https://api.semanticscholar.org/CorpusID:32530881](https://api.semanticscholar.org/CorpusID:32530881). [↗](https://api.semanticscholar.org/CorpusID:32530881)
106. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., and Bouchard, G. [Complex embeddings for simple link prediction](https://md.research.elamur.ai/free/en/papers/trouillon-2016-complex-embeddings-simple-link-prediction/). ArXiv, abs/1606.06357, 2016. URL [https://api.semanticscholar.org/CorpusID:15150247](https://api.semanticscholar.org/CorpusID:15150247). [↗](https://api.semanticscholar.org/CorpusID:15150247)
107. Whitley, L. D. Fundamental principles of deception in genetic search. In Foundations of genetic algorithms, volume 1, pp. 221–241. Elsevier, 1991. [find](https://scholar.google.com/scholar?q=Whitley%2C+L%2E+D%2E+Fundamental+principles+of+deception+in+genetic+search%2E+In+Foundations+of+genetic+algorithms%2C+volume+1%2C+pp%2E+221%E2%80%93241%2E+Elsevier%2C+1991%2E)
108. Vaithilingam, P., Zhang, T., and Glassman, E. L. Expectation vs. experience: Evaluating the usability of code generation tools powered by large Wolf, Y., Wies, N., Levine, Y., and Shashua, A. [Fundamental limitations of alignment in large language models](https://md.research.elamur.ai/free/en/papers/wolf-2023-fundamental-limitations-alignment-large-language/). ArXiv, abs/2304.11082, 2023. URL https://api…
109. Zaw, K., Hamilton, D., and Darity, W. A. J. Race, wealth and incarceration: Results from the national longitudinal survey of youth. Race and Social Problems, 8:103–115, 2016. URL [https://api.semanticscholar.org/CorpusID:13709779](https://api.semanticscholar.org/CorpusID:13709779). [↗](https://api.semanticscholar.org/CorpusID:13709779)
110. Zhang, J., Lehman, J., Stanley, K., and Clune, J. Omni: Open-endedness via models of human notions of interestingness. arXiv preprint arXiv:2306.01711, 2023. [find](https://scholar.google.com/scholar?q=Zhang%2C+J%2E%2C+Lehman%2C+J%2E%2C+Stanley%2C+K%2E%2C+and+Clune%2C+J%2E+Omni%3A+Open%2Dendedness+via+models+of+human+notions+of+interestingness%2E+arXiv+preprint+arXiv%3A2306%2E01711%2C+2023%2E)
111. Zhou, X., Zhu, H., Mathur, L., Zhang, R., Yu, H., Qi, Z., Morency, L.-P., Bisk, Y., Fried, D., Neubig, G., and Sap, M. [Sotopia: Interactive evaluation for social intelligence in language agents](https://md.research.elamur.ai/free/en/papers/zhou-2023-sotopia-interactive-evaluation-social-intelligence/). ArXiv, abs/2310.11667, 2023. URL [https://api.semanticscholar.org/CorpusID:264289186](https://api.semanticscholar.org/CorpusID:264289186). *Figure 4. Agent St… [↗](https://api.semanticscholar.org/CorpusID:264289186)

*Updated 30 October 2026*