---
title: "DiscoveryBench — paper by Majumder et al. (2024)"
description: "DiscoveryBench tests whether LLM-based agents can independently form and check hypotheses… 55 references, cited by 53 papers here."
authors: ["Bodhisattwa Prasad Majumder", "Harshit Surana", "Dhruv Agarwal", "Bhavana Dalvi Mishra", "Abhijeetsingh Meena", "Aryan Prakhar", "Tirth Vora", "Tushar Khot", "Ashish Sabharwal", "Peter Clark"]
year: 2024
venue: "arXiv:2407.01725"
source: "https://arxiv.org/abs/2407.01725"
doi: "10.48550/arxiv.2407.01725"
arxiv: "2407.01725"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/majumder-2024-discoverybench-data-driven-discovery/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/majumder-2024-discoverybench-data-driven-discovery/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/7/) / DiscoveryBench: Towards Data-Driven Discovery with Large Language Models

# DiscoveryBench: Towards Data-Driven Discovery with Large Language Models

**Authors:** [Bodhisattwa Prasad Majumder](https://md.research.elamur.ai/free/en/authors/bodhisattwa-prasad-majumder/), [Harshit Surana](https://md.research.elamur.ai/free/en/authors/harshit-surana/), [Dhruv Agarwal](https://md.research.elamur.ai/free/en/authors/dhruv-agarwal/), [Bhavana Dalvi Mishra](https://md.research.elamur.ai/free/en/authors/bhavana-dalvi-mishra/), Abhijeetsingh Meena, Aryan Prakhar, Tirth Vora, [Tushar Khot](https://md.research.elamur.ai/free/en/authors/tushar-khot/), [Ashish Sabharwal](https://md.research.elamur.ai/free/en/authors/ashish-sabharwal/) and [Peter Clark](https://md.research.elamur.ai/free/en/authors/peter-clark/) · **Year:** 2024 · **Venue:** arXiv:2407.01725 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2407.01725) · [PDF](https://arxiv.org/pdf/2407.01725) · [DOI 10.48550/arxiv.2407.01725](https://doi.org/10.48550/arxiv.2407.01725) · [arXiv 2407.01725](https://arxiv.org/abs/2407.01725) · [OpenAlex](https://openalex.org/W4400373784)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [53](https://md.research.elamur.ai/free/en/papers/majumder-2024-discoverybench-data-driven-discovery/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
DiscoveryBench tests whether LLM-based agents can independently form and check hypotheses using only supplied datasets, a setting the authors call data-driven discovery. They model a hypothesis as a combination of a context, a set of variables and a relationship. A task hides one of these and gives a natural-language goal plus dataset(s) with metadata. The real portion, DB-REAL, has 264 tasks built by hand from discovery workflows in more than 20 published studies across six fields, including sociology and engineering. DB-SYNTH adds 903 LLM-generated tasks spanning 48 domains, with controllable difficulty.

Because answers are open-ended, scoring compares predicted and gold sub-hypotheses facet by facet. Several agent frameworks were run with both open and closed LLMs, and the strongest system reaches only about 25%. Errors grow with workflow length and with the need for specialized statistics or domain knowledge. The benchmark exposes how far autonomous scientific analysis still has to go and offers a reproducible way to track progress.

## Key points
- **Resource and evaluation:** DB-REAL (264 tasks) and DB-SYNTH (903 tasks) are released on GitHub and Hugging Face under allenai/discoverybench. The evaluation metric is the Hypothesis Matching Score (HMS), which is built from context F1, variable F1 and relationship accuracy.
- **Baselines:** The tested agents were CodeGen, React, DataVoyager, Reflexion (Oracle) and a NoDataGuess memorization check. They were paired with GPT-4o, GPT-4p and Llama-3. React prompting and DataVoyager's planner-critic design did not beat plain CodeGen. Only Reflexion, which receives oracle feedback, improved substantially.
- **Task types:** Correlation analysis was the best-handled workflow category for the top non-oracle agent, at about 55%. Spatial analysis, pollen dating and ecological modeling all scored 0%.
- **Domains:** Economics (25%) and sociology (23%) did best, while biology (0%) and engineering (7%) did worst. Recovering a relationship when context and variables are given is easier than finding the context or the variables.
- **Domain hints:** Adding domain-knowledge hints to archaeology tasks raised DataVoyager (GPT-4p) from 9.9% to 17.5%. Scores also drop steeply as gold workflow length or semantic-tree height increases. Results on synthetic and real tasks were similar.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) — Research institute, Seattle, United States · 304 works · 106,138 citations
- [University of Massachusetts Amherst](https://md.research.elamur.ai/free/en/universities/university-of-massachusetts-amherst/) — University, Amherst Center, United States · 114 works · 68,731 citations

📚 Shelves (8): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 249 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 254 works · [🎯 Information Systems and Management](https://md.research.elamur.ai/free/en/shelves/decision-sciences/information-systems-and-management/) · 52 works

## Cite

Cite as: Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Bhavana Dalvi Mishra, Abhijeetsingh Meena, Aryan Prakhar, Tirth Vora, Tushar Khot, Ashish Sabharwal and Peter Clark. 2024. DiscoveryBench: Towards Data-Driven Discovery with Large Language Models. arXiv:2407.01725.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{majumder2024discoverybench,
  title = {{DiscoveryBench: Towards Data-Driven Discovery with Large Language Models}},
  author = {Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Bhavana Dalvi Mishra, Abhijeetsingh Meena, Aryan Prakhar, Tirth Vora, Tushar Khot, Ashish Sabharwal and Peter Clark},
  year = {2024},
  doi = {10.48550/arxiv.2407.01725},
  eprint = {2407.01725},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2407.01725}
}
```

- [← previous Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts](https://md.research.elamur.ai/free/en/papers/nguyen-2024-democratizing-llms-low-resource-languages/)
- [next → Flex Attention: A Programming Model for Generating Optimized Attention Kernels](https://md.research.elamur.ai/free/en/papers/dong-2024-flex-attention-programming-model-generating/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/7/)

## References

55 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. K. L. Alexander, C. Riordan, J. Fennessey, and A. M. Pallas. Social background, academic resources, and college graduation: Recent evidence from the national longitudinal survey. American Journal of Education, 90(4):315–333, 1982. [find](https://scholar.google.com/scholar?q=K%2E+L%2E+Alexander%2C+C%2E+Riordan%2C+J%2E+Fennessey%2C+and+A%2E+M%2E+Pallas%2E+Social+background%2C+academic+resources%2C+and+college+graduation%3A+Recent+evidence+from+the+national+longitudinal+survey%2E+American+Journal+of+E)
2. A. P. S. Alves, M. Kalinowski, G. Giray, D. Mendez, N. Lavesson, K. Azevedo, H. Villamizar, T. Escovedo, H. Lopes, S. Biffl, et al. Status quo and problems of requirements engineering for machine learning: Results from an international survey. In International Conference on Product-Focused Software… [find](https://scholar.google.com/scholar?q=A%2E+P%2E+S%2E+Alves%2C+M%2E+Kalinowski%2C+G%2E+Giray%2C+D%2E+Mendez%2C+N%2E+Lavesson%2C+K%2E+Azevedo%2C+H%2E+Villamizar%2C+T%2E+Escovedo%2C+H%2E+Lopes%2C+S%2E+Biffl%2C+et+al%2E+Status+quo+and+problems+of+requirements+engineering+for+machine+lear)
3. R. Apel and G. Sweeten. The impact of incarceration on employment during the transition to adulthood. Social problems, 57(3):448–479, 2010. [find](https://scholar.google.com/scholar?q=R%2E+Apel+and+G%2E+Sweeten%2E+The+impact+of+incarceration+on+employment+during+the+transition+to+adulthood%2E+Social+problems%2C+57%283%29%3A448%E2%80%93479%2C+2010%2E)
4. E. N. Appiah. The effect of education expenditure on per capita gdp in developing countries. International Journal of Economics and Finance, 9(10):136–144, 2017. [find](https://scholar.google.com/scholar?q=E%2E+N%2E+Appiah%2E+The+effect+of+education+expenditure+on+per+capita+gdp+in+developing+countries%2E+International+Journal+of+Economics+and+Finance%2C+9%2810%29%3A136%E2%80%93144%2C+2017%2E)
5. J. Brinkmann. Copper output, demand for wood and energy expenditure–evaluating economic aspects of bronze age metallurgy. How’s life, pages 11–34, 2019. [find](https://scholar.google.com/scholar?q=J%2E+Brinkmann%2E+Copper+output%2C+demand+for+wood+and+energy+expenditure%E2%80%93evaluating+economic+aspects+of+bronze+age+metallurgy%2E+How%E2%80%99s+life%2C+pages+11%E2%80%9334%2C+2019%2E)
6. J. P. Brozio, J. Müller, M. Furholt, W. Kirleis, S. Dreibrodt, I. Feeser, W. Dörfler, M. Weinelt, H. Raese, and A. Bock. Monuments and economies: What drove their variability in the middle-holocene neolithic? The Holocene, 29(10):1558–1571, 2019. [find](https://scholar.google.com/scholar?q=J%2E+P%2E+Brozio%2C+J%2E+M%C3%BCller%2C+M%2E+Furholt%2C+W%2E+Kirleis%2C+S%2E+Dreibrodt%2C+I%2E+Feeser%2C+W%2E+D%C3%B6rfler%2C+M%2E+Weinelt%2C+H%2E+Raese%2C+and+A%2E+Bock%2E+Monuments+and+economies%3A+What+drove+their+variability+in+the+middle%2Dholocene+ne)
7. J. P. Brozio, J. Kneisel, S. Schaefer-Di Maida, J. Laabs, I. Feeser, A. Ribeiro, and S. Schultrich. Patterns of socio-economic cultural transformations in neolithic and bronze age societies in the central northern european plain: Human-environmental interaction concerning bourdieu’s forms of capital… [find](https://scholar.google.com/scholar?q=J%2E+P%2E+Brozio%2C+J%2E+Kneisel%2C+S%2E+Schaefer%2DDi+Maida%2C+J%2E+Laabs%2C+I%2E+Feeser%2C+A%2E+Ribeiro%2C+and+S%2E+Schultrich%2E+Patterns+of+socio%2Deconomic+cultural+transformations+in+neolithic+and+bronze+age+societies+in+the+cen)
8. F. O. Cerezer, C. S. Dambros, M. T. Coelho, F. A. Cassemiro, E. Barreto, J. S. Albert, R. O. Wüest, and C. H. Graham. Accelerated body size evolution in upland environments is correlated with recent speciation in south american freshwater fishes. Nature Communications, 14(1):6070, 2023. [find](https://scholar.google.com/scholar?q=F%2E+O%2E+Cerezer%2C+C%2E+S%2E+Dambros%2C+M%2E+T%2E+Coelho%2C+F%2E+A%2E+Cassemiro%2C+E%2E+Barreto%2C+J%2E+S%2E+Albert%2C+R%2E+O%2E+W%C3%BCest%2C+and+C%2E+H%2E+Graham%2E+Accelerated+body+size+evolution+in+upland+environments+is+correlated+with+recent+s)
9. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374, 2021.
10. M. Dal Corso, W. Kirleis, J. Kneisel, N. Taylor, M. Wieckowska-Lüth, and M. Zanon. How’s Life? Living Conditions in the 2nd and 1st Millennia BCE. Sidestone Press, 2019. [find](https://scholar.google.com/scholar?q=M%2E+Dal+Corso%2C+W%2E+Kirleis%2C+J%2E+Kneisel%2C+N%2E+Taylor%2C+M%2E+Wieckowska%2DL%C3%BCth%2C+and+M%2E+Zanon%2E+How%E2%80%99s+Life%3F+Living+Conditions+in+the+2nd+and+1st+Millennia+BCE%2E+Sidestone+Press%2C+2019%2E)
11. C. Dougherty. Numeracy, literacy and earnings: Evidence from the national longitudinal survey of youth. Economics of education review, 22(5):511–521, 2003. [find](https://scholar.google.com/scholar?q=C%2E+Dougherty%2E+Numeracy%2C+literacy+and+earnings%3A+Evidence+from+the+national+longitudinal+survey+of+youth%2E+Economics+of+education+review%2C+22%285%29%3A511%E2%80%93521%2C+2003%2E)
12. I. Feeser, W. Dörfler, J. Kneisel, M. Hinz, and S. Dreibrodt. Human impact and population dynamics in the neolithic and bronze age: Multi-proxy evidence from north-western central europe. The Holocene, 29(10):1596–1606, 2019. [find](https://scholar.google.com/scholar?q=I%2E+Feeser%2C+W%2E+D%C3%B6rfler%2C+J%2E+Kneisel%2C+M%2E+Hinz%2C+and+S%2E+Dreibrodt%2E+Human+impact+and+population+dynamics+in+the+neolithic+and+bronze+age%3A+Multi%2Dproxy+evidence+from+north%2Dwestern+central+europe%2E+The+Holocene)
13. M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter. Efficient and robust automated machine learning. In NeurIPS, 2015. [find](https://scholar.google.com/scholar?q=M%2E+Feurer%2C+A%2E+Klein%2C+K%2E+Eggensperger%2C+J%2E+Springenberg%2C+M%2E+Blum%2C+and+F%2E+Hutter%2E+Efficient+and+robust+automated+machine+learning%2E+In+NeurIPS%2C+2015%2E)
14. J. Fu, S.-K. Ng, Z. Jiang, and P. Liu. Gptscore: Evaluate as you desire. ArXiv, abs/2302.04166, 2023. URL [https://api.semanticscholar.org/CorpusID:256662188](https://api.semanticscholar.org/CorpusID:256662188). [↗](https://api.semanticscholar.org/CorpusID:256662188)
15. P. Gijsbers, M. L. P. Bueno, S. Coors, E. LeDell, S. Poirier, J. Thomas, B. Bischl, and J. Vanschoren. Amlb: an automl benchmark. ArXiv, abs/2207.12560, 2022. URL [https://api.semanticscholar.org/CorpusID:251066648](https://api.semanticscholar.org/CorpusID:251066648). — [arXiv:2207.12560](https://arxiv.org/abs/2207.12560) [↗](https://api.semanticscholar.org/CorpusID:251066648)
16. D. J. Glass and N. Hall. A brief history of the hypothesis. Cell, 134(3):378–381, 2008. [find](https://scholar.google.com/scholar?q=D%2E+J%2E+Glass+and+N%2E+Hall%2E+A+brief+history+of+the+hypothesis%2E+Cell%2C+134%283%29%3A378%E2%80%93381%2C+2008%2E)
17. R. Heyard and L. Held. Meta-regression to explain shrinkage and heterogeneity in large-scale replication projects. Technical report, Center for Open Science, 2024. [find](https://scholar.google.com/scholar?q=R%2E+Heyard+and+L%2E+Held%2E+Meta%2Dregression+to+explain+shrinkage+and+heterogeneity+in+large%2Dscale+replication+projects%2E+Technical+report%2C+Center+for+Open+Science%2C+2024%2E)
18. H. Jin, F. Chollet, Q. Song, and X. Hu. Autokeras: An automl library for deep learning. J. Mach. Learn. Res., 24:6:1–6:6, 2023. [find](https://scholar.google.com/scholar?q=H%2E+Jin%2C+F%2E+Chollet%2C+Q%2E+Song%2C+and+X%2E+Hu%2E+Autokeras%3A+An+automl+library+for+deep+learning%2E+J%2E+Mach%2E+Learn%2E+Res%2E%2C+24%3A6%3A1%E2%80%936%3A6%2C+2023%2E)
19. J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko, et al. [Highly accurate protein structure prediction with alphafold](https://md.research.elamur.ai/free/en/papers/jumper-2021-highly-accurate-protein-structure-prediction/). Nature, 596(7873):583–589, 2021.
20. E. Kaiser and W. Schier. Time and Materiality: Periodization and Regional Chronologies at the Transition from Bronze to Iron Age in Eurasia (1200-600 BCE). Edited by Elke Kaiser and Wolfram Schier. Verlag Marie Leidorf GmbH, 2021. [find](https://scholar.google.com/scholar?q=E%2E+Kaiser+and+W%2E+Schier%2E+Time+and+Materiality%3A+Periodization+and+Regional+Chronologies+at+the+Transition+from+Bronze+to+Iron+Age+in+Eurasia+%281200%2D600+BCE%29%2E+Edited+by+Elke+Kaiser+and+Wolfram+Schier%2E+Ve)
21. H. Kitano. Artificial intelligence to win the nobel prize and beyond: Creating the engine for scientific discovery. AI magazine, 37(1):39–49, 2016. [find](https://scholar.google.com/scholar?q=H%2E+Kitano%2E+Artificial+intelligence+to+win+the+nobel+prize+and+beyond%3A+Creating+the+engine+for+scientific+discovery%2E+AI+magazine%2C+37%281%29%3A39%E2%80%9349%2C+2016%2E)
22. J. Kneisel. Chronology and transformation. the transition from bronze to iron age in northern europe. Time and materiality: Periodization and regional chronologies at the transition from Bronze to Iron Age in Eurasia (1200–600 BCE), pages 237–263, 2021. [find](https://scholar.google.com/scholar?q=J%2E+Kneisel%2E+Chronology+and+transformation%2E+the+transition+from+bronze+to+iron+age+in+northern+europe%2E+Time+and+materiality%3A+Periodization+and+regional+chronologies+at+the+transition+from+Bronze+to+Iro)
23. P. Langley. Data-driven discovery of physical laws. Cogn. Sci., 5:31–54, 1981. URL [https://api.semanticscholar.org/CorpusID:39694251](https://api.semanticscholar.org/CorpusID:39694251). [↗](https://api.semanticscholar.org/CorpusID:39694251)
24. E. LeDell and S. Poirier. H2O AutoML: Scalable automatic machine learning. In Proceedings of the AutoML Workshop at ICML, 2020. [find](https://scholar.google.com/scholar?q=E%2E+LeDell+and+S%2E+Poirier%2E+H2O+AutoML%3A+Scalable+automatic+machine+learning%2E+In+Proceedings+of+the+AutoML+Workshop+at+ICML%2C+2020%2E)
25. M. Y. Li, E. B. Fox, and N. D. Goodman. Automated statistical model discovery with language models. ArXiv, abs/2402.17879, 2024. URL [https://api.semanticscholar.org/](https://api.semanticscholar.org/) CorpusID:268041863. [↗](https://api.semanticscholar.org/)
26. X. Li, T. Zhang, Y. Dubois, R. Taori, I. Gulrajani, C. Guestrin, P. Liang, and T. B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models. [https://github.com/](https://github.com/) tatsu-lab/alpaca_eval, 2023. [↗](https://github.com/)
27. B. Y. Lin, K. Chandu, F. Brahman, Y. Deng, A. Ravichander, V. Pyatkin, R. L. Bras, and Y. Choi. Wildbench: Benchmarking language models with challenging tasks from real users in the wild, 2024. URL [https://huggingface.co/spaces/allenai/WildBench](https://huggingface.co/spaces/allenai/WildBench). [↗](https://huggingface.co/spaces/allenai/WildBench)
28. X. Liu, Z. Wu, X. Wu, P. Lu, K.-W. Chang, and Y. Feng. Are llms capable of data-based statistical and causal reasoning? benchmarking advanced quantitative reasoning with data. arXiv preprint arXiv:2402.17644, 2024. [find](https://scholar.google.com/scholar?q=X%2E+Liu%2C+Z%2E+Wu%2C+X%2E+Wu%2C+P%2E+Lu%2C+K%2E%2DW%2E+Chang%2C+and+Y%2E+Feng%2E+Are+llms+capable+of+data%2Dbased+statistical+and+causal+reasoning%3F+benchmarking+advanced+quantitative+reasoning+with+data%2E+arXiv+preprint+arXiv%3A240)
29. L. Lorenz. Kommunikationsstrukturen mittelneolithischer Gesellschaften im nordmitteleuropäis-chen Tiefland. Verlag Dr. Rudolf Habelt GmbH, in Kommission, 2018. [find](https://scholar.google.com/scholar?q=L%2E+Lorenz%2E+Kommunikationsstrukturen+mittelneolithischer+Gesellschaften+im+nordmitteleurop%C3%A4is%2Dchen+Tiefland%2E+Verlag+Dr%2E+Rudolf+Habelt+GmbH%2C+in+Kommission%2C+2018%2E)
30. A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, S. Gupta, B. P. Majumder, K. Hermann, S. Welleck, A. Yazdanbakhsh, and P. Clark. Self-refine: Iterative refinement with self-feedback. In Thirty-seventh Conference on Neural Information Proc… [find](https://scholar.google.com/scholar?q=A%2E+Madaan%2C+N%2E+Tandon%2C+P%2E+Gupta%2C+S%2E+Hallinan%2C+L%2E+Gao%2C+S%2E+Wiegreffe%2C+U%2E+Alon%2C+N%2E+Dziri%2C+S%2E+Prabhumoye%2C+Y%2E+Yang%2C+S%2E+Gupta%2C+B%2E+P%2E+Majumder%2C+K%2E+Hermann%2C+S%2E+Welleck%2C+A%2E+Yazdanbakhsh%2C+and+P%2E+Clark%2E+Self%2Drefi)
31. S.-D. Maida et al. Unter hügeln: Bronzezeitliche transformationsprozesse in schleswig-holstein am beispiel des fundplatzes von mang de bargen (bornhöved, kr. segeberg) band 1, 2023. [find](https://scholar.google.com/scholar?q=S%2E%2DD%2E+Maida+et+al%2E+Unter+h%C3%BCgeln%3A+Bronzezeitliche+transformationsprozesse+in+schleswig%2Dholstein+am+beispiel+des+fundplatzes+von+mang+de+bargen+%28bornh%C3%B6ved%2C+kr%2E+segeberg%29+band+1%2C+2023%2E)
32. B. P. Majumder, B. D. Mishra, P. Jansen, O. Tafjord, N. Tandon, L. Zhang, C. Callison-Burch, and P. Clark. [CLIN: A continually learning language agent for rapid task adaptation and generalization](https://md.research.elamur.ai/free/en/papers/majumder-2023-clin-continually-learning-language-agent/). arXiv preprint arXiv:2310.10134, 2023.
33. B. P. Majumder, H. Surana, D. Agarwal, S. Hazra, A. Sabharwal, and P. Clark. [Data-driven discovery with large generative models](https://md.research.elamur.ai/free/en/papers/majumder-2024-data-driven-discovery-large-generative/). ICML, 2024.
34. G. I. P. Ottaviano, G. Peri, and G. C. Wright. Immigration, offshoring, and american jobs. American Economic Review, 103(5):1925–1959, 2013. [find](https://scholar.google.com/scholar?q=G%2E+I%2E+P%2E+Ottaviano%2C+G%2E+Peri%2C+and+G%2E+C%2E+Wright%2E+Immigration%2C+offshoring%2C+and+american+jobs%2E+American+Economic+Review%2C+103%285%29%3A1925%E2%80%931959%2C+2013%2E)
35. L. C. Pal. Impact of education on economic development. Khazanah Pendidikan Islam, 5(1): 10–19, 2023. [find](https://scholar.google.com/scholar?q=L%2E+C%2E+Pal%2E+Impact+of+education+on+economic+development%2E+Khazanah+Pendidikan+Islam%2C+5%281%29%3A+10%E2%80%9319%2C+2023%2E)
36. A. Palmisano, A. Bevan, A. Kabelindde, N. Roberts, and S. Shennan. Long-term demographic trends in prehistoric italy: Climate impacts and regionalised socio-ecological trajectories. Journal of world prehistory, 34(3):381–432, 2021. [find](https://scholar.google.com/scholar?q=A%2E+Palmisano%2C+A%2E+Bevan%2C+A%2E+Kabelindde%2C+N%2E+Roberts%2C+and+S%2E+Shennan%2E+Long%2Dterm+demographic+trends+in+prehistoric+italy%3A+Climate+impacts+and+regionalised+socio%2Decological+trajectories%2E+Journal+of+world+p)
37. E. W. Parkinson, T. R. McLaughlin, C. Esposito, S. Stoddart, and C. Malone. Radiocarbon dated trends and central mediterranean prehistory. Journal of world prehistory, 34:317–379, 2021. [find](https://scholar.google.com/scholar?q=E%2E+W%2E+Parkinson%2C+T%2E+R%2E+McLaughlin%2C+C%2E+Esposito%2C+S%2E+Stoddart%2C+and+C%2E+Malone%2E+Radiocarbon+dated+trends+and+central+mediterranean+prehistory%2E+Journal+of+world+prehistory%2C+34%3A317%E2%80%93379%2C+2021%2E)
38. N. Rambeli, D. A. A. Marikan, J. M. Podivinsky, R. Amiruddin, and I. Ismail. The dynamic impact of government expenditure in education on economic growth. International Journal of Business and Society, 22(3):1487–1507, 2021. [find](https://scholar.google.com/scholar?q=N%2E+Rambeli%2C+D%2E+A%2E+A%2E+Marikan%2C+J%2E+M%2E+Podivinsky%2C+R%2E+Amiruddin%2C+and+I%2E+Ismail%2E+The+dynamic+impact+of+government+expenditure+in+education+on+economic+growth%2E+International+Journal+of+Business+and+Society)
39. M. Riera, J. Pino, L. Sáez, P. Aymerich, and Y. Melero. Effect of introduction pathways on the invasion success of non-native plants along environmental gradients. Biological Invasions, pages 1–20, 2024. [find](https://scholar.google.com/scholar?q=M%2E+Riera%2C+J%2E+Pino%2C+L%2E+S%C3%A1ez%2C+P%2E+Aymerich%2C+and+Y%2E+Melero%2E+Effect+of+introduction+pathways+on+the+invasion+success+of+non%2Dnative+plants+along+environmental+gradients%2E+Biological+Invasions%2C+pages+1%E2%80%9320%2C+20)
40. B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, T. Remez, J. Rapin, et al. [Code llama: Open foundation models for code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/). arXiv preprint arXiv:2308.12950, 2023.
41. T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom. [Toolformer: Language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). Advances in Neural Information Processing Systems, 36, 2024.
42. Z. Shao, F. Wang, Y. Xu, W. Wei, C. Yu, Z. Zhang, D. Yao, G. Jin, X. Cao, G. Cong, C. S. Jensen, and X. Cheng. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. ArXiv, abs/2310.06119, 2023. — [arXiv:2310.06119](https://arxiv.org/abs/2310.06119) [find](https://scholar.google.com/scholar?q=Z%2E+Shao%2C+F%2E+Wang%2C+Y%2E+Xu%2C+W%2E+Wei%2C+C%2E+Yu%2C+Z%2E+Zhang%2C+D%2E+Yao%2C+G%2E+Jin%2C+X%2E+Cao%2C+G%2E+Cong%2C+C%2E+S%2E+Jensen%2C+and+X%2E+Cheng%2E+Exploring+progress+in+multivariate+time+series+forecasting%3A+Comprehensive+benchmarking+an)
43. S. Shashidhar, A. Chinta, V. Sahai, Z. Wang, and H. Ji. Democratizing llms: An exploration of cost-performance trade-offs in self-refined open-source models. ArXiv, abs/2310.07611, 2023. URL [https://api.semanticscholar.org/CorpusID:263834891](https://api.semanticscholar.org/CorpusID:263834891). [↗](https://api.semanticscholar.org/CorpusID:263834891)
44. N. Shinn, F. Cassano, B. Labash, A. Gopinath, K. Narasimhan, and S. Yao. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). In NeurIPS, 2023. URL [https://api.semanticscholar.org/CorpusID:258833055](https://api.semanticscholar.org/CorpusID:258833055). [↗](https://api.semanticscholar.org/CorpusID:258833055)
45. P. K. Smith, B. Bogin, and D. Bishai. Are time preference and body mass index associated?: Evidence from the national longitudinal survey of youth. Economics & Human Biology, 3(2): 259–270, 2005. [find](https://scholar.google.com/scholar?q=P%2E+K%2E+Smith%2C+B%2E+Bogin%2C+and+D%2E+Bishai%2E+Are+time+preference+and+body+mass+index+associated%3F%3A+Evidence+from+the+national+longitudinal+survey+of+youth%2E+Economics+%26+Human+Biology%2C+3%282%29%3A+259%E2%80%93270%2C+2005%2E)
46. C. Sommerfeld. Gerätegeld Sichel: studien zur monetären Struktur bronzezeitlicher Horte im nördlichen Mitteleuropa, volume 19. Walter de Gruyter, 2013. [find](https://scholar.google.com/scholar?q=C%2E+Sommerfeld%2E+Ger%C3%A4tegeld+Sichel%3A+studien+zur+monet%C3%A4ren+Struktur+bronzezeitlicher+Horte+im+n%C3%B6rdlichen+Mitteleuropa%2C+volume+19%2E+Walter+de+Gruyter%2C+2013%2E)
47. W. H. Thompson and S. Skau. On the scope of scientific hypotheses. Royal Society Open Science, 10(8):230607, 2023. [find](https://scholar.google.com/scholar?q=W%2E+H%2E+Thompson+and+S%2E+Skau%2E+On+the+scope+of+scientific+hypotheses%2E+Royal+Society+Open+Science%2C+10%288%29%3A230607%2C+2023%2E)
48. H. Weatherly, K. Lopez, and C. Tierra. The impact of education on gdp per capita. 2022. [find](https://scholar.google.com/scholar?q=H%2E+Weatherly%2C+K%2E+Lopez%2C+and+C%2E+Tierra%2E+The+impact+of+education+on+gdp+per+capita%2E+2022%2E)
49. Wes McKinney. [Data Structures for Statistical Computing in Python](https://md.research.elamur.ai/free/en/papers/mckinney-2010-data-structures-statistical-computing-python/). In Stéfan van der Walt and Jarrod Millman, editors, Proceedings of the 9th Python in Science Conference, pages 56 – 61, 2010. doi: 10.25080/Majora-92bf1922-00a.
50. Z. Yang, X. Liu, T. Li, D. Wu, J. Wang, Y. Zhao, and H. Han. A systematic literature review of methods and datasets for anomaly-based network intrusion detection. Comput. Secur., 116: 102675, 2022. [find](https://scholar.google.com/scholar?q=Z%2E+Yang%2C+X%2E+Liu%2C+T%2E+Li%2C+D%2E+Wu%2C+J%2E+Wang%2C+Y%2E+Zhao%2C+and+H%2E+Han%2E+A+systematic+literature+review+of+methods+and+datasets+for+anomaly%2Dbased+network+intrusion+detection%2E+Comput%2E+Secur%2E%2C+116%3A+102675%2C+2022%2E)
51. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao. [ReAct: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In ICLR, 2023. URL [https://openreview.net/](https://openreview.net/) forum?id=WE_vluYUL-X. [↗](https://openreview.net/)
52. Z. Yuan, J. Liu, Q. Zi, M. Liu, X. Peng, and Y. Lou. Evaluating instruction-tuned large language models on code comprehension and generation. ArXiv, abs/2308.01240, 2023. URL [https://api.semanticscholar.org/CorpusID:260379087](https://api.semanticscholar.org/CorpusID:260379087). — [arXiv:2308.01240](https://arxiv.org/abs/2308.01240) [↗](https://api.semanticscholar.org/CorpusID:260379087)
53. K. Zaw, D. Hamilton, and W. Darity. Race, wealth and incarceration: Results from the national longitudinal survey of youth. Race and Social Problems, 8:103–115, 2016. [find](https://scholar.google.com/scholar?q=K%2E+Zaw%2C+D%2E+Hamilton%2C+and+W%2E+Darity%2E+Race%2C+wealth+and+incarceration%3A+Results+from+the+national+longitudinal+survey+of+youth%2E+Race+and+Social+Problems%2C+8%3A103%E2%80%93115%2C+2016%2E)
54. Z. Zeng, J. Yu, T. Gao, Y. Meng, T. Goyal, and D. Chen. Evaluating large language models at evaluating instruction following. ArXiv, abs/2310.07641, 2023. URL [https://api.semanticscholar.org/CorpusID:263834884](https://api.semanticscholar.org/CorpusID:263834884). [↗](https://api.semanticscholar.org/CorpusID:263834884)
55. S. Zhang, C. Gong, L. Wu, X. Liu, and M. Zhou. [AutoML-GPT: Automatic machine learning with GPT](https://md.research.elamur.ai/free/en/papers/zhang-2023-automl-gpt-automatic-machine-learning/). ArXiv, abs/2305.02499, 2023.

*Updated 2 November 2026*