---
title: "Large Language Models are Zero Shot Hypothesis Proposers"
description: "The paper examines whether large language models can formulate new scientific hypotheses without… 44 references, cited by 40 papers here."
authors: ["Biqing Qi", "Kaiyan Zhang", "Haoxiang Li", "Kai Tian", "Sihang Zeng", "Zhang-Ren Chen", "Bowen Zhou"]
year: 2023
venue: "arXiv:2311.05965"
source: "https://arxiv.org/abs/2311.05965"
doi: "10.48550/arxiv.2311.05965"
arxiv: "2311.05965"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qi-2023-large-language-models-zero-shot/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qi-2023-large-language-models-zero-shot/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/11/) / Large Language Models are Zero Shot Hypothesis Proposers

# Large Language Models are Zero Shot Hypothesis Proposers

**Authors:** [Biqing Qi](https://md.research.elamur.ai/free/en/authors/biqing-qi/), [Kaiyan Zhang](https://md.research.elamur.ai/free/en/authors/kaiyan-zhang/), [Haoxiang Li](https://md.research.elamur.ai/free/en/authors/haoxiang-li/), [Kai Tian](https://md.research.elamur.ai/free/en/authors/kai-tian/), [Sihang Zeng](https://md.research.elamur.ai/free/en/authors/sihang-zeng/), [Zhang-Ren Chen](https://md.research.elamur.ai/free/en/authors/zhang-ren-chen/) and [Bowen Zhou](https://md.research.elamur.ai/free/en/authors/bowen-zhou/) · **Year:** 2023 · **Venue:** arXiv:2311.05965 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2311.05965) · [PDF](https://arxiv.org/pdf/2311.05965) · [DOI 10.48550/arxiv.2311.05965](https://doi.org/10.48550/arxiv.2311.05965) · [arXiv 2311.05965](https://arxiv.org/abs/2311.05965) · [OpenAlex](https://openalex.org/W4388650906)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [40](https://md.research.elamur.ai/free/en/papers/qi-2023-large-language-models-zero-shot/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper examines whether large language models can formulate new scientific hypotheses without being trained on them, a step in research that has received far less study than hypothesis validation. The authors built a biomedical dataset of background-knowledge and hypothesis pairs drawn from literature published between January 2000 and September 2023. They split it by publication date to control what models could have seen. Training data predates January 2023, and an unseen test set comes from papers published in August 2023, after the evaluated models' training cutoffs. Closed and open-source instruction-tuned LLMs were tested in zero-shot, few-shot and fine-tuned settings. The authors also built a role-based multi-agent framework with PubMed search. Outputs were scored with four criteria, using both ChatGPT-based and human evaluation.

The models produced hypotheses that were absent from their training data but later supported by the test literature. Fine-tuned models performed comparably to ChatGPT, and in some cases better. The authors argue that greater uncertainty, for example from multi-agent interaction, broadens the space of candidate hypotheses. This makes LLMs credible assistants for scientific discovery.

## Key points
- The dataset is temporally partitioned into training, seen and unseen test sets. The unseen set uses August 2023 publications to reduce data-leakage concerns.
- Generated hypotheses were rated on four dimensions: Novelty, Relevance, Significance and Verifiability. Evaluation used ChatGPT/GPT-4 as well as human judges, and the authors report that the two correlate.
- In one example, a fine-tuned 65B LLaMA proposed a sex-dependent link between metformin use and esophageal cancer risk in T2DM patients. A paper published in August 2023 supports this hypothesis.
- The multi-agent system assigns four roles: Analyst, Engineer, Scientist and Critic. The Engineer queries PubMed, with results restricted to papers published before January 2023. Tool use was implemented via ReAct and OpenAI function calling.
- On the unseen set, a single agent scored 1.92 on the GPT-4 average. Multi-agent setups scored about 2.07–2.09. Tool use alone lowered the score to 1.49–1.56, indicating that external retrieval adds little by itself.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 422 works · 213,539 citations
- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 439 works · 94,783 citations
- [First Affiliated Hospital of Nanchang University](https://md.research.elamur.ai/free/en/hospitals/first-affiliated-hospital-of-nanchang-university/) — Hospital, Nanchang, China · 2 works · 64 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,621 works · [🧬 Biochemistry, Genetics and Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/) · 273 works · [🧬 Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/molecular-biology/) · 270 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works

## Cite

Cite as: Biqing Qi, Kaiyan Zhang, Haoxiang Li, Kai Tian, Sihang Zeng, Zhang-Ren Chen and Bowen Zhou. 2023. Large Language Models are Zero Shot Hypothesis Proposers. arXiv:2311.05965.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{qi2023large,
  title = {{Large Language Models are Zero Shot Hypothesis Proposers}},
  author = {Biqing Qi, Kaiyan Zhang, Haoxiang Li, Kai Tian, Sihang Zeng, Zhang-Ren Chen and Bowen Zhou},
  year = {2023},
  doi = {10.48550/arxiv.2311.05965},
  eprint = {2311.05965},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2311.05965}
}
```

- [← previous DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition](https://md.research.elamur.ai/free/en/papers/chan-2023-discoprompt-path-prediction-prompt-tuning/)
- [next → Lost in Translation: Large Language Models in Non-English Content Analysis](https://md.research.elamur.ai/free/en/papers/nicholas-2023-lost-translation-large-language-models/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/11/)

## References

44 references, 32 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Daniil A Boiko, Robert MacKnight, and Gabe Gomes. [Emergent autonomous scientific research capabilities of large language models](https://md.research.elamur.ai/free/en/papers/boiko-2023-emergent-autonomous-scientific-research/). arXiv preprint arXiv:2304.05332, 2023.
2. Andres M. Bran, Sam Cox, Andrew D. White, and Philippe Schwaller. [ChemCrow: Augmenting large-language models with chemistry tools](https://md.research.elamur.ai/free/en/papers/bran-2023-chemcrow-augmenting-large-language/), June 2023. arXiv:2304.05376.
3. Boxi Cao, Hongyu Lin, Xianpei Han, and Le Sun. [The Life Cycle of Knowledge in Big Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/cao-2023-life-cycle-knowledge-big-language/), March 2023. arXiv:2303.07616 [cs].
4. Zhuo Chang, Jing Zhang, Yilun Liu, Huajian Gao, and Guang-Kui Xu. New Mechanical Markers for Tracking the Progression of Myocardial Infarction. Nano Letters, 23(16):7350–7357, August 2023. [find](https://scholar.google.com/scholar?q=Zhuo+Chang%2C+Jing+Zhang%2C+Yilun+Liu%2C+Huajian+Gao%2C+and+Guang%2DKui+Xu%2E+New+Mechanical+Markers+for+Tracking+the+Progression+of+Myocardial+Infarction%2E+Nano+Letters%2C+23%2816%29%3A7350%E2%80%937357%2C+August+2023%2E)
5. Xiaoyu Chen, Shenao Zhang, Pushi Zhang, Li Zhao, and Jianyu Chen. Asking Before Action: Gather Information in Embodied Decision Making with Language Models, May 2023. arXiv:2305.15695 [cs]. — [arXiv:2305.15695](https://arxiv.org/abs/2305.15695) [find](https://scholar.google.com/scholar?q=Xiaoyu+Chen%2C+Shenao+Zhang%2C+Pushi+Zhang%2C+Li+Zhao%2C+and+Jianyu+Chen%2E+Asking+Before+Action%3A+Gather+Information+in+Embodied+Decision+Making+with+Language+Models%2C+May+2023%2E+arXiv%3A2305%2E15695+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2)
6. Yulin Chen, Ning Ding, Hai-Tao Zheng, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Empowering Private Tutoring by Chaining Large Language Models, September 2023. arXiv:2309.08112 [cs] version: 1. — [arXiv:2309.08112](https://arxiv.org/abs/2309.08112) [find](https://scholar.google.com/scholar?q=Yulin+Chen%2C+Ning+Ding%2C+Hai%2DTao+Zheng%2C+Zhiyuan+Liu%2C+Maosong+Sun%2C+and+Bowen+Zhou%2E+Empowering+Private+Tutoring+by+Chaining+Large+Language+Models%2C+September+2023%2E+arXiv%3A2309%2E08112+%5Bcs%5D+version%3A+1%2E+%E2%80%94+%5BarXi)
7. Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://md.research.elamur.ai/free/en/papers/fu-2023-improving-language-model-negotiation-self/), May 2023. arXiv:2305.10142 [cs].
8. Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://md.research.elamur.ai/free/en/papers/gilardi-2023-chatgpt-outperforms-crowd-workers-text/). Proceedings of the National Academy of Sciences, 120(30):e2305016120, July 2023. arXiv:2303.15056 [cs].
9. Ran Gong, Qiuyuan Huang, Xiaojian Ma, Hoi Vo, Zane Durante, Yusuke Noda, Zilong Zheng, Song-Chun Zhu, Demetri Terzopoulos, Li Fei-Fei, and Jianfeng Gao. MindAgent: Emergent Gaming Interaction, September 2023. arXiv:2309.09971 [cs] version: 1. [find](https://scholar.google.com/scholar?q=Ran+Gong%2C+Qiuyuan+Huang%2C+Xiaojian+Ma%2C+Hoi+Vo%2C+Zane+Durante%2C+Yusuke+Noda%2C+Zilong+Zheng%2C+Song%2DChun+Zhu%2C+Demetri+Terzopoulos%2C+Li+Fei%2DFei%2C+and+Jianfeng+Gao%2E+MindAgent%3A+Emergent+Gaming+Interaction%2C+Septemb)
10. Tianyu Han, Lisa C Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander Löser, Daniel Truhn, and Keno K Bressem. Medalpaca–an open-source collection of medical conversational ai models and training data. arXiv preprint arXiv:2304.[08247, 2023](https://md.research.elamur.ai/free/en/papers/han-2023-medalpaca-open-source-collection-medical/).
11. Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, and Chenglin Wu. MetaGPT: Meta Programming for Multi-Agent Collaborative Framework, August 2023. [arXiv:2308.00352 cs](https://md.research.elamur.ai/free/en/papers/hong-2023-metagpt-meta-programming-multi/).
12. Chenxu Hu, Jie Fu, Chenzhuang Du, Simian Luo, Junbo Zhao, and Hang Zhao. ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory, June 2023. arXiv:2306.03901 [cs]. — [arXiv:2306.03901](https://arxiv.org/abs/2306.03901) [find](https://scholar.google.com/scholar?q=Chenxu+Hu%2C+Jie+Fu%2C+Chenzhuang+Du%2C+Simian+Luo%2C+Junbo+Zhao%2C+and+Hang+Zhao%2E+ChatDB%3A+Augmenting+LLMs+with+Databases+as+Their+Symbolic+Memory%2C+June+2023%2E+arXiv%3A2306%2E03901+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2306%2E03901%5D%28https%3A%2F)
13. Moksh Jain, Tristan Deleu, Jason Hartford, Cheng-Hao Liu, Alex Hernandez-Garcia, and Yoshua Bengio. Gflownets for ai-driven scientific discovery. Digital Discovery, 2(3):557–577, 2023. [find](https://scholar.google.com/scholar?q=Moksh+Jain%2C+Tristan+Deleu%2C+Jason+Hartford%2C+Cheng%2DHao+Liu%2C+Alex+Hernandez%2DGarcia%2C+and+Yoshua+Bengio%2E+Gflownets+for+ai%2Ddriven+scientific+discovery%2E+Digital+Discovery%2C+2%283%29%3A557%E2%80%93577%2C+2023%2E)
14. Jikun Kang, Romain Laroche, Xindi Yuan, Adam Trischler, Xue Liu, and Jie Fu. Think Before You Act: Decision Transformers with Internal Working Memory, May 2023. 0 citations (Semantic Scholar/arXiv) [2023-05-30] arXiv:2305.16338 [cs]. — [arXiv:2305.16338](https://arxiv.org/abs/2305.16338) [find](https://scholar.google.com/scholar?q=Jikun+Kang%2C+Romain+Laroche%2C+Xindi+Yuan%2C+Adam+Trischler%2C+Xue+Liu%2C+and+Jie+Fu%2E+Think+Before+You+Act%3A+Decision+Transformers+with+Internal+Working+Memory%2C+May+2023%2E+0+citations+%28Semantic+Scholar%2FarXiv%29+%5B2)
15. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society, March 2023. [arXiv:2303.17760 cs](https://md.research.elamur.ai/free/en/papers/li-2023-camel-communicative-agents-mind/).
16. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://md.research.elamur.ai/free/en/papers/liu-2023-g-eval-nlg-evaluation-gpt/), May 2023. arXiv:2303.16634 [cs].
17. Philipp Maas, Frank Carey, Chris Wheeler, Edward Saatchi, Pete Billington, and Jessica Yaffa Shamash. SHOW-1 and Showrunner Agents in Multi-Agent Simulations. arXiv preprint, 2023. [find](https://scholar.google.com/scholar?q=Philipp+Maas%2C+Frank+Carey%2C+Chris+Wheeler%2C+Edward+Saatchi%2C+Pete+Billington%2C+and+Jessica+Yaffa+Shamash%2E+SHOW%2D1+and+Showrunner+Agents+in+Multi%2DAgent+Simulations%2E+arXiv+preprint%2C+2023%2E)
18. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. WebGPT: Browser-assisted qu… — [WebGPT: Browser-assisted question-answering with human feedback](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/)
19. Aaron Parisi, Yao Zhao, and Noah Fiedel. [TALM: Tool Augmented Language Models](https://md.research.elamur.ai/free/en/papers/parisi-2022-talm-tool-augmented-language-models/), May 2022. arXiv:2205.12255 [cs].
20. Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative Agents: Interactive Simulacra of Human Behavior, August 2023. arXiv:2304.03442 [cs]. [find](https://scholar.google.com/scholar?q=Joon+Sung+Park%2C+Joseph+C%2E+O%E2%80%99Brien%2C+Carrie+J%2E+Cai%2C+Meredith+Ringel+Morris%2C+Percy+Liang%2C+and+Michael+S%2E+Bernstein%2E+Generative+Agents%3A+Interactive+Simulacra+of+Human+Behavior%2C+August+2023%2E+arXiv%3A2304%2E034)
21. Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. [Gorilla: Large Language Model Connected with Massive APIs](https://md.research.elamur.ai/free/en/papers/patil-2023-gorilla-large-language-model-connected/), May 2023. arXiv:2305.15334 [cs].
22. Chen Qian, Xin Cong, Wei Liu, Cheng Yang, Weize Chen, Yusheng Su, Yufan Dang, Jiahao Li, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. Communicative Agents for Software Development, August 2023. [arXiv:2307.07924 cs](https://md.research.elamur.ai/free/en/papers/qian-2023-chatdev-communicative-agents-software/).
23. Vipula Rawte, Amit Sheth, and Amitava Das. [A Survey of Hallucination in Large Foundation Models](https://md.research.elamur.ai/free/en/papers/rawte-2023-survey-hallucination-large-foundation-models/), September 2023. arXiv:2309.05922 [cs].
24. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. [Toolformer: Language Models Can Teach Themselves to Use Tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/), February 2023. arXiv:2302.04761.
25. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/), May 2023. arXiv:2303.17580 [cs].
26. Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. [Reflexion: Language Agents with Verbal Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/), June 2023. arXiv:2303.11366 [cs].
27. Yifan Song, Weimin Xiong, Dawei Zhu, Cheng Li, Ke Wang, Ye Tian, and Sujian Li. RestGPT: Connecting Large Language Models with Real-World Applications via RESTful APIs, June 2023. [arXiv:2306.06624 cs](https://md.research.elamur.ai/free/en/papers/song-2023-restgpt-connecting-large-language-models/).
28. Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. [Galactica: A large language model for science](https://md.research.elamur.ai/free/en/papers/taylor-2022-galactica-large-language-model-science/). arXiv preprint arXiv:2211.09085, 2022.
29. Karthik Valmeekam, Matthew Marquez, Sarath Sreedharan, and Subbarao Kambhampati. On the Planning Abilities of Large Language Models – A Critical Investigation, May 2023. [arXiv:2305.15771 cs](https://md.research.elamur.ai/free/en/papers/valmeekam-2023-planning-abilities-large-language-models/).
30. Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, et al. Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47–60, 2023. [find](https://scholar.google.com/scholar?q=Hanchen+Wang%2C+Tianfan+Fu%2C+Yuanqi+Du%2C+Wenhao+Gao%2C+Kexin+Huang%2C+Ziming+Liu%2C+Payal+Chandak%2C+Shengchao+Liu%2C+Peter+Van+Katwyk%2C+Andreea+Deac%2C+et+al%2E+Scientific+discovery+in+the+age+of+artificial+intelligenc)
31. Qingyun Wang, Doug Downey, Heng Ji, and Tom Hope. Learning to generate novel scientific directions with contextualized literature-based discovery. arXiv preprint arXiv:2305.[14259, 2023](https://md.research.elamur.ai/free/en/papers/wang-2023-scimon-scientific-inspiration-machines/).
32. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560, 2022. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2Dinstruct%3A+Aligning+language+model+with+self+generated+instructions%2E+arXiv+preprint+)
33. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/), January 2023. arXiv:2201.11903 [cs].
34. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in Neural Information Processing Systems, 35:24824–24837, 2022.
35. Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. [Pmc-llama: Towards building open-source language models for medicine](https://md.research.elamur.ai/free/en/papers/wu-2023-pmc-llama-towards-building-open/), 2023.
36. Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang,… — [The Rise and Potential of Large Language Model Based Agents: A Survey](https://md.research.elamur.ai/free/en/papers/xi-2023-rise-potential-large-language/)
37. Hui Yang, Sifu Yue, and Yunzhong He. [Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions](https://md.research.elamur.ai/free/en/papers/yang-2023-auto-gpt-online-decision-making/), June 2023. arXiv:2306.02224.
38. Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. [Large language models for automated open-domain scientific hypotheses discovery](https://md.research.elamur.ai/free/en/papers/yang-2023-large-language-models-automated/). arXiv preprint arXiv:2309.02726, 2023.
39. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of Thoughts: Deliberate Problem Solving with Large Language Models, May 2023. arXiv:2305.10601 [cs]. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Thomas+L%2E+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+Tree+of+Thoughts%3A+Deliberate+Problem+Solving+with+Large+Language+Models%2C+May+2023%2E+arXiv%3A2305%2E1)
40. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. [ReAct: Synergizing Reasoning and Acting in Language Models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/), March 2023. arXiv:2210.03629.
41. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models, September 2023. [arXiv:2309.01219 cs](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/).
42. Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, and Jacob Steinhardt. [Goal driven discovery of distributional differences via language descriptions](https://md.research.elamur.ai/free/en/papers/zhong-2023-goal-driven-discovery-distributional-differences/). arXiv preprint arXiv:2302.14233, 2023.
43. Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, Yu Qiao, Zhaoxiang Zhang, and Jifeng Dai. [Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory](https://md.research.elamur.ai/free/en/papers/zhu-2023-ghost-minecraft-generally-capable-agents/), J…
44. Xuekai Zhu, Biqing Qi, Kaiyan Zhang, Xingwei Long, and Bowen Zhou. Pad: Program-aided distillation specializes large models in reasoning. arXiv preprint arXiv:2305.[13888, 2023](https://md.research.elamur.ai/free/en/papers/zhu-2024-pad-program-aided-distillation-can/).

*Updated 3 November 2026*