---
title: "AutoAgents: A Framework for Automatic Agent Generation (2023)"
description: "Most LLM-based multi-agent systems use agents whose roles people define in advance. 41 references, cited by 65 papers here."
authors: ["Guangyao Chen", "Siwei Dong", "Yu Shu", "Ge Zhang", "Jaward Sesay", "Borje F. Karlsson", "Jie Fu", "Yemin Shi"]
year: 2023
venue: "arXiv:2309.17288"
source: "https://arxiv.org/abs/2309.17288"
doi: "10.48550/arxiv.2309.17288"
arxiv: "2309.17288"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chen-2023-autoagents-framework-automatic-agent-generation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chen-2023-autoagents-framework-automatic-agent-generation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/) / AutoAgents: A Framework for Automatic Agent Generation

# AutoAgents: A Framework for Automatic Agent Generation

**Authors:** Guangyao Chen, Siwei Dong, Yu Shu, [Ge Zhang](https://md.research.elamur.ai/free/en/authors/ge-zhang/), Jaward Sesay, [Borje F. Karlsson](https://md.research.elamur.ai/free/en/authors/borje-f-karlsson/), [Jie Fu](https://md.research.elamur.ai/free/en/authors/jie-fu/) and Yemin Shi · **Year:** 2023 · **Venue:** arXiv:2309.17288 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2309.17288) · [PDF](https://arxiv.org/pdf/2309.17288) · [DOI 10.48550/arxiv.2309.17288](https://doi.org/10.48550/arxiv.2309.17288) · [arXiv 2309.17288](https://arxiv.org/abs/2309.17288) · [OpenAlex](https://openalex.org/W4401024643)

**Citations:** 42 citations (OpenAlex, 2026-10-14) · [65](https://md.research.elamur.ai/free/en/papers/chen-2023-autoagents-framework-automatic-agent-generation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most LLM-based multi-agent systems use agents whose roles people define in advance. This makes them hard to adapt to new task types and expensive to scale. AutoAgents removes that dependence by assembling a task-specific team of expert agents automatically.

The framework runs in two phases. In the Drafting Stage, three fixed helper agents (Planner, Agent Observer and Plan Observer) discuss the input task. Together they decide which specialists to create and write an execution plan. In the Execution Stage, the generated specialists carry out the plan. Each one improves its own output through self-refinement, and they pool expertise through collaborative refinement. An Action Observer coordinates the team and uses a dynamic memory of earlier actions.

On two challenging tasks, including trivia creative writing, AutoAgents produced more coherent and accurate results than other agent-generation frameworks such as AgentVerse and SPP. A Tetris software case study shows practical use. The work suggests that fitting roles to each task and structuring team cooperation help LLMs with knowledge-heavy, cross-domain problems.

## Key points
- The pipeline has a Drafting Stage, with Planner, Agent Observer and Plan Observer, and an Execution Stage, coordinated by an Action Observer with dynamic long-term memory.
- Unlike AutoGPT, BabyAGI, MetaGPT, Camel or AutoGen, it generates an unlimited number of agents dynamically. Compared with SPP and AgentVerse, it adds both self-refinement and collaborative refinement actions.
- Ablations on trivia creative writing (Table 4) measured these drops:
  - about 3% without self-refinement
  - about 2% without collaborative refinement
  - about 1% without dynamic memory
- In the case study, the system built a Python Tetris game by generating game design, UI design, programming and debugging experts.
- The code is available at github.com/Link-AGI/AutoAgents (arXiv:2309.17288).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) — University, Beijing, China · 315 works · 43,267 citations
- [Hong Kong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/hong-kong-university-of-science-and-technology/) — University, Hong Kong, China · 170 works · 26,341 citations
- [University of Waterloo](https://md.research.elamur.ai/free/en/universities/university-of-waterloo/) — University, Waterloo, Canada · 80 works · 15,030 citations
- [Beijing Academy of Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/beijing-academy-of-artificial-intelligence/) — Research institute, Beijing, China · 46 works · 8,682 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works

## Cite

Cite as: Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Borje F. Karlsson, Jie Fu and Yemin Shi. 2023. AutoAgents: A Framework for Automatic Agent Generation. arXiv:2309.17288.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{chen2023autoagents,
  title = {{AutoAgents: A Framework for Automatic Agent Generation}},
  author = {Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Borje F. Karlsson, Jie Fu and Yemin Shi},
  year = {2023},
  doi = {10.48550/arxiv.2309.17288},
  eprint = {2309.17288},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.17288}
}
```

- [← previous AppAgent: Multimodal Agents as Smartphone Users](https://md.research.elamur.ai/free/en/papers/zhang-2023-appagent-multimodal-agents-smartphone-users/)
- [next → Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models](https://md.research.elamur.ai/free/en/papers/arora-2023-have-llms-advanced-enough-challenging/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/)

## References

41 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Brigid Barron. Achieving coordination in collaborative problem-solving groups. The journal of the learning sciences, 9(4):403–436, 2000. 1 [find](https://scholar.google.com/scholar?q=Brigid+Barron%2E+Achieving+coordination+in+collaborative+problem%2Dsolving+groups%2E+The+journal+of+the+learning+sciences%2C+9%284%29%3A403%E2%80%93436%2C+2000%2E+1)
2. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. [Sparks of artificial general intelligence: Early experiments with gpt-4](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/). arXiv preprint arXiv:2303.12712, 2023. 1
3. Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. [Chateval: Towards better llm-based evaluators through multi-agent debate](https://md.research.elamur.ai/free/en/papers/chan-2023-chateval-better-llm-based/). arXiv preprint arXiv:2308.07201, 2023. 6
4. Guangyao Chen, Peixi Peng, Yangru Huang, Mengyue Geng, and Yonghong Tian. Adaptive discovering and merging for incremental novel class discovery. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 11276–11284, 2024. 21 [find](https://scholar.google.com/scholar?q=Guangyao+Chen%2C+Peixi+Peng%2C+Yangru+Huang%2C+Mengyue+Geng%2C+and+Yonghong+Tian%2E+Adaptive+discovering+and+merging+for+incremental+novel+class+discovery%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+In)
5. Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, et al. [Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors](https://md.research.elamur.ai/free/en/papers/chen-2023-agentverse-facilitating-multi-agent/) in agents. arXiv preprint arXiv:2308.10848, 2023. 2, 4, 5, 6, 17
6. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128, 2023. 10 [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Maxwell+Lin%2C+Nathanael+Sch%C3%A4rli%2C+and+Denny+Zhou%2E+Teaching+large+language+models+to+self%2Ddebug%2E+arXiv+preprint+arXiv%3A2304%2E05128%2C+2023%2E+10)
7. Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. [Improving factuality and reasoning in language models through multiagent debate](https://md.research.elamur.ai/free/en/papers/du-2023-improving-factuality-reasoning-language-models/). arXiv preprint arXiv:2305.14325, 2023. 1, 3
8. Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. [Improving language model negotiation with self-play and in-context learning from ai feedback](https://md.research.elamur.ai/free/en/papers/fu-2023-improving-language-model-negotiation-self/). arXiv preprint arXiv:2305.10142, 2023. 2
9. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. [Critic: Large language models can self-correct with tool-interactive critiquing](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/), 2023. 10
10. Significant Gravitas. Auto-gpt: An autonomous gpt-4 experiment, 2023. URL [https://github.com/Significant-Gravitas/Auto-GPT](https://github.com/Significant-Gravitas/Auto-GPT), 2023. 2, 5, 7 [↗](https://github.com/Significant-Gravitas/Auto-GPT)
11. Rui Hao, Linmei Hu, Weijian Qi, Qingliu Wu, Yirui Zhang, and Liqiang Nie. Chatllm network: More brains, more intelligence. arXiv preprint arXiv:2304.12998, 2023. 1, 3 — [arXiv:2304.12998](https://arxiv.org/abs/2304.12998) [find](https://scholar.google.com/scholar?q=Rui+Hao%2C+Linmei+Hu%2C+Weijian+Qi%2C+Qingliu+Wu%2C+Yirui+Zhang%2C+and+Liqiang+Nie%2E+Chatllm+network%3A+More+brains%2C+more+intelligence%2E+arXiv+preprint+arXiv%3A2304%2E12998%2C+2023%2E+1%2C+3+%E2%80%94+%5BarXiv%3A2304%2E12998%5D%28https%3A%2F%2Farxi)
12. Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.[00352, 2023](https://md.research.elamur.ai/free/en/papers/hong-2023-metagpt-meta-programming-multi/). 2, 3, 5, 6
13. Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608, 2022. 10 [find](https://scholar.google.com/scholar?q=Wenlong+Huang%2C+Fei+Xia%2C+Ted+Xiao%2C+Harris+Chan%2C+Jacky+Liang%2C+Pete+Florence%2C+Andy+Zeng%2C+Jonathan+Tompson%2C+Igor+Mordatch%2C+Yevgen+Chebotar%2C+et+al%2E+Inner+monologue%3A+Embodied+reasoning+through+planning+with)
14. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1601–1611. Association for Computational Linguisti…
15. Martin Josifoski, Lars Klein, Maxime Peyrard, Yifei Li, Saibo Geng, Julian Paul Schnitzler, Yuxing Yao, Jiheng Wei, Debjit Paul, and Robert West. Flows: Building blocks of reasoning and collaborating ai. arXiv preprint arXiv:2308.01285, 2023. 3 — [arXiv:2308.01285](https://arxiv.org/abs/2308.01285) [find](https://scholar.google.com/scholar?q=Martin+Josifoski%2C+Lars+Klein%2C+Maxime+Peyrard%2C+Yifei+Li%2C+Saibo+Geng%2C+Julian+Paul+Schnitzler%2C+Yuxing+Yao%2C+Jiheng+Wei%2C+Debjit+Paul%2C+and+Robert+West%2E+Flows%3A+Building+blocks+of+reasoning+and+collaborating+)
16. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for" mind" exploration of large scale language model society. arXiv preprint arXiv:2303.[17760, 2023](https://md.research.elamur.ai/free/en/papers/li-2023-camel-communicative-agents-mind/). 2, 3
17. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. [Encouraging divergent thinking in large language models through multi-agent debate](https://md.research.elamur.ai/free/en/papers/liang-2023-encouraging-divergent-thinking-large-language/). arXiv preprint arXiv:2305.19118, 2023. 1, 3
18. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651, 2023. 10 [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+et+al%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Dfeedbac)
19. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. [On faithfulness and factuality in abstractive summarization](https://md.research.elamur.ai/free/en/papers/maynez-2020-faithfulness-factuality-abstractive-summarization/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online, July 2020. Association for Computational Linguistics. 1
20. Y Nakajima. Task-driven autonomous agent utilizing gpt-4, pinecone, and langchain for diverse applications. See [https://yoheinakajima.com/task-driven-autonomous-agent-utilizing-gpt-4-pinecone-and-langchain-for-diverse-applications](https://yoheinakajima.com/task-driven-autonomous-agent-utilizing-gpt-4-pinecone-and-langchain-for-diverse-applications) (accessed 18 April 2023), 2023. 2, 3 [↗](https://yoheinakajima.com/task-driven-autonomous-agent-utilizing-gpt-4-pinecone-and-langchain-for-diverse-applications)
21. Laurie Miller Nelson. Collaborative problem solving. In Instructional-design theories and models, pages 241–267. Routledge, 2013. 1 [find](https://scholar.google.com/scholar?q=Laurie+Miller+Nelson%2E+Collaborative+problem+solving%2E+In+Instructional%2Ddesign+theories+and+models%2C+pages+241%E2%80%93267%2E+Routledge%2C+2013%2E+1)
22. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), 2023. 1
23. Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442, 2023. 2 [find](https://scholar.google.com/scholar?q=Joon+Sung+Park%2C+Joseph+C+O%E2%80%99Brien%2C+Carrie+J+Cai%2C+Meredith+Ringel+Morris%2C+Percy+Liang%2C+and+Michael+S+Bernstein%2E+Generative+agents%3A+Interactive+simulacra+of+human+behavior%2E+arXiv+preprint+arXiv%3A2304%2E0344)
24. Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. [Social simulacra: Creating populated prototypes for social computing systems](https://md.research.elamur.ai/free/en/papers/park-2022-social-simulacra-creating-populated-prototypes/). In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, pages 1–18, 2022. 2, 4,…
25. Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. arXiv preprint arXiv:2307.[07924, 2023](https://md.research.elamur.ai/free/en/papers/qian-2023-chatdev-communicative-agents-software/). 6
26. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. [Is chatgpt a general-purpose natural language processing task solver?](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/) arXiv preprint arXiv:2302.06476, 2023. 1
27. Jeremy Roschelle and Stephanie D Teasley. The construction of shared knowledge in collaborative problem solving. In Computer supported collaborative learning, pages 69–97. Springer, 1995. 1 [find](https://scholar.google.com/scholar?q=Jeremy+Roschelle+and+Stephanie+D+Teasley%2E+The+construction+of+shared+knowledge+in+collaborative+problem+solving%2E+In+Computer+supported+collaborative+learning%2C+pages+69%E2%80%9397%2E+Springer%2C+1995%2E+1)
28. Noah Shinn, Beck Labash, and Ashwin Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.[11366, 2023](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). 10
29. Yu Shu, Siwei Dong, Guangyao Chen, Wenhao Huang, Ruihua Zhang, Daochen Shi, Qiqi Xiang, and Yemin Shi. Llasm: Large language and speech model. arXiv preprint arXiv:2308.15930, 2023. 1 — [arXiv:2308.15930](https://arxiv.org/abs/2308.15930) [find](https://scholar.google.com/scholar?q=Yu+Shu%2C+Siwei+Dong%2C+Guangyao+Chen%2C+Wenhao+Huang%2C+Ruihua+Zhang%2C+Daochen+Shi%2C+Qiqi+Xiang%2C+and+Yemin+Shi%2E+Llasm%3A+Large+language+and+speech+model%2E+arXiv+preprint+arXiv%3A2308%2E15930%2C+2023%2E+1+%E2%80%94+%5BarXiv%3A2308%2E15)
30. Steven A Sloman. The empirical case for two systems of reasoning. Psychological bulletin, 119(1):3, 1996. 1 [find](https://scholar.google.com/scholar?q=Steven+A+Sloman%2E+The+empirical+case+for+two+systems+of+reasoning%2E+Psychological+bulletin%2C+119%281%29%3A3%2C+1996%2E+1)
31. Yashar Talebirad and Amirhossein Nadiri. [Multi-agent collaboration: Harnessing the power of intelligent llm agents](https://md.research.elamur.ai/free/en/papers/talebirad-2023-multi-agent-collaboration-harnessing-power/). arXiv preprint arXiv:2306.03314, 2023. 3
32. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, et al. [Decodingtrust: A comprehensive assessment of trustworthiness in gpt models](https://md.research.elamur.ai/free/en/papers/wang-2023-decodingtrust-comprehensive-assessment-trustworthiness-gpt/). arXiv preprint arXiv:2306.11698, 2023. 1
33. Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926, 2023. 9 [find](https://scholar.google.com/scholar?q=Peiyi+Wang%2C+Lei+Li%2C+Liang+Chen%2C+Dawei+Zhu%2C+Binghuai+Lin%2C+Yunbo+Cao%2C+Qi+Liu%2C+Tianyu+Liu%2C+and+Zhifang+Sui%2E+Large+language+models+are+not+fair+evaluators%2E+arXiv+preprint+arXiv%3A2305%2E17926%2C+2023%2E+9)
34. Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unleashing cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. arXiv preprint arXiv:2307.[05300, 2023](https://md.research.elamur.ai/free/en/papers/wang-2023-unleashing-emergent-cognitive-synergy-large/). 1, 2, 3, 4, 5, 6, 9, 10, 17
35. Ross Williams, Niyousha Hosseinichimeh, Aritra Majumdar, and Navid Ghaffarzadegan. Epidemic modeling with generative agents. arXiv preprint arXiv:2307.04986, 2023. 2, 4, 17 — [arXiv:2307.04986](https://arxiv.org/abs/2307.04986) [find](https://scholar.google.com/scholar?q=Ross+Williams%2C+Niyousha+Hosseinichimeh%2C+Aritra+Majumdar%2C+and+Navid+Ghaffarzadegan%2E+Epidemic+modeling+with+generative+agents%2E+arXiv+preprint+arXiv%3A2307%2E04986%2C+2023%2E+2%2C+4%2C+17+%E2%80%94+%5BarXiv%3A2307%2E04986%5D%28https%3A)
36. Anita Williams Woolley, Ishani Aggarwal, and Thomas W Malone. Collective intelligence and group performance. Current Directions in Psychological Science, 24(6):420–424, 2015. 5 [find](https://scholar.google.com/scholar?q=Anita+Williams+Woolley%2C+Ishani+Aggarwal%2C+and+Thomas+W+Malone%2E+Collective+intelligence+and+group+performance%2E+Current+Directions+in+Psychological+Science%2C+24%286%29%3A420%E2%80%93424%2C+2015%2E+5)
37. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. [Autogen: Enabling next-gen llm applications via multi-agent conversation](https://md.research.elamur.ai/free/en/papers/wu-2023-autogen-enabling-next-gen-llm/) framework. arXiv preprint arXiv:2308.08155, 2023. 2, 3, 5
38. Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, and Zhendong Mao. [Expertprompting: Instructing large language models to be distinguished experts](https://md.research.elamur.ai/free/en/papers/xu-2023-expertprompting-instructing-large-language-models/). arXiv preprint arXiv:2305.14688, 2023. 2, 4, 5
39. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601, 2023. 9, 10 [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Thomas+L+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+arXiv+preprint+arXiv%3A23)
40. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). arXiv preprint arXiv:2210.03629, 2022. 7, 10
41. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). arXiv preprint arXiv:2306.05685, 2023. 9

*Updated 2 November 2026*