---
title: "Gentopia: A Collaborative Platform for Tool-Augmented LLMs"
description: "The core framework builds an agent from a single YAML file that names its LLM, interaction type… 38 references, cited by 9 papers here."
authors: ["Binfeng Xu", "Xukun Liu", "Hua Shen", "Zeyu Han", "Yuhan Li", "Murong Yue", "Zhiyuan Peng", "Yuchen Liu", "Ziyu Yao", "Dongkuan Xu"]
year: 2023
venue: "arXiv:2308.04030"
source: "https://arxiv.org/abs/2308.04030"
doi: "10.48550/arxiv.2308.04030"
arxiv: "2308.04030"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xu-2023-gentopia-collaborative-platform-tool-augmented/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xu-2023-gentopia-collaborative-platform-tool-augmented/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/) / Gentopia: A Collaborative Platform for Tool-Augmented LLMs

# Gentopia: A Collaborative Platform for Tool-Augmented LLMs

**Authors:** [Binfeng Xu](https://md.research.elamur.ai/free/en/authors/binfeng-xu/), Xukun Liu, [Hua Shen](https://md.research.elamur.ai/free/en/authors/hua-shen/), [Zeyu Han](https://md.research.elamur.ai/free/en/authors/zeyu-han/), [Yuhan Li](https://md.research.elamur.ai/free/en/authors/yuhan-li/), Murong Yue, [Zhiyuan Peng](https://md.research.elamur.ai/free/en/authors/zhiyuan-peng/), [Yuchen Liu](https://md.research.elamur.ai/free/en/authors/yuchen-liu/), [Ziyu Yao](https://md.research.elamur.ai/free/en/authors/ziyu-yao/) and [Dongkuan Xu](https://md.research.elamur.ai/free/en/authors/dongkuan-xu/) · **Year:** 2023 · **Venue:** arXiv:2308.04030 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2308.04030) · [PDF](https://arxiv.org/pdf/2308.04030) · [DOI 10.48550/arxiv.2308.04030](https://doi.org/10.48550/arxiv.2308.04030) · [arXiv 2308.04030](https://arxiv.org/abs/2308.04030) · [OpenAlex](https://openalex.org/W4385714628)

**Citations:** 6 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/xu-2023-gentopia-collaborative-platform-tool-augmented/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Existing toolkits for tool-augmented language models (ALMs, or "agents"), such as AutoGPT, LangChain and HuggingGPT, are often tied to narrow task sets and need heavy boilerplate to tune. They also offer no shared place to publish agents and no benchmark that tests agents beyond accuracy. Gentopia addresses these gaps with three linked parts. The core framework builds an agent from a single YAML file that names its LLM, interaction type, prompt, plugins, memory and sub-agents, and an AgentAssembler turns that file into a working instance. GentPool is a public registry where users contribute agents, each documented on a Wiki page, and registered agents can be nested so they cooperate. GentBench, bundled with GentPool, scores agents on reasoning, knowledge, safety, multilingual ability and efficiency through configurable grading pipelines. The paper describes a system rather than reporting experiments, and its contribution is the open-source toolkit itself. It matters because it lets researchers build small, task-specialized agents cheaply, reproduce them and combine them, instead of relying on one huge general model.

## Key points
- **Agent types and models:** agents run as Vanilla, ReAct, ReWOO, OpenAI (Function API) or OpenAI Memory. The authors note that switching from ReAct to ReWOO greatly cuts runtime on observation-independent tasks such as search-and-summarize. Supported backends include OpenAI models (GPT-4, GPT-3.5-turbo) and more than 10 HuggingFace open LLM families, such as Llama, Alpaca, Vicuna, Falcon, Flan and MPT, on both CPU and GPU.
- **Configuration operators:** YAML configs use `!prompt` (custom prompt template), `!tool` (user-defined tool), `!include` (sub-agent as a plugin), `!file` (local text file) and `!env` (environment variable). Memory is either a short-term scratch-pad or a long-term VectorDB such as chroma.
- **GentBench tasks:** the reasoning tasks draw on MATH, GSM8K, HumanEval, MBPP, APPS, LLM-Plan and BBH. Other categories cover knowledge and web retrieval, safety (integrity, harmlessness), multilingual translation and understanding, and efficiency (token usage, runtime). Graders include GatedGrader, DojoGrader, InstructionGrader and CodeGrader.
- **Workflow:** users create or clone an agent from templates (e.g. `react_template`). They chat with it through the CLI via `assemble.py` or the GUI, evaluate it with `evaluate.py`, and publish it to GentPool through a GitHub Pull Request.
- **Availability:** all components are released under the MIT license at github.com/Gentopia-AI.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works

## Cite

Cite as: Binfeng Xu, Xukun Liu, Hua Shen, Zeyu Han, Yuhan Li, Murong Yue, Zhiyuan Peng, Yuchen Liu, Ziyu Yao and Dongkuan Xu. 2023. Gentopia: A Collaborative Platform for Tool-Augmented LLMs. arXiv:2308.04030.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{xu2023gentopia,
  title = {{Gentopia: A Collaborative Platform for Tool-Augmented LLMs}},
  author = {Binfeng Xu, Xukun Liu, Hua Shen, Zeyu Han, Yuhan Li, Murong Yue, Zhiyuan Peng, Yuchen Liu, Ziyu Yao and Dongkuan Xu},
  year = {2023},
  doi = {10.48550/arxiv.2308.04030},
  eprint = {2308.04030},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.04030}
}
```

- [← previous Extension Multi30K: Multimodal Dataset for Integrated Vision and Language Research in Ukrainian](https://md.research.elamur.ai/free/en/papers/saichyshyna-2023-extension-multi30k-multimodal-dataset-integrated/)
- [next → HeRo: RoBERTa and Longformer Hebrew Language Models](https://md.research.elamur.ai/free/en/papers/shalumov-2023-hero-roberta-longformer-hebrew-language/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/)

## References

38 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Merouane Debbah, Etienne Goffinet, Daniel Heslow, Julien Launay, Quentin Malartic, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. Falcon-40B: an open large language model with state-of-t… [find](https://scholar.google.com/scholar?q=Ebtesam+Almazrouei%2C+Hamza+Alobeidli%2C+Abdulaziz+Alshamsi%2C+Alessandro+Cappelli%2C+Ruxandra+Cojocaru%2C+Merouane+Debbah%2C+Etienne+Goffinet%2C+Daniel+Heslow%2C+Julien+Launay%2C+Quentin+Malartic%2C+Badreddine+Noune%2C+Ba)
2. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv preprint arXiv:2108.07732.
3. Systems, CHI ’23, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Systems%2C+CHI+%E2%80%9923%2C+New+York%2C+NY%2C+USA%2E+Association+for+Computing+Machinery%2E)
4. Shawn Callegari. 2023. Semantic Kernel: Integrate cutting-edge LLM technology quickly and easily into your apps. [find](https://scholar.google.com/scholar?q=Shawn+Callegari%2E+2023%2E+Semantic+Kernel%3A+Integrate+cutting%2Dedge+LLM+technology+quickly+and+easily+into+your+apps%2E)
5. Harrison Chase. 2023. LangChain: Next Generation Language Processing. [find](https://scholar.google.com/scholar?q=Harrison+Chase%2E+2023%2E+LangChain%3A+Next+Generation+Language+Processing%2E)
6. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374.
7. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Zhuohan+Li%2C+Zi+Lin%2C+Ying+Sheng%2C+Zhanghao+Wu%2C+Hao+Zhang%2C+Lianmin+Zheng%2C+Siyuan+Zhuang%2C+Yonghao+Zhuang%2C+Joseph+E%2E+Gonzalez%2C+Ion+Stoica%2C+and+Eric+P%2E+Xing%2E+2023%2E+Vicuna%3A+An+open%2Dsource+cha)
8. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
9. Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021. Bold: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 862–872… [find](https://scholar.google.com/scholar?q=Jwala+Dhamala%2C+Tony+Sun%2C+Varun+Kumar%2C+Satyapriya+Krishna%2C+Yada+Pruksachatkun%2C+Kai%2DWei+Chang%2C+and+Rahul+Gupta%2E+2021%2E+Bold%3A+Dataset+and+metrics+for+measuring+biases+in+open%2Dended+language+generation%2E+In)
10. Ronen Eldan and Yuanzhi Li. 2023. [Tinystories: How small can language models be and still speak coherent english?](https://md.research.elamur.ai/free/en/papers/eldan-2023-tinystories-small-language-models-still/) arXiv preprint arXiv:2305.07759.
11. Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal, and Tushar Khot. 2023. [Specializing smaller language models towards multi-step reasoning](https://md.research.elamur.ai/free/en/papers/fu-2023-specializing-smaller-language-models-towards/). arXiv preprint arXiv:2301.12726.
12. Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. 2023. [A real-world webagent with planning, long context understanding, and program synthesis](https://md.research.elamur.ai/free/en/papers/gur-2023-real-world-webagent-planning-long/). arXiv preprint arXiv:2307.12856.
13. BIG bench authors. 2023. [Beyond the imitation game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). Transactions on Machine Learning Research.
14. Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. 2021a. [Measuring coding challenge competence with apps](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-coding-challenge-competence-apps/). In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (R…
15. Ángel Alexander Cabrera, Erica Fu, Donald Bertucci, Kenneth Holstein, Ameet Talwalkar, Jason I. Hong, and Adam Perer. 2023. Zeno: An interactive framework for behavioral evaluation of machine learning. In CHI Conference on Human Factors in Computing Dan Hendrycks, Collin Burns, Steven Basart, Andy Z… [find](https://scholar.google.com/scholar?q=%C3%81ngel+Alexander+Cabrera%2C+Erica+Fu%2C+Donald+Bertucci%2C+Kenneth+Holstein%2C+Ameet+Talwalkar%2C+Jason+I%2E+Hong%2C+and+Adam+Perer%2E+2023%2E+Zeno%3A+An+interactive+framework+for+behavioral+evaluation+of+machine+learning)
16. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021b. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
17. Shima Imani, Liang Du, and Harsh Shrivastava. 2023. [Mathprompter: Mathematical reasoning using large language models](https://md.research.elamur.ai/free/en/papers/imani-2023-mathprompter-mathematical-reasoning-large-language/). arXiv preprint arXiv:2303.05398.
18. Zhou, , and Jason Wei. 2022. [Challenging big-bench tasks and whether chain-of-thought can solve them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/). arXiv preprint arXiv:2210.09261.
19. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Alpaca: A strong, replicable instruction-following model. Stanford Center for Research on Foundation Models., 3(6):7. [find](https://scholar.google.com/scholar?q=Rohan+Taori%2C+Ishaan+Gulrajani%2C+Tianyi+Zhang%2C+Yann+Dubois%2C+Xuechen+Li%2C+Carlos+Guestrin%2C+Percy+Liang%2C+and+Tatsunori+B+Hashimoto%2E+2023%2E+Alpaca%3A+A+strong%2C+replicable+instruction%2Dfollowing+model%2E+Stanford+)
20. Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645. [find](https://scholar.google.com/scholar?q=Jinhao+Jiang%2C+Kun+Zhou%2C+Zican+Dong%2C+Keming+Ye%2C+Wayne+Xin+Zhao%2C+and+Ji%2DRong+Wen%2E+2023%2E+Structgpt%3A+A+general+framework+for+large+language+model+to+reason+over+structured+data%2E+arXiv+preprint+arXiv%3A2305%2E)
21. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
22. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). Advances in neural information processing systems, 35:22199– 22213.
23. Karthik Valmeekam, Sarath Sreedharan, Matthew Marquez, Alberto Olmo, and Subbarao Kambhampati. 2023. [On the planning abilities of large language models (a critical investigation with a proposed benchmark)](https://md.research.elamur.ai/free/en/papers/valmeekam-2023-planning-abilities-large-language-models/). arXiv preprint arXiv:2302.06706.
24. Abhay Kondi. 2023. SuperAGI: Open-source framework to build, manage and run useful Autonomous AI Agents. [find](https://scholar.google.com/scholar?q=Abhay+Kondi%2E+2023%2E+SuperAGI%3A+Open%2Dsource+framework+to+build%2C+manage+and+run+useful+Autonomous+AI+Agents%2E)
25. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv: Arxiv-2305.16291. [find](https://scholar.google.com/scholar?q=Guanzhi+Wang%2C+Yuqi+Xie%2C+Yunfan+Jiang%2C+Ajay+Mandlekar%2C+Chaowei+Xiao%2C+Yuke+Zhu%2C+Linxi+Fan%2C+and+Anima+Anandkumar%2E+2023%2E+Voyager%3A+An+open%2Dended+embodied+agent+with+large+language+models%2E+arXiv+preprint+ar)
26. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). arXiv preprint arXiv:2305.20050.
27. MosaicML NLP Team. 2023. Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs. [find](https://scholar.google.com/scholar?q=MosaicML+NLP+Team%2E+2023%2E+Introducing+MPT%2D7B%3A+A+New+Standard+for+Open%2DSource%2C+Commercially+Usable+LLMs%2E)
28. Anton Osika. 2023. GPT-Engineer: Specify what you want it to build, the AI asks for clarification, and then builds it. [find](https://scholar.google.com/scholar?q=Anton+Osika%2E+2023%2E+GPT%2DEngineer%3A+Specify+what+you+want+it+to+build%2C+the+AI+asks+for+clarification%2C+and+then+builds+it%2E)
29. Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2023. [Unifying large language models and knowledge graphs: A roadmap](https://md.research.elamur.ai/free/en/papers/pan-2023-unifying-large-language-models-knowledge/). arXiv preprint arXiv:2306.08302.
30. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R Bowman. 2021. [Bbq: A hand-built bias benchmark for question answering](https://md.research.elamur.ai/free/en/papers/parrish-2022-bbq-hand-built-bias-benchmark/). arXiv preprint arXiv:2110.08193.
31. Toran Bruce Richards. 2023. Auto-GPT: An Autonomous GPT-4 Experiment. [find](https://scholar.google.com/scholar?q=Toran+Bruce+Richards%2E+2023%2E+Auto%2DGPT%3A+An+Autonomous+GPT%2D4+Experiment%2E)
32. Sasha Rush. 2023. MiniChain: A tiny library for coding with large language models. [find](https://scholar.google.com/scholar?q=Sasha+Rush%2E+2023%2E+MiniChain%3A+A+tiny+library+for+coding+with+large+language+models%2E)
33. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. [Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/). arXiv preprint arXiv:2303.17580.
34. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. [Finetuned language models are zero…](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/)
35. Alexander Wu. 2023. MetaGPT: The Multi-Role Meta Programming Framework. [find](https://scholar.google.com/scholar?q=Alexander+Wu%2E+2023%2E+MetaGPT%3A+The+Multi%2DRole+Meta+Programming+Framework%2E)
36. Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, and Dongkuan Xu. 2023. [Rewoo: Decoupling reasoning from observations for efficient augmented language models](https://md.research.elamur.ai/free/en/papers/xu-2023-rewoo-decoupling-reasoning-observations-efficient/). arXiv preprint arXiv:2305.18323.
37. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). arXiv preprint arXiv:2210.03629.
38. Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. 2023. [Agieval: A human-centric benchmark for evaluating foundation models](https://md.research.elamur.ai/free/en/papers/zhong-2024-agieval-human-centric-benchmark-evaluating/). arXiv preprint arXiv:2304.06364.

*Updated 3 November 2026*