---
title: "RestGPT: Connecting Large Language Models with — Song 2023"
description: "Most tool-augmented LLMs work with a small set of custom-built tools and struggle with multi-step requests, so they fall short in realistic settings."
authors: ["Yifan Song", "Weimin Xiong", "Dawei Zhu", "Wenhao Wu", "Han Qian", "Mingbo Song", "Hailiang Huang", "Cheng Li", "Ke Wang", "Rong Yao", "Ye Tian", "Sujian Li"]
year: 2023
venue: "arXiv:2306.06624"
source: "https://arxiv.org/abs/2306.06624"
doi: "10.48550/arxiv.2306.06624"
arxiv: "2306.06624"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/song-2023-restgpt-connecting-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/song-2023-restgpt-connecting-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/) / RestGPT: Connecting Large Language Models with Real-World RESTful APIs

# RestGPT: Connecting Large Language Models with Real-World RESTful APIs

**Authors:** Yifan Song, Weimin Xiong, [Dawei Zhu](https://md.research.elamur.ai/free/en/authors/dawei-zhu/), Wenhao Wu, Han Qian, Mingbo Song, Hailiang Huang, [Cheng Li](https://md.research.elamur.ai/free/en/authors/cheng-li/), [Ke Wang](https://md.research.elamur.ai/free/en/authors/ke-wang/), Rong Yao, [Ye Tian](https://md.research.elamur.ai/free/en/authors/ye-tian/), [Sujian Li](https://md.research.elamur.ai/free/en/authors/sujian-li/) · **Year:** 2023 · **Venue:** arXiv:2306.06624 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2306.06624) · [PDF](https://arxiv.org/pdf/2306.06624) · [DOI 10.48550/arxiv.2306.06624](https://doi.org/10.48550/arxiv.2306.06624) · [arXiv 2306.06624](https://arxiv.org/abs/2306.06624) · [OpenAlex](https://openalex.org/W4380552023)

**Citations:** 17 citations (openalex-snapshot · 2026-10-14) · [91](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Most tool-augmented LLMs work with a small set of custom-built tools and struggle with multi-step requests, so they fall short in realistic settings. This paper links LLMs to RESTful APIs, the dominant style for web services, which are documented through the OpenAPI Specification (OAS). The proposed framework, RestGPT, splits work among three prompted LLM modules. A Planner proposes the next sub-task in natural language. An API Selector turns that sub-task into concrete API calls. An Executor then runs the calls through two parts: a Caller, which builds parameters from the API docs, and a Parser, which writes Python code from the OAS response schema to extract data from the JSON output. Planning happens online and iteratively, so the system revises its next step after seeing each execution result. The authors also release RestBench, a benchmark of human-written instructions with annotated gold solution paths in two domains: the TMDB movie database and the Spotify music player. RestGPT beats offline plan-then-execute and ReAct baselines, especially on long tasks. This suggests LLM agents can plug into existing web applications without hand-built tool wrappers.

## Key points
- RestGPT supports over 100 RESTful APIs. Earlier systems use far fewer, such as the 15 tools in Chameleon. Because RestGPT works with any OAS-described service, it can be added to an application without custom integration.
- Each module reads only the part of the OAS it needs: the selector uses endpoint descriptions, the Caller uses detailed docs, and the Parser uses response schemas. This helps fit large API sets into the LLM's limited context window.
- On instructions whose gold path has length 4, all baselines largely fail, while RestGPT still succeeds more than 40% of the time.
- In a scaling test, the authors started with 10 TMDB APIs and 15 instructions, then added irrelevant "noise" APIs. Baseline performance dropped as noise grew, but RestGPT stayed nearly unaffected.
- In a Spotify playlist case study, RestGPT completed the task in 4 API calls versus 6 for ReAct. The offline planner picked a wrong API and ignored dependencies between calls.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) · [AI & machine learning](https://md.research.elamur.ai/free/en/shelves/ai/)

## Cited by

Cited by

91 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 84 not yet.

- [Evaluating Large Language Models: A Comprehensive Survey](https://md.research.elamur.ai/free/en/papers/guo-2023-evaluating-large-language-models-comprehensive/)
- [A Comprehensive Overview of Large Language Models](https://md.research.elamur.ai/free/en/papers/naveed-2023-comprehensive-overview-large-language-models/)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://md.research.elamur.ai/free/en/papers/qin-2023-toolllm-facilitating-large-language-models/)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://md.research.elamur.ai/free/en/papers/tang-2023-toolalpaca-generalized-tool-learning-language/)
- [MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback](https://md.research.elamur.ai/free/en/papers/wang-2023-mint-evaluating-llms-multi-turn/)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/)
- [大语言模型的工具使用综述](https://md.research.elamur.ai/free/en/papers/zheng-2025-tool-use-large/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{song2023restgpt,
  title = {{RestGPT: Connecting Large Language Models with Real-World RESTful APIs}},
  author = {Song, Yifan and Xiong, Weimin and Zhu, Dawei and Wu, Wenhao and Qian, Han and Song, Mingbo and Huang, Hailiang and Li, Cheng and Wang, Ke and Yao, Rong and Tian, Ye and Li, Sujian},
  year = {2023},
  doi = {10.48550/arxiv.2306.06624},
  eprint = {2306.06624},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2306.06624}
}
```

- [← previous Representation Engineering: A Top-Down Approach to AI Transparency](https://md.research.elamur.ai/free/en/papers/zou-2023-representation-engineering-top-down-approach/)
- [next → Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent](https://md.research.elamur.ai/free/en/papers/freitag-2023-results-wmt23-metrics-shared-task/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/)

## References

26 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al%2E+Language+models+are+few%2Dshot+learners%2E+)
2. OpenAI. Chatgpt, 2022. URL [https://openai.com/blog/chatgpt](https://openai.com/blog/chatgpt). [↗](https://openai.com/blog/chatgpt)
3. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. A survey for in-context learning. arXiv preprint arXiv:2301.00234, 2022. [find](https://scholar.google.com/scholar?q=Qingxiu+Dong%2C+Lei+Li%2C+Damai+Dai%2C+Ce+Zheng%2C+Zhiyong+Wu%2C+Baobao+Chang%2C+Xu+Sun%2C+Jingjing+Xu%2C+and+Zhifang+Sui%2E+A+survey+for+in%2Dcontext+learning%2E+arXiv+preprint+arXiv%3A2301%2E00234%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)
4. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Maarten+Bosma%2C+Ed+Chi%2C+Quoc+Le%2C+and+Denny+Zhou%2E+Chain+of+thought+prompting+elicits+reasoning+in+large+language+models%2E+arXiv+preprint+arXiv%3A2201%2E11903%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
5. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682, 2022. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Yi+Tay%2C+Rishi+Bommasani%2C+Colin+Raffel%2C+Barret+Zoph%2C+Sebastian+Borgeaud%2C+Dani+Yogatama%2C+Maarten+Bosma%2C+Denny+Zhou%2C+Donald+Metzler%2C+et+al%2E+Emergent+abilities+of+large+language+models%2E+arXiv+p) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/)
6. Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International Conference on Machine Learning, pages 9118–9147. PMLR, 2022. [find](https://scholar.google.com/scholar?q=Wenlong+Huang%2C+Pieter+Abbeel%2C+Deepak+Pathak%2C+and+Igor+Mordatch%2E+Language+models+as+zero%2Dshot+planners%3A+Extracting+actionable+knowledge+for+embodied+agents%2E+In+International+Conference+on+Machine+Learn) [our card](https://md.research.elamur.ai/free/en/papers/huang-2022-language-models-zero-shot-planners/)
7. Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560, 2023. [find](https://scholar.google.com/scholar?q=Zihao+Wang%2C+Shaofei+Cai%2C+Anji+Liu%2C+Xiaojian+Ma%2C+and+Yitao+Liang%2E+Describe%2C+explain%2C+plan+and+select%3A+Interactive+planning+with+large+language+models+enables+open%2Dworld+multi%2Dtask+agents%2E+arXiv+preprin) [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-describe-explain-plan-select-interactive/)
8. Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, et al. Augmented language models: a survey. arXiv preprint arXiv:2302.07842, 2023. [find](https://scholar.google.com/scholar?q=Gr%C3%A9goire+Mialon%2C+Roberto+Dess%C3%AC%2C+Maria+Lomeli%2C+Christoforos+Nalmpantis%2C+Ram+Pasunuru%2C+Roberta+Raileanu%2C+Baptiste+Rozi%C3%A8re%2C+Timo+Schick%2C+Jane+Dwivedi%2DYu%2C+Asli+Celikyilmaz%2C+et+al%2E+Augmented+language+model) [our card](https://md.research.elamur.ai/free/en/papers/mialon-2023-augmented-language-models-survey/)
9. Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, et al. Tool learning with foundation models. arXiv preprint arXiv:2304.08354, 2023. [find](https://scholar.google.com/scholar?q=Yujia+Qin%2C+Shengding+Hu%2C+Yankai+Lin%2C+Weize+Chen%2C+Ning+Ding%2C+Ganqu+Cui%2C+Zheni+Zeng%2C+Yufei+Huang%2C+Chaojun+Xiao%2C+Chi+Han%2C+et+al%2E+Tool+learning+with+foundation+models%2E+arXiv+preprint+arXiv%3A2304%2E08354%2C+202) [our card](https://md.research.elamur.ai/free/en/papers/qin-2023-tool-learning-foundation-models/)
10. Aaron Parisi, Yao Zhao, and Noah Fiedel. Talm: Tool augmented language models. arXiv preprint arXiv:2205.12255, 2022. [find](https://scholar.google.com/scholar?q=Aaron+Parisi%2C+Yao+Zhao%2C+and+Noah+Fiedel%2E+Talm%3A+Tool+augmented+language+models%2E+arXiv+preprint+arXiv%3A2205%2E12255%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/parisi-2022-talm-tool-augmented-language-models/)
11. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023. [find](https://scholar.google.com/scholar?q=Timo+Schick%2C+Jane+Dwivedi%2DYu%2C+Roberto+Dess%C3%AC%2C+Roberta+Raileanu%2C+Maria+Lomeli%2C+Luke+Zettlemoyer%2C+Nicola+Cancedda%2C+and+Thomas+Scialom%2E+Toolformer%3A+Language+models+can+teach+themselves+to+use+tools%2E+arXiv) [our card](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/)
12. Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language models. arXiv preprint arXiv:2304.09842, 2023. [find](https://scholar.google.com/scholar?q=Pan+Lu%2C+Baolin+Peng%2C+Hao+Cheng%2C+Michel+Galley%2C+Kai%2DWei+Chang%2C+Ying+Nian+Wu%2C+Song%2DChun+Zhu%2C+and+Jianfeng+Gao%2E+Chameleon%3A+Plug%2Dand%2Dplay+compositional+reasoning+with+large+language+models%2E+arXiv+preprint) [our card](https://md.research.elamur.ai/free/en/papers/lu-2023-chameleon-plug-play-compositional-reasoning/)
13. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint arXiv:2303.17580, 2023. [find](https://scholar.google.com/scholar?q=Yongliang+Shen%2C+Kaitao+Song%2C+Xu+Tan%2C+Dongsheng+Li%2C+Weiming+Lu%2C+and+Yueting+Zhuang%2E+Hugginggpt%3A+Solving+ai+tasks+with+chatgpt+and+its+friends+in+huggingface%2E+arXiv+preprint+arXiv%3A2303%2E17580%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/)
14. Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671, 2023. [find](https://scholar.google.com/scholar?q=Chenfei+Wu%2C+Shengming+Yin%2C+Weizhen+Qi%2C+Xiaodong+Wang%2C+Zecheng+Tang%2C+and+Nan+Duan%2E+Visual+chatgpt%3A+Talking%2C+drawing+and+editing+with+visual+foundation+models%2E+arXiv+preprint+arXiv%3A2303%2E04671%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/wu-2023-visual-chatgpt-talking-drawing-editing/)
15. Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, and Ying Shan. Gpt4tools: Teaching large language model to use tools via self-instruction. arXiv preprint arXiv:2305.18752, 2023. [find](https://scholar.google.com/scholar?q=Rui+Yang%2C+Lin+Song%2C+Yanwei+Li%2C+Sijie+Zhao%2C+Yixiao+Ge%2C+Xiu+Li%2C+and+Ying+Shan%2E+Gpt4tools%3A+Teaching+large+language+model+to+use+tools+via+self%2Dinstruction%2E+arXiv+preprint+arXiv%3A2305%2E18752%2C+2023%2E)
16. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Jeffrey+Zhao%2C+Dian+Yu%2C+Nan+Du%2C+Izhak+Shafran%2C+Karthik+Narasimhan%2C+and+Yuan+Cao%2E+React%3A+Synergizing+reasoning+and+acting+in+language+models%2E+arXiv+preprint+arXiv%3A2210%2E03629%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/)
17. Li Li, Wu Chou, Wei Zhou, and Min Luo. Design patterns and extensibility of rest api for networking applications. IEEE Transactions on Network and Service Management, 13(1):154–167, 2016. [find](https://scholar.google.com/scholar?q=Li+Li%2C+Wu+Chou%2C+Wei+Zhou%2C+and+Min+Luo%2E+Design+patterns+and+extensibility+of+rest+api+for+networking+applications%2E+IEEE+Transactions+on+Network+and+Service+Management%2C+13%281%29%3A154%E2%80%93167%2C+2016%2E)
18. SmartBear. Swagger, 2023. URL [https://swagger.io/](https://swagger.io/). [↗](https://swagger.io/)
19. Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. Art: Automatic multi-step reasoning and tool-use for large language models. arXiv preprint arXiv:2303.09014, 2023. [find](https://scholar.google.com/scholar?q=Bhargavi+Paranjape%2C+Scott+Lundberg%2C+Sameer+Singh%2C+Hannaneh+Hajishirzi%2C+Luke+Zettlemoyer%2C+and+Marco+Tulio+Ribeiro%2E+Art%3A+Automatic+multi%2Dstep+reasoning+and+tool%2Duse+for+large+language+models%2E+arXiv+prep)
20. Dídac Surís, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. arXiv preprint arXiv:2303.08128, 2023. [find](https://scholar.google.com/scholar?q=D%C3%ADdac+Sur%C3%ADs%2C+Sachit+Menon%2C+and+Carl+Vondrick%2E+Vipergpt%3A+Visual+inference+via+python+execution+for+reasoning%2E+arXiv+preprint+arXiv%3A2303%2E08128%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/suris-2023-vipergpt-visual-inference-python-execution/)
21. Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.15334, 2023. [find](https://scholar.google.com/scholar?q=Shishir+G+Patil%2C+Tianjun+Zhang%2C+Xin+Wang%2C+and+Joseph+E+Gonzalez%2E+Gorilla%3A+Large+language+model+connected+with+massive+apis%2E+arXiv+preprint+arXiv%3A2305%2E15334%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/patil-2023-gorilla-large-language-model-connected/)
22. Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, and Le Sun. Toolalpaca: Generalized tool learning for language models with 3000 simulated cases. arXiv preprint arXiv:2306.05301, 2023. [find](https://scholar.google.com/scholar?q=Qiaoyu+Tang%2C+Ziliang+Deng%2C+Hongyu+Lin%2C+Xianpei+Han%2C+Qiao+Liang%2C+and+Le+Sun%2E+Toolalpaca%3A+Generalized+tool+learning+for+language+models+with+3000+simulated+cases%2E+arXiv+preprint+arXiv%3A2306%2E05301%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/tang-2023-toolalpaca-generalized-tool-learning-language/)
23. Minghao Li, Feifan Song, Bowen Yu, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. Api-bank: A benchmark for tool-augmented llms. arXiv preprint arXiv:2304.08244, 2023. — [arXiv:2304.08244](https://arxiv.org/abs/2304.08244) [find](https://scholar.google.com/scholar?q=Minghao+Li%2C+Feifan+Song%2C+Bowen+Yu%2C+Haiyang+Yu%2C+Zhoujun+Li%2C+Fei+Huang%2C+and+Yongbin+Li%2E+Api%2Dbank%3A+A+benchmark+for+tool%2Daugmented+llms%2E+arXiv+preprint+arXiv%3A2304%2E08244%2C+2023%2E+%E2%80%94+%5BarXiv%3A2304%2E08244%5D%28https%3A%2F)
24. Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926, 2023. [find](https://scholar.google.com/scholar?q=Peiyi+Wang%2C+Lei+Li%2C+Liang+Chen%2C+Dawei+Zhu%2C+Binghuai+Lin%2C+Yunbo+Cao%2C+Qi+Liu%2C+Tianyu+Liu%2C+and+Zhifang+Sui%2E+Large+language+models+are+not+fair+evaluators%2E+arXiv+preprint+arXiv%3A2305%2E17926%2C+2023%2E)
25. Mark Masse. REST API design rulebook: designing consistent RESTful web service interfaces. " O’Reilly Media, Inc.", 2011. [find](https://scholar.google.com/scholar?q=Mark+Masse%2E+REST+API+design+rulebook%3A+designing+consistent+RESTful+web+service+interfaces%2E+%22+O%E2%80%99Reilly+Media%2C+Inc%2E%22%2C+2011%2E)
26. Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366, 2023. *Figure 6: A RESTful API from TMDB.* [find](https://scholar.google.com/scholar?q=Noah+Shinn%2C+Federico+Cassano%2C+Beck+Labash%2C+Ashwin+Gopinath%2C+Karthik+Narasimhan%2C+and+Shunyu+Yao%2E+Reflexion%3A+Language+agents+with+verbal+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2303%2E11366%2C+2023%2E+%2AF) [our card](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/)