---
title: "Tool Learning with Large Language Models: A Survey (2024)"
description: "Large language models rely on frozen parametric knowledge. 199 references, cited by 96 papers here."
authors: ["Changle Qu", "Sunhao Dai", "Xiaochi Wei", "Hengyi Cai", "Shuaiqiang Wang", "Dawei Yin", "Jun Xu", "Ji-Rong Wen"]
year: 2024
venue: "arXiv:2405.17935"
source: "https://arxiv.org/abs/2405.17935"
doi: "10.1007/s11704-024-40678-2"
arxiv: "2405.17935"
licence: "CC BY-NC-ND 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-nc-nd/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qu-2024-tool-learning-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qu-2024-tool-learning-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/) / Tool Learning with Large Language Models: A Survey

# Tool Learning with Large Language Models: A Survey

**Authors:** [Changle Qu](https://md.research.elamur.ai/free/en/authors/changle-qu/), [Sunhao Dai](https://md.research.elamur.ai/free/en/authors/sunhao-dai/), [Xiaochi Wei](https://md.research.elamur.ai/free/en/authors/xiaochi-wei/), [Hengyi Cai](https://md.research.elamur.ai/free/en/authors/hengyi-cai/), [Shuaiqiang Wang](https://md.research.elamur.ai/free/en/authors/shuaiqiang-wang/), [Dawei Yin](https://md.research.elamur.ai/free/en/authors/dawei-yin/), [Jun Xu](https://md.research.elamur.ai/free/en/authors/jun-xu/) and [Ji-Rong Wen](https://md.research.elamur.ai/free/en/authors/ji-rong-wen/) · **Year:** 2024 · **Venue:** [arXiv:2405.17935](https://md.research.elamur.ai/free/en/venues/frontiers-of-computer-science/) ([Frontiers of Computer Science 2024](https://md.research.elamur.ai/free/en/venues/frontiers-of-computer-science/2024/)) · **Publisher:** [Higher Education Press](https://md.research.elamur.ai/free/en/publishers/higher-education-press/) · **Licence:** [CC BY-NC-ND 4.0 (arXiv)](http://creativecommons.org/licenses/by-nc-nd/4.0/)

**Links:** [Publisher's page](https://doi.org/10.1007/s11704-024-40678-2) · [PDF](https://arxiv.org/pdf/2405.17935) · [DOI 10.1007/s11704-024-40678-2](https://doi.org/10.1007/s11704-024-40678-2) · [arXiv 2405.17935](https://arxiv.org/abs/2405.17935) · [OpenAlex](https://openalex.org/W4406297511)

**Citations:** 99 citations (OpenAlex, 2026-10-14) · [96](https://md.research.elamur.ai/free/en/papers/qu-2024-tool-learning-large-language-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large language models rely on frozen parametric knowledge. As a result, they often miscalculate or give outdated or invented answers. Connecting them to external tools such as search engines, calculators, code interpreters and APIs is a promising fix, but the research on this has been scattered and hard for newcomers to navigate. This survey organizes the field along two questions.

- **Why use tools?** The authors list six benefits. Four come from what tools add: knowledge acquisition, expertise enhancement, automation and efficiency, and richer interaction. Two come from the tool learning paradigm itself: interpretability and user trust, and robustness and adaptability.
- **How is tool use carried out?** The authors break the workflow into a sequence of four steps: planning the task, picking a suitable tool, invoking that tool, and finally producing the response. Within the stages, they separate tuning-free methods from tuning-based ones, and retriever-based selection from LLM-based selection.

The survey also maps benchmarks and evaluation metrics to the stage each one measures. It closes with open problems. The result is a structured entry point and reference for researchers and engineers building tool-augmented LLM systems.

## Key points
- The survey covers more than 150 papers. A companion GitHub repository (quchangle1/LLM-Tool-Survey) keeps the paper list current.
- The authors distinguish two usage paradigms: single invocation and iterative invocation of tools. Representative systems include Toolformer, ReACT, HuggingGPT, ToolLLaMA, Gorilla, ToolkenGPT and RestGPT.
- Covered benchmarks include ToolBench, APIBench, API-Bank, T-Eval, MetaTool, ToolAlpaca, StableToolBench, ToolQA and AppWorld.
- Metrics are matched to stages:
  - planning: pass rate and accuracy;
  - selection: Recall, NDCG and COMP;
  - response generation: BLEU, ROUGE-L and Exact Match.
- The authors name seven open challenges:
  - high latency;
  - more rigorous evaluation;
  - accessible tools;
  - safety and robustness (for example, RoTBench uses five noise levels and ToolSword defines six safety scenarios, and even GPT-4 struggles on them);
  - a unified framework;
  - benchmarks built from real user queries;
  - multi-modal tool learning.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Baidu](https://md.research.elamur.ai/free/en/companies/baidu/) — Company, Beijing, China · 84 works · 17,681 citations
- [Renmin University of China](https://md.research.elamur.ai/free/en/universities/renmin-university-of-china/) — University, Beijing, China · 139 works · 17,752 citations
- [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) — Government body, Beijing, China · 246 works · 24,951 citations
- [Institute of Computing Technology](https://md.research.elamur.ai/free/en/labs/institute-of-computing-technology/) — Research institute, Beijing, China · 57 works · 3,782 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

## Cite

Cite as: Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu and Ji-Rong Wen. 2024. Tool Learning with Large Language Models: A Survey. arXiv:2405.17935.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{qu2024tool,
  title = {{Tool Learning with Large Language Models: A Survey}},
  author = {Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu and Ji-Rong Wen},
  year = {2024},
  doi = {10.1007/s11704-024-40678-2},
  eprint = {2405.17935},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1007/s11704-024-40678-2}
}
```

- [← previous LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://md.research.elamur.ai/free/en/papers/li-2024-llms-judges-comprehensive-survey-llm/)
- [next → InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://md.research.elamur.ai/free/en/papers/zhan-2024-injecagent-benchmarking-indirect-prompt-injections/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/)

## References

199 references, 84 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Washburn S L. Tools and human evolution. Scientific American, 1960, 203(3): 62–75 [find](https://scholar.google.com/scholar?q=Washburn+S+L%2E+Tools+and+human+evolution%2E+Scientific+American%2C+1960%2C+203%283%29%3A+62%E2%80%9375)
2. Gibson K R, Gibson K R, Ingold T. Tools, language and cognition in human evolution. Cambridge University Press, 1993 [find](https://scholar.google.com/scholar?q=Gibson+K+R%2C+Gibson+K+R%2C+Ingold+T%2E+Tools%2C+language+and+cognition+in+human+evolution%2E+Cambridge+University+Press%2C+1993)
3. Von Eckardt B. What is cognitive science? The MIT press, 1995 [find](https://scholar.google.com/scholar?q=Von+Eckardt+B%2E+What+is+cognitive+science%3F+The+MIT+press%2C+1995)
4. Shumaker R W, Walkup K R, Beck B B. Animal tool behavior: the use and manufacture of tools by animals. JHU Press, 2011 [find](https://scholar.google.com/scholar?q=Shumaker+R+W%2C+Walkup+K+R%2C+Beck+B+B%2E+Animal+tool+behavior%3A+the+use+and+manufacture+of+tools+by+animals%2E+JHU+Press%2C+2011)
5. Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman F L, Almeida D, Altenschmidt J, Altman S, Anadkat S, others . [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774, 2023
6. El-Kassas W S, Salama C R, Rafea A A, Mohamed H K. Automatic text summarization: A comprehensive survey. Expert systems with applications, 2021, 165: 113679 [find](https://scholar.google.com/scholar?q=El%2DKassas+W+S%2C+Salama+C+R%2C+Rafea+A+A%2C+Mohamed+H+K%2E+Automatic+text+summarization%3A+A+comprehensive+survey%2E+Expert+systems+with+applications%2C+2021%2C+165%3A+113679)
7. Zhang T, Ladhak F, Durmus E, Liang P, McKeown K, Hashimoto T B. [Benchmarking large language models for news summarization](https://md.research.elamur.ai/free/en/papers/zhang-2024-benchmarking-large-language-models-news/). Transactions of the Association for Computational Linguistics, 2024
8. Zhang B, Haddow B, Birch A. [Prompting large language model for machine translation: A case study](https://md.research.elamur.ai/free/en/papers/zhang-2023-prompting-large-language-model-machine/). In: International Conference on Machine Learning. 2023, 41092–41110
9. Feng Z, Zhang Y, Li H, Liu W, Lang J, Feng Y, Wu J, Liu Z. Improving llm-based machine translation with systematic self-correction. arXiv preprint arXiv:2402.16379, 2024 — [arXiv:2402.16379](https://arxiv.org/abs/2402.16379) [find](https://scholar.google.com/scholar?q=Feng+Z%2C+Zhang+Y%2C+Li+H%2C+Liu+W%2C+Lang+J%2C+Feng+Y%2C+Wu+J%2C+Liu+Z%2E+Improving+llm%2Dbased+machine+translation+with+systematic+self%2Dcorrection%2E+arXiv+preprint+arXiv%3A2402%2E16379%2C+2024+%E2%80%94+%5BarXiv%3A2402%2E16379%5D%28https%3A%2F%2Fa)
10. Yang Z, Qi P, Zhang S, Bengio Y, Cohen W W, Salakhutdinov R, Manning C D. [Hotpotqa: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). arXiv preprint arXiv:1809.09600, 2018
11. Kwiatkowski T, Palomaki J, Redfield O, Collins M, Parikh A, Alberti C, Epstein D, Polosukhin I, Devlin J, Lee K, others . [Natural questions: a benchmark for question answering research](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/). Transactions of the Association for Computational Linguistics, 2019, 7: 453–466
12. Mallen A, Asai A, Zhong V, Das R, Khashabi D, Hajishirzi H. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. arXiv preprint arXiv:2212.10511, 2022 [find](https://scholar.google.com/scholar?q=Mallen+A%2C+Asai+A%2C+Zhong+V%2C+Das+R%2C+Khashabi+D%2C+Hajishirzi+H%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+memories%2E+arXiv+preprint+arXiv%3A2212%2E10511%2C+2)
13. Vu T, Iyyer M, Wang X, Constant N, Wei J, Wei J, Tar C, Sung Y H, Zhou D, Le Q, others . [Freshllms: Refreshing large language models with search engine augmentation](https://md.research.elamur.ai/free/en/papers/vu-2023-freshllms-refreshing-large-language-models/). arXiv preprint arXiv:2310.03214, 2023
14. Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishii E, Bang Y J, Madotto A, Fung P. [Survey of hallucination in natural language generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/). ACM Computing Surveys, 2023, 55(12): 1–38
15. Zhang Y, Li Y, Cui L, Cai D, Liu L, Fu T, Huang X, Zhao E, Zhang Y, Chen Y, others . Siren’s song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.[01219, 2023](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/)
16. Qin Y, Hu S, Lin Y, Chen W, Ding N, Cui G, Zeng Z, Huang Y, Xiao C, Han C, others . [Tool learning with foundation models](https://md.research.elamur.ai/free/en/papers/qin-2023-tool-learning-foundation-models/). arXiv preprint arXiv:2304.08354, 2023
17. Schick T, Dwivedi-Yu J, Dessı̀ R, Raileanu R, Lomeli M, Hambro E, Zettlemoyer L, Cancedda N, Scialom T. [Toolformer: Language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). Advances in Neural Information Processing Systems, 2024, 36
18. Qin Y, Liang S, Ye Y, Zhu K, Yan L, Lu Y, Lin Y, Cong X, Tang X, Qian B, others . [Toolllm: Facilitating large language models to master 16000+ real-world apis](https://md.research.elamur.ai/free/en/papers/qin-2023-toolllm-facilitating-large-language-models/). In Proceedings of the 12th ICLR, 2024
19. Tang Q, Deng Z, Lin H, Han X, Liang Q, Sun L. [Toolalpaca: Generalized tool learning for language models with 3000 simulated cases](https://md.research.elamur.ai/free/en/papers/tang-2023-toolalpaca-generalized-tool-learning-language/). arXiv preprint arXiv:2306.05301, 2023
20. Wang H, Qin Y, Lin Y, Pan J Z, Wong K F. Empowering large language models: Tool learning for real-world interaction. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2024 [find](https://scholar.google.com/scholar?q=Wang+H%2C+Qin+Y%2C+Lin+Y%2C+Pan+J+Z%2C+Wong+K+F%2E+Empowering+large+language+models%3A+Tool+learning+for+real%2Dworld+interaction%2E+In%3A+Proceedings+of+the+47th+International+ACM+SIGIR+Conference+on+Research+and+Deve)
21. Yao S, Chen H, Yang J, Narasimhan K. [Webshop: Towards scalable real-world web interaction with grounded language agents](https://md.research.elamur.ai/free/en/papers/yao-2022-webshop-towards-scalable-real-world/). Advances in Neural Information Processing Systems, 2022, 35: 20744–20757
22. Lazaridou A, Gribovskaya E, Stokowiec W, Grigorev N. [Internet-augmented language models through few-shot prompting for open-domain question answering](https://md.research.elamur.ai/free/en/papers/lazaridou-2022-internet-augmented-language-models-through/). arXiv preprint arXiv:2203.05115, 2022
23. Lu Y, Yu H, Khashabi D. Gear: Augmenting language models with generalizable and efficient tool resolution. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2023 [find](https://scholar.google.com/scholar?q=Lu+Y%2C+Yu+H%2C+Khashabi+D%2E+Gear%3A+Augmenting+language+models+with+generalizable+and+efficient+tool+resolution%2E+In+Proceedings+of+the+18th+Conference+of+the+European+Chapter+of+the+Association+for+Computat)
24. Pan L, Wu X, Lu X, Luu A T, Wang W Y, Kan M Y, Nakov P. [Fact-checking complex claims with program-guided reasoning](https://md.research.elamur.ai/free/en/papers/pan-2023-fact-checking-complex-claims-program/). In: Rogers A, Boyd-Graber J, Okazaki N, eds, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). July 2023, 6981–7004
25. Wang X, Wang Z, Liu J, Chen Y, Yuan L, Peng H, Ji H. [Mint: Evaluating llms in multi-turn interaction with tools and language feedback](https://md.research.elamur.ai/free/en/papers/wang-2023-mint-evaluating-llms-multi-turn/). In Proceedings of 12th ICLR., 2024
26. Parisi A, Zhao Y, Fiedel N. [Talm: Tool augmented language models](https://md.research.elamur.ai/free/en/papers/parisi-2022-talm-tool-augmented-language-models/). arXiv preprint arXiv:2205.12255, 2022
27. Karpas E, Abend O, Belinkov Y, Lenz B, Lieber O, Ratner N, Shoham Y, Bata H, Levine Y, Leyton-Brown K, others . Mrkl systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning. arXiv preprint arXiv:2205.00445, 2022 — [arXiv:2205.00445](https://arxiv.org/abs/2205.00445) [find](https://scholar.google.com/scholar?q=Karpas+E%2C+Abend+O%2C+Belinkov+Y%2C+Lenz+B%2C+Lieber+O%2C+Ratner+N%2C+Shoham+Y%2C+Bata+H%2C+Levine+Y%2C+Leyton%2DBrown+K%2C+others+%2E+Mrkl+systems%3A+A+modular%2C+neuro%2Dsymbolic+architecture+that+combines+large+language+models)
28. Nakano R, Hilton J, Balaji S, Wu J, Ouyang L, Kim C, Hesse C, Jain S, Kosaraju V, Saunders W, others . [Webgpt: Browser-assisted question-answering with human feedback](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/). arXiv preprint arXiv:2112.09332, 2021
29. Surı́s D, Menon S, Vondrick C. [Vipergpt: Visual inference via python execution for reasoning](https://md.research.elamur.ai/free/en/papers/suris-2023-vipergpt-visual-inference-python-execution/). In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023, 11888–11898
30. Li M, Zhao Y, Yu B, Song F, Li H, Yu H, Li Z, Huang F, Li Y. [Api-bank: A comprehensive benchmark for tool-augmented llms](https://md.research.elamur.ai/free/en/papers/li-2023-api-bank-comprehensive-benchmark-tool/). In: The 2023 Conference on Empirical Methods in Natural Language Processing. 2023
31. Huang Y, Shi J, Li Y, Fan C, Wu S, Zhang Q, Liu Y, Zhou P, Wan Y, Gong N Z, others . [Metatool benchmark for large language models: Deciding whether to use tools and which to use](https://md.research.elamur.ai/free/en/papers/huang-2023-metatool-benchmark-large-language-models/). In Proceedings of 12th ICLR., 2024
32. Chen Z, Du W, Zhang W, Liu K, Liu J, Zheng M, Zhuo J, Zhang S, Lin D, Chen K, others . T-eval: Evaluating the tool utilization capability step by step. arXiv preprint arXiv:2312.14033, 2023 — [arXiv:2312.14033](https://arxiv.org/abs/2312.14033) [find](https://scholar.google.com/scholar?q=Chen+Z%2C+Du+W%2C+Zhang+W%2C+Liu+K%2C+Liu+J%2C+Zheng+M%2C+Zhuo+J%2C+Zhang+S%2C+Lin+D%2C+Chen+K%2C+others+%2E+T%2Deval%3A+Evaluating+the+tool+utilization+capability+step+by+step%2E+arXiv+preprint+arXiv%3A2312%2E14033%2C+2023+%E2%80%94+%5BarXiv%3A2)
33. Xu Q, Hong F, Li B, Hu C, Chen Z, Zhang J. [On the tool manipulation capability of open-source large language models](https://md.research.elamur.ai/free/en/papers/xu-2023-tool-manipulation-capability-open-source/). arXiv preprint arXiv:2305.16504, 2023
34. Gao S, Shi Z, Zhu M, Fang B, Xin X, Ren P, Chen Z, Ma J, Ren Z. Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum. In: In Proceedings of 38th Conference on Artificial Intelligence (AAAI). 2024 [find](https://scholar.google.com/scholar?q=Gao+S%2C+Shi+Z%2C+Zhu+M%2C+Fang+B%2C+Xin+X%2C+Ren+P%2C+Chen+Z%2C+Ma+J%2C+Ren+Z%2E+Confucius%3A+Iterative+tool+learning+from+introspection+feedback+by+easy%2Dto%2Ddifficult+curriculum%2E+In%3A+In+Proceedings+of+38th+Conference+on)
35. Zhao Y, Wu J, Wang X, Tang W, Wang D, De Rijke M. Let me do it for you: Towards llm empowered recommendation via tool learning. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2024 [find](https://scholar.google.com/scholar?q=Zhao+Y%2C+Wu+J%2C+Wang+X%2C+Tang+W%2C+Wang+D%2C+De+Rijke+M%2E+Let+me+do+it+for+you%3A+Towards+llm+empowered+recommendation+via+tool+learning%2E+In%3A+Proceedings+of+the+47th+International+ACM+SIGIR+Conference+on+Resear)
36. Zhao W X, Zhou K, Li J, Tang T, Wang X, Hou Y, Min Y, Zhang B, Zhang J, Dong Z, others . [A survey of large language models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/). arXiv preprint arXiv:2303.18223, 2023
37. Huang X, Liu W, Chen X, Wang X, Wang H, Lian D, Wang Y, Tang R, Chen E. [Understanding the planning of llm agents: A survey](https://md.research.elamur.ai/free/en/papers/huang-2024-understanding-planning-llm-agents-survey/). arXiv preprint arXiv:2402.02716, 2024
38. Qiao S, Ou Y, Zhang N, Chen X, Yao Y, Deng S, Tan C, Huang F, Chen H. [Reasoning with language model prompting: A survey](https://md.research.elamur.ai/free/en/papers/qiao-2022-reasoning-language-model-prompting-survey/). arXiv preprint arXiv:2212.09597, 2022
39. Sun J, Zheng C, Xie E, Liu Z, Chu R, Qiu J, Xu J, Ding M, Li H, Geng M, others . A survey of reasoning with foundation models. arXiv preprint arXiv:2312.11562, 2023 — [arXiv:2312.11562](https://arxiv.org/abs/2312.11562) [find](https://scholar.google.com/scholar?q=Sun+J%2C+Zheng+C%2C+Xie+E%2C+Liu+Z%2C+Chu+R%2C+Qiu+J%2C+Xu+J%2C+Ding+M%2C+Li+H%2C+Geng+M%2C+others+%2E+A+survey+of+reasoning+with+foundation+models%2E+arXiv+preprint+arXiv%3A2312%2E11562%2C+2023+%E2%80%94+%5BarXiv%3A2312%2E11562%5D%28https%3A%2F%2Farxiv%2E)
40. Wang L, Ma C, Feng X, Zhang Z, Yang H, Zhang J, Chen Z, Tang J, Chen X, Lin Y, others . [A survey on large language model based autonomous agents](https://md.research.elamur.ai/free/en/papers/wang-2023-survey-large-language-model-based/). Frontiers of Computer Science, 2024, 18(6): 1–26
41. Sumers T, Yao S, Narasimhan K, Griffiths T. Cognitive architectures for language agents. Transactions on Machine Learning Research, 2024 [find](https://scholar.google.com/scholar?q=Sumers+T%2C+Yao+S%2C+Narasimhan+K%2C+Griffiths+T%2E+Cognitive+architectures+for+language+agents%2E+Transactions+on+Machine+Learning+Research%2C+2024)
42. Xi Z, Chen W, Guo X, He W, Ding Y, Hong B, Zhang M, Wang J, Jin S, Zhou E, others . [The rise and potential of large language model based agents: A survey](https://md.research.elamur.ai/free/en/papers/xi-2023-rise-potential-large-language/). arXiv preprint arXiv:2309.07864, 2023
43. Gao Y, Xiong Y, Gao X, Jia K, Pan J, Bi Y, Dai Y, Sun J, Wang H. [Retrieval-augmented generation for large language models: A survey](https://md.research.elamur.ai/free/en/papers/gao-2023-retrieval-augmented-generation-large-language/). arXiv preprint arXiv:2312.10997, 2023
44. Zhao P, Zhang H, Yu Q, Wang Z, Geng Y, Fu F, Yang L, Zhang W, Cui B. [Retrieval-augmented generation for ai-generated content: A survey](https://md.research.elamur.ai/free/en/papers/zhao-2024-retrieval-augmented-generation-ai-generated/). arXiv preprint arXiv:2402.19473, 2024
45. Mialon G, Dessı̀ R, Lomeli M, Nalmpantis C, Pasunuru R, Raileanu R, Rozière B, Schick T, Dwivedi-Yu J, Celikyilmaz A, others . [Augmented language models: a survey](https://md.research.elamur.ai/free/en/papers/mialon-2023-augmented-language-models-survey/). arXiv preprint arXiv:2302.07842, 2023
46. Wang Z, Cheng Z, Zhu H, Fried D, Neubig G. What are tools anyway? a survey from the language model perspective, 2024 [find](https://scholar.google.com/scholar?q=Wang+Z%2C+Cheng+Z%2C+Zhu+H%2C+Fried+D%2C+Neubig+G%2E+What+are+tools+anyway%3F+a+survey+from+the+language+model+perspective%2C+2024)
47. Komeili M, Shuster K, Weston J. [Internet-augmented dialogue generation](https://md.research.elamur.ai/free/en/papers/komeili-2022-internet-augmented-dialogue-generation/). In Proceedings of the 60st Annual Meeting of the Association for Computational Linguistics (ACL), 2022
48. Zhang K, Zhang H, Li G, Li J, Li Z, Jin Z. Toolcoder: Teach code generation models to use api search tools. arXiv preprint arXiv:2305.04032, 2023 — [arXiv:2305.04032](https://arxiv.org/abs/2305.04032) [find](https://scholar.google.com/scholar?q=Zhang+K%2C+Zhang+H%2C+Li+G%2C+Li+J%2C+Li+Z%2C+Jin+Z%2E+Toolcoder%3A+Teach+code+generation+models+to+use+api+search+tools%2E+arXiv+preprint+arXiv%3A2305%2E04032%2C+2023+%E2%80%94+%5BarXiv%3A2305%2E04032%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2305%2E04032%29)
49. Shi W, Min S, Yasunaga M, Seo M, James R, Lewis M, Zettlemoyer L, Yih W t. [Replug: Retrieval-augmented black-box language models](https://md.research.elamur.ai/free/en/papers/shi-2023-replug-retrieval-augmented-black-box/). arXiv preprint arXiv:2301.12652, 2023
50. Paranjape B, Lundberg S, Singh S, Hajishirzi H, Zettlemoyer L, Ribeiro M T. [Art: Automatic multi-step reasoning and tool-use for large language models](https://md.research.elamur.ai/free/en/papers/paranjape-2023-art-automatic-multi-step-reasoning/). arXiv preprint arXiv:2303.09014, 2023
51. Gou Z, Shao Z, Gong Y, Shen Y, Yang Y, Duan N, Chen W. [Critic: Large language models can self-correct with tool-interactive critiquing](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/). In Proceedings of the 12th ICLR, 2024
52. Thoppilan R, De Freitas D, Hall J, Shazeer N, Kulshreshtha A, Cheng H T, Jin A, Bos T, Baker L, Du Y, others . [Lamda: Language models for dialog applications](https://md.research.elamur.ai/free/en/papers/thoppilan-2022-lamda-language-models-dialog-applications/). arXiv preprint arXiv:2201.08239, 2022
53. Patil S G, Zhang T, Wang X, Gonzalez J E. [Gorilla: Large language model connected with massive apis](https://md.research.elamur.ai/free/en/papers/patil-2023-gorilla-large-language-model-connected/). arXiv preprint arXiv:2305.15334, 2023
54. Hao S, Liu T, Wang Z, Hu Z. [Toolkengpt: Augmenting frozen language models with massive tools via tool embeddings](https://md.research.elamur.ai/free/en/papers/hao-2023-toolkengpt-augmenting-frozen-language-models/). Advances in neural information processing systems, 2024, 36
55. Zhuang Y, Yu Y, Wang K, Sun H, Zhang C. [Toolqa: A dataset for llm question answering with external tools](https://md.research.elamur.ai/free/en/papers/zhuang-2023-toolqa-dataset-llm-question-answering/). Advances in Neural Information Processing Systems, 2024, 36
56. Zhang K, Chen H, Li L, Wang W. Syntax error-free and generalizable tool use for llms via finite-state decoding. Advances in Neural Information Processing Systems, 2024 [find](https://scholar.google.com/scholar?q=Zhang+K%2C+Chen+H%2C+Li+L%2C+Wang+W%2E+Syntax+error%2Dfree+and+generalizable+tool+use+for+llms+via+finite%2Dstate+decoding%2E+Advances+in+Neural+Information+Processing+Systems%2C+2024)
57. Gu Y, Shu Y, Yu H, Liu X, Dong Y, Tang J, Srinivasa J, Latapie H, Su Y. Middleware for llms: Tools are instrumental for language agents in complex environments. arXiv preprint arXiv:2402.14672, 2024 [find](https://scholar.google.com/scholar?q=Gu+Y%2C+Shu+Y%2C+Yu+H%2C+Liu+X%2C+Dong+Y%2C+Tang+J%2C+Srinivasa+J%2C+Latapie+H%2C+Su+Y%2E+Middleware+for+llms%3A+Tools+are+instrumental+for+language+agents+in+complex+environments%2E+arXiv+preprint+arXiv%3A2402%2E14672%2C+2024)
58. Cobbe K, Kosaraju V, Bavarian M, Chen M, Jun H, Kaiser L, Plappert M, Tworek J, Hilton J, Nakano R, others . [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021
59. Shao Z, Huang F, Huang M. Chaining simultaneous thoughts for numerical reasoning. arXiv preprint arXiv:2211.16482, 2022 [find](https://scholar.google.com/scholar?q=Shao+Z%2C+Huang+F%2C+Huang+M%2E+Chaining+simultaneous+thoughts+for+numerical+reasoning%2E+arXiv+preprint+arXiv%3A2211%2E16482%2C+2022)
60. Kadlčı́k M, Štefánik M, Sotolar O, Martinek V. Calc-x and calcformers: Empowering arithmetical chain-of-thought through interaction with symbolic systems. In: The 2023 Conference on Empirical Methods in Natural Language Processing. 2023 [find](https://scholar.google.com/scholar?q=Kadlc%CC%8C%C4%B1%CC%81k+M%2C+S%CC%8Ctefa%CC%81nik+M%2C+Sotolar+O%2C+Martinek+V%2E+Calc%2Dx+and+calcformers%3A+Empowering+arithmetical+chain%2Dof%2Dthought+through+interaction+with+symbolic+systems%2E+In%3A+The+2023+Conference+on+Empirical+Metho)
61. He-Yueya J, Poesia G, Wang R E, Goodman N D. [Solving math word problems by combining language models with symbolic solvers](https://md.research.elamur.ai/free/en/papers/yueya-2023-solving-math-word-problems-combining/). In Proceedings of the 2023 Annual Conference on Neural Information Processing Systems (NeurIPS), 2023
62. Zhang B, Zhou K, Wei X, Zhao X, Sha J, Wang S, Wen J R. Evaluating and improving tool-augmented computation-intensive math reasoning. Advances in Neural Information Processing Systems, 2024, 36 [find](https://scholar.google.com/scholar?q=Zhang+B%2C+Zhou+K%2C+Wei+X%2C+Zhao+X%2C+Sha+J%2C+Wang+S%2C+Wen+J+R%2E+Evaluating+and+improving+tool%2Daugmented+computation%2Dintensive+math+reasoning%2E+Advances+in+Neural+Information+Processing+Systems%2C+2024%2C+36)
63. Gou Z, Shao Z, Gong Y, Yang Y, Huang M, Duan N, Chen W, others . [Tora: A tool-integrated reasoning agent for mathematical problem solving](https://md.research.elamur.ai/free/en/papers/gou-2023-tora-tool-integrated-reasoning-agent/). In Proceedings of the 12th ICLR, 2024
64. Das D, Banerjee D, Aditya S, Kulkarni A. Mathsensei: A tool-augmented large language model for mathematical reasoning. arXiv preprint arXiv:2402.17231, 2024 [find](https://scholar.google.com/scholar?q=Das+D%2C+Banerjee+D%2C+Aditya+S%2C+Kulkarni+A%2E+Mathsensei%3A+A+tool%2Daugmented+large+language+model+for+mathematical+reasoning%2E+arXiv+preprint+arXiv%3A2402%2E17231%2C+2024)
65. Veerendranath V, Shah V, Ghate K. Calc-cmu at semeval-2024 task 7: Pre-calc – learning to use the calculator improves numeracy in language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-… [find](https://scholar.google.com/scholar?q=Veerendranath+V%2C+Shah+V%2C+Ghate+K%2E+Calc%2Dcmu+at+semeval%2D2024+task+7%3A+Pre%2Dcalc+%E2%80%93+learning+to+use+the+calculator+improves+numeracy+in+language+models%2E+In+Proceedings+of+the+2024+Conference+of+the+North+Am)
66. Bulusu A, Man B, Jagmohan A, Vempaty A, Mari-Wyka J, Akkil D. Mathviz-e: A case-study in domain-specialized tool-using agents. arXiv preprint arXiv:2407.17544, 2024 — [arXiv:2407.17544](https://arxiv.org/abs/2407.17544) [find](https://scholar.google.com/scholar?q=Bulusu+A%2C+Man+B%2C+Jagmohan+A%2C+Vempaty+A%2C+Mari%2DWyka+J%2C+Akkil+D%2E+Mathviz%2De%3A+A+case%2Dstudy+in+domain%2Dspecialized+tool%2Dusing+agents%2E+arXiv+preprint+arXiv%3A2407%2E17544%2C+2024+%E2%80%94+%5BarXiv%3A2407%2E17544%5D%28https%3A%2F%2Farxiv%2E)
67. Gao L, Madaan A, Zhou S, Alon U, Liu P, Yang Y, Callan J, Neubig G. Pal: Program-aided language models. In: International Conference on Machine Learning. 2023, 10764–10799 [find](https://scholar.google.com/scholar?q=Gao+L%2C+Madaan+A%2C+Zhou+S%2C+Alon+U%2C+Liu+P%2C+Yang+Y%2C+Callan+J%2C+Neubig+G%2E+Pal%3A+Program%2Daided+language+models%2E+In%3A+International+Conference+on+Machine+Learning%2E+2023%2C+10764%E2%80%9310799)
68. Chen W, Ma X, Wang X, Cohen W W. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588, 2022 [find](https://scholar.google.com/scholar?q=Chen+W%2C+Ma+X%2C+Wang+X%2C+Cohen+W+W%2E+Program+of+thoughts+prompting%3A+Disentangling+computation+from+reasoning+for+numerical+reasoning+tasks%2E+arXiv+preprint+arXiv%3A2211%2E12588%2C+2022)
69. Lu P, Peng B, Cheng H, Galley M, Chang K W, Wu Y N, Zhu S C, Gao J. [Chameleon: Plug-and-play compositional reasoning with large language models](https://md.research.elamur.ai/free/en/papers/lu-2023-chameleon-plug-play-compositional-reasoning/). Advances in Neural Information Processing Systems, 2024, 36
70. Wang X, Peng H, Jabbarvand R, Ji H. Leti: Learning to generate from textual interactions. arXiv preprint arXiv:2305.10314, 2023 [find](https://scholar.google.com/scholar?q=Wang+X%2C+Peng+H%2C+Jabbarvand+R%2C+Ji+H%2E+Leti%3A+Learning+to+generate+from+textual+interactions%2E+arXiv+preprint+arXiv%3A2305%2E10314%2C+2023)
71. Wu J, Zhu R, Chen N, Sun Q, Li X, Gao M. Structure-aware fine-tuning for code pre-trained models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), 2024 [find](https://scholar.google.com/scholar?q=Wu+J%2C+Zhu+R%2C+Chen+N%2C+Sun+Q%2C+Li+X%2C+Gao+M%2E+Structure%2Daware+fine%2Dtuning+for+code+pre%2Dtrained+models%2E+In+Proceedings+of+the+2024+Joint+International+Conference+on+Computational+Linguistics%2C+Language+Resou)
72. Zhang K, Li J, Li G, Shi X, Jin Z. [Codeagent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges](https://md.research.elamur.ai/free/en/papers/zhang-2024-codeagent-enhancing-code-generation-tool/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), 2024
73. Inaba T, Kiyomaru H, Cheng F, Kurohashi S. MultiTool-CoT: GPT-3 can use multiple external tools with chain of thought prompting. In: Rogers A, Boyd-Graber J, Okazaki N, eds, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). July 2023, 1… [find](https://scholar.google.com/scholar?q=Inaba+T%2C+Kiyomaru+H%2C+Cheng+F%2C+Kurohashi+S%2E+MultiTool%2DCoT%3A+GPT%2D3+can+use+multiple+external+tools+with+chain+of+thought+prompting%2E+In%3A+Rogers+A%2C+Boyd%2DGraber+J%2C+Okazaki+N%2C+eds%2C+Proceedings+of+the+61st+An)
74. M. Bran A, Cox S, Schilter O, Baldassari C, White A D, Schwaller P. Augmenting large language models with chemistry tools. Nature Machine Intelligence, 2024 [find](https://scholar.google.com/scholar?q=M%2E+Bran+A%2C+Cox+S%2C+Schilter+O%2C+Baldassari+C%2C+White+A+D%2C+Schwaller+P%2E+Augmenting+large+language+models+with+chemistry+tools%2E+Nature+Machine+Intelligence%2C+2024)
75. Ramos M C, Collison C J, White A D. [A review of large language models and autonomous agents in chemistry](https://md.research.elamur.ai/free/en/papers/ramos-2024-review-large-language-models-autonomous/). arXiv preprint arXiv:2407.01603, 2024
76. Jin Q, Yang Y, Chen Q, Lu Z. [Genegpt: Augmenting large language models with domain tools for improved access to biomedical information](https://md.research.elamur.ai/free/en/papers/jin-2023-genegpt-augmenting-large-language-models/). Bioinformatics, 2024, 40(2): btae075
77. Theuma A, Shareghi E. Equipping language models with tool use capability for tabular data analysis in finance. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2024 [find](https://scholar.google.com/scholar?q=Theuma+A%2C+Shareghi+E%2E+Equipping+language+models+with+tool+use+capability+for+tabular+data+analysis+in+finance%2E+In+Proceedings+of+the+18th+Conference+of+the+European+Chapter+of+the+Association+for+Comp)
78. Gao S, Wen Y, Zhu M, Wei J, Cheng Y, Zhang Q, Shang S. Simulating financial market via large language model based agents. arXiv preprint arXiv:2406.19966, 2024 — [arXiv:2406.19966](https://arxiv.org/abs/2406.19966) [find](https://scholar.google.com/scholar?q=Gao+S%2C+Wen+Y%2C+Zhu+M%2C+Wei+J%2C+Cheng+Y%2C+Zhang+Q%2C+Shang+S%2E+Simulating+financial+market+via+large+language+model+based+agents%2E+arXiv+preprint+arXiv%3A2406%2E19966%2C+2024+%E2%80%94+%5BarXiv%3A2406%2E19966%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
79. hang W, Zhao L, Xia H, Sun S, Sun J, Qin M, Li X, Zhao Y, Zhao Y, Cai X, others . A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024 [find](https://scholar.google.com/scholar?q=hang+W%2C+Zhao+L%2C+Xia+H%2C+Sun+S%2C+Sun+J%2C+Qin+M%2C+Li+X%2C+Zhao+Y%2C+Zhao+Y%2C+Cai+X%2C+others+%2E+A+multimodal+foundation+agent+for+financial+trading%3A+Tool%2Daugmented%2C+diversified%2C+and+generalist%2E+In%3A+Proceedings+of+t)
80. Jin Q, Wang Z, Yang Y, Zhu Q, Wright D, Huang T, Wilbur W J, He Z, Taylor A, Chen Q, others . Agentmd: Empowering language agents for risk prediction with large-scale clinical tool learning. arXiv preprint arXiv:2402.13225, 2024 — [arXiv:2402.13225](https://arxiv.org/abs/2402.13225) [find](https://scholar.google.com/scholar?q=Jin+Q%2C+Wang+Z%2C+Yang+Y%2C+Zhu+Q%2C+Wright+D%2C+Huang+T%2C+Wilbur+W+J%2C+He+Z%2C+Taylor+A%2C+Chen+Q%2C+others+%2E+Agentmd%3A+Empowering+language+agents+for+risk+prediction+with+large%2Dscale+clinical+tool+learning%2E+arXiv+pre)
81. Li B, Yan T, Pan Y, Xu Z, Luo J, Ji R, Liu S, Dong H, Lin Z, Wang Y. Mmedagent: Learning to use medical tools with multi-modal agent. arXiv preprint arXiv:2407.02483, 2024 [find](https://scholar.google.com/scholar?q=Li+B%2C+Yan+T%2C+Pan+Y%2C+Xu+Z%2C+Luo+J%2C+Ji+R%2C+Liu+S%2C+Dong+H%2C+Lin+Z%2C+Wang+Y%2E+Mmedagent%3A+Learning+to+use+medical+tools+with+multi%2Dmodal+agent%2E+arXiv+preprint+arXiv%3A2407%2E02483%2C+2024)
82. Yang Z, Li L, Wang J, Lin K, Azarnasab E, Ahmed F, Liu Z, Liu C, Zeng M, Wang L. [Mm-react: Prompting chatgpt for multimodal reasoning and action](https://md.research.elamur.ai/free/en/papers/yang-2023-mm-react-prompting-chatgpt-multimodal/). arXiv preprint arXiv:2303.11381, 2023
83. Liu Z, He Y, Wang W, Wang W, Wang Y, Chen S, Zhang Q, Yang Y, Li Q, Yu J, others . Internchat: Solving vision-centric tasks by interacting with chatbots beyond language. arXiv preprint arXiv:2305.05662, 2023 — [arXiv:2305.05662](https://arxiv.org/abs/2305.05662) [find](https://scholar.google.com/scholar?q=Liu+Z%2C+He+Y%2C+Wang+W%2C+Wang+W%2C+Wang+Y%2C+Chen+S%2C+Zhang+Q%2C+Yang+Y%2C+Li+Q%2C+Yu+J%2C+others+%2E+Internchat%3A+Solving+vision%2Dcentric+tasks+by+interacting+with+chatbots+beyond+language%2E+arXiv+preprint+arXiv%3A2305%2E0566)
84. Gao D, Ji L, Zhou L, Lin K Q, Chen J, Fan Z, Shou M Z. Assistgpt: A general multi-modal assistant that can plan, execute, inspect, and learn. arXiv preprint arXiv:2306.08640, 2023 — [arXiv:2306.08640](https://arxiv.org/abs/2306.08640) [find](https://scholar.google.com/scholar?q=Gao+D%2C+Ji+L%2C+Zhou+L%2C+Lin+K+Q%2C+Chen+J%2C+Fan+Z%2C+Shou+M+Z%2E+Assistgpt%3A+A+general+multi%2Dmodal+assistant+that+can+plan%2C+execute%2C+inspect%2C+and+learn%2E+arXiv+preprint+arXiv%3A2306%2E08640%2C+2023+%E2%80%94+%5BarXiv%3A2306%2E08640%5D)
85. Gao Z, Du Y, Zhang X, Ma X, Han W, Zhu S C, Li Q. Clova: A closed-loop visual assistant with tool usage and update. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024 [find](https://scholar.google.com/scholar?q=Gao+Z%2C+Du+Y%2C+Zhang+X%2C+Ma+X%2C+Han+W%2C+Zhu+S+C%2C+Li+Q%2E+Clova%3A+A+closed%2Dloop+visual+assistant+with+tool+usage+and+update%2E+In+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CV)
86. Zhao L, Yang Y, Zhang K, Shao W, Zhang Y, Qiao Y, Luo P, Ji R. Diffagent: Fast and accurate text-to-image api selection with large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024 [find](https://scholar.google.com/scholar?q=Zhao+L%2C+Yang+Y%2C+Zhang+K%2C+Shao+W%2C+Zhang+Y%2C+Qiao+Y%2C+Luo+P%2C+Ji+R%2E+Diffagent%3A+Fast+and+accurate+text%2Dto%2Dimage+api+selection+with+large+language+model%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer)
87. Ma Z, Huang W, Zhang J, Gupta T, Krishna R. m&m’s: A benchmark to evaluate tool-use for multi-step multimodal tasks. arXiv preprint arXiv:2403.11085, 2024 [find](https://scholar.google.com/scholar?q=Ma+Z%2C+Huang+W%2C+Zhang+J%2C+Gupta+T%2C+Krishna+R%2E+m%26m%E2%80%99s%3A+A+benchmark+to+evaluate+tool%2Duse+for+multi%2Dstep+multimodal+tasks%2E+arXiv+preprint+arXiv%3A2403%2E11085%2C+2024)
88. Wang C, Luo W, Chen Q, Mai H, Guo J, Dong S, Li Z, Ma L, Gao S, others . Tool-lmm: A large multimodal model for tool agent learning. arXiv preprint arXiv:2401.10727, 2024 — [arXiv:2401.10727](https://arxiv.org/abs/2401.10727) [find](https://scholar.google.com/scholar?q=Wang+C%2C+Luo+W%2C+Chen+Q%2C+Mai+H%2C+Guo+J%2C+Dong+S%2C+Li+Z%2C+Ma+L%2C+Gao+S%2C+others+%2E+Tool%2Dlmm%3A+A+large+multimodal+model+for+tool+agent+learning%2E+arXiv+preprint+arXiv%3A2401%2E10727%2C+2024+%E2%80%94+%5BarXiv%3A2401%2E10727%5D%28https%3A%2F%2F)
89. Shen Y, Song K, Tan X, Li D, Lu W, Zhuang Y. [Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/). Advances in Neural Information Processing Systems, 2024, 36
90. Lyu B, Cong X, Yu H, Yang P, Qin Y, Ye Y, Lu Y, Zhang Z, Yan Y, Lin Y, others . Gitagent: Facilitating autonomous agent with github by tool extension. arXiv preprint arXiv:2312.17294, 2023 — [arXiv:2312.17294](https://arxiv.org/abs/2312.17294) [find](https://scholar.google.com/scholar?q=Lyu+B%2C+Cong+X%2C+Yu+H%2C+Yang+P%2C+Qin+Y%2C+Ye+Y%2C+Lu+Y%2C+Zhang+Z%2C+Yan+Y%2C+Lin+Y%2C+others+%2E+Gitagent%3A+Facilitating+autonomous+agent+with+github+by+tool+extension%2E+arXiv+preprint+arXiv%3A2312%2E17294%2C+2023+%E2%80%94+%5BarXiv%3A23)
91. Wei J, Wang X, Schuurmans D, Bosma M, Xia F, Chi E, Le Q V, Zhou D, others . [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 2022, 35: 24824–24837
92. Yao S, Zhao J, Yu D, Du N, Shafran I, Narasimhan K, Cao Y. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). arXiv preprint arXiv:2210.03629, 2022
93. Song Y, Xiong W, Zhu D, Li C, Wang K, Tian Y, Li S. Restgpt: Connecting large language models with real-world applications via restful apis. arXiv preprint arXiv:2306.[06624, 2023](https://md.research.elamur.ai/free/en/papers/song-2023-restgpt-connecting-large-language-models/)
94. Ruan J, Chen Y, Zhang B, Xu Z, Bao T, Du G, Shi S, Mao H, Zeng X, Zhao R. Tptu: Task planning and tool usage of large language model-based ai agents. arXiv preprint arXiv:2308.03427, 2023 — [arXiv:2308.03427](https://arxiv.org/abs/2308.03427) [find](https://scholar.google.com/scholar?q=Ruan+J%2C+Chen+Y%2C+Zhang+B%2C+Xu+Z%2C+Bao+T%2C+Du+G%2C+Shi+S%2C+Mao+H%2C+Zeng+X%2C+Zhao+R%2E+Tptu%3A+Task+planning+and+tool+usage+of+large+language+model%2Dbased+ai+agents%2E+arXiv+preprint+arXiv%3A2308%2E03427%2C+2023+%E2%80%94+%5BarXiv%3A230)
95. Zhuang Y, Chen X, Yu T, Mitra S, Bursztyn V, Rossi R A, Sarkhel S, Zhang C. Toolchain*: Efficient action space navigation in large language models with a* search. In Proceedings of the 12th ICLR, 2024 [find](https://scholar.google.com/scholar?q=Zhuang+Y%2C+Chen+X%2C+Yu+T%2C+Mitra+S%2C+Bursztyn+V%2C+Rossi+R+A%2C+Sarkhel+S%2C+Zhang+C%2E+Toolchain%2A%3A+Efficient+action+space+navigation+in+large+language+models+with+a%2A+search%2E+In+Proceedings+of+the+12th+ICLR%2C+2024)
96. Liu Z, Lai Z, Gao Z, Cui E, Li Z, Zhu X, Lu L, Chen Q, Qiao Y, Dai J, others . Controlllm: Augment language models with tools by searching on graphs. arXiv preprint arXiv:2310.17796, 2023 — [arXiv:2310.17796](https://arxiv.org/abs/2310.17796) [find](https://scholar.google.com/scholar?q=Liu+Z%2C+Lai+Z%2C+Gao+Z%2C+Cui+E%2C+Li+Z%2C+Zhu+X%2C+Lu+L%2C+Chen+Q%2C+Qiao+Y%2C+Dai+J%2C+others+%2E+Controlllm%3A+Augment+language+models+with+tools+by+searching+on+graphs%2E+arXiv+preprint+arXiv%3A2310%2E17796%2C+2023+%E2%80%94+%5BarXiv%3A231)
97. Chen Y, Lv A, Lin T E, Chen C, Wu Y, Huang F, Li Y, Yan R. Fortify the shortest stave in attention: Enhancing context awareness of large language models for effective tool use. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024 [find](https://scholar.google.com/scholar?q=Chen+Y%2C+Lv+A%2C+Lin+T+E%2C+Chen+C%2C+Wu+Y%2C+Huang+F%2C+Li+Y%2C+Yan+R%2E+Fortify+the+shortest+stave+in+attention%3A+Enhancing+context+awareness+of+large+language+models+for+effective+tool+use%2E+In+Proceedings+of+the+6)
98. Huang T, Jung D, Chen M. Planning and editing what you retrieve for enhanced tool learning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2024 [find](https://scholar.google.com/scholar?q=Huang+T%2C+Jung+D%2C+Chen+M%2E+Planning+and+editing+what+you+retrieve+for+enhanced+tool+learning%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Lin)
99. Shi Z, Gao S, Chen X, Feng Y, Yan L, Shi H, Yin D, Chen Z, Verberne S, Ren Z. Chain of tools: Large language model is an automatic multi-tool learner. arXiv preprint arXiv:2405.16533, 2024 — [arXiv:2405.16533](https://arxiv.org/abs/2405.16533) [find](https://scholar.google.com/scholar?q=Shi+Z%2C+Gao+S%2C+Chen+X%2C+Feng+Y%2C+Yan+L%2C+Shi+H%2C+Yin+D%2C+Chen+Z%2C+Verberne+S%2C+Ren+Z%2E+Chain+of+tools%3A+Large+language+model+is+an+automatic+multi%2Dtool+learner%2E+arXiv+preprint+arXiv%3A2405%2E16533%2C+2024+%E2%80%94+%5BarXiv%3A24)
100. Wu X, Shen Y, Shan C, Song K, Wang S, Zhang B, Feng J, Cheng H, Chen W, Xiong Y, others . Can graph learning improve task planning? arXiv preprint arXiv:2405.19119, 2024 — [arXiv:2405.19119](https://arxiv.org/abs/2405.19119) [find](https://scholar.google.com/scholar?q=Wu+X%2C+Shen+Y%2C+Shan+C%2C+Song+K%2C+Wang+S%2C+Zhang+B%2C+Feng+J%2C+Cheng+H%2C+Chen+W%2C+Xiong+Y%2C+others+%2E+Can+graph+learning+improve+task+planning%3F+arXiv+preprint+arXiv%3A2405%2E19119%2C+2024+%E2%80%94+%5BarXiv%3A2405%2E19119%5D%28https%3A%2F%2Fa)
101. Liu Y, Yuan Y, Wang C, Han J, Ma Y, Zhang L, Zheng N, Xu H. From summary to action: Enhancing large language models for complex tasks with open world apis. arXiv preprint arXiv:2402.18157, 2024 — [arXiv:2402.18157](https://arxiv.org/abs/2402.18157) [find](https://scholar.google.com/scholar?q=Liu+Y%2C+Yuan+Y%2C+Wang+C%2C+Han+J%2C+Ma+Y%2C+Zhang+L%2C+Zheng+N%2C+Xu+H%2E+From+summary+to+action%3A+Enhancing+large+language+models+for+complex+tasks+with+open+world+apis%2E+arXiv+preprint+arXiv%3A2402%2E18157%2C+2024+%E2%80%94+%5BarX)
102. Zheng Y, Li P, Yan M, Zhang J, Huang F, Liu Y. Budget-constrained tool learning with planning. Findings of the Association for Computational Linguistics: ACL 2024, 2024 [find](https://scholar.google.com/scholar?q=Zheng+Y%2C+Li+P%2C+Yan+M%2C+Zhang+J%2C+Huang+F%2C+Liu+Y%2E+Budget%2Dconstrained+tool+learning+with+planning%2E+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2024%2C+2024)
103. Qu C, Dai S, Wei X, Cai H, Wang S, Yin D, Xu J, Wen J R. From exploration to mastery: Enabling llms to master tools via self-driven interactions. arXiv preprint arXiv:2410.08197, 2024 — [arXiv:2410.08197](https://arxiv.org/abs/2410.08197) [find](https://scholar.google.com/scholar?q=Qu+C%2C+Dai+S%2C+Wei+X%2C+Cai+H%2C+Wang+S%2C+Yin+D%2C+Xu+J%2C+Wen+J+R%2E+From+exploration+to+mastery%3A+Enabling+llms+to+master+tools+via+self%2Ddriven+interactions%2E+arXiv+preprint+arXiv%3A2410%2E08197%2C+2024+%E2%80%94+%5BarXiv%3A2410%2E08)
104. Liang Y, Wu C, Song T, Wu W, Xia Y, Liu Y, Ou Y, Lu S, Ji L, Mao S, others . Taskmatrix. ai: Completing tasks by connecting foundation models with millions of apis. Advances in Neural Information Processing Systems, 2024 [find](https://scholar.google.com/scholar?q=Liang+Y%2C+Wu+C%2C+Song+T%2C+Wu+W%2C+Xia+Y%2C+Liu+Y%2C+Ou+Y%2C+Lu+S%2C+Ji+L%2C+Mao+S%2C+others+%2E+Taskmatrix%2E+ai%3A+Completing+tasks+by+connecting+foundation+models+with+millions+of+apis%2E+Advances+in+Neural+Information+Proc)
105. Qian C, Xiong C, Liu Z, Liu Z. Toolink: Linking toolkit creation and using through chain-of-solving on open-source model. arXiv preprint arXiv:2310.05155, 2023 [find](https://scholar.google.com/scholar?q=Qian+C%2C+Xiong+C%2C+Liu+Z%2C+Liu+Z%2E+Toolink%3A+Linking+toolkit+creation+and+using+through+chain%2Dof%2Dsolving+on+open%2Dsource+model%2E+arXiv+preprint+arXiv%3A2310%2E05155%2C+2023)
106. Kong Y, Ruan J, Chen Y, Zhang B, Bao T, Shi S, Du G, Hu X, Mao H, Li Z, Zeng X, Zhao R. Tptu-v2: Boosting task planning and tool usage of large language model-based agents in real-world systems, 2023 [find](https://scholar.google.com/scholar?q=Kong+Y%2C+Ruan+J%2C+Chen+Y%2C+Zhang+B%2C+Bao+T%2C+Shi+S%2C+Du+G%2C+Hu+X%2C+Mao+H%2C+Li+Z%2C+Zeng+X%2C+Zhao+R%2E+Tptu%2Dv2%3A+Boosting+task+planning+and+tool+usage+of+large+language+model%2Dbased+agents+in+real%2Dworld+systems%2C+2023)
107. Shen W, Li C, Chen H, Yan M, Quan X, Chen H, Zhang J, Huang F. [Small llms are weak tool learners: A multi-llm agent](https://md.research.elamur.ai/free/en/papers/shen-2024-small-llms-are-weak-tool/). arXiv preprint arXiv:2401.07324, 2024
108. Gao S, Dwivedi-Yu J, Yu P, Tan X E, Pasunuru R, Golovneva O, Sinha K, Celikyilmaz A, Bosselut A, Wang T. [Efficient tool use with chain-of-abstraction reasoning](https://md.research.elamur.ai/free/en/papers/gao-2025-efficient-tool-use-chain-abstraction/). arXiv preprint arXiv:2401.17464, 2024
109. Gui A, Li J, Dai Y, Du N, Xiao H. Look before you leap: Towards decision-aware and generalizable tool-usage for large language models. arXiv preprint arXiv:2402.16696, 2024 — [arXiv:2402.16696](https://arxiv.org/abs/2402.16696) [find](https://scholar.google.com/scholar?q=Gui+A%2C+Li+J%2C+Dai+Y%2C+Du+N%2C+Xiao+H%2E+Look+before+you+leap%3A+Towards+decision%2Daware+and+generalizable+tool%2Dusage+for+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E16696%2C+2024+%E2%80%94+%5BarXiv%3A2402%2E16696%5D%28https%3A)
110. Ge Y, Hua W, Mei K, Tan J, Xu S, Li Z, Zhang Y, others . Openagi: When llm meets domain experts. Advances in Neural Information Processing Systems, 2024, 36 [find](https://scholar.google.com/scholar?q=Ge+Y%2C+Hua+W%2C+Mei+K%2C+Tan+J%2C+Xu+S%2C+Li+Z%2C+Zhang+Y%2C+others+%2E+Openagi%3A+When+llm+meets+domain+experts%2E+Advances+in+Neural+Information+Processing+Systems%2C+2024%2C+36)
111. Wang Y, Yu J, Yao Z, Zhang J, Xie Y, Tu S, Fu Y, Feng Y, Zhang J, Zhang J, others . A solution-based llm api-using methodology for academic information seeking. arXiv preprint arXiv:2405.15165, 2024 — [arXiv:2405.15165](https://arxiv.org/abs/2405.15165) [find](https://scholar.google.com/scholar?q=Wang+Y%2C+Yu+J%2C+Yao+Z%2C+Zhang+J%2C+Xie+Y%2C+Tu+S%2C+Fu+Y%2C+Feng+Y%2C+Zhang+J%2C+Zhang+J%2C+others+%2E+A+solution%2Dbased+llm+api%2Dusing+methodology+for+academic+information+seeking%2E+arXiv+preprint+arXiv%3A2405%2E15165%2C+2024+%E2%80%94)
112. Chen S, Wang Y, Wu Y F, Chen Q G, Xu Z, Luo W, Zhang K, Zhang L. Advancing tool-augmented large language models: Integrating insights from errors in inference trees. arXiv preprint arXiv:2406.07115, 2024 — [arXiv:2406.07115](https://arxiv.org/abs/2406.07115) [find](https://scholar.google.com/scholar?q=Chen+S%2C+Wang+Y%2C+Wu+Y+F%2C+Chen+Q+G%2C+Xu+Z%2C+Luo+W%2C+Zhang+K%2C+Zhang+L%2E+Advancing+tool%2Daugmented+large+language+models%3A+Integrating+insights+from+errors+in+inference+trees%2E+arXiv+preprint+arXiv%3A2406%2E07115%2C+2)
113. Liu Z, Hoang T, Zhang J, Zhu M, Lan T, Kokane S, Tan J, Yao W, Liu Z, Feng Y, others . Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets. arXiv preprint arXiv:2406.18518, 2024 — [arXiv:2406.18518](https://arxiv.org/abs/2406.18518) [find](https://scholar.google.com/scholar?q=Liu+Z%2C+Hoang+T%2C+Zhang+J%2C+Zhu+M%2C+Lan+T%2C+Kokane+S%2C+Tan+J%2C+Yao+W%2C+Liu+Z%2C+Feng+Y%2C+others+%2E+Apigen%3A+Automated+pipeline+for+generating+verifiable+and+diverse+function%2Dcalling+datasets%2E+arXiv+preprint+arXiv%3A)
114. Sparck Jones K. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation, 1972, 28(1): 11–21 [find](https://scholar.google.com/scholar?q=Sparck+Jones+K%2E+A+statistical+interpretation+of+term+specificity+and+its+application+in+retrieval%2E+Journal+of+documentation%2C+1972%2C+28%281%29%3A+11%E2%80%9321)
115. Robertson S, Zaragoza H, others . The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 2009, 3(4) [find](https://scholar.google.com/scholar?q=Robertson+S%2C+Zaragoza+H%2C+others+%2E+The+probabilistic+relevance+framework%3A+Bm25+and+beyond%2E+Foundations+and+Trends%C2%AE+in+Information+Retrieval%2C+2009%2C+3%284%29)
116. Reimers N, Gurevych I. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, 2019
117. Xiong L, Xiong C, Li Y, Tang K F, Liu J, Bennett P, Ahmed J, Overwijk A. [Approximate nearest neighbor negative contrastive learning for dense text retrieval](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/). In Proceedings of 9th ICLR, 2021
118. Hofstätter S, Lin S C, Yang J H, Lin J, Hanbury A. [Efficiently teaching an effective dense retriever with balanced topic aware sampling](https://md.research.elamur.ai/free/en/papers/hofstatter-2021-efficiently-teaching-effective-dense-retriever/). In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 113–122
119. Izacard G, Caron M, Hosseini L, Riedel S, Bojanowski P, Joulin A, Grave E. [Unsupervised dense information retrieval with contrastive learning](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/). arXiv preprint arXiv:2112.09118, 2021
120. Gao L, Callan J. [Unsupervised corpus aware language model pre-training for dense passage retrieval](https://md.research.elamur.ai/free/en/papers/gao-2022-unsupervised-corpus-aware-language-model/). In Proceedings of the 60st Annual Meeting of the Association for Computational Linguistics (ACL), 2022
121. Yuan L, Chen Y, Wang X, Fung Y R, Peng H, Ji H. Craft: Customizing llms by creating and retrieving from specialized toolsets. In Proceedings of 12th ICLR., 2024 [find](https://scholar.google.com/scholar?q=Yuan+L%2C+Chen+Y%2C+Wang+X%2C+Fung+Y+R%2C+Peng+H%2C+Ji+H%2E+Craft%3A+Customizing+llms+by+creating+and+retrieving+from+specialized+toolsets%2E+In+Proceedings+of+12th+ICLR%2E%2C+2024)
122. Anantha R, Bandyopadhyay B, Kashi A, Mahinder S, Hill A W, Chappidi S. Protip: Progressive tool retrieval improves planning. arXiv preprint arXiv:2312.10332, 2023 — [arXiv:2312.10332](https://arxiv.org/abs/2312.10332) [find](https://scholar.google.com/scholar?q=Anantha+R%2C+Bandyopadhyay+B%2C+Kashi+A%2C+Mahinder+S%2C+Hill+A+W%2C+Chappidi+S%2E+Protip%3A+Progressive+tool+retrieval+improves+planning%2E+arXiv+preprint+arXiv%3A2312%2E10332%2C+2023+%E2%80%94+%5BarXiv%3A2312%2E10332%5D%28https%3A%2F%2Farxiv%2Eor)
123. Zheng Y, Li P, Liu W, Liu Y, Luan J, Wang B. Toolrerank: Adaptive and hierarchy-aware reranking for tool retrieval. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), 2024 [find](https://scholar.google.com/scholar?q=Zheng+Y%2C+Li+P%2C+Liu+W%2C+Liu+Y%2C+Luan+J%2C+Wang+B%2E+Toolrerank%3A+Adaptive+and+hierarchy%2Daware+reranking+for+tool+retrieval%2E+In+Proceedings+of+the+2024+Joint+International+Conference+on+Computational+Linguisti)
124. Qu C, Dai S, Wei X, Cai H, Wang S, Yin D, Xu J, Wen J R. Towards completeness-oriented tool retrieval for large language models. In: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. 2024, 1930–1940 [find](https://scholar.google.com/scholar?q=Qu+C%2C+Dai+S%2C+Wei+X%2C+Cai+H%2C+Wang+S%2C+Yin+D%2C+Xu+J%2C+Wen+J+R%2E+Towards+completeness%2Doriented+tool+retrieval+for+large+language+models%2E+In%3A+Proceedings+of+the+33rd+ACM+International+Conference+on+Information)
125. Chen Z, Zhou K, Zhang B, Gong Z, Zhao X, Wen J R. Chatcot: Tool-augmented chain-of-thought reasoning on chat-based large language models. Findings of the Association for Computational Linguistics: EMNLP 2023, 2024 [find](https://scholar.google.com/scholar?q=Chen+Z%2C+Zhou+K%2C+Zhang+B%2C+Gong+Z%2C+Zhao+X%2C+Wen+J+R%2E+Chatcot%3A+Tool%2Daugmented+chain%2Dof%2Dthought+reasoning+on+chat%2Dbased+large+language+models%2E+Findings+of+the+Association+for+Computational+Linguistics%3A+EMN)
126. Liu X, Peng Z, Yi X, Xie X, Xiang L, Liu Y, Xu D. Toolnet: Connecting large language models with massive tools via tool graph. arXiv preprint arXiv:2403.00839, 2024 — [arXiv:2403.00839](https://arxiv.org/abs/2403.00839) [find](https://scholar.google.com/scholar?q=Liu+X%2C+Peng+Z%2C+Yi+X%2C+Xie+X%2C+Xiang+L%2C+Liu+Y%2C+Xu+D%2E+Toolnet%3A+Connecting+large+language+models+with+massive+tools+via+tool+graph%2E+arXiv+preprint+arXiv%3A2403%2E00839%2C+2024+%E2%80%94+%5BarXiv%3A2403%2E00839%5D%28https%3A%2F%2Farxiv%2E)
127. Mekala D, Weston J, Lanchantin J, Raileanu R, Lomeli M, Shang J, Dwivedi-Yu J. Toolverifier: Generalization to new tools via self-verification. arXiv preprint arXiv:2402.14158, 2024 [find](https://scholar.google.com/scholar?q=Mekala+D%2C+Weston+J%2C+Lanchantin+J%2C+Raileanu+R%2C+Lomeli+M%2C+Shang+J%2C+Dwivedi%2DYu+J%2E+Toolverifier%3A+Generalization+to+new+tools+via+self%2Dverification%2E+arXiv+preprint+arXiv%3A2402%2E14158%2C+2024)
128. Qiao S, Gui H, Chen H, Zhang N. Making language models better tool learners with execution feedback. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2024 [find](https://scholar.google.com/scholar?q=Qiao+S%2C+Gui+H%2C+Chen+H%2C+Zhang+N%2E+Making+language+models+better+tool+learners+with+execution+feedback%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Association+for+Computat)
129. Du Y, Wei F, Zhang H. Anytool: Self-reflective, hierarchical agents for large-scale api calls. In Proceedings of ICML 2024, 2024 [find](https://scholar.google.com/scholar?q=Du+Y%2C+Wei+F%2C+Zhang+H%2E+Anytool%3A+Self%2Dreflective%2C+hierarchical+agents+for+large%2Dscale+api+calls%2E+In+Proceedings+of+ICML+2024%2C+2024)
130. Fore M, Singh S, Stamoulis D. Geckopt: Llm system efficiency via intent-based tool selection. arXiv preprint arXiv:2404.15804, 2024 — [arXiv:2404.15804](https://arxiv.org/abs/2404.15804) [find](https://scholar.google.com/scholar?q=Fore+M%2C+Singh+S%2C+Stamoulis+D%2E+Geckopt%3A+Llm+system+efficiency+via+intent%2Dbased+tool+selection%2E+arXiv+preprint+arXiv%3A2404%2E15804%2C+2024+%E2%80%94+%5BarXiv%3A2404%2E15804%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2404%2E15804%29)
131. Zhang Y, Cai H, Chen Y, Sun R, Zheng J. Reverse chain: A generic-rule for llms to master multi-api planning. arXiv preprint arXiv:2310.04474, 2023 [find](https://scholar.google.com/scholar?q=Zhang+Y%2C+Cai+H%2C+Chen+Y%2C+Sun+R%2C+Zheng+J%2E+Reverse+chain%3A+A+generic%2Drule+for+llms+to+master+multi%2Dapi+planning%2E+arXiv+preprint+arXiv%3A2310%2E04474%2C+2023)
132. Yuan S, Song K, Chen J, Tan X, Shen Y, Kan R, Li D, Yang D. Easytool: Enhancing llm-based agents with concise tool instruction. arXiv preprint arXiv:2401.06201, 2024 [find](https://scholar.google.com/scholar?q=Yuan+S%2C+Song+K%2C+Chen+J%2C+Tan+X%2C+Shen+Y%2C+Kan+R%2C+Li+D%2C+Yang+D%2E+Easytool%3A+Enhancing+llm%2Dbased+agents+with+concise+tool+instruction%2E+arXiv+preprint+arXiv%3A2401%2E06201%2C+2024)
133. Shi Z, Gao S, Chen X, Yan L, Shi H, Yin D, Chen Z, Ren P, Verberne S, Ren Z. Learning to use tools via cooperative and interactive agents. arXiv preprint arXiv:2403.03031, 2024 [find](https://scholar.google.com/scholar?q=Shi+Z%2C+Gao+S%2C+Chen+X%2C+Yan+L%2C+Shi+H%2C+Yin+D%2C+Chen+Z%2C+Ren+P%2C+Verberne+S%2C+Ren+Z%2E+Learning+to+use+tools+via+cooperative+and+interactive+agents%2E+arXiv+preprint+arXiv%3A2403%2E03031%2C+2024)
134. Yang R, Song L, Li Y, Zhao S, Ge Y, Li X, Shan Y. [Gpt4tools: Teaching large language model to use tools via self-instruction](https://md.research.elamur.ai/free/en/papers/yang-2023-gpt4tools-teaching-large-language-model/). Advances in Neural Information Processing Systems, 2024, 36
135. Li L, Chai Y, Wang S, Sun Y, Tian H, Zhang N, Wu H. Tool-augmented reward modeling. arXiv preprint arXiv:2310.01045, 2023 — [arXiv:2310.01045](https://arxiv.org/abs/2310.01045) [find](https://scholar.google.com/scholar?q=Li+L%2C+Chai+Y%2C+Wang+S%2C+Sun+Y%2C+Tian+H%2C+Zhang+N%2C+Wu+H%2E+Tool%2Daugmented+reward+modeling%2E+arXiv+preprint+arXiv%3A2310%2E01045%2C+2023+%E2%80%94+%5BarXiv%3A2310%2E01045%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E01045%29)
136. Wang B, Fang H, Eisner J, Van Durme B, Su Y. Llms in the imaginarium: Tool learning through simulated trial and error. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024 [find](https://scholar.google.com/scholar?q=Wang+B%2C+Fang+H%2C+Eisner+J%2C+Van+Durme+B%2C+Su+Y%2E+Llms+in+the+imaginarium%3A+Tool+learning+through+simulated+trial+and+error%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computational+Li)
137. Xu F, Shi W, Choi E. [Recomp: Improving retrieval-augmented lms with compression and selective augmentation](https://md.research.elamur.ai/free/en/papers/xu-2023-recomp-improving-retrieval-augmented-lms/). In Proceedings of 12th ICLR., 2024
138. Ye J, Li G, Gao S, Huang C, Wu Y, Li S, Fan X, Dou S, Zhang Q, Gui T, others . [Tooleyes: Fine-grained evaluation for tool learning capabilities of large language models in real-world scenarios](https://md.research.elamur.ai/free/en/papers/ye-2025-tooleyes-fine-grained-evaluation-tool/). arXiv preprint arXiv:2401.00741, 2024
139. Huang S, Zhong W, Lu J, Zhu Q, Gao J, Liu W, Hou Y, Zeng X, Wang Y, Shang L, others . Planning, creation, usage: Benchmarking llms for comprehensive tool utilization in real-world complex scenarios. arXiv preprint arXiv:2401.17167, 2024 [find](https://scholar.google.com/scholar?q=Huang+S%2C+Zhong+W%2C+Lu+J%2C+Zhu+Q%2C+Gao+J%2C+Liu+W%2C+Hou+Y%2C+Zeng+X%2C+Wang+Y%2C+Shang+L%2C+others+%2E+Planning%2C+creation%2C+usage%3A+Benchmarking+llms+for+comprehensive+tool+utilization+in+real%2Dworld+complex+scenarios%2E+a)
140. Wu M, Zhu T, Han H, Tan C, Zhang X, Chen W. Seal-tools: Self-instruct tool learning dataset for agent tuning and detailed benchmark. arXiv preprint arXiv:2405.08355, 2024 — [arXiv:2405.08355](https://arxiv.org/abs/2405.08355) [find](https://scholar.google.com/scholar?q=Wu+M%2C+Zhu+T%2C+Han+H%2C+Tan+C%2C+Zhang+X%2C+Chen+W%2E+Seal%2Dtools%3A+Self%2Dinstruct+tool+learning+dataset+for+agent+tuning+and+detailed+benchmark%2E+arXiv+preprint+arXiv%3A2405%2E08355%2C+2024+%E2%80%94+%5BarXiv%3A2405%2E08355%5D%28https%3A%2F%2F)
141. Basu K, Abdelaziz I, Chaudhury S, Dan S, Crouse M, Munawar A, Kumaravel S, Muthusamy V, Kapanipathi P, Lastras L A. Api-blend: A comprehensive corpora for training and benchmarking api llms. arXiv preprint arXiv:2402.15491, 2024 [find](https://scholar.google.com/scholar?q=Basu+K%2C+Abdelaziz+I%2C+Chaudhury+S%2C+Dan+S%2C+Crouse+M%2C+Munawar+A%2C+Kumaravel+S%2C+Muthusamy+V%2C+Kapanipathi+P%2C+Lastras+L+A%2E+Api%2Dblend%3A+A+comprehensive+corpora+for+training+and+benchmarking+api+llms%2E+arXiv+pre)
142. Shen H, Li Y, Meng D, Cai D, Qi S, Zhang L, Xu M, Ma Y. Shortcutsbench: A large-scale real-world benchmark for api-based agents. arXiv preprint arXiv:2407.00132, 2024 — [arXiv:2407.00132](https://arxiv.org/abs/2407.00132) [find](https://scholar.google.com/scholar?q=Shen+H%2C+Li+Y%2C+Meng+D%2C+Cai+D%2C+Qi+S%2C+Zhang+L%2C+Xu+M%2C+Ma+Y%2E+Shortcutsbench%3A+A+large%2Dscale+real%2Dworld+benchmark+for+api%2Dbased+agents%2E+arXiv+preprint+arXiv%3A2407%2E00132%2C+2024+%E2%80%94+%5BarXiv%3A2407%2E00132%5D%28https%3A%2F%2Farxi)
143. Wang J, Ma Z, Li Y, Zhang S, Chen C, Chen K, Le X. Gta: A benchmark for general tool agents. arXiv preprint arXiv:2407.08713, 2024 — [arXiv:2407.08713](https://arxiv.org/abs/2407.08713) [find](https://scholar.google.com/scholar?q=Wang+J%2C+Ma+Z%2C+Li+Y%2C+Zhang+S%2C+Chen+C%2C+Chen+K%2C+Le+X%2E+Gta%3A+A+benchmark+for+general+tool+agents%2E+arXiv+preprint+arXiv%3A2407%2E08713%2C+2024+%E2%80%94+%5BarXiv%3A2407%2E08713%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2407%2E08713%29)
144. Ning K, Su Y, Lv X, Zhang Y, Liu J, Liu K, Xu J. Wtu-eval: A whether-or-not tool usage evaluation benchmark for large language models. arXiv preprint arXiv:2407.12823, 2024 — [arXiv:2407.12823](https://arxiv.org/abs/2407.12823) [find](https://scholar.google.com/scholar?q=Ning+K%2C+Su+Y%2C+Lv+X%2C+Zhang+Y%2C+Liu+J%2C+Liu+K%2C+Xu+J%2E+Wtu%2Deval%3A+A+whether%2Dor%2Dnot+tool+usage+evaluation+benchmark+for+large+language+models%2E+arXiv+preprint+arXiv%3A2407%2E12823%2C+2024+%E2%80%94+%5BarXiv%3A2407%2E12823%5D%28https%3A)
145. Trivedi H, Khot T, Hartmann M, Manku R, Dong V, Li E, Gupta S, Sabharwal A, Balasubramanian N. [Appworld: A controllable world of apps and people for benchmarking interactive coding agents](https://md.research.elamur.ai/free/en/papers/trivedi-2024-appworld-controllable-world-apps-people/). arXiv preprint arXiv:2407.18901, 2024
146. Ruan Y, Dong H, Wang A, Pitis S, Zhou Y, Ba J, Dubois Y, Maddison C J, Hashimoto T. [Identifying the risks of lm agents with an lm-emulated sandbox](https://md.research.elamur.ai/free/en/papers/ruan-2023-identifying-risks-lm-agents-lm/). arXiv preprint arXiv:2309.15817, 2023
147. Farn N, Shin R. Tooltalk: Evaluating tool-usage in a conversational setting. arXiv preprint arXiv:2311.10775, 2023 — [arXiv:2311.10775](https://arxiv.org/abs/2311.10775) [find](https://scholar.google.com/scholar?q=Farn+N%2C+Shin+R%2E+Tooltalk%3A+Evaluating+tool%2Dusage+in+a+conversational+setting%2E+arXiv+preprint+arXiv%3A2311%2E10775%2C+2023+%E2%80%94+%5BarXiv%3A2311%2E10775%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2311%2E10775%29)
148. Zhong Y, Qi M, Wang R, Qiu Y, Zhang Y, Ma H. Viotgpt: Learning to schedule vision tools towards intelligent video internet of things. arXiv preprint arXiv:2312.00401, 2023 — [arXiv:2312.00401](https://arxiv.org/abs/2312.00401) [find](https://scholar.google.com/scholar?q=Zhong+Y%2C+Qi+M%2C+Wang+R%2C+Qiu+Y%2C+Zhang+Y%2C+Ma+H%2E+Viotgpt%3A+Learning+to+schedule+vision+tools+towards+intelligent+video+internet+of+things%2E+arXiv+preprint+arXiv%3A2312%2E00401%2C+2023+%E2%80%94+%5BarXiv%3A2312%2E00401%5D%28https%3A%2F)
149. Ye J, Wu Y, Gao S, Li S, Li G, Fan X, Zhang Q, Gui T, Huang X. Rotbench: A multi-level benchmark for evaluating the robustness of large language models in tool learning. arXiv preprint arXiv:2401.08326, 2024 [find](https://scholar.google.com/scholar?q=Ye+J%2C+Wu+Y%2C+Gao+S%2C+Li+S%2C+Li+G%2C+Fan+X%2C+Zhang+Q%2C+Gui+T%2C+Huang+X%2E+Rotbench%3A+A+multi%2Dlevel+benchmark+for+evaluating+the+robustness+of+large+language+models+in+tool+learning%2E+arXiv+preprint+arXiv%3A2401%2E0832)
150. Ye J, Li S, Li G, Huang C, Gao S, Wu Y, Zhang Q, Gui T, Huang X. Toolsword: Unveiling safety issues of large language models in tool learning across three stages. arXiv preprint arXiv:2402.10753, 2024 [find](https://scholar.google.com/scholar?q=Ye+J%2C+Li+S%2C+Li+G%2C+Huang+C%2C+Gao+S%2C+Wu+Y%2C+Zhang+Q%2C+Gui+T%2C+Huang+X%2E+Toolsword%3A+Unveiling+safety+issues+of+large+language+models+in+tool+learning+across+three+stages%2E+arXiv+preprint+arXiv%3A2402%2E10753%2C+2024)
151. Ma Y, Gou Z, Hao J, Xu R, Wang S, Pan L, Yang Y, Cao Y, Sun A. [Sciagent: Tool-augmented language models for scientific reasoning](https://md.research.elamur.ai/free/en/papers/ma-2024-sciagent-tool-augmented-language-models/). arXiv preprint arXiv:2402.11451, 2024
152. Guo Z, Cheng S, Wang H, Liang S, Qin Y, Li P, Liu Z, Sun M, Liu Y. Stabletoolbench: Towards stable large-scale benchmarking on tool learning of large language models. arXiv preprint arXiv:2403.07714, 2024 [find](https://scholar.google.com/scholar?q=Guo+Z%2C+Cheng+S%2C+Wang+H%2C+Liang+S%2C+Qin+Y%2C+Li+P%2C+Liu+Z%2C+Sun+M%2C+Liu+Y%2E+Stabletoolbench%3A+Towards+stable+large%2Dscale+benchmarking+on+tool+learning+of+large+language+models%2E+arXiv+preprint+arXiv%3A2403%2E07714%2C+)
153. Zhan Q, Liang Z, Ying Z, Kang D. [Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents](https://md.research.elamur.ai/free/en/papers/zhan-2024-injecagent-benchmarking-indirect-prompt-injections/). arXiv preprint arXiv:2403.02691, 2024
154. Guo Z, Huang Y, Xiong D. Ctooleval: A chinese benchmark for llm-powered agent evaluation in real-world api interactions. Findings of the Association for Computational Linguistics: ACL 2024, 2024 [find](https://scholar.google.com/scholar?q=Guo+Z%2C+Huang+Y%2C+Xiong+D%2E+Ctooleval%3A+A+chinese+benchmark+for+llm%2Dpowered+agent+evaluation+in+real%2Dworld+api+interactions%2E+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2024%2C+2024)
155. Lu J, Holleis T, Zhang Y, Aumayer B, Nan F, Bai F, Ma S, Ma S, Li M, Yin G, others . [Toolsandbox: A stateful, conversational, interactive evaluation benchmark for llm tool use capabilities](https://md.research.elamur.ai/free/en/papers/lu-2025-toolsandbox-stateful-conversational-interactive-evaluation/). arXiv preprint arXiv:2408.04682, 2024
156. Zhu M. Recall, precision and average precision. Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, 2004, 2(30): 6 [find](https://scholar.google.com/scholar?q=Zhu+M%2E+Recall%2C+precision+and+average+precision%2E+Department+of+Statistics+and+Actuarial+Science%2C+University+of+Waterloo%2C+Waterloo%2C+2004%2C+2%2830%29%3A+6)
157. Järvelin K, Kekäläinen J. Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 2002, 20(4): 422–446 [find](https://scholar.google.com/scholar?q=Ja%CC%88rvelin+K%2C+Keka%CC%88la%CC%88inen+J%2E+Cumulated+gain%2Dbased+evaluation+of+ir+techniques%2E+ACM+Transactions+on+Information+Systems+%28TOIS%29%2C+2002%2C+20%284%29%3A+422%E2%80%93446)
158. Papineni K, Roukos S, Ward T, Zhu W J. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In: Isabelle P, Charniak E, Lin D, eds, Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. July 2002, 311–318
159. Lin C Y. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In: Text summarization branches out. 2004, 74–81
160. Blackwell M, Iacus S, King G, Porro G. cem: Coarsened exact matching in stata. The Stata Journal, 2009, 9(4): 524–546 [find](https://scholar.google.com/scholar?q=Blackwell+M%2C+Iacus+S%2C+King+G%2C+Porro+G%2E+cem%3A+Coarsened+exact+matching+in+stata%2E+The+Stata+Journal%2C+2009%2C+9%284%29%3A+524%E2%80%93546)
161. Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A, others . [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 2022, 35: 27730–27744
162. Dao X Q, Le N B. Investigating the effectiveness of chatgpt in mathematical reasoning and problem solving: Evidence from the vietnamese national high school graduation examination. arXiv preprint arXiv:2306.06331, 2023 [find](https://scholar.google.com/scholar?q=Dao+X+Q%2C+Le+N+B%2E+Investigating+the+effectiveness+of+chatgpt+in+mathematical+reasoning+and+problem+solving%3A+Evidence+from+the+vietnamese+national+high+school+graduation+examination%2E+arXiv+preprint+arXi)
163. Wei T, Luan J, Liu W, Dong S, Wang B. [Cmath: can your language model pass chinese elementary school math test?](https://md.research.elamur.ai/free/en/papers/wei-2023-cmath-language-model-pass/) arXiv preprint arXiv:2306.16636, 2023
164. Chen M, Tworek J, Jun H, Yuan Q, Pinto H P d O, Kaplan J, Edwards H, Burda Y, Joseph N, Brockman G, others . [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374, 2021
165. Austin J, Odena A, Nye M, Bosma M, Michalewski H, Dohan D, Jiang E, Cai C, Terry M, Le Q, others . [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv preprint arXiv:2108.07732, 2021
166. Singh S, Fore M, Stamoulis D. Evaluating tool-augmented agents in remote sensing platforms. arXiv preprint arXiv:2405.00709, 2024 — [arXiv:2405.00709](https://arxiv.org/abs/2405.00709) [find](https://scholar.google.com/scholar?q=Singh+S%2C+Fore+M%2C+Stamoulis+D%2E+Evaluating+tool%2Daugmented+agents+in+remote+sensing+platforms%2E+arXiv+preprint+arXiv%3A2405%2E00709%2C+2024+%E2%80%94+%5BarXiv%3A2405%2E00709%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2405%2E00709%29)
167. Linardatos P, Papastefanopoulos V, Kotsiantis S. Explainable ai: A review of machine learning interpretability methods. Entropy, 2020, 23(1): 18 [find](https://scholar.google.com/scholar?q=Linardatos+P%2C+Papastefanopoulos+V%2C+Kotsiantis+S%2E+Explainable+ai%3A+A+review+of+machine+learning+interpretability+methods%2E+Entropy%2C+2020%2C+23%281%29%3A+18)
168. Zhao H, Chen H, Yang F, Liu N, Deng H, Cai H, Wang S, Yin D, Du M. [Explainability for large language models: A survey](https://md.research.elamur.ai/free/en/papers/zhao-2023-explainability-large-language-models-survey/). ACM Transactions on Intelligent Systems and Technology, 2024, 15(2): 1–38
169. Weidinger L, Mellor J, Rauh M, Griffin C, Uesato J, Huang P S, Cheng M, Glaese M, Balle B, Kasirzadeh A, others . [Ethical and social risks of harm from language models](https://md.research.elamur.ai/free/en/papers/weidinger-2021-ethical-social-risks-harm-language/). arXiv preprint arXiv:2112.04359, 2021
170. Gao T, Yen H, Yu J, Chen D. [Enabling large language models to generate text with citations](https://md.research.elamur.ai/free/en/papers/gao-2023-enabling-large-language-models-generate/). In: Bouamor H, Pino J, Bali K, eds, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. December 2023, 6465–6488
171. Sun H, Cai H, Wang B, Hou Y, Wei X, Wang S, Zhang Y, Yin D. Towards verifiable text generation with evolving memory and self-reflection. arXiv preprint arXiv:2312.09075, 2023 [find](https://scholar.google.com/scholar?q=Sun+H%2C+Cai+H%2C+Wang+B%2C+Hou+Y%2C+Wei+X%2C+Wang+S%2C+Zhang+Y%2C+Yin+D%2E+Towards+verifiable+text+generation+with+evolving+memory+and+self%2Dreflection%2E+arXiv+preprint+arXiv%3A2312%2E09075%2C+2023)
172. Wallace E, Feng S, Kandpal N, Gardner M, Singh S. Universal adversarial triggers for attacking and analyzing NLP. In: Inui K, Jiang J, Ng V, Wan X, eds, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Langua… [find](https://scholar.google.com/scholar?q=Wallace+E%2C+Feng+S%2C+Kandpal+N%2C+Gardner+M%2C+Singh+S%2E+Universal+adversarial+triggers+for+attacking+and+analyzing+NLP%2E+In%3A+Inui+K%2C+Jiang+J%2C+Ng+V%2C+Wan+X%2C+eds%2C+Proceedings+of+the+2019+Conference+on+Empirical)
173. Jin D, Jin Z, Zhou J T, Szolovits P. [Is bert really robust? a strong baseline for natural language attack on text classification and entailment](https://md.research.elamur.ai/free/en/papers/jin-2019-bert-really-robust-strong-baseline/). In: Proceedings of the AAAI conference on artificial intelligence. 2020
174. Wu F, Zhang N, Jha S, McDaniel P, Xiao C. A new era in llm security: Exploring security concerns in real-world llm-based systems. arXiv preprint arXiv:2402.18649, 2024 — [arXiv:2402.18649](https://arxiv.org/abs/2402.18649) [find](https://scholar.google.com/scholar?q=Wu+F%2C+Zhang+N%2C+Jha+S%2C+McDaniel+P%2C+Xiao+C%2E+A+new+era+in+llm+security%3A+Exploring+security+concerns+in+real%2Dworld+llm%2Dbased+systems%2E+arXiv+preprint+arXiv%3A2402%2E18649%2C+2024+%E2%80%94+%5BarXiv%3A2402%2E18649%5D%28https%3A%2F%2Farx)
175. Zhang J. Graph-toolformer: To empower llms with graph reasoning ability via prompt augmented by chatgpt. arXiv preprint arXiv:2304.11116, 2023 [find](https://scholar.google.com/scholar?q=Zhang+J%2E+Graph%2Dtoolformer%3A+To+empower+llms+with+graph+reasoning+ability+via+prompt+augmented+by+chatgpt%2E+arXiv+preprint+arXiv%3A2304%2E11116%2C+2023)
176. Li C, Yang R, Li T, Bafarassat M, Sharifi K, Bergemann D, Yang Z. Stride: A tool-assisted llm agent framework for strategic and interactive decision-making. arXiv preprint arXiv:2405.16376, 2024 — [arXiv:2405.16376](https://arxiv.org/abs/2405.16376) [find](https://scholar.google.com/scholar?q=Li+C%2C+Yang+R%2C+Li+T%2C+Bafarassat+M%2C+Sharifi+K%2C+Bergemann+D%2C+Yang+Z%2E+Stride%3A+A+tool%2Dassisted+llm+agent+framework+for+strategic+and+interactive+decision%2Dmaking%2E+arXiv+preprint+arXiv%3A2405%2E16376%2C+2024+%E2%80%94+%5Bar)
177. Huang W, Abbeel P, Pathak D, Mordatch I. [Language models as zero-shot planners: Extracting actionable knowledge for embodied agents](https://md.research.elamur.ai/free/en/papers/huang-2022-language-models-zero-shot-planners/). In: International Conference on Machine Learning. 2022, 9118–9147
178. Chern I, Chern S, Chen S, Yuan W, Feng K, Zhou C, He J, Neubig G, Liu P, others . Factool: Factuality detection in generative ai–a tool augmented framework for multi-task and multi-domain scenarios. arXiv preprint arXiv:2307.[13528, 2023](https://md.research.elamur.ai/free/en/papers/chern-2023-factool-factuality-detection-generative-ai/)
179. Xu S, Pang L, Shen H, Cheng X, Chua T s. Search-in-the-chain: Towards the accurate, credible and traceable content generation for complex knowledge-intensive tasks. In: Proceedings of the ACM Web Conference 2024. 2024 [find](https://scholar.google.com/scholar?q=Xu+S%2C+Pang+L%2C+Shen+H%2C+Cheng+X%2C+Chua+T+s%2E+Search%2Din%2Dthe%2Dchain%3A+Towards+the+accurate%2C+credible+and+traceable+content+generation+for+complex+knowledge%2Dintensive+tasks%2E+In%3A+Proceedings+of+the+ACM+Web+Conf)
180. Kim G, Baldi P, McAleer S. [Language models can solve computer tasks](https://md.research.elamur.ai/free/en/papers/kim-2023-language-models-solve-computer-tasks/). Advances in Neural Information Processing Systems, 2024, 36
181. Liu Y, Peng X, Zhang Y, Cao J, Zhang X, Cheng S, Wang X, Yin J, Du T. Tool-planner: Dynamic solution tree planning for large language model with tool clustering. arXiv preprint arXiv:2406.03807, 2024 — [arXiv:2406.03807](https://arxiv.org/abs/2406.03807) [find](https://scholar.google.com/scholar?q=Liu+Y%2C+Peng+X%2C+Zhang+Y%2C+Cao+J%2C+Zhang+X%2C+Cheng+S%2C+Wang+X%2C+Yin+J%2C+Du+T%2E+Tool%2Dplanner%3A+Dynamic+solution+tree+planning+for+large+language+model+with+tool+clustering%2E+arXiv+preprint+arXiv%3A2406%2E03807%2C+2024+)
182. Erbacher P, Falissar L, Guigue V, Soulier L. Navigating uncertainty: Optimizing api dependency for hallucination reduction in closed-book question answering. arXiv preprint arXiv:2401.01780, 2024 — [arXiv:2401.01780](https://arxiv.org/abs/2401.01780) [find](https://scholar.google.com/scholar?q=Erbacher+P%2C+Falissar+L%2C+Guigue+V%2C+Soulier+L%2E+Navigating+uncertainty%3A+Optimizing+api+dependency+for+hallucination+reduction+in+closed%2Dbook+question+answering%2E+arXiv+preprint+arXiv%3A2401%2E01780%2C+2024+%E2%80%94+%5Ba)
183. Xu Q, Li Y, Xia H, Li W. Enhancing tool retrieval with iterative feedback from large language models. arXiv preprint arXiv:2406.17465, 2024 [find](https://scholar.google.com/scholar?q=Xu+Q%2C+Li+Y%2C+Xia+H%2C+Li+W%2E+Enhancing+tool+retrieval+with+iterative+feedback+from+large+language+models%2E+arXiv+preprint+arXiv%3A2406%2E17465%2C+2024)
184. Xiao S, Liu Z, Zhang P, Muennighoff N. C-pack: Packaged resources to advance general chinese embedding, 2023 [find](https://scholar.google.com/scholar?q=Xiao+S%2C+Liu+Z%2C+Zhang+P%2C+Muennighoff+N%2E+C%2Dpack%3A+Packaged+resources+to+advance+general+chinese+embedding%2C+2023)
185. Team G, Anil R, Borgeaud S, Alayrac J B, Yu J, Soricut R, Schalkwyk J, Dai A M, Hauth A, others . [Gemini: A family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/), 2024
186. Hsieh C Y, Chen S A, Li C L, Fujii Y, Ratner A, Lee C Y, Krishna R, Pfister T. Tool documentation enables zero-shot tool-usage with large language models. arXiv preprint arXiv:2308.00675, 2023 [find](https://scholar.google.com/scholar?q=Hsieh+C+Y%2C+Chen+S+A%2C+Li+C+L%2C+Fujii+Y%2C+Ratner+A%2C+Lee+C+Y%2C+Krishna+R%2C+Pfister+T%2E+Tool+documentation+enables+zero%2Dshot+tool%2Dusage+with+large+language+models%2E+arXiv+preprint+arXiv%3A2308%2E00675%2C+2023)
187. Xu Y, Feng Y, Mu H, Hou Y, Li Y, Wang X, Zhong W, Li Z, Tu D, Zhu Q, others . Concise and precise context compression for tool-using language models. Findings of the Association for Computational Linguistics: ACL 2024, 2024 [find](https://scholar.google.com/scholar?q=Xu+Y%2C+Feng+Y%2C+Mu+H%2C+Hou+Y%2C+Li+Y%2C+Wang+X%2C+Zhong+W%2C+Li+Z%2C+Tu+D%2C+Zhu+Q%2C+others+%2E+Concise+and+precise+context+compression+for+tool%2Dusing+language+models%2E+Findings+of+the+Association+for+Computational+Ling)
188. Shen Y, Song K, Tan X, Zhang W, Ren K, Yuan S, Lu W, Li D, Zhuang Y. Taskbench: Benchmarking large language models for task automation. arXiv preprint arXiv:2311.18760, 2023 — [arXiv:2311.18760](https://arxiv.org/abs/2311.18760) [find](https://scholar.google.com/scholar?q=Shen+Y%2C+Song+K%2C+Tan+X%2C+Zhang+W%2C+Ren+K%2C+Yuan+S%2C+Lu+W%2C+Li+D%2C+Zhuang+Y%2E+Taskbench%3A+Benchmarking+large+language+models+for+task+automation%2E+arXiv+preprint+arXiv%3A2311%2E18760%2C+2023+%E2%80%94+%5BarXiv%3A2311%2E18760%5D%28https)
189. Wang H, Wang H, Wang L, Hu M, Wang R, Xue B, Lu H, Mi F, Wong K F. Tpe: Towards better compositional reasoning over conceptual tools with multi-persona collaboration. arXiv preprint arXiv:2309.16090, 2023 — [arXiv:2309.16090](https://arxiv.org/abs/2309.16090) [find](https://scholar.google.com/scholar?q=Wang+H%2C+Wang+H%2C+Wang+L%2C+Hu+M%2C+Wang+R%2C+Xue+B%2C+Lu+H%2C+Mi+F%2C+Wong+K+F%2E+Tpe%3A+Towards+better+compositional+reasoning+over+conceptual+tools+with+multi%2Dpersona+collaboration%2E+arXiv+preprint+arXiv%3A2309%2E16090%2C+)
190. Qian C, Han C, Fung Y, Qin Y, Liu Z, Ji H. [Creator: Tool creation for disentangling abstract and concrete reasoning of large language models](https://md.research.elamur.ai/free/en/papers/qian-2023-creator-tool-creation-disentangling-abstract/). In: Findings of the Association for Computational Linguistics: EMNLP 2023. 2023, 6922–6939
191. Jacovi A, Caciularu A, Herzig J, Aharoni R, Bohnet B, Geva M. A comprehensive evaluation of tool-assisted generation strategies. Findings of the Association for Computational Linguistics: EMNLP 2023, 2023 [find](https://scholar.google.com/scholar?q=Jacovi+A%2C+Caciularu+A%2C+Herzig+J%2C+Aharoni+R%2C+Bohnet+B%2C+Geva+M%2E+A+comprehensive+evaluation+of+tool%2Dassisted+generation+strategies%2E+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2023%2C+)
192. Nathani D, Wang D, Pan L, Wang W Y. Maf: Multi-aspect feedback for improving reasoning in large language models. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023 [find](https://scholar.google.com/scholar?q=Nathani+D%2C+Wang+D%2C+Pan+L%2C+Wang+W+Y%2E+Maf%3A+Multi%2Daspect+feedback+for+improving+reasoning+in+large+language+models%2E+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C)
193. Yao Y, Duan J, Xu K, Cai Y, Sun Z, Zhang Y. [A survey on large language model (llm) security and privacy: The good, the bad, and the ugly](https://md.research.elamur.ai/free/en/papers/yao-2023-survey-large-language-model-llm/). High-Confidence Computing, 2024, 100211
194. Cui T, Wang Y, Fu C, Xiao Y, Li S, Deng X, Liu Y, Zhang Q, Qiu Z, Li P, others . Risk taxonomy, mitigation, and assessment benchmarks of large language model systems. arXiv preprint arXiv:2401.05778, 2024 — [arXiv:2401.05778](https://arxiv.org/abs/2401.05778) [find](https://scholar.google.com/scholar?q=Cui+T%2C+Wang+Y%2C+Fu+C%2C+Xiao+Y%2C+Li+S%2C+Deng+X%2C+Liu+Y%2C+Zhang+Q%2C+Qiu+Z%2C+Li+P%2C+others+%2E+Risk+taxonomy%2C+mitigation%2C+and+assessment+benchmarks+of+large+language+model+systems%2E+arXiv+preprint+arXiv%3A2401%2E05778%2C+)
195. Das B C, Amini M H, Wu Y. [Security and privacy challenges of large language models: A survey](https://md.research.elamur.ai/free/en/papers/das-2024-security-privacy-challenges-large-language/). arXiv preprint arXiv:2402.00888, 2024
196. Qin Y, Cai Z, Jin D, Yan L, Liang S, Zhu K, Lin Y, Han X, Ding N, Wang H, Xie R, Qi F, Liu Z, Sun M, Zhou J. Webcpm: Interactive web search for chinese long-form question answering. In: Proceedings of ACL 2023. 2023 [find](https://scholar.google.com/scholar?q=Qin+Y%2C+Cai+Z%2C+Jin+D%2C+Yan+L%2C+Liang+S%2C+Zhu+K%2C+Lin+Y%2C+Han+X%2C+Ding+N%2C+Wang+H%2C+Xie+R%2C+Qi+F%2C+Liu+Z%2C+Sun+M%2C+Zhou+J%2E+Webcpm%3A+Interactive+web+search+for+chinese+long%2Dform+question+answering%2E+In%3A+Proceedings+of)
197. Miao X, Oliaro G, Zhang Z, Cheng X, Jin H, Chen T, Jia Z. [Towards efficient generative large language model serving: A survey from algorithms to systems](https://md.research.elamur.ai/free/en/papers/miao-2023-towards-efficient-generative-large-language/). arXiv preprint arXiv:2312.15234, 2023
198. Cai T, Wang X, Ma T, Chen X, Zhou D. [Large language models as tool makers](https://md.research.elamur.ai/free/en/papers/cai-2023-large-language-models-tool-makers/). In Proceedings of the 12th ICLR., 2024
199. Wang Z, Fried D, Neubig G. Trove: Inducing verifiable and efficient toolboxes for solving programmatic tasks. In Proceedings of ICML 2024, 2024 Changle Qu is currently pursuing the Ph.D. degree at Gaoling School of Artificial Intelligence, Renmin University of China. His current research interests m… [find](https://scholar.google.com/scholar?q=Wang+Z%2C+Fried+D%2C+Neubig+G%2E+Trove%3A+Inducing+verifiable+and+efficient+toolboxes+for+solving+programmatic+tasks%2E+In+Proceedings+of+ICML+2024%2C+2024+Changle+Qu+is+currently+pursuing+the+Ph%2ED%2E+degree+at+Gao)

*Updated 2 November 2026*