---
title: "NesTools — paper by Han et al. (2025) — summary and ideas"
description: "The authors build an automatic generation pipeline that exploits LLMs' familiarity with code. 27 references, cited by 7 papers here."
authors: ["Han Han", "Tong Zhu", "Xiang Zhang", "MengSong Wu", "Xiong Hao", "Wenliang Chen"]
year: 2025
venue: "Proceedings of the 31st International Conference on Computational Linguistics"
source: "https://aclanthology.org/2025.coling-main.657/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/han-2025-nestools-dataset-evaluating-nested-tool/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/han-2025-nestools-dataset-evaluating-nested-tool/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/10/) / NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of Large Language Models

# NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of Large Language Models

**Authors:** Han Han, [Tong Zhu](https://md.research.elamur.ai/free/en/authors/tong-zhu/), [Xiang Zhang](https://md.research.elamur.ai/free/en/authors/xiang-zhang/), MengSong Wu, Xiong Hao and [Wenliang Chen](https://md.research.elamur.ai/free/en/authors/wenliang-chen/) · **Year:** 2025 · **Venue:** [Proceedings of the 31st International Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2025](https://md.research.elamur.ai/free/en/venues/coling/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.coling-main.657/) · [PDF](https://aclanthology.org/2025.coling-main.657.pdf) · [OpenAlex](https://openalex.org/W4403575298)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/han-2025-nestools-dataset-evaluating-nested-tool/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
NesTools tackles a gap in tool-learning evaluation for large language models: nested tool calls, where one tool's output feeds the parameters of a later call. Existing benchmarks rarely contain such cases or judge them coarsely. The authors build an automatic generation pipeline that exploits LLMs' familiarity with code. GPT-3.5 writes Python-style tool definitions and a call chain in which return values are named variables passed to later functions, covering domains taken from Seal-Tools. GPT-3.5 then drafts a matching user query, and GPT-4 rewrites it. Each candidate passes AST and format checks, followed by annotator and expert review. The resulting benchmark contains 1,000 verified instances. Models are scored deterministically along four dimensions: choosing the right tools, invoking them in the correct sequence, supplying correct argument values, and correctly supplying the arguments that come from earlier calls' outputs (nested arguments). Tests on 22 proprietary and open-weight LLMs show three main patterns. Larger models do better, but even basic tool selection remains error-prone. Performance also falls as nesting gets deeper or more structurally complex. The resource gives a fine-grained way to diagnose multi-step tool use.

## Key points
- **Dataset scale:** NesTools has 3,034 tools, averaging 2.24 required parameters each. Of its 1,000 instances, 830 involve nested calls, with about 3.04 tool calls per instance. Domain coverage comes from Seal-Tools, which spans 146 domains and 5,860 sub-domains.
- **Generation pipeline:** Call chains follow a `data0, ... = func1(...)` pattern so later calls can reference specific return values. Tools are sampled at temperature 0.95, and in-context examples are rotated to increase diversity.
- **Nesting structure effects:** Shallow nesting yields higher scores. Chain-like structures that add an extra cross-link, such as 1→2→3 combined with 1→3, score lowest.
- **Robustness tests on LLaMA3.1-8B:**
  - Masking tool return types causes only small drops.
  - Removing connective words like "then" from queries hurts much more, especially the Order metric.
- **Error analysis (Qwen2-7B vs LLaMA3.1-8B):**
  - Qwen2-7B often treats nested parameters as ordinary ones.
  - LLaMA3.1-8B tends to recognize nesting but fills in wrong values.
  - Smaller models more often produce output in unparseable formats.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Soochow University](https://md.research.elamur.ai/free/en/universities/soochow-university/) — University, Suzhou, China · 63 works · 4,507 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Han Han, Tong Zhu, Xiang Zhang, MengSong Wu, Xiong Hao and Wenliang Chen. 2025. NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of Large Language Models. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{han2025nestools,
  title = {{NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of Large Language Models}},
  author = {Han Han, Tong Zhu, Xiang Zhang, MengSong Wu, Xiong Hao and Wenliang Chen},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.657/}
}
```

- [← previous MPO: Multilingual Safety Alignment via Reward Gap Optimization](https://md.research.elamur.ai/free/en/papers/zhao-2025-mpo-multilingual-safety-alignment-reward/)
- [next → NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark](https://md.research.elamur.ai/free/en/papers/mikhailov-2025-noreval-norwegian-language-understanding-generation/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/10/)

## References

27 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). ArXiv preprint, abs/2303.08774.
2. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. [Qwen technical report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/). ArXiv preprint, abs/2309.16609.
3. Kinjal Basu, Ibrahim Abdelaziz, Subhajit Chaudhury, Soham Dan, Maxwell Crouse, Asim Munawar, Vernon Austel, Sadhana Kumaravel, Vinod Muthusamy, Pavan Kapanipathi, and Luis Lastras. 2024. API-BLEND: A comprehensive corpora for training and benchmarking API LLMs. In Proceedings of the 62nd Annual Meet… [find](https://scholar.google.com/scholar?q=Kinjal+Basu%2C+Ibrahim+Abdelaziz%2C+Subhajit+Chaudhury%2C+Soham+Dan%2C+Maxwell+Crouse%2C+Asim+Munawar%2C+Vernon+Austel%2C+Sadhana+Kumaravel%2C+Vinod+Muthusamy%2C+Pavan+Kapanipathi%2C+and+Luis+Lastras%2E+2024%2E+API%2DBLEND%3A+A+)
4. Zehui Chen, Weihua Du, Wenwei Zhang, Kuikun Liu, Jiangning Liu, Miao Zheng, Jingming Zhuo, Songyang Zhang, Dahua Lin, Kai Chen, and Feng Zhao. 2024. T-eval: Evaluating the tool utilization capability of large language models step by step. In Proceedings of the 62nd Annual Meeting of the Association… [find](https://scholar.google.com/scholar?q=Zehui+Chen%2C+Weihua+Du%2C+Wenwei+Zhang%2C+Kuikun+Liu%2C+Jiangning+Liu%2C+Miao+Zheng%2C+Jingming+Zhuo%2C+Songyang+Zhang%2C+Dahua+Lin%2C+Kai+Chen%2C+and+Feng+Zhao%2E+2024%2E+T%2Deval%3A+Evaluating+the+tool+utilization+capability+)
5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). ArXiv preprint, abs/2407.21783.
6. Shijue Huang, Wanjun Zhong, Jianqiao Lu, Qi Zhu, Jiahui Gao, Weiwen Liu, Yutai Hou, Xingshan Zeng, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruifeng Xu, and Qun Liu. 2024. Planning, creation, usage: Benchmarking LLMs for comprehensive tool utilization in real-world complex scenarios. In Findings of the… [find](https://scholar.google.com/scholar?q=Shijue+Huang%2C+Wanjun+Zhong%2C+Jianqiao+Lu%2C+Qi+Zhu%2C+Jiahui+Gao%2C+Weiwen+Liu%2C+Yutai+Hou%2C+Xingshan+Zeng%2C+Yasheng+Wang%2C+Lifeng+Shang%2C+Xin+Jiang%2C+Ruifeng+Xu%2C+and+Qun+Liu%2E+2024%2E+Planning%2C+creation%2C+usage%3A+Benc)
7. Yue Huang, Jiawen Shi, Yuan Li, Chenrui Fan, Siyuan Wu, Qihui Zhang, Yixin Liu, Pan Zhou, Yao Wan, Neil Zhenqiang Gong, et al. 2023. Metatool benchmark for large language models: Deciding whether to use tools and which to use. ArXiv preprint, abs/2310.03128. [find](https://scholar.google.com/scholar?q=Yue+Huang%2C+Jiawen+Shi%2C+Yuan+Li%2C+Chenrui+Fan%2C+Siyuan+Wu%2C+Qihui+Zhang%2C+Yixin+Liu%2C+Pan+Zhou%2C+Yao+Wan%2C+Neil+Zhenqiang+Gong%2C+et+al%2E+2023%2E+Metatool+benchmark+for+large+language+models%3A+Deciding+whether+to+u)
8. Tatsuro Inaba, Hirokazu Kiyomaru, Fei Cheng, and Sadao Kurohashi. 2023. MultiTool-CoT: GPT-3 can use multiple external tools with chain of thought prompting. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1522–1532, Toronto,… [find](https://scholar.google.com/scholar?q=Tatsuro+Inaba%2C+Hirokazu+Kiyomaru%2C+Fei+Cheng%2C+and+Sadao+Kurohashi%2E+2023%2E+MultiTool%2DCoT%3A+GPT%2D3+can+use+multiple+external+tools+with+chain+of+thought+prompting%2E+In+Proceedings+of+the+61st+Annual+Meeting+)
9. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. ArXiv preprint, abs/2310.06825. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+et+al%2E+2023%2E+Mist)
10. Yongliang Shen, Kaitao Song, Xu Tan, Wenqi Zhang, Kan Ren, Siyu Yuan, Weiming Lu, Dongsheng Li, and Yueting Zhuang. 2023. Taskbench: Benchmarking large language models for task automation. ArXiv preprint, abs/2311.18760. — [arXiv:2311.18760](https://arxiv.org/abs/2311.18760) [find](https://scholar.google.com/scholar?q=Yongliang+Shen%2C+Kaitao+Song%2C+Xu+Tan%2C+Wenqi+Zhang%2C+Kan+Ren%2C+Siyu+Yuan%2C+Weiming+Lu%2C+Dongsheng+Li%2C+and+Yueting+Zhuang%2E+2023%2E+Taskbench%3A+Benchmarking+large+language+models+for+task+automation%2E+ArXiv+prepr)
11. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. ArXiv preprint, abs/2401.04088. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+et+al%2E+2024%2E+Mixtral+o)
12. Yifan Song, Weimin Xiong, Dawei Zhu, Wenhao Wu, Han Qian, Mingbo Song, Hailiang Huang, Cheng Li, Ke Wang, Rong Yao, et al. 2023. [Restgpt: Connecting large language models with real-world restful apis](https://md.research.elamur.ai/free/en/papers/song-2023-restgpt-connecting-large-language-models/). ArXiv preprint, abs/2306.06624.
13. Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. [API-bank: A comprehensive benchmark for tool-augmented LLMs](https://md.research.elamur.ai/free/en/papers/li-2023-api-bank-comprehensive-benchmark-tool/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3102–3116, Singapore. Ass…
14. Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, Boxi Cao, and Le Sun. 2023. [Toolalpaca: Generalized tool learning for language models with 3000 simulated cases](https://md.research.elamur.ai/free/en/papers/tang-2023-toolalpaca-generalized-tool-learning-language/). ArXiv preprint, abs/2306.05301.
15. Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. 2023. Art: Automatic multi-step reasoning and tool-use for large language models. ArXiv preprint, abs/2303.09014. [find](https://scholar.google.com/scholar?q=Bhargavi+Paranjape%2C+Scott+Lundberg%2C+Sameer+Singh%2C+Hannaneh+Hajishirzi%2C+Luke+Zettlemoyer%2C+and+Marco+Tulio+Ribeiro%2E+2023%2E+Art%3A+Automatic+multi%2Dstep+reasoning+and+tool%2Duse+for+large+language+models%2E+ArXi)
16. Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. 2023. [Gorilla: Large language model connected with massive apis](https://md.research.elamur.ai/free/en/papers/patil-2023-gorilla-large-language-model-connected/). ArXiv preprint, abs/2305.15334.
17. Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, Yi Ren Fung, Yusheng Su, Huadong Wang, Cheng Qian, Runchu Tian, Kunlun Zhu, Shihao Liang, Xingyu Shen, Bokai Xu, Zhen Zhang, Yining Ye, Bowen Li, Ziwei Tang, Jing Yi, Yuzhang Zhu, Z… — [Tool Learning with Foundation Models](https://md.research.elamur.ai/free/en/papers/qin-2023-tool-learning-foundation-models/)
18. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023b. [Toolllm: Facilitating large language models to master 16000+ real-world apis](https://md.research.elamur.ai/free/en/papers/qin-2023-toolllm-facilitating-large-language-models/). ArXiv preprint, abs/2307.16789.
19. Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2024. Tool learning with [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). ArXiv preprint, abs/2405.17935.
20. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. [Toolformer: Language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). In Advances in Neural Information Processing Systems 36: Annual Conference on Neural I…
21. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+In+Proc)
22. Mengsong Wu, Tong Zhu, Han Han, Chuanyuan Tan, Xiang Zhang, and Wenliang Chen. 2024. Seal-tools: Self-instruct tool learning dataset for agent tuning and detailed benchmark. ArXiv preprint, abs/2405.08355. — [arXiv:2405.08355](https://arxiv.org/abs/2405.08355) [find](https://scholar.google.com/scholar?q=Mengsong+Wu%2C+Tong+Zhu%2C+Han+Han%2C+Chuanyuan+Tan%2C+Xiang+Zhang%2C+and+Wenliang+Chen%2E+2024%2E+Seal%2Dtools%3A+Self%2Dinstruct+tool+learning+dataset+for+agent+tuning+and+detailed+benchmark%2E+ArXiv+preprint%2C+abs%2F2405%2E0)
23. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. ArXiv preprint, abs/2407.10671. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+et+al%2E+2024%2E+Qwen2+technical+report%2E+ArXiv+preprint%2C+abs%2F2407%2E10671%2E)
24. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
25. Junjie Ye, Guanyu Li, Songyang Gao, Caishuang Huang, Yilong Wu, Sixian Li, Xiaoran Fan, Shihan Dou, Qi Zhang, Tao Gui, et al. 2024. [Tooleyes: Fine-grained evaluation for tool learning capabilities of large language models in real-world scenarios](https://md.research.elamur.ai/free/en/papers/ye-2025-tooleyes-fine-grained-evaluation-tool/). ArXiv preprint, abs/2401.00741.
26. Junjie Yu, Xing Wang, and Wenliang Chen. 2024. Reliable data generation and selection for low-resource relation extraction. In AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Junjie+Yu%2C+Xing+Wang%2C+and+Wenliang+Chen%2E+2024%2E+Reliable+data+generation+and+selection+for+low%2Dresource+relation+extraction%2E+In+AAAI+Conference+on+Artificial+Intelligence%2E)
27. Tong Zhu, Xiaoye Qu, Daize Dong, Jiacheng Ruan, Jingqi Tong, Conghui He, and Yu Cheng. 2024. [Llama-moe: Building mixture-of-experts from llama with continual pre-training](https://md.research.elamur.ai/free/en/papers/zhu-2024-llama-moe-building-mixture-experts/). ArXiv preprint, abs/2406.16554.

*Updated 30 October 2026*