---
title: "ToolSafe — paper by Mou et al. (2026) — summary and ideas"
description: "Finally, they proposed TS-Flow, which feeds TS-Guard's verdicts back into a ReAct-style agent so it can… 53 references, cited by 1 paper here."
authors: ["Yutao Mou", "Zhangchi Xue", "Lijun Li", "Peiyang Liu", "Shikun Zhang", "Wei Ye", "Jing Shao"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1850/"
doi: "10.18653/v1/2026.findings-acl.1850"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mou-2026-toolsafe-enhancing-tool-invocation-safety/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mou-2026-toolsafe-enhancing-tool-invocation-safety/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/) / ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback

# ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback

**Authors:** [Yutao Mou](https://md.research.elamur.ai/free/en/authors/yutao-mou/), Zhangchi Xue, [Lijun Li](https://md.research.elamur.ai/free/en/authors/lijun-li/), [Peiyang Liu](https://md.research.elamur.ai/free/en/authors/peiyang-liu/), [Shikun Zhang](https://md.research.elamur.ai/free/en/authors/shikun-zhang/), [Wei Ye](https://md.research.elamur.ai/free/en/authors/wei-ye/) and [Jing Shao](https://md.research.elamur.ai/free/en/authors/jing-shao/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1850/) · [PDF](https://aclanthology.org/2026.findings-acl.1850.pdf) · [DOI 10.18653/v1/2026.findings-acl.1850](https://doi.org/10.18653/v1/2026.findings-acl.1850) · [OpenAlex](https://openalex.org/W7166886473)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
LLM agents that call external tools can be steered into harmful actions, either by malicious users or by prompt injections hidden in tool outputs. Most existing guardrails either moderate static text or judge whole trajectories after the fact. This paper targets a different point in the process: checking each individual tool call before it runs. The authors first studied logs from several agent-safety datasets and found four recurring risk patterns. From these they built TS-Bench, a step-level benchmark for detecting unsafe tool invocations. They then trained TS-Guard, a guardrail model optimized with multi-task reinforcement learning. For each candidate action, TS-Guard outputs a short analysis, a judgment of whether the request is malicious, a judgment of whether the agent is under attack, and a harmfulness rating. Finally, they proposed TS-Flow, which feeds TS-Guard's verdicts back into a ReAct-style agent so it can reason its way to a safer action instead of being halted. Across experiments, TS-Flow cut harmful tool calls by about 65% on average and raised benign task success by roughly 10% under injection attacks. This makes real-time, pre-execution agent protection practical.

## Key points
- **Four unsafe patterns:** the taxonomy combines two causes (Malicious User Request, Prompt Injection) with two manifestations (Harmful Tools, Benign Tools with Risky Arguments).
- **TS-Bench data:**
  - The evaluation split draws on AgentHarm-Traj (731 samples), ASB-Traj (5237) and AgentDojo-Traj (1220).
  - The training split uses AgentAlign-Traj (673) and ASB-Traj (1520).
  - Labels are safe, controversial or unsafe.
- **AgentDojo results with TS-Flow:**
  - With GPT-4o, attack success rate (ASR) dropped from 17.59% to 1.79%.
  - With Qwen2.5-14B-Instruct, ASR fell from 56.16% to 1.16%, and utility rose from 26.87% to 42.78%.
  - Agents also needed fewer interaction steps.
- **Overhead:** input context grew by 36% for GPT-4o and 89% for Qwen2.5-14B, staying below a doubling. Guardrail feedback fired about once per task.
- **Value of richer feedback:** passing the full TS-Guard output rather than just the safety rating lowered ASR on ASB-OPI from 26.04% to 7.00%. The authors also report that guardrail feedback raises agent token entropy, which encourages exploration of safer actions.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) — University, Beijing, China · 244 works · 39,359 citations
- [Shanghai Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/shanghai-artificial-intelligence-laboratory/) — Research institute, Shanghai, China · 117 works · 11,281 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works

🧭 Topics (1): [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Yutao Mou, Zhangchi Xue, Lijun Li, Peiyang Liu, Shikun Zhang, Wei Ye and Jing Shao. 2026. ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mou2026toolsafe,
  title = {{ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback}},
  author = {Yutao Mou, Zhangchi Xue, Lijun Li, Peiyang Liu, Shikun Zhang, Wei Ye and Jing Shao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1850},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1850}
}
```

- [← previous ToolGate: Contract-Grounded and Verified Tool Execution for LLMs](https://md.research.elamur.ai/free/en/papers/liu-2026-toolgate-contract-grounded-verified-tool/)
- [next → TreeDiff: AST-Guided Code Generation with Diffusion LLMs](https://md.research.elamur.ai/free/en/papers/zeng-2026-treediff-ast-guided-code-generation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/)

## References

53 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hengyu An, Jinghuai Zhang, Tianyu Du, Chunyi Zhou, Qingming Li, Tao Lin, and Shouling Ji. 2025. Ipiguard: A novel tool dependency graph-based defense against indirect prompt injection in llm agents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 1023… [find](https://scholar.google.com/scholar?q=Hengyu+An%2C+Jinghuai+Zhang%2C+Tianyu+Du%2C+Chunyi+Zhou%2C+Qingming+Li%2C+Tao+Lin%2C+and+Shouling+Ji%2E+2025%2E+Ipiguard%3A+A+novel+tool+dependency+graph%2Dbased+defense+against+indirect+prompt+injection+in+llm+agents%2E+I)
2. Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrikson, and 1 others. 2024. Agentharm: A benchmark for measuring harmfulness of llm agents. arXiv preprint arXiv:2410.09024. — [arXiv:2410.09024](https://arxiv.org/abs/2410.09024) [find](https://scholar.google.com/scholar?q=Maksym+Andriushchenko%2C+Alexandra+Souly%2C+Mateusz+Dziemian%2C+Derek+Duenas%2C+Maxwell+Lin%2C+Justin+Wang%2C+Dan+Hendrycks%2C+Andy+Zou%2C+Zico+Kolter%2C+Matt+Fredrikson%2C+and+1+others%2E+2024%2E+Agentharm%3A+A+benchmark+for+)
3. Zhaorun Chen, Mintong Kang, and Bo Li. 2025. Shieldagent: Shielding agents via verifiable safety policy reasoning. arXiv preprint arXiv:2503.22738. — [arXiv:2503.22738](https://arxiv.org/abs/2503.22738) [find](https://scholar.google.com/scholar?q=Zhaorun+Chen%2C+Mintong+Kang%2C+and+Bo+Li%2E+2025%2E+Shieldagent%3A+Shielding+agents+via+verifiable+safety+policy+reasoning%2E+arXiv+preprint+arXiv%3A2503%2E22738%2E+%E2%80%94+%5BarXiv%3A2503%2E22738%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2503%2E2273)
4. Zhi-Yuan Chen, Shiqi Shen, Guangyao Shen, Gong Zhi, Xu Chen, and Yankai Lin. 2024. Towards tool use alignment of large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1382–1400, Miami, Florida, USA. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Zhi%2DYuan+Chen%2C+Shiqi+Shen%2C+Guangyao+Shen%2C+Gong+Zhi%2C+Xu+Chen%2C+and+Yankai+Lin%2E+2024%2E+Towards+tool+use+alignment+of+large+language+models%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Na)
5. Sahana Chennabasappa, Cyrus Nikolaidis, Daniel Song, David Molnar, Stephanie Ding, Shengye Wan, Spencer Whitman, Lauren Deason, Nicholas Doucette, Abraham Montilla, and 1 others. 2025. Llamafirewall: An open source guardrail system for building secure ai agents. arXiv preprint arXiv:2505.03574. — [arXiv:2505.03574](https://arxiv.org/abs/2505.03574) [find](https://scholar.google.com/scholar?q=Sahana+Chennabasappa%2C+Cyrus+Nikolaidis%2C+Daniel+Song%2C+David+Molnar%2C+Stephanie+Ding%2C+Shengye+Wan%2C+Spencer+Whitman%2C+Lauren+Deason%2C+Nicholas+Doucette%2C+Abraham+Montilla%2C+and+1+others%2E+2025%2E+Llamafirewall%3A+)
6. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. 2025. [Sft memorizes, rl generalizes: A comparative study of foundation model post-training](https://md.research.elamur.ai/free/en/papers/chu-2025-sft-memorizes-rl-generalizes-comparative/). arXiv preprint arXiv:2501.17161.
7. Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, and 1 others. 2025. The entropy mechanism of reinforcement learning for reasoning language models. arXiv preprint arXiv:2505.22617. [find](https://scholar.google.com/scholar?q=Ganqu+Cui%2C+Yuchen+Zhang%2C+Jiacheng+Chen%2C+Lifan+Yuan%2C+Zhi+Wang%2C+Yuxin+Zuo%2C+Haozhan+Li%2C+Yuchen+Fan%2C+Huayu+Chen%2C+Weize+Chen%2C+and+1+others%2E+2025%2E+The+entropy+mechanism+of+reinforcement+learning+for+reasoni)
8. Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. 2024. Agentdojo: A dynamic environment to evaluate attacks and defenses for llm agents. arXiv e-prints, pages arXiv–2406. [find](https://scholar.google.com/scholar?q=Edoardo+Debenedetti%2C+Jie+Zhang%2C+Mislav+Balunovic%CC%81%2C+Luca+Beurer%2DKellner%2C+Marc+Fischer%2C+and+Florian+Tram%C3%A8r%2E+2024%2E+Agentdojo%3A+A+dynamic+environment+to+evaluate+attacks+and+defenses+for+llm+agents%2E+arXiv+)
9. Ivan Evtimov, Arman Zharmagambetov, Aaron Grattafiori, Chuan Guo, and Kamalika Chaudhuri. 2025. Wasp: Benchmarking web agent security against prompt injection attacks. arXiv preprint arXiv:2504.18575. — [arXiv:2504.18575](https://arxiv.org/abs/2504.18575) [find](https://scholar.google.com/scholar?q=Ivan+Evtimov%2C+Arman+Zharmagambetov%2C+Aaron+Grattafiori%2C+Chuan+Guo%2C+and+Kamalika+Chaudhuri%2E+2025%2E+Wasp%3A+Benchmarking+web+agent+security+against+prompt+injection+attacks%2E+arXiv+preprint+arXiv%3A2504%2E18575%2E)
10. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
11. Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. 2024. [Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms](https://md.research.elamur.ai/free/en/papers/han-2024-wildguard-open-one-stop-moderation/). Advances in Neural Information Processing Systems, 37:8093–8131.
12. Yihao Huang, Chong Wang, Xiaojun Jia, Qing Guo, Felix Juefei-Xu, Jian Zhang, Yang Liu, and Geguang Pu. 2025a. Efficient universal goal hijacking with semantics-guided prompt organization. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper… [find](https://scholar.google.com/scholar?q=Yihao+Huang%2C+Chong+Wang%2C+Xiaojun+Jia%2C+Qing+Guo%2C+Felix+Juefei%2DXu%2C+Jian+Zhang%2C+Yang+Liu%2C+and+Geguang+Pu%2E+2025a%2E+Efficient+universal+goal+hijacking+with+semantics%2Dguided+prompt+organization%2E+In+Proceedin)
13. Yue Huang, Hang Hua, Yujun Zhou, Pengcheng Jing, Manish Nagireddy, Inkit Padhi, Greta Dolcetti, Zhangchen Xu, Subhajit Chaudhury, Ambrish Rawat, and 1 others. 2025b. Building a foundational guardrail for general agentic systems via synthetic data. arXiv preprint arXiv:2510.09781. — [arXiv:2510.09781](https://arxiv.org/abs/2510.09781) [find](https://scholar.google.com/scholar?q=Yue+Huang%2C+Hang+Hua%2C+Yujun+Zhou%2C+Pengcheng+Jing%2C+Manish+Nagireddy%2C+Inkit+Padhi%2C+Greta+Dolcetti%2C+Zhangchen+Xu%2C+Subhajit+Chaudhury%2C+Ambrish+Rawat%2C+and+1+others%2E+2025b%2E+Building+a+foundational+guardrail+)
14. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). arXiv preprint arXiv:2410.21276.
15. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and 1 others. 2023. [Llama guard: Llm-based input-output safeguard for human-ai conversations](https://md.research.elamur.ai/free/en/papers/inan-2023-llama-guard-llm-based-input/). arXiv preprint arXiv:2312.06674.
16. Aidan Kierans, Avijit Ghosh, Hananel Hazan, and Shiri Dori-Hacohen. 2025. Quantifying misalignment between agents: Towards a sociotechnical understanding of alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27365–27373. [find](https://scholar.google.com/scholar?q=Aidan+Kierans%2C+Avijit+Ghosh%2C+Hananel+Hazan%2C+and+Shiri+Dori%2DHacohen%2E+2025%2E+Quantifying+misalignment+between+agents%3A+Towards+a+sociotechnical+understanding+of+alignment%2E+In+Proceedings+of+the+AAAI+Confe)
17. Priyanshu Kumar, Devansh Jain, Akhila Yerukola, Liwei Jiang, Himanshu Beniwal, Thomas Hartvigsen, and Maarten Sap. 2025. Polyguard: A multilingual safety moderation tool for 17 languages. arXiv preprint arXiv:2504.04377. — [arXiv:2504.04377](https://arxiv.org/abs/2504.04377) [find](https://scholar.google.com/scholar?q=Priyanshu+Kumar%2C+Devansh+Jain%2C+Akhila+Yerukola%2C+Liwei+Jiang%2C+Himanshu+Beniwal%2C+Thomas+Hartvigsen%2C+and+Maarten+Sap%2E+2025%2E+Polyguard%3A+A+multilingual+safety+moderation+tool+for+17+languages%2E+arXiv+prepri)
18. Hao Li and Xiaogeng Liu. 2024. Injecguard: Benchmarking and mitigating over-defense in prompt injection guardrail models. arXiv preprint arXiv:2410.22770. — [arXiv:2410.22770](https://arxiv.org/abs/2410.22770) [find](https://scholar.google.com/scholar?q=Hao+Li+and+Xiaogeng+Liu%2E+2024%2E+Injecguard%3A+Benchmarking+and+mitigating+over%2Ddefense+in+prompt+injection+guardrail+models%2E+arXiv+preprint+arXiv%3A2410%2E22770%2E+%E2%80%94+%5BarXiv%3A2410%2E22770%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F24)
19. Hao Li, Xiaogeng Liu, Ning Zhang, and Chaowei Xiao. 2025. [Piguard: Prompt injection guardrail via mitigating overdefense for free](https://md.research.elamur.ai/free/en/papers/li-2025-piguard-prompt-injection-guardrail-mitigating/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 30420–30437. defensive_measures/sandwich_defens…
20. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
21. Idan Shenfeld, Jyothish Pari, and Pulkit Agrawal. 2025. Rl’s razor: Why online reinforcement learning forgets less. arXiv preprint arXiv:2509.04259. — [arXiv:2509.04259](https://arxiv.org/abs/2509.04259) [find](https://scholar.google.com/scholar?q=Idan+Shenfeld%2C+Jyothish+Pari%2C+and+Pulkit+Agrawal%2E+2025%2E+Rl%E2%80%99s+razor%3A+Why+online+reinforcement+learning+forgets+less%2E+arXiv+preprint+arXiv%3A2509%2E04259%2E+%E2%80%94+%5BarXiv%3A2509%2E04259%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2509%2E042)
22. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297.
23. Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce C Ho, Carl Yang, and May Dongmei Wang. 2024. Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records. In Proceedings of the 2024 Conference on Empirical Methods… [find](https://scholar.google.com/scholar?q=Wenqi+Shi%2C+Ran+Xu%2C+Yuchen+Zhuang%2C+Yue+Yu%2C+Jieyu+Zhang%2C+Hang+Wu%2C+Yuanda+Zhu%2C+Joyce+C+Ho%2C+Carl+Yang%2C+and+May+Dongmei+Wang%2E+2024%2E+Ehragent%3A+Code+empowers+large+language+models+for+few%2Dshot+complex+tabula)
24. Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li, Zichen Ding, Qi Liu, and 1 others. 2025. [Os-sentinel: Towards safety-enhanced mobile gui agents via hybrid validation in realistic workflows](https://md.research.elamur.ai/free/en/papers/sun-2026-os-sentinel-safety-enhanced-mobile/). arXiv preprint arXiv:2510.24411.
25. Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. 2024. [Salad-bench: A hierarchical and comprehensive safety benchmark for large language models](https://md.research.elamur.ai/free/en/papers/li-2024-salad-bench-hierarchical-comprehensive-safety/). arXiv preprint arXiv:2402.05044.
26. Gokul Swamy, Sanjiban Choudhury, Wen Sun, Zhiwei Steven Wu, and J Andrew Bagnell. 2025. All roads lead to likelihood: The value of reinforcement learning in fine-tuning. arXiv preprint arXiv:2503.01067. — [arXiv:2503.01067](https://arxiv.org/abs/2503.01067) [find](https://scholar.google.com/scholar?q=Gokul+Swamy%2C+Sanjiban+Choudhury%2C+Wen+Sun%2C+Zhiwei+Steven+Wu%2C+and+J+Andrew+Bagnell%2E+2025%2E+All+roads+lead+to+likelihood%3A+The+value+of+reinforcement+learning+in+fine%2Dtuning%2E+arXiv+preprint+arXiv%3A2503%2E0106)
27. Hanjun Luo, Shenyu Dai, Chiming Ni, Xinfeng Li, Guibin Zhang, Kun Wang, Tongliang Liu, and Hanan Salam. 2025a. Agentauditor: Human-level safety and security evaluation for llm agents. arXiv preprint arXiv:2506.00641. — [arXiv:2506.00641](https://arxiv.org/abs/2506.00641) [find](https://scholar.google.com/scholar?q=Hanjun+Luo%2C+Shenyu+Dai%2C+Chiming+Ni%2C+Xinfeng+Li%2C+Guibin+Zhang%2C+Kun+Wang%2C+Tongliang+Liu%2C+and+Hanan+Salam%2E+2025a%2E+Agentauditor%3A+Human%2Dlevel+safety+and+security+evaluation+for+llm+agents%2E+arXiv+preprint+a)
28. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, and 1 others. 2025. [Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/). arXiv preprint arXiv:2506.01939.
29. Weidi Luo, Shenghong Dai, Xiaogeng Liu, Suman Banerjee, Huan Sun, Muhao Chen, and Chaowei Xiao. 2025b. [AGrail: A lifelong agent guardrail with effective and adaptive safety detection](https://md.research.elamur.ai/free/en/papers/luo-2025-agrail-lifelong-agent-guardrail-effective/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),…
30. Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Shishir+G%2E+Patil%2C+Huanzhi+Mao%2C+Charlie+Cheng%2DJie+Ji%2C+Fanjia+Yan%2C+Vishnu+Suresh%2C+Ion+Stoica%2C+and+Joseph+E%2E+Gonzalez%2E+2025%2E+The+berkeley+function+calling+leaderboard+%28bfcl%29%3A+From+tool+use+to+agentic+eva)
31. Sander Schulhoff. 2024. Sandwich defense. [https://learnprompting.org/docs/prompt_hacking/](https://learnprompting.org/docs/prompt_hacking/) Yifei Wang, Dizhan Xue, Shengjie Zhang, and Shengsheng Qian. 2024. Badagent: Inserting and activating backdoor attacks in llm agents. arXiv preprint arXiv:2406.03007. [↗](https://learnprompting.org/docs/prompt_hacking/)
32. Qianshan Wei, Tengchao Yang, Yaochen Wang, Xinfeng Li, Lijun Li, Zhenfei Yin, Yi Zhan, Thorsten Holz, Zhiqiang Lin, and XiaoFeng Wang. 2025. A-memguard: A proactive defense framework for llm-based agent memory. arXiv preprint arXiv:2510.02373. — [arXiv:2510.02373](https://arxiv.org/abs/2510.02373) [find](https://scholar.google.com/scholar?q=Qianshan+Wei%2C+Tengchao+Yang%2C+Yaochen+Wang%2C+Xinfeng+Li%2C+Lijun+Li%2C+Zhenfei+Yin%2C+Yi+Zhan%2C+Thorsten+Holz%2C+Zhiqiang+Lin%2C+and+XiaoFeng+Wang%2E+2025%2E+A%2Dmemguard%3A+A+proactive+defense+framework+for+llm%2Dbased+age)
33. Yaozu Wu, Jizhou Guo, Dongyuan Li, Henry Peng Zou, Wei-Chieh Huang, Yankai Chen, Zhen Wang, Weizhi Zhang, Yangning Li, Meng Zhang, and 1 others. 2025. Psg-agent: Personality-aware safety guardrail for llm-based agents. arXiv preprint arXiv:2509.23614. — [arXiv:2509.23614](https://arxiv.org/abs/2509.23614) [find](https://scholar.google.com/scholar?q=Yaozu+Wu%2C+Jizhou+Guo%2C+Dongyuan+Li%2C+Henry+Peng+Zou%2C+Wei%2DChieh+Huang%2C+Yankai+Chen%2C+Zhen+Wang%2C+Weizhi+Zhang%2C+Yangning+Li%2C+Meng+Zhang%2C+and+1+others%2E+2025%2E+Psg%2Dagent%3A+Personality%2Daware+safety+guardrail+for)
34. Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, and 1 others. 2024. Guardagent: Safeguard llm agents by a guard agent via knowledge-enabled reasoning. arXiv preprint arXiv:2406.09187. — [arXiv:2406.09187](https://arxiv.org/abs/2406.09187) [find](https://scholar.google.com/scholar?q=Zhen+Xiang%2C+Linzhi+Zheng%2C+Yanjie+Li%2C+Junyuan+Hong%2C+Qinbin+Li%2C+Han+Xie%2C+Jiawei+Zhang%2C+Zidi+Xiong%2C+Chulin+Xie%2C+Carl+Yang%2C+and+1+others%2E+2024%2E+Guardagent%3A+Safeguard+llm+agents+by+a+guard+agent+via+knowle)
35. Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, and 1 others. 2024. [Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments](https://md.research.elamur.ai/free/en/papers/xie-2024-osworld-benchmarking-multimodal-agents-open/). Advances in Neural Information Processing Systems, 37…
36. Chejian Xu, Mintong Kang, Jiawei Zhang, Zeyi Liao, Lingbo Mo, Mengqi Yuan, Huan Sun, and Bo Li. 2024. Advagent: Controllable blackbox red-teaming on web agents. arXiv preprint arXiv:2410.17401. — [arXiv:2410.17401](https://arxiv.org/abs/2410.17401) [find](https://scholar.google.com/scholar?q=Chejian+Xu%2C+Mintong+Kang%2C+Jiawei+Zhang%2C+Zeyi+Liao%2C+Lingbo+Mo%2C+Mengqi+Yuan%2C+Huan+Sun%2C+and+Bo+Li%2E+2024%2E+Advagent%3A+Controllable+blackbox+red%2Dteaming+on+web+agents%2E+arXiv+preprint+arXiv%3A2410%2E17401%2E+%E2%80%94+%5BarX)
37. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
38. Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu Sun. 2024. Watch out for your agents! investigating backdoor threats to llm-based agents. Advances in Neural Information Processing Systems, 37:100938–100964. [find](https://scholar.google.com/scholar?q=Wenkai+Yang%2C+Xiaohan+Bi%2C+Yankai+Lin%2C+Sishuo+Chen%2C+Jie+Zhou%2C+and+Xu+Sun%2E+2024%2E+Watch+out+for+your+agents%21+investigating+backdoor+threats+to+llm%2Dbased+agents%2E+Advances+in+Neural+Information+Processing+S)
39. Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. 2024. [τ -bench: A benchmark for tool-agent-user interaction in real-world domains](https://md.research.elamur.ai/free/en/papers/yao-2024-tau-bench-benchmark-tool/). ArXiv, abs/2406.12045.
40. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In The eleventh international conference on learning representations.
41. Junjie Ye, Sixian Li, Guanyu Li, Caishuang Huang, Songyang Gao, Yilong Wu, Qi Zhang, Tao Gui, and Xuan-Jing Huang. 2024. Toolsword: Unveiling safety issues of large language models in tool learning across three stages. In Proceedings of the 62nd Annual Meeting of the Association for Computational Li… [find](https://scholar.google.com/scholar?q=Junjie+Ye%2C+Sixian+Li%2C+Guanyu+Li%2C+Caishuang+Huang%2C+Songyang+Gao%2C+Yilong+Wu%2C+Qi+Zhang%2C+Tao+Gui%2C+and+Xuan%2DJing+Huang%2E+2024%2E+Toolsword%3A+Unveiling+safety+issues+of+large+language+models+in+tool+learning+ac)
42. Lingzhi Yuan, Xinfeng Li, Chejian Xu, Guanhong Tao, Xiaojun Jia, Yihao Huang, Wei Dong, Yang Liu, and Bo Li. 2025. Promptguard: Soft prompt-guided unsafe content moderation for text-to-image models. arXiv preprint arXiv:2501.03544. — [arXiv:2501.03544](https://arxiv.org/abs/2501.03544) [find](https://scholar.google.com/scholar?q=Lingzhi+Yuan%2C+Xinfeng+Li%2C+Chejian+Xu%2C+Guanhong+Tao%2C+Xiaojun+Jia%2C+Yihao+Huang%2C+Wei+Dong%2C+Yang+Liu%2C+and+Bo+Li%2E+2025%2E+Promptguard%3A+Soft+prompt%2Dguided+unsafe+content+moderation+for+text%2Dto%2Dimage+models%2E+a)
43. Tongxin Yuan, Zhiwei He, Lingzhong Dong, Yiming Wang, Ruijie Zhao, Tian Xia, Lizhen Xu, Binglin Zhou, Fangqi Li, Zhuosheng Zhang, and 1 others. 2024. [R-judge: Benchmarking safety risk awareness for llm agents](https://md.research.elamur.ai/free/en/papers/yuan-2024-r-judge-benchmarking-safety-risk/). arXiv preprint arXiv:2401.10019.
44. Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, and 1 others. 2025. [Glm-4.5: Agentic, reasoning, and coding (arc) foundation models](https://md.research.elamur.ai/free/en/papers/team-2025-4-glm-4-5-agentic/). arXiv preprint arXiv:2508.06471.
45. Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, and 1 others. 2024. Shieldgemma: Generative ai content moderation based on gemma. arXiv preprint arXiv:2407.21772. — [arXiv:2407.21772](https://arxiv.org/abs/2407.21772) [find](https://scholar.google.com/scholar?q=Wenjun+Zeng%2C+Yuchi+Liu%2C+Ryan+Mullins%2C+Ludovic+Peran%2C+Joe+Fernandez%2C+Hamza+Harkous%2C+Karthik+Narasimhan%2C+Drew+Proud%2C+Piyush+Kumar%2C+Bhaktipriya+Radharapu%2C+and+1+others%2E+2024%2E+Shieldgemma%3A+Generative+ai+c)
46. Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. [InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents](https://md.research.elamur.ai/free/en/papers/zhan-2024-injecagent-benchmarking-indirect-prompt-injections/). In Findings of the Association for Computational Linguistics: ACL 2024, pages 10471– 10506, Bangkok, Thailand. Association for Computatio…
47. Chaoyun Zhang, Liqun Li, Shilin He, Xu Zhang, Bo Qiao, Si Qin, Minghua Ma, Yu Kang, Qingwei Lin, Saravan Rajmohan, and 1 others. 2025a. [Ufo: A ui-focused agent for windows os interaction](https://md.research.elamur.ai/free/en/papers/zhang-2025-ufo-ui-focused-agent-windows/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computationa…
48. Jinchuan Zhang, Lu Yin, Yan Zhou, and Songlin Hu. 2025b. Agentalign: Navigating safety alignment in the shift from informative to agentic large language models. arXiv preprint arXiv:2505.23020. — [arXiv:2505.23020](https://arxiv.org/abs/2505.23020) [find](https://scholar.google.com/scholar?q=Jinchuan+Zhang%2C+Lu+Yin%2C+Yan+Zhou%2C+and+Songlin+Hu%2E+2025b%2E+Agentalign%3A+Navigating+safety+alignment+in+the+shift+from+informative+to+agentic+large+language+models%2E+arXiv+preprint+arXiv%3A2505%2E23020%2E+%E2%80%94+%5BarX)
49. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, and 1 others. 2025c. [Qwen3 embedding: Advancing text embedding and reranking through foundation models](https://md.research.elamur.ai/free/en/papers/zhang-2025-qwen3-embedding-advancing-text/). arXiv preprint arXiv:2506.05176.
50. Zhexin Zhang, Shiyao Cui, Yida Lu, Jingzhuo Zhou, Junxiao Yang, Hongning Wang, and Minlie Huang. 2024. Agent-safetybench: Evaluating the safety of llm agents. arXiv preprint arXiv:2412.14470. — [arXiv:2412.14470](https://arxiv.org/abs/2412.14470) [find](https://scholar.google.com/scholar?q=Zhexin+Zhang%2C+Shiyao+Cui%2C+Yida+Lu%2C+Jingzhuo+Zhou%2C+Junxiao+Yang%2C+Hongning+Wang%2C+and+Minlie+Huang%2E+2024%2E+Agent%2Dsafetybench%3A+Evaluating+the+safety+of+llm+agents%2E+arXiv+preprint+arXiv%3A2412%2E14470%2E+%E2%80%94+%5BarXiv)
51. Haiquan Zhao, Chenhan Yuan, Fei Huang, Xiaomeng Hu, Yichang Zhang, An Yang, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin, and 1 others. 2025. [Qwen3guard technical report](https://md.research.elamur.ai/free/en/papers/zhao-2025-qwen3guard-technical-report/). arXiv preprint arXiv:2510.14276.
52. Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, and 1 others. 2023. [Webarena: A realistic web environment for building autonomous agents](https://md.research.elamur.ai/free/en/papers/zhou-2023-webarena-realistic-web-environment-building/). arXiv preprint arXiv:2307.13854.
53. Xin Zhou, Martin Weyssow, Ratnadira Widyasari, Ting Zhang, Junda He, Yunbo Lyu, Jianming Chang, Beiqi Zhang, Dan Huang, and David Lo. 2025. Lessleak-bench: A first investigation of data leakage in llms across 83 software engineering benchmarks. arXiv preprint arXiv:2502.06215. — [arXiv:2502.06215](https://arxiv.org/abs/2502.06215) [find](https://scholar.google.com/scholar?q=Xin+Zhou%2C+Martin+Weyssow%2C+Ratnadira+Widyasari%2C+Ting+Zhang%2C+Junda+He%2C+Yunbo+Lyu%2C+Jianming+Chang%2C+Beiqi+Zhang%2C+Dan+Huang%2C+and+David+Lo%2E+2025%2E+Lessleak%2Dbench%3A+A+first+investigation+of+data+leakage+in+llm)

*Updated 31 October 2026*