---
title: "Near-Miss: Latent Policy Failure Detection in Agentic Workflows"
description: "The authors build on ToolGuard, which compiles natural-language policies into executable Python guard functions for… 24 references."
authors: ["Ella Rabinovich", "David Boaz", "Naama Zwerdling", "Ateret Anaby Tavor"]
year: 2026
venue: "Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM)"
source: "https://aclanthology.org/2026.gem-main.30/"
doi: "10.18653/v1/2026.gem-main.30"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rabinovich-2026-near-miss-latent-policy-failure/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rabinovich-2026-near-miss-latent-policy-failure/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/37/) / Near-Miss: Latent Policy Failure Detection in Agentic Workflows

# Near-Miss: Latent Policy Failure Detection in Agentic Workflows

**Authors:** [Ella Rabinovich](https://md.research.elamur.ai/free/en/authors/ella-rabinovich/), David Boaz, Naama Zwerdling and [Ateret Anaby Tavor](https://md.research.elamur.ai/free/en/authors/ateret-anaby-tavor/) · **Year:** 2026 · **Venue:** [Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM)](https://md.research.elamur.ai/free/en/venues/gem/) ([GEM 2026](https://md.research.elamur.ai/free/en/venues/gem/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.gem-main.30/) · [PDF](https://aclanthology.org/2026.gem-main.30.pdf) · [DOI 10.18653/v1/2026.gem-main.30](https://doi.org/10.18653/v1/2026.gem-main.30) · [OpenAlex](https://openalex.org/W7166832590)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
LLM agents that automate business processes must follow policies before they change system state, for example checking eligibility before cancelling a booking. Benchmarks usually judge this only by comparing the final database state with a gold reference. That approach misses "near-misses," or latent failures: the agent skips a required check, but the outcome is still correct because the user happened to give accurate information. In adversarial settings, such skipped checks could be exploited. The authors build on ToolGuard, which compiles natural-language policies into executable Python guard functions for state-changing tools. They use it offline to inspect agent trajectories and test whether each mutating tool call was backed by the information the policy requires. They evaluate six open and proprietary LLM agents on the τ²-verified Airlines domain. Among trajectories with mutating tool calls, 8–17% contain latent failures even though the final state matches the gold one. The work exposes a blind spot in outcome-only evaluation and argues for metrics that also assess the agent's decision process.

## Key points
- **Data and models:** The benchmark is τ²-verified Airlines, with 200 simulations per model. The agents are GPT5-chat, Claude-Sonnet4 and Gemini-3-pro (closed), plus GPT-oss-120b, Qwen2.5-72b and Kimi-K2.5 (open). All guard code was generated with Claude-Sonnet4, the best-performing generator.
- **Near-miss rates:** Among trajectories with mutating tool calls, the near-miss rate ranges from about 0.084 (Qwen2.5-72b) to 0.173 (GPT5-chat). Closed models show slightly more latent failures than open ones.
- **Explicit violations:** By the authors' estimate, explicit policy violations make up roughly 25% of all simulation failures in the τ² Airlines domain.
- **Where failures occur:** update_reservation_flights() is the mutating tool most often called without the required checks. get_flight_status() is the read-only check agents skip most often. No near-misses were found for update_reservation_passengers().
- **Limitations and code:** Results depend on the quality of the LLM-generated guards, and only one benchmark was tested. Code is released at github.com/AgentToolkit/toolguard-violation-analysis.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [IBM](https://md.research.elamur.ai/free/en/companies/ibm/) — Company, Armonk, United States · 139 works · 70,691 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [⚙️ Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/) · 133 works · [💻 Software](https://md.research.elamur.ai/free/en/shelves/computer-science/software/) · 92 works · [⚙️ Industrial and Manufacturing Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/industrial-and-manufacturing-engineering/) · 10 works · [💼 Business, Management and Accounting](https://md.research.elamur.ai/free/en/shelves/business-management-and-accounting/) · 32 works · [💼 Management Information Systems](https://md.research.elamur.ai/free/en/shelves/business-management-and-accounting/management-information-systems/) · 4 works

## Cite

Cite as: Ella Rabinovich, David Boaz, Naama Zwerdling and Ateret Anaby Tavor. 2026. Near-Miss: Latent Policy Failure Detection in Agentic Workflows. Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rabinovich2026near,
  title = {{Near-Miss: Latent Policy Failure Detection in Agentic Workflows}},
  author = {Ella Rabinovich, David Boaz, Naama Zwerdling and Ateret Anaby Tavor},
  booktitle = {Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM)},
  year = {2026},
  doi = {10.18653/v1/2026.gem-main.30},
  url = {https://doi.org/10.18653/v1/2026.gem-main.30}
}
```

- [← previous NE-LID: A Fast and Accurate Language Identification System for Northeast Indian Languages](https://md.research.elamur.ai/free/en/papers/nyalang-2026-ne-lid-fast-accurate-language/)
- [next → NEAT-IR: Neural Explainable Analysis Tool for Information Retrieval](https://md.research.elamur.ai/free/en/papers/sukherman-2026-neat-ir-neural-explainable-analysis/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/37/)

## References

24 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Anthropic. 2025. Claude model documentation. [https://docs.anthropic.com/claude/](https://docs.anthropic.com/claude/) docs/models-overview. Accessed: 2026. [↗](https://docs.anthropic.com/claude/)
2. Sumanth Balaji, Piyush Mishra, Aashraya Sachdeva, and Suraj Agrawal. 2026. Beyond ivr: Benchmarking customer support llm agents for business-adherence. Preprint, arXiv:2601.00596. [find](https://scholar.google.com/scholar?q=Sumanth+Balaji%2C+Piyush+Mishra%2C+Aashraya+Sachdeva%2C+and+Suraj+Agrawal%2E+2026%2E+Beyond+ivr%3A+Benchmarking+customer+support+llm+agents+for+business%2Dadherence%2E+Preprint%2C+arXiv%3A2601%2E00596%2E)
3. Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. 2025. τ 2 -bench: Evaluating conversational agents in a dual-control environment. Preprint, arXiv:2506.07982. [find](https://scholar.google.com/scholar?q=Victor+Barres%2C+Honghua+Dong%2C+Soham+Ray%2C+Xujie+Si%2C+and+Karthik+Narasimhan%2E+2025%2E+%CF%84+2+%2Dbench%3A+Evaluating+conversational+agents+in+a+dual%2Dcontrol+environment%2E+Preprint%2C+arXiv%3A2506%2E07982%2E)
4. Zhaorun Chen, Mintong Kang, and Bo Li. 2025. Shield-Agent: Shielding agents via verifiable safety policy reasoning. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 8313–8344. PMLR. [find](https://scholar.google.com/scholar?q=Zhaorun+Chen%2C+Mintong+Kang%2C+and+Bo+Li%2E+2025%2E+Shield%2DAgent%3A+Shielding+agents+via+verifiable+safety+policy+reasoning%2E+In+Proceedings+of+the+42nd+International+Conference+on+Machine+Learning%2C+volume+267+)
5. Alejandro Cuadron, Pengfei Yu, Yang Liu, and Arpit Gupta. 2025. Saber: Small actions, big errors - safeguarding mutating steps in llm agents. Preprint, arXiv:2512.07850. — [arXiv:2512.07850](https://arxiv.org/abs/2512.07850) [find](https://scholar.google.com/scholar?q=Alejandro+Cuadron%2C+Pengfei+Yu%2C+Yang+Liu%2C+and+Arpit+Gupta%2E+2025%2E+Saber%3A+Small+actions%2C+big+errors+%2D+safeguarding+mutating+steps+in+llm+agents%2E+Preprint%2C+arXiv%3A2512%2E07850%2E+%E2%80%94+%5BarXiv%3A2512%2E07850%5D%28https%3A%2F%2Fa)
6. Sarik Ghazarian, Abhinav Gullapalli, Swair Shah, Anurag Beniwal, Nanyun Peng, Narayanan Sadagopan, and Zhou Yu. 2025. TOD-procbench: Benchmarking complex instruction-following in task-oriented dialogues. In First Workshop on Multi-Turn Interactions in Large Language Models. [find](https://scholar.google.com/scholar?q=Sarik+Ghazarian%2C+Abhinav+Gullapalli%2C+Swair+Shah%2C+Anurag+Beniwal%2C+Nanyun+Peng%2C+Narayanan+Sadagopan%2C+and+Zhou+Yu%2E+2025%2E+TOD%2Dprocbench%3A+Benchmarking+complex+instruction%2Dfollowing+in+task%2Doriented+dialogu)
7. Google DeepMind. 2025. Gemini 3: A family of highly capable multimodal models. [https://deepmind.google/technologies/gemini/](https://deepmind.google/technologies/gemini/). [↗](https://deepmind.google/technologies/gemini/)
8. Kung-Hsiang Huang, Akshara Prabhakar, Sidharth Dhawan, Yixin Mao, Huan Wang, Silvio Savarese, Caiming Xiong, Philippe Laban, and Chien-Sheng Wu. 2025. [CRMArena: Understanding the capacity of LLM agents to perform professional CRM tasks in realistic environments](https://md.research.elamur.ai/free/en/papers/huang-2025-crmarena-understanding-capacity-llm-agents/). In Proceedings of the 2025 Conference…
9. Zekun Li, Shinda Huang, Jiangtian Wang, Nathan Zhang, Antonis Antoniades, Wenyue Hua, Kaijie Zhu, Sirui Zeng, William Yang Wang, and Xifeng Yan. 2025. Agentorca: A dual-system framework to evaluate language agents on operational routine and constraint adherence. arXiv preprint arXiv:2503.08669. — [arXiv:2503.08669](https://arxiv.org/abs/2503.08669) [find](https://scholar.google.com/scholar?q=Zekun+Li%2C+Shinda+Huang%2C+Jiangtian+Wang%2C+Nathan+Zhang%2C+Antonis+Antoniades%2C+Wenyue+Hua%2C+Kaijie+Zhu%2C+Sirui+Zeng%2C+William+Yang+Wang%2C+and+Xifeng+Yan%2E+2025%2E+Agentorca%3A+A+dual%2Dsystem+framework+to+evaluate+la)
10. Nikita Mehandru, Brenda Y Miao, Eduardo Rodriguez Almaraz, Madhumita Sushil, Atul J Butte, and Ahmed Alaa. 2024. Evaluating large language models as agents in the clinic. NPJ digital medicine, 7(1):84. [find](https://scholar.google.com/scholar?q=Nikita+Mehandru%2C+Brenda+Y+Miao%2C+Eduardo+Rodriguez+Almaraz%2C+Madhumita+Sushil%2C+Atul+J+Butte%2C+and+Ahmed+Alaa%2E+2024%2E+Evaluating+large+language+models+as+agents+in+the+clinic%2E+NPJ+digital+medicine%2C+7%281%29%3A84)
11. Moonshot AI. 2024. Kimi: An efficient large language model for long context reasoning. [https://kimi.moonshot.cn](https://kimi.moonshot.cn). [↗](https://kimi.moonshot.cn)
12. Itay Nakash, George Kour, Koren Lazar, Matan Vetzler, Guy Uziel, and Ateret Anaby Tavor. 2025. Effective red-teaming of policy-adherent agents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2250–2268, Suzhou, China. Association for Computational Lin… [find](https://scholar.google.com/scholar?q=Itay+Nakash%2C+George+Kour%2C+Koren+Lazar%2C+Matan+Vetzler%2C+Guy+Uziel%2C+and+Ateret+Anaby+Tavor%2E+2025%2E+Effective+red%2Dteaming+of+policy%2Dadherent+agents%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Metho)
13. OpenAI. 2025a. Codex models. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025a%2E+Codex+models%2E)
14. OpenAI. 2025b. Gpt-4.1 model family. [https://platform.openai.com/docs/models/gpt-4.1](https://platform.openai.com/docs/models/gpt-4.1). Accessed: 2026. [↗](https://platform.openai.com/docs/models/gpt-4.1)
15. OpenAI. 2025c. Gpt-5 technical report. [https://openai.com](https://openai.com). GPT-5 model family. [↗](https://openai.com)
16. OpenAI. 2025d. Gpt-oss: Open source large language models. [https://openai.com](https://openai.com). [↗](https://openai.com)
17. Nils Palumbo, Sarthak Choudhary, Jihye Choi, Prasad Chalasani, and Somesh Jha. 2026. Policy compiler for secure agentic systems. Preprint, arXiv:2602.16708. — [arXiv:2602.16708](https://arxiv.org/abs/2602.16708) [find](https://scholar.google.com/scholar?q=Nils+Palumbo%2C+Sarthak+Choudhary%2C+Jihye+Choi%2C+Prasad+Chalasani%2C+and+Somesh+Jha%2E+2026%2E+Policy+compiler+for+secure+agentic+systems%2E+Preprint%2C+arXiv%3A2602%2E16708%2E+%E2%80%94+%5BarXiv%3A2602%2E16708%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
18. Qwen Team. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2024%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)
19. Francesco Romeo, Luigi Arena, Francesco Blefari, Francesco Aurelio Pironti, Matteo Lupinacci, and Angelo Furfaro. 2025. Arpaccino: an agentic-rag for policy as code compliance. In European Conference on Advances in Databases and Information Systems, pages 467–481. Springer. [find](https://scholar.google.com/scholar?q=Francesco+Romeo%2C+Luigi+Arena%2C+Francesco+Blefari%2C+Francesco+Aurelio+Pironti%2C+Matteo+Lupinacci%2C+and+Angelo+Furfaro%2E+2025%2E+Arpaccino%3A+an+agentic%2Drag+for+policy+as+code+compliance%2E+In+European+Conference+)
20. Haoyu Wang, Christopher M. Poskitt, and Jun Sun. 2025. Agentspec: Customizable runtime enforcement for safe and reliable llm agents. arXiv preprint arXiv:2503.18666. — [arXiv:2503.18666](https://arxiv.org/abs/2503.18666) [find](https://scholar.google.com/scholar?q=Haoyu+Wang%2C+Christopher+M%2E+Poskitt%2C+and+Jun+Sun%2E+2025%2E+Agentspec%3A+Customizable+runtime+enforcement+for+safe+and+reliable+llm+agents%2E+arXiv+preprint+arXiv%3A2503%2E18666%2E+%E2%80%94+%5BarXiv%3A2503%2E18666%5D%28https%3A%2F%2Farxiv)
21. Yijia Xiao, Edward Sun, Di Luo, and Wei Wang. 2024. TradingAgents: Multi-Agents LLM Financial Trading Framework. arXiv preprint arXiv:2412.20138. — [arXiv:2412.20138](https://arxiv.org/abs/2412.20138) [find](https://scholar.google.com/scholar?q=Yijia+Xiao%2C+Edward+Sun%2C+Di+Luo%2C+and+Wei+Wang%2E+2024%2E+TradingAgents%3A+Multi%2DAgents+LLM+Financial+Trading+Framework%2E+arXiv+preprint+arXiv%3A2412%2E20138%2E+%E2%80%94+%5BarXiv%3A2412%2E20138%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2412%2E20138%29)
22. Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. 2024. [tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://md.research.elamur.ai/free/en/papers/yao-2024-tau-bench-benchmark-tool/). arXiv preprint arXiv:2406.12045.
23. Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, and 1 others. 2023. [Webarena: A realistic web environment for building autonomous agents](https://md.research.elamur.ai/free/en/papers/zhou-2023-webarena-realistic-web-environment-building/). arXiv preprint arXiv:2307.13854.
24. Naama Zwerdling, David Boaz, Ella Rabinovich, Guy Uziel, David Amid, and Ateret Anaby Tavor. 2025. Towards enforcing company policy adherence in agentic workflows. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 595–606. Association fo… [find](https://scholar.google.com/scholar?q=Naama+Zwerdling%2C+David+Boaz%2C+Ella+Rabinovich%2C+Guy+Uziel%2C+David+Amid%2C+and+Ateret+Anaby+Tavor%2E+2025%2E+Towards+enforcing+company+policy+adherence+in+agentic+workflows%2E+In+Proceedings+of+the+2025+Conferenc)

*Updated 1 November 2026*