---
title: "VET — paper by Wang et al. (2026) — summary and ideas"
description: "LLM-based text-to-SQL systems often hallucinate schema elements, data values or query logic. 28 references."
authors: ["Dongyu Wang", "Jingyu Li", "Lan Zhang", "Ganggang Yu", "Liang Huang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1544/"
doi: "10.18653/v1/2026.findings-acl.1544"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-vet-verifiable-execution-tracing-reliable/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-vet-verifiable-execution-tracing-reliable/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/47/) / VET: Verifiable Execution Tracing for Reliable Text-to-SQL Generation

# VET: Verifiable Execution Tracing for Reliable Text-to-SQL Generation

**Authors:** Dongyu Wang, Jingyu Li, [Lan Zhang](https://md.research.elamur.ai/free/en/authors/lan-zhang/), Ganggang Yu and [Liang Huang](https://md.research.elamur.ai/free/en/authors/liang-huang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1544/) · [PDF](https://aclanthology.org/2026.findings-acl.1544.pdf) · [DOI 10.18653/v1/2026.findings-acl.1544](https://doi.org/10.18653/v1/2026.findings-acl.1544) · [OpenAlex](https://openalex.org/W7166850356)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
LLM-based text-to-SQL systems often hallucinate schema elements, data values or query logic. Their intermediate reasoning is either free text that nothing can check or code that runs only once at the end. VET (Verifiable Execution Tracing) is a training-free framework that replaces textual rationales with a chain of small Python operations, each run on the actual database.

The pipeline has three stages:
1. **Schema-constrained initialization.** Schema linking narrows the reasoning to a set of candidate columns.
2. **Iterative executable reasoning.** Each step's output, such as an intermediate DataFrame, is inspected. Errors or failed checks trigger a repair, and data probes resolve ambiguous values.
3. **SQL synthesis.** The final query is built from the verified trace and accepted only if two checks pass. Its result must agree with the trace's final output, and a question reconstructed from it must stay close to the original.

Under comparable training-free conditions, VET attains 37.04% execution accuracy on Spider 2.0-lite and 70.93% on BIRD, improving most on difficult queries. The work shows that grounding each reasoning step in observable database state makes generation more reliable and easier to debug.

## Key points
- **Formulation:** generation is modeled as Y = G(V(E(Q, S), D)), where E builds the executable trace, V verifies it against database D, and G synthesizes the SQL.
- **Spider 2.0-lite with DeepSeek-Chat:** the baseline scores 14.82% and full VET reaches 37.04%. Removing any single component lowers the score.
- **Ablation on BIRD-Dev (overall execution accuracy):**
  - Full VET scores 68.38%.
  - Dropping execution feedback gives 63.04%, with challenging queries falling from 61.11% to 45.83%.
  - Dropping step-wise verification gives 63.56%.
  - Dropping the unified consistency check gives 63.30%.
  - Dropping schema constraints gives 66.30%.
- **Positioning against LEVER:** the authors describe their verification as a "generative white-box" approach, in contrast to LEVER's trained discriminator that ranks candidates.
- **Limitations:**
  - Step-wise execution adds inference latency.
  - Deployment involves a privacy–observability trade-off between a Secure mode and a Transparent/Auditing mode.
  - The authors recommend read-only database credentials and bounded execution budgets.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 108 works · 112,698 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🌐 Computer Networks and Communications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-networks-and-communications/) · 64 works

🧭 Topics (3): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [🧾 Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/) · 185 works · [💻 Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/) · 129 works

## Cite

Cite as: Dongyu Wang, Jingyu Li, Lan Zhang, Ganggang Yu and Liang Huang. 2026. VET: Verifiable Execution Tracing for Reliable Text-to-SQL Generation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026vet,
  title = {{VET: Verifiable Execution Tracing for Reliable Text-to-SQL Generation}},
  author = {Dongyu Wang, Jingyu Li, Lan Zhang, Ganggang Yu and Liang Huang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1544},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1544}
}
```

- [← previous Vertical Routing: A Cost-Efficient Collaboration Routing Framework](https://md.research.elamur.ai/free/en/papers/shen-2026-vertical-routing-cost-efficient-collaboration/)
- [next → VIDA: A Visual Intent-driven Design Assistant for Proactive Multimodal Clarification](https://md.research.elamur.ai/free/en/papers/liu-2026-vida-visual-intent-driven-design/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/47/)

## References

28 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Zhenbiao Cao, Yuanlei Zheng, Zhihao Fan, Xiaojin Zhang, Wei Chen, and Xiang Bai. 2024. Rsl-sql: Robust schema linking in text-to-sql generation. Preprint, arXiv:2411.00073. — [arXiv:2411.00073](https://arxiv.org/abs/2411.00073) [find](https://scholar.google.com/scholar?q=Zhenbiao+Cao%2C+Yuanlei+Zheng%2C+Zhihao+Fan%2C+Xiaojin+Zhang%2C+Wei+Chen%2C+and+Xiang+Bai%2E+2024%2E+Rsl%2Dsql%3A+Robust+schema+linking+in+text%2Dto%2Dsql+generation%2E+Preprint%2C+arXiv%3A2411%2E00073%2E+%E2%80%94+%5BarXiv%3A2411%2E00073%5D%28https%3A)
2. Yongdong chi, Hanqing Wang, Zonghan Yang, Jian Yang, Xiao Yan, Yun Chen, and Guanhua Chen. 2025. Pi-sql: Enhancing text-to-sql with fine-grained guidance from pivot programming languages. Preprint, arXiv:2506.00912. [find](https://scholar.google.com/scholar?q=Yongdong+chi%2C+Hanqing+Wang%2C+Zonghan+Yang%2C+Jian+Yang%2C+Xiao+Yan%2C+Yun+Chen%2C+and+Guanhua+Chen%2E+2025%2E+Pi%2Dsql%3A+Enhancing+text%2Dto%2Dsql+with+fine%2Dgrained+guidance+from+pivot+programming+languages%2E+Preprint%2C+ar)
3. Yaxun Dai, Haiqin Yang, Mou Hao, and Pingfu Chao. 2025. PARSQL: Enhancing text-to-SQL through SQL parsing and reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 661–681, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yaxun+Dai%2C+Haiqin+Yang%2C+Mou+Hao%2C+and+Pingfu+Chao%2E+2025%2E+PARSQL%3A+Enhancing+text%2Dto%2DSQL+through+SQL+parsing+and+reasoning%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2025%2C+pages+6)
4. Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. 2023a. [Text-to-sql empowered by large language models: A benchmark evaluation](https://md.research.elamur.ai/free/en/papers/gao-2023-text-sql-empowered-large-language/). Preprint, arXiv:2308.15363.
5. Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. 2023b. [Text-to-sql empowered by large language models: A benchmark evaluation](https://md.research.elamur.ai/free/en/papers/gao-2023-text-sql-empowered-large-language/). arXiv preprint arXiv:2308.15363.
6. Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023c. Pal: Program-aided language models. Preprint, arXiv:2211.10435. [find](https://scholar.google.com/scholar?q=Luyu+Gao%2C+Aman+Madaan%2C+Shuyan+Zhou%2C+Uri+Alon%2C+Pengfei+Liu%2C+Yiming+Yang%2C+Jamie+Callan%2C+and+Graham+Neubig%2E+2023c%2E+Pal%3A+Program%2Daided+language+models%2E+Preprint%2C+arXiv%3A2211%2E10435%2E)
7. Omid Reza Heidari, Siobhan Reid, and Yassine Yaakoubi. 2025. Agentiql: An agent-inspired multi-expert framework for text-to-sql generation. Preprint, arXiv:2510.10661. — [arXiv:2510.10661](https://arxiv.org/abs/2510.10661) [find](https://scholar.google.com/scholar?q=Omid+Reza+Heidari%2C+Siobhan+Reid%2C+and+Yassine+Yaakoubi%2E+2025%2E+Agentiql%3A+An+agent%2Dinspired+multi%2Dexpert+framework+for+text%2Dto%2Dsql+generation%2E+Preprint%2C+arXiv%3A2510%2E10661%2E+%E2%80%94+%5BarXiv%3A2510%2E10661%5D%28https%3A%2F%2Farx)
8. Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, and 1 others. Spider 2.0: Evaluating language models on real-world enterprise text-to-sql work. [find](https://scholar.google.com/scholar?q=Fangyu+Lei%2C+Jixuan+Chen%2C+Yuxiao+Ye%2C+Ruisheng+Cao%2C+Dongchan+Shin%2C+Hongjin+Su%2C+Zhaoqing+Suo%2C+Hongcheng+Gao%2C+Wenjing+Hu%2C+Pengcheng+Yin%2C+and+1+others%2E+Spider+2%2E0%3A+Evaluating+language+models+on+real%2Dworld+)
9. Boyan Li, Yuyu Luo, Chengliang Chai, Guoliang Li, and Nan Tang. 2024a. The dawn of natural language to sql: Are we fully ready? Proceedings of the VLDB Endowment, 17(11):33183331. [find](https://scholar.google.com/scholar?q=Boyan+Li%2C+Yuyu+Luo%2C+Chengliang+Chai%2C+Guoliang+Li%2C+and+Nan+Tang%2E+2024a%2E+The+dawn+of+natural+language+to+sql%3A+Are+we+fully+ready%3F+Proceedings+of+the+VLDB+Endowment%2C+17%2811%29%3A33183331%2E)
10. Boyan Li, Jiayi Zhang, Ju Fan, Yanwei Xu, Chong Chen, Nan Tang, and Yuyu Luo. 2025a. Alpha-sql: Zero-shot text-to-sql using monte carlo tree search. Preprint, arXiv:2502.17248. — [arXiv:2502.17248](https://arxiv.org/abs/2502.17248) [find](https://scholar.google.com/scholar?q=Boyan+Li%2C+Jiayi+Zhang%2C+Ju+Fan%2C+Yanwei+Xu%2C+Chong+Chen%2C+Nan+Tang%2C+and+Yuyu+Luo%2E+2025a%2E+Alpha%2Dsql%3A+Zero%2Dshot+text%2Dto%2Dsql+using+monte+carlo+tree+search%2E+Preprint%2C+arXiv%3A2502%2E17248%2E+%E2%80%94+%5BarXiv%3A2502%2E17248%5D%28ht)
11. Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, and 1 others. 2024b. [Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls](https://md.research.elamur.ai/free/en/papers/li-2023-llm-already-serve-database-interface/). Advances in Neural Information Processing Systems, 36.
12. Miaoran Li, Jiangning Chen, Minghua Xu, and Xiaolong Wang. 2025b. Hallucination detection in structured query generation via llm self-debating. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 16102–16113. [find](https://scholar.google.com/scholar?q=Miaoran+Li%2C+Jiangning+Chen%2C+Minghua+Xu%2C+and+Xiaolong+Wang%2E+2025b%2E+Hallucination+detection+in+structured+query+generation+via+llm+self%2Ddebating%2E+In+Findings+of+the+Association+for+Computational+Linguis)
13. Hanbing Liu, Haoyang Li, Xiaokang Zhang, Ruotong Chen, Haiyong Xu, Tian Tian, Qi Qi, and Jing Zhang. 2025. Uncovering the impact of chain-of-thought reasoning for direct preference optimization: Lessons from text-to-sql. Preprint, arXiv:2502.11656. [find](https://scholar.google.com/scholar?q=Hanbing+Liu%2C+Haoyang+Li%2C+Xiaokang+Zhang%2C+Ruotong+Chen%2C+Haiyong+Xu%2C+Tian+Tian%2C+Qi+Qi%2C+and+Jing+Zhang%2E+2025%2E+Uncovering+the+impact+of+chain%2Dof%2Dthought+reasoning+for+direct+preference+optimization%3A+Lesso)
14. Giang Nguyen, Ivan Brugere, Shubham Sharma, Sanjay Kariyappa, Anh Totti Nguyen, and Freddy Lecue. 2025. Interpretable llm-based table question answering. Preprint, arXiv:2412.12386. — [arXiv:2412.12386](https://arxiv.org/abs/2412.12386) [find](https://scholar.google.com/scholar?q=Giang+Nguyen%2C+Ivan+Brugere%2C+Shubham+Sharma%2C+Sanjay+Kariyappa%2C+Anh+Totti+Nguyen%2C+and+Freddy+Lecue%2E+2025%2E+Interpretable+llm%2Dbased+table+question+answering%2E+Preprint%2C+arXiv%3A2412%2E12386%2E+%E2%80%94+%5BarXiv%3A2412%2E1238)
15. Ansong Ni, Srini Iyer, Dragomir Radev, Veselin Stoyanov, Wen-tau Yih, Sida Wang, and Xi Victoria Lin. 2023. [Lever: Learning to verify language-to-code generation with execution](https://md.research.elamur.ai/free/en/papers/ni-2023-lever-learning-verify-language-code/). In International Conference on Machine Learning, pages 26106–26128. PMLR.
16. Mohammadreza Pourreza and Davood Rafiei. 2024. [Din-sql: Decomposed in-context learning of text-to-sql with self-correction](https://md.research.elamur.ai/free/en/papers/pourreza-2023-din-sql-decomposed-context-learning/). Advances in Neural Information Processing Systems, 36.
17. Ge Qu, Jinyang Li, Bowen Li, Bowen Qin, Nan Huo, Chenhao Ma, and Reynold Cheng. 2024. Before generation, align it! a novel and effective strategy for mitigating hallucinations in text-to-sql generation. arXiv preprint arXiv:2405.15307. [find](https://scholar.google.com/scholar?q=Ge+Qu%2C+Jinyang+Li%2C+Bowen+Li%2C+Bowen+Qin%2C+Nan+Huo%2C+Chenhao+Ma%2C+and+Reynold+Cheng%2E+2024%2E+Before+generation%2C+align+it%21+a+novel+and+effective+strategy+for+mitigating+hallucinations+in+text%2Dto%2Dsql+generatio)
18. Ohad Rubin and Jonathan Berant. 2020. Smbop: Semi-autoregressive bottom-up semantic parsing. arXiv preprint arXiv:2010.12412. [find](https://scholar.google.com/scholar?q=Ohad+Rubin+and+Jonathan+Berant%2E+2020%2E+Smbop%3A+Semi%2Dautoregressive+bottom%2Dup+semantic+parsing%2E+arXiv+preprint+arXiv%3A2010%2E12412%2E)
19. Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. Chess: Contextual harnessing for efficient sql synthesis. arXiv preprint arXiv:2405.16755. — [arXiv:2405.16755](https://arxiv.org/abs/2405.16755) [find](https://scholar.google.com/scholar?q=Shayan+Talaei%2C+Mohammadreza+Pourreza%2C+Yu%2DChen+Chang%2C+Azalia+Mirhoseini%2C+and+Amin+Saberi%2E+2024%2E+Chess%3A+Contextual+harnessing+for+efficient+sql+synthesis%2E+arXiv+preprint+arXiv%3A2405%2E16755%2E+%E2%80%94+%5BarXiv%3A2405%2E)
20. Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. 2019. [Rat-sql: Relation-aware schema encoding and linking for text-to-sql parsers](https://md.research.elamur.ai/free/en/papers/wang-2020-rat-sql-relation-aware-schema/). arXiv preprint arXiv:1911.04942.
21. Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, LinZheng Chai, Zhao Yan, Qian-Wen Zhang, Di Yin, Xing Sun, and Zhoujun Li. 2025a. [find](https://scholar.google.com/scholar?q=Bing+Wang%2C+Changyu+Ren%2C+Jian+Yang%2C+Xinnian+Liang%2C+Jiaqi+Bai%2C+LinZheng+Chai%2C+Zhao+Yan%2C+Qian%2DWen+Zhang%2C+Di+Yin%2C+Xing+Sun%2C+and+Zhoujun+Li%2E+2025a%2E)
22. [MAC-SQL: A multi-agent collaborative framework for text-to-SQL](https://md.research.elamur.ai/free/en/papers/wang-2025-mac-sql-multi-agent-collaborative/). In Proceedings of the 31st International Conference on Computational Linguistics, pages 540–557, Abu Dhabi, UAE. Association for Computational Linguistics.
23. Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, LinZheng Chai, Zhao Yan, Qian-Wen Zhang, Di Yin, Xing Sun, and Zhoujun Li. 2025b. [Mac-sql: A multi-agent collaborative framework for text-to-sql](https://md.research.elamur.ai/free/en/papers/wang-2025-mac-sql-multi-agent-collaborative/). Preprint, arXiv:2312.11242.
24. Yihan Wang, Peiyu Liu, and Xin Yang. 2025c. Linkalign: Scalable schema linking for real-world Preprint, large-scale multi-database text-to-sql. arXiv:2503.18596. — [arXiv:2503.18596](https://arxiv.org/abs/2503.18596) [find](https://scholar.google.com/scholar?q=Yihan+Wang%2C+Peiyu+Liu%2C+and+Xin+Yang%2E+2025c%2E+Linkalign%3A+Scalable+schema+linking+for+real%2Dworld+Preprint%2C+large%2Dscale+multi%2Ddatabase+text%2Dto%2Dsql%2E+arXiv%3A2503%2E18596%2E+%E2%80%94+%5BarXiv%3A2503%2E18596%5D%28https%3A%2F%2Farxiv%2Eorg)
25. Hanchen Xia, Feng Jiang, Naihao Deng, Cunxiang Wang, Guojiang Zhao, Rada Mihalcea, and Yue Zhang. 2024. r 3 : "this is my sql, are you with me?" a consensus-based multi-agent system for text-to-sql tasks. Preprint, arXiv:2402.14851. [find](https://scholar.google.com/scholar?q=Hanchen+Xia%2C+Feng+Jiang%2C+Naihao+Deng%2C+Cunxiang+Wang%2C+Guojiang+Zhao%2C+Rada+Mihalcea%2C+and+Yue+Zhang%2E+2024%2E+r+3+%3A+%22this+is+my+sql%2C+are+you+with+me%3F%22+a+consensus%2Dbased+multi%2Dagent+system+for+text%2Dto%2Dsql+ta)
26. Bo Yang, Yinfen Xia, Weisong Sun, and Yang Liu. 2025. Hallucination detection for llm-based text-to-sql generation via two-stage metamorphic testing. Preprint, arXiv:2512.22250. — [arXiv:2512.22250](https://arxiv.org/abs/2512.22250) [find](https://scholar.google.com/scholar?q=Bo+Yang%2C+Yinfen+Xia%2C+Weisong+Sun%2C+and+Yang+Liu%2E+2025%2E+Hallucination+detection+for+llm%2Dbased+text%2Dto%2Dsql+generation+via+two%2Dstage+metamorphic+testing%2E+Preprint%2C+arXiv%3A2512%2E22250%2E+%E2%80%94+%5BarXiv%3A2512%2E22250%5D%28h)
27. Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2019. [Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task](https://md.research.elamur.ai/free/en/papers/yu-2018-spider-large-scale-human-labeled/). Preprint, arXiv:1809.08887.
28. Bohan Zhai, Canwen Xu, Yuxiong He, and Zhewei Yao. 2025. Excot: Optimizing reasoning for Preprint, text-to-sql with execution feedback. arXiv:2503.19988. — [arXiv:2503.19988](https://arxiv.org/abs/2503.19988) [find](https://scholar.google.com/scholar?q=Bohan+Zhai%2C+Canwen+Xu%2C+Yuxiong+He%2C+and+Zhewei+Yao%2E+2025%2E+Excot%3A+Optimizing+reasoning+for+Preprint%2C+text%2Dto%2Dsql+with+execution+feedback%2E+arXiv%3A2503%2E19988%2E+%E2%80%94+%5BarXiv%3A2503%2E19988%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F250)

*Updated 1 November 2026*