---
title: "Thesis Proposal — paper by Mao (2026) — summary and ideas"
description: "LLM agents that call external tools make long chains of dependent decisions. 35 references."
authors: ["Zhenjiang Mao"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)"
source: "https://aclanthology.org/2026.acl-srw.78/"
doi: "10.18653/v1/2026.acl-srw.78"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mao-2026-thesis-proposal-when-does-agent/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mao-2026-thesis-proposal-when-does-agent/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/) / Thesis Proposal: When Does an Agent Know It Is Lost? Confidence Trajectory Analysis for Tool-Using LLMs

# Thesis Proposal: When Does an Agent Know It Is Lost? Confidence Trajectory Analysis for Tool-Using LLMs

**Authors:** [Zhenjiang Mao](https://md.research.elamur.ai/free/en/authors/zhenjiang-mao/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-srw.78/) · [PDF](https://aclanthology.org/2026.acl-srw.78.pdf) · [DOI 10.18653/v1/2026.acl-srw.78](https://doi.org/10.18653/v1/2026.acl-srw.78) · [OpenAlex](https://openalex.org/W7166878287)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
LLM agents that call external tools make long chains of dependent decisions. One early slip, such as an invented API argument, can quietly spoil every later step. Current uncertainty quantification (UQ) methods score steps independently and give every kind of action the same treatment. This thesis proposal sketches a three-stage pipeline for tracking confidence across a whole episode.

- **Stage 1** adapts the training-free RCC estimator to think-act-observe loops. Confidence is derived from the logits of the Think segment, which is divided into planning, tool-choice and argument-building parts. The score is then propagated recurrently, with a discount that lowers it when a tool's output contradicts what the agent anticipated.
- **Stage 2** maps raw actions to a few semantic categories, using embedding clustering refined by an LLM. The result is a typed confidence trajectory.
- **Stage 3** mines Signal Temporal Logic (STL) formulas with type-aware predicates to forecast task success and explain failures.

Pilot experiments indicate well-calibrated step-level confidence and compressible, discriminative structure in agent action spaces. The approach could give early, interpretable warnings that an agent is going off track.

## Key points
- Preliminary results: the RCC estimator achieves ECE as low as 3.63%. SVD-based action aggregation reaches AUROC up to 0.935. Effective ranks of 3–7 across three benchmarks hint at a plausible number of action types.
- The novel components are:
  - an observation-consistency discount γ(o_{t−1}) in the recurrent gate;
  - the typed trajectory C⁺ = {(c_t, k_t)};
  - STL extended with typed atomic propositions and type-transition predicates.
- Planned evaluation uses AgentBench, API-Bank and ToolBench, later extended to WebArena, SWE-rebench (67,074 trajectories) and AgentReward-Bench (1,302 annotated trajectories).
- Each stage has its own success criterion:
  - Stage 1: ECE/NLL.
  - Stage 2: Krippendorff's α ≥ 0.6 against human labels from three researchers on 200 actions per benchmark.
  - Stage 3: AUROC and early-exit lead time.
- Key hyperparameters are K ∈ [6, 10] action types and formula depth D ≤ 3, tuned on a 20% validation split. The full method requires token log-probabilities, so it is limited to open-weight models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Florida](https://md.research.elamur.ai/free/en/universities/university-of-florida/) — University, Gainesville, United States · 33 works · 364 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Zhenjiang Mao. 2026. Thesis Proposal: When Does an Agent Know It Is Lost? Confidence Trajectory Analysis for Tool-Using LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mao2026thesis,
  title = {{Thesis Proposal: When Does an Agent Know It Is Lost? Confidence Trajectory Analysis for Tool-Using LLMs}},
  author = {Zhenjiang Mao},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.78},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.78}
}
```

- [← previous Thesis Proposal: Toward a Human-Centered and Perspective-Aware Framework for Reproducible ML Evaluation and AI Alignment](https://md.research.elamur.ai/free/en/papers/pandita-2026-thesis-proposal-human-centered-perspective/)
- [next → Think Before Writing: Feature-Level Multi-Objective Optimization for Generative Citation Visibility](https://md.research.elamur.ai/free/en/papers/liu-2026-think-before-writing-feature-level/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/)

## References

35 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. 2025. Why do multi-agent LLM systems fail? arXiv preprint arXiv:2503.13657. [find](https://scholar.google.com/scholar?q=Mert+Cemri%2C+Melissa+Z%2E+Pan%2C+Shuyi+Yang%2C+Lakshya+A%2E+Agrawal%2C+Bhavya+Chopra%2C+Rishabh+Tiwari%2C+Kurt+Keutzer%2C+Aditya+Parameswaran%2C+Dan+Klein%2C+Kannan+Ramchandran%2C+Matei+Zaharia%2C+Joseph+E%2E+Gonzalez%2C+and+Ion+)
2. Jinhao Duan, Hao Cheng, Shiqi Wang, Alexander Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. 2024. SAR: Shifting attention to relevance—towards the predictive uncertainty quantification of free-form large language models. In Proceedings of the 62nd Annual Meeting of the Associatio… [find](https://scholar.google.com/scholar?q=Jinhao+Duan%2C+Hao+Cheng%2C+Shiqi+Wang%2C+Alexander+Zavalny%2C+Chenan+Wang%2C+Renjing+Xu%2C+Bhavya+Kailkhura%2C+and+Kaidi+Xu%2E+2024%2E+SAR%3A+Shifting+attention+to+relevance%E2%80%94towards+the+predictive+uncertainty+quantifica)
3. Jinhao Duan, James Diffenderfer, Sandeep Madireddy, Tianlong Chen, Bhavya Kailkhura, and Kaidi Xu. 2025. UProp: Investigating the uncertainty propagation of LLMs in multi-step agentic decision-making. arXiv preprint arXiv:2506.17419. — [arXiv:2506.17419](https://arxiv.org/abs/2506.17419) [find](https://scholar.google.com/scholar?q=Jinhao+Duan%2C+James+Diffenderfer%2C+Sandeep+Madireddy%2C+Tianlong+Chen%2C+Bhavya+Kailkhura%2C+and+Kaidi+Xu%2E+2025%2E+UProp%3A+Investigating+the+uncertainty+propagation+of+LLMs+in+multi%2Dstep+agentic+decision%2Dmaking%2E)
4. Isaac Gibbs and Emmanuel Candès. 2021. Adaptive conformal inference under distribution shift. In Advances in Neural Information Processing Systems (NeurIPS). [find](https://scholar.google.com/scholar?q=Isaac+Gibbs+and+Emmanuel+Cand%C3%A8s%2E+2021%2E+Adaptive+conformal+inference+under+distribution+shift%2E+In+Advances+in+Neural+Information+Processing+Systems+%28NeurIPS%29%2E)
5. Michael Kirchhof, Gjergji Kasneci, and Enkelejda Kasneci. 2025. Position: Uncertainty quantification needs reassessment for large-language model agents. In Proceedings of the 42nd International Conference on Machine Learning (ICML). [find](https://scholar.google.com/scholar?q=Michael+Kirchhof%2C+Gjergji+Kasneci%2C+and+Enkelejda+Kasneci%2E+2025%2E+Position%3A+Uncertainty+quantification+needs+reassessment+for+large%2Dlanguage+model+agents%2E+In+Proceedings+of+the+42nd+International+Confer)
6. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. In The Eleventh International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Lorenz+Kuhn%2C+Yarin+Gal%2C+and+Sebastian+Farquhar%2E+2023%2E+Semantic+uncertainty%3A+Linguistic+invariances+for+uncertainty+estimation+in+natural+language+generation%2E+In+The+Eleventh+International+Conference+o)
7. Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. [API-Bank: A comprehensive benchmark for tool-augmented LLMs](https://md.research.elamur.ai/free/en/papers/li-2023-api-bank-comprehensive-benchmark-tool/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP).
8. Yinghao Li and 1 others. 2025. Language model uncertainty quantification with attention chain. arXiv preprint arXiv:2503.19168. — [arXiv:2503.19168](https://arxiv.org/abs/2503.19168) [find](https://scholar.google.com/scholar?q=Yinghao+Li+and+1+others%2E+2025%2E+Language+model+uncertainty+quantification+with+attention+chain%2E+arXiv+preprint+arXiv%3A2503%2E19168%2E+%E2%80%94+%5BarXiv%3A2503%2E19168%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2503%2E19168%29)
9. Dongrui Liu, Qihan Ren, Chen Qian, Shuai Shao, Yuejin Xie, Yu Li, Zhonghao Yang, Haoyu Luo, Peng Wang, Qingyu Liu, and 1 others. 2026. AgentDoG: A diagnostic guardrail framework for AI agent safety and security. arXiv preprint arXiv:2601.18491. — [arXiv:2601.18491](https://arxiv.org/abs/2601.18491) [find](https://scholar.google.com/scholar?q=Dongrui+Liu%2C+Qihan+Ren%2C+Chen+Qian%2C+Shuai+Shao%2C+Yuejin+Xie%2C+Yu+Li%2C+Zhonghao+Yang%2C+Haoyu+Luo%2C+Peng+Wang%2C+Qingyu+Liu%2C+and+1+others%2E+2026%2E+AgentDoG%3A+A+diagnostic+guardrail+framework+for+AI+agent+safety+an)
10. Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, and 1 others. 2024. [AgentBench: Evaluating LLMs as agents](https://md.research.elamur.ai/free/en/papers/liu-2023-agentbench-evaluating-llms-agents/). In The Twelfth International Conference on Learning Representations (ICLR).
11. Jiaying Lu, Bo Pan, Jieyi Chen, Yingchaojie Feng, Jingyuan Hu, Yuchen Peng, and Wei Chen. 2025. AgentLens: Visual analysis for agent behaviors in LLM-based autonomous systems. IEEE Transactions on Visualization and Computer Graphics, 31:4182–4194. [find](https://scholar.google.com/scholar?q=Jiaying+Lu%2C+Bo+Pan%2C+Jieyi+Chen%2C+Yingchaojie+Feng%2C+Jingyuan+Hu%2C+Yuchen+Peng%2C+and+Wei+Chen%2E+2025%2E+AgentLens%3A+Visual+analysis+for+agent+behaviors+in+LLM%2Dbased+autonomous+systems%2E+IEEE+Transactions+on+Vis)
12. Xing Han Lù, Amirhossein Kazemnejad, Nicholas Meade, Arkil Patel, Dongchan Shin, Alejandra Zambrano, Karolina Stańczak, Peter Shaw, Christopher J. Pal, and Siva Reddy. 2025. AgentRewardBench: Evaluating automatic evaluations of web agent trajectories. arXiv preprint arXiv:2504.08942. — [arXiv:2504.08942](https://arxiv.org/abs/2504.08942) [find](https://scholar.google.com/scholar?q=Xing+Han+L%C3%B9%2C+Amirhossein+Kazemnejad%2C+Nicholas+Meade%2C+Arkil+Patel%2C+Dongchan+Shin%2C+Alejandra+Zambrano%2C+Karolina+Stan%CC%81czak%2C+Peter+Shaw%2C+Christopher+J%2E+Pal%2C+and+Siva+Reddy%2E+2025%2E+AgentRewardBench%3A+Evaluat)
13. Zhenjiang Mao, Artem Bisliouk, Rakesh Ravi Nama, and Ivan Ruchkin. 2025a. Temporalizing confidence: Evaluation of chain-of-thought reasoning with signal temporal logic. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications. [find](https://scholar.google.com/scholar?q=Zhenjiang+Mao%2C+Artem+Bisliouk%2C+Rakesh+Ravi+Nama%2C+and+Ivan+Ruchkin%2E+2025a%2E+Temporalizing+confidence%3A+Evaluation+of+chain%2Dof%2Dthought+reasoning+with+signal+temporal+logic%2E+In+Proceedings+of+the+20th+Work)
14. Zhenjiang Mao, Mrinall Eashaan Umasudhan, and Ivan Ruchkin. 2024. How safe Am I given what I see? Calibrated prediction of safety chances for image-controlled autonomy. In Proceedings of the 6th Annual Learning for Dynamics and Control Conference (L4DC). [find](https://scholar.google.com/scholar?q=Zhenjiang+Mao%2C+Mrinall+Eashaan+Umasudhan%2C+and+Ivan+Ruchkin%2E+2024%2E+How+safe+Am+I+given+what+I+see%3F+Calibrated+prediction+of+safety+chances+for+image%2Dcontrolled+autonomy%2E+In+Proceedings+of+the+6th+Annua)
15. Zhenjiang Mao, Mrinall Eashaan Umasudhan, and Ivan Ruchkin. 2025b. How safe will I be given what I saw? Calibrated prediction of safety chances for image-controlled autonomy. arXiv preprint arXiv:2508.09346. — [arXiv:2508.09346](https://arxiv.org/abs/2508.09346) [find](https://scholar.google.com/scholar?q=Zhenjiang+Mao%2C+Mrinall+Eashaan+Umasudhan%2C+and+Ivan+Ruchkin%2E+2025b%2E+How+safe+will+I+be+given+what+I+saw%3F+Calibrated+prediction+of+safety+chances+for+image%2Dcontrolled+autonomy%2E+arXiv+preprint+arXiv%3A2508)
16. Zhenjiang Mao and Anirudhh Venkat. 2026. Recurrent confidence chain: Temporal-aware uncertainty quantification in large language models. arXiv preprint arXiv:2601.13368. — [arXiv:2601.13368](https://arxiv.org/abs/2601.13368) [find](https://scholar.google.com/scholar?q=Zhenjiang+Mao+and+Anirudhh+Venkat%2E+2026%2E+Recurrent+confidence+chain%3A+Temporal%2Daware+uncertainty+quantification+in+large+language+models%2E+arXiv+preprint+arXiv%3A2601%2E13368%2E+%E2%80%94+%5BarXiv%3A2601%2E13368%5D%28https%3A%2F%2Fa)
17. Zhenjiang Mao, Anirudhh Venkat, Artem Bisliouk, Akshat Kothiyal, Sindhura Kumbakonam Subramanian, Saithej Singhu, and Ivan Ruchkin. 2026. [Confidence over time: Confidence calibration with temporal logic for large language model reasoning](https://md.research.elamur.ai/free/en/papers/mao-2026-confidence-over-time-confidence-calibration/). arXiv preprint arXiv:2601.13387.
18. Changdae Oh, Seongheon Park, To Eun Kim, Jiatong Li, Wendi Li, Samuel Yeh, Xuefeng Du, Hamed Hassani, Paul Bogdan, Dawn Song, and Sharon Li. 2026. [Uncertainty quantification in LLM agents: Foundations, emerging challenges, and opportunities](https://md.research.elamur.ai/free/en/papers/oh-2026-uncertainty-quantification-llm-agents-foundations/). Preprint, arXiv:2602.05073.
19. Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2024. [Gorilla: Large language model connected with massive APIs](https://md.research.elamur.ai/free/en/papers/patil-2023-gorilla-large-language-model-connected/). In Advances in Neural Information Processing Systems (NeurIPS).
20. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, and 1 others. 2024. [ToolLLM: Facilitating large language models to master 16000+ real-world APIs](https://md.research.elamur.ai/free/en/papers/qin-2023-toolllm-facilitating-large-language-models/). In The Twelfth International Conference on Learning Representations (ICLR).
21. Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi Jaakkola, and Regina Barzilay. 2024. Conformal language modeling. In The Twelfth International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Victor+Quach%2C+Adam+Fisch%2C+Tal+Schuster%2C+Adam+Yala%2C+Jae+Ho+Sohn%2C+Tommi+Jaakkola%2C+and+Regina+Barzilay%2E+2024%2E+Conformal+language+modeling%2E+In+The+Twelfth+International+Conference+on+Learning+Representati)
22. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. [Toolformer: Language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). In Advances in Neural Information Processing Systems (NeurIPS).
23. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2021. [ALFWorld: Aligning text and embodied environments for interactive learning](https://md.research.elamur.ai/free/en/papers/shridhar-2020-alfworld-aligning-text-embodied-environments/). In Proceedings of the International Conference on Learning Representations (ICLR).
24. Caleb Sobolewski, Zhenjiang Mao, Kasper Vejre, and Ivan Ruchkin. 2025. Generalizable image repair for robust visual control. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). [find](https://scholar.google.com/scholar?q=Caleb+Sobolewski%2C+Zhenjiang+Mao%2C+Kasper+Vejre%2C+and+Ivan+Ruchkin%2E+2025%2E+Generalizable+image+repair+for+robust+visual+control%2E+In+2025+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems)
25. Qiwei Zhao, Dong Li, Yanchi Liu, Wei Cheng, Yiyou Sun, Mika Oishi, Takao Osaki, Katsushi Matsuda, Huaxiu Yao, Chen Zhao, Haifeng Chen, and Xujiang Zhao. 2025. Uncertainty propagation on LLM agent. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: L… [find](https://scholar.google.com/scholar?q=Qiwei+Zhao%2C+Dong+Li%2C+Yanchi+Liu%2C+Wei+Cheng%2C+Yiyou+Sun%2C+Mika+Oishi%2C+Takao+Osaki%2C+Katsushi+Matsuda%2C+Huaxiu+Yao%2C+Chen+Zhao%2C+Haifeng+Chen%2C+and+Xujiang+Zhao%2E+2025%2E+Uncertainty+propagation+on+LLM+agent%2E+In+)
26. Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. 2024. [WebArena: A realistic web environment for building autonomous agents](https://md.research.elamur.ai/free/en/papers/zhou-2023-webarena-realistic-web-environment-building/). In The Twelfth International Conference on Learning Representations (ICLR).
27. Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, and Dinesh Manocha. 2026. [Structured uncertainty guided clarification for LLM agents](https://md.research.elamur.ai/free/en/papers/suri-2026-structured-uncertainty-guided-clarification-llm/). Preprint, arXiv:2511.08798.
28. Alex Tamkin and 1 others. 2024. Clio: Privacy-preserving insights into real-world AI use. arXiv preprint arXiv:2412.13678. [find](https://scholar.google.com/scholar?q=Alex+Tamkin+and+1+others%2E+2024%2E+Clio%3A+Privacy%2Dpreserving+insights+into+real%2Dworld+AI+use%2E+arXiv+preprint+arXiv%3A2412%2E13678%2E)
29. Maria Trofimova, Anton Shevtsov, Ibragim Badertdinov, Konstantin Pyaev, Simon Karasik, and Alexander Golubev. 2025. SWE-rebench Open-Hands trajectories. Hugging Face Datasets, [https://huggingface.co/datasets/nebius/](https://huggingface.co/datasets/nebius/) SWE-rebench-openhands-trajectories. [↗](https://huggingface.co/datasets/nebius/)
30. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). In The Eleventh International Conference on Learning Representations (ICLR).
31. Chen Xu and Yao Xie. 2023. Sequential predictive conformal inference for time series. In Proceedings of the 40th International Conference on Machine Learning (ICML). [find](https://scholar.google.com/scholar?q=Chen+Xu+and+Yao+Xie%2E+2023%2E+Sequential+predictive+conformal+inference+for+time+series%2E+In+Proceedings+of+the+40th+International+Conference+on+Machine+Learning+%28ICML%29%2E)
32. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. [ReAct: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In The Eleventh International Conference on Learning Representations (ICLR).
33. Boxuan Zhang and Ruqi Zhang. 2025. CoT-UQ: Improving response-wise uncertainty quantification in LLMs with chain-of-thought. In Findings of the Association for Computational Linguistics: ACL 2025. [find](https://scholar.google.com/scholar?q=Boxuan+Zhang+and+Ruqi+Zhang%2E+2025%2E+CoT%2DUQ%3A+Improving+response%2Dwise+uncertainty+quantification+in+LLMs+with+chain%2Dof%2Dthought%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2025%2E)
34. Jiaxin Zhang, Caiming Xiong, and Chien-Sheng Wu. 2026. Agentic confidence calibration. arXiv preprint arXiv:2601.15778. — [arXiv:2601.15778](https://arxiv.org/abs/2601.15778) [find](https://scholar.google.com/scholar?q=Jiaxin+Zhang%2C+Caiming+Xiong%2C+and+Chien%2DSheng+Wu%2E+2026%2E+Agentic+confidence+calibration%2E+arXiv+preprint+arXiv%3A2601%2E15778%2E+%E2%80%94+%5BarXiv%3A2601%2E15778%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2601%2E15778%29)
35. Yuwei Zhang, Zihan Wang, and Jingbo Shang. 2023. ClusterLLM: Large language models as a guide for text clustering. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Yuwei+Zhang%2C+Zihan+Wang%2C+and+Jingbo+Shang%2E+2023%2E+ClusterLLM%3A+Large+language+models+as+a+guide+for+text+clustering%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Proces)

*Updated 1 November 2026*