---
title: "ADVICE: Answer-Dependent Verbalized Confidence Estimation"
description: "They compare confidence distributions produced for different candidate answers to the same question… 39 references, cited by 1 paper here."
authors: ["KiJung Seo", "Sehun Lim", "Taeuk Kim"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1098/"
doi: "10.18653/v1/2026.acl-long.1098"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/seo-2026-advice-answer-dependent-verbalized-confidence/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/seo-2026-advice-answer-dependent-verbalized-confidence/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/9/) / ADVICE: Answer-Dependent Verbalized Confidence Estimation

# ADVICE: Answer-Dependent Verbalized Confidence Estimation

**Authors:** KiJung Seo, Sehun Lim and [Taeuk Kim](https://md.research.elamur.ai/free/en/authors/taeuk-kim/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1098/) · [PDF](https://aclanthology.org/2026.acl-long.1098.pdf) · [DOI 10.18653/v1/2026.acl-long.1098](https://doi.org/10.18653/v1/2026.acl-long.1098) · [OpenAlex](https://openalex.org/W7166846701)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large language models can state their confidence in words, but this verbalized confidence tends to be inflated whether or not the answer is correct. Seo, Lim and Kim ask why this happens. They compare confidence distributions produced for different candidate answers to the same question using Jensen–Shannon divergence, and they probe the models with Attention Rollout and Integrated Gradients. Both analyses show that Gemma-2-9B-IT and Llama-3.1-8B-Instruct barely consider their own answer when reporting confidence. The authors call this answer-independence and argue that it is the main driver of overconfidence.

Their remedy, ADVICE, is a lightweight fine-tuning framework. It is trained on TriviaQA triplets that pair a question with a correct and a wrong model-generated answer. Four objectives tie the confidence to the answer while keeping general task ability intact. ADVICE reaches calibration comparable to state-of-the-art sampling- and fine-tuning-based methods. It also transfers to unseen confidence formats, adds gains when combined with other methods, and needs few extra tokens. Follow-up probing confirms that answer tokens gain influence after training. Trustworthy confidence matters for high-stakes uses such as law and healthcare.

## Key points
- **Diagnostic analysis.** For each question, the authors sample up to 30 answers with top-p sampling and compute pairwise JSD between the resulting confidence distributions on TriviaQA, MMLU and LogiQA. Most pairs fall near zero (JSD ≤ 0.1), meaning confidence hardly changes with the answer.
- **Attribution evidence.** Attention Rollout shows weaker Confidence→Answer flow than Answer→Question or Confidence→Question flow. Integrated Gradients gives answer tokens less weight than tokens like "user" or BOS.
- **Training data.** ADVICE starts from 4,000 TriviaQA training instances and keeps only those the model answers correctly under greedy decoding. Each kept question is paired with correct and wrong answers to form contrastive training signals.
- **Causal check.** After training with ADVICE, Confidence→Answer attention rises significantly (t-test) for both models. The rank of the answer token among the top-10 attributed tokens also climbs as training progresses.
- **Limitations.** Building contrastive pairs adds data construction cost, and the evaluation is restricted to short-form and multiple-choice QA. On high-accuracy data such as SciQ (>90% accuracy), the base model is already the best calibrated.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hanyang University](https://md.research.elamur.ai/free/en/universities/hanyang-university/) — University, Seoul, South Korea · 6 works · 150 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: KiJung Seo, Sehun Lim and Taeuk Kim. 2026. ADVICE: Answer-Dependent Verbalized Confidence Estimation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{seo2026advice,
  title = {{ADVICE: Answer-Dependent Verbalized Confidence Estimation}},
  author = {KiJung Seo, Sehun Lim and Taeuk Kim},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1098},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1098}
}
```

- [← previous Adversarial Decoding: Generating Readable Documents for Adversarial Objectives](https://md.research.elamur.ai/free/en/papers/zhang-2026-adversarial-decoding-generating-readable-documents/)
- [next → AFourP at SemEval-2026 Task 2: Predicting Variation in Emotional Valence and Arousal over Time from Ecological Essays](https://md.research.elamur.ai/free/en/papers/thota-2026-afourp-semeval-2026-task-2/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/9/)

## References

39 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Samira Abnar and Willem Zuidema. 2020. [Quantifying attention flow in transformers](https://md.research.elamur.ai/free/en/papers/abnar-2020-quantifying-attention-flow-transformers/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4190–4197, Online. Association for Computational Linguistics.
2. Kendrick Boyd, Kevin H. Eng, and C. David Page. 2013. Area under the precision-recall curve: Point estimates and confidence intervals. In Machine Learning and Knowledge Discovery in Databases, pages 451–466, Berlin, Heidelberg. Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Kendrick+Boyd%2C+Kevin+H%2E+Eng%2C+and+C%2E+David+Page%2E+2013%2E+Area+under+the+precision%2Drecall+curve%3A+Point+estimates+and+confidence+intervals%2E+In+Machine+Learning+and+Knowledge+Discovery+in+Databases%2C+pages+4)
3. GLENN W. Brier. 1950. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1):1 – 3. [find](https://scholar.google.com/scholar?q=GLENN+W%2E+Brier%2E+1950%2E+Verification+of+forecasts+expressed+in+terms+of+probability%2E+Monthly+Weather+Review%2C+78%281%29%3A1+%E2%80%93+3%2E)
4. Oscar Blessed Deho, Michael Bewong, Selasi Kwashie, Jiuyong Li, Jixue Liu, Lin Liu, and Srecko Joksi-movic. 2025. Is it still fair? a comparative evaluation of fairness algorithms through the lens of covariate drift. Mach. Learn., 114(1). [find](https://scholar.google.com/scholar?q=Oscar+Blessed+Deho%2C+Michael+Bewong%2C+Selasi+Kwashie%2C+Jiuyong+Li%2C+Jixue+Liu%2C+Lin+Liu%2C+and+Srecko+Joksi%2Dmovic%2E+2025%2E+Is+it+still+fair%3F+a+comparative+evaluation+of+fairness+algorithms+through+the+lens+of+)
5. Kobe Desender, K Richard Ridderinkhof, and Peter R Murphy. 2021. Understanding neural signals of post-decisional performance monitoring: An integrative review. eLife, 10:e67556. [find](https://scholar.google.com/scholar?q=Kobe+Desender%2C+K+Richard+Ridderinkhof%2C+and+Peter+R+Murphy%2E+2021%2E+Understanding+neural+signals+of+post%2Ddecisional+performance+monitoring%3A+An+integrative+review%2E+eLife%2C+10%3Ae67556%2E)
6. Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models. In The 2023 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Jasmijn+Bastings%2C+Katja+Filippova%2C+and+Amir+Globerson%2E+2023%2E+Dissecting+recall+of+factual+associations+in+auto%2Dregressive+language+models%2E+In+The+2023+Conference+on+Empirical+Methods+in+Natu)
7. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
8. Tobias Groot and Matias Valdenegro Toro. 2024. Overconfidence is key: Verbalized uncertainty evaluation in large language and vision-language models. In Proceedings of the 4th Workshop on Trustworthy Natural Language Processing (TrustNLP 2024), pages 145–171, Mexico City, Mexico. Association for Com… [find](https://scholar.google.com/scholar?q=Tobias+Groot+and+Matias+Valdenegro+Toro%2E+2024%2E+Overconfidence+is+key%3A+Verbalized+uncertainty+evaluation+in+large+language+and+vision%2Dlanguage+models%2E+In+Proceedings+of+the+4th+Workshop+on+Trustworthy+)
9. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1321– 1330. PMLR. [find](https://scholar.google.com/scholar?q=Chuan+Guo%2C+Geoff+Pleiss%2C+Yu+Sun%2C+and+Kilian+Q%2E+Weinberger%2E+2017%2E+On+calibration+of+modern+neural+networks%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning%2C+volume+70+of+Proceed)
10. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations.
11. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
12. Thanmay Jayakumar, Fauzan Farooqui, and Luqman Farooqui. 2023. Large language models are legal but they are not: Making the case for a powerful Legal-LLM. In Proceedings of the Natural Legal Language Processing Workshop 2023, pages 223–229, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Thanmay+Jayakumar%2C+Fauzan+Farooqui%2C+and+Luqman+Farooqui%2E+2023%2E+Large+language+models+are+legal+but+they+are+not%3A+Making+the+case+for+a+powerful+Legal%2DLLM%2E+In+Proceedings+of+the+Natural+Legal+Language+)
13. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. [Survey of hallucination in natural language generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/). ACM Computing Surveys, 55(12):1–38.
14. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
15. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611, Vancouver…
16. Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. 2025. Why language models hallucinate. Preprint, arXiv:2509.04664. — [arXiv:2509.04664](https://arxiv.org/abs/2509.04664) [find](https://scholar.google.com/scholar?q=Adam+Tauman+Kalai%2C+Ofir+Nachum%2C+Santosh+S%2E+Vempala%2C+and+Edwin+Zhang%2E+2025%2E+Why+language+models+hallucinate%2E+Preprint%2C+arXiv%3A2509%2E04664%2E+%E2%80%94+%5BarXiv%3A2509%2E04664%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2509%2E04664%29)
17. Jixuan Leng, Chengsong Huang, Banghua Zhu, and Jiaxin Huang. 2025. Taming overconfidence in LLMs: Reward calibration in RLHF. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jixuan+Leng%2C+Chengsong+Huang%2C+Banghua+Zhu%2C+and+Jiaxin+Huang%2E+2025%2E+Taming+overconfidence+in+LLMs%3A+Reward+calibration+in+RLHF%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations%2E)
18. Yibo Li, Miao Xiong, Jiaying Wu, and Bryan Hooi. 2025. Conftuner: Training large language models to express their confidence verbally. Preprint, arXiv:2508.18847. — [arXiv:2508.18847](https://arxiv.org/abs/2508.18847) [find](https://scholar.google.com/scholar?q=Yibo+Li%2C+Miao+Xiong%2C+Jiaying+Wu%2C+and+Bryan+Hooi%2E+2025%2E+Conftuner%3A+Training+large+language+models+to+express+their+confidence+verbally%2E+Preprint%2C+arXiv%3A2508%2E18847%2E+%E2%80%94+%5BarXiv%3A2508%2E18847%5D%28https%3A%2F%2Farxiv%2Eor)
19. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Teaching models to express their uncertainty in words. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Stephanie+Lin%2C+Jacob+Hilton%2C+and+Owain+Evans%2E+2022%2E+Teaching+models+to+express+their+uncertainty+in+words%2E+Transactions+on+Machine+Learning+Research%2E)
20. Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2024. Generating with confidence: Uncertainty quantification for black-box large language models. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Zhen+Lin%2C+Shubhendu+Trivedi%2C+and+Jimeng+Sun%2E+2024%2E+Generating+with+confidence%3A+Uncertainty+quantification+for+black%2Dbox+large+language+models%2E+Transactions+on+Machine+Learning+Research%2E)
21. Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2021. [Logiqa: a challenge dataset for machine reading comprehension with logical reasoning](https://md.research.elamur.ai/free/en/papers/liu-2020-logiqa-challenge-dataset-machine-reading/). In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI’20.
22. Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan. 2022. PEFT: State-of-the-art parameter-efficient fine-tuning methods. [https://github.com/huggingface/peft](https://github.com/huggingface/peft). [↗](https://github.com/huggingface/peft)
23. M.L. Menéndez, J.A. Pardo, L. Pardo, and M.C. Pardo. 1997. The jensen-shannon divergence. Journal of the Franklin Institute, 334(2):307–318. [find](https://scholar.google.com/scholar?q=M%2EL%2E+Men%C3%A9ndez%2C+J%2EA%2E+Pardo%2C+L%2E+Pardo%2C+and+M%2EC%2E+Pardo%2E+1997%2E+The+jensen%2Dshannon+divergence%2E+Journal+of+the+Franklin+Institute%2C+334%282%29%3A307%E2%80%93318%2E)
24. Aiswariya Milan Kummaya, Amudha Joseph, Kumar Rajamani, and George Ghinea. 2025. Fed-hetero: A self-evaluating federated learning framework for data heterogeneity. Applied System Innovation, 8(2). [find](https://scholar.google.com/scholar?q=Aiswariya+Milan+Kummaya%2C+Amudha+Joseph%2C+Kumar+Rajamani%2C+and+George+Ghinea%2E+2025%2E+Fed%2Dhetero%3A+A+self%2Devaluating+federated+learning+framework+for+data+heterogeneity%2E+Applied+System+Innovation%2C+8%282%29%2E)
25. OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, and 401 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/).…
26. Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. 2015. Obtaining well calibrated probabilities using bayesian binning. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1). [find](https://scholar.google.com/scholar?q=Mahdi+Pakdaman+Naeini%2C+Gregory+Cooper%2C+and+Milos+Hauskrecht%2E+2015%2E+Obtaining+well+calibrated+probabilities+using+bayesian+binning%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+29%281%29%2E)
27. Hajar Sakai and Sarah S. Lam. 2025. Large language models for healthcare text classification: A systematic review. Preprint, arXiv:2503.01159. — [arXiv:2503.01159](https://arxiv.org/abs/2503.01159) [find](https://scholar.google.com/scholar?q=Hajar+Sakai+and+Sarah+S%2E+Lam%2E+2025%2E+Large+language+models+for+healthcare+text+classification%3A+A+systematic+review%2E+Preprint%2C+arXiv%3A2503%2E01159%2E+%E2%80%94+%5BarXiv%3A2503%2E01159%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2503%2E01159%29)
28. Paul Stangel, David Bani-Harouni, Chantal Pellegrini, Ege Özsoy, Kamilia Zaripova, Matthias Keicher, and Nassir Navab. 2025. Rewarding doubt: A reinforcement learning approach to confidence calibration of large language models. CoRR, abs/2503.02623. — [arXiv:2503.02623](https://arxiv.org/abs/2503.02623) [find](https://scholar.google.com/scholar?q=Paul+Stangel%2C+David+Bani%2DHarouni%2C+Chantal+Pellegrini%2C+Ege+%C3%96zsoy%2C+Kamilia+Zaripova%2C+Matthias+Keicher%2C+and+Nassir+Navab%2E+2025%2E+Rewarding+doubt%3A+A+reinforcement+learning+approach+to+confidence+calibratio)
29. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 3319–3328. JMLR.org. [find](https://scholar.google.com/scholar?q=Mukund+Sundararajan%2C+Ankur+Taly%2C+and+Qiqi+Yan%2E+2017%2E+Axiomatic+attribution+for+deep+networks%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning+%2D+Volume+70%2C+ICML%E2%80%9917%2C+page+3319%E2%80%9333)
30. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, and 179… [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+Cassidy+Hardin%2C+Surya+Bhupatiraju%2C+L%C3%A9onard+Hussenot%2C+Thomas+Mesnard%2C+Bobak+Shahriari%2C+Alexandre+Ram%C3%A9%2C+Johan+Ferret%2C+Peter+Liu%2C+Pouya+Ta)
31. Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher Manning. 2023. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. In Proceedings of the 2023 Conferen… [find](https://scholar.google.com/scholar?q=Katherine+Tian%2C+Eric+Mitchell%2C+Allan+Zhou%2C+Archit+Sharma%2C+Rafael+Rafailov%2C+Huaxiu+Yao%2C+Chelsea+Finn%2C+and+Christopher+Manning%2E+2023%2E+Just+ask+for+calibration%3A+Strategies+for+eliciting+calibrated+confid)
32. Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. 2024. Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Miao+Xiong%2C+Zhiyuan+Hu%2C+Xinyang+Lu%2C+YIFEI+LI%2C+Jie+Fu%2C+Junxian+He%2C+and+Bryan+Hooi%2E+2024%2E+Can+LLMs+express+their+uncertainty%3F+an+empirical+evaluation+of+confidence+elicitation+in+LLMs%2E+In+The+Twelfth+In)
33. Hadi Mohammadi, Ayoub Bagheri, Anastasia Giachanou, and Daniel L. Oberski. 2025. Explainability in practice: A survey of explainable nlp across various domains. Preprint, arXiv:2502.00837. — [arXiv:2502.00837](https://arxiv.org/abs/2502.00837) [find](https://scholar.google.com/scholar?q=Hadi+Mohammadi%2C+Ayoub+Bagheri%2C+Anastasia+Giachanou%2C+and+Daniel+L%2E+Oberski%2E+2025%2E+Explainability+in+practice%3A+A+survey+of+explainable+nlp+across+various+domains%2E+Preprint%2C+arXiv%3A2502%2E00837%2E+%E2%80%94+%5BarXiv%3A25)
34. Chenjun Xu, Bingbing Wen, Bin Han, Robert Wolfe, Lucy Lu Wang, and Bill Howe. 2025. Do language models mirror human confidence? exploring psychological insights to address overconfidence in LLMs. In Findings of the Association for Computational Linguistics: ACL 2025, pages 25655–25672, Vienna, Austr… [find](https://scholar.google.com/scholar?q=Chenjun+Xu%2C+Bingbing+Wen%2C+Bin+Han%2C+Robert+Wolfe%2C+Lucy+Lu+Wang%2C+and+Bill+Howe%2E+2025%2E+Do+language+models+mirror+human+confidence%3F+exploring+psychological+insights+to+address+overconfidence+in+LLMs%2E+In+F)
35. Joaquin Navajas, Bahador Bahrami, and Peter E Latham. 2016. Post-decisional accounts of biases in confidence. Current Opinion in Behavioral Sciences, 11:55–60. Computational modeling. [find](https://scholar.google.com/scholar?q=Joaquin+Navajas%2C+Bahador+Bahrami%2C+and+Peter+E+Latham%2E+2016%2E+Post%2Ddecisional+accounts+of+biases+in+confidence%2E+Current+Opinion+in+Behavioral+Sciences%2C+11%3A55%E2%80%9360%2E+Computational+modeling%2E)
36. Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2024. [Hallucination is inevitable: An innate limitation of large language models](https://md.research.elamur.ai/free/en/papers/xu-2024-hallucination-inevitable-innate-limitation-large/). arXiv preprint arXiv:2401.11817.
37. Daniel Yang, Yao-Hung Hubert Tsai, and Makoto Yamada. 2025. On verbalized confidence scores for LLMs. In ICLR Workshop: Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI. [find](https://scholar.google.com/scholar?q=Daniel+Yang%2C+Yao%2DHung+Hubert+Tsai%2C+and+Makoto+Yamada%2E+2025%2E+On+verbalized+confidence+scores+for+LLMs%2E+In+ICLR+Workshop%3A+Quantify+Uncertainty+and+Hallucination+in+Foundation+Models%3A+The+Next+Frontier+i)
38. Xinran Zhao, Hongming Zhang, Xiaoman Pan, Wenlin Yao, Dong Yu, Tongshuang Wu, and Jianshu Chen. 2024. Fact-and-reflection (FaR) improves confidence calibration of large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 8702–8718, Bangkok, Thailand. Associ… [find](https://scholar.google.com/scholar?q=Xinran+Zhao%2C+Hongming+Zhang%2C+Xiaoman+Pan%2C+Wenlin+Yao%2C+Dong+Yu%2C+Tongshuang+Wu%2C+and+Jianshu+Chen%2E+2024%2E+Fact%2Dand%2Dreflection+%28FaR%29+improves+confidence+calibration+of+large+language+models%2E+In+Findings+of)
39. Ziang Zhou, Tianyuan Jin, Jieming Shi, and Qing Li. 2025. Steerconf: Steering llms for confidence elicitation. Preprint, arXiv:2503.02863. — [arXiv:2503.02863](https://arxiv.org/abs/2503.02863) [find](https://scholar.google.com/scholar?q=Ziang+Zhou%2C+Tianyuan+Jin%2C+Jieming+Shi%2C+and+Qing+Li%2E+2025%2E+Steerconf%3A+Steering+llms+for+confidence+elicitation%2E+Preprint%2C+arXiv%3A2503%2E02863%2E+%E2%80%94+%5BarXiv%3A2503%2E02863%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2503%2E02863%29)

*Updated 31 October 2026*