---
title: "Anchoring Depends on Confidence and Post-Training in"
description: "Large language models can be swayed by irrelevant numbers placed before a quantitative question. 13 references."
authors: ["Hillary N. Owusu", "Naomi H. Feldman"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)"
source: "https://aclanthology.org/2026.acl-short.16/"
doi: "10.18653/v1/2026.acl-short.16"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/owusu-2026-anchoring-depends-confidence-post-training/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/owusu-2026-anchoring-depends-confidence-post-training/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/22/) / Anchoring Depends on Confidence and Post-Training in Language Models

# Anchoring Depends on Confidence and Post-Training in Language Models

**Authors:** Hillary N. Owusu and Naomi H. Feldman · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-short.16/) · [PDF](https://aclanthology.org/2026.acl-short.16.pdf) · [DOI 10.18653/v1/2026.acl-short.16](https://doi.org/10.18653/v1/2026.acl-short.16) · [OpenAlex](https://openalex.org/W7166835177)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Large language models can be swayed by irrelevant numbers placed before a quantitative question. This is the anchoring effect known from human psychology. The authors ask whether a model's confidence or its factual correctness predicts how strongly anchors pull it. They test six models: base, instruction-tuned and DeepSeek-R1-distilled versions of Llama-3.1-8B and Qwen-7B. The test set is 50 questions from the Open Anchoring Dataset (OpAQ), each preceded by an unrelated "slot machine" number that is either low or high. Instead of sampling, they score nine fixed candidate answers with length-normalized log-likelihoods. They then use Total Variation Distance to measure how far the anchored distribution moves from the control one. Regression analyses (pooled OLS, mixed-effects and per-model) show that sharper baseline distributions resist anchoring. Distance from the true answer adds no predictive value, so a confidently wrong model is as robust as a correct one. Post-training strengthens this link, and high anchors move models more than low ones. This gives practitioners a signal for flagging vulnerable numerical outputs that does not require ground truth.

## Key points
- **Models:** Llama-3.1-8B (Base, Instruct, DeepSeek-R1-Distill), Qwen2.5-7B (Base, Instruct) and DeepSeek-R1-Distill-Qwen-7B, which is built on Qwen2.5-Math-7B.
- **Data:** 600 observations from 50 OpAQ items × 2 anchor conditions × 6 models. The findings also hold on 99 items (1,188 observations), with main effects at p \< .001.
- **Certainty vs. accuracy:** Certainty is the maximum candidate probability in the control condition. It lowers TVD in pooled OLS (β = −0.037, p = .001) and in the mixed-effects model (β = −0.052, p \< .001). Accuracy, measured as log-transformed relative MAE, is not significant (p = .48 and p = .74).
- **Effect of post-training:** In pooled data, base models show a weak certainty slope (β = −0.012, n.s.). Instruct (interaction β = −0.072) and Distill (interaction β = −0.054) slopes are significantly steeper. Per-model fits show that base models do have an internal link, but their confidence scales are not aligned with each other.
- **Asymmetry:** High anchors have a stronger effect than low ones, peaking at medium certainty. Instruction-tuned models keep 78–86% directional consistency for high anchors but only 50–64% for low anchors, compared with ≥92% for base models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Maryland, College Park](https://md.research.elamur.ai/free/en/universities/university-of-maryland-college-park/) — University, College Park, United States · 141 works · 29,174 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 249 works · [🎯 General Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/general-decision-sciences/) · 4 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Hillary N. Owusu and Naomi H. Feldman. 2026. Anchoring Depends on Confidence and Post-Training in Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{owusu2026anchoring,
  title = {{Anchoring Depends on Confidence and Post-Training in Language Models}},
  author = {Hillary N. Owusu and Naomi H. Feldman},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-short.16},
  url = {https://doi.org/10.18653/v1/2026.acl-short.16}
}
```

- [← previous Anchored Sliding Window: Toward Robust and Imperceptible Linguistic Steganography](https://md.research.elamur.ai/free/en/papers/yan-2026-anchored-sliding-window-robust-imperceptible/)
- [next → AnchorMem: Anchored Facts with Associative Contexts for Building Memory in Large Language Models](https://md.research.elamur.ai/free/en/papers/shen-2026-anchormem-anchored-facts-associative-contexts/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/22/)

## References

13 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dale J. Barr, R. Levy, Christoph Scheepers, and Harry J. Tily. 2013. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of memory and language, 68 3. [find](https://scholar.google.com/scholar?q=Dale+J%2E+Barr%2C+R%2E+Levy%2C+Christoph+Scheepers%2C+and+Harry+J%2E+Tily%2E+2013%2E+Random+effects+structure+for+confirmatory+hypothesis+testing%3A+Keep+it+maximal%2E+Journal+of+memory+and+language%2C+68+3%2E)
2. Jessica Echterhoff, Yao Liu, Abeer Alessa, Julian McAuley, and Zexue He. 2024. Cognitive bias in decision-making with llms. Preprint, arXiv:2403.00811. [find](https://scholar.google.com/scholar?q=Jessica+Echterhoff%2C+Yao+Liu%2C+Abeer+Alessa%2C+Julian+McAuley%2C+and+Zexue+He%2E+2024%2E+Cognitive+bias+in+decision%2Dmaking+with+llms%2E+Preprint%2C+arXiv%3A2403%2E00811%2E)
3. Thilo Hagendorff, Sarah Fabi, and Michal Kosinski. 2023. Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in chatgpt. Nature Computational Science, 3(10):833–838. [find](https://scholar.google.com/scholar?q=Thilo+Hagendorff%2C+Sarah+Fabi%2C+and+Michal+Kosinski%2E+2023%2E+Human%2Dlike+intuitive+behavior+and+reasoning+biases+emerged+in+large+language+models+but+disappeared+in+chatgpt%2E+Nature+Computational+Science%2C+3)
4. Yiming Huang, Biquan Bie, Zuqiu Na, Weilin Ruan, Songxin Lei, Yutao Yue, and Xinlei He. 2025. An empirical study of the anchoring effect in llms: Existence, mechanism, and potential mitigations. Preprint, arXiv:2505.15392. — [arXiv:2505.15392](https://arxiv.org/abs/2505.15392) [find](https://scholar.google.com/scholar?q=Yiming+Huang%2C+Biquan+Bie%2C+Zuqiu+Na%2C+Weilin+Ruan%2C+Songxin+Lei%2C+Yutao+Yue%2C+and+Xinlei+He%2E+2025%2E+An+empirical+study+of+the+anchoring+effect+in+llms%3A+Existence%2C+mechanism%2C+and+potential+mitigations%2E+Prepr)
5. Karen E. Jacowitz and Daniel Kahneman. 1995. Measures of anchoring in estimation tasks. Personality and Social Psychology Bulletin, 21(11):1161–1166. [find](https://scholar.google.com/scholar?q=Karen+E%2E+Jacowitz+and+Daniel+Kahneman%2E+1995%2E+Measures+of+anchoring+in+estimation+tasks%2E+Personality+and+Social+Psychology+Bulletin%2C+21%2811%29%3A1161%E2%80%931166%2E)
6. Each candidate string y i corresponds to a unique numeric value v i ∈ R. To measure the extent to which the distributional shift moves probability mass toward the anchor value, we compute the percentage change in the model’s estimate in each anchoring condition relative to the control baseline: Falk… [find](https://scholar.google.com/scholar?q=Each+candidate+string+y+i+corresponds+to+a+unique+numeric+value+v+i+%E2%88%88+R%2E+To+measure+the+extent+to+which+the+distributional+shift+moves+probability+mass+toward+the+anchor+value%2C+we+compute+the+percenta)
7. E[v \| x ctl ] − E[v \| x low ] E[v \| x ctl ] (4) ∆ high = E[v \| x high ] − E[v \| x ctl ] E[v \| x ctl ] (5) where E[v \| x] is the expected value over the candidate set: Jiaxu Lou and Yifan Sun. 2024. Anchoring bias in large language models: An experimental study. Preprint, arXiv:2412.06593. — [arXiv:2412.06593](https://arxiv.org/abs/2412.06593) [find](https://scholar.google.com/scholar?q=E%5Bv+%5C%7C+x+ctl+%5D+%E2%88%92+E%5Bv+%5C%7C+x+low+%5D+E%5Bv+%5C%7C+x+ctl+%5D+%284%29+%E2%88%86+high+%3D+E%5Bv+%5C%7C+x+high+%5D+%E2%88%92+E%5Bv+%5C%7C+x+ctl+%5D+E%5Bv+%5C%7C+x+ctl+%5D+%285%29+where+E%5Bv+%5C%7C+x%5D+is+the+expected+value+over+the+candidate+set%3A+Jiaxu+Lou+and+Yifan+Sun%2E+2)
8. Jeremy K. Nguyen. 2024. Human bias in ai models? anchoring effects and mitigation strategies in large language models. Journal of Behavioral and Experimental Finance, 43(C):None. [find](https://scholar.google.com/scholar?q=Jeremy+K%2E+Nguyen%2E+2024%2E+Human+bias+in+ai+models%3F+anchoring+effects+and+mitigation+strategies+in+large+language+models%2E+Journal+of+Behavioral+and+Experimental+Finance%2C+43%28C%29%3ANone%2E)
9. Lukas Röseler, Lucia Weber, Ena P. B. Stijović, Katharina A. K. Jaekel, J. F. (Janne)ke M. T. (Janneke) G. (Gijsbers) Gijsbers, and Nir Milstein. 2022. The Open Anchoring Quest Dataset: Anchored Estimates from 96 Studies on Anchoring Effects. Journal of Open Psychology Data, 10(1):16. [find](https://scholar.google.com/scholar?q=Lukas+R%C3%B6seler%2C+Lucia+Weber%2C+Ena+P%2E+B%2E+Stijovic%CC%81%2C+Katharina+A%2E+K%2E+Jaekel%2C+J%2E+F%2E+%28Janne%29ke+M%2E+T%2E+%28Janneke%29+G%2E+%28Gijsbers%29+Gijsbers%2C+and+Nir+Milstein%2E+2022%2E+The+Open+Anchoring+Quest+Dataset%3A+Anchored+Esti)
10. Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D. Manning. 2023. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. Preprint, arXiv:2305.14975. [find](https://scholar.google.com/scholar?q=Katherine+Tian%2C+Eric+Mitchell%2C+Allan+Zhou%2C+Archit+Sharma%2C+Rafael+Rafailov%2C+Huaxiu+Yao%2C+Chelsea+Finn%2C+and+Christopher+D%2E+Manning%2E+2023%2E+Just+ask+for+calibration%3A+Strategies+for+eliciting+calibrated+con)
11. Amos Tversky and Daniel Kahneman. 1974. Judgment under uncertainty: Heuristics and biases. Science, 185(4157):1124–1131. [find](https://scholar.google.com/scholar?q=Amos+Tversky+and+Daniel+Kahneman%2E+1974%2E+Judgment+under+uncertainty%3A+Heuristics+and+biases%2E+Science%2C+185%284157%29%3A1124%E2%80%931131%2E)
12. Felipe Valencia-Clavijo. 2025. Anchors in the machine: Behavioral and attributional evidence of anchoring bias in llms. Preprint, arXiv:2511.05766. — [arXiv:2511.05766](https://arxiv.org/abs/2511.05766) [find](https://scholar.google.com/scholar?q=Felipe+Valencia%2DClavijo%2E+2025%2E+Anchors+in+the+machine%3A+Behavioral+and+attributional+evidence+of+anchoring+bias+in+llms%2E+Preprint%2C+arXiv%3A2511%2E05766%2E+%E2%80%94+%5BarXiv%3A2511%2E05766%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2511%2E0576)
13. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. [Huggin…](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)

*Updated 1 November 2026*