---
title: "ART — paper by Luo et al. (2026) — summary and ideas"
description: "This paper links hallucination in LLMs to the way attention is distributed in the earliest layers. 42 references."
authors: ["Ziqin Luo", "Yihao Quan", "Xiaofeng Zhang", "Xiaosong Yuan", "Chen Shen"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1571/"
doi: "10.18653/v1/2026.acl-long.1571"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/luo-2026-art-attention-replacement-technique-improve/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/luo-2026-art-attention-replacement-technique-improve/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/10/) / ART: Attention Replacement Technique to Improve Factuality in LLMs

# ART: Attention Replacement Technique to Improve Factuality in LLMs

**Authors:** Ziqin Luo, [Yihao Quan](https://md.research.elamur.ai/free/en/authors/yihao-quan/), [Xiaofeng Zhang](https://md.research.elamur.ai/free/en/authors/xiaofeng-zhang/), [Xiaosong Yuan](https://md.research.elamur.ai/free/en/authors/xiaosong-yuan/) and [Chen Shen](https://md.research.elamur.ai/free/en/authors/chen-shen/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1571/) · [PDF](https://aclanthology.org/2026.acl-long.1571.pdf) · [DOI 10.18653/v1/2026.acl-long.1571](https://doi.org/10.18653/v1/2026.acl-long.1571) · [OpenAlex](https://openalex.org/W7166804848)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
This paper links hallucination in LLMs to the way attention is distributed in the earliest layers. Inspecting attention maps of shallow layers (e.g., Layer 0 of Qwen2.5-7B-Instruct on a GSM8K sample), the authors group heads into three types. Uniform heads spread weight evenly over the context, local heads look mostly at neighbouring tokens, and scattered heads concentrate on a few earlier tokens. They argue that uniform heads are largely redundant and can pull the model away from relevant context.

Their training-free fix, ART (Attention Replacement Technique), works at inference time. In shallow layers it finds the k most uniform heads and swaps their attention for patterns derived from local heads. It needs no fine-tuning, extra data, or task-specific probing.

Tests on several open LLMs show consistent accuracy gains. ART matches or outperforms intervention methods such as ITI, ACT and DoLa, and it combines well with beam search. This makes it a cheap plug-in option for improving factuality.

## Key points
- Models evaluated include Qwen2.5-7B-Instruct, Llama3.1-8B-Instruct, Llama2-7B-Chat and Llama3-8B-Instruct. ART-mean is the default variant, and ART-max is also described.
- The default number of replaced heads is k = ⌊0.1·N_h⌋. Ablations on 200 sampled training examples per dataset show accuracy rising and then falling as k grows, peaking at this value.
- The reverse swap, ART-inverse (local replaced by uniform), breaks Qwen2.5-7B-Instruct, dropping scores from about 51/51/93 to 0.0/2.0/7.5. This underlines how important local attention is.
- With ART-scattered, scattered attention serves as the replacement target. It still helps but less than standard ART, suggesting the choice of target is flexible.
- Combining ART with beam search (ART-beam) gives the best scores on three benchmarks:
  - Llama2-7B-Chat improves from 19.0/28.4/27.2 to 22.7/30.5/30.1.
  - Llama3-8B-Instruct improves from 45.1/38.5/83.1 to 46.4/42.7/85.8.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/) — University, Shanghai, China · 202 works · 22,509 citations
- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 201 works · 17,173 citations
- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 212 works · 19,544 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Ziqin Luo, Yihao Quan, Xiaofeng Zhang, Xiaosong Yuan and Chen Shen. 2026. ART: Attention Replacement Technique to Improve Factuality in LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{luo2026art,
  title = {{ART: Attention Replacement Technique to Improve Factuality in LLMs}},
  author = {Ziqin Luo, Yihao Quan, Xiaofeng Zhang, Xiaosong Yuan and Chen Shen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1571},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1571}
}
```

- [← previous ART: Adaptive Reasoning Trees for Explainable Claim Verification](https://md.research.elamur.ai/free/en/papers/wadhwa-2026-art-adaptive-reasoning-trees-explainable/)
- [next → arXiv2Table: Toward Realistic Benchmarking and Evaluation for LLM-Based Literature-Review Table Generation](https://md.research.elamur.ai/free/en/papers/wang-2026-arxiv2table-realistic-benchmarking-evaluation-llm/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/10/)

## References

42 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. AI@Meta. 2024. Llama 3 model card. [find](https://scholar.google.com/scholar?q=AI%40Meta%2E+2024%2E+Llama+3+model+card%2E)
2. Liwei Che, Zhiyu Xue, Yihao Quan, Benlin Liu, Zeru Shi, Michelle Hurst, Jacob Feldman, Ruixiang Tang, Ranjay Krishna, and Vladimir Pavlovic. 2026. Counting circuits: Mechanistic interpretability of visual reasoning in large vision-language models. arXiv preprint arXiv:2603.18523. [find](https://scholar.google.com/scholar?q=Liwei+Che%2C+Zhiyu+Xue%2C+Yihao+Quan%2C+Benlin+Liu%2C+Zeru+Shi%2C+Michelle+Hurst%2C+Jacob+Feldman%2C+Ruixiang+Tang%2C+Ranjay+Krishna%2C+and+Vladimir+Pavlovic%2E+2026%2E+Counting+circuits%3A+Mechanistic+interpretability+of+vi)
3. Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. 2024a. [An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models](https://md.research.elamur.ai/free/en/papers/chen-2024-image-worth-1-2-tokens/). In Proceedings of the 18th European Conference on Computer Vision (ECCV), volume 15139,…
4. Tianxiang Chen, Zhentao Tan, Tao Gong, Yue Wu, Qi Chu, Bin Liu, Jieping Ye, and Nenghai Yu. 2024b. Llama slayer 8b: Shallow layers hold the key to knowledge injection. In Proceedings of the 29th Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5991–6002. [find](https://scholar.google.com/scholar?q=Tianxiang+Chen%2C+Zhentao+Tan%2C+Tao+Gong%2C+Yue+Wu%2C+Qi+Chu%2C+Bin+Liu%2C+Jieping+Ye%2C+and+Nenghai+Yu%2E+2024b%2E+Llama+slayer+8b%3A+Shallow+layers+hold+the+key+to+knowledge+injection%2E+In+Proceedings+of+the+29th+Confe)
5. Tianxiang Chen, Zhentao Tan, Tao Gong, Yue Wu, Qi Chu, Bin Liu, Jieping Ye, and Nenghai Yu. 2024c. Llama slayer 8b: Shallow layers hold the key to knowledge injection. arXiv preprint arXiv:2410.02330. [find](https://scholar.google.com/scholar?q=Tianxiang+Chen%2C+Zhentao+Tan%2C+Tao+Gong%2C+Yue+Wu%2C+Qi+Chu%2C+Bin+Liu%2C+Jieping+Ye%2C+and+Nenghai+Yu%2E+2024c%2E+Llama+slayer+8b%3A+Shallow+layers+hold+the+key+to+knowledge+injection%2E+arXiv+preprint+arXiv%3A2410%2E02330%2E)
6. Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. 2024. [Dola: Decoding by contrasting layers improves factuality in large language models](https://md.research.elamur.ai/free/en/papers/chuang-2023-dola-decoding-contrasting-layers-improves/). In Proceedings of the 12th International Conference on Learning Representations (ICLR).
7. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
8. Siqi Fan, Xin Jiang, Xiang Li, Xuying Meng, Peng Han, Shuo Shang, Aixin Sun, Yequan Wang, and Zhongyuan Wang. 2024. Not all layers of llms are necessary during inference. arXiv preprint arXiv:2403.02181. — [arXiv:2403.02181](https://arxiv.org/abs/2403.02181) [find](https://scholar.google.com/scholar?q=Siqi+Fan%2C+Xin+Jiang%2C+Xiang+Li%2C+Xuying+Meng%2C+Peng+Han%2C+Shuo+Shang%2C+Aixin+Sun%2C+Yequan+Wang%2C+and+Zhongyuan+Wang%2E+2024%2E+Not+all+layers+of+llms+are+necessary+during+inference%2E+arXiv+preprint+arXiv%3A2403%2E021)
9. Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. 2024. [Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation](https://md.research.elamur.ai/free/en/papers/huang-2023-opera-alleviating-hallucination-multi-modal/). In Proceedings of the 41st IEEE/CVF Conference on Compute…
10. Kenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister, and Martin Wattenberg. 2023. [Inference-time intervention: Eliciting truthful answers from a language model](https://md.research.elamur.ai/free/en/papers/li-2023-inference-time-intervention-eliciting-truthful/). In Advances in Neural Information Processing Systems 36 (NIPS).
11. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), pages 3214–3252.
12. Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2020. [Logiqa: A challenge dataset for machine reading comprehension with logical reasoning](https://md.research.elamur.ai/free/en/papers/liu-2020-logiqa-challenge-dataset-machine-reading/). In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI), pages 3622–3628.
13. Paul Michel, Omer Levy, and Graham Neubig. 2019. [Are sixteen heads really better than one?](https://md.research.elamur.ai/free/en/papers/michel-2019-sixteen-heads-really-better-than/) In Advances in Neural Information Processing Systems 32 (NIPS).
14. Yihao Quan, Zeru Shi, Jinman Zhao, and Ruixiang Tang. 2026. Reinforcing consistency in video mllms with structured rewards. arXiv preprint arXiv:2604.01460. — [arXiv:2604.01460](https://arxiv.org/abs/2604.01460) [find](https://scholar.google.com/scholar?q=Yihao+Quan%2C+Zeru+Shi%2C+Jinman+Zhao%2C+and+Ruixiang+Tang%2E+2026%2E+Reinforcing+consistency+in+video+mllms+with+structured+rewards%2E+arXiv+preprint+arXiv%3A2604%2E01460%2E+%E2%80%94+%5BarXiv%3A2604%2E01460%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
15. Zeru Shi, Kai Mei, Yihao Quan, Dimitris N Metaxas, and Ruixiang Tang. 2026. Improving visual reasoning with iterative evidence refinement. arXiv preprint arXiv:2603.14117. — [arXiv:2603.14117](https://arxiv.org/abs/2603.14117) [find](https://scholar.google.com/scholar?q=Zeru+Shi%2C+Kai+Mei%2C+Yihao+Quan%2C+Dimitris+N+Metaxas%2C+and+Ruixiang+Tang%2E+2026%2E+Improving+visual+reasoning+with+iterative+evidence+refinement%2E+arXiv+preprint+arXiv%3A2603%2E14117%2E+%E2%80%94+%5BarXiv%3A2603%2E14117%5D%28https%3A%2F)
16. Zeru Shi, Yingjia Wan, Zhenting Wang, Qifan Wang, Fan Yang, Elisa Kreiss, and Ruixiang Tang. 2025. Meaningless tokens, meaningful gains: How activation shifts enhance llm reasoning. arXiv preprint arXiv:2510.01032. [find](https://scholar.google.com/scholar?q=Zeru+Shi%2C+Yingjia+Wan%2C+Zhenting+Wang%2C+Qifan+Wang%2C+Fan+Yang%2C+Elisa+Kreiss%2C+and+Ruixiang+Tang%2E+2025%2E+Meaningless+tokens%2C+meaningful+gains%3A+How+activation+shifts+enhance+llm+reasoning%2E+arXiv+preprint+arX)
17. Qi Sun, Marc Pickett, Aakash Kumar Nain, and Llion Jones. 2024. Transformer layers as painters. arXiv preprint arXiv:2407.09298. [find](https://scholar.google.com/scholar?q=Qi+Sun%2C+Marc+Pickett%2C+Aakash+Kumar+Nain%2C+and+Llion+Jones%2E+2024%2E+Transformer+layers+as+painters%2E+arXiv+preprint+arXiv%3A2407%2E09298%2E)
18. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. [Commonsenseqa: A question answering challenge targeting commonsense knowledge](https://md.research.elamur.ai/free/en/papers/talmor-2019-commonsenseqa-question-answering-challenge-targeting/). In Proceedings of the 17th Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 4149–4158.
19. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, et al. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
20. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems 30 (NIPS), pages 5998–6008.
21. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems 35 (NIPS).
22. Jinfeng Wei and Xiaofeng Zhang. 2024a. Dopra: Decoding over-accumulation penalization and reallocation in specific weighting layer. Proceedings of the 32nd ACM International Conference on Multimedia. [find](https://scholar.google.com/scholar?q=Jinfeng+Wei+and+Xiaofeng+Zhang%2E+2024a%2E+Dopra%3A+Decoding+over%2Daccumulation+penalization+and+reallocation+in+specific+weighting+layer%2E+Proceedings+of+the+32nd+ACM+International+Conference+on+Multimedia%2E)
23. Jinfeng Wei and Xiaofeng Zhang. 2024b. Dopra: Decoding over-accumulation penalization and reallocation in specific weighting layer. arXiv preprint arXiv:2407.15130. — [arXiv:2407.15130](https://arxiv.org/abs/2407.15130) [find](https://scholar.google.com/scholar?q=Jinfeng+Wei+and+Xiaofeng+Zhang%2E+2024b%2E+Dopra%3A+Decoding+over%2Daccumulation+penalization+and+reallocation+in+specific+weighting+layer%2E+arXiv+preprint+arXiv%3A2407%2E15130%2E+%E2%80%94+%5BarXiv%3A2407%2E15130%5D%28https%3A%2F%2Farxiv%2E)
24. Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. 2024. [Retrieval head mechanistically explains long-context factuality](https://md.research.elamur.ai/free/en/papers/wu-2024-retrieval-head-mechanistically-explains-long/). arXiv preprint arXiv:2404.15574.
25. Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, Junxian Guo, Shang Yang, Haotian Tang, Yao Fu, and Song Han. 2024a. [Duoattention: Efficient long-context llm inference with retrieval and streaming heads](https://md.research.elamur.ai/free/en/papers/xiao-2024-duoattention-efficient-long-context-llm/). arXiv preprint arXiv:2410.10819.
26. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024b. Efficient streaming language models with attention sinks. In Proceedings of the 12th International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Guangxuan+Xiao%2C+Yuandong+Tian%2C+Beidi+Chen%2C+Song+Han%2C+and+Mike+Lewis%2E+2024b%2E+Efficient+streaming+language+models+with+attention+sinks%2E+In+Proceedings+of+the+12th+International+Conference+on+Learning+Re)
27. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+et+al%2E+2024%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)
28. Fangcong Yin, Xi Ye, and Greg Durrett. 2024. Lofit: Localized fine-tuning on llm representations. arXiv preprint arXiv:2406.01563. — [arXiv:2406.01563](https://arxiv.org/abs/2406.01563) [find](https://scholar.google.com/scholar?q=Fangcong+Yin%2C+Xi+Ye%2C+and+Greg+Durrett%2E+2024%2E+Lofit%3A+Localized+fine%2Dtuning+on+llm+representations%2E+arXiv+preprint+arXiv%3A2406%2E01563%2E+%E2%80%94+%5BarXiv%3A2406%2E01563%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2406%2E01563%29)
29. Zhongzhi Yu, Zheng Wang, Yonggan Fu, Huihong Shi, Khalid Shaikh, and Yingyan Celine Lin. 2024. Unveiling and harnessing hidden attention sinks: Enhancing large language models without training through attention calibration. In Proceedings of the 41st International Conference on Machine Learning (ICM… [find](https://scholar.google.com/scholar?q=Zhongzhi+Yu%2C+Zheng+Wang%2C+Yonggan+Fu%2C+Huihong+Shi%2C+Khalid+Shaikh%2C+and+Yingyan+Celine+Lin%2E+2024%2E+Unveiling+and+harnessing+hidden+attention+sinks%3A+Enhancing+large+language+models+without+training+through)
30. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 3512–3534. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+3512%E2%80%933534%2E)
31. Xiaofeng Zhang, Yihao Quan, Chen Shen, Xiaosong Yuan, Shaotian Yan, Liang Xie, Wenxiao Wang, Chaochen Gu, Hao Tang, and Jieping Ye. 2025c. [From redundancy to relevance: Information flow in lvlms across reasoning tasks](https://md.research.elamur.ai/free/en/papers/zhang-2025-redundancy-relevance-information-flow-lvlms/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of t…
32. Xiaofeng Zhang, Fanshuo Zeng, Yihao Quan, Zheng Hui, and Jiawei Yao. 2025d. Enhancing multimodal large language models complex reason via similarity computation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 10203– 10211. [find](https://scholar.google.com/scholar?q=Xiaofeng+Zhang%2C+Fanshuo+Zeng%2C+Yihao+Quan%2C+Zheng+Hui%2C+and+Jiawei+Yao%2E+2025d%2E+Enhancing+multimodal+large+language+models+complex+reason+via+similarity+computation%2E+In+Proceedings+of+the+AAAI+Conference+)
33. Xiaofeng Zhang, Yuanchao Zhu, Chaochen Gu, Jiawei Cao, Hao Cheng, and Kaijie Wu. 2026a. What drives attention sinks? a study of massive activations and rotational positional encoding in large vision–language models. Information Processing & Management, 63(2):104431. [find](https://scholar.google.com/scholar?q=Xiaofeng+Zhang%2C+Yuanchao+Zhu%2C+Chaochen+Gu%2C+Jiawei+Cao%2C+Hao+Cheng%2C+and+Kaijie+Wu%2E+2026a%2E+What+drives+attention+sinks%3F+a+study+of+massive+activations+and+rotational+positional+encoding+in+large+vision%E2%80%93l)
34. Xiaofeng Zhang, Yuanchao Zhu, Chaochen Gu, Xiaosong Yuan, Qiyan Zhao, Jiawei Cao, Feilong Tang, Sinan Fan, Yaomin Shen, Chen Shen, et al. 2026b. Hallucination begins where saliency drops. In The Fourteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Xiaofeng+Zhang%2C+Yuanchao+Zhu%2C+Chaochen+Gu%2C+Xiaosong+Yuan%2C+Qiyan+Zhao%2C+Jiawei+Cao%2C+Feilong+Tang%2C+Sinan+Fan%2C+Yaomin+Shen%2C+Chen+Shen%2C+et+al%2E+2026b%2E+Hallucination+begins+where+saliency+drops%2E+In+The+Fourt)
35. Xiaosong Yuan, Chen Shen, Shaotian Yan, Xiaofeng Zhang, Liang Xie, Wenxiao Wang, Renchu Guan, Ying Wang, and Jieping Ye. 2024. Instance-adaptive zero-shot chain-of-thought prompting. arXiv preprint arXiv:2409.20441. — [arXiv:2409.20441](https://arxiv.org/abs/2409.20441) [find](https://scholar.google.com/scholar?q=Xiaosong+Yuan%2C+Chen+Shen%2C+Shaotian+Yan%2C+Xiaofeng+Zhang%2C+Liang+Xie%2C+Wenxiao+Wang%2C+Renchu+Guan%2C+Ying+Wang%2C+and+Jieping+Ye%2E+2024%2E+Instance%2Dadaptive+zero%2Dshot+chain%2Dof%2Dthought+prompting%2E+arXiv+preprint+ar)
36. Qiyan Zhao, Xiaofeng Zhang, Shuochen Chang, Qianyu Chen, Xiaosong Yuan, Xuhang Chen, Luoqi Liu, Jiajun Zhang, Xu-Yao Zhang, and Da-Han Wang. 2026. Context tokens are anchors: Understanding the repetition curse in dmllms from an information flow perspective. In The Fourteenth International Conference… [find](https://scholar.google.com/scholar?q=Qiyan+Zhao%2C+Xiaofeng+Zhang%2C+Shuochen+Chang%2C+Qianyu+Chen%2C+Xiaosong+Yuan%2C+Xuhang+Chen%2C+Luoqi+Liu%2C+Jiajun+Zhang%2C+Xu%2DYao+Zhang%2C+and+Da%2DHan+Wang%2E+2026%2E+Context+tokens+are+anchors%3A+Understanding+the+repetit)
37. Boxuan Zhang, Yi Yu, Jiaxuan Guo, and Jing Shao. 2025a. Dive into the agent matrix: A realistic evaluation of self-replication risk in llm agents. arXiv preprint arXiv:2509.25302. — [arXiv:2509.25302](https://arxiv.org/abs/2509.25302) [find](https://scholar.google.com/scholar?q=Boxuan+Zhang%2C+Yi+Yu%2C+Jiaxuan+Guo%2C+and+Jing+Shao%2E+2025a%2E+Dive+into+the+agent+matrix%3A+A+realistic+evaluation+of+self%2Dreplication+risk+in+llm+agents%2E+arXiv+preprint+arXiv%3A2509%2E25302%2E+%E2%80%94+%5BarXiv%3A2509%2E25302%5D)
38. Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Mingchuan Yang, Bo Tang, Feiyu Xiong, and Zhiyu Li. 2024. Attention heads of [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). arXiv preprint arXiv:2409.03752. — [arXiv:2409.03752](https://arxiv.org/abs/2409.03752)
39. Boxuan Zhang and Ruqi Zhang. 2025. Cot-uq: Improving response-wise uncertainty quantification in llms with chain-of-thought. In Findings of the Association for Computational Linguistics: ACL 2025, pages 26114–26133. [find](https://scholar.google.com/scholar?q=Boxuan+Zhang+and+Ruqi+Zhang%2E+2025%2E+Cot%2Duq%3A+Improving+response%2Dwise+uncertainty+quantification+in+llms+with+chain%2Dof%2Dthought%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2025%2C+pag)
40. Guanyu Zhou, Yibo Yan, Xin Zou, Kun Wang, Aiwei Liu, and Xuming Hu. 2025. Mitigating modality prior-induced hallucinations in multimodal large language models via deciphering attention causality. In Proceedings of the 13th International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Guanyu+Zhou%2C+Yibo+Yan%2C+Xin+Zou%2C+Kun+Wang%2C+Aiwei+Liu%2C+and+Xuming+Hu%2E+2025%2E+Mitigating+modality+prior%2Dinduced+hallucinations+in+multimodal+large+language+models+via+deciphering+attention+causality%2E+In+P)
41. Xiaofeng Zhang, Yihao Quan, Chaochen Gu, Chen Shen, Xiaosong Yuan, Shaotian Yan, Hao Cheng, Kaijie Wu, and Jieping Ye. 2024. Seeing clearly by layer two: Enhancing attention heads to alleviate hallucination in lvlms. arXiv preprint arXiv:2411.09968. — [arXiv:2411.09968](https://arxiv.org/abs/2411.09968) [find](https://scholar.google.com/scholar?q=Xiaofeng+Zhang%2C+Yihao+Quan%2C+Chaochen+Gu%2C+Chen+Shen%2C+Xiaosong+Yuan%2C+Shaotian+Yan%2C+Hao+Cheng%2C+Kaijie+Wu%2C+and+Jieping+Ye%2E+2024%2E+Seeing+clearly+by+layer+two%3A+Enhancing+attention+heads+to+alleviate+halluci)
42. Xiaofeng Zhang, Yihao Quan, Chen Shen, Chaochen Gu, Xiaosong Yuan, Shaotian Yan, Jiawei Cao, Hao Cheng, Kaijie Wu, and Jieping Ye. 2025b. [Shallow focus, deep fixes: Enhancing shallow layers vision attention sinks to alleviate hallucination in lvlms](https://md.research.elamur.ai/free/en/papers/zhang-2025-shallow-focus-deep-fixes-enhancing/).

*Updated 2 November 2026*