---
title: "From Heads to Neurons — paper by Wang et al. (2026)"
description: "As a remedy, the authors introduce a two-stage technique, called HONES (short for Head-Oriented Neuron Explanation &… 48 references."
authors: ["Qidong Wang", "Junjie Hu", "Ming Jiang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1802/"
doi: "10.18653/v1/2026.findings-acl.1802"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-heads-neurons-causal-attribution-steering/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-heads-neurons-causal-attribution-steering/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) / From Heads to Neurons: Causal Attribution and Steering in Multi-Task Vision–Language Models

# From Heads to Neurons: Causal Attribution and Steering in Multi-Task Vision–Language Models

**Authors:** Qidong Wang, [Junjie Hu](https://md.research.elamur.ai/free/en/authors/junjie-hu/) and [Ming Jiang](https://md.research.elamur.ai/free/en/authors/ming-jiang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1802/) · [PDF](https://aclanthology.org/2026.findings-acl.1802.pdf) · [DOI 10.18653/v1/2026.findings-acl.1802](https://doi.org/10.18653/v1/2026.findings-acl.1802) · [OpenAlex](https://openalex.org/W7166806917)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Neuron-level interpretability work on vision–language models usually studies one task at a time and scores each FFN neuron independently. This makes importance scores hard to compare across tasks. It also inflates the scores of polysemantic neurons that serve several tasks. As a remedy, the authors introduce a two-stage technique, called HONES (short for Head-Oriented Neuron Explanation & Steering), which needs no gradient computation. First, it uses causal ablations to find the attention heads that carry task-relevant information. Second, it ranks FFN neurons by how much they write into the residual stream given those heads, computing all scores together in a few forward passes. Steering then keeps the backbone frozen and learns sparse per-neuron scaling factors for the selected neurons. Experiments cover VQA, OCR, captioning and image-to-text retrieval on a shared image set, using LLaVA-1.5-7B and Qwen2.5-VL-7B. HONES identifies neurons whose masking hurts performance more than competing ranking methods do, and its steering improves all four tasks on both models. This gives a cheap, model-native tool for diagnosing and controlling multi-task VLMs.

## Key points
- **Task metrics:** each task is scored with its own metric (VQAv2 accuracy, ANLS, BLEU-4, NDCG@5). Each task's data is divided into discovery, dev and test splits that share no images.
- **Neuron patterns:**
  - High activation often does not mean causal importance.
  - Critical neurons follow a "rise–valley–surge" pattern across layers, with task-specific anchoring.
  - Neurons shared across tasks form a sparse hub-and-bridge structure centred on VQA, and they matter more than task-specific ones.
- **In-domain steering:** steering acts only on the dominant salient neuron group drawn from the top-1% neurons. It beats a random-neuron control (RandNeuron) and a uniform-amplifier Grid-Search baseline.
- **Out-of-distribution transfer:** tested on GQA, TextVQA and Flickr30k.
  - Reusing the in-domain scaling factors zero-shot already improves results.
  - Re-tuning the scales on 20% of the OOD data, without re-identifying heads or neurons, gives further gains (averaged over 10 random splits).
- **Limitations and code:** only 7B dense models were tested, with four coarse task categories, and the ablations are computationally costly. Code: github.com/petergit1/HONES.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Wisconsin–Madison](https://md.research.elamur.ai/free/en/universities/university-of-wisconsin-madison/) — University, Madison, United States · 54 works · 49,089 citations
- [Tongji University](https://md.research.elamur.ai/free/en/universities/tongji-university/) — University, Shanghai, China · 33 works · 2,993 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works

## Cite

Cite as: Qidong Wang, Junjie Hu and Ming Jiang. 2026. From Heads to Neurons: Causal Attribution and Steering in Multi-Task Vision–Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026heads,
  title = {{From Heads to Neurons: Causal Attribution and Steering in Multi-Task Vision–Language Models}},
  author = {Qidong Wang, Junjie Hu and Ming Jiang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1802},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1802}
}
```

- [← previous From Guidelines to Guarantees: A Graph-Based Evaluation Harness for Domain-Specific Evaluation of LLMs](https://md.research.elamur.ai/free/en/papers/lundin-2026-guidelines-guarantees-graph-based-evaluation/)
- [next → From ID to LLM: Rethinking Representation Learning for Recommendation](https://md.research.elamur.ai/free/en/papers/wu-2026-id-llm-rethinking-representation-learning/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/)

## References

48 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L. Menick, Sebastian Borgeaud, and 8… — [Flamingo: a Visual Language Model for Few-Shot Learning](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/)
2. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025. [Qwen2.5-vl technical report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/). Preprint, arXiv:2502.13…
3. Samyadeep Basu, Martin Grayson, Cecily Morrison, Besmira Nushi, Soheil Feizi, and Daniela Massiceti. 2024. Understanding information storage and transfer in multi-modal large language models. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 7400–7426. [find](https://scholar.google.com/scholar?q=Samyadeep+Basu%2C+Martin+Grayson%2C+Cecily+Morrison%2C+Besmira+Nushi%2C+Soheil+Feizi%2C+and+Daniela+Massiceti%2E+2024%2E+Understanding+information+storage+and+transfer+in+multi%2Dmodal+large+language+models%2E+In+Advan)
4. Ali Furkan Biten, Ruben Tito, Andres Mafla, Lluis Gomez, Marcal Rusinol, Ernest Valveny, C. V. Jawahar, and Dimosthenis Karatzas. 2019. Scene text Lihu Chen, Adam Dejl, and Francesca Toni. 2025. Identifying query-relevant neurons in large language models for long-form texts. In Proceedings of the AA… [find](https://scholar.google.com/scholar?q=Ali+Furkan+Biten%2C+Ruben+Tito%2C+Andres+Mafla%2C+Lluis+Gomez%2C+Marcal+Rusinol%2C+Ernest+Valveny%2C+C%2E+V%2E+Jawahar%2C+and+Dimosthenis+Karatzas%2E+2019%2E+Scene+text+Lihu+Chen%2C+Adam+Dejl%2C+and+Francesca+Toni%2E+2025%2E+Ident)
5. Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C. Lawrence Zitnick. 2015. [Microsoft COCO captions: Data collection and evaluation server](https://md.research.elamur.ai/free/en/papers/chen-2015-microsoft-coco-captions-data-collection/). arXiv preprint arXiv:1504.00325.
6. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. 2024. [Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks](https://md.research.elamur.ai/free/en/papers/chen-2023-internvl-scaling-up-vision/). In Proceedings of the IEEE/CV…
7. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493– 8502. [find](https://scholar.google.com/scholar?q=Damai+Dai%2C+Li+Dong%2C+Yaru+Hao%2C+Zhifang+Sui%2C+Baobao+Chang%2C+and+Furu+Wei%2E+2022%2E+Knowledge+neurons+in+pretrained+transformers%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computationa)
8. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. 2023. [Instructblip: Towards general-purpose vision-language models with instruction tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/). In Advances in Neural Information Processing Systems, volume 36, pages 49250–49267.
9. Yunkai Dang, Kaichen Huang, Jiahao Huo, Yibo Yan, Sirui Huang, Dongrui Liu, Mengxi Gao, Jie Zhang, Chen Qian, Kun Wang, Yong Liu, Jing Shao, Hui Xiong, and Xuming Hu. 2024. Explainable and interpretable multimodal large language models: A comprehensive survey. CoRR, abs/2412.02104. — [arXiv:2412.02104](https://arxiv.org/abs/2412.02104) [find](https://scholar.google.com/scholar?q=Yunkai+Dang%2C+Kaichen+Huang%2C+Jiahao+Huo%2C+Yibo+Yan%2C+Sirui+Huang%2C+Dongrui+Liu%2C+Mengxi+Gao%2C+Jie+Zhang%2C+Chen+Qian%2C+Kun+Wang%2C+Yong+Liu%2C+Jing+Shao%2C+Hui+Xiong%2C+and+Xuming+Hu%2E+2024%2E+Explainable+and+interpretab)
10. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, and 6 others. 2021. A mathe… [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Neel+Nanda%2C+Catherine+Olsson%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Tom+Conerly%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Zac+Hatfield%2DDodds%2C+Da)
11. Junfeng Fang, Zongze Bi, Ruipeng Wang, Houcheng Jiang, Yuan Gao, Kun Wang, An Zhang, Jie Shi, Xiang Wang, and Tat-Seng Chua. 2024. Towards neuron attributions in multi-modal large language models. In Advances in Neural Information Processing Systems, volume 37, pages 122867–122890. Curran Associates… [find](https://scholar.google.com/scholar?q=Junfeng+Fang%2C+Zongze+Bi%2C+Ruipeng+Wang%2C+Houcheng+Jiang%2C+Yuan+Gao%2C+Kun+Wang%2C+An+Zhang%2C+Jie+Shi%2C+Xiang+Wang%2C+and+Tat%2DSeng+Chua%2E+2024%2E+Towards+neuron+attributions+in+multi%2Dmodal+large+language+models%2E+In+)
12. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495. [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Roei+Schuster%2C+Jonathan+Berant%2C+and+Omer+Levy%2E+2021%2E+Transformer+feed%2Dforward+layers+are+key%2Dvalue+memories%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
13. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. [Making the v in vqa matter: Elevating the role of image understanding in visual question answering](https://md.research.elamur.ai/free/en/papers/goyal-2016-making-v-vqa-matter-elevating/). In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6325–6334.
14. Muhammad Umair Haider, Hammad Rizwan, Hassan Sajjad, Peizhong Ju, and A. B. Siddique. 2025. Neurons speak in ranges: Breaking free from discrete neuronal attribution. In Mechanistic Interpretability Workshop at NeurIPS 2025. [find](https://scholar.google.com/scholar?q=Muhammad+Umair+Haider%2C+Hammad+Rizwan%2C+Hassan+Sajjad%2C+Peizhong+Ju%2C+and+A%2E+B%2E+Siddique%2E+2025%2E+Neurons+speak+in+ranges%3A+Breaking+free+from+discrete+neuronal+attribution%2E+In+Mechanistic+Interpretability+W)
15. Anwen Hu, Haiyang Xu, Liang Zhang, Jiabo Ye, Ming Yan, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. 2025. [mPLUG-DocOwl2: High-resolution compressing for OCR-free multi-page document understanding](https://md.research.elamur.ai/free/en/papers/hu-2025-mplug-docowl2-high-resolution-compressing/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1…
16. Kaichen Huang, Jiahao Huo, Yibo Yan, Kun Wang, Yutao Yue, and Xuming Hu. 2024. MINER: Mining the underlying pattern of modality-specific neurons in multimodal large language models. arXiv preprint arXiv:2410.04819. — [arXiv:2410.04819](https://arxiv.org/abs/2410.04819) [find](https://scholar.google.com/scholar?q=Kaichen+Huang%2C+Jiahao+Huo%2C+Yibo+Yan%2C+Kun+Wang%2C+Yutao+Yue%2C+and+Xuming+Hu%2E+2024%2E+MINER%3A+Mining+the+underlying+pattern+of+modality%2Dspecific+neurons+in+multimodal+large+language+models%2E+arXiv+preprint+arX)
17. Drew A. Hudson and Christopher D. Manning. 2019. [Gqa: A new dataset for real-world visual reasoning and compositional question answering](https://md.research.elamur.ai/free/en/papers/hudson-2019-gqa-new-dataset-real-world/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6700– 6709.
18. Jiahao Huo, Yibo Yan, Boren Hu, Yutao Yue, and Xuming Hu. 2024. MMNeuron: Discovering neuron-level domain-specific interpretation in multimodal large language model. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6801–6816, Miami, Florida, USA. Assoc… [find](https://scholar.google.com/scholar?q=Jiahao+Huo%2C+Yibo+Yan%2C+Boren+Hu%2C+Yutao+Yue%2C+and+Xuming+Hu%2E+2024%2E+MMNeuron%3A+Discovering+neuron%2Dlevel+domain%2Dspecific+interpretation+in+multimodal+large+language+model%2E+In+Proceedings+of+the+2024+Confere)
19. Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):422– 446. [find](https://scholar.google.com/scholar?q=Kalervo+J%C3%A4rvelin+and+Jaana+Kek%C3%A4l%C3%A4inen%2E+2002%2E+Cumulated+gain%2Dbased+evaluation+of+IR+techniques%2E+ACM+Transactions+on+Information+Systems%2C+20%284%29%3A422%E2%80%93+446%2E)
20. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. [BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages…
21. Hyesu Lim, Jinho Choi, Jaegul Choo, and Steffen Schneider. 2025. Sparse autoencoders reveal selective remapping of visual concepts during adaptation. In International Conference on Learning Representations (ICLR). Poster. [find](https://scholar.google.com/scholar?q=Hyesu+Lim%2C+Jinho+Choi%2C+Jaegul+Choo%2C+and+Steffen+Schneider%2E+2025%2E+Sparse+autoencoders+reveal+selective+remapping+of+visual+concepts+during+adaptation%2E+In+International+Conference+on+Learning+Representa)
22. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. [Microsoft COCO: Common objects in context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/). In European Conference on Computer Vision (ECCV), pages 740–755.
23. Zihao Lin, Samyadeep Basu, Mohammad Beigi, Varun Manjunatha, Ryan A. Rossi, Zichao Wang, Yufan Zhou, Sriram Balasubramanian, Arman Zarei, Keivan Rezaei, Ying Shen, Barry Menglong Yao, Zhiyang Xu, Qin Liu, Yuxiang Zhang, Yan Sun, Shilong Liu, Li Shen, Hongxuan Li, and 2 others. 2025. A survey on mech… — [arXiv:2502.17516](https://arxiv.org/abs/2502.17516) [find](https://scholar.google.com/scholar?q=Zihao+Lin%2C+Samyadeep+Basu%2C+Mohammad+Beigi%2C+Varun+Manjunatha%2C+Ryan+A%2E+Rossi%2C+Zichao+Wang%2C+Yufan+Zhou%2C+Sriram+Balasubramanian%2C+Arman+Zarei%2C+Keivan+Rezaei%2C+Ying+Shen%2C+Barry+Menglong+Yao%2C+Zhiyang+Xu%2C+Qin+)
24. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). In Advances in Neural Information Processing Systems, volume 36, pages 34892–34916.
25. Zhihang Liu, Chen-Wei Xie, Bin Wen, Feiwu Yu, Jixuan Chen, Pandeng Li, Boqiang Zhang, Nianzu Yang, Yinglu Li, Zuan Gao, Yun Zheng, and Hongtao Xie. 2025. Capability: A comprehensive visual caption benchmark for evaluating both correctness and thoroughness. arXiv preprint arXiv:2502.14914. — [arXiv:2502.14914](https://arxiv.org/abs/2502.14914) [find](https://scholar.google.com/scholar?q=Zhihang+Liu%2C+Chen%2DWei+Xie%2C+Bin+Wen%2C+Feiwu+Yu%2C+Jixuan+Chen%2C+Pandeng+Li%2C+Boqiang+Zhang%2C+Nianzu+Yang%2C+Yinglu+Li%2C+Zuan+Gao%2C+Yun+Zheng%2C+and+Hongtao+Xie%2E+2025%2E+Capability%3A+A+comprehensive+visual+caption+ben)
26. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. In Advances in Neural Information Processing Systems, pages 17359–17372. [find](https://scholar.google.com/scholar?q=Kevin+Meng%2C+David+Bau%2C+Alex+Andonian%2C+and+Yonatan+Belinkov%2E+2022%2E+Locating+and+editing+factual+associations+in+gpt%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+17359%E2%80%9317372%2E)
27. Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, and Fazl Barez. 2025. Towards interpreting visual information processing in vision-language models. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Clement+Neo%2C+Luke+Ong%2C+Philip+Torr%2C+Mor+Geva%2C+David+Krueger%2C+and+Fazl+Barez%2E+2025%2E+Towards+interpreting+visual+information+processing+in+vision%2Dlanguage+models%2E+In+International+Conference+on+Learning)
28. Tuomas Oikarinen and Tsui-Wei Weng. 2024. Linear explanations for individual neurons. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 38639–38662. PMLR. [find](https://scholar.google.com/scholar?q=Tuomas+Oikarinen+and+Tsui%2DWei+Weng%2E+2024%2E+Linear+explanations+for+individual+neurons%2E+In+Proceedings+of+the+41st+International+Conference+on+Machine+Learning%2C+volume+235+of+Proceedings+of+Machine+Lear)
29. Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, and Zeynep Akata. 2025. Sparse autoencoders learn monosemantic features in vision-language models. arXiv preprint arXiv:2504.02821. — [arXiv:2504.02821](https://arxiv.org/abs/2504.02821) [find](https://scholar.google.com/scholar?q=Mateusz+Pach%2C+Shyamgopal+Karthik%2C+Quentin+Bouniot%2C+Serge+Belongie%2C+and+Zeynep+Akata%2E+2025%2E+Sparse+autoencoders+learn+monosemantic+features+in+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2504%2E02821%2E+%E2%80%94)
30. Haowen Pan, Yixin Cao, Xiaozhi Wang, Xun Yang, and Meng Wang. 2024. Finding and editing multi-modal neurons in pre-trained transformers. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1012–1037, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Haowen+Pan%2C+Yixin+Cao%2C+Xiaozhi+Wang%2C+Xun+Yang%2C+and+Meng+Wang%2E+2024%2E+Finding+and+editing+multi%2Dmodal+neurons+in+pre%2Dtrained+transformers%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+A)
31. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [BLEU: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pages 311–318.
32. Hassan Sajjad, Nadir Durrani, and Fahim Dalvi. 2022. [Neuron-level interpretation of deep NLP models: A survey](https://md.research.elamur.ai/free/en/papers/sajjad-2022-neuron-level-interpretation-deep-nlp/). Transactions of the Association for Computational Linguistics, 10:1285–1303.
33. Sarah Schwettmann, Neil Chowdhury, Samuel Klein, David Bau, and Antonio Torralba. 2023. Multimodal neurons in pretrained text-only transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pages 2862– 2867. [find](https://scholar.google.com/scholar?q=Sarah+Schwettmann%2C+Neil+Chowdhury%2C+Samuel+Klein%2C+David+Bau%2C+and+Antonio+Torralba%2E+2023%2E+Multimodal+neurons+in+pretrained+text%2Donly+transformers%2E+In+Proceedings+of+the+IEEE%2FCVF+International+Conference)
34. Dong Shu, Xuansheng Wu, Haiyan Zhao, Daking Rai, Ziyu Yao, Ninghao Liu, and Mengnan Du. 2025. A survey on sparse autoencoders: Interpreting the internal mechanisms of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 1690–1712, Suzhou, China. Asso… [find](https://scholar.google.com/scholar?q=Dong+Shu%2C+Xuansheng+Wu%2C+Haiyan+Zhao%2C+Daking+Rai%2C+Ziyu+Yao%2C+Ninghao+Liu%2C+and+Mengnan+Du%2E+2025%2E+A+survey+on+sparse+autoencoders%3A+Interpreting+the+internal+mechanisms+of+large+language+models%2E+In+Finding)
35. Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards vqa models that can read. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8309–8318. [find](https://scholar.google.com/scholar?q=Amanpreet+Singh%2C+Vivek+Natarajan%2C+Meet+Shah%2C+Yu+Jiang%2C+Xinlei+Chen%2C+Dhruv+Batra%2C+Devi+Parikh%2C+and+Marcus+Rohrbach%2E+2019%2E+Towards+vqa+models+that+can+read%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+)
36. Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. [Language-specific neurons: The key to multilingual capabilities in large language models](https://md.research.elamur.ai/free/en/papers/tang-2024-language-specific-neurons-key-multilingual/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu…
37. Andreas Veit, Tomas Matera, Lukas Neumann, Jiri Matas, and Serge Belongie. 2016. COCO-text: Dataset and benchmark for text detection and recognition in natural images. arXiv preprint arXiv:1601.07140. — [arXiv:1601.07140](https://arxiv.org/abs/1601.07140) [find](https://scholar.google.com/scholar?q=Andreas+Veit%2C+Tomas+Matera%2C+Lukas+Neumann%2C+Jiri+Matas%2C+and+Serge+Belongie%2E+2016%2E+COCO%2Dtext%3A+Dataset+and+benchmark+for+text+detection+and+recognition+in+natural+images%2E+arXiv+preprint+arXiv%3A1601%2E07140%2E)
38. Feiyu Wang, Ziran Zhao, Dong Yu, and Pengyuan Liu. 2025a. Attribution and application of multiple neurons in multimodal large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 11649–11662. [find](https://scholar.google.com/scholar?q=Feiyu+Wang%2C+Ziran+Zhao%2C+Dong+Yu%2C+and+Pengyuan+Liu%2E+2025a%2E+Attribution+and+application+of+multiple+neurons+in+multimodal+large+language+models%2E+In+Findings+of+the+Association+for+Computational+Linguist)
39. Qidong Wang, Junjie Hu, and Ming Jiang. 2025b. V-SEAM: visual semantic editing and attention modulating for causal interpretability of vision-language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 17396–17420, Suzhou, China. Association for… [find](https://scholar.google.com/scholar?q=Qidong+Wang%2C+Junjie+Hu%2C+and+Ming+Jiang%2E+2025b%2E+V%2DSEAM%3A+visual+semantic+editing+and+attention+modulating+for+causal+interpretability+of+vision%2Dlanguage+models%2E+In+Proceedings+of+the+2025+Conference+on+)
40. Jiaqi Xu, Cuiling Lan, and Yan Lu. 2025. Deciphering functions of neurons in vision-language models. In Proceedings of the 33rd ACM International Conference on Multimedia, page 3173–3181. [find](https://scholar.google.com/scholar?q=Jiaqi+Xu%2C+Cuiling+Lan%2C+and+Yan+Lu%2E+2025%2E+Deciphering+functions+of+neurons+in+vision%2Dlanguage+models%2E+In+Proceedings+of+the+33rd+ACM+International+Conference+on+Multimedia%2C+page+3173%E2%80%933181%2E)
41. Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, Qin Jin, Liang He, Xin Lin, and Fei Huang. 2023. [UReader: Universal OCR-free visually-situated language understanding with multimodal large language model](https://md.research.elamur.ai/free/en/papers/ye-2023-ureader-universal-ocr-free-visually/). In Findings of the Association f…
42. Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. 2024. mplug-owi2: Revolutionizing multi-modal large language model with modality collaboration. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13040–13051. [find](https://scholar.google.com/scholar?q=Qinghao+Ye%2C+Haiyang+Xu%2C+Jiabo+Ye%2C+Ming+Yan%2C+Anwen+Hu%2C+Haowei+Liu%2C+Qi+Qian%2C+Ji+Zhang%2C+and+Fei+Huang%2E+2024%2E+mplug%2Dowi2%3A+Revolutionizing+multi%2Dmodal+large+language+model+with+modality+collaboration%2E+In+2)
43. Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67–78. [find](https://scholar.google.com/scholar?q=Peter+Young%2C+Alice+Lai%2C+Micah+Hodosh%2C+and+Julia+Hockenmaier%2E+2014%2E+From+image+descriptions+to+visual+denotations%3A+New+similarity+metrics+for+semantic+inference+over+event+descriptions%2E+Transactions+of)
44. Zeping Yu and Sophia Ananiadou. 2024. Neuron-level knowledge attribution in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3267–3280, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zeping+Yu+and+Sophia+Ananiadou%2E+2024%2E+Neuron%2Dlevel+knowledge+attribution+in+large+language+models%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+3267)
45. Zhuoran Yu and Yong Jae Lee. 2025. How multimodal llms solve image tasks: A lens on visual grounding, task reasoning, and answer decoding. Preprint, arXiv:2508.20279. — [arXiv:2508.20279](https://arxiv.org/abs/2508.20279) [find](https://scholar.google.com/scholar?q=Zhuoran+Yu+and+Yong+Jae+Lee%2E+2025%2E+How+multimodal+llms+solve+image+tasks%3A+A+lens+on+visual+grounding%2C+task+reasoning%2C+and+answer+decoding%2E+Preprint%2C+arXiv%3A2508%2E20279%2E+%E2%80%94+%5BarXiv%3A2508%2E20279%5D%28https%3A%2F%2Farxi)
46. Fred Zhang and Neel Nanda. 2024. [Towards best practices of activation patching in language models: Metrics and methods](https://md.research.elamur.ai/free/en/papers/zhang-2023-towards-best-practices-activation-patching/). In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.
47. Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. [Explainability for large language models: A survey](https://md.research.elamur.ai/free/en/papers/zhao-2023-explainability-large-language-models-survey/). ACM Trans. Intell. Syst. Technol., 15(2):1–38.
48. Xiutian Zhao, Rochelle Choenni, Rohit Saxena, and Ivan Titov. 2026. [Finding culture-sensitive neurons in vision-language models](https://md.research.elamur.ai/free/en/papers/zhao-2026-finding-culture-sensitive-neurons-vision/). In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2026 - Volume 1: Long Papers, Rabat, Morocco, March 24…

*Updated 2 November 2026*