---
title: "ReAttn — paper by Tian et al. (2026) — summary and ideas"
description: "Attention-based re-ranking scores candidate documents from an LLM's internal attention rather than from generated output. 52 references."
authors: ["Yuxing Tian", "Fengran Mo", "Weixu Zhang", "Yiyan Qi", "Jian-Yun Nie"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: EACL 2026"
source: "https://aclanthology.org/2026.findings-eacl.66/"
doi: "10.18653/v1/2026.findings-eacl.66"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tian-2026-reattn-improving-attention-based-re/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tian-2026-reattn-improving-attention-based-re/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/) / ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting

# ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting

**Authors:** [Yuxing Tian](https://md.research.elamur.ai/free/en/authors/yuxing-tian/), [Fengran Mo](https://md.research.elamur.ai/free/en/authors/fengran-mo/), [Weixu Zhang](https://md.research.elamur.ai/free/en/authors/weixu-zhang/), [Yiyan Qi](https://md.research.elamur.ai/free/en/authors/yiyan-qi/) and [Jian-Yun Nie](https://md.research.elamur.ai/free/en/authors/jian-yun-nie/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.66/) · [PDF](https://aclanthology.org/2026.findings-eacl.66.pdf) · [DOI 10.18653/v1/2026.findings-eacl.66](https://doi.org/10.18653/v1/2026.findings-eacl.66) · [OpenAlex](https://openalex.org/W7140078186)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Attention-based re-ranking scores candidate documents from an LLM's internal attention rather than from generated output. The documents and the query are fed in together, and the attention that query tokens give each document's tokens is summed into a relevance score. The authors identify two weaknesses in this signal. First, attention mass collapses onto a few tokens in a few documents, leaving most candidates hard to tell apart. Second, it favours surface word overlap with the query, so lexically similar but irrelevant passages rank too high.

ReAttn is a training-free, post-hoc correction applied on top of existing methods such as ICR and QRhead. It rescales token scores with an IDF computed over the candidate set, so query terms shared by many candidates count less. It then reweights each document by the entropy of its token-score distribution, favouring documents whose attention is spread out. Tests on BEIR datasets (nDCG@10) and long-context multi-hop reasoning, across several base LLMs, show consistent gains at little extra cost. This makes zero-shot LLM re-ranking cheaper to improve and more trustworthy.

## Key points
- **Two named failure modes:** "signal concentration" (attention dominated by a few tokens and documents) and "lexical bias" (over-weighting of query-overlapping terms).
- **Two components:** a cross-document IDF reweighting at token level and an entropy-based regularization at document level. Both operate on existing attention weights with no fine-tuning or labels.
- **Setup:** ReAttn is applied to the attention-based re-rankers ICR (Chen et al., 2025) and QR heads/QRhead (Zhang et al., 2025c). Evaluation uses BEIR with nDCG@10, plus multi-hop long-context reasoning, across different base LLMs.
- **Ablation findings:**
  - IDF alone delivers most of the gain on keyword-heavy sets such as NQ, NFCorpus, FiQA and DBPedia.
  - Entropy helps more on long, heterogeneous collections such as Robust04, News and Climate.
  - On Scidocs, IDF alone slightly hurts, but the full combination is best, suggesting the two parts are complementary.
  - Qualitative attention maps on FEVER illustrate the effect.
- **Limitations:** Experiments cover only English datasets and general-purpose (not IR-tuned) LLMs. Behaviour on multilingual data, including Ukrainian, remains untested.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works

## Cite

Cite as: Yuxing Tian, Fengran Mo, Weixu Zhang, Yiyan Qi and Jian-Yun Nie. 2026. ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tian2026reattn,
  title = {{ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting}},
  author = {Yuxing Tian, Fengran Mo, Weixu Zhang, Yiyan Qi and Jian-Yun Nie},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.66},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.66}
}
```

- [← previous Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey](https://md.research.elamur.ai/free/en/papers/romberg-2026-reassessing-active-learning-adoption-contemporary/)
- [next → RECAP: REwriting Conversations for Intent Understanding in Agentic Planning](https://md.research.elamur.ai/free/en/papers/mitra-2026-recap-rewriting-conversations-intent-understanding/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/)

## References

52 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. 2015. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). In 3rd International Conference on Learning Representations, ICLR 2015.
2. Vera Boteva, Demian Gholipour, Artem Sokolov, and Stefan Riezler. 2016. A full-text learning to rank dataset for medical information retrieval. In Advances in Information Retrieval: 38th European Conference on IR Research, ECIR 2016, Padua, Italy, March 20–23, 2016. Proceedings 38, pages 716–722. Sp… [find](https://scholar.google.com/scholar?q=Vera+Boteva%2C+Demian+Gholipour%2C+Artem+Sokolov%2C+and+Stefan+Riezler%2E+2016%2E+A+full%2Dtext+learning+to+rank+dataset+for+medical+information+retrieval%2E+In+Advances+in+Information+Retrieval%3A+38th+European+Conf)
3. Shijie Chen, Bernal Jimenez Gutierrez, and Yu Su. 2025. Attention in large language models yields efficient zero-shot re-rankers. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Shijie+Chen%2C+Bernal+Jimenez+Gutierrez%2C+and+Yu+Su%2E+2025%2E+Attention+in+large+language+models+yields+efficient+zero%2Dshot+re%2Drankers%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations)
4. Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. 2023. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595. [find](https://scholar.google.com/scholar?q=Shouyuan+Chen%2C+Sherman+Wong%2C+Liangjian+Chen%2C+and+Yuandong+Tian%2E+2023%2E+Extending+context+window+of+large+language+models+via+positional+interpolation%2E+arXiv+preprint+arXiv%3A2306%2E15595%2E)
5. Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel S Weld. 2020. [Specter: Document-level representation learning using citation-informed transformers](https://md.research.elamur.ai/free/en/papers/cohan-2020-specter-document-level-representation-learning/). arXiv preprint arXiv:2004.07180.
6. Thomas Diggelmann, Jordan Boyd-Graber, Jannis Bulian, Massimiliano Ciaramita, and Markus Leippold. 2020. [Climate-fever: A dataset for verification of real-world climate claims](https://md.research.elamur.ai/free/en/papers/diggelmann-2020-climate-fever-dataset-verification-real/). arXiv preprint arXiv:2012.00614.
7. Yao Fu, Rameswar Panda, Xinyao Niu, Xiang Yue, Hannaneh Hajishirzi, Yoon Kim, and Hao Peng. 2024. [Data engineering for scaling language models to 128k context](https://md.research.elamur.ai/free/en/papers/fu-2024-data-engineering-scaling-language-models/). arXiv preprint arXiv:2402.10171.
8. Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. [Bias and fairness in large language models: A survey](https://md.research.elamur.ai/free/en/papers/gallegos-2023-bias-fairness-large-language-models/). Computational Linguistics, pages 1–79.
9. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. 2023. [Retrieval-augmented generation for large language models: A survey](https://md.research.elamur.ai/free/en/papers/gao-2023-retrieval-augmented-generation-large-language/). arXiv preprint arXiv:2312.10997, 2(1).
10. Xueguang Ma, Xinyu Zhang, Ronak Pradeep, and Jimmy Lin. 2023. [Zero-shot listwise document reranking with a large language model](https://md.research.elamur.ai/free/en/papers/ma-2023-zero-shot-listwise-document-reranking/). arXiv preprint arXiv:2305.02156.
11. Faegheh Hasibi, Fedor Nikolaev, Chenyan Xiong, Krisztian Balog, Svein Erik Bratsberg, Alexander Kotov, and Jamie Callan. 2017. Dbpedia-entity v2: a test collection for entity search. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, p… [find](https://scholar.google.com/scholar?q=Faegheh+Hasibi%2C+Fedor+Nikolaev%2C+Chenyan+Xiong%2C+Krisztian+Balog%2C+Svein+Erik+Bratsberg%2C+Alexander+Kotov%2C+and+Jamie+Callan%2E+2017%2E+Dbpedia%2Dentity+v2%3A+a+test+collection+for+entity+search%2E+In+Proceedings+of)
12. Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. 2018. Www’18 open challenge: Financial opinion mining and question answering. In Companion Proceedings of the The Web Conference 2018, WWW ’18, page 1941–1942, Republic and Canton of G… [find](https://scholar.google.com/scholar?q=Macedo+Maia%2C+Siegfried+Handschuh%2C+Andr%C3%A9+Freitas%2C+Brian+Davis%2C+Ross+McDermott%2C+Manel+Zarrouk%2C+and+Alexandra+Balahur%2E+2018%2E+Www%E2%80%9918+open+challenge%3A+Financial+opinion+mining+and+question+answering%2E+In+Com)
13. Gautier Izacard and Edouard Grave. 2021. Distilling knowledge from reader to retriever for question answering. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Gautier+Izacard+and+Edouard+Grave%2E+2021%2E+Distilling+knowledge+from+reader+to+retriever+for+question+answering%2E+In+International+Conference+on+Learning+Representations%2E)
14. Paul Michel, Omer Levy, and Graham Neubig. 2019. [Are sixteen heads really better than one?](https://md.research.elamur.ai/free/en/papers/michel-2019-sixteen-heads-really-better-than/) Advances in neural information processing systems, 32.
15. Vitor Jeronymo, Mauricio Nascimento, Roberto Lotufo, and Rodrigo Nogueira. 2022. mrobust04: A multilingual version of the trec robust 2004 benchmark. Preprint, arXiv:2209.13738. — [arXiv:2209.13738](https://arxiv.org/abs/2209.13738) [find](https://scholar.google.com/scholar?q=Vitor+Jeronymo%2C+Mauricio+Nascimento%2C+Roberto+Lotufo%2C+and+Rodrigo+Nogueira%2E+2022%2E+mrobust04%3A+A+multilingual+version+of+the+trec+robust+2004+benchmark%2E+Preprint%2C+arXiv%3A2209%2E13738%2E+%E2%80%94+%5BarXiv%3A2209%2E13738%5D%28h)
16. Hongye Jin, Xiaotian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia-Yuan Chang, Huiyuan Chen, and Xia Hu. 2024a. [Llm maybe longlm: Self-extend llm context window without tuning](https://md.research.elamur.ai/free/en/papers/jin-2024-llm-maybe-longlm-self-extend/). arXiv preprint arXiv:2401.01325.
17. Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao. 2024b. Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge conflicts in language models. In Findings of the Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Zhuoran+Jin%2C+Pengfei+Cao%2C+Hongbang+Yuan%2C+Yubo+Chen%2C+Jiexin+Xu%2C+Huaijun+Li%2C+Xiaojian+Jiang%2C+Kang+Liu%2C+and+Jun+Zhao%2E+2024b%2E+Cutting+off+the+head+ends+the+conflict%3A+A+mechanism+for+interpreting+and+mitig)
18. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). arXiv preprint arXiv:2004.04906.
19. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [Natural…](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
20. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in Neural Information Processing Systems, 33:9459–…
21. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, and 1 others. 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110. [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Rishi+Bommasani%2C+Tony+Lee%2C+Dimitris+Tsipras%2C+Dilara+Soylu%2C+Michihiro+Yasunaga%2C+Yian+Zhang%2C+Deepak+Narayanan%2C+Yuhuai+Wu%2C+Ananya+Kumar%2C+and+1+others%2E+2022%2E+Holistic+evaluation+of+language+m)
22. Fengran Mo, Yifan Gao, Chuan Meng, Xin Liu, Zhuofeng Wu, Kelong Mao, Zhengyang Wang, Pei Chen, Zheng Li, Xian Li, and 1 others. 2025. Uniconv: Unifying retrieval and response generation for large language models in conversations. In Proceedings of the 63rd Annual Meeting of the Association for Compu… [find](https://scholar.google.com/scholar?q=Fengran+Mo%2C+Yifan+Gao%2C+Chuan+Meng%2C+Xin+Liu%2C+Zhuofeng+Wu%2C+Kelong+Mao%2C+Zhengyang+Wang%2C+Pei+Chen%2C+Zheng+Li%2C+Xian+Li%2C+and+1+others%2E+2025%2E+Uniconv%3A+Unifying+retrieval+and+response+generation+for+large+lang)
23. Fengran Mo, Kelong Mao, Yutao Zhu, Yihong Wu, Kaiyu Huang, and Jian-Yun Nie. 2023. Convgqr: Generative query reformulation for conversational search. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4998–5012. [find](https://scholar.google.com/scholar?q=Fengran+Mo%2C+Kelong+Mao%2C+Yutao+Zhu%2C+Yihong+Wu%2C+Kaiyu+Huang%2C+and+Jian%2DYun+Nie%2E+2023%2E+Convgqr%3A+Generative+query+reformulation+for+conversational+search%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+A)
24. Fengran Mo, Zhan Su, Yuchen Hui, Jinghan Zhang, Jia Ao Sun, Zheyuan Liu, Chao Zhang, Tetsuya Sakai, and Jian-Yun Nie. 2026. Opendecoder: Open large language model decoding to incorporate document quality in rag. arXiv preprint arXiv:2601.09028. — [arXiv:2601.09028](https://arxiv.org/abs/2601.09028) [find](https://scholar.google.com/scholar?q=Fengran+Mo%2C+Zhan+Su%2C+Yuchen+Hui%2C+Jinghan+Zhang%2C+Jia+Ao+Sun%2C+Zheyuan+Liu%2C+Chao+Zhang%2C+Tetsuya+Sakai%2C+and+Jian%2DYun+Nie%2E+2026%2E+Opendecoder%3A+Open+large+language+model+decoding+to+incorporate+document+qual)
25. Rodrigo Nogueira, Zhiying Jiang, and Jimmy Lin. 2020. [Document ranking with a pretrained sequence-to-sequence model](https://md.research.elamur.ai/free/en/papers/nogueira-2020-document-ranking-pretrained-sequence-sequence/). arXiv preprint arXiv:2003.06713.
26. Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. 2019. Multi-stage document ranking with bert. arXiv preprint arXiv:1910.14424. [find](https://scholar.google.com/scholar?q=Rodrigo+Nogueira%2C+Wei+Yang%2C+Kyunghyun+Cho%2C+and+Jimmy+Lin%2E+2019%2E+Multi%2Dstage+document+ranking+with+bert%2E+arXiv+preprint+arXiv%3A1910%2E14424%2E)
27. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, and 1 others. 2022. [In-context learning and induction heads](https://md.research.elamur.ai/free/en/papers/olsson-2022-context-learning-induction-heads/). arXiv preprint arXiv:2209.11895.
28. Alexander Peysakhovich and Adam Lerer. 2023. Attention sorting combats recency bias in long context language models. ArXiv, abs/2310.01427. — [arXiv:2310.01427](https://arxiv.org/abs/2310.01427) [find](https://scholar.google.com/scholar?q=Alexander+Peysakhovich+and+Adam+Lerer%2E+2023%2E+Attention+sorting+combats+recency+bias+in+long+context+language+models%2E+ArXiv%2C+abs%2F2310%2E01427%2E+%E2%80%94+%5BarXiv%3A2310%2E01427%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E01427%29)
29. Chau Minh Pham, Yapei Chang, and Mohit Iyyer. 2025. Clipper: Compression enables long-context synthetic data generation. Preprint, arXiv:2502.14854. — [arXiv:2502.14854](https://arxiv.org/abs/2502.14854) [find](https://scholar.google.com/scholar?q=Chau+Minh+Pham%2C+Yapei+Chang%2C+and+Mohit+Iyyer%2E+2025%2E+Clipper%3A+Compression+enables+long%2Dcontext+synthetic+data+generation%2E+Preprint%2C+arXiv%3A2502%2E14854%2E+%E2%80%94+%5BarXiv%3A2502%2E14854%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2502%2E148)
30. Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, and 1 others. 2024. [Large language models are effective text rankers with pairwise ranking prompting](https://md.research.elamur.ai/free/en/papers/qin-2024-large-language-models-are-effective/). In Findings of the Association for Computational Linguistics: NAACL 2024, pag…
31. Jie Ren, Qipeng Guo, Hang Yan, Dongrui Liu, Quanshi Zhang, Xipeng Qiu, and Dahua Lin. 2024. Identifying semantic induction heads to understand in-context learning. In Findings of the Association for Computational Linguistics ACL 2024, pages 6916– 6932. [find](https://scholar.google.com/scholar?q=Jie+Ren%2C+Qipeng+Guo%2C+Hang+Yan%2C+Dongrui+Liu%2C+Quanshi+Zhang%2C+Xipeng+Qiu%2C+and+Dahua+Lin%2E+2024%2E+Identifying+semantic+induction+heads+to+understand+in%2Dcontext+learning%2E+In+Findings+of+the+Association+for+C)
32. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3(4):333–389. [find](https://scholar.google.com/scholar?q=Stephen+Robertson+and+Hugo+Zaragoza%2E+2009%2E+The+probabilistic+relevance+framework%3A+Bm25+and+beyond%2E+Found%2E+Trends+Inf%2E+Retr%2E%2C+3%284%29%3A333%E2%80%93389%2E)
33. Devendra Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen-tau Yih, Joelle Pineau, and Luke Zettlemoyer. 2022. [Improving passage retrieval with zero-shot question generation](https://md.research.elamur.ai/free/en/papers/sachan-2022-improving-passage-retrieval-zero-shot/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3781–3797.
34. Sofia Serrano and Noah A. Smith. 2019. [Is attention interpretable?](https://md.research.elamur.ai/free/en/papers/serrano-2019-is-attention-interpretable/) In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2931–2951, Florence, Italy. Association for Computational Linguistics.
35. Dan Shi, Renren Jin, Tianhao Shen, Weilong Dong, Xinwei Wu, and Deyi Xiong. 2024. Ircan: Mitigating knowledge conflicts in llm generation via identifying and reweighting context-aware neurons. Advances in Neural Information Processing Systems, 37:4997– 5024. [find](https://scholar.google.com/scholar?q=Dan+Shi%2C+Renren+Jin%2C+Tianhao+Shen%2C+Weilong+Dong%2C+Xinwei+Wu%2C+and+Deyi+Xiong%2E+2024%2E+Ircan%3A+Mitigating+knowledge+conflicts+in+llm+generation+via+identifying+and+reweighting+context%2Daware+neurons%2E+Advance)
36. Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023a. [Is chatgpt good at search? investigating large language models as re-ranking agents](https://md.research.elamur.ai/free/en/papers/sun-2023-chatgpt-good-search-investigating-large/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 149…
37. Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023b. [Is ChatGPT good at search? investigating large language models as re-ranking agents](https://md.research.elamur.ai/free/en/papers/sun-2023-chatgpt-good-search-investigating-large/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 149…
38. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). [find](https://scholar.google.com/scholar?q=Nandan+Thakur%2C+Nils+Reimers%2C+Andreas+R%C3%BCckl%C3%A9%2C+Abhishek+Srivastava%2C+and+Iryna+Gurevych%2E+2021%2E+BEIR%3A+A+heterogeneous+benchmark+for+zero%2Dshot+evaluation+of+information+retrieval+models%2E+In+Thirty%2Dfifth+Co)
39. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. [Fever: a large-scale dataset for fact extraction and verification](https://md.research.elamur.ai/free/en/papers/thorne-2018-fever-large-scale-dataset-fact/). arXiv preprint arXiv:1803.05355.
40. Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. [Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned](https://md.research.elamur.ai/free/en/papers/voita-2019-analyzing-multi-head-self-attention/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5797–5808.
41. Ellen Voorhees, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, William R Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, and Lucy Lu Wang. 2021. Trec-covid: constructing a pandemic information retrieval test collection. In ACM SIGIR Forum, volume 54, pages 1–12. ACM New York, NY, USA. [find](https://scholar.google.com/scholar?q=Ellen+Voorhees%2C+Tasmeer+Alam%2C+Steven+Bedrick%2C+Dina+Demner%2DFushman%2C+William+R+Hersh%2C+Kyle+Lo%2C+Kirk+Roberts%2C+Ian+Soboroff%2C+and+Lucy+Lu+Wang%2E+2021%2E+Trec%2Dcovid%3A+constructing+a+pandemic+information+retriev)
42. David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. [Fact or fiction: Verifying scientific claims](https://md.research.elamur.ai/free/en/papers/wadden-2020-fact-or-fiction-verifying-scientific/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534–7550, Online. Association…
43. Sarah Wiegreffe and Yuval Pinter. 2019. [Attention is not not explanation](https://md.research.elamur.ai/free/en/papers/wiegreffe-2019-attention-is-not-not-explanation/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 11–20, Hong Kong, China. Association for…
44. Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2025a. [Longmemeval: Benchmarking chat assistants on long-term interactive memory](https://md.research.elamur.ai/free/en/papers/wu-2024-longmemeval-benchmarking-chat-assistants-long/). Preprint, arXiv:2410.10813.
45. Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. 2025b. [Retrieval head mechanistically explains long-context factuality](https://md.research.elamur.ai/free/en/papers/wu-2024-retrieval-head-mechanistically-explains-long/). In The Thirteenth International Conference on Learning Representations.
46. Kayo Yin and Jacob Steinhardt. 2025. Which attention heads matter for in-context learning? In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Kayo+Yin+and+Jacob+Steinhardt%2E+2025%2E+Which+attention+heads+matter+for+in%2Dcontext+learning%3F+In+Forty%2Dsecond+International+Conference+on+Machine+Learning%2E)
47. Jinghan Zhang, Fengran Mo, Xiting Wang, and Kunpeng Liu. 2024. Blind spot navigation in llm reasoning with thought space explorer. arXiv preprint arXiv:2410.24155. — [arXiv:2410.24155](https://arxiv.org/abs/2410.24155) [find](https://scholar.google.com/scholar?q=Jinghan+Zhang%2C+Fengran+Mo%2C+Xiting+Wang%2C+and+Kunpeng+Liu%2E+2024%2E+Blind+spot+navigation+in+llm+reasoning+with+thought+space+explorer%2E+arXiv+preprint+arXiv%3A2410%2E24155%2E+%E2%80%94+%5BarXiv%3A2410%2E24155%5D%28https%3A%2F%2Farxiv%2Eo)
48. Jinghan Zhang, Xiting Wang, Fengran Mo, Yeyang Zhou, Wanfu Gao, and Kunpeng Liu. 2025a. Entropy-based exploration conduction for multi-step reasoning. arXiv preprint arXiv:2503.15848. [find](https://scholar.google.com/scholar?q=Jinghan+Zhang%2C+Xiting+Wang%2C+Fengran+Mo%2C+Yeyang+Zhou%2C+Wanfu+Gao%2C+and+Kunpeng+Liu%2E+2025a%2E+Entropy%2Dbased+exploration+conduction+for+multi%2Dstep+reasoning%2E+arXiv+preprint+arXiv%3A2503%2E15848%2E)
49. Jinghan Zhang, Xiting Wang, Weijieying Ren, Lu Jiang, Dongjie Wang, and Kunpeng Liu. 2025b. Ratt: A thought structure for coherent and correct llm reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 26733–26741. [find](https://scholar.google.com/scholar?q=Jinghan+Zhang%2C+Xiting+Wang%2C+Weijieying+Ren%2C+Lu+Jiang%2C+Dongjie+Wang%2C+and+Kunpeng+Liu%2E+2025b%2E+Ratt%3A+A+thought+structure+for+coherent+and+correct+llm+reasoning%2E+In+Proceedings+of+the+AAAI+Conference+on+A)
50. Wuwei Zhang, Fangcong Yin, Howard Yen, Danqi Chen, and Xi Ye. 2025c. Query-focused retrieval heads improve long-context reasoning and re-ranking. arXiv preprint arXiv:2506.09944. [find](https://scholar.google.com/scholar?q=Wuwei+Zhang%2C+Fangcong+Yin%2C+Howard+Yen%2C+Danqi+Chen%2C+and+Xi+Ye%2E+2025c%2E+Query%2Dfocused+retrieval+heads+improve+long%2Dcontext+reasoning+and+re%2Dranking%2E+arXiv+preprint+arXiv%3A2506%2E09944%2E)
51. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. [Calibrate before use: Improving few-shot performance of language models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/). In International conference on machine learning, pages 12697–12706. PMLR.
52. Youxiang Zhu, Ruochen Li, Danqing Wang, Daniel Haehn, and Xiaohui Liang. 2025. Focus directions make your language models pay more attention to relevant contexts. arXiv preprint arXiv:2503.23306. — [arXiv:2503.23306](https://arxiv.org/abs/2503.23306) [find](https://scholar.google.com/scholar?q=Youxiang+Zhu%2C+Ruochen+Li%2C+Danqing+Wang%2C+Daniel+Haehn%2C+and+Xiaohui+Liang%2E+2025%2E+Focus+directions+make+your+language+models+pay+more+attention+to+relevant+contexts%2E+arXiv+preprint+arXiv%3A2503%2E23306%2E+%E2%80%94+%5Ba)

*Updated 2 November 2026*