---
title: "The Distracting Effect: Understanding Irrelevant Passages in RAG"
description: "The work offers a practical way to build harder RAG test sets and more robust… 7.6 points on WebQA. 32 references, cited by 13 papers here."
authors: ["Chen Amiraz", "Florin Cuconasu", "Simone Filice", "Zohar Karnin"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.892/"
doi: "10.18653/v1/2025.acl-long.892"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/amiraz-2025-distracting-effect-understanding-irrelevant-passages/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/amiraz-2025-distracting-effect-understanding-irrelevant-passages/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/) / The Distracting Effect: Understanding Irrelevant Passages in RAG

# The Distracting Effect: Understanding Irrelevant Passages in RAG

**Authors:** Chen Amiraz, [Florin Cuconasu](https://md.research.elamur.ai/free/en/authors/florin-cuconasu/), [Simone Filice](https://md.research.elamur.ai/free/en/authors/simone-filice/) and Zohar Karnin · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.892/) · [PDF](https://aclanthology.org/2025.acl-long.892.pdf) · [DOI 10.18653/v1/2025.acl-long.892](https://doi.org/10.18653/v1/2025.acl-long.892) · [OpenAlex](https://openalex.org/W4412945011)

**Citations:** 8 citations (OpenAlex, 2026-10-14) · [13](https://md.research.elamur.ai/free/en/papers/amiraz-2025-distracting-effect-understanding-irrelevant-passages/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This paper examines a familiar weakness of Retrieval Augmented Generation: passages that do not answer a question but are topically close to it can lead the generating LLM to a wrong answer. Instead of labelling irrelevant passages simply as random or distracting, the authors treat distraction as a graded property. They measure it by asking the model to answer from a single passage or output "NO-RESPONSE", and define the distracting effect as one minus the probability of that abstention. The resulting scores correlate well across different LLMs, and passages with higher scores reduce accuracy more when added next to the gold passage. To find hard distractors beyond what ordinary retrieval returns, the authors combine standard retrieval, a skewed retriever, and LLM-generated passages from several categories. Fine-tuning on these harder distractors improves question-answering accuracy over training on conventional retrieval-based RAG data, with the abstract reporting gains of up to 7.5%. The work offers a practical way to build harder RAG test sets and more robust training data.

## Key points
- **Distracting effect:** DE_q(p) = 1 − p_LLM(NO-RESPONSE | q, p). It needs only one forward pass, with no generated tokens and no reference model. It is used to rank passages for the same query relative to each other.
- **Models evaluated:** Llama-3.2-3B, Llama-3.1-8B, Llama-3.3-70B, Falcon-3-3B, Falcon-3-7B, Qwen-2.5-3B and Qwen-2.5-7B. The distraction scores agree closely across these models.
- **Effect on accuracy:** Irrelevant passages from stronger retrievers, and those ranked higher, tend to be more distracting. Adding a distractor alongside the gold passage lowers accuracy by about 6–11 points.
- **Fine-tuning results:** For Llama-3.2-3B, training on hard distractors gives lifts of 6.7 points on TriviaQA and 7.6 points on WebQA over the baselines. For Llama-3.1-8B, it wins on 3 of 4 benchmarks, with most of the gain on ungrounded examples (where the gold passage is absent).
- **Limitations:** The study covers only English benchmarks and the question-answering task. The taxonomy of generated distractor types (e.g., G_modal, G_rel) may not be complete.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Sapienza University of Rome](https://md.research.elamur.ai/free/en/universities/sapienza-university-of-rome/) — University, Rome, Italy · 57 works · 9,564 citations
- [Technology Innovation Institute](https://md.research.elamur.ai/free/en/labs/technology-innovation-institute/) — Research institute, Abu Dhabi, United Arab Emirates · 7 works · 543 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

## Cite

Cite as: Chen Amiraz, Florin Cuconasu, Simone Filice and Zohar Karnin. 2025. The Distracting Effect: Understanding Irrelevant Passages in RAG. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{amiraz2025distracting,
  title = {{The Distracting Effect: Understanding Irrelevant Passages in RAG}},
  author = {Chen Amiraz, Florin Cuconasu, Simone Filice and Zohar Karnin},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.892},
  url = {https://doi.org/10.18653/v1/2025.acl-long.892}
}
```

- [← previous Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model](https://md.research.elamur.ai/free/en/papers/ma-2025-step-video-t2v-technical/)
- [next → Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement](https://md.research.elamur.ai/free/en/papers/kong-2025-time-mqa-time-series-multi/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/)

## References

32 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rustam Abdumalikov, Pasquale Minervini, and Yova Kementchedjhieva. 2024. Answerability in retrieval-augmented open-domain question answering. arXiv preprint arXiv:2403.01461. — [arXiv:2403.01461](https://arxiv.org/abs/2403.01461) [find](https://scholar.google.com/scholar?q=Rustam+Abdumalikov%2C+Pasquale+Minervini%2C+and+Yova+Kementchedjhieva%2E+2024%2E+Answerability+in+retrieval%2Daugmented+open%2Ddomain+question+answering%2E+arXiv+preprint+arXiv%3A2403%2E01461%2E+%E2%80%94+%5BarXiv%3A2403%2E01461%5D%28http)
2. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. [Self-RAG: Learning to retrieve, generate, and critique through self-reflection](https://md.research.elamur.ai/free/en/papers/asai-2023-self-rag-learning-retrieve-generate/). In The Twelfth International Conference on Learning Representations.
3. Victoria Basmov, Yoav Goldberg, and Reut Tsarfaty. 2024. LLMs’ reading comprehension is affected by parametric knowledge and struggles with hypothetical statements. arXiv preprint arXiv:2404.06283. — [arXiv:2404.06283](https://arxiv.org/abs/2404.06283) [find](https://scholar.google.com/scholar?q=Victoria+Basmov%2C+Yoav+Goldberg%2C+and+Reut+Tsarfaty%2E+2024%2E+LLMs%E2%80%99+reading+comprehension+is+affected+by+parametric+knowledge+and+struggles+with+hypothetical+statements%2E+arXiv+preprint+arXiv%3A2404%2E06283%2E+%E2%80%94+)
4. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. [Semantic parsing on Freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533–1544, Seattle, Washington, USA. Association for Computational Linguistics.
5. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. [Reading wikipedia to answer open-domain questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870–1879.
6. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. [BGE M3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). Preprint, arXiv:2402.03216.
7. Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024. [The power of noise: Redefining retrieval for RAG systems](https://md.research.elamur.ai/free/en/papers/cuconasu-2024-power-noise-redefining-retrieval-rag/). In Proceedings of the 47th International ACM SIGIR Conference on Research and Developm…
8. Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. [A survey on RAG meeting LLMs: Towards retrieval-augmented large language models](https://md.research.elamur.ai/free/en/papers/fan-2024-survey-rag-meeting-llms-towards/). In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6491–6501.
9. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. [Retrieval-augmented generation for large language models: A survey](https://md.research.elamur.ai/free/en/papers/gao-2023-retrieval-augmented-generation-large-language/). arXiv preprint arXiv:2312.10997.
10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
11. Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. [True: Re-evaluating factual consistency evaluation](https://md.research.elamur.ai/free/en/papers/honovich-2022-true-re-evaluating-factual-consistency/). In Proceedings of the 2022 Conference of the North American Chapter of the Associa…
12. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). arXiv preprint arXiv:2106.09685.
13. Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan O Arik. 2024. Long-context LLMs meet RAG: Overcoming challenges for long inputs in RAG. arXiv preprint arXiv:2410.05983. — [arXiv:2410.05983](https://arxiv.org/abs/2410.05983) [find](https://scholar.google.com/scholar?q=Bowen+Jin%2C+Jinsung+Yoon%2C+Jiawei+Han%2C+and+Sercan+O+Arik%2E+2024%2E+Long%2Dcontext+LLMs+meet+RAG%3A+Overcoming+challenges+for+long+inputs+in+RAG%2E+arXiv+preprint+arXiv%3A2410%2E05983%2E+%E2%80%94+%5BarXiv%3A2410%2E05983%5D%28https%3A%2F%2Far)
14. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611.
15. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, and 1 others. 2019. [Natural questions: a benchmark for question answering research](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/). Transactions of the Association for Computational Lin…
16. Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, and Sanjiv Kumar. 2023. [Large language models with controllable working memory](https://md.research.elamur.ai/free/en/papers/li-2023-large-language-models-controllable-working/). In Findings of the Association for Computational Linguistics: ACL 2023, pages 1774–1793.
17. Xi Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Richard James, Pedro Rodriguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis, Luke Zettlemoyer, and Scott Yih. 2024. [RA-DIT: Retrieval-augmented dual instruction tuning](https://md.research.elamur.ai/free/en/papers/lin-2023-ra-dit-retrieval-augmented-dual/). In The Twelfth International Conference on Learning Representation…
18. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. [Lost in the middle: How language models use long contexts](https://md.research.elamur.ai/free/en/papers/liu-2023-lost-middle-language-models-use/). Preprint, arXiv:2307.03172.
19. Hongyin Luo, Yung-Sung Chuang, Yuan Gong, Tianhua Zhang, Yoon Kim, Xixin Wu, Danny Fox, Helen Meng, and James Glass. 2023. Sail: Search-augmented instruction learning. arXiv preprint arXiv:2305.15225. [find](https://scholar.google.com/scholar?q=Hongyin+Luo%2C+Yung%2DSung+Chuang%2C+Yuan+Gong%2C+Tianhua+Zhang%2C+Yoon+Kim%2C+Xixin+Wu%2C+Danny+Fox%2C+Helen+Meng%2C+and+James+Glass%2E+2023%2E+Sail%3A+Search%2Daugmented+instruction+learning%2E+arXiv+preprint+arXiv%3A2305%2E15225%2E)
20. Alex Troy Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In The 61st Annual Meeting Of The Association For Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alex+Troy+Mallen%2C+Akari+Asai%2C+Victor+Zhong%2C+Rajarshi+Das%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametri)
21. Gabriel de Souza P Moreira, Radek Osmulski, Mengyao Xu, Ronay Ak, Benedikt Schifferer, and Even Oldridge. 2024. NV-Retriever: Improving text embedding models with effective hard-negative mining. arXiv preprint arXiv:2407.15831. — [arXiv:2407.15831](https://arxiv.org/abs/2407.15831) [find](https://scholar.google.com/scholar?q=Gabriel+de+Souza+P+Moreira%2C+Radek+Osmulski%2C+Mengyao+Xu%2C+Ronay+Ak%2C+Benedikt+Schifferer%2C+and+Even+Oldridge%2E+2024%2E+NV%2DRetriever%3A+Improving+text+embedding+models+with+effective+hard%2Dnegative+mining%2E+arXiv)
22. Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. [KILT: a benchmark for knowledge intensive language tasks](https://md.research.elamur.ai/free/en/papers/petroni-2020-kilt-benchmark-knowledge-intensive-language/). In Proceedings of th…
23. Falcon-LLM Team. 2024. The falcon 3 family of open models. [find](https://scholar.google.com/scholar?q=Falcon%2DLLM+Team%2E+2024%2E+The+falcon+3+family+of+open+models%2E)
24. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. [Text embeddings by weakly-supervised contrastive pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/). arXiv preprint arXiv:2212.03533.
25. Zhepei Wei, Wei-Lin Chen, and Yu Meng. 2024. Instructrag: Instructing retrieval augmented generation via self-synthesized rationales. In Adaptive Foundation Models: Evolving AI for Personalized and Efficient Learning. [find](https://scholar.google.com/scholar?q=Zhepei+Wei%2C+Wei%2DLin+Chen%2C+and+Yu+Meng%2E+2024%2E+Instructrag%3A+Instructing+retrieval+augmented+generation+via+self%2Dsynthesized+rationales%2E+In+Adaptive+Foundation+Models%3A+Evolving+AI+for+Personalized+and+Ef)
26. Orion Weller, Benjamin Van Durme, Dawn Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel. 2024. Promptriever: Instruction-trained retrievers can be prompted like language models. arXiv preprint arXiv:2409.11136. — [arXiv:2409.11136](https://arxiv.org/abs/2409.11136) [find](https://scholar.google.com/scholar?q=Orion+Weller%2C+Benjamin+Van+Durme%2C+Dawn+Lawrie%2C+Ashwin+Paranjape%2C+Yuhao+Zhang%2C+and+Jack+Hessel%2E+2024%2E+Promptriever%3A+Instruction%2Dtrained+retrievers+can+be+prompted+like+language+models%2E+arXiv+preprint+a)
27. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. [Transf…](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
28. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. [Approximate nearest neighbor negative contrastive learning for dense text retrieval](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/). In International Conference on Learning Representations.
29. Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. 2024. [Corrective retrieval augmented generation](https://md.research.elamur.ai/free/en/papers/yan-2024-corrective-retrieval-augmented-generation/). arXiv preprint arXiv:2401.15884.
30. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 23 others. 2025. Qwen2.5 technical report. Preprint, arXiv:241… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+Yang%2C+Jin)
31. Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2024. [Making retrieval-augmented language models robust to irrelevant context](https://md.research.elamur.ai/free/en/papers/yoran-2023-making-retrieval-augmented-language-models/). In The Twelfth International Conference on Learning Representations.
32. Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, and Dong Yu. 2023. [Chain-of-note: Enhancing robustness in retrieval-augmented language models](https://md.research.elamur.ai/free/en/papers/yu-2024-chain-note-enhancing-robustness-retrieval/). arXiv preprint arXiv:2311.09210.

*Updated 31 October 2026*