--- title: "When Retrieval Hurts — paper by Mohan et al. (2026)" description: "The authors also propose lightweight metrics for checking whether answers use the correct writing system. 1 reference, cited by 1 paper here." authors: ["Varalekshmy M Mohan", "Swathi Jayakumar", "Gadha Saji Menon", "Sachin Kurup", "Veena G", "Vani Kanjirangat"] year: 2026 venue: "Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026)" source: "https://aclanthology.org/2026.mellm-1.9/" doi: "10.18653/v1/2026.mellm-1.9" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/mohan-2026-when-retrieval-hurts-evidence-utilization/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mohan-2026-when-retrieval-hurts-evidence-utilization/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/) / When Retrieval Hurts: Evidence Utilization, Script Fidelity, and Knowledge Conflicts in Multilingual RAG # When Retrieval Hurts: Evidence Utilization, Script Fidelity, and Knowledge Conflicts in Multilingual RAG **Authors:** Varalekshmy M Mohan, Swathi Jayakumar, Gadha Saji Menon, Sachin Kurup, Veena G and [Vani Kanjirangat](https://md.research.elamur.ai/free/en/authors/vani-kanjirangat/) · **Year:** 2026 · **Venue:** [Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026)](https://md.research.elamur.ai/free/en/venues/mellm/) ([Workshop on Multilinguality in the Era of Large Language Models 2026](https://md.research.elamur.ai/free/en/venues/mellm/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.mellm-1.9/) · [PDF](https://aclanthology.org/2026.mellm-1.9.pdf) · [DOI 10.18653/v1/2026.mellm-1.9](https://doi.org/10.18653/v1/2026.mellm-1.9) · [OpenAlex](https://openalex.org/W7166811013) **Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper asks why retrieval-augmented generation (RAG) often fails to improve, and sometimes worsens, extractive question answering across many languages. The authors compare RAG with a non-RAG baseline over 21 typologically diverse languages in 9 writing scripts, using TyDiQA and XQuAD. They test five instruction-tuned LLMs (Gemma-2-9B-IT, Qwen2.5-7B-Instruct, LLaMA-2-7B-Chat, Aya-23-8B, EuroLLM-9B-Instruct) with five prompting strategies. The retrieval pipeline uses BGE-M3 dense retrieval with FAISS and sentence-aware chunking tuned to each language. The central finding is that retrieval is rarely the weak link. Relevant evidence is almost always retrieved, yet three of the five models score lower with RAG, because they fail to make use of what they receive. Models with stronger built-in knowledge suffer most from this interference. The authors also propose lightweight metrics for checking whether answers use the correct writing system. These metrics expose script errors that EM and F1 miss. The work suggests that multilingual RAG needs better grounding on the generator side and script-aware handling of conflicts, not just better retrievers. ## Key points - **Data:** 3,930 evaluation samples in total. - TyDiQA contributes 1,730 samples across 9 languages. Its secondary task (gold passage supplied) serves as No-RAG, and its primary task (full-corpus retrieval) serves as RAG. - XQuAD contributes 2,200 samples, evaluated without retrieval. - **Evidence-utilization gap:** the retrieval hit rate (RHR) is 0.927, yet RAG lowers performance for three of five models. Gemma and Qwen are the most affected. - **New script-fidelity diagnostics:** the authors introduce three measures, namely Transfer Rate (TR), Script Consistency (SC) and Language Match Rate (LMR). - Qwen produced Latin characters in Russian answers (LMR 0.565). Adding a Cyrillic-enforcing clause to the prompt raised LMR to 0.984 without retraining. - Korean Latin leakage (LMR 0.148) was also detected. - **Language-specific failures:** - Telugu is the hardest language under RAG (F1 0.162–0.357), which the authors attribute to abugida tokenization fragmenting retrieved chunks. - In Arabic, negated facts are frequently turned into affirmatives (conflict rate about 0.50–0.60). - For Indonesian, minimal prompting reduced hallucination. - **XQuAD without retrieval:** Aya-23-8B performs best (macro F1 0.757). This suggests that internal multilingual knowledge and the ability to exploit retrieved context are separate skills. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Amrita Vishwa Vidyapeetham](https://md.research.elamur.ai/free/en/universities/amrita-vishwa-vidyapeetham/) — University, Coimbatore, India · 7 works · 31 citations - [Dalle Molle Institute for Artificial Intelligence Research](https://md.research.elamur.ai/free/en/labs/dalle-molle-institute-for-artificial-intelligence-research/) — Research institute, Lugano, Switzerland · 5 works · 18 citations - [University of Applied Sciences and Arts of Southern Switzerland](https://md.research.elamur.ai/free/en/universities/university-of-applied-sciences-and-arts-of-southern-switzerland/) — University, Manno, Switzerland · 4 works · 17 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works 🧭 Topics (2): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works ## Cite Cite as: Varalekshmy M Mohan, Swathi Jayakumar, Gadha Saji Menon, Sachin Kurup, Veena G and Vani Kanjirangat. 2026. When Retrieval Hurts: Evidence Utilization, Script Fidelity, and Knowledge Conflicts in Multilingual RAG. Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{mohan2026when, title = {{When Retrieval Hurts: Evidence Utilization, Script Fidelity, and Knowledge Conflicts in Multilingual RAG}}, author = {Varalekshmy M Mohan, Swathi Jayakumar, Gadha Saji Menon, Sachin Kurup, Veena G and Vani Kanjirangat}, booktitle = {Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026)}, year = {2026}, doi = {10.18653/v1/2026.mellm-1.9}, url = {https://doi.org/10.18653/v1/2026.mellm-1.9} } ``` - [← previous When Retrieval Doesn't Help: A Large-Scale Study of Biomedical RAG](https://md.research.elamur.ai/free/en/papers/nourbakhsh-2026-when-retrieval-doesn-t-help/) - [next → When Tasks Share Structure: A Comparative Study of Training Strategies for Generative Event Extraction](https://md.research.elamur.ai/free/en/papers/ravikumar-2026-when-tasks-share-structure-comparative/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/) ## References 1 references, 1 of them with a card of ours. ↗ leads to the work. 1. Kabir Ahuja, Harshita Diddee, Rishav Hada, and 1 others. 2023. [MEGA: Multilingual evaluation of generative AI](https://md.research.elamur.ai/free/en/papers/ahuja-2023-mega-multilingual-evaluation-generative-ai/). In Proceedings of EMNLP. *Updated 1 November 2026*