---
title: "Evaluating the Retrieval Component in a — Naguib 2026"
description: "The paper addresses how to condense lengthy French patient records into structured summaries for emergency and intensive care clinicians, focusing on the…"
authors: ["Marco Naguib", "Christel Gérardin", "Victor Beaucoté", "Cyril Charron", "Adrien Joseph", "Aurélie Névéol", "Xavier Tannier"]
year: 2026
venue: "Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026"
source: "https://aclanthology.org/2026.clinicalnlp-1.7/"
doi: "10.63317/4cy8xxinjw7z"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/naguib-2026-evaluating-retrieval-component-retrieval-augmented/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/naguib-2026-evaluating-retrieval-component-retrieval-augmented/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) / Evaluating the Retrieval Component in a Retrieval-Augmented Summarization System for Patient Records in French

# Evaluating the Retrieval Component in a Retrieval-Augmented Summarization System for Patient Records in French

**Authors:** Marco Naguib, Christel Gérardin, Victor Beaucoté, Cyril Charron, Adrien Joseph, [Aurélie Névéol](https://md.research.elamur.ai/free/en/authors/aurelie-neveol/), [Xavier Tannier](https://md.research.elamur.ai/free/en/authors/xavier-tannier/) · **Year:** 2026 · **Venue:** Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026 · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.clinicalnlp-1.7/) · [PDF](https://aclanthology.org/2026.clinicalnlp-1.7.pdf) · [DOI 10.63317/4cy8xxinjw7z](https://doi.org/10.63317/4cy8xxinjw7z) · [OpenAlex](https://openalex.org/W7166552659)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
The paper addresses how to condense lengthy French patient records into structured summaries for emergency and intensive care clinicians, focusing on the retrieval stage of a Retrieval-Augmented Generation (RAG) pipeline. Generation is left for future work. The authors use real records from the AP–HP hospital. Critical care physicians marked clinically important spans under six section labels on a subset of patients with a typical number of documents. Records are split into fixed-size chunks. Sparse retrievers (TF-IDF, BM25) and dense sentence-embedding models are then compared under seven query formulations. Five are hand-written text queries. The other two are built directly from annotations, either by averaging encoded annotated spans or by averaging annotated chunks into a single prototype query vector. The annotation-derived queries consistently beat prompt-style queries, with especially large gains for lightweight TF-IDF retrieval. Preliminary full-pipeline tests suggested that passing whole records to a long-context LLM yields verbose, poorly organized summaries, so targeted retrieval still matters. The work provides the first systematic benchmark of retrieval and embedding models for French clinical text.

## Key points
- **Data:** The source corpus has 10,000 AP–HP clinical documents. Patients with the median count of 61 documents were chosen, giving 11 patients and 671 annotated documents. Three of these patients were used for inter-annotator agreement, and eight were annotated by a single physician. Four critical care physicians did the annotation.
- **Six labels:** Lifestyle, Medical history, Surgical history, Treatments, Allergies, and Biometrics.
- **Query strategies:** There are seven in total.
  - Five textual queries: Specification-Based Prompts, Examples, Keywords, Contextual Prompts, and Structure Prompts.
  - Two data-driven queries: Query-from-Annotations and Query-from-Annotation-Chunks.
- **Main finding:** Annotation-based querying outperforms traditional prompting, with notable benefits for TF-IDF.
- **Limitations and code:** The approach depends on costly expert annotations, and the dataset is small. The code is available at github.com/marconaguib/rag.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) · [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) · [Life sciences & medicine](https://md.research.elamur.ai/free/en/shelves/life-sciences/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{naguib2026evaluating,
  title = {{Evaluating the Retrieval Component in a Retrieval-Augmented Summarization System for Patient Records in French}},
  author = {Naguib, Marco and Gérardin, Christel and Beaucoté, Victor and Charron, Cyril and Joseph, Adrien and Névéol, Aurélie and Tannier, Xavier},
  booktitle = {Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026},
  year = {2026},
  doi = {10.63317/4cy8xxinjw7z},
  url = {https://doi.org/10.63317/4cy8xxinjw7z}
}
```

- [← previous Evaluating the Impact of Source Diversity for RAG in Historical Research](https://md.research.elamur.ai/free/en/papers/mahadeshwar-2026-evaluating-impact-source-diversity-rag/)
- [next → Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?](https://md.research.elamur.ai/free/en/papers/rajput-2026-evaluation-drift-llm-personality-induction/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/)

## References

20 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mohammad Alkhalaf, Ping Yu, Mengyang Yin, and Chao Deng. 2024. Applying generative AI with retrieval augmented generation to summarize and extract key clinical information from electronic health records. Journal of Biomedical Informatics, 156:104662. [↗](https://doi.org/10.1016/j.jbi.2024.104662)
2. Wojciech Kryściński, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the Factual Consistency of Abstractive Text Summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 9332–9346. [find](https://scholar.google.com/scholar?q=Wojciech+Kry%C5%9Bci%C5%84ski%2C+Bryan+McCann%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+2020%2E+Evaluating+the+Factual+Consistency+of+Abstractive+Text+Summarization%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Met)
3. Tianchi Cai, Zhiwen Tan, Xierui Song, Tao Sun, Jiyan Jiang, Yunqi Xu, Yinger Zhang, and Jinjie Gu. 2024. FoRAG: Factuality-optimized Retrieval Augmented Generation for Web-enhanced Long-form Question Answering. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’24, page 199–210, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Tianchi+Cai%2C+Zhiwen+Tan%2C+Xierui+Song%2C+Tao+Sun%2C+Jiyan+Jiang%2C+Yunqi+Xu%2C+Yinger+Zhang%2C+and+Jinjie+Gu%2E+2024%2E+FoRAG%3A+Factuality%2Doptimized+Retrieval+Augmented+Generation+for+Web%2Denhanced+Long%2Dform+Question+)
4. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Processing Systems, volume 33, pages 9459–9474. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Patrick+Lewis%2C+Ethan+Perez%2C+Aleksandra+Piktus%2C+Fabio+Petroni%2C+Vladimir+Karpukhin%2C+Naman+Goyal%2C+Heinrich+K%C3%BCttler%2C+Mike+Lewis%2C+Wen%2Dtau+Yih%2C+Tim+Rockt%C3%A4schel%2C+Sebastian+Riedel%2C+and+Douwe+Kiela%2E+2020%2E+Retr)
5. Mathieu Ciancone, Imene Kerboua, Marion Schaeffer, and Wissam Siblini. 2024. MTEB-French: Resources for French Sentence Embedding Evaluation and Analysis. arXiv preprint arXiv:2405.20468. [↗](https://arxiv.org/abs/2405.20468)
6. Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 74–81. [find](https://scholar.google.com/scholar?q=Chin%2DYew+Lin%2E+2004%2E+ROUGE%3A+A+Package+for+Automatic+Evaluation+of+Summaries%2E+In+Text+Summarization+Branches+Out%3A+Proceedings+of+the+ACL%2D04+Workshop%2C+pages+74%E2%80%9381%2E)
7. Shengjie Liu, Jing Wu, Jingyuan Bao, Wenyi Wang, Naira Hovakimyan, and Christopher G Healey. 2024. Towards a Robust Retrieval-Based Summarization System. [↗](https://doi.org/10.48550/arxiv.2403.19889)
8. Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2025. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. [find](https://scholar.google.com/scholar?q=Darren+Edge%2C+Ha+Trinh%2C+Newman+Cheng%2C+Joshua+Bradley%2C+Alex+Chao%2C+Apurva+Mody%2C+Steven+Truitt%2C+Dasha+Metropolitansky%2C+Robert+Osazuwa+Ness%2C+and+Jonathan+Larson%2E+2025%2E+From+Local+to+Global%3A+A+Graph+RAG+App)
9. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On Faithfulness and Factuality in Abstractive Summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919. [find](https://scholar.google.com/scholar?q=Joshua+Maynez%2C+Shashi+Narayan%2C+Bernd+Bohnet%2C+and+Ryan+McDonald%2E+2020%2E+On+Faithfulness+and+Factuality+in+Abstractive+Summarization%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Comp)
10. Tanguy Herserant and Vincent Guigue. 2025. Seval-ex : Un paradigme basé sur les phrases atomiques pour une évaluation explicable de la qualité des résumés. In Actes de la 20e Conférence en Recherche d’Information et Applications (CORIA), pages 217–229, Marseille, France. ATALA \\& ARIA. [find](https://scholar.google.com/scholar?q=Tanguy+Herserant+and+Vincent+Guigue%2E+2025%2E+Seval%2Dex+%3A+Un+paradigme+bas%C3%A9+sur+les+phrases+atomiques+pour+une+%C3%A9valuation+explicable+de+la+qualit%C3%A9+des+r%C3%A9sum%C3%A9s%2E+In+Actes+de+la+20e+Conf%C3%A9rence+en+Recherche+d)
11. Rashmi Mishra, Jiantao Bian, Marcelo Fiszman, Charlene R. Weir, Siddhartha Jonnalagadda, Javed Mostafa, and Guilherme Del Fiol. 2014. Text summarization in the biomedical domain: A systematic review of recent research. Journal of Biomedical Informatics, 52:457–467. Special Section: Methods in Clinical Research Informatics. [↗](https://doi.org/10.1016/j.jbi.2014.06.009)
12. Yuelyu Ji, Zhuochun Li, Rui Meng, Sonish Sivarajkumar, Yanshan Wang, Zeshui Yu, Hui Ji, Yushui Han, Hanyu Zeng, and Daqing He. 2024. RAG-RLRC-LaySum at BioLaySumm: Integrating Retrieval-Augmented Generation and Readability Control for Layman Summarization of Biomedical Texts. [↗](https://doi.org/10.48550/arxiv.2405.13179)
13. Hans Moen, Laura-Maria Peltonen, Juho Heimonen, Antti Airola, Tapio Pahikkala, Tapio Salakoski, and Sanna Salanterä. 2016. Comparison of automatic summarisation methods for clinical free text notes. Artificial Intelligence in Medicine, 67:25–37. [find](https://scholar.google.com/scholar?q=Hans+Moen%2C+Laura%2DMaria+Peltonen%2C+Juho+Heimonen%2C+Antti+Airola%2C+Tapio+Pahikkala%2C+Tapio+Salakoski%2C+and+Sanna+Salanter%C3%A4%2E+2016%2E+Comparison+of+automatic+summarisation+methods+for+clinical+free+text+notes%2E+A)
14. Vasileios Katranidis and Gabor Barany. 2024. FaaF: facts as a function for the evaluation of RAG systems. arXiv e-prints, pages arXiv–2403. [find](https://scholar.google.com/scholar?q=Vasileios+Katranidis+and+Gabor+Barany%2E+2024%2E+FaaF%3A+facts+as+a+function+for+the+evaluation+of+RAG+systems%2E+arXiv+e%2Dprints%2C+pages+arXiv%E2%80%932403%2E)
15. Daniel Keszthelyi, Christophe Gaudet-Blavignac, Mina Bjelogrlic, Christian Lovis, et al. 2023. Patient information summarization in clinical settings: scoping review. JMIR Medical Informatics, 11(1):e44639. [↗](https://doi.org/10.2196/44639)
16. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. [↗](https://doi.org/10.3115/1073083.1073135) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
17. Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Rimma Pivovarov and Noémie Elhadad. 2015. Automated methods for the summarization of electronic health records. Journal of the American Medical Informatics Association, 22(5):938–947. [find](https://scholar.google.com/scholar?q=Satyapriya+Krishna%2C+Kalpesh+Krishna%2C+Anhad+Mohananey%2C+Steven+Schwarcz%2C+Adam+Stambler%2C+Rimma+Pivovarov+and+No%C3%A9mie+Elhadad%2E+2015%2E+Automated+methods+for+the+summarization+of+electronic+health+records%2E+Jo)
18. Rishabh Uapadhyay and Marco Viviani. 2025. Enhancing Health Information Retrieval with RAG by Prioritizing Topical Relevance and Factual Accuracy. arXiv preprint arXiv:2502.04666. [↗](https://arxiv.org/abs/2502.04666)
19. Yuhao Zhang, Qian Chi, and Xibin Xia. 2018. Learning to Summarize Radiology Findings. In Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis, pages 204–213. [find](https://scholar.google.com/scholar?q=Yuhao+Zhang%2C+Qian+Chi%2C+and+Xibin+Xia%2E+2018%2E+Learning+to+Summarize+Radiology+Findings%2E+In+Proceedings+of+the+Ninth+International+Workshop+on+Health+Text+Mining+and+Information+Analysis%2C+pages+204%E2%80%93213%2E)
20. Yuhao Zhang, Derek Merck, Emily Tsai, Christopher D. Manning, and Curtis Langlotz. 2020. Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5108–5120, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yuhao+Zhang%2C+Derek+Merck%2C+Emily+Tsai%2C+Christopher+D%2E+Manning%2C+and+Curtis+Langlotz%2E+2020%2E+Optimizing+the+Factual+Correctness+of+a+Summary%3A+A+Study+of+Summarizing+Radiology+Reports%2E+In+Proceedings+of+th) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2020-optimizing-factual-correctness-summary-study/)