---
title: "MedEvi-NS at ArchEHR-QA 2026: Using Clinical Reasoning"
description: "The paper addresses subtask 4 of the ArchEHR-QA 2026 shared task. 77.4 F1. 15 references."
authors: ["Mengxuan Sun", "Nicolay Rusnachenko"]
year: 2026
venue: "Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026"
source: "https://aclanthology.org/2026.cl4health-1.44/"
doi: "10.63317/2uoe94adhkng"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sun-2026-medevi-ns-archehr-qa-2026/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sun-2026-medevi-ns-archehr-qa-2026/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/36/) / MedEvi-NS at ArchEHR-QA 2026: Using Clinical Reasoning Principles to Improve Zero-shot Capabilities of Large Language Models in Evidence Alignment

# MedEvi-NS at ArchEHR-QA 2026: Using Clinical Reasoning Principles to Improve Zero-shot Capabilities of Large Language Models in Evidence Alignment

**Authors:** Mengxuan Sun and [Nicolay Rusnachenko](https://md.research.elamur.ai/free/en/authors/nicolay-rusnachenko/) · **Year:** 2026 · **Venue:** [Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026](https://md.research.elamur.ai/free/en/venues/cl4health/) ([Workshop on Patient-Oriented Language Processing @ LREC 2026](https://md.research.elamur.ai/free/en/venues/cl4health/2026/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.cl4health-1.44/) · [PDF](https://aclanthology.org/2026.cl4health-1.44.pdf) · [DOI 10.63317/2uoe94adhkng](https://doi.org/10.63317/2uoe94adhkng) · [OpenAlex](https://openalex.org/W7166528008)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses subtask 4 of the ArchEHR-QA 2026 shared task. Given a clinician's reading of a patient question, an excerpt from an EHR note and an answer text, a system must link every answer sentence to the note sentences that back it up. Similarity-based retrieval can fail here because clinical notes often describe several related events. A matching keyword may therefore point to the wrong episode or stage of treatment.

The authors use GPT-5.2 in zero-shot mode with a structured prompt that mimics clinical reasoning. The model first identifies the episode an answer sentence refers to and decides its stage of care (finding, decision, procedure or outcome). It then selects a minimal set of sentences that document completed events and checks that every fact is covered. The output is parsed with regular expressions and matched to answer sentences using Levenshtein distance.

On the development split, the reasoning principles raised F1 overall by about 2 points. The final test submission scored 77.4 F1 overall and ranked 10th of 16 teams. This suggests that domain-informed prompting can make LLM evidence grounding more faithful in clinical QA.

## Key points
- Data: the official ArchEHR-QA dataset v1.5, with 20 development cases and 147 test cases. Answers are about 5.32 times shorter (in sentences) than the note excerpts.
- Models: GPT-5.2 (2025-12-11) was the main model and LLaMA-3-70B-instruct served as a comparison, both at temperature 0.1. LLaMA-3-70B scored lower (67.95 F1 overall on dev) and did not benefit from the reasoning principles.
- On the dev split, GPT-5.2 reached 74.77 F1 overall without the reasoning principles and 76.24 with them, mainly through higher precision.
- On the test split, the system scored 77.41 F1 overall, 10th of 16 teams. It improved on team "boris123" by +15.8 F1 micro.
- On the dev split, F1 overall fell by roughly 10% as excerpts grew from 10 to 50 sentences. The code is released at github.com/nicolay-r/ArchEHR-QA-2026-Task-4-MedEvi-NS.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Aberdeen](https://md.research.elamur.ai/free/en/universities/university-of-aberdeen/) — University, Aberdeen, United Kingdom · 20 works · 1,385 citations
- [Bournemouth University](https://md.research.elamur.ai/free/en/universities/bournemouth-university/) — University, Poole, United Kingdom · 5 works · 276 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 241 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 184 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works

## Cite

Cite as: Mengxuan Sun and Nicolay Rusnachenko. 2026. MedEvi-NS at ArchEHR-QA 2026: Using Clinical Reasoning Principles to Improve Zero-shot Capabilities of Large Language Models in Evidence Alignment. Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sun2026medevi,
  title = {{MedEvi-NS at ArchEHR-QA 2026: Using Clinical Reasoning Principles to Improve Zero-shot Capabilities of Large Language Models in Evidence Alignment}},
  author = {Mengxuan Sun and Nicolay Rusnachenko},
  booktitle = {Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026},
  year = {2026},
  doi = {10.63317/2uoe94adhkng},
  url = {https://doi.org/10.63317/2uoe94adhkng}
}
```

- [← previous MedCPI: A Construct–Personalize–Integrate Framework for KG-enhanced Clinical Prediction](https://md.research.elamur.ai/free/en/papers/wang-2026-medcpi-construct-personalize-integrate-framework/)
- [next → Medical Context Variation: A source of impairment for Event classification](https://md.research.elamur.ai/free/en/papers/sinha-2026-medical-context-variation-source-impairment/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/36/)

## References

15 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Akanksha Dhamija and Arun Sharma. 2025. Prompt-enhanced question answering for clinical texts using instruction-tuned language models. In 2025 1st IEEE Uttar Pradesh Section Women in Engineering International Conference on Electrical Electronics and Computer Engineering (UPWIECON), pages 308–312. IE… [find](https://scholar.google.com/scholar?q=Akanksha+Dhamija+and+Arun+Sharma%2E+2025%2E+Prompt%2Denhanced+question+answering+for+clinical+texts+using+instruction%2Dtuned+language+models%2E+In+2025+1st+IEEE+Uttar+Pradesh+Section+Women+in+Engineering+Inter)
2. Shashi Kant Gupta, Aditya Basu, Mauro Nievas, Jerrin Thomas, Nathan Wolfrath, Adhitya Ramamurthi, Bradley Taylor, Anai N Kothari, Regina Schwind, Therica M Miller, et al. 2024. Prism: Patient records interpretation for semantic clinical trial matching using large language models. arXiv preprint arXi… — [arXiv:2404.15549](https://arxiv.org/abs/2404.15549) [find](https://scholar.google.com/scholar?q=Shashi+Kant+Gupta%2C+Aditya+Basu%2C+Mauro+Nievas%2C+Jerrin+Thomas%2C+Nathan+Wolfrath%2C+Adhitya+Ramamurthi%2C+Bradley+Taylor%2C+Anai+N+Kothari%2C+Regina+Schwind%2C+Therica+M+Miller%2C+et+al%2E+2024%2E+Prism%3A+Patient+records+)
3. Brian Hazlehurst, H Robert Frost, Dean F Sittig, and Victor J Stevens. 2005. Mediclass: A system for detecting and classifying encounter-based clinical events in any electronic medical record. Journal of the American Medical Informatics Association, 12(5):517–529. [find](https://scholar.google.com/scholar?q=Brian+Hazlehurst%2C+H+Robert+Frost%2C+Dean+F+Sittig%2C+and+Victor+J+Stevens%2E+2005%2E+Mediclass%3A+A+system+for+detecting+and+classifying+encounter%2Dbased+clinical+events+in+any+electronic+medical+record%2E+Journal)
4. K Karthik, Sowmya Kamath, R Supreetha, and Ashish Katlam. 2025. Content-based medical retrieval systems with evidence-based diagnosis for enhanced clinical decision support. Expert Systems with Applications, 272:126678. [find](https://scholar.google.com/scholar?q=K+Karthik%2C+Sowmya+Kamath%2C+R+Supreetha%2C+and+Ashish+Katlam%2E+2025%2E+Content%2Dbased+medical+retrieval+systems+with+evidence%2Dbased+diagnosis+for+enhanced+clinical+decision+support%2E+Expert+Systems+with+Applic)
5. Yubin Kim, Hyewon Jeong, Shan Chen, Shuyue Stella Li, Chanwoo Park, Mingyu Lu, Kumail Alhamoud, Jimin Mun, Cristina Grau, Minseok Jung, et al. 2025. Medical hallucinations in foundation models and their impact on healthcare. arXiv preprint arXiv:2503.05777. — [arXiv:2503.05777](https://arxiv.org/abs/2503.05777) [find](https://scholar.google.com/scholar?q=Yubin+Kim%2C+Hyewon+Jeong%2C+Shan+Chen%2C+Shuyue+Stella+Li%2C+Chanwoo+Park%2C+Mingyu+Lu%2C+Kumail+Alhamoud%2C+Jimin+Mun%2C+Cristina+Grau%2C+Minseok+Jung%2C+et+al%2E+2025%2E+Medical+hallucinations+in+foundation+models+and+the)
6. Vladimir I Levenshtein et al. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, volume 10, pages 707–710. Soviet Union. [find](https://scholar.google.com/scholar?q=Vladimir+I+Levenshtein+et+al%2E+1966%2E+Binary+codes+capable+of+correcting+deletions%2C+insertions%2C+and+reversals%2E+In+Soviet+physics+doklady%2C+volume+10%2C+pages+707%E2%80%93710%2E+Soviet+Union%2E)
7. Jialin Liu, Fang Liu, Changyu Wang, and Siru Liu. 2025. Prompt engineering in clinical practice: tutorial for clinicians. Journal of Medical Internet Research, 27:e72644. [find](https://scholar.google.com/scholar?q=Jialin+Liu%2C+Fang+Liu%2C+Changyu+Wang%2C+and+Siru+Liu%2E+2025%2E+Prompt+engineering+in+clinical+practice%3A+tutorial+for+clinicians%2E+Journal+of+Medical+Internet+Research%2C+27%3Ae72644%2E)
8. Kathleen N Lohr, Kristen Eleazer, and Josephine Mauskopf. 1998. Health policy issues and applications for evidence-based medicine and clinical practice guidelines. Health policy, 46(1):1–19. [find](https://scholar.google.com/scholar?q=Kathleen+N+Lohr%2C+Kristen+Eleazer%2C+and+Josephine+Mauskopf%2E+1998%2E+Health+policy+issues+and+applications+for+evidence%2Dbased+medicine+and+clinical+practice+guidelines%2E+Health+policy%2C+46%281%29%3A1%E2%80%9319%2E)
9. Pasi Shailendra, Rudra Chandra Ghosh, Rajdeep Kumar, and Nitin Sharma. 2024. Survey of large language models for answering questions across various fields. In 2024 10th International Conference on Advanced Computing and Communication Systems (ICACCS), volume 1, pages 520–527. IEEE. [find](https://scholar.google.com/scholar?q=Pasi+Shailendra%2C+Rudra+Chandra+Ghosh%2C+Rajdeep+Kumar%2C+and+Nitin+Sharma%2E+2024%2E+Survey+of+large+language+models+for+answering+questions+across+various+fields%2E+In+2024+10th+International+Conference+on+Adv)
10. Sarvesh Soni and Dina Demner-Fushman. 2026a. A dataset for addressing patient’s information needs related to clinical course of hospitalization. Scientific Data. [find](https://scholar.google.com/scholar?q=Sarvesh+Soni+and+Dina+Demner%2DFushman%2E+2026a%2E+A+dataset+for+addressing+patient%E2%80%99s+information+needs+related+to+clinical+course+of+hospitalization%2E+Scientific+Data%2E)
11. Sarvesh Soni and Dina Demner-Fushman. 2026b. Overview of the archehr-qa 2026 shared task on grounded question answering from electronic health records. In Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health), Palma, Mallorca (Spain). ELRA. [find](https://scholar.google.com/scholar?q=Sarvesh+Soni+and+Dina+Demner%2DFushman%2E+2026b%2E+Overview+of+the+archehr%2Dqa+2026+shared+task+on+grounded+question+answering+from+electronic+health+records%2E+In+Proceedings+of+the+Third+Workshop+on+Patient%2D)
12. Aslıhan Uzun Bektaş, Balahan Bora, and Erbil Ünsal. 2025. Comparative evaluation of chatgpt and llama for reliability, quality, and accuracy in familial mediterranean fever. European Journal of Pediatrics, 184(8):491. [find](https://scholar.google.com/scholar?q=Asl%C4%B1han+Uzun+Bekta%C5%9F%2C+Balahan+Bora%2C+and+Erbil+%C3%9Cnsal%2E+2025%2E+Comparative+evaluation+of+chatgpt+and+llama+for+reliability%2C+quality%2C+and+accuracy+in+familial+mediterranean+fever%2E+European+Journal+of+Pediat)
13. Laura Zakowski, Christine Seibert, and Wisconsin Selma VanEyck. 2004. Evidence-based medicine: answering questions of diagnosis. Clinical medicine & research, 2(1):63–69. [find](https://scholar.google.com/scholar?q=Laura+Zakowski%2C+Christine+Seibert%2C+and+Wisconsin+Selma+VanEyck%2E+2004%2E+Evidence%2Dbased+medicine%3A+answering+questions+of+diagnosis%2E+Clinical+medicine+%26+research%2C+2%281%29%3A63%E2%80%9369%2E)
14. Xinping Zhao, Dongfang Li, Yan Zhong, Boren Hu, Yibin Chen, Baotian Hu, and Min Zhang. 2024. Seer: Self-aligned evidence extraction for retrieval-augmented generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3027– 3041. [find](https://scholar.google.com/scholar?q=Xinping+Zhao%2C+Dongfang+Li%2C+Yan+Zhong%2C+Boren+Hu%2C+Yibin+Chen%2C+Baotian+Hu%2C+and+Min+Zhang%2E+2024%2E+Seer%3A+Self%2Daligned+evidence+extraction+for+retrieval%2Daugmented+generation%2E+In+Proceedings+of+the+2024+Confe)
15. Chenqian Le, Ziheng Gong, Chihang Wang, Haowei Ni, Panfeng Li, and Xupeng Chen. 2025. [find](https://scholar.google.com/scholar?q=Chenqian+Le%2C+Ziheng+Gong%2C+Chihang+Wang%2C+Haowei+Ni%2C+Panfeng+Li%2C+and+Xupeng+Chen%2E+2025%2E)

*Updated 1 November 2026*