---
title: "MediEval — paper by Qu and Färber (2026) — summary and ideas"
description: "As a fix, the authors propose CoRFu, a DPO variant that trains on quadrant-structured preference pairs and penalises unsafe… 39 references."
authors: ["Zhan Qu", "Michael Färber"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.734/"
doi: "10.18653/v1/2026.acl-long.734"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qu-2026-medieval-unified-medical-benchmark-patient/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qu-2026-medieval-unified-medical-benchmark-patient/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/41/) / MediEval: A Unified Medical Benchmark for Patient-Contextual and Knowledge-Grounded Reasoning in LLMs

# MediEval: A Unified Medical Benchmark for Patient-Contextual and Knowledge-Grounded Reasoning in LLMs

**Authors:** Zhan Qu and [Michael Färber](https://md.research.elamur.ai/free/en/authors/michael-farber/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.734/) · [PDF](https://aclanthology.org/2026.acl-long.734.pdf) · [DOI 10.18653/v1/2026.acl-long.734](https://doi.org/10.18653/v1/2026.acl-long.734) · [OpenAlex](https://openalex.org/W7166801843)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Medical LLM evaluations usually check one of two things. Some test isolated medical facts. Others test reasoning over patient records without confirming that the conclusions are medically correct. MediEval combines both checks. It pairs MIMIC-IV electronic health records (discharge summaries plus coded diagnoses, procedures and medications) with a knowledge base built from UMLS, SNOMED CT and RxNorm. Codes are mapped to UMLS CUIs and linked through an ontology graph. Graph-guided substitutions then produce true and counterfactual statements tied to each patient's context.

Each statement falls into one of four quadrants, crossing factual truth (True/False) with support in the record (Supported/Unsupported). Dangerous mistakes are tracked by two newly introduced safety metrics, one called the Hallucinated Support Rate (HSR), the other the Truth Inversion Rate (TIR). Proprietary, general open-source and biomedical LLMs all show systematic quadrant-level weaknesses.

As a fix, the authors propose CoRFu, a DPO variant that trains on quadrant-structured preference pairs and penalises unsafe confusions more heavily. It raises macro-F1 by 16.4 points and removes truth inversion errors. The work shows that headline accuracy can hide safety-critical failures in clinical reasoning.

## Key points
- **Benchmark scale:** MediEval draws on about 2k MIMIC-IV admissions and contains roughly 37k statements. The source database holds 546,028 admissions from 223,452 patients. The construction pipeline is automated and released at github.com/ZhanQu945/MediEval.
- **Four-quadrant labels:** Q1 = True–Supported, Q2 = True–Unsupported, Q3 = False–Supported, Q4 = False–Unsupported. HSR and TIR measure the safety-critical confusions between these quadrants.
- **Best Llama result:** With Llama-3.1-8B-Instruct, CoRFu trained on Q1 vs. Q2 pairs reaches 77.9% macro-F1. HSR drops to 18.2% and TIR falls to 0.
- **Qwen3-8B result:** The Q1 vs. Q2 pairing gives 71.0% macro-F1 (+11.1 over base) and TIR of 0.0. Curriculum training gives the lowest HSR, at 17.3%.
- **Penalty ablation:** Setting the asymmetric penalty λ to 0 reduces CoRFu to vanilla DPO. Values of 0.5–1.0 work best, while values above 1.0 over-penalise and hurt performance.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Technische Universität Dresden](https://md.research.elamur.ai/free/en/universities/technische-universitat-dresden/) — University, Dresden, Germany · 13 works · 7,336 citations
- [Center for Scalable Data Analytics and Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/center-for-scalable-data-analytics-and-artificial-intelligence/) — University, Dresden, Germany · 10 works · 77 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 275 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 214 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works

## Cite

Cite as: Zhan Qu and Michael Färber. 2026. MediEval: A Unified Medical Benchmark for Patient-Contextual and Knowledge-Grounded Reasoning in LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{qu2026medieval,
  title = {{MediEval: A Unified Medical Benchmark for Patient-Contextual and Knowledge-Grounded Reasoning in LLMs}},
  author = {Zhan Qu and Michael Färber},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.734},
  url = {https://doi.org/10.18653/v1/2026.acl-long.734}
}
```

- [← previous Medical Summarization in Practice: Design, Deployment, and Analysis of a Clinical Summarization System for a German Hospital](https://md.research.elamur.ai/free/en/papers/rauf-2026-medical-summarization-practice-design-deployment/)
- [next → Mediocrity is the key for LLM as a Judge Anchor Selection](https://md.research.elamur.ai/free/en/papers/yehiya-2026-mediocrity-is-key-llm-as/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/41/)

## References

39 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Alaa, Thomas Hartvigsen, Niloufar Golchini, Shiladitya Dutta, Frances Dean, Inioluwa Deborah Raji, and Travis Zack. 2025. Position: Medical large language model benchmarks should prioritize construct validity. In Forty-second International Conference on Machine Learning Position Paper Track. [find](https://scholar.google.com/scholar?q=Ahmed+Alaa%2C+Thomas+Hartvigsen%2C+Niloufar+Golchini%2C+Shiladitya+Dutta%2C+Frances+Dean%2C+Inioluwa+Deborah+Raji%2C+and+Travis+Zack%2E+2025%2E+Position%3A+Medical+large+language+model+benchmarks+should+prioritize+cons)
2. Olivier Bodenreider. 2004. The unified medical language system (umls): integrating biomedical terminology. Nucleic acids research, 32(suppl_1):D267– D270. [find](https://scholar.google.com/scholar?q=Olivier+Bodenreider%2E+2004%2E+The+unified+medical+language+system+%28umls%29%3A+integrating+biomedical+terminology%2E+Nucleic+acids+research%2C+32%28suppl%5F1%29%3AD267%E2%80%93+D270%2E)
3. Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, and 1 others. 2023. [Meditron-70b: Scaling medical pretraining for large language models](https://md.research.elamur.ai/free/en/papers/chen-2023-meditron-70b-scaling-medical-pretraining/). arXiv preprint arXiv:2311.16079.
4. Clément Christophe, Praveen K Kanithi, Prateek Munjal, Tathagata Raha, Nasir Hayat, Ronnie Rajan, Ahmed Al-Mahrooqi, Avani Gupta, Muhammad Umar Salman, Gurpreet Gosal, and 1 others. 2024. Med42–evaluating fine-tuning strategies for medical llms: full-parameter vs. parameter-efficient approaches. arX… — [arXiv:2404.14779](https://arxiv.org/abs/2404.14779) [find](https://scholar.google.com/scholar?q=Cl%C3%A9ment+Christophe%2C+Praveen+K+Kanithi%2C+Prateek+Munjal%2C+Tathagata+Raha%2C+Nasir+Hayat%2C+Ronnie+Rajan%2C+Ahmed+Al%2DMahrooqi%2C+Avani+Gupta%2C+Muhammad+Umar+Salman%2C+Gurpreet+Gosal%2C+and+1+others%2E+2024%2E+Med42%E2%80%93evalua)
5. Kevin Donnelly and 1 others. 2006. Snomed-ct: The advanced terminology and coding system for ehealth. Studies in health technology and informatics, 121:279. [find](https://scholar.google.com/scholar?q=Kevin+Donnelly+and+1+others%2E+2006%2E+Snomed%2Dct%3A+The+advanced+terminology+and+coding+system+for+ehealth%2E+Studies+in+health+technology+and+informatics%2C+121%3A279%2E)
6. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
7. Maxime Griot, Jean Vanderdonckt, Demet Yuksel, and Coralie Hemptinne. 2025. Pattern recognition or medical knowledge? the problem with multiple-choice questions in medicine. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5321… [find](https://scholar.google.com/scholar?q=Maxime+Griot%2C+Jean+Vanderdonckt%2C+Demet+Yuksel%2C+and+Coralie+Hemptinne%2E+2025%2E+Pattern+recognition+or+medical+knowledge%3F+the+problem+with+multiple%2Dchoice+questions+in+medicine%2E+In+Proceedings+of+the+63rd)
8. Joschka Haltaufderheide and Robert Ranisch. 2024. The ethics of chatgpt in medicine and healthcare: a systematic review on large language models (llms). NPJ digital medicine, 7(1):183. [find](https://scholar.google.com/scholar?q=Joschka+Haltaufderheide+and+Robert+Ranisch%2E+2024%2E+The+ethics+of+chatgpt+in+medicine+and+healthcare%3A+a+systematic+review+on+large+language+models+%28llms%29%2E+NPJ+digital+medicine%2C+7%281%29%3A183%2E)
9. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). arXiv preprint arXiv:2410.21276.
10. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guill)
11. Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, and 1 others. 2023. [Mimic-iv, a freely accessible electronic health record dataset](https://md.research.elamur.ai/free/en/papers/johnson-2023-mimic-iv-freely-accessible-electronic/). Scientific data, 10(1):1.
12. Simon Liu, Wei Ma, Robin Moore, Vikraman Ganesan, and Stuart Nelson. 2005. Rxnorm: prescription for electronic drug information exchange. IT professional, 7(5):17–23. [find](https://scholar.google.com/scholar?q=Simon+Liu%2C+Wei+Ma%2C+Robin+Moore%2C+Vikraman+Ganesan%2C+and+Stuart+Nelson%2E+2005%2E+Rxnorm%3A+prescription+for+electronic+drug+information+exchange%2E+IT+professional%2C+7%285%29%3A17%E2%80%9323%2E)
13. Jesús Lovón-Melgarejo, Martin Mouysset, Jo Olei-wan, Jose G Moreno, Christine Damase-Michel, and Lynda Tamine. 2025. Evaluating llm abilities to understand tabular electronic health records: A comprehensive study of patient data extraction and retrieval. In European Conference on Information Retriev… [find](https://scholar.google.com/scholar?q=Jes%C3%BAs+Lov%C3%B3n%2DMelgarejo%2C+Martin+Mouysset%2C+Jo+Olei%2Dwan%2C+Jose+G+Moreno%2C+Christine+Damase%2DMichel%2C+and+Lynda+Tamine%2E+2025%2E+Evaluating+llm+abilities+to+understand+tabular+electronic+health+records%3A+A+compreh)
14. Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric J Topol, and Pranav Rajpurkar. 2023. Foundation models for generalist medical artificial intelligence. Nature, 616(7956):259–265. [find](https://scholar.google.com/scholar?q=Michael+Moor%2C+Oishi+Banerjee%2C+Zahra+Shakeri+Hossein+Abad%2C+Harlan+M+Krumholz%2C+Jure+Leskovec%2C+Eric+J+Topol%2C+and+Pranav+Rajpurkar%2E+2023%2E+Foundation+models+for+generalist+medical+artificial+intelligence%2E+)
15. Stuart J Nelson, Kelly Zeng, John Kilbourne, Tammy Powell, and Robin Moore. 2011. Normalized names for clinical drugs: Rxnorm at 6 years. Journal of the American Medical Informatics Association, 18(4):441–448. [find](https://scholar.google.com/scholar?q=Stuart+J+Nelson%2C+Kelly+Zeng%2C+John+Kilbourne%2C+Tammy+Powell%2C+and+Robin+Moore%2E+2011%2E+Normalized+names+for+clinical+drugs%3A+Rxnorm+at+6+years%2E+Journal+of+the+American+Medical+Informatics+Association%2C+18%284%29)
16. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 6 others. 2… — [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
17. Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. [Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering](https://md.research.elamur.ai/free/en/papers/pal-2022-medmcqa-large-scale-multi-subject/). In Conference on health, inference, and learning, pages 248–260. PMLR.
18. Zhan Qu and Michael Färber. 2026a. Grail: Geometry-aware retrieval-augmented inference with llms over hyperbolic representations of patient trajectories. arXiv preprint arXiv:2602.12828. — [arXiv:2602.12828](https://arxiv.org/abs/2602.12828) [find](https://scholar.google.com/scholar?q=Zhan+Qu+and+Michael+F%C3%A4rber%2E+2026a%2E+Grail%3A+Geometry%2Daware+retrieval%2Daugmented+inference+with+llms+over+hyperbolic+representations+of+patient+trajectories%2E+arXiv+preprint+arXiv%3A2602%2E12828%2E+%E2%80%94+%5BarXiv%3A2602)
19. Zhan Qu and Michael Färber. 2026b. Trace: Temporal reasoning via agentic context evolution for streaming electronic health records (ehrs). arXiv preprint arXiv:2602.12833. — [arXiv:2602.12833](https://arxiv.org/abs/2602.12833) [find](https://scholar.google.com/scholar?q=Zhan+Qu+and+Michael+F%C3%A4rber%2E+2026b%2E+Trace%3A+Temporal+reasoning+via+agentic+context+evolution+for+streaming+electronic+health+records+%28ehrs%29%2E+arXiv+preprint+arXiv%3A2602%2E12833%2E+%E2%80%94+%5BarXiv%3A2602%2E12833%5D%28https%3A%2F)
20. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. 2023. [Direct preference optimization: your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 53728–53741…
21. Pranav Rajpurkar, Emma Chen, Oishi Banerjee, and Eric J Topol. 2022. Ai in health and medicine. Nature medicine, 28(1):31–38. [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Emma+Chen%2C+Oishi+Banerjee%2C+and+Eric+J+Topol%2E+2022%2E+Ai+in+health+and+medicine%2E+Nature+medicine%2C+28%281%29%3A31%E2%80%9338%2E)
22. Andrey Sakhovskiy and Elena Tutubalina. 2025. Bali: Enhancing biomedical language representations through knowledge graph and language model alignment. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1152–1164. [find](https://scholar.google.com/scholar?q=Andrey+Sakhovskiy+and+Elena+Tutubalina%2E+2025%2E+Bali%3A+Enhancing+biomedical+language+representations+through+knowledge+graph+and+language+model+alignment%2E+In+Proceedings+of+the+48th+International+ACM+SIG)
23. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, and 1 others. 2025. [Openai gpt-5 system card](https://md.research.elamur.ai/free/en/papers/singh-2025-openai-gpt-5-system-card/). arXiv preprint arXiv:2601.03267.
24. Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, and 1 others. 2023. [Large language models encode clinical knowledge](https://md.research.elamur.ai/free/en/papers/singhal-2022-large-language-models-encode-clinical/). Nature, 620(7972):172–180.
25. Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, and 1 others. 2025. Toward expert-level medical question answering with large language models. Nature Medicine, 31(3):943–950. [find](https://scholar.google.com/scholar?q=Karan+Singhal%2C+Tao+Tu%2C+Juraj+Gottweis%2C+Rory+Sayres%2C+Ellery+Wulczyn%2C+Mohamed+Amin%2C+Le+Hou%2C+Kevin+Clark%2C+Stephen+R+Pfohl%2C+Heather+Cole%2DLewis%2C+and+1+others%2E+2025%2E+Toward+expert%2Dlevel+medical+question+ans)
26. Xiaorui Su, Yibo Wang, Shanghua Gao, Xiaolong Liu, Valentina Giunchiglia, Djork-Arné Clevert, and Marinka Zitnik. 2025. KGARevion: An AI agent for knowledge-intensive biomedical QA. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Xiaorui+Su%2C+Yibo+Wang%2C+Shanghua+Gao%2C+Xiaolong+Liu%2C+Valentina+Giunchiglia%2C+Djork%2DArn%C3%A9+Clevert%2C+and+Marinka+Zitnik%2E+2025%2E+KGARevion%3A+An+AI+agent+for+knowledge%2Dintensive+biomedical+QA%2E+In+The+Thirteenth+)
27. Mujeen Sung, Jinhyuk Lee, Sean Yi, Minji Jeon, Sungdong Kim, and Jaewoo Kang. 2021. [Can language models be biomedical knowledge bases?](https://md.research.elamur.ai/free/en/papers/sung-2021-can-language-models-be-biomedical/) In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4723–4734, Online and Punta Cana, Dominican Republic. Association fo…
28. Yanchao Tan, Hang Lv, Yunfei Zhan, Guofang Ma, Bo Xiong, and Carl Yang. 2025. BoxLM: Unifying structures and semantics of medical concepts for diagnosis prediction in healthcare. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Yanchao+Tan%2C+Hang+Lv%2C+Yunfei+Zhan%2C+Guofang+Ma%2C+Bo+Xiong%2C+and+Carl+Yang%2E+2025%2E+BoxLM%3A+Unifying+structures+and+semantics+of+medical+concepts+for+diagnosis+prediction+in+healthcare%2E+In+Forty%2Dsecond+Inter)
29. Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine, 29(8):1930– 1940. [find](https://scholar.google.com/scholar?q=Arun+James+Thirunavukarasu%2C+Darren+Shu+Jeng+Ting%2C+Kabilan+Elangovan%2C+Laura+Gutierrez%2C+Ting+Fang+Tan%2C+and+Daniel+Shu+Wei+Ting%2E+2023%2E+Large+language+models+in+medicine%2E+Nature+medicine%2C+29%288%29%3A1930%E2%80%93+1940)
30. Augustin Toma, Patrick R Lawler, Jimmy Ba, Rahul G Krishnan, Barry B Rubin, and Bo Wang. 2023. Clinical camel: An open expert-level medical language model with dialogue-based knowledge encoding. arXiv preprint arXiv:2305.12031. [find](https://scholar.google.com/scholar?q=Augustin+Toma%2C+Patrick+R+Lawler%2C+Jimmy+Ba%2C+Rahul+G+Krishnan%2C+Barry+B+Rubin%2C+and+Bo+Wang%2E+2023%2E+Clinical+camel%3A+An+open+expert%2Dlevel+medical+language+model+with+dialogue%2Dbased+knowledge+encoding%2E+arXiv)
31. Michael Wornow, Suhana Bedi, Miguel Angel Fuentes Hernandez, Ethan Steinberg, Jason Alan Fries, Christopher Re, Sanmi Koyejo, and Nigam Shah. 2025. Context clues: Evaluating long context models for clinical prediction tasks on EHR data. In The Thirteenth International Conference on Learning Represen… [find](https://scholar.google.com/scholar?q=Michael+Wornow%2C+Suhana+Bedi%2C+Miguel+Angel+Fuentes+Hernandez%2C+Ethan+Steinberg%2C+Jason+Alan+Fries%2C+Christopher+Re%2C+Sanmi+Koyejo%2C+and+Nigam+Shah%2E+2025%2E+Context+clues%3A+Evaluating+long+context+models+for+cl)
32. Jiahuan Yan, Jintai Chen, Chaowen Hu, Bo Zheng, Yaojun Hu, Jimeng Sun, and Jian Wu. 2025. Small models are LLM knowledge triggers for medical tabular prediction. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jiahuan+Yan%2C+Jintai+Chen%2C+Chaowen+Hu%2C+Bo+Zheng%2C+Yaojun+Hu%2C+Jimeng+Sun%2C+and+Jian+Wu%2E+2025%2E+Small+models+are+LLM+knowledge+triggers+for+medical+tabular+prediction%2E+In+The+Thirteenth+International+Confer)
33. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
34. Rui Yang, Ting Fang Tan, Wei Lu, Arun James Thirunavukarasu, Daniel Shu Wei Ting, and Nan Liu. 2023. Large language models in health care: Development, applications, and challenges. Health Care Science, 2(4):255–263. [find](https://scholar.google.com/scholar?q=Rui+Yang%2C+Ting+Fang+Tan%2C+Wei+Lu%2C+Arun+James+Thirunavukarasu%2C+Daniel+Shu+Wei+Ting%2C+and+Nan+Liu%2E+2023%2E+Large+language+models+in+health+care%3A+Development%2C+applications%2C+and+challenges%2E+Health+Care+Scienc)
35. Xiechi Zhang, Zetian Ouyang, Linlin Wang, Gerard De Melo, Zhu Cao, Xiaoling Wang, Ya Zhang, Yanfeng Wang, and Liang He. 2025. AutoMedEval: Harnessing language models for automatic medical capability evaluation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Xiechi+Zhang%2C+Zetian+Ouyang%2C+Linlin+Wang%2C+Gerard+De+Melo%2C+Zhu+Cao%2C+Xiaoling+Wang%2C+Ya+Zhang%2C+Yanfeng+Wang%2C+and+Liang+He%2E+2025%2E+AutoMedEval%3A+Harnessing+language+models+for+automatic+medical+capability+e)
36. Yuxuan Zhou, Xien Liu, Chen Ning, and Ji Wu. 2024. Multifaceteval: multifaceted evaluation to probe llms in mastering medical knowledge. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 6669–6677. [find](https://scholar.google.com/scholar?q=Yuxuan+Zhou%2C+Xien+Liu%2C+Chen+Ning%2C+and+Ji+Wu%2E+2024%2E+Multifaceteval%3A+multifaceted+evaluation+to+probe+llms+in+mastering+medical+knowledge%2E+In+Proceedings+of+the+Thirty%2DThird+International+Joint+Conferen)
37. Yuxuan Zhou, Xien Liu, Chen Ning, Xiao Zhang, and Ji Wu. 2025a. Reliable and diverse evaluation of LLM medical knowledge mastery. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yuxuan+Zhou%2C+Xien+Liu%2C+Chen+Ning%2C+Xiao+Zhang%2C+and+Ji+Wu%2E+2025a%2E+Reliable+and+diverse+evaluation+of+LLM+medical+knowledge+mastery%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations)
38. Yuxuan Zhou, Xien Liu, Chenwei Yan, Chen Ning, Xiao Zhang, Boxun Li, Xiangling Fu, Shijin Wang, Guoping Hu, Yu Wang, and Ji Wu. 2025b. Evaluating LLMs across multi-cognitive levels: From medical knowledge mastery to scenario-based problem solving. In Forty-second International Conference on Machine… [find](https://scholar.google.com/scholar?q=Yuxuan+Zhou%2C+Xien+Liu%2C+Chenwei+Yan%2C+Chen+Ning%2C+Xiao+Zhang%2C+Boxun+Li%2C+Xiangling+Fu%2C+Shijin+Wang%2C+Guoping+Hu%2C+Yu+Wang%2C+and+Ji+Wu%2E+2025b%2E+Evaluating+LLMs+across+multi%2Dcognitive+levels%3A+From+medical+knowl)
39. Junde Wu, Jiayuan Zhu, Yunli Qi, Jingkun Chen, Min Xu, Filippo Menolascina, Yueming Jin, and Vicente Grau. 2025. [Medical graph RAG: Evidence-based medical large language model via graph retrieval-augmented generation](https://md.research.elamur.ai/free/en/papers/wu-2025-medical-graph-rag-evidence-based/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Lin…

*Updated 1 November 2026*