---
title: "How Robust Are Large Language Models for Clinical Numeracy?"
description: "Using LLMs safely for clinical decision support depends on handling patient measurements correctly. 43 references."
authors: ["Minh-Vuong Nguyen", "Fatemeh Shiri", "Zhuang Li", "Karin Verspoor"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.901/"
doi: "10.18653/v1/2026.findings-acl.901"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nguyen-2026-how-robust-are-large-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nguyen-2026-how-robust-are-large-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/) / How Robust Are Large Language Models for Clinical Numeracy? An Empirical Study on Numerical Reasoning Abilities in Clinical Contexts

# How Robust Are Large Language Models for Clinical Numeracy? An Empirical Study on Numerical Reasoning Abilities in Clinical Contexts

**Authors:** Minh-Vuong Nguyen, Fatemeh Shiri, [Zhuang Li](https://md.research.elamur.ai/free/en/authors/zhuang-li/) and [Karin Verspoor](https://md.research.elamur.ai/free/en/authors/karin-verspoor/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.901/) · [PDF](https://aclanthology.org/2026.findings-acl.901.pdf) · [DOI 10.18653/v1/2026.findings-acl.901](https://doi.org/10.18653/v1/2026.findings-acl.901) · [OpenAlex](https://openalex.org/W7166840400)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Using LLMs safely for clinical decision support depends on handling patient measurements correctly. It also depends on that handling staying correct when the same values are recorded in different ways. Earlier clinical benchmarks, such as MedCalc-Bench, mostly test arithmetic and ignore documentation format. The authors built ClinicNumRobBench to cover both gaps.

The benchmark samples longitudinal vital-sign records and demographics from MIMIC-IV and MIMIC-IV ED. Each patient context is rendered in three equivalent forms:
- a JSON-like structured format
- a single fixed verbalization template
- note-style variants modeled on real notes from the Open Patients dataset

Template-generated questions probe four operations: value retrieval, arithmetic, relational comparison and aggregation. Correct answers are computed programmatically.

Across 17 LLMs, retrieval proved largely solved, but comparison and aggregation were weak. Medically fine-tuned models sometimes reasoned with numbers worse than their base versions. Note-style formatting also lowered accuracy, especially when abbreviations appeared or units were missing.

The benchmark gives developers and clinicians a controlled way to locate where numerical reasoning breaks down before deployment.

## Key points
- **Benchmark design:** ClinicNumRobBench contains 1,624 context–question instances generated from 42 question templates. Contexts come from 200 MIMIC-IV patients, stratified by age and gender, each with 1–50 records.
- **Note-style variants:** These were built in stages from Open Patients notes:
  - 180K notes were filtered down to 15K numerically rich candidates.
  - gpt-4.1-mini extracted vital-sign mentions from them, leaving 2,300 notes with validated spans.
  - A manual audit of 100 substituted outputs found 96% correct.
- **Results by operation:** Most models scored above 85% on value retrieval. On comparison and aggregation, some fell below 15%.
- **Effect of medical fine-tuning:** It could reduce numeracy by more than 30% relative to the base model, with comparison tasks most affected.
- **Format sensitivity:** Abbreviations and omitted units hurt performance most, while reordering attributes or changing separators had little effect. The data is released under PhysioNet credentialed access.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [RMIT University](https://md.research.elamur.ai/free/en/universities/rmit-university/) — University, Melbourne, Australia · 17 works · 703 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 241 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 184 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Minh-Vuong Nguyen, Fatemeh Shiri, Zhuang Li and Karin Verspoor. 2026. How Robust Are Large Language Models for Clinical Numeracy? An Empirical Study on Numerical Reasoning Abilities in Clinical Contexts. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nguyen2026how,
  title = {{How Robust Are Large Language Models for Clinical Numeracy? An Empirical Study on Numerical Reasoning Abilities in Clinical Contexts}},
  author = {Minh-Vuong Nguyen, Fatemeh Shiri, Zhuang Li and Karin Verspoor},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.901},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.901}
}
```

- [← previous How multilingual are multilingual LLMs? A case study in Northern Sámi-Finnish Translation](https://md.research.elamur.ai/free/en/papers/saleva-2026-how-multilingual-are-multilingual-llms/)
- [next → How to Set the Learning Rate for Large-Scale Pre-training?](https://md.research.elamur.ai/free/en/papers/zhou-2026-how-set-learning-rate-large/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/)

## References

43 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. 2024. [Phi-4 Technical Report](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-4-technical-report/). arXiv preprint arXiv:2412.08905.
2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
3. Pulkit Arora, Akbar Karimi, and Lucie Flek. 2025. Exploring Robustness of LLMs to Paraphrasing Based on Sociodemographic Factors. [find](https://scholar.google.com/scholar?q=Pulkit+Arora%2C+Akbar+Karimi%2C+and+Lucie+Flek%2E+2025%2E+Exploring+Robustness+of+LLMs+to+Paraphrasing+Based+on+Sociodemographic+Factors%2E)
4. Suhana Bedi, Hejie Cui, Miguel Fuentes, Alyssa Unell, Michael Wornow, Juan M Banda, Nikesh Kotecha, Timothy Keyes, Yifan Mai, Mert Oez, et al. 2025. MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks. arXiv preprint arXiv:2505.23802. [find](https://scholar.google.com/scholar?q=Suhana+Bedi%2C+Hejie+Cui%2C+Miguel+Fuentes%2C+Alyssa+Unell%2C+Michael+Wornow%2C+Juan+M+Banda%2C+Nikesh+Kotecha%2C+Timothy+Keyes%2C+Yifan+Mai%2C+Mert+Oez%2C+et+al%2E+2025%2E+MedHELM%3A+Holistic+Evaluation+of+Large+Language+Mode)
5. Hanjie Chen, Zhouxiang Fang, Yash Singla, and Mark Dredze. 2025a. [Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions](https://md.research.elamur.ai/free/en/papers/chen-2025-benchmarking-large-language-models-answering/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Languag…
6. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. 2024a. HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs. arXiv preprint arXiv:2412.18925. [find](https://scholar.google.com/scholar?q=Junying+Chen%2C+Zhenyang+Cai%2C+Ke+Ji%2C+Xidong+Wang%2C+Wanlong+Liu%2C+Rongsheng+Wang%2C+Jianye+Hou%2C+and+Benyou+Wang%2E+2024a%2E+HuatuoGPT%2Do1%2C+Towards+Medical+Complex+Reasoning+with+LLMs%2E+arXiv+preprint+arXiv%3A2412%2E18)
7. Sijia Chen, Xiaomin Li, Mengxue Zhang, Eric Hanchen Jiang, Qin Zeng, and Chen-Hsiang Yu. 2025b. CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMs. ArXiv, abs/2505.11413. — [arXiv:2505.11413](https://arxiv.org/abs/2505.11413) [find](https://scholar.google.com/scholar?q=Sijia+Chen%2C+Xiaomin+Li%2C+Mengxue+Zhang%2C+Eric+Hanchen+Jiang%2C+Qin+Zeng%2C+and+Chen%2DHsiang+Yu%2E+2025b%2E+CARES%3A+Comprehensive+Evaluation+of+Safety+and+Adversarial+Robustness+in+Medical+LLMs%2E+ArXiv%2C+abs%2F2505%2E11)
8. Zeming Chen, Angelika Romanou, Antoine Bonnet, Alejandro Hernández-Cano, Badr Alkhamissi, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, et al. 2024b. MEDITRON: Open Medical Foundation Models Adapted for Clinical Practice. [find](https://scholar.google.com/scholar?q=Zeming+Chen%2C+Angelika+Romanou%2C+Antoine+Bonnet%2C+Alejandro+Hern%C3%A1ndez%2DCano%2C+Badr+Alkhamissi%2C+Kyle+Matoba%2C+Francesco+Salvi%2C+Matteo+Pagliardini%2C+Simin+Fan%2C+Andreas+K%C3%B6pf%2C+et+al%2E+2024b%2E+MEDITRON%3A+Open+Medica)
9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. [Training Verifiers to Solve Math Word Problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
10. Jean-Philippe Corbeil, Amin Dada, Jean-Michel Attendu, Asma Ben Abacha, Alessandro Sordoni, Lucas Caccia, François Beaulieu, Thomas Lin, Jens Kleesiek, and Paul Vozila. 2025. A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks… [find](https://scholar.google.com/scholar?q=Jean%2DPhilippe+Corbeil%2C+Amin+Dada%2C+Jean%2DMichel+Attendu%2C+Asma+Ben+Abacha%2C+Alessandro+Sordoni%2C+Lucas+Caccia%2C+Fran%C3%A7ois+Beaulieu%2C+Thomas+Lin%2C+Jens+Kleesiek%2C+and+Paul+Vozila%2E+2025%2E+A+Modular+Approach+for+Cl)
11. DeepSeek-AI. 2025. [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/).
12. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv e-prints, pages arXiv–2407.
13. Abinitha Gourabathina, Walter Gerych, Eileen Pan, and Marzyeh Ghassemi. 2025. The Medium is the Message: How Non-Clinical Information Shapes Clinical Decisions in LLMs. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, pages 1805–1828. [find](https://scholar.google.com/scholar?q=Abinitha+Gourabathina%2C+Walter+Gerych%2C+Eileen+Pan%2C+and+Marzyeh+Ghassemi%2E+2025%2E+The+Medium+is+the+Message%3A+How+Non%2DClinical+Information+Shapes+Clinical+Decisions+in+LLMs%2E+In+Proceedings+of+the+2025+ACM+)
14. Pierre Guiraud. 1959. Problèmes et méthodes de la statistique linguistique. [find](https://scholar.google.com/scholar?q=Pierre+Guiraud%2E+1959%2E+Probl%C3%A8mes+et+m%C3%A9thodes+de+la+statistique+linguistique%2E)
15. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring Mathematical Problem Solving With the MATH Dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). NeurIPS.
16. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2020. [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://md.research.elamur.ai/free/en/papers/jin-2020-disease-this-patient-have-large/). arXiv preprint arXiv:2009.13081.
17. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. [PubMedQA: A Dataset for Biomedical Research Question Answering](https://md.research.elamur.ai/free/en/papers/jin-2019-pubmedqa-dataset-biomedical-research-question/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce…
18. A Johnson, L Bulgarelli, T Pollard, B Gow, B Moody, S Horng, LA Celi, and R Mark. 2024. MIMIC-IV (Version 3.1). PhysioNet. RRID: SCR_007345. [find](https://scholar.google.com/scholar?q=A+Johnson%2C+L+Bulgarelli%2C+T+Pollard%2C+B+Gow%2C+B+Moody%2C+S+Horng%2C+LA+Celi%2C+and+R+Mark%2E+2024%2E+MIMIC%2DIV+%28Version+3%2E1%29%2E+PhysioNet%2E+RRID%3A+SCR%5F007345%2E)
19. Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Leo Anthony Celi, Roger Mark, and Steven Horng. 2023a. MIMIC-IV-ED v2.2. PhysioNet. [find](https://scholar.google.com/scholar?q=Alistair+Johnson%2C+Lucas+Bulgarelli%2C+Tom+Pollard%2C+Leo+Anthony+Celi%2C+Roger+Mark%2C+and+Steven+Horng%2E+2023a%2E+MIMIC%2DIV%2DED+v2%2E2%2E+PhysioNet%2E)
20. Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, et al. 2023b. [MIMIC-IV, a freely accessible electronic health record dataset](https://md.research.elamur.ai/free/en/papers/johnson-2023-mimic-iv-freely-accessible-electronic/). Scientific data, 10(1):1.
21. Nikhil Khandekar, Qiao Jin, Guangzhi Xiong, Soren Dunn, Serina Applebaum, Zain Anwar, Maame Sarfo-Gyamfi, Conrad Safranek, Abid Anwar, Andrew Zhang, et al. 2024. MedCalc-Bench: Evaluating Large Language Models for Medical Calculations. Advances in Neural Information Processing Systems, 37:84730–8474… [find](https://scholar.google.com/scholar?q=Nikhil+Khandekar%2C+Qiao+Jin%2C+Guangzhi+Xiong%2C+Soren+Dunn%2C+Serina+Applebaum%2C+Zain+Anwar%2C+Maame+Sarfo%2DGyamfi%2C+Conrad+Safranek%2C+Abid+Anwar%2C+Andrew+Zhang%2C+et+al%2E+2024%2E+MedCalc%2DBench%3A+Evaluating+Large+Langua)
22. Haoyang Li, Xuejia Chen, Zhanchao Xu, Darian Li, Nicole Hu, Fei Teng, Yiming Li, Luyu Qiu, Chen Jason Zhang, Li Qing, et al. 2025. Exposing Numeracy Gaps: A Benchmark to Evaluate Fundamental Numerical Abilities in Large Language Models. In Findings of the Association for Computational Linguistics: A… [find](https://scholar.google.com/scholar?q=Haoyang+Li%2C+Xuejia+Chen%2C+Zhanchao+Xu%2C+Darian+Li%2C+Nicole+Hu%2C+Fei+Teng%2C+Yiming+Li%2C+Luyu+Qiu%2C+Chen+Jason+Zhang%2C+Li+Qing%2C+et+al%2E+2025%2E+Exposing+Numeracy+Gaps%3A+A+Benchmark+to+Evaluate+Fundamental+Numerical)
23. Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. 2024. [GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers](https://md.research.elamur.ai/free/en/papers/li-2024-gsm-plus-comprehensive-benchmark-evaluating/). ArXiv, abs/2402.19255.
24. Zefei Long, Zhenbiao Cao, Wei Chen, and Zhongyu Wei. 2025. EMGLLM: Data-to-Text Alignment for Electromyogram Diagnosis Generation with Medical Numerical Data Encoding. In Findings of the Association for Computational Linguistics: ACL 2025, pages 20470–20480. [find](https://scholar.google.com/scholar?q=Zefei+Long%2C+Zhenbiao+Cao%2C+Wei+Chen%2C+and+Zhongyu+Wei%2E+2025%2E+EMGLLM%3A+Data%2Dto%2DText+Alignment+for+Electromyogram+Diagnosis+Generation+with+Medical+Numerical+Data+Encoding%2E+In+Findings+of+the+Association+f)
25. Rahmad Mahendra, Damiano Spina, Lawrence Cavedon, and Karin Verspoor. 2024. Do numbers matter? Types and prevalence of numbers in clinical texts. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 409–415. [find](https://scholar.google.com/scholar?q=Rahmad+Mahendra%2C+Damiano+Spina%2C+Lawrence+Cavedon%2C+and+Karin+Verspoor%2E+2024%2E+Do+numbers+matter%3F+Types+and+prevalence+of+numbers+in+clinical+texts%2E+In+Proceedings+of+the+23rd+Workshop+on+Biomedical+Natu)
26. Rahmad Mahendra, Damiano Spina, Lawrence Cavedon, and Karin Verspoor. 2025. [Evaluating Numeracy of Language Models as a Natural Language Inference Task](https://md.research.elamur.ai/free/en/papers/mahendra-2025-evaluating-numeracy-language-models-as/). In Findings of the Association for Computational Linguistics: NAACL 2025, pages 8336– 8361.
27. David Malvern, Brian Richards, Ngoni Chipere, and Pilar Durán. 2004. Lexical Diversity and Language Development. Springer. [find](https://scholar.google.com/scholar?q=David+Malvern%2C+Brian+Richards%2C+Ngoni+Chipere%2C+and+Pilar+Dur%C3%A1n%2E+2004%2E+Lexical+Diversity+and+Language+Development%2E+Springer%2E)
28. Philip M McCarthy and Scott Jarvis. 2010. MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior research methods, 42(2):381–392. [find](https://scholar.google.com/scholar?q=Philip+M+McCarthy+and+Scott+Jarvis%2E+2010%2E+MTLD%2C+vocd%2DD%2C+and+HD%2DD%3A+A+validation+study+of+sophisticated+approaches+to+lexical+diversity+assessment%2E+Behavior+research+methods%2C+42%282%29%3A381%E2%80%93392%2E)
29. Shreshth Mehrotra, Raghavendra P, Balraj Prajesh, Hrishikesh Kambale, and Puspita Majumdar. 2025. Unmasking Bias in Financial AI: A Robust Framework for Evaluating and Mitigating Hidden Biases in LLMs. Proceedings of the 6th ACM International Conference on AI in Finance. [find](https://scholar.google.com/scholar?q=Shreshth+Mehrotra%2C+Raghavendra+P%2C+Balraj+Prajesh%2C+Hrishikesh+Kambale%2C+and+Puspita+Majumdar%2E+2025%2E+Unmasking+Bias+in+Financial+AI%3A+A+Robust+Framework+for+Evaluating+and+Mitigating+Hidden+Biases+in+LLMs)
30. Robert Osazuwa Ness, Katie Matton, Hayden S. Helm, Sheng Zhang, Junaid Bajwa, Carey E. Priebe, and Eric Horvitz. 2024. MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answering. ArXiv, abs/2406.06573. — [arXiv:2406.06573](https://arxiv.org/abs/2406.06573) [find](https://scholar.google.com/scholar?q=Robert+Osazuwa+Ness%2C+Katie+Matton%2C+Hayden+S%2E+Helm%2C+Sheng+Zhang%2C+Junaid+Bajwa%2C+Carey+E%2E+Priebe%2C+and+Eric+Horvitz%2E+2024%2E+MedFuzz%3A+Exploring+the+Robustness+of+Large+Language+Models+in+Medical+Question+An)
31. Tuan Dung Nguyen, Thanh Trung Huynh, Minh Hieu Phan, Quoc Viet Hung Nguyen, and Phi Le Nguyen. 2024. CARER-ClinicAl Reasoning-Enhanced Representation for Temporal Health Risk Prediction. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10392–10407. [find](https://scholar.google.com/scholar?q=Tuan+Dung+Nguyen%2C+Thanh+Trung+Huynh%2C+Minh+Hieu+Phan%2C+Quoc+Viet+Hung+Nguyen%2C+and+Phi+Le+Nguyen%2E+2024%2E+CARER%2DClinicAl+Reasoning%2DEnhanced+Representation+for+Temporal+Health+Risk+Prediction%2E+In+Proceeding)
32. Benjamin Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, Iain Marshall, Ani Nenkova, and Byron C Wallace. 2018. [A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature](https://md.research.elamur.ai/free/en/papers/nye-2018-corpus-multi-level-annotations-patients/). In Proceedings of the 56th Annual Meeting of the Associat…
33. Juliana Damasio Oliveira, Henrique DP Santos, Ana Helena DPS Ulbrich, Julia Colleoni Couto, Marcelo Arocha, Joaquim Santos, Manuela Martins Costa, Daniela Faccio, Fabio O Tabalipa, and Rodrigo F Nogueira. 2025. Development and evaluation of a clinical note summarization system using large language m… [find](https://scholar.google.com/scholar?q=Juliana+Damasio+Oliveira%2C+Henrique+DP+Santos%2C+Ana+Helena+DPS+Ulbrich%2C+Julia+Colleoni+Couto%2C+Marcelo+Arocha%2C+Joaquim+Santos%2C+Manuela+Martins+Costa%2C+Daniela+Faccio%2C+Fabio+O+Tabalipa%2C+and+Rodrigo+F+Nogue)
34. Kirk Roberts, Dina Demner-Fushman, Ellen M Voorhees, Steven Bedrick, and William R Hersh. 2022. Overview of the TREC 2022 Clinical Trials Track. In TREC. [find](https://scholar.google.com/scholar?q=Kirk+Roberts%2C+Dina+Demner%2DFushman%2C+Ellen+M+Voorhees%2C+Steven+Bedrick%2C+and+William+R+Hersh%2E+2022%2E+Overview+of+the+TREC+2022+Clinical+Trials+Track%2E+In+TREC%2E)
35. Russell L. Rothman, Victor M. Montori, Andrea L Cherrington, and Michael Pignone. 2008. Perspective: The Role of Numeracy in Health Care. Journal of Health Communication, 13:583 – 595. [find](https://scholar.google.com/scholar?q=Russell+L%2E+Rothman%2C+Victor+M%2E+Montori%2C+Andrea+L+Cherrington%2C+and+Michael+Pignone%2E+2008%2E+Perspective%3A+The+Role+of+Numeracy+in+Health+Care%2E+Journal+of+Health+Communication%2C+13%3A583+%E2%80%93+595%2E)
36. Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, et al. 2025. MedGemma Technical Report. arXiv preprint arXiv:2507.05201. [find](https://scholar.google.com/scholar?q=Andrew+Sellergren%2C+Sahar+Kazemzadeh%2C+Tiam+Jaroensri%2C+Atilla+Kiraly%2C+Madeleine+Traverse%2C+Timo+Kohlberger%2C+Shawn+Xu%2C+Fayaz+Jamil%2C+C%C3%ADan+Hughes%2C+Charles+Lau%2C+et+al%2E+2025%2E+MedGemma+Technical+Report%2E+arXiv+)
37. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Jun-Mei Song, Mingchuan Zhang, Y. K. Li, Yu Wu, and Daya Guo. 2024. [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). ArXiv, abs/2402.03300.
38. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. 2025. Gemma 3 Technical Report. arXiv preprint arXiv:2503.19786. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+et+al%2E+2025%2E+Gemma+3+Technical+Report%2E+ar)
39. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. [Qwen3 Technical Report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
40. Kaiyan Zhang, Sihang Zeng, Ermo Hua, Ning Ding, Zhang-Ren Chen, Zhiyuan Ma, Haoxin Li, Ganqu Cui, Biqing Qi, Xuekai Zhu, et al. 2024. UltraMedical: Building Specialized Generalists in Biomedicine. Advances in Neural Information Processing Systems, 37:26045–26081. [find](https://scholar.google.com/scholar?q=Kaiyan+Zhang%2C+Sihang+Zeng%2C+Ermo+Hua%2C+Ning+Ding%2C+Zhang%2DRen+Chen%2C+Zhiyuan+Ma%2C+Haoxin+Li%2C+Ganqu+Cui%2C+Biqing+Qi%2C+Xuekai+Zhu%2C+et+al%2E+2024%2E+UltraMedical%3A+Building+Specialized+Generalists+in+Biomedicine%2E+Adv)
41. Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/).
42. Jingming Zhuo, Songyang Zhang, Xinyu Fang, Haodong Duan, Dahua Lin, and Kai Chen. 2024. ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs. In Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Jingming+Zhuo%2C+Songyang+Zhang%2C+Xinyu+Fang%2C+Haodong+Duan%2C+Dahua+Lin%2C+and+Kai+Chen%2E+2024%2E+ProSA%3A+Assessing+and+Understanding+the+Prompt+Sensitivity+of+LLMs%2E+In+Conference+on+Empirical+Methods+in+Natural)
43. Yuxin Zuo, Shang Qu, Yifei Li, Zhangren Chen, Xuekai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. 2025. MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding. [find](https://scholar.google.com/scholar?q=Yuxin+Zuo%2C+Shang+Qu%2C+Yifei+Li%2C+Zhangren+Chen%2C+Xuekai+Zhu%2C+Ermo+Hua%2C+Kaiyan+Zhang%2C+Ning+Ding%2C+and+Bowen+Zhou%2E+2025%2E+MedXpertQA%3A+Benchmarking+Expert%2DLevel+Medical+Reasoning+and+Understanding%2E)

*Updated 1 November 2026*