---
title: "Evaluation of GPT-3.5 and GPT-4 for supporting real-world"
description: "The work offers an early template for evaluating LLMs at the point of care. 35 references, cited by 11 papers here."
authors: ["Debadutta Dash", "Rahul Thapa", "Juan M. Banda", "Akshay Swaminathan", "Morgan Cheatham", "Mehr Kashyap", "Nikesh Kotecha", "Jonathan H. Chen", "Saurabh Gombar", "Lance Downing", "Rachel Pedreira", "Ethan Goh", "Angel Arnaout", "Garret Kenn Morris", "Honor Magon", "Matthew P Lungren", "Eric Horvitz", "Nigam H. Shah"]
year: 2023
venue: "arXiv:2304.13714"
source: "https://arxiv.org/abs/2304.13714"
doi: "10.48550/arxiv.2304.13714"
arxiv: "2304.13714"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/dash-2023-evaluation-gpt-3-5-gpt/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dash-2023-evaluation-gpt-3-5-gpt/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/11/) / Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery

# Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery

**Authors:** Debadutta Dash, [Rahul Thapa](https://md.research.elamur.ai/free/en/authors/rahul-thapa/), Juan M. Banda, [Akshay Swaminathan](https://md.research.elamur.ai/free/en/authors/akshay-swaminathan/), [Morgan Cheatham](https://md.research.elamur.ai/free/en/authors/morgan-cheatham/), Mehr Kashyap, Nikesh Kotecha, Jonathan H. Chen, Saurabh Gombar, Lance Downing, Rachel Pedreira, Ethan Goh, Angel Arnaout, Garret Kenn Morris, Honor Magon, Matthew P Lungren, [Eric Horvitz](https://md.research.elamur.ai/free/en/authors/eric-horvitz/) and Nigam H. Shah · **Year:** 2023 · **Venue:** arXiv:2304.13714 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2304.13714) · [PDF](https://arxiv.org/pdf/2304.13714) · [DOI 10.48550/arxiv.2304.13714](https://doi.org/10.48550/arxiv.2304.13714) · [arXiv 2304.13714](https://arxiv.org/abs/2304.13714) · [OpenAlex](https://openalex.org/W4367191144)

**Citations:** 33 citations (OpenAlex, 2026-10-14) · [11](https://md.research.elamur.ai/free/en/papers/dash-2023-evaluation-gpt-3-5-gpt/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most tests of medical LLMs rely on exam-style benchmarks, which say little about real clinical use. This study asks whether GPT-3.5 and GPT-4 can safely and usefully answer questions physicians actually raised during patient care. The authors drew on questions sent to a Stanford informatics consultation service between 2017 and 2019. That service answered each question by analysing de-identified records of similar patients. After filtering, 66 questions remained. Each was sent to both models with only a short role-assigning prompt. Twelve physicians then judged every response on two criteria: whether it could harm a patient, and whether it matched the consult report's conclusion. A response counted as harmful or concordant only if more than six reviewers agreed. By majority vote, no response from either model was judged harmful. However, agreement with the consult reports was low, some responses cited invented references, and reviewers often reached no majority. The work offers an early template for evaluating LLMs at the point of care. It also suggests that grounding, prompt design and local adaptation are still required.

## Key points
- **Question set:** 154 consult questions (Feb 2017 – Aug 2019) were filtered down to 66. Excluded were questions lacking a clear PICOT form, questions tied specifically to Stanford patients, duplicates, and questions without a usable report.
- **Harm:** By majority vote (>6 of 12 physicians), harmful responses were 0/66 for both GPT-3.5 and GPT-4. Inter-rater agreement on harm was very low (Fleiss kappa 0.0926).
- **Concordance with consult reports:**
  - GPT-3.5: 8 agree, 20 disagree, 9 unable to assess, 29 with no majority.
  - GPT-4: 13 agree, 15 disagree, 3 unable to assess, 35 with no majority.
  - Agreement therefore rose by about 63% and disagreement fell by 25%.
- **Hallucinated references:** GPT-3.5 produced 2 and GPT-4 produced 9. Using Bing Chat's 'More Precise' mode yielded answers without these fabricated citations.
- **Limitations:** The question set is small and skewed toward internal medicine, dermatology and descriptive questions. The local EHR-based reports were treated as ground truth, which the models had no access to.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 567 works · 335,189 citations
- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 404 works · 163,863 citations
- [Brown University](https://md.research.elamur.ai/free/en/universities/brown-university/) — University, Providence, United States · 85 works · 26,322 citations
- [University of Ottawa](https://md.research.elamur.ai/free/en/universities/university-of-ottawa/) — University, Ottawa, Canada · 14 works · 2,558 citations
- [Stanford Medicine](https://md.research.elamur.ai/free/en/hospitals/stanford-medicine/) — Hospital, Stanford, United States · 6 works · 1,507 citations

Also on the byline: [Stanford Health Care](https://ror.org/019wqcg20).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 288 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 227 works · [🏥 Health Professions](https://md.research.elamur.ai/free/en/shelves/health-professions/) · 76 works · [🏥 Health Information Management](https://md.research.elamur.ai/free/en/shelves/health-professions/health-information-management/) · 57 works

🧭 Topics (1): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,394 works

## Cite

Cite as: Debadutta Dash, Rahul Thapa, Juan M. Banda, Akshay Swaminathan, Morgan Cheatham, Mehr Kashyap, Nikesh Kotecha, Jonathan H. Chen, Saurabh Gombar, Lance Downing, Rachel Pedreira, Ethan Goh, Angel Arnaout, Garret Kenn Morris, Honor Magon, Matthew P Lungren, Eric Horvitz and Nigam H. Shah. 2023. Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery. arXiv:2304.13714.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{dash2023evaluation,
  title = {{Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery}},
  author = {Debadutta Dash, Rahul Thapa, Juan M. Banda, Akshay Swaminathan, Morgan Cheatham, Mehr Kashyap, Nikesh Kotecha, Jonathan H. Chen, Saurabh Gombar, Lance Downing, Rachel Pedreira, Ethan Goh, Angel Arnaout, Garret Kenn Morris, Honor Magon, Matthew P Lungren, Eric Horvitz and Nigam H. Shah},
  year = {2023},
  doi = {10.48550/arxiv.2304.13714},
  eprint = {2304.13714},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2304.13714}
}
```

- [← previous Understanding Catastrophic Forgetting in Language Models via Implicit Inference](https://md.research.elamur.ai/free/en/papers/kotha-2023-understanding-catastrophic-forgetting-language-models/)
- [next → Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning](https://md.research.elamur.ai/free/en/papers/shi-2023-language-models-improve-event-prediction/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/11/)

## References

35 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sevilla J, Heim L, Ho A, Besiroglu T, Hobbhahn M, Villalobos P. Compute Trends Across Three Eras of Machine Learning. arXiv [csLG]. Published online February 11, 2022. http://arxiv.org/abs/2202.05924 — [arXiv:2202.05924](https://arxiv.org/abs/2202.05924) [↗](http://arxiv.org/abs/2202.05924)
2. Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, et al. [On the Opportunities and Risks of Foundation Models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/). arXiv [csLG]. Published online August 16, 2021. http://arxiv.org/abs/2108.07258 [↗](http://arxiv.org/abs/2108.07258)
3. OpenAI. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv [csCL]. Published online March 15, 2023. http://arxiv.org/abs/2303.08774 [↗](http://arxiv.org/abs/2303.08774)
4. Ramesh A, Dhariwal P, Nichol A, Chu C, Chen M. [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://md.research.elamur.ai/free/en/papers/ramesh-2022-hierarchical-text-conditional-image-generation/). arXiv [csCV]. Published online April 13, 2022. http://arxiv.org/abs/2204.06125 [↗](http://arxiv.org/abs/2204.06125)
5. Zhou C, Li Q, Li C, Yu J, Liu Y, Wang G, et al. A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT. arXiv [csAI]. Published online February 18, 2023. http://arxiv.org/abs/2302.09419 [↗](http://arxiv.org/abs/2302.09419)
6. Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Adv Neural Inf Process Syst. 2020;33:1877-1901.
7. Zhang T, Ladhak F, Durmus E, Liang P, McKeown K, Hashimoto TB. [Benchmarking Large Language Models for News Summarization](https://md.research.elamur.ai/free/en/papers/zhang-2024-benchmarking-large-language-models-news/). arXiv [csCL]. Published online January 31, 2023. http://arxiv.org/abs/2301.13848 [↗](http://arxiv.org/abs/2301.13848)
8. Vilar D, Freitag M, Cherry C, Luo J, Ratnakar V, Foster G. Prompting PaLM for Translation: Assessing Strategies and Performance. arXiv [csCL]. Published online November 16, 2022. http://arxiv.org/abs/2211.09102 [↗](http://arxiv.org/abs/2211.09102)
9. Jumper J, Evans R, Pritzel A, Green T, Figurnov M, Ronneberger O, et al. [Highly accurate protein structure prediction with AlphaFold](https://md.research.elamur.ai/free/en/papers/jumper-2021-highly-accurate-protein-structure-prediction/). Nature. 2021;596(7873):583-589.
10. Vemprala S, Bonatti R, Bucker A, Kapoor A. Chatgpt for robotics: Design principles and model abilities. Published online 2023. [https://www.microsoft.com/en-us/research/uploads/prod/2023/02/ChatGPT___Robotics.pdf](https://www.microsoft.com/en-us/research/uploads/prod/2023/02/ChatGPT___Robotics.pdf) [↗](https://www.microsoft.com/en-us/research/uploads/prod/2023/02/ChatGPT___Robotics.pdf)
11. Castro Nascimento CM, Pimentel AS. Do Large Language Models Understand Chemistry? A Conversation with ChatGPT. J Chem Inf Model. 2023;63(6):1649-1655. [find](https://scholar.google.com/scholar?q=Castro+Nascimento+CM%2C+Pimentel+AS%2E+Do+Large+Language+Models+Understand+Chemistry%3F+A+Conversation+with+ChatGPT%2E+J+Chem+Inf+Model%2E+2023%3B63%286%29%3A1649%2D1655%2E)
12. Wornow M, Xu Y, Thapa R, Patel B, Steinberg E, Fleming S, et al. The Shaky Foundations of Clinical Foundation Models: [A Survey of Large Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/) and Foundation Models for EMRs. arXiv [csLG]. Published online March 22, 2023. http://arxiv.org/abs/2303.12961 — [arXiv:2303.12961](https://arxiv.org/abs/2303.12961) [↗](http://arxiv.org/abs/2303.12961)
13. Eysenbach G. The Role of ChatGPT, Generative Language Models, and Artificial Intelligence in Medical Education: A Conversation With ChatGPT and a Call for Papers. JMIR Med Educ. 2023;9:e46885. [find](https://scholar.google.com/scholar?q=Eysenbach+G%2E+The+Role+of+ChatGPT%2C+Generative+Language+Models%2C+and+Artificial+Intelligence+in+Medical+Education%3A+A+Conversation+With+ChatGPT+and+a+Call+for+Papers%2E+JMIR+Med+Educ%2E+2023%3B9%3Ae46885%2E)
14. Jin D, Pan E, Oufattole N, Weng WH, Fang H, Szolovits P. [What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical Exams](https://md.research.elamur.ai/free/en/papers/jin-2020-disease-this-patient-have-large/). NATO Adv Sci Inst Ser E Appl Sci. 2021;11(14):6421.
15. Liévin V, Hother CE, Winther O. Can large language models reason about medical questions? arXiv [csCL]. Published online July 17, 2022. http://arxiv.org/abs/2207.08143 [↗](http://arxiv.org/abs/2207.08143)
16. Singhal K, Azizi S, Tu T, Sara Mahdavi S, Wei J, Chung HW, et al. [Large Language Models Encode Clinical Knowledge](https://md.research.elamur.ai/free/en/papers/singhal-2022-large-language-models-encode-clinical/). arXiv [csCL]. Published online December 26, 2022. http://arxiv.org/abs/2212.13138 [↗](http://arxiv.org/abs/2212.13138)
17. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. [Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models](https://md.research.elamur.ai/free/en/papers/kung-2023-performance-chatgpt-usmle-potential-ai/). PLOS Digit Health. 2023;2(2):e0000198.
18. Jeblick K, Schachtner B, Dexl J, Mittermeier A, Stüber AT, Topalis J, et al. ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reports. arXiv [csCL]. Published online December 30, 2022. http://arxiv.org/abs/2212.14882 — [arXiv:2212.14882](https://arxiv.org/abs/2212.14882) [↗](http://arxiv.org/abs/2212.14882)
19. Macdonald C, Adeloye D, Sheikh A, Rudan I. Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis. J Glob Health. 2023;13:01003. [find](https://scholar.google.com/scholar?q=Macdonald+C%2C+Adeloye+D%2C+Sheikh+A%2C+Rudan+I%2E+Can+ChatGPT+draft+a+research+article%3F+An+example+of+population%2Dlevel+vaccine+effectiveness+analysis%2E+J+Glob+Health%2E+2023%3B13%3A01003%2E)
20. Adamson B, Waskom M, Blarre A, Kelly J, Krismer K, Nemeth S, et al. Approach to machine learning for extraction of real-world data variables from electronic health records. bioRxiv. Published online March 6, 2023. doi:10.1101/2023.03.02.23286522 — [doi:10.1101/2023.03.02.23286522](https://doi.org/10.1101/2023.03.02.23286522) [find](https://scholar.google.com/scholar?q=Adamson+B%2C+Waskom+M%2C+Blarre+A%2C+Kelly+J%2C+Krismer+K%2C+Nemeth+S%2C+et+al%2E+Approach+to+machine+learning+for+extraction+of+real%2Dworld+data+variables+from+electronic+health+records%2E+bioRxiv%2E+Published+online+M)
21. Raji D, Denton E, Bender EM, Hanna A, Paullada A. AI and the Everything in the Whole Wide World Benchmark. In: Vanschoren J, Yeung S, eds. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. Vol 1. Curran; 2021. [https://datasets-benchmarks-proceedings.neurips.c…](https://datasets-benchmarks-proceedings.neurips.c…) [↗](https://datasets-benchmarks-proceedings.neurips.c…)
22. Narayanan A, Kapoor S. GPT-4 and professional benchmarks: the wrong answer to the wrong question. AI Snake Oil. Published March 20, 2023. [https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks](https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks). Accessed March 29, 2023 [↗](https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks)
23. Wang A, Pruksachatkun Y, Nangia N, Singh A, Michael J, Hill F, et al. [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://md.research.elamur.ai/free/en/papers/wang-2019-superglue-stickier-benchmark-general-purpose/). arXiv [csCL]. Published online May 2, 2019. http://arxiv.org/abs/1905.00537 [↗](http://arxiv.org/abs/1905.00537)
24. Srivastava A, Rastogi A, Rao A, Shoeb AAM, Abid A, Fisch A, et al. [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). arXiv [csCL]. Published online June 9, 2022. http://arxiv.org/abs/2206.04615 [↗](http://arxiv.org/abs/2206.04615)
25. Gao L, Tow J, Biderman S, Lovering C, Phang J, Thite A, et al. EleutherAI/lm-Evaluation-Harness: v0.3.0.; 2022. doi:10.5281/zenodo.7413426 — [doi:10.5281/zenodo.7413426](https://doi.org/10.5281/zenodo.7413426) [find](https://scholar.google.com/scholar?q=Gao+L%2C+Tow+J%2C+Biderman+S%2C+Lovering+C%2C+Phang+J%2C+Thite+A%2C+et+al%2E+EleutherAI%2Flm%2DEvaluation%2DHarness%3A+v0%2E3%2E0%2E%3B+2022%2E+doi%3A10%2E5281%2Fzenodo%2E7413426+%E2%80%94+%5Bdoi%3A10%2E5281%2Fzenodo%2E7413426%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E5281%2Fzenodo)
26. Mrabet Y, Demner-Fushman D. HOLMS: Alternative Summary Evaluation with Large Language Models. In: Proceedings of the 28th International Conference on Computational Linguistics. International Committee on Computational Linguistics; 2020:5679-5688. [find](https://scholar.google.com/scholar?q=Mrabet+Y%2C+Demner%2DFushman+D%2E+HOLMS%3A+Alternative+Summary+Evaluation+with+Large+Language+Models%2E+In%3A+Proceedings+of+the+28th+International+Conference+on+Computational+Linguistics%2E+International+Committee)
27. Liang P, Bommasani R, Lee T, Tsipras D, Soylu D, Yasunaga M, et al. Holistic Evaluation of Language Models. arXiv [csCL]. Published online November 16, 2022. http://arxiv.org/abs/2211.09110 [↗](http://arxiv.org/abs/2211.09110)
28. Lee P, Bubeck S, Petro J, Eng. M. Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine. [find](https://scholar.google.com/scholar?q=Lee+P%2C+Bubeck+S%2C+Petro+J%2C+Eng%2E+M%2E+Benefits%2C+Limits%2C+and+Risks+of+GPT%2D4+as+an+AI+Chatbot+for+Medicine%2E)
29. Verghese A, Shah NH, Harrington RA. What This Computer Needs Is a Physician: Humanism and Artificial Intelligence. JAMA. 2018;319(1):19-20. [find](https://scholar.google.com/scholar?q=Verghese+A%2C+Shah+NH%2C+Harrington+RA%2E+What+This+Computer+Needs+Is+a+Physician%3A+Humanism+and+Artificial+Intelligence%2E+JAMA%2E+2018%3B319%281%29%3A19%2D20%2E)
30. Nori H, King N, McKinney SM, Carignan D, Horvitz E. [Capabilities of GPT-4 on Medical Challenge Problems](https://md.research.elamur.ai/free/en/papers/nori-2023-capabilities-gpt-4-medical-challenge/). arXiv [csCL]. Published online March 20, 2023. http://arxiv.org/abs/2303.13375 [↗](http://arxiv.org/abs/2303.13375)
31. Callahan Alison, Gombar Saurabh, Cahan Eli M., Jung Kenneth, Steinberg Ethan, Polony Vladimir, et al. Using Aggregate Patient Data at the Bedside via an On-Demand Consultation Service. NEJM Catalyst. 2(10). doi:10.1056/CAT.21.0224 — [doi:10.1056/cat.21.0224](https://doi.org/10.1056/cat.21.0224) [find](https://scholar.google.com/scholar?q=Callahan+Alison%2C+Gombar+Saurabh%2C+Cahan+Eli+M%2E%2C+Jung+Kenneth%2C+Steinberg+Ethan%2C+Polony+Vladimir%2C+et+al%2E+Using+Aggregate+Patient+Data+at+the+Bedside+via+an+On%2DDemand+Consultation+Service%2E+NEJM+Catalyst%2E+)
32. Callahan Alison, Gombar Saurabh, Cahan Eli M., Jung Kenneth, Steinberg Ethan, Polony Vladimir, et al. Using Aggregate Patient Data at the Bedside via an On-Demand Consultation Service. NEJM Catalyst. 2(10). doi:10.1056/CAT.21.0224 — [doi:10.1056/cat.21.0224](https://doi.org/10.1056/cat.21.0224) [find](https://scholar.google.com/scholar?q=Callahan+Alison%2C+Gombar+Saurabh%2C+Cahan+Eli+M%2E%2C+Jung+Kenneth%2C+Steinberg+Ethan%2C+Polony+Vladimir%2C+et+al%2E+Using+Aggregate+Patient+Data+at+the+Bedside+via+an+On%2DDemand+Consultation+Service%2E+NEJM+Catalyst%2E+)
33. Riva JJ, Malik KMP, Burnie SJ, Endicott AR, Busse JW. What is your research question? An introduction to the PICOT format for clinicians. J Can Chiropr Assoc. 2012;56(3):167-171. [find](https://scholar.google.com/scholar?q=Riva+JJ%2C+Malik+KMP%2C+Burnie+SJ%2C+Endicott+AR%2C+Busse+JW%2E+What+is+your+research+question%3F+An+introduction+to+the+PICOT+format+for+clinicians%2E+J+Can+Chiropr+Assoc%2E+2012%3B56%283%29%3A167%2D171%2E)
34. How Well Do Large Language Models Support Clinician Information Needs? Stanford HAI. [https://hai.stanford.edu/news/how-well-do-large-language-models-support-clinician-informat](https://hai.stanford.edu/news/how-well-do-large-language-models-support-clinician-informat) ion-needs. Accessed April 18, 2023 [↗](https://hai.stanford.edu/news/how-well-do-large-language-models-support-clinician-informat)
35. Qiu J, Li L, Sun J, Peng J, Shi P, Zhang R, et al. Large AI Models in Health Informatics: Applications, Challenges, and the Future. Published online March 21, 2023. http://arxiv.org/abs/2303.11568. Accessed April 24, 2023 — [arXiv:2303.11568](https://arxiv.org/abs/2303.11568) [↗](http://arxiv.org/abs/2303.11568)

*Updated 3 November 2026*