---
title: "Benchmarking and Mitigating the Impact of Noisy User Prompts"
description: "As a remedy, the authors propose SFT-R, a supervised fine-tuning scheme with cross-modal reflection. 38 references."
authors: ["Zhiyu Xue", "Reza Abbasi-Asl", "Ramtin Pedarsani"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)"
source: "https://aclanthology.org/2026.eacl-industry.67/"
doi: "10.18653/v1/2026.eacl-industry.67"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xue-2026-benchmarking-mitigating-impact-noisy-user/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xue-2026-benchmarking-mitigating-impact-noisy-user/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/) / Benchmarking and Mitigating the Impact of Noisy User Prompts in Medical VLMs via Cross-Modal Reflection

# Benchmarking and Mitigating the Impact of Noisy User Prompts in Medical VLMs via Cross-Modal Reflection

**Authors:** [Zhiyu Xue](https://md.research.elamur.ai/free/en/authors/zhiyu-xue/), [Reza Abbasi-Asl](https://md.research.elamur.ai/free/en/authors/reza-abbasi-asl/) and [Ramtin Pedarsani](https://md.research.elamur.ai/free/en/authors/ramtin-pedarsani/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-industry.67/) · [PDF](https://aclanthology.org/2026.eacl-industry.67.pdf) · [DOI 10.18653/v1/2026.eacl-industry.67](https://doi.org/10.18653/v1/2026.eacl-industry.67) · [OpenAlex](https://openalex.org/W7140117659)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Medical vision-language models (Med-VLMs) are often given user-supplied clinical hints, but it was unclear how they react when those hints are wrong. The authors built Med-CP, a multiple-choice VQA benchmark derived from OmniMedVQA. Each question comes with a structured user prompt that states a confidence level, a preferred answer and supporting evidence. A prompt counts as noisy when the preferred answer differs from the ground truth, and every prompt is also rewritten in four writing styles that imitate different clinicians. Tests on open, medical and closed models show that they largely defer to whatever the user suggests: accuracy rises with correct hints and falls sharply with incorrect ones. Larger models, medical fine-tuning, RL-trained reasoning and inference-time CoT do not solve this. As a remedy, the authors propose SFT-R, a supervised fine-tuning scheme with cross-modal reflection. The model writes separate reasoning about the image and about the prompt, then reconciles the two before answering. This gives clear robustness gains, especially on out-of-domain data, which matters for safe clinical use.

## Key points
- **Med-CP benchmark:** built from 43 public OmniMedVQA datasets, giving 89,727 VQA pairs. A compact version, Med-CP-Small, has 407 items. HuatuoGPTV-7B generated the evidence and GPT-4o produced the style rewrites.
- **MedGemma-4B on lesion grading:** accuracy fell from 47% to 0% under noisy prompts. Adding evidence to a prompt made its influence stronger.
- **Vulnerability across model types:**
  - MedVLM-R1 dropped from 72.72 to 33.16.
  - GPT-4o dropped from 82.55 to 71.01.
  - Qwen2.5VL-32B was no more robust than Qwen2.5VL-7B.
- **Plain SFT on clinical prompts:** the in-domain mean rose from 44.75 to 91.18. However, the out-of-domain mean fell from 52.01 to 44.05, indicating overfitting.
- **SFT-R out-of-domain results:** the OOD mean reached 68.31; for example, BioMediTech scored 76.34 versus 46.39 for the base model. Code is available at github.com/chrisyxue/Med_CP.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 275 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 214 works

## Cite

Cite as: Zhiyu Xue, Reza Abbasi-Asl and Ramtin Pedarsani. 2026. Benchmarking and Mitigating the Impact of Noisy User Prompts in Medical VLMs via Cross-Modal Reflection. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xue2026benchmarking,
  title = {{Benchmarking and Mitigating the Impact of Noisy User Prompts in Medical VLMs via Cross-Modal Reflection}},
  author = {Zhiyu Xue, Reza Abbasi-Asl and Ramtin Pedarsani},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.67},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.67}
}
```

- [← previous Benchmark Data Contamination in Underrepresented Languages: A Comprehensive Analysis Using Brazilian Data](https://md.research.elamur.ai/free/en/papers/vilar-2026-benchmark-data-contamination-underrepresented-languages/)
- [next → Benchmarking Arabic Authorship Attribution and Style Transfer with Large Language Models](https://md.research.elamur.ai/free/en/papers/hamed-2026-benchmarking-arabic-authorship-attribution-style/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/)

## References

38 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Where we stand and what lies ahead. arXiv preprint arXiv:2504.00294. Junying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, and 1 others. 2024a. Huatuogpt-vision, towards injecting medical visual knowledge into multimodal… [find](https://scholar.google.com/scholar?q=Where+we+stand+and+what+lies+ahead%2E+arXiv+preprint+arXiv%3A2504%2E00294%2E+Junying+Chen%2C+Chi+Gui%2C+Ruyi+Ouyang%2C+Anningzhe+Gao%2C+Shunian+Chen%2C+Guiming+Hardy+Chen%2C+Xidong+Wang%2C+Ruifei+Zhang%2C+Zhenyang+Cai%2C+Ke+Ji)
3. Sizhe Chen, Arman Zharmagambetov, David Wagner, and Chuan Guo. 2025a. Meta secalign: A secure foundation llm against prompt injection attacks. arXiv preprint arXiv:2507.02735. — [arXiv:2507.02735](https://arxiv.org/abs/2507.02735) [find](https://scholar.google.com/scholar?q=Sizhe+Chen%2C+Arman+Zharmagambetov%2C+David+Wagner%2C+and+Chuan+Guo%2E+2025a%2E+Meta+secalign%3A+A+secure+foundation+llm+against+prompt+injection+attacks%2E+arXiv+preprint+arXiv%3A2507%2E02735%2E+%E2%80%94+%5BarXiv%3A2507%2E02735%5D%28htt)
4. Yulin Chen, Haoran Li, Yuan Sui, Yufei He, Yue Liu, Yangqiu Song, and Bryan Hooi. 2025b. [Can indirect prompt injection attacks be detected and removed?](https://md.research.elamur.ai/free/en/papers/chen-2025-can-indirect-prompt-injection-attacks/) arXiv preprint arXiv:2502.16580.
5. Zhihong Chen, Maya Varma, Jean-Benoit Delbrouck, Magdalini Paschali, Louis Blankemeier, Dave Van Veen, Jeya Maria Jose Valanarasu, Alaa Youssef, Joseph Paul Cohen, Eduardo Pontes Reis, and 1 others. 2024b. Chexagent: Towards a foundation model for chest x-ray interpretation. arXiv preprint arXiv:240… — [arXiv:2401.12208](https://arxiv.org/abs/2401.12208) [find](https://scholar.google.com/scholar?q=Zhihong+Chen%2C+Maya+Varma%2C+Jean%2DBenoit+Delbrouck%2C+Magdalini+Paschali%2C+Louis+Blankemeier%2C+Dave+Van+Veen%2C+Jeya+Maria+Jose+Valanarasu%2C+Alaa+Youssef%2C+Joseph+Paul+Cohen%2C+Eduardo+Pontes+Reis%2C+and+1+others%2E+2)
6. Jan Clusmann, Dyke Ferber, Isabella C Wiest, Carolin V Schneider, Titus J Brinker, Sebastian Foersch, Daniel Truhn, and Jakob Nikolas Kather. 2025. Prompt injection attacks on vision language models in oncology. Nature Communications, 16(1):1239. [find](https://scholar.google.com/scholar?q=Jan+Clusmann%2C+Dyke+Ferber%2C+Isabella+C+Wiest%2C+Carolin+V+Schneider%2C+Titus+J+Brinker%2C+Sebastian+Foersch%2C+Daniel+Truhn%2C+and+Jakob+Nikolas+Kather%2E+2025%2E+Prompt+injection+attacks+on+vision+language+models+i)
7. Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. 2024. Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents. Advances in Neural Information Processing Systems, 37:82895–82920. [find](https://scholar.google.com/scholar?q=Edoardo+Debenedetti%2C+Jie+Zhang%2C+Mislav+Balunovic%2C+Luca+Beurer%2DKellner%2C+Marc+Fischer%2C+and+Florian+Tram%C3%A8r%2E+2024%2E+Agentdojo%3A+A+dynamic+environment+to+evaluate+prompt+injection+attacks+and+defenses+for+ll)
8. Google Deepmind. 2025. Medgemma: A gemma 3 variant optimized for medical text and image comprehension. [https://deepmind.google/models/](https://deepmind.google/models/) gemma/medgemma/. Accessed: 2025-06-24. [↗](https://deepmind.google/models/)
9. Melody Y Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, and 1 others. 2024. [Deliberative alignment: Reasoning enables safer language models](https://md.research.elamur.ai/free/en/papers/guan-2024-deliberative-alignment-reasoning-enables-safer/). arXiv preprint arXiv:2412.16339.
10. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
11. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
12. Vidhisha Balachandran, Jingya Chen, Lingjiao Chen, Shivam Garg, Neel Joshi, Yash Lara, John Langford, Besmira Nushi, Vibhav Vineet, Yue Wu, and 1 others. 2025. [Inference-time scaling for complex tasks: Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizh…](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/)
13. Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He, Yu Qiao, and Ping Luo. 2024. Omnimedvqa: A new large-scale comprehensive evaluation benchmark for medical lvlm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22170–22183. [find](https://scholar.google.com/scholar?q=Yutao+Hu%2C+Tianbin+Li%2C+Quanfeng+Lu%2C+Wenqi+Shao%2C+Junjun+He%2C+Yu+Qiao%2C+and+Ping+Luo%2E+2024%2E+Omnimedvqa%3A+A+new+large%2Dscale+comprehensive+evaluation+benchmark+for+medical+lvlm%2E+In+Proceedings+of+the+IEEE%2FCVF)
14. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
15. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361.
16. Sanjay Subramanian, Lucy Lu Wang, Ben Bogin, Sachin Mehta, Madeleine van Zuylen, Sravanthi Parasa, Sameer Singh, Matt Gardner, and Hannaneh Hajishirzi. 2020. Medicat: A dataset of medical images, captions, and textual references. Findings of the Association for Computational Linguistics: EMNLP. [find](https://scholar.google.com/scholar?q=Sanjay+Subramanian%2C+Lucy+Lu+Wang%2C+Ben+Bogin%2C+Sachin+Mehta%2C+Madeleine+van+Zuylen%2C+Sravanthi+Parasa%2C+Sameer+Singh%2C+Matt+Gardner%2C+and+Hannaneh+Hajishirzi%2E+2020%2E+Medicat%3A+A+dataset+of+medical+images%2C+capt)
17. Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2024. [Llava-med: Training a large language-and-vision assistant for biomedicine in one day](https://md.research.elamur.ai/free/en/papers/li-2023-llava-med-training-large-language/). Advances in Neural Information Processing Systems, 36.
18. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805.
19. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024a. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306.
20. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024b. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). Advances in neural information processing systems, 36.
21. Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, and 1 others. 2023. [Prompt injection attack against llm-integrated applications](https://md.research.elamur.ai/free/en/papers/liu-2023-prompt-injection-attack-against-llm/). arXiv preprint arXiv:2306.05499.
22. Ming Y Lu, Bowen Chen, Drew FK Williamson, Richard J Chen, Melissa Zhao, Aaron K Chow, Kenji Ikemura, Ahrong Kim, Dimitra Pouli, Ankush Patel, and 1 others. 2024. A multimodal generative ai copilot for human pathology. Nature, 634(8033):466– 473. [find](https://scholar.google.com/scholar?q=Ming+Y+Lu%2C+Bowen+Chen%2C+Drew+FK+Williamson%2C+Richard+J+Chen%2C+Melissa+Zhao%2C+Aaron+K+Chow%2C+Kenji+Ikemura%2C+Ahrong+Kim%2C+Dimitra+Pouli%2C+Ankush+Patel%2C+and+1+others%2E+2024%2E+A+multimodal+generative+ai+copilot+fo)
23. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, and 1 others. 2024. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/). arXiv preprint arXiv:2403.05530.
24. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+and+1+others%2E+2025%2E+Gemma+3+technical+rep)
25. Omkar Thawkar, Abdelrahman Shaker, Sahal Shaji Mullappilly, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Jorma Laaksonen, and Fahad Shahbaz Khan. 2023. Xraygpt: Chest radiographs summarization using medical vision-language models. arXiv preprint arXiv:2306.07971. — [arXiv:2306.07971](https://arxiv.org/abs/2306.07971) [find](https://scholar.google.com/scholar?q=Omkar+Thawkar%2C+Abdelrahman+Shaker%2C+Sahal+Shaji+Mullappilly%2C+Hisham+Cholakkal%2C+Rao+Muhammad+Anwer%2C+Salman+Khan%2C+Jorma+Laaksonen%2C+and+Fahad+Shahbaz+Khan%2E+2023%2E+Xraygpt%3A+Chest+radiographs+summarization+u)
26. Minheng Ni, YuTao Fan, Lei Zhang, and Wangmeng Zuo. Visual-o1: Understanding ambiguous instructions via multi-modal multi-turn chain-of-thoughts reasoning. In The Thirteenth International Conference on Learning Representations. Xiyao Wang, Zhengyuan Yang, Linjie Li, Hongjin Lu, Yuancheng Xu, Chung-C… [find](https://scholar.google.com/scholar?q=Minheng+Ni%2C+YuTao+Fan%2C+Lei+Zhang%2C+and+Wangmeng+Zuo%2E+Visual%2Do1%3A+Understanding+ambiguous+instructions+via+multi%2Dmodal+multi%2Dturn+chain%2Dof%2Dthoughts+reasoning%2E+In+The+Thirteenth+International+Conference+o)
27. Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. 2025. Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. arXiv preprint arXiv:2502.19634. — [arXiv:2502.19634](https://arxiv.org/abs/2502.19634) [find](https://scholar.google.com/scholar?q=Jiazhen+Pan%2C+Che+Liu%2C+Junde+Wu%2C+Fenglin+Liu%2C+Jiayuan+Zhu%2C+Hongwei+Bran+Li%2C+Chen+Chen%2C+Cheng+Ouyang%2C+and+Daniel+Rueckert%2E+2025%2E+Medvlm%2Dr1%3A+Incentivizing+medical+reasoning+capability+of+vision%2Dlanguage+)
28. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). In The Eleventh International Conference on Learning Representations.
29. Obioma Pelka, Sven Koitka, Johannes Rückert, Felix Nensa, and Christoph M Friedrich. 2018. Radiology objects in context (roco): a multimodal image dataset. In Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis: 7th Joint Inte… [find](https://scholar.google.com/scholar?q=Obioma+Pelka%2C+Sven+Koitka%2C+Johannes+R%C3%BCckert%2C+Felix+Nensa%2C+and+Christoph+M+Friedrich%2E+2018%2E+Radiology+objects+in+context+%28roco%29%3A+a+multimodal+image+dataset%2E+In+Intravascular+Imaging+and+Computer+Assist)
30. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. [Jailbroken: How does llm safety training fail?](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/) Advances in Neural Information Processing Systems, 36:80079–80110.
31. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, and 1 others. [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/). Transactions on Machine Learning Research. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, F…
32. R Patrick Xian, Noah R Baker, Tom David, Qiming Cui, A Jay Holmgren, Stefan Bauer, Madhumita Sushil, and Reza Abbasi-Asl. 2025. Robustness tests for biomedical foundation models should tailor to specifications. npj Digital Medicine, 8(1):557. [find](https://scholar.google.com/scholar?q=R+Patrick+Xian%2C+Noah+R+Baker%2C+Tom+David%2C+Qiming+Cui%2C+A+Jay+Holmgren%2C+Stefan+Bauer%2C+Madhumita+Sushil%2C+and+Reza+Abbasi%2DAsl%2E+2025%2E+Robustness+tests+for+biomedical+foundation+models+should+tailor+to+speci)
33. R Patrick Xian, Alex Jihun Lee, Satvik Lolla, Vincent Wang, Russell Ro, Qiming Cui, and Reza Abbasi-Asl. 2024. Assessing biomedical knowledge robustness in large language models by query-efficient sampling attacks. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=R+Patrick+Xian%2C+Alex+Jihun+Lee%2C+Satvik+Lolla%2C+Vincent+Wang%2C+Russell+Ro%2C+Qiming+Cui%2C+and+Reza+Abbasi%2DAsl%2E+2024%2E+Assessing+biomedical+knowledge+robustness+in+large+language+models+by+query%2Defficient+sam)
34. Zhiyu Xue, Reza Abbasi-Asl, and Ramtin Pedarsani. 2025. [Enhancing the safety of medical vision-language models by synthetic demonstrations](https://md.research.elamur.ai/free/en/papers/xue-2026-enhancing-safety-medical-vision-language/). arXiv preprint arXiv:2506.09067.
35. Jin Ye, Guoan Wang, Yanjun Li, Zhongying Deng, Wei Li, Tianbin Li, Haodong Duan, Ziyan Huang, Yanzhou Su, Benyou Wang, and 1 others. 2024. Gmai-mmbench: A comprehensive multimodal evaluation benchmark towards general medical ai. Advances in Neural Information Processing Systems, 37:94327–94427. [find](https://scholar.google.com/scholar?q=Jin+Ye%2C+Guoan+Wang%2C+Yanjun+Li%2C+Zhongying+Deng%2C+Wei+Li%2C+Tianbin+Li%2C+Haodong+Duan%2C+Ziyan+Huang%2C+Yanzhou+Su%2C+Benyou+Wang%2C+and+1+others%2E+2024%2E+Gmai%2Dmmbench%3A+A+comprehensive+multimodal+evaluation+benchmark)
36. Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, and 1 others. 2023. Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915. [find](https://scholar.google.com/scholar?q=Sheng+Zhang%2C+Yanbo+Xu%2C+Naoto+Usuyama%2C+Hanwen+Xu%2C+Jaspreet+Bagga%2C+Robert+Tinn%2C+Sam+Preston%2C+Rajesh+Rao%2C+Mu+Wei%2C+Naveen+Valluri%2C+and+1+others%2E+2023%2E+Biomedclip%3A+a+multimodal+biomedical+foundation+model+)
37. Zheyuan Zhang, Muhammad Ibtsaam Qadir, Matthias Carstens, Evan Hongyang Zhang, Madison Sarah Loiselle, Farren Marc Martinus, Maksymilian Ksaw-ier Mroczkowski, Jan Clusmann, Jakob Nikolas Kather, and Fiona R Kolbinger. 2025. Prompt injection attacks on vision-language models for surgical decision sup… [find](https://scholar.google.com/scholar?q=Zheyuan+Zhang%2C+Muhammad+Ibtsaam+Qadir%2C+Matthias+Carstens%2C+Evan+Hongyang+Zhang%2C+Madison+Sarah+Loiselle%2C+Farren+Marc+Martinus%2C+Maksymilian+Ksaw%2Dier+Mroczkowski%2C+Jan+Clusmann%2C+Jakob+Nikolas+Kather%2C+and+F)
38. Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. [Minigpt-4: Enhancing vision-language understanding with advanced large language models](https://md.research.elamur.ai/free/en/papers/zhu-2023-minigpt-4-enhancing-vision-language/). arXiv preprint arXiv:2304.10592.

*Updated 2 November 2026*