---
title: "Do Mixed-Vendor Multi-Agent LLMs Improve Clinical Diagnosis?"
description: "The paper tests whether LLM \"doctor\" teams diagnose better when their members come from different vendors instead of… 11 references."
authors: ["Grace Chang Yuan", "Xiaoman Zhang", "Sung Eun Kim", "Pranav Rajpurkar"]
year: 2026
venue: "Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026)"
source: "https://aclanthology.org/2026.healing-1.1/"
doi: "10.18653/v1/2026.healing-1.1"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yuan-2026-do-mixed-vendor-multi-agent/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yuan-2026-do-mixed-vendor-multi-agent/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) / Do Mixed-Vendor Multi-Agent LLMs Improve Clinical Diagnosis?

# Do Mixed-Vendor Multi-Agent LLMs Improve Clinical Diagnosis?

**Authors:** Grace Chang Yuan, [Xiaoman Zhang](https://md.research.elamur.ai/free/en/authors/xiaoman-zhang/), Sung Eun Kim and [Pranav Rajpurkar](https://md.research.elamur.ai/free/en/authors/pranav-rajpurkar/) · **Year:** 2026 · **Venue:** [Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026)](https://md.research.elamur.ai/free/en/venues/healing/) ([Workshop on Linguistic Analysis for Health 2026](https://md.research.elamur.ai/free/en/venues/healing/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.healing-1.1/) · [PDF](https://aclanthology.org/2026.healing-1.1.pdf) · [DOI 10.18653/v1/2026.healing-1.1](https://doi.org/10.18653/v1/2026.healing-1.1) · [OpenAlex](https://openalex.org/W7140135492)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
The paper tests whether LLM "doctor" teams diagnose better when their members come from different vendors instead of copies of one model, which may share blind spots. The authors reuse the Multi-Agent Conversation (MAC) framework: three doctor agents take turns proposing ranked differentials and a supervisor (o4-mini by default) gives feedback and issues the final answer. All doctors get identical generic prompts. Three set-ups are compared: a lone model, a single-vendor MAC (three copies of o4-mini, Gemini-2.5-Pro or Claude-4.5-Sonnet) and a mixed-vendor MAC (one agent from each vendor). Evaluation covers rare-disease cases from RareBench (MME, HMS, LIRICAL) and a 2024-only subset of DiagnosisArena chosen to limit contamination. The mixed team gives the best recall and accuracy, and the gains hold under a second, embedding-based judge and with other supervisors. Overlap analysis suggests the agents' differing priors complement one another, while homogeneous teams tend to reinforce errors and sometimes drop correct answers their base model had found. For anyone building clinical assistants, this points to model diversity as a practical design choice.

## Key points
- Data: RareBench subsets MME (40 cases, 17 diseases), HMS (88 cases, 39 diseases) and LIRICAL (370 cases, 252 diseases), plus 165 DiagnosisArena cases from 2024, drawn from a pool of 1,113.
- Protocol: doctors produce Top-10 lists in round-robin order, with a cap of 13 turns. Scoring uses Recall@1/3/5/10 on RareBench and Top-1/Top-5 accuracy on DiagnosisArena.
- On combined RareBench, the mixed-vendor MAC reaches 39.31 R@1 and 61.35 R@10, ahead of the strongest single model (Gemini: 37.58 / 56.41) and every single-vendor team.
- Its lead also holds under a second judge, BioLORD (FRemyCompany/BioLORD-2023), which maps predictions to standardized disease names from OMIM, Orphanet and CCRD. The primary judge is o4-mini.
- In the supervisor ablation on MME, swapping in Gemini or Claude as supervisor leaves the mixed team at 40.0–42.5% R@1, which is 7.5–15.0 points above the best homogeneous team.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 243 works · 76,506 citations
- [Harvard University](https://md.research.elamur.ai/free/en/universities/harvard-university/) — University, Cambridge, United States · 126 works · 45,130 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 275 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 214 works · [🏥 Health Professions](https://md.research.elamur.ai/free/en/shelves/health-professions/) · 75 works · [🏥 Health Information Management](https://md.research.elamur.ai/free/en/shelves/health-professions/health-information-management/) · 56 works

## Cite

Cite as: Grace Chang Yuan, Xiaoman Zhang, Sung Eun Kim and Pranav Rajpurkar. 2026. Do Mixed-Vendor Multi-Agent LLMs Improve Clinical Diagnosis?. Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yuan2026do,
  title = {{Do Mixed-Vendor Multi-Agent LLMs Improve Clinical Diagnosis?}},
  author = {Grace Chang Yuan, Xiaoman Zhang, Sung Eun Kim and Pranav Rajpurkar},
  booktitle = {Proceedings of the 1st Workshop on Linguistic Analysis for Health (HeaLing 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.healing-1.1},
  url = {https://doi.org/10.18653/v1/2026.healing-1.1}
}
```

- [← previous Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework](https://md.research.elamur.ai/free/en/papers/luo-2026-do-llms-really-memorize-personally/)
- [next → Does Machine "know" interpersonal pragmatics? Evidence from MARBERT's learning of emoji pragmatics in Arabic digital discourse](https://md.research.elamur.ai/free/en/papers/shormani-2026-machine-know-interpersonal-pragmatics-evidence/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/)

## References

11 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. X. Li and et al. 2025. The alignment paradox of medical large language models in infertility care. arXiv preprint arXiv:2511.00924. — [arXiv:2511.00924](https://arxiv.org/abs/2511.00924) [find](https://scholar.google.com/scholar?q=X%2E+Li+and+et+al%2E+2025%2E+The+alignment+paradox+of+medical+large+language+models+in+infertility+care%2E+arXiv+preprint+arXiv%3A2511%2E00924%2E+%E2%80%94+%5BarXiv%3A2511%2E00924%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2511%2E00924%29)
2. Harsha Nori and 1 others. 2025. agnosis with language models. arXiv:2506.22405. — [arXiv:2506.22405](https://arxiv.org/abs/2506.22405) [find](https://scholar.google.com/scholar?q=Harsha+Nori+and+1+others%2E+2025%2E+agnosis+with+language+models%2E+arXiv%3A2506%2E22405%2E+%E2%80%94+%5BarXiv%3A2506%2E22405%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2506%2E22405%29)
3. Sequential di-arXiv preprint OpenAI. 2025. Openai o4-mini system card. [https://openai.com/index/o4-mini-system-card](https://openai.com/index/o4-mini-system-card). Accessed: 2025-12-10. [↗](https://openai.com/index/o4-mini-system-card)
4. Ana Rath, Annie Olry, Ferdinand Dhombres, Marc M Brandt, Bruno Urbero, and Ségolène Ayme. 2012. Representation of rare diseases in health information systems: the orphanet approach to serve a wide range of end users. Human Mutation, 33(5):803–808. [find](https://scholar.google.com/scholar?q=Ana+Rath%2C+Annie+Olry%2C+Ferdinand+Dhombres%2C+Marc+M+Brandt%2C+Bruno+Urbero%2C+and+S%C3%A9gol%C3%A8ne+Ayme%2E+2012%2E+Representation+of+rare+diseases+in+health+information+systems%3A+the+orphanet+approach+to+serve+a+wide+ran)
5. François Remy, Kris Demuynck, and Thomas Demeester. 2024. Biolord-2023: Semantic textual representations fusing large language models and clinical knowledge graph insights. Journal of the American Medical Informatics Association, 31(9):1844–1855. [find](https://scholar.google.com/scholar?q=Fran%C3%A7ois+Remy%2C+Kris+Demuynck%2C+and+Thomas+Demeester%2E+2024%2E+Biolord%2D2023%3A+Semantic+textual+representations+fusing+large+language+models+and+clinical+knowledge+graph+insights%2E+Journal+of+the+American+Med)
6. Khaled Saab, Tao Tu, and 1 others. 2024. [Capabilities of gemini models in medicine](https://md.research.elamur.ai/free/en/papers/saab-2024-capabilities-gemini-models-medicine/). arXiv preprint arXiv:2404.18416.
7. Tieliu Shi and Xiaoxia Lin. 2024. Catalogue of chinese rare diseases (ccrd). [http://www.unimd.org/](http://www.unimd.org/) ccrd/. Accessed: 2025-12-19. [↗](http://www.unimd.org/)
8. Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, and 1 others. 2025. Toward expert-level medical question answering with large language models. Nature Medicine, 31:943–950. [find](https://scholar.google.com/scholar?q=Karan+Singhal%2C+Shekoofeh+Azizi%2C+Tao+Tu%2C+S+Sara+Mahdavi%2C+Jason+Wei%2C+Hyung+Won+Chung%2C+Nathan+Scales%2C+Ajay+Tanwani%2C+Heather+Cole%2DLewis%2C+Stephen+Pfohl%2C+and+1+others%2E+2025%2E+Toward+expert%2Dlevel+medical+ques)
9. Akinori Sonoda, Yuka Horiuchi, Takayuki Abe, Genta Shirota, Tomohiro Mikami, Shuhei Nakao, Kengo Yoshii, Misaki Sone, Kazuma Ryuzaki, Saki Kageyama, and 1 others. 2024. Structured clinical reasoning prompt enhances LLM’s diagnostic capabilities in diagnosis please quiz cases. Japanese Journal of Rad… [find](https://scholar.google.com/scholar?q=Akinori+Sonoda%2C+Yuka+Horiuchi%2C+Takayuki+Abe%2C+Genta+Shirota%2C+Tomohiro+Mikami%2C+Shuhei+Nakao%2C+Kengo+Yoshii%2C+Misaki+Sone%2C+Kazuma+Ryuzaki%2C+Saki+Kageyama%2C+and+1+others%2E+2024%2E+Structured+clinical+reasoning+p)
10. Michael J. Sorich, Arduino A. Mangoni, Stephen Bacchi, Bradley D. Menz, and Ashley M. Hopkins. 2024. The triage and diagnostic accuracy of frontier large language models: Updated comparison to physician performance. Journal of Medical Internet Research, 26:e67409. [find](https://scholar.google.com/scholar?q=Michael+J%2E+Sorich%2C+Arduino+A%2E+Mangoni%2C+Stephen+Bacchi%2C+Bradley+D%2E+Menz%2C+and+Ashley+M%2E+Hopkins%2E+2024%2E+The+triage+and+diagnostic+accuracy+of+frontier+large+language+models%3A+Updated+comparison+to+physici)
11. H. Yang, M. Li, and et al. 2024. One llm is not enough: Harnessing the power of ensemble learning for medical question answering. medRxiv. [find](https://scholar.google.com/scholar?q=H%2E+Yang%2C+M%2E+Li%2C+and+et+al%2E+2024%2E+One+llm+is+not+enough%3A+Harnessing+the+power+of+ensemble+learning+for+medical+question+answering%2E+medRxiv%2E)

*Updated 2 November 2026*