---
title: "Beyond Distillation — paper by Liu et al. (2025)"
description: "Medical LLMs usually learn to reason through supervised fine-tuning on chain-of-thought (CoT) traces… 52 references, cited by 10 papers here."
authors: ["Che Liu", "Haozhe Wang", "Jiazhen Pan", "Zhongwei Wan", "Yong Dai", "Fangzhen Lin", "Wenjia Bai", "Daniel Rueckert", "Rossella Arcucci"]
year: 2025
venue: "arXiv:2505.17952"
source: "https://arxiv.org/abs/2505.17952"
doi: "10.48550/arxiv.2505.17952"
arxiv: "2505.17952"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2025-distillation-pushing-limits-medical-llm/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2025-distillation-pushing-limits-medical-llm/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/) / Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RL

# Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RL

**Authors:** [Che Liu](https://md.research.elamur.ai/free/en/authors/che-liu/), [Haozhe Wang](https://md.research.elamur.ai/free/en/authors/haozhe-wang/), [Jiazhen Pan](https://md.research.elamur.ai/free/en/authors/jiazhen-pan/), [Zhongwei Wan](https://md.research.elamur.ai/free/en/authors/zhongwei-wan/), [Yong Dai](https://md.research.elamur.ai/free/en/authors/yong-dai/), Fangzhen Lin, Wenjia Bai, [Daniel Rueckert](https://md.research.elamur.ai/free/en/authors/daniel-rueckert/) and Rossella Arcucci · **Year:** 2025 · **Venue:** arXiv:2505.17952 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2505.17952) · [PDF](https://arxiv.org/pdf/2505.17952) · [DOI 10.48550/arxiv.2505.17952](https://doi.org/10.48550/arxiv.2505.17952) · [arXiv 2505.17952](https://arxiv.org/abs/2505.17952) · [OpenAlex](https://openalex.org/W6910806205)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [10](https://md.research.elamur.ai/free/en/papers/liu-2025-distillation-pushing-limits-medical-llm/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Medical LLMs usually learn to reason through supervised fine-tuning on chain-of-thought (CoT) traces, often distilled from proprietary systems such as GPT-4o. Producing these traces is expensive and ties model development to external models. This paper asks whether reinforcement learning alone, with a trivially simple reward, can produce reasoning instead. The authors train Llama3.1-8B-Instruct and Llama3.1-70B-Instruct with GRPO. The model receives a reward of 1 only when its final boxed multiple-choice answer is correct, and no SFT stage, preference data or learned reward model is used. The training data come from public QA sets, MedQA and MedMCQA. A data-centric study examines how quantity, diversity, informativeness and question difficulty affect results. The resulting models, called AlphaMed, set new state-of-the-art scores on six medical QA benchmarks and beat SFT+RL pipelines like HuatuoGPT-o1 and m1. The 70B version also outperforms GPT-4o, Claude-3.5-Sonnet and DeepSeek-V3-671B on harder sets. The work suggests that carefully curated multiple-choice data can replace costly distilled rationales. It also notes that current benchmarks only partly capture reasoning gains.

## Key points
- **Training setup:** full-parameter GRPO via the verl framework for 300 steps. Each batch of 512 consists of 64 questions with 8 sampled answers per question, and the reward is a binary check of the `\boxed{}` answer. The 8B model was trained on 8 A800 GPUs and the 70B model on 64.
- **Training set:** 19,178 QA pairs, made up of all of MedQA plus 1,600 MedMCQA items per difficulty level. PubMedQA was dropped because it carried little information and hurt performance.
- **Evaluation:** six benchmarks, namely MedQA, MedMCQA, PubMedQA, MMLU-ProM, GPQA-M and MedXpert. The last four are treated as out-of-domain, and MedXpert is labelled "hard+".
- **Scores:** AlphaMed(8B) reaches 76.19 on MedQA and 22.14 on MedXpert. AlphaMed(70B) reaches 87.52 on MedQA and 32.56 on MedXpert.
- **Data findings:**
  - Informativeness of the training data drives gains.
  - Easy data already triggers reasoning, and hard data mainly helps on MedXpert.
  - Mixing difficulty levels gives the best generalization.
  - Trends that differ across benchmarks point to weaknesses in current evaluation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/) — University, Shanghai, China · 167 works · 20,987 citations
- [Hong Kong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/hong-kong-university-of-science-and-technology/) — University, Hong Kong, China · 142 works · 25,136 citations
- [The Ohio State University](https://md.research.elamur.ai/free/en/universities/the-ohio-state-university/) — University, Columbus, United States · 82 works · 18,502 citations
- [Imperial College London](https://md.research.elamur.ai/free/en/universities/imperial-college-london/) — University, London, United Kingdom · 48 works · 12,181 citations
- [Technical University of Munich](https://md.research.elamur.ai/free/en/universities/technical-university-of-munich/) — University, Munich, Germany · 55 works · 48,474 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 241 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 184 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

## Cite

Cite as: Che Liu, Haozhe Wang, Jiazhen Pan, Zhongwei Wan, Yong Dai, Fangzhen Lin, Wenjia Bai, Daniel Rueckert and Rossella Arcucci. 2025. Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RL. arXiv:2505.17952.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{liu2025beyond,
  title = {{Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RL}},
  author = {Che Liu, Haozhe Wang, Jiazhen Pan, Zhongwei Wan, Yong Dai, Fangzhen Lin, Wenjia Bai, Daniel Rueckert and Rossella Arcucci},
  year = {2025},
  doi = {10.48550/arxiv.2505.17952},
  eprint = {2505.17952},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2505.17952}
}
```

- [← previous Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model](https://md.research.elamur.ai/free/en/papers/yu-2025-beyond-dialogue-profile-dialogue-alignment/)
- [next → Beyond the Surface: Measuring Self-Preference in LLM Judgments](https://md.research.elamur.ai/free/en/papers/chen-2025-beyond-surface-measuring-self-preference/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/)

## References

52 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Y. Qin, X. Li, H. Zou, Y. Liu, S. Xia, Z. Huang, Y. Ye, W. Yuan, H. Liu, Y. Li et al., “O1 replication journey: A strategic progress report–part 1,” arXiv preprint arXiv:2410.18982, 2024. [find](https://scholar.google.com/scholar?q=Y%2E+Qin%2C+X%2E+Li%2C+H%2E+Zou%2C+Y%2E+Liu%2C+S%2E+Xia%2C+Z%2E+Huang%2C+Y%2E+Ye%2C+W%2E+Yuan%2C+H%2E+Liu%2C+Y%2E+Li+et+al%2E%2C+%E2%80%9CO1+replication+journey%3A+A+strategic+progress+report%E2%80%93part+1%2C%E2%80%9D+arXiv+preprint+arXiv%3A2410%2E18982%2C+2024%2E)
2. Z. Zeng, Q. Cheng, Z. Yin, B. Wang, S. Li, Y. Zhou, Q. Guo, X. Huang, and X. Qiu, “Scaling of search and learning: A roadmap to reproduce o1 from reinforcement learning perspective,” arXiv preprint arXiv:2412.14135, 2024. [find](https://scholar.google.com/scholar?q=Z%2E+Zeng%2C+Q%2E+Cheng%2C+Z%2E+Yin%2C+B%2E+Wang%2C+S%2E+Li%2C+Y%2E+Zhou%2C+Q%2E+Guo%2C+X%2E+Huang%2C+and+X%2E+Qiu%2C+%E2%80%9CScaling+of+search+and+learning%3A+A+roadmap+to+reproduce+o1+from+reinforcement+learning+perspective%2C%E2%80%9D+arXiv+preprint+ar)
3. J. Wang, M. Fang, Z. Wan, M. Wen, J. Zhu, A. Liu, Z. Gong, Y. Song, L. Chen, L. M. Ni et al., “Openr: An open source framework for advanced reasoning with large language models,” arXiv preprint arXiv:2410.09671, 2024. [find](https://scholar.google.com/scholar?q=J%2E+Wang%2C+M%2E+Fang%2C+Z%2E+Wan%2C+M%2E+Wen%2C+J%2E+Zhu%2C+A%2E+Liu%2C+Z%2E+Gong%2C+Y%2E+Song%2C+L%2E+Chen%2C+L%2E+M%2E+Ni+et+al%2E%2C+%E2%80%9COpenr%3A+An+open+source+framework+for+advanced+reasoning+with+large+language+models%2C%E2%80%9D+arXiv+preprint+arXiv%3A)
4. M. Y. Guan, M. Joglekar, E. Wallace, S. Jain, B. Barak, A. Heylar, R. Dias, A. Vallone, H. Ren, J. Wei, H. W. Chung, S. Toyer, J. Heidecke, A. Beutel, and A. Glaese, “[Deliberative alignment: Reasoning enables safer language models](https://md.research.elamur.ai/free/en/papers/guan-2024-deliberative-alignment-reasoning-enables-safer/),” OpenAI Blog, 2024. [Online]. Available: [https://openai.com/index/de…](https://openai.com/index/de…) [↗](https://openai.com/index/de…)
5. K. Saab, T. Tu, W.-H. Weng, R. Tanno, D. Stutz, E. Wulczyn, F. Zhang, T. Strother, C. Park, E. Vedadi et al., “Capabilities of gemini models in medicine,” arXiv preprint arXiv:2404.18416, 2024. [find](https://scholar.google.com/scholar?q=K%2E+Saab%2C+T%2E+Tu%2C+W%2E%2DH%2E+Weng%2C+R%2E+Tanno%2C+D%2E+Stutz%2C+E%2E+Wulczyn%2C+F%2E+Zhang%2C+T%2E+Strother%2C+C%2E+Park%2C+E%2E+Vedadi+et+al%2E%2C+%E2%80%9CCapabilities+of+gemini+models+in+medicine%2C%E2%80%9D+arXiv+preprint+arXiv%3A2404%2E18416%2C+2024%2E)
6. J. Chen, C. Gui, A. Gao, K. Ji, X. Wang, X. Wan, and B. Wang, “Cod, towards an interpretable medical agent using chain of diagnosis,” arXiv preprint arXiv:2407.13301, 2024. [find](https://scholar.google.com/scholar?q=J%2E+Chen%2C+C%2E+Gui%2C+A%2E+Gao%2C+K%2E+Ji%2C+X%2E+Wang%2C+X%2E+Wan%2C+and+B%2E+Wang%2C+%E2%80%9CCod%2C+towards+an+interpretable+medical+agent+using+chain+of+diagnosis%2C%E2%80%9D+arXiv+preprint+arXiv%3A2407%2E13301%2C+2024%2E)
7. V. L. Patel, J. F. Arocha, and J. Zhang, “Thinking and reasoning in medicine,” The Cambridge handbook of thinking and reasoning, vol. 14, pp. 727–750, 2005. [find](https://scholar.google.com/scholar?q=V%2E+L%2E+Patel%2C+J%2E+F%2E+Arocha%2C+and+J%2E+Zhang%2C+%E2%80%9CThinking+and+reasoning+in+medicine%2C%E2%80%9D+The+Cambridge+handbook+of+thinking+and+reasoning%2C+vol%2E+14%2C+pp%2E+727%E2%80%93750%2C+2005%2E)
8. S. Xu, Y. Zhou, Z. Liu, Z. Wu, T. Zhong, H. Zhao, Y. Li, H. Jiang, Y. Pan, J. Chen et al., “Towards next-generation medical agent: How o1 is reshaping decision-making in medical scenarios,” arXiv preprint arXiv:2411.14461, 2024. — [arXiv:2411.14461](https://arxiv.org/abs/2411.14461) [find](https://scholar.google.com/scholar?q=S%2E+Xu%2C+Y%2E+Zhou%2C+Z%2E+Liu%2C+Z%2E+Wu%2C+T%2E+Zhong%2C+H%2E+Zhao%2C+Y%2E+Li%2C+H%2E+Jiang%2C+Y%2E+Pan%2C+J%2E+Chen+et+al%2E%2C+%E2%80%9CTowards+next%2Dgeneration+medical+agent%3A+How+o1+is+reshaping+decision%2Dmaking+in+medical+scenarios%2C%E2%80%9D+arXiv+prep)
9. M.-H. Temsah, A. Jamal, K. Alhasan, A. A. Temsah, and K. H. Malki, “Openai o1-preview vs. chatgpt in healthcare: A new frontier in medical ai reasoning,” Cureus, vol. 16, no. 10, p. e70640, 2024. [find](https://scholar.google.com/scholar?q=M%2E%2DH%2E+Temsah%2C+A%2E+Jamal%2C+K%2E+Alhasan%2C+A%2E+A%2E+Temsah%2C+and+K%2E+H%2E+Malki%2C+%E2%80%9COpenai+o1%2Dpreview+vs%2E+chatgpt+in+healthcare%3A+A+new+frontier+in+medical+ai+reasoning%2C%E2%80%9D+Cureus%2C+vol%2E+16%2C+no%2E+10%2C+p%2E+e70640%2C+2024%2E)
10. Y. Xie, J. Wu, H. Tu, S. Yang, B. Zhao, Y. Zong, Q. Jin, C. Xie, and Y. Zhou, “A preliminary study of o1 in medicine: Are we closer to an ai doctor?” arXiv preprint arXiv:2409.15277, 2024. — [arXiv:2409.15277](https://arxiv.org/abs/2409.15277) [find](https://scholar.google.com/scholar?q=Y%2E+Xie%2C+J%2E+Wu%2C+H%2E+Tu%2C+S%2E+Yang%2C+B%2E+Zhao%2C+Y%2E+Zong%2C+Q%2E+Jin%2C+C%2E+Xie%2C+and+Y%2E+Zhou%2C+%E2%80%9CA+preliminary+study+of+o1+in+medicine%3A+Are+we+closer+to+an+ai+doctor%3F%E2%80%9D+arXiv+preprint+arXiv%3A2409%2E15277%2C+2024%2E+%E2%80%94+%5BarXiv%3A24)
11. J. Chen, X. Wang, K. Ji, A. Gao, F. Jiang, S. Chen, H. Zhang, D. Song, W. Xie, C. Kong et al., “Huatuogpt-ii, one-stage training for medical adaption of llms,” arXiv preprint arXiv:2311.09774, 2023. — [arXiv:2311.09774](https://arxiv.org/abs/2311.09774) [find](https://scholar.google.com/scholar?q=J%2E+Chen%2C+X%2E+Wang%2C+K%2E+Ji%2C+A%2E+Gao%2C+F%2E+Jiang%2C+S%2E+Chen%2C+H%2E+Zhang%2C+D%2E+Song%2C+W%2E+Xie%2C+C%2E+Kong+et+al%2E%2C+%E2%80%9CHuatuogpt%2Dii%2C+one%2Dstage+training+for+medical+adaption+of+llms%2C%E2%80%9D+arXiv+preprint+arXiv%3A2311%2E09774%2C+2023%2E+%E2%80%94)
12. J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, “[Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/),” Advances in Neural Information Processing Systems, vol. 35, pp. 24 824–24 837, 2022.
13. H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, J. Wei et al., “Scaling instruction-finetuned language models,” arXiv preprint arXiv:2210.11416, 2022. [find](https://scholar.google.com/scholar?q=H%2E+W%2E+Chung%2C+L%2E+Hou%2C+S%2E+Longpre%2C+B%2E+Zoph%2C+Y%2E+Tay%2C+W%2E+Fedus%2C+Y%2E+Li%2C+X%2E+Wang%2C+J%2E+Wei+et+al%2E%2C+%E2%80%9CScaling+instruction%2Dfinetuned+language+models%2C%E2%80%9D+arXiv+preprint+arXiv%3A2210%2E11416%2C+2022%2E)
14. E. Zelikman, Y. Wu, J. Mu, and N. D. Goodman, “[Star: Bootstrapping reasoning with reasoning](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/),” Advances in Neural Information Processing Systems, vol. 35, pp. 15 476–15 488, 2022.
15. T. Chu, Y. Zhai, J. Yang, S. Tong, S. Xie, D. Schuurmans, Q. V. Le, S. Levine, and Y. Ma, “[Sft memorizes, rl generalizes: A comparative study of foundation model post-training](https://md.research.elamur.ai/free/en/papers/chu-2025-sft-memorizes-rl-generalizes-comparative/),” arXiv preprint arXiv:2501.17161, 2025.
16. H. Wang, C. Qu, Z. Huang, W. Chu, F. Lin, and W. Chen, “Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning,” arXiv preprint arXiv:2504.08837, 2025. [find](https://scholar.google.com/scholar?q=H%2E+Wang%2C+C%2E+Qu%2C+Z%2E+Huang%2C+W%2E+Chu%2C+F%2E+Lin%2C+and+W%2E+Chen%2C+%E2%80%9CVl%2Drethinker%3A+Incentivizing+self%2Dreflection+of+vision%2Dlanguage+models+with+reinforcement+learning%2C%E2%80%9D+arXiv+preprint+arXiv%3A2504%2E08837%2C+2025%2E)
17. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “[Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/),” Advances in Neural Information Processing Systems, vol. 35, pp. 27 730–27 744, 2022.
18. H. Wang, L. Li, C. Qu, F. Zhu, W. Xu, W. Chu, and F. Lin, “Learning autonomous code integration for math language models,” arXiv preprint arXiv:2502.00691, 2025. [find](https://scholar.google.com/scholar?q=H%2E+Wang%2C+L%2E+Li%2C+C%2E+Qu%2C+F%2E+Zhu%2C+W%2E+Xu%2C+W%2E+Chu%2C+and+F%2E+Lin%2C+%E2%80%9CLearning+autonomous+code+integration+for+math+language+models%2C%E2%80%9D+arXiv+preprint+arXiv%3A2502%2E00691%2C+2025%2E)
19. R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “[Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/),” Advances in Neural Information Processing Systems, vol. 36, 2023.
20. A. Ura, “Openbiollm-70b: Advancing open-source biomedical llms with direct preference optimization,” Hugging Face Blog, 2024, available at [https://huggingface.co/blog/aaditya/openbiollm](https://huggingface.co/blog/aaditya/openbiollm). [↗](https://huggingface.co/blog/aaditya/openbiollm)
21. D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi et al., “[Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/),” arXiv preprint arXiv:2501.12948, 2025.
22. H. Zeng, D. Jiang, H. Wang, P. Nie, X. Chen, and W. Chen, “Acecoder: Acing coder rl via automated test-case synthesis,” arXiv preprint arXiv:2502.01718, 2025. [find](https://scholar.google.com/scholar?q=H%2E+Zeng%2C+D%2E+Jiang%2C+H%2E+Wang%2C+P%2E+Nie%2C+X%2E+Chen%2C+and+W%2E+Chen%2C+%E2%80%9CAcecoder%3A+Acing+coder+rl+via+automated+test%2Dcase+synthesis%2C%E2%80%9D+arXiv+preprint+arXiv%3A2502%2E01718%2C+2025%2E)
23. J. Pan, C. Liu, J. Wu, F. Liu, J. Zhu, H. B. Li, C. Chen, C. Ouyang, and D. Rueckert, “Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning,” arXiv preprint arXiv:2502.19634, 2025. — [arXiv:2502.19634](https://arxiv.org/abs/2502.19634) [find](https://scholar.google.com/scholar?q=J%2E+Pan%2C+C%2E+Liu%2C+J%2E+Wu%2C+F%2E+Liu%2C+J%2E+Zhu%2C+H%2E+B%2E+Li%2C+C%2E+Chen%2C+C%2E+Ouyang%2C+and+D%2E+Rueckert%2C+%E2%80%9CMedvlm%2Dr1%3A+Incentivizing+medical+reasoning+capability+of+vision%2Dlanguage+models+%28vlms%29+via+reinforcement+learning)
24. H. Zhang, J. Chen, F. Jiang, F. Yu, Z. Chen, J. Li, G. Chen, X. Wu, Z. Zhang, Q. Xiao et al., “[Huatuogpt, towards taming language model to be a doctor](https://md.research.elamur.ai/free/en/papers/zhang-2023-huatuogpt-taming-language-model-be/),” arXiv preprint arXiv:2305.15075, 2023.
25. Y. Labrak, A. Bazoge, E. Morin, P.-A. Gourraud, M. Rouvier, and R. Dufour, “[Biomistral: A collection of open-source pretrained large language models for medical domains](https://md.research.elamur.ai/free/en/papers/labrak-2024-biomistral-collection-open-source-pretrained/),” arXiv preprint arXiv:2402.10373, 2024.
26. Z. Ke, Y. Shao, H. Lin, T. Konishi, G. Kim, and B. Liu, “Continual pre-training of language models,” arXiv preprint arXiv:2302.03241, 2023. [find](https://scholar.google.com/scholar?q=Z%2E+Ke%2C+Y%2E+Shao%2C+H%2E+Lin%2C+T%2E+Konishi%2C+G%2E+Kim%2C+and+B%2E+Liu%2C+%E2%80%9CContinual+pre%2Dtraining+of+language+models%2C%E2%80%9D+arXiv+preprint+arXiv%3A2302%2E03241%2C+2023%2E)
27. K. Zhang, S. Zeng, E. Hua, N. Ding, Z.-R. Chen, Z. Ma, H. Li, G. Cui, B. Qi, X. Zhu et al., “Ultramedical: Building specialized generalists in biomedicine,” Advances in Neural Information Processing Systems, vol. 37, pp. 26 045–26 081, 2024. [find](https://scholar.google.com/scholar?q=K%2E+Zhang%2C+S%2E+Zeng%2C+E%2E+Hua%2C+N%2E+Ding%2C+Z%2E%2DR%2E+Chen%2C+Z%2E+Ma%2C+H%2E+Li%2C+G%2E+Cui%2C+B%2E+Qi%2C+X%2E+Zhu+et+al%2E%2C+%E2%80%9CUltramedical%3A+Building+specialized+generalists+in+biomedicine%2C%E2%80%9D+Advances+in+Neural+Information+Processing+S)
28. J. Chen, Z. Cai, K. Ji, X. Wang, W. Liu, R. Wang, J. Hou, and B. Wang, “Huatuogpt-o1: Towards medical complex reasoning with llms,” arXiv preprint arXiv:2412.18925, 2024. [find](https://scholar.google.com/scholar?q=J%2E+Chen%2C+Z%2E+Cai%2C+K%2E+Ji%2C+X%2E+Wang%2C+W%2E+Liu%2C+R%2E+Wang%2C+J%2E+Hou%2C+and+B%2E+Wang%2C+%E2%80%9CHuatuogpt%2Do1%3A+Towards+medical+complex+reasoning+with+llms%2C%E2%80%9D+arXiv+preprint+arXiv%3A2412%2E18925%2C+2024%2E)
29. X. Huang, J. Wu, H. Liu, X. Tang, and Y. Zhou, “m1: Unleash the potential of test-time scaling for medical reasoning with large language models,” arXiv preprint arXiv:2504.00869, 2025. — [arXiv:2504.00869](https://arxiv.org/abs/2504.00869) [find](https://scholar.google.com/scholar?q=X%2E+Huang%2C+J%2E+Wu%2C+H%2E+Liu%2C+X%2E+Tang%2C+and+Y%2E+Zhou%2C+%E2%80%9Cm1%3A+Unleash+the+potential+of+test%2Dtime+scaling+for+medical+reasoning+with+large+language+models%2C%E2%80%9D+arXiv+preprint+arXiv%3A2504%2E00869%2C+2025%2E+%E2%80%94+%5BarXiv%3A2504%2E0)
30. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017. [find](https://scholar.google.com/scholar?q=J%2E+Schulman%2C+F%2E+Wolski%2C+P%2E+Dhariwal%2C+A%2E+Radford%2C+and+O%2E+Klimov%2C+%E2%80%9CProximal+policy+optimization+algorithms%2C%E2%80%9D+arXiv+preprint+arXiv%3A1707%2E06347%2C+2017%2E)
31. G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu, “[Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/),” arXiv preprint arXiv:2409.19256, 2024.
32. D. Jin, E. Pan, N. Oufattole, W.-H. Weng, H. Fang, and P. Szolovits, “[What disease does this patient have? a large-scale open domain question answering dataset from medical exams](https://md.research.elamur.ai/free/en/papers/jin-2020-disease-this-patient-have-large/),” Applied Sciences, vol. 11, no. 14, p. 6421, 2021.
33. A. Pal, L. K. Umapathi, and M. Sankarasubbu, “Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering,” in Conference on health, inference, and learning. PMLR, 2022, pp. 248–260. — [MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering](https://md.research.elamur.ai/free/en/papers/pal-2022-medmcqa-large-scale-multi-subject/)
34. Q. Jin, B. Dhingra, Z. Liu, W. W. Cohen, and X. Lu, “[Pubmedqa: A dataset for biomedical research question answering](https://md.research.elamur.ai/free/en/papers/jin-2019-pubmedqa-dataset-biomedical-research-question/),” arXiv preprint arXiv:1909.06146, 2019.
35. Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang et al., “[Mmlu-pro: A more robust and challenging multi-task language understanding benchmark](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/),” in The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
36. D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman, “[Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/),” in First Conference on Language Modeling, 2024.
37. Y. Zuo, S. Qu, Y. Li, Z. Chen, X. Zhu, E. Hua, K. Zhang, N. Ding, and B. Zhou, “Medxpertqa: Benchmarking expert-level medical reasoning and understanding,” arXiv preprint arXiv:2501.18362, 2025. [find](https://scholar.google.com/scholar?q=Y%2E+Zuo%2C+S%2E+Qu%2C+Y%2E+Li%2C+Z%2E+Chen%2C+X%2E+Zhu%2C+E%2E+Hua%2C+K%2E+Zhang%2C+N%2E+Ding%2C+and+B%2E+Zhou%2C+%E2%80%9CMedxpertqa%3A+Benchmarking+expert%2Dlevel+medical+reasoning+and+understanding%2C%E2%80%9D+arXiv+preprint+arXiv%3A2501%2E18362%2C+2025%2E)
38. X. Tang, D. Shao, J. Sohn, J. Chen, J. Zhang, J. Xiang, F. Wu, Y. Zhao, C. Wu, W. Shi et al., “Medagents-bench: Benchmarking thinking models and agent frameworks for complex medical reasoning,” arXiv preprint arXiv:2503.07459, 2025. — [arXiv:2503.07459](https://arxiv.org/abs/2503.07459) [find](https://scholar.google.com/scholar?q=X%2E+Tang%2C+D%2E+Shao%2C+J%2E+Sohn%2C+J%2E+Chen%2C+J%2E+Zhang%2C+J%2E+Xiang%2C+F%2E+Wu%2C+Y%2E+Zhao%2C+C%2E+Wu%2C+W%2E+Shi+et+al%2E%2C+%E2%80%9CMedagents%2Dbench%3A+Benchmarking+thinking+models+and+agent+frameworks+for+complex+medical+reasoning%2C%E2%80%9D+arXiv+)
39. Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang et al., “[Mmlu-pro: A more robust and challenging multi-task language understanding benchmark](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/),” arXiv preprint arXiv:2406.01574, 2024.
40. M. S. A. Pal and M. Sankarasubbu, “Openbiollms: Advancing open-source large language models for healthcare and life sciences,” 2024. [find](https://scholar.google.com/scholar?q=M%2E+S%2E+A%2E+Pal+and+M%2E+Sankarasubbu%2C+%E2%80%9COpenbiollms%3A+Advancing+open%2Dsource+large+language+models+for+healthcare+and+life+sciences%2C%E2%80%9D+2024%2E)
41. P. Qiu, C. Wu, X. Zhang, W. Lin, H. Wang, Y. Zhang, Y. Wang, and W. Xie, “[Towards building multilingual language model for medicine](https://md.research.elamur.ai/free/en/papers/qiu-2024-towards-building-multilingual-language-model/),” Nature Communications, vol. 15, no. 1, p. 8384, 2024.
42. C. Christophe, P. K. Kanithi, P. Munjal, T. Raha, N. Hayat, R. Rajan, A. Al-Mahrooqi, A. Gupta, M. U. Salman, G. Gosal et al., “Med42–evaluating fine-tuning strategies for medical llms: Full-parameter vs. parameter-efficient approaches,” arXiv preprint arXiv:2404.14779, 2024. — [arXiv:2404.14779](https://arxiv.org/abs/2404.14779) [find](https://scholar.google.com/scholar?q=C%2E+Christophe%2C+P%2E+K%2E+Kanithi%2C+P%2E+Munjal%2C+T%2E+Raha%2C+N%2E+Hayat%2C+R%2E+Rajan%2C+A%2E+Al%2DMahrooqi%2C+A%2E+Gupta%2C+M%2E+U%2E+Salman%2C+G%2E+Gosal+et+al%2E%2C+%E2%80%9CMed42%E2%80%93evaluating+fine%2Dtuning+strategies+for+medical+llms%3A+Full%2Dparameter)
43. D. Jin, E. Pan, N. Oufattole, W.-H. Weng, H. Fang, and P. Szolovits, “[What disease does this patient have? a large-scale open domain question answering dataset from medical exams](https://md.research.elamur.ai/free/en/papers/jin-2020-disease-this-patient-have-large/),” Applied Sciences, vol. 11, no. 14, p. 6421, 2021.
44. A. Pal, L. K. Umapathi, and M. Sankarasubbu, “Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering,” in Conference on Health, Inference, and Learning. PMLR, 2022, pp. 248–260. — [MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering](https://md.research.elamur.ai/free/en/papers/pal-2022-medmcqa-large-scale-multi-subject/)
45. A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan et al., “[The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/),” arXiv preprint arXiv:2407.21783, 2024.
46. J. Chen, Z. Cai, K. Ji, X. Wang, W. Liu, R. Wang, J. Hou, and B. Wang, “Huatuogpt-o1, towards medical complex reasoning with llms,” arXiv preprint arXiv:2412.18925, 2024. [find](https://scholar.google.com/scholar?q=J%2E+Chen%2C+Z%2E+Cai%2C+K%2E+Ji%2C+X%2E+Wang%2C+W%2E+Liu%2C+R%2E+Wang%2C+J%2E+Hou%2C+and+B%2E+Wang%2C+%E2%80%9CHuatuogpt%2Do1%2C+towards+medical+complex+reasoning+with+llms%2C%E2%80%9D+arXiv+preprint+arXiv%3A2412%2E18925%2C+2024%2E)
47. A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford et al., “[Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/),” arXiv preprint arXiv:2410.21276, 2024.
48. Q. Team, “Qwq: Reflect deeply on the boundaries of the unknown,” November 2024. [Online]. Available: [https://qwenlm.github.io/blog/qwq-32b-preview/](https://qwenlm.github.io/blog/qwq-32b-preview/) [↗](https://qwenlm.github.io/blog/qwq-32b-preview/)
49. Anthropic, “The claude 3 model family: Opus, sonnet, haiku,” [https://www-cdn.anthropic.com/](https://www-cdn.anthropic.com/) de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf, 2024. [↗](https://www-cdn.anthropic.com/)
50. A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan et al., “Deepseekv3 technical report,” arXiv preprint arXiv:2412.19437, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Liu%2C+B%2E+Feng%2C+B%2E+Xue%2C+B%2E+Wang%2C+B%2E+Wu%2C+C%2E+Lu%2C+C%2E+Zhao%2C+C%2E+Deng%2C+C%2E+Zhang%2C+C%2E+Ruan+et+al%2E%2C+%E2%80%9CDeepseekv3+technical+report%2C%E2%80%9D+arXiv+preprint+arXiv%3A2412%2E19437%2C+2024%2E)
51. D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman, “[Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/),” arXiv preprint arXiv:2311.12022, 2023.
52. Q. Team, “Qwen2.5: A party of foundation models,” September 2024. [Online]. Available: [https://qwenlm.github.io/blog/qwen2.5/Limitations](https://qwenlm.github.io/blog/qwen2.5/Limitations) and Future Work [↗](https://qwenlm.github.io/blog/qwen2.5/Limitations)

*Updated 31 October 2026*