---
title: "J-ClinicalBench — Shimizu 2026"
description: "Clinical LLM evaluation is well established for English but not for Japanese."
authors: ["Seiji Shimizu", "Tomohiro Nishiyama", "HISADA Shohei", "Yamato Himi", "Shoko Wakamiya", "Yuki Yanagisawa", "Masami Tsuchiya", "Satoko Hori", "Eiji ARAMAKI"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.28/"
doi: "10.63317/2uwf25atuoom"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shimizu-2026-j-clinicalbench-benchmark-evaluating-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shimizu-2026-j-clinicalbench-benchmark-evaluating-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/7/) / J-ClinicalBench: A Benchmark for Evaluating Large Language Models on Practical Clinical Tasks in Japanese

# J-ClinicalBench: A Benchmark for Evaluating Large Language Models on Practical Clinical Tasks in Japanese

**Authors:** Seiji Shimizu, [Tomohiro Nishiyama](https://md.research.elamur.ai/free/en/authors/tomohiro-nishiyama/), HISADA Shohei, Yamato Himi, [Shoko Wakamiya](https://md.research.elamur.ai/free/en/authors/shoko-wakamiya/), Yuki Yanagisawa, Masami Tsuchiya, Satoko Hori, [Eiji ARAMAKI](https://md.research.elamur.ai/free/en/authors/eiji-aramaki/) · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.28/) · [PDF](https://aclanthology.org/2026.lrec-1.28.pdf) · [DOI 10.63317/2uwf25atuoom](https://doi.org/10.63317/2uwf25atuoom) · [OpenAlex](https://openalex.org/W7160075701)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Clinical LLM evaluation is well established for English but not for Japanese. Strict privacy rules leave almost no shareable Japanese clinical text, and current Japanese medical benchmarks mostly rely on translated English data or licensing-exam questions. Neither reflects how real hospital records are written. J-ClinicalBench addresses this with a public benchmark built from original Japanese material. Domain experts wrote 227 simulated clinical documents, such as progress notes and discharge summaries modelled on in-hospital records. From these, the authors derived five new datasets. Combined with adapted existing resources, the benchmark has nine tasks grouped into reasoning, generation and understanding, scored with F1, accuracy or ROUGE-L. The authors set baselines for proprietary state-of-the-art models and for Japanese general and medical-domain open-source LLMs. They present this as the first look at how such models handle practice-oriented Japanese clinical work. The data and evaluation code are released openly. This gives Japanese clinical NLP a shared testbed and offers a model for other languages that lack clinical corpora.

## Key points
- **Resources:** 227 expert-written Japanese clinical documents underpin five newly built datasets:
  - Clinical Code Assignment (CCA)
  - Medication Recommendation (MR)
  - Hospitalization Summarization (HS)
  - Incident Report Summarization (IRS)
  - Clinical Text Simplification (CTS)
- **Reasoning tasks:**
  - CCA assigns diagnostic codes using MedDRA.
  - MR recommends suitable drugs from the case description.
  - ExamQA reuses clinical-vignette items from IGAKU QA, which draws on the Japanese national medical board exam.
- **Generation tasks:**
  - HS condenses progress notes into discharge summaries, then discharge summaries into referral notes.
  - IRS produces short summaries of patient-safety incident reports.
  - CTS rewrites radiology reports, progress notes and case reports in plain Japanese.
- **Understanding tasks:**
  - NER and RE (entity–entity and temporal relations) are adapted from the MedTxt corpus.
  - Document Classification covers ADE detection and TNM lung-cancer staging, based on NTCIR-18 RadNLP 2024.
- **Evaluation and release:** Models fall into three groups: proprietary, Japanese-pretrained and Japanese medical-pretrained LLMs. The benchmark is available at github.com/seiji-shimizu/J-ClinicalBench-release. Per-model scores were not in the supplied text, so none are reported here.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shimizu2026j,
  title = {{J-ClinicalBench: A Benchmark for Evaluating Large Language Models on Practical Clinical Tasks in Japanese}},
  author = {Shimizu, Seiji and Nishiyama, Tomohiro and Shohei, HISADA and Himi, Yamato and Wakamiya, Shoko and Yanagisawa, Yuki and Tsuchiya, Masami and Hori, Satoko and ARAMAKI, Eiji},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2uwf25atuoom},
  url = {https://doi.org/10.63317/2uwf25atuoom}
}
```

- [← previous Iʿrāb, Tanwīn and Status Constructus: Some Thoughts on the History of the Nominal Declension in Semitic and Arabic](https://md.research.elamur.ai/free/en/papers/stempel-2026-irab-tanwin-status-constructus-some/)
- [next → JFC-Recipe: A Dataset for Nutrient Estimation from Japanese User-Generated Cooking Recipes](https://md.research.elamur.ai/free/en/papers/shirai-2026-jfc-recipe-dataset-nutrient-estimation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/7/)

## References

44 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. [find](https://scholar.google.com/scholar?q=Josh+Achiam%2C+Steven+Adler%2C+Sandhini+Agarwal%2C+Lama+Ahmad%2C+Ilge+Akkaya%2C+Florencia+Leoni+Aleman%2C+Diogo+Almeida%2C+Janko+Altenschmidt%2C+Sam+Altman%2C+Shyamal+Anadkat%2C+et+al%2E+2023%2E+Gpt%2D4+technical+report%2E+arXiv) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
2. Lisa Adams, Felix Busch, Tianyu Han, Jean-Baptiste Excoffier, Matthieu Ortala, Alexander Löser, Hugo JWL Aerts, Jakob Nikolas Kather, Daniel Truhn, and Keno Bressem. 2025. Longhealth: A question answering benchmark with long clinical documents. Journal of Healthcare Informatics Research, pages 1–17. [find](https://scholar.google.com/scholar?q=Lisa+Adams%2C+Felix+Busch%2C+Tianyu+Han%2C+Jean%2DBaptiste+Excoffier%2C+Matthieu+Ortala%2C+Alexander+L%C3%B6ser%2C+Hugo+JWL+Aerts%2C+Jakob+Nikolas+Kather%2C+Daniel+Truhn%2C+and+Keno+Bressem%2E+2025%2E+Longhealth%3A+A+question+answe)
3. AI@Meta. 2024. Llama 3 model card. [find](https://scholar.google.com/scholar?q=AI%40Meta%2E+2024%2E+Llama+3+model+card%2E)
4. Joseph S Boyle, Antanas Kascenas, Pat Lok, Maria Liakata, and Alison Q O’Neil. 2023. Automated clinical coding using off-the-shelf large language models. arXiv preprint arXiv:2310.06552. — [arXiv:2310.06552](https://arxiv.org/abs/2310.06552) [find](https://scholar.google.com/scholar?q=Joseph+S+Boyle%2C+Antanas+Kascenas%2C+Pat+Lok%2C+Maria+Liakata%2C+and+Alison+Q+O%E2%80%99Neil%2E+2023%2E+Automated+clinical+coding+using+off%2Dthe%2Dshelf+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E06552%2E+%E2%80%94+%5BarXiv%3A2310)
5. Elliot G Brown, Louise Wood, and Sue Wood. 1999. The medical dictionary for regulatory activities (meddra). Drug safety, 20(2):109–117. [find](https://scholar.google.com/scholar?q=Elliot+G+Brown%2C+Louise+Wood%2C+and+Sue+Wood%2E+1999%2E+The+medical+dictionary+for+regulatory+activities+%28meddra%29%2E+Drug+safety%2C+20%282%29%3A109%E2%80%93117%2E)
6. Qingyu Chen, Yan Hu, Xueqing Peng, Qianqian Xie, Qiao Jin, Aidan Gilson, Maxwell B Singer, Xuguang Ai, Po-Ting Lai, Zhizheng Wang, et al. 2025. Benchmarking large language models for biomedical natural language processing applications and recommendations. Nature communications, 16(1):3280. [find](https://scholar.google.com/scholar?q=Qingyu+Chen%2C+Yan+Hu%2C+Xueqing+Peng%2C+Qianqian+Xie%2C+Qiao+Jin%2C+Aidan+Gilson%2C+Maxwell+B+Singer%2C+Xuguang+Ai%2C+Po%2DTing+Lai%2C+Zhizheng+Wang%2C+et+al%2E+2025%2E+Benchmarking+large+language+models+for+biomedical+natura)
7. Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, et al. 2023. Meditron-70b: Scaling medical pretraining for large language models. arXiv preprint arXiv:2311.16079. [find](https://scholar.google.com/scholar?q=Zeming+Chen%2C+Alejandro+Hern%C3%A1ndez+Cano%2C+Angelika+Romanou%2C+Antoine+Bonnet%2C+Kyle+Matoba%2C+Francesco+Salvi%2C+Matteo+Pagliardini%2C+Simin+Fan%2C+Andreas+K%C3%B6pf%2C+Amirkeivan+Mohtashami%2C+et+al%2E+2023%2E+Meditron%2D70b%3A+Sc) [our card](https://md.research.elamur.ai/free/en/papers/chen-2023-meditron-70b-scaling-medical-pretraining/)
8. Jean-Philippe Corbeil, Amin Dada, Jean-Michel Attendu, Asma Ben Abacha, Alessandro Sordoni, Lucas Caccia, Francois Beaulieu, Thomas Lin, Jens Kleesiek, and Paul Vozila. 2025. A modular approach for clinical SLMs driven by synthetic data with pre-instruction tuning, model merging, and clinical-tasks… [find](https://scholar.google.com/scholar?q=Jean%2DPhilippe+Corbeil%2C+Amin+Dada%2C+Jean%2DMichel+Attendu%2C+Asma+Ben+Abacha%2C+Alessandro+Sordoni%2C+Lucas+Caccia%2C+Francois+Beaulieu%2C+Thomas+Lin%2C+Jens+Kleesiek%2C+and+Paul+Vozila%2E+2025%2E+A+modular+approach+for+cl)
9. Emma Croxford, Yanjun Gao, Nicholas Pellegrino, Karen Wong, Graham Wills, Elliot First, Frank Liao, Cherodeep Goswami, Brian Patterson, and Majid Afshar. 2025. Current and future state of evaluation of large language models for medical summarization tasks. npj Health Systems, 2(1):6. [find](https://scholar.google.com/scholar?q=Emma+Croxford%2C+Yanjun+Gao%2C+Nicholas+Pellegrino%2C+Karen+Wong%2C+Graham+Wills%2C+Elliot+First%2C+Frank+Liao%2C+Cherodeep+Goswami%2C+Brian+Patterson%2C+and+Majid+Afshar%2E+2025%2E+Current+and+future+state+of+evaluation+o)
10. Amin Dada, Osman Alperen Koraş, Marie Bauer, Jean-Philippe Corbeil, Amanda Butler Contreras, Constantin Marc Seibold, Kaleb E Smith, Julian Friedrich, and Jens Kleesiek. 2025. Does biomedical training lead to better medical performance? In Proceedings of the Fourth Workshop on Generation, Evaluation… [find](https://scholar.google.com/scholar?q=Amin+Dada%2C+Osman+Alperen+Kora%C5%9F%2C+Marie+Bauer%2C+Jean%2DPhilippe+Corbeil%2C+Amanda+Butler+Contreras%2C+Constantin+Marc+Seibold%2C+Kaleb+E+Smith%2C+Julian+Friedrich%2C+and+Jens+Kleesiek%2E+2025%2E+Does+biomedical+training)
11. Kerstin Denecke and Helmut Paula. 2024. Analysis of critical incident reports using natural language processing. In dHealth 2024, pages 1–6. IOS Press. [find](https://scholar.google.com/scholar?q=Kerstin+Denecke+and+Helmut+Paula%2E+2024%2E+Analysis+of+critical+incident+reports+using+natural+language+processing%2E+In+dHealth+2024%2C+pages+1%E2%80%936%2E+IOS+Press%2E)
12. Felix J Dorfner, Amin Dada, Felix Busch, Marcus R Makowski, Tianyu Han, Daniel Truhn, Jens Kleesiek, Madhumita Sushil, Jacqueline Lammert, Lisa C Adams, et al. 2024. Biomedical large languages models seem not to be superior to generalist models on unseen medical data. arXiv preprint arXiv:2408.13833… — [arXiv:2408.13833](https://arxiv.org/abs/2408.13833) [find](https://scholar.google.com/scholar?q=Felix+J+Dorfner%2C+Amin+Dada%2C+Felix+Busch%2C+Marcus+R+Makowski%2C+Tianyu+Han%2C+Daniel+Truhn%2C+Jens+Kleesiek%2C+Madhumita+Sushil%2C+Jacqueline+Lammert%2C+Lisa+C+Adams%2C+et+al%2E+2024%2E+Biomedical+large+languages+models+)
13. Maarten Falter, Dries Godderis, Martijn Scher-renberg, Sevda Ece Kizilkilic, Linqi Xu, Marc Mertens, Jan Jansen, Pascal Legroux, Hanne Kindermans, Peter Sinnaeve, Frank Neven, and Paul Dendale. 2024. Using natural language processing for automated classification of disease and to identify misclassif… [find](https://scholar.google.com/scholar?q=Maarten+Falter%2C+Dries+Godderis%2C+Martijn+Scher%2Drenberg%2C+Sevda+Ece+Kizilkilic%2C+Linqi+Xu%2C+Marc+Mertens%2C+Jan+Jansen%2C+Pascal+Legroux%2C+Hanne+Kindermans%2C+Peter+Sinnaeve%2C+Frank+Neven%2C+and+Paul+Dendale%2E+2024%2E+)
14. Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki Iida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, and Naoaki Okazaki. 2024. Continual pre-training for cross-lingual llm adaptation: Enhancing japanese language capabilities. In Proceedings of the First Conference on Langua… [find](https://scholar.google.com/scholar?q=Kazuki+Fujii%2C+Taishi+Nakamura%2C+Mengsay+Loem%2C+Hiroki+Iida%2C+Masanari+Ohi%2C+Kakeru+Hattori%2C+Hirai+Shota%2C+Sakae+Mizuki%2C+Rio+Yokota%2C+and+Naoaki+Okazaki%2E+2024%2E+Continual+pre%2Dtraining+for+cross%2Dlingual+llm+ad) [our card](https://md.research.elamur.ai/free/en/papers/fujii-2024-continual-pre-training-cross-lingual/)
15. Emily Gesner, Priscilla Gazarian, and Patricia Dykes. 2019. The burden and burnout in documenting patient care: an integrative literature review. [find](https://scholar.google.com/scholar?q=Emily+Gesner%2C+Priscilla+Gazarian%2C+and+Patricia+Dykes%2E+2019%2E+The+burden+and+burnout+in+documenting+patient+care%3A+an+integrative+literature+review%2E)
16. MEDINFO 2019: Health and wellbeing e-networks for all, pages 1194–1198. [find](https://scholar.google.com/scholar?q=MEDINFO+2019%3A+Health+and+wellbeing+e%2Dnetworks+for+all%2C+pages+1194%E2%80%931198%2E)
17. Maxime Griot, Jean Vanderdonckt, Demet Yuksel, and Coralie Hemptinne. 2025. Pattern recognition or medical knowledge? the problem with multiple-choice questions in medicine. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5321… [find](https://scholar.google.com/scholar?q=Maxime+Griot%2C+Jean+Vanderdonckt%2C+Demet+Yuksel%2C+and+Coralie+Hemptinne%2E+2025%2E+Pattern+recognition+or+medical+knowledge%3F+the+problem+with+multiple%2Dchoice+questions+in+medicine%2E+In+Proceedings+of+the+63rd)
18. Tianyu Han, Lisa C Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander Löser, Daniel Truhn, and Keno K Bressem. 2023. MedAlpaca–an open-source collection of medical conversational AI models and training data. arXiv preprint arXiv:2304.08247. [find](https://scholar.google.com/scholar?q=Tianyu+Han%2C+Lisa+C+Adams%2C+Jens%2DMichalis+Papaioannou%2C+Paul+Grundmann%2C+Tom+Oberhauser%2C+Alexander+L%C3%B6ser%2C+Daniel+Truhn%2C+and+Keno+K+Bressem%2E+2023%2E+MedAlpaca%E2%80%93an+open%2Dsource+collection+of+medical+conversatio) [our card](https://md.research.elamur.ai/free/en/papers/han-2023-medalpaca-open-source-collection-medical/)
19. Nicolas Hiebel, Olivier Ferret, Karen Fort, and Aurélie Névéol. 2023. Can synthetic text help clinical named entity recognition? a study of electronic health records in French. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2320–… [find](https://scholar.google.com/scholar?q=Nicolas+Hiebel%2C+Olivier+Ferret%2C+Karen+Fort%2C+and+Aur%C3%A9lie+N%C3%A9v%C3%A9ol%2E+2023%2E+Can+synthetic+text+help+clinical+named+entity+recognition%3F+a+study+of+electronic+health+records+in+French%2E+In+Proceedings+of+the+1)
20. Julia Ive, Natalia Viani, Joyce Kam, Lucia Yin, Somain Verma, Stephen Puntis, Rudolf N Cardinal, Angus Roberts, Robert Stewart, and Sumithra Velupillai. 2020. Generation and evaluation of artificial mental health records for natural language processing. npj Digital Medicine, 3(1):69. [find](https://scholar.google.com/scholar?q=Julia+Ive%2C+Natalia+Viani%2C+Joyce+Kam%2C+Lucia+Yin%2C+Somain+Verma%2C+Stephen+Puntis%2C+Rudolf+N+Cardinal%2C+Angus+Roberts%2C+Robert+Stewart%2C+and+Sumithra+Velupillai%2E+2020%2E+Generation+and+evaluation+of+artificial+m)
21. Junfeng Jiang, Jiahao Huang, and Akiko Aizawa. 2025. JMedBench: A Benchmark for Evaluating Japanese Biomedical Large Language Models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5918– 5935, Abu Dhabi, UAE. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Junfeng+Jiang%2C+Jiahao+Huang%2C+and+Akiko+Aizawa%2E+2025%2E+JMedBench%3A+A+Benchmark+for+Evaluating+Japanese+Biomedical+Large+Language+Models%2E+In+Proceedings+of+the+31st+International+Conference+on+Computation)
22. Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, et al. 2023. MIMIC-IV, a freely accessible electronic health record dataset. Scientific data, 10(1):1. [find](https://scholar.google.com/scholar?q=Alistair+EW+Johnson%2C+Lucas+Bulgarelli%2C+Lu+Shen%2C+Alvin+Gayles%2C+Ayad+Shammout%2C+Steven+Horng%2C+Tom+J+Pollard%2C+Sicheng+Hao%2C+Benjamin+Moody%2C+Brian+Gow%2C+et+al%2E+2023%2E+MIMIC%2DIV%2C+a+freely+accessible+electronic+) [our card](https://md.research.elamur.ai/free/en/papers/johnson-2023-mimic-iv-freely-accessible-electronic/)
23. Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific data, 3(1):1–9. [find](https://scholar.google.com/scholar?q=Alistair+EW+Johnson%2C+Tom+J+Pollard%2C+Lu+Shen%2C+Li%2Dwei+H+Lehman%2C+Mengling+Feng%2C+Mohammad+Ghassemi%2C+Benjamin+Moody%2C+Peter+Szolovits%2C+Leo+Anthony+Celi%2C+and+Roger+G+Mark%2E+2016%2E+MIMIC%2DIII%2C+a+freely+accessibl) [our card](https://md.research.elamur.ai/free/en/papers/johnson-2016-mimic-iii-freely-accessible-critical/)
24. Jungo Kasai, Yuhei Kasai, Keisuke Sakaguchi, Yutaro Yamada, and Dragomir Radev. 2023. Evaluating GPT-4 and ChatGPT on Japanese medical licensing examinations. arXiv preprint arXiv:2303.18027. [find](https://scholar.google.com/scholar?q=Jungo+Kasai%2C+Yuhei+Kasai%2C+Keisuke+Sakaguchi%2C+Yutaro+Yamada%2C+and+Dragomir+Radev%2E+2023%2E+Evaluating+GPT%2D4+and+ChatGPT+on+Japanese+medical+licensing+examinations%2E+arXiv+preprint+arXiv%3A2303%2E18027%2E)
25. Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickaël Rouvier, and Richard Dufour. 2024. BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5848–5864. [find](https://scholar.google.com/scholar?q=Yanis+Labrak%2C+Adrien+Bazoge%2C+Emmanuel+Morin%2C+Pierre%2DAntoine+Gourraud%2C+Micka%C3%ABl+Rouvier%2C+and+Richard+Dufour%2E+2024%2E+BioMistral%3A+A+Collection+of+Open%2DSource+Pretrained+Large+Language+Models+for+Medical+Do) [our card](https://md.research.elamur.ai/free/en/papers/labrak-2024-biomistral-collection-open-source-pretrained/)
26. Egoitz Laparra, Steven Bethard, and Timothy A Miller. 2020. Rethinking domain adaptation for machine learning over clinical language. JAMIA open, 3(2):146–150. [find](https://scholar.google.com/scholar?q=Egoitz+Laparra%2C+Steven+Bethard%2C+and+Timothy+A+Miller%2E+2020%2E+Rethinking+domain+adaptation+for+machine+learning+over+clinical+language%2E+JAMIA+open%2C+3%282%29%3A146%E2%80%93150%2E)
27. Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, and David A. Clifton. 2024. Large language models are poor clinical… [find](https://scholar.google.com/scholar?q=Fenglin+Liu%2C+Zheng+Li%2C+Hongjian+Zhou%2C+Qingyu+Yin%2C+Jingfeng+Yang%2C+Xianfeng+Tang%2C+Chen+Luo%2C+Ming+Zeng%2C+Haoming+Jiang%2C+Yifan+Gao%2C+Priyanka+Nigam%2C+Sreyashi+Nag%2C+Bing+Yin%2C+Yining+Hua%2C+Xuan+Zhou%2C+Omid+Rohan)
28. Yixin Liu, Alex Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, and Dragomir Radev. 2023. Revisiting the gold standard: Grounding summarization evaluation with robust human evaluation. In Proceedings of the 61st Annual Meeting of the… [find](https://scholar.google.com/scholar?q=Yixin+Liu%2C+Alex+Fabbri%2C+Pengfei+Liu%2C+Yilun+Zhao%2C+Linyong+Nan%2C+Ruilin+Han%2C+Simeng+Han%2C+Shafiq+Joty%2C+Chien%2DSheng+Wu%2C+Caiming+Xiong%2C+and+Dragomir+Radev%2E+2023%2E+Revisiting+the+gold+standard%3A+Grounding+summ) [our card](https://md.research.elamur.ai/free/en/papers/liu-2022-revisiting-gold-standard-grounding-summarization/)
29. Christopher McMaster, Julia Chan, David F.L. Liew, Elizabeth Su, Albert G. Frauman, Wendy W. Chapman, and Douglas E.V. Pires. 2023. Developing a deep learning natural language processing algorithm for automated reporting of adverse drug reactions. Journal of Biomedical Informatics, 137:104265. [find](https://scholar.google.com/scholar?q=Christopher+McMaster%2C+Julia+Chan%2C+David+F%2EL%2E+Liew%2C+Elizabeth+Su%2C+Albert+G%2E+Frauman%2C+Wendy+W%2E+Chapman%2C+and+Douglas+E%2EV%2E+Pires%2E+2023%2E+Developing+a+deep+learning+natural+language+processing+algorithm+for)
30. Taishi Nakamura, Takuya Akiba, Kazuki Fujii, Yusuke Oda, Rio Yokota, and Jun Suzuki. 2025a. Drop-upcycling: Training sparse mixture of experts with partial re-initialization. In Proceedings of the Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Taishi+Nakamura%2C+Takuya+Akiba%2C+Kazuki+Fujii%2C+Yusuke+Oda%2C+Rio+Yokota%2C+and+Jun+Suzuki%2E+2025a%2E+Drop%2Dupcycling%3A+Training+sparse+mixture+of+experts+with+partial+re%2Dinitialization%2E+In+Proceedings+of+the+Thi)
31. Yuta Nakamura, Koji Fujimoto, Jonas Kluckert, Michael Krauthammer, Jun Kanzawa, Akira Katayama, Tomohiro Kikuchi, Ryo Kurokawa, Wataru Gonoi, Peitao Han, et al. 2025b. NTCIR-18 RadNLP 2024 Overview: Dataset and Solutions for Automated Lung Cancer Staging. In Proceedings of the NTCIR-18 Conference. [find](https://scholar.google.com/scholar?q=Yuta+Nakamura%2C+Koji+Fujimoto%2C+Jonas+Kluckert%2C+Michael+Krauthammer%2C+Jun+Kanzawa%2C+Akira+Katayama%2C+Tomohiro+Kikuchi%2C+Ryo+Kurokawa%2C+Wataru+Gonoi%2C+Peitao+Han%2C+et+al%2E+2025b%2E+NTCIR%2D18+RadNLP+2024+Overview%3A+D)
32. Tuan Dung Nguyen, Thanh Trung Huynh, Minh Hieu Phan, Quoc Viet Hung Nguyen, and Phi Le Nguyen. 2024. CARER - ClinicAl reasoning-enhanced representation for temporal health risk prediction. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10392– 10407,… [find](https://scholar.google.com/scholar?q=Tuan+Dung+Nguyen%2C+Thanh+Trung+Huynh%2C+Minh+Hieu+Phan%2C+Quoc+Viet+Hung+Nguyen%2C+and+Phi+Le+Nguyen%2E+2024%2E+CARER+%2D+ClinicAl+reasoning%2Denhanced+representation+for+temporal+health+risk+prediction%2E+In+Proceedi)
33. OpenAI. 2025. Introducing gpt-5. OpenAI. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025%2E+Introducing+gpt%2D5%2E+OpenAI%2E)
34. Hassan Shakil, Atqiya Munawara Mahi, Phuoc Nguyen, Zeydy Ortiz, Jugal Kalita, and Mamoun T Mardini. 2024. Evaluating text summaries generated by large language models using openai’s gpt. In 2024 International Conference on Machine Learning and Applications (ICMLA), pages 951–956. IEEE. [find](https://scholar.google.com/scholar?q=Hassan+Shakil%2C+Atqiya+Munawara+Mahi%2C+Phuoc+Nguyen%2C+Zeydy+Ortiz%2C+Jugal+Kalita%2C+and+Mamoun+T+Mardini%2E+2024%2E+Evaluating+text+summaries+generated+by+large+language+models+using+openai%E2%80%99s+gpt%2E+In+2024+Inter)
35. Seiji Shimizu, Tomohiro Nishiyama, Hiroyuki Nagai, Shoko Wakamiya, and Eiji Aramaki. 2025. Toward cross-hospital deployment of natural language processing systems: Model development and validation of fine-tuned large language models for disease name recognition in japanese. JMIR Med Inform, 13:e7677… [find](https://scholar.google.com/scholar?q=Seiji+Shimizu%2C+Tomohiro+Nishiyama%2C+Hiroyuki+Nagai%2C+Shoko+Wakamiya%2C+and+Eiji+Aramaki%2E+2025%2E+Toward+cross%2Dhospital+deployment+of+natural+language+processing+systems%3A+Model+development+and+validation+of+)
36. Weiyi Sun, Anna Rumshisky, and Ozlem Uzuner. 2013. Evaluating temporal relations in clinical text: 2012 i2b2 challenge. Journal of the American Medical Informatics Association, 20(5):806– 813. [find](https://scholar.google.com/scholar?q=Weiyi+Sun%2C+Anna+Rumshisky%2C+and+Ozlem+Uzuner%2E+2013%2E+Evaluating+temporal+relations+in+clinical+text%3A+2012+i2b2+challenge%2E+Journal+of+the+American+Medical+Informatics+Association%2C+20%285%29%3A806%E2%80%93+813%2E)
37. Liyan Tang, Zhaoyi Sun, Betina Idnay, Jordan G Nestor, Ali Soroush, Pierre A Elias, Ziyang Xu, Ying Ding, Greg Durrett, Justin F Rousseau, et al. 2023. Evaluating large language models on medical evidence summarization. npj Digital Medicine, 6(1):158. [find](https://scholar.google.com/scholar?q=Liyan+Tang%2C+Zhaoyi+Sun%2C+Betina+Idnay%2C+Jordan+G+Nestor%2C+Ali+Soroush%2C+Pierre+A+Elias%2C+Ziyang+Xu%2C+Ying+Ding%2C+Greg+Durrett%2C+Justin+F+Rousseau%2C+et+al%2E+2023%2E+Evaluating+large+language+models+on+medical+evid)
38. Augustin Toma, Patrick R Lawler, Jimmy Ba, Rahul G Krishnan, Barry B Rubin, and Bo Wang. 2023. Clinical camel: An open expert-level medical language model with dialogue-based knowledge encoding. arXiv preprint arXiv:2305.12031. [find](https://scholar.google.com/scholar?q=Augustin+Toma%2C+Patrick+R+Lawler%2C+Jimmy+Ba%2C+Rahul+G+Krishnan%2C+Barry+B+Rubin%2C+and+Bo+Wang%2E+2023%2E+Clinical+camel%3A+An+open+expert%2Dlevel+medical+language+model+with+dialogue%2Dbased+knowledge+encoding%2E+arXiv)
39. Dave Van Veen, Cara Van Uden, Louis Blankemeier, Jean-Benoit Delbrouck, Asad Aali, Christian Bluethgen, Anuj Pareek, Malgorzata Polacin, Eduardo Pontes Reis, Anna Seehofnerová, et al. 2024. Adapted large language models can outperform medical experts in clinical text summarization. Nature medicine,… [find](https://scholar.google.com/scholar?q=Dave+Van+Veen%2C+Cara+Van+Uden%2C+Louis+Blankemeier%2C+Jean%2DBenoit+Delbrouck%2C+Asad+Aali%2C+Christian+Bluethgen%2C+Anuj+Pareek%2C+Malgorzata+Polacin%2C+Eduardo+Pontes+Reis%2C+Anna+Seehofnerov%C3%A1%2C+et+al%2E+2024%2E+Adapted+la) [our card](https://md.research.elamur.ai/free/en/papers/veen-2023-adapted-large-language-models-outperform/)
40. Elizabeth Geena Woo, Michael C Burkhart, Emily Alsentzer, and Brett K Beaulieu-Jones. 2025. Synthetic data distillation enables the extraction of clinical information at scale. npj Digital Medicine, 8(1):267. [find](https://scholar.google.com/scholar?q=Elizabeth+Geena+Woo%2C+Michael+C+Burkhart%2C+Emily+Alsentzer%2C+and+Brett+K+Beaulieu%2DJones%2E+2025%2E+Synthetic+data+distillation+enables+the+extraction+of+clinical+information+at+scale%2E+npj+Digital+Medicine%2C+8)
41. Chenghao Xiao, Kun Zhao, Xiao Wang, Siwei Wu, Sixing Yan, Tomas Goldsack, Sophia Ananiadou, Noura Al Moubayed, Liang Zhan, William Cheung, and Chenghua Lin. 2025. Overview of the biolaysumm 2025 shared task on lay summarization of biomedical research articles and radiology reports. In The 24th Works… [find](https://scholar.google.com/scholar?q=Chenghao+Xiao%2C+Kun+Zhao%2C+Xiao+Wang%2C+Siwei+Wu%2C+Sixing+Yan%2C+Tomas+Goldsack%2C+Sophia+Ananiadou%2C+Noura+Al+Moubayed%2C+Liang+Zhan%2C+William+Cheung%2C+and+Chenghua+Lin%2E+2025%2E+Overview+of+the+biolaysumm+2025+share)
42. Shuntaro Yada, Yuta Nakamura, Shoko Wakamiya, and Eiji Aramaki. 2024. Cross-lingual natural language processing on limited annotated case/radiology reports in English and Japanese: Insights from the Real-MedNLP workshop. Methods of Information in Medicine, 63(05/06):145– 163. 11. Language Resource R… [find](https://scholar.google.com/scholar?q=Shuntaro+Yada%2C+Yuta+Nakamura%2C+Shoko+Wakamiya%2C+and+Eiji+Aramaki%2E+2024%2E+Cross%2Dlingual+natural+language+processing+on+limited+annotated+case%2Fradiology+reports+in+English+and+Japanese%3A+Insights+from+the+R)
43. Overview: Dataset and Solutions for Automated Lung Cancer Staging. NTCIR. [find](https://scholar.google.com/scholar?q=Overview%3A+Dataset+and+Solutions+for+Automated+Lung+Cancer+Staging%2E+NTCIR%2E)
44. Shuntaro Yada and Yuta Nakamura and Shoko Wakamiya and Eiji Aramaki. 2024. Cross-lingual natural language processing on limited annotated case/radiology reports in English and Japanese: Insights from the Real-MedNLP workshop. Georg Thieme Verlag KG. [find](https://scholar.google.com/scholar?q=Shuntaro+Yada+and+Yuta+Nakamura+and+Shoko+Wakamiya+and+Eiji+Aramaki%2E+2024%2E+Cross%2Dlingual+natural+language+processing+on+limited+annotated+case%2Fradiology+reports+in+English+and+Japanese%3A+Insights+from+)