---
title: "Patient-Similarity Cohort Reasoning in Clinical Text-to-SQL (2026)"
description: "They want to compare groups of similar patients under clinical constraints such as diagnosis codes… 51 references, cited by 4 papers here."
authors: ["Yifei Shen", "Yilun Zhao", "Justice Ou", "Tinglin Huang", "Arman Cohan"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.64/"
doi: "10.18653/v1/2026.eacl-long.64"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shen-2026-patient-similarity-cohort-reasoning-clinical/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shen-2026-patient-similarity-cohort-reasoning-clinical/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) / Patient-Similarity Cohort Reasoning in Clinical Text-to-SQL

# Patient-Similarity Cohort Reasoning in Clinical Text-to-SQL

**Authors:** [Yifei Shen](https://md.research.elamur.ai/free/en/authors/yifei-shen/), [Yilun Zhao](https://md.research.elamur.ai/free/en/authors/yilun-zhao/), [Justice Ou](https://md.research.elamur.ai/free/en/authors/justice-ou/), [Tinglin Huang](https://md.research.elamur.ai/free/en/authors/tinglin-huang/) and [Arman Cohan](https://md.research.elamur.ai/free/en/authors/arman-cohan/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.64/) · [PDF](https://aclanthology.org/2026.eacl-long.64.pdf) · [DOI 10.18653/v1/2026.eacl-long.64](https://doi.org/10.18653/v1/2026.eacl-long.64) · [OpenAlex](https://openalex.org/W7140087449)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/shen-2026-patient-similarity-cohort-reasoning-clinical/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

Clinicians often ask for more than one patient's record. They want to compare groups of similar patients under clinical constraints such as diagnosis codes, time windows and lab ranges. Earlier EHR text-to-SQL datasets like MIMICSQL and EHRSQL mostly target single patients or simple statistics. The authors built CLINSQL to fill this gap. It contains 633 expert-annotated tasks on MIMIC-IV v3.1, spread across six clinical scenario families and split into Easy, Medium and Hard levels.

Solving a task means mapping the question onto the schema and coding systems (ICD, itemids), joining many tables, applying temporal logic and computing statistics for a patient cohort. Scoring uses two rubric trees, one for the SQL and one for the results. Critical requirements are weighted first, and execution checks test clinical plausibility while still accepting equivalent formulations.

The authors tested 22 models with Chain-of-Thought self-refinement, and none came close to clinical reliability. Most errors came from cohort definitions that drifted from the question. Adding schema hints improved results. The benchmark gives a realistic target for trustworthy EHR analytics.

## Key points

- **Data:** 633 tasks on MIMIC-IV v3.1, using the `mimiciv_hosp` and `mimiciv_icu` modules. Each task has expert-written gold SQL and rubric trees for both the SQL and the results.
- **Test-set results:**
  - GPT-5-mini is best overall at 74.7% execution.
  - DeepSeek-R1 is the best open-source model at 69.2% (the introduction gives 69.1%).
  - Gemini-2.5-Pro scores 85.5% on Easy tasks but only 67.2% on Hard ones.
- **Error analysis** (60 GPT-5-mini validation cases):
  - 54% concerned cohort specification and coding, such as replacing exact ICD or itemid filters with keyword heuristics.
  - 24% were output schema or formatting problems.
  - 14% were aggregation or statistics mistakes.
- **Schema-hinted inference:** prompts that supply validated ICD filters and the expected output columns improved GPT-5-mini at every difficulty level, with the largest gains on Medium and Hard. One reported figure shows a rise from 72.7% to 75.2%.
- **Limitations:** the benchmark covers a single health system and a single SQL environment. It also depends on heavy expert annotation, which makes it hard to scale.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 402 works · 212,762 citations
- [Yale University](https://md.research.elamur.ai/free/en/universities/yale-university/) — University, New Haven, United States · 101 works · 13,302 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🧬 Biochemistry, Genetics and Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/) · 263 works · [🧬 Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/molecular-biology/) · 260 works

🧭 Topics (3): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works · [🧾 Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/) · 193 works · [💻 Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/) · 168 works

## Cite

Cite as: Yifei Shen, Yilun Zhao, Justice Ou, Tinglin Huang and Arman Cohan. 2026. Patient-Similarity Cohort Reasoning in Clinical Text-to-SQL. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shen2026patient,
  title = {{Patient-Similarity Cohort Reasoning in Clinical Text-to-SQL}},
  author = {Yifei Shen, Yilun Zhao, Justice Ou, Tinglin Huang and Arman Cohan},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.64},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.64}
}
```

- [← previous PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inference](https://md.research.elamur.ai/free/en/papers/venkata-2026-pagedeviction-structured-block-wise-kv/)
- [next → PII-Bench: Evaluating Query-Aware Privacy Protection Systems](https://md.research.elamur.ai/free/en/papers/shen-2026-pii-bench-evaluating-query-aware/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/)

## References

51 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Seongsu Bae, Sunjun Kweon, Taewoong Jang, and Edward Choi. 2023. Ehrxqa: A multi-modal question answering dataset for electronic health records with chest x-ray images. arXiv preprint arXiv:2310.18652. — [arXiv:2310.18652](https://arxiv.org/abs/2310.18652) [find](https://scholar.google.com/scholar?q=Seongsu+Bae%2C+Sunjun+Kweon%2C+Taewoong+Jang%2C+and+Edward+Choi%2E+2023%2E+Ehrxqa%3A+A+multi%2Dmodal+question+answering+dataset+for+electronic+health+records+with+chest+x%2Dray+images%2E+arXiv+preprint+arXiv%3A2310%2E18652)
2. Baichuan AI. 2025. Baichuan-M2-32B. [https://huggingface.co/baichuan-inc/](https://huggingface.co/baichuan-inc/) Baichuan-M2-32B. Model card. [↗](https://huggingface.co/baichuan-inc/)
3. Jayetri Bardhan, Anthony Colas, Kirk Roberts, and Daisy Zhe Wang. 2022. DrugEHRQA: A question answering dataset on structured and unstructured electronic health records for medicine related queries. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 1083–1097, Marse… [find](https://scholar.google.com/scholar?q=Jayetri+Bardhan%2C+Anthony+Colas%2C+Kirk+Roberts%2C+and+Daisy+Zhe+Wang%2E+2022%2E+DrugEHRQA%3A+A+question+answering+dataset+on+structured+and+unstructured+electronic+health+records+for+medicine+related+queries%2E+I)
4. Hanjie Chen, Zhouxiang Fang, Yash Singla, and Mark Dredze. 2025. [Benchmarking large language models on answering and explaining challenging medical questions](https://md.research.elamur.ai/free/en/papers/chen-2025-benchmarking-large-language-models-answering/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language…
5. Gheorghe Comanici, many others, and the Gemini Team. 2025. [Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/).
6. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan W… [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2C+Daya+Guo%2C+Dejian+Yang%2C+Haowei+Zhang%2C+Junxiao+Song%2C+Ruoyu+Zhang%2C+Runxin+Xu%2C+Qihao+Zhu%2C+Shirong+Ma%2C+Peiyi+Wang%2C+Xiao+Bi%2C+Xiaokang+Zhang%2C+Xingkai+Yu%2C+Yu+Wu%2C+Z%2E+F%2E+Wu%2C+Zhibin+Gou%2C+Zhihong+Sha)
7. Defog.ai. 2024. SQLCoder-7B-2. [https://huggingface.co/defog/sqlcoder-7b-2](https://huggingface.co/defog/sqlcoder-7b-2). Model card. [↗](https://huggingface.co/defog/sqlcoder-7b-2)
8. Google DeepMind. 2025. MedGemma 27B Text-only (medgemma-27b-text-it). [https://huggingface.co/google/](https://huggingface.co/google/) medgemma-27b-text-it. Model card. [↗](https://huggingface.co/google/)
9. Boyu Gou, Zanming Huang, Yuting Ning, Yu Gu, Michael Lin, Weijian Qi, Andrei Kopanev, Botao Yu, Bernal Jiménez Gutiérrez, Yiheng Shu, Chan Hee Song, Jiaman Wu, Shijie Chen, Hanane Nour Moussa, Tianshu Zhang, Jian Xie, Yifei Li, Tianci Xue, Zeyi Liao, Kai Zhang, Boyuan Zheng, Zhaowei Cai, Viktor Rozg… [find](https://scholar.google.com/scholar?q=Boyu+Gou%2C+Zanming+Huang%2C+Yuting+Ning%2C+Yu+Gu%2C+Michael+Lin%2C+Weijian+Qi%2C+Andrei+Kopanev%2C+Botao+Yu%2C+Bernal+Jim%C3%A9nez+Guti%C3%A9rrez%2C+Yiheng+Shu%2C+Chan+Hee+Song%2C+Jiaman+Wu%2C+Shijie+Chen%2C+Hanane+Nour+Moussa%2C+Tianshu)
10. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations.
11. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. [What disease does this patient have? a large-scale open domain question answering dataset from medical exams](https://md.research.elamur.ai/free/en/papers/jin-2020-disease-this-patient-have-large/). Applied Sciences, 11(14):6421.
12. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019a. [Pubmedqa: A dataset for biomedical research question answering](https://md.research.elamur.ai/free/en/papers/jin-2019-pubmedqa-dataset-biomedical-research-question/). Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process…
13. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019b. [PubMedQA: A dataset for biomedical research question answering](https://md.research.elamur.ai/free/en/papers/jin-2019-pubmedqa-dataset-biomedical-research-question/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proc…
14. Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, et al. 2023. [Mimic-iv, a freely accessible electronic health record dataset](https://md.research.elamur.ai/free/en/papers/johnson-2023-mimic-iv-freely-accessible-electronic/). Scientific Data, 10(1):1–9.
15. Zong Ke, Yuqing Cao, Zhenrui Chen, Yuchen Yin, Shouchao He, and Yu Cheng. 2025a. Early warning of cryptocurrency reversal risks via multi-source data. Finance Research Letters, page 107890. [find](https://scholar.google.com/scholar?q=Zong+Ke%2C+Yuqing+Cao%2C+Zhenrui+Chen%2C+Yuchen+Yin%2C+Shouchao+He%2C+and+Yu+Cheng%2E+2025a%2E+Early+warning+of+cryptocurrency+reversal+risks+via+multi%2Dsource+data%2E+Finance+Research+Letters%2C+page+107890%2E)
16. Zong Ke, Jiaqing Shen, Xuanyi Zhao, Xinghao Fu, Yang Wang, Zichao Li, Lingjie Liu, and Huailing Mu. 2025b. A stable technical feature with gru-cnn-ga fusion. Applied Soft Computing, page 114302. [find](https://scholar.google.com/scholar?q=Zong+Ke%2C+Jiaqing+Shen%2C+Xuanyi+Zhao%2C+Xinghao+Fu%2C+Yang+Wang%2C+Zichao+Li%2C+Lingjie+Liu%2C+and+Huailing+Mu%2E+2025b%2E+A+stable+technical+feature+with+gru%2Dcnn%2Dga+fusion%2E+Applied+Soft+Computing%2C+page+114302%2E)
17. Hajung Kim, Chanhwi Kim, Hoonick Lee, Kyochul Jang, Jiwoo Lee, Kyungjae Lee, Gangwoo Kim, and Jaewoo Kang. 2024a. KU-DMIS at EHRSQL 2024 : Generating SQL query via question templatization in EHR. In Proceedings of the 6th Clinical Natural Language Processing Workshop, pages 672–686, Mexico City, Mex… [find](https://scholar.google.com/scholar?q=Hajung+Kim%2C+Chanhwi+Kim%2C+Hoonick+Lee%2C+Kyochul+Jang%2C+Jiwoo+Lee%2C+Kyungjae+Lee%2C+Gangwoo+Kim%2C+and+Jaewoo+Kang%2E+2024a%2E+KU%2DDMIS+at+EHRSQL+2024+%3A+Generating+SQL+query+via+question+templatization+in+EHR%2E+In+P)
18. Yunsoo Kim, Jinge Wu, Yusuf Abdulle, and Honghan Wu. 2024b. MedExQA: Medical question answering benchmark with multiple explanations. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 167–181, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yunsoo+Kim%2C+Jinge+Wu%2C+Yusuf+Abdulle%2C+and+Honghan+Wu%2E+2024b%2E+MedExQA%3A+Medical+question+answering+benchmark+with+multiple+explanations%2E+In+Proceedings+of+the+23rd+Workshop+on+Biomedical+Natural+Language)
19. Sunjun Kweon, Jiyoun Lee, Joon Sung Yi, and Edward Choi. 2024. Ehrnoteqa: An llm benchmark for real-world clinical practice using discharge summaries. arXiv preprint arXiv:2402.16040. — [arXiv:2402.16040](https://arxiv.org/abs/2402.16040) [find](https://scholar.google.com/scholar?q=Sunjun+Kweon%2C+Jiyoun+Lee%2C+Joon+Sung+Yi%2C+and+Edward+Choi%2E+2024%2E+Ehrnoteqa%3A+An+llm+benchmark+for+real%2Dworld+clinical+practice+using+discharge+summaries%2E+arXiv+preprint+arXiv%3A2402%2E16040%2E+%E2%80%94+%5BarXiv%3A2402%2E16)
20. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/).
21. Chia-Hsuan Lee, Oleksandr Polozov, and Matthew Richardson. 2021. Kaggledbqa: Realistic evaluation of text-to-sql parsers. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: L… [find](https://scholar.google.com/scholar?q=Chia%2DHsuan+Lee%2C+Oleksandr+Polozov%2C+and+Matthew+Richardson%2E+2021%2E+Kaggledbqa%3A+Realistic+evaluation+of+text%2Dto%2Dsql+parsers%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational)
22. Qwen Team. 2025. Qwen3 technical report. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2025%2E+Qwen3+technical+report%2E)
23. Gyubok Lee, Hyeonji Hwang, Seongsu Bae, Yeonsu Kwon, Woncheol Shin, Seongjun Yang, Minjoon Seo, Jong-Yeup Kim, and Edward Choi. 2022. Ehrsql: A practical text-to-sql benchmark for electronic health records. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks T… [find](https://scholar.google.com/scholar?q=Gyubok+Lee%2C+Hyeonji+Hwang%2C+Seongsu+Bae%2C+Yeonsu+Kwon%2C+Woncheol+Shin%2C+Seongjun+Yang%2C+Minjoon+Seo%2C+Jong%2DYeup+Kim%2C+and+Edward+Choi%2E+2022%2E+Ehrsql%3A+A+practical+text%2Dto%2Dsql+benchmark+for+electronic+health+re)
24. Jaehee Ryu, Seonhee Cho, Gyubok Lee, and Edward Choi. 2024. EHR-SeqSQL : A sequential text-to-SQL dataset for interactively exploring electronic health records. In Findings of the Association for Computational Linguistics: ACL 2024, pages 16388– 16407, Bangkok, Thailand. Association for Computationa… [find](https://scholar.google.com/scholar?q=Jaehee+Ryu%2C+Seonhee+Cho%2C+Gyubok+Lee%2C+and+Edward+Choi%2E+2024%2E+EHR%2DSeqSQL+%3A+A+sequential+text%2Dto%2DSQL+dataset+for+interactively+exploring+electronic+health+records%2E+In+Findings+of+the+Association+for+Comp)
25. Gyubok Lee, Sunjun Kweon, Seongsu Bae, and Edward Choi. 2024. Overview of the EHRSQL 2024 shared task on reliable text-to-SQL modeling on electronic health records. In Proceedings of the 6th Clinical Natural Language Processing Workshop, pages 644– 654, Mexico City, Mexico. Association for Computati… [find](https://scholar.google.com/scholar?q=Gyubok+Lee%2C+Sunjun+Kweon%2C+Seongsu+Bae%2C+and+Edward+Choi%2E+2024%2E+Overview+of+the+EHRSQL+2024+shared+task+on+reliable+text%2Dto%2DSQL+modeling+on+electronic+health+records%2E+In+Proceedings+of+the+6th+Clinical+)
26. Shadi Saleh and Pavel Pecina. 2019. An extended CLEF ehealth test collection for cross-lingual information retrieval in the medical domain. In Advances in Information Retrieval: 41st European Conference on Information Retrieval (ECIR 2019), volume 11438 of Lecture Notes in Computer Science, pages 18… [find](https://scholar.google.com/scholar?q=Shadi+Saleh+and+Pavel+Pecina%2E+2019%2E+An+extended+CLEF+ehealth+test+collection+for+cross%2Dlingual+information+retrieval+in+the+medical+domain%2E+In+Advances+in+Information+Retrieval%3A+41st+European+Conferen)
27. Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al. 2023. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sql evaluation. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Jinyang+Li%2C+Binyuan+Hui%2C+Ge+Qu%2C+Jiaxi+Yang%2C+Binhua+Li%2C+Bowen+Li%2C+Bailin+Wang%2C+Bowen+Qin%2C+Ruiying+Geng%2C+Nan+Huo%2C+et+al%2E+2023%2E+Can+llm+already+serve+as+a+database+interface%3F+a+big+bench+for+large%2Dscale+)
28. Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Nathaneal Scharli, Aakanksha Chowdhery, Philip Mansfield, Blaise Aguera y Arcas, Dale Webste… — [Large Language Models Encode Clinical Knowledge](https://md.research.elamur.ai/free/en/papers/singhal-2022-large-language-models-encode-clinical/)
29. Meta AI. 2024. Introducing llama 4: Advancing multimodal intelligence. [find](https://scholar.google.com/scholar?q=Meta+AI%2E+2024%2E+Introducing+llama+4%3A+Advancing+multimodal+intelligence%2E)
30. Mistral AI. 2025. Mistral medium: Model overview and documentation. Product Documentation. [find](https://scholar.google.com/scholar?q=Mistral+AI%2E+2025%2E+Mistral+medium%3A+Model+overview+and+documentation%2E+Product+Documentation%2E)
31. OpenAI. 2025a. Gpt-4.1 and gpt-4.1 mini release notes. Release notes (14 May 2025) announcing GPT-4.1 and GPT-4.1 mini; GPT-4.1 excels at coding and precise instruction following, while GPT-4.1 mini is a fast, efficient model with a 1 M-token context window. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025a%2E+Gpt%2D4%2E1+and+gpt%2D4%2E1+mini+release+notes%2E+Release+notes+%2814+May+2025%29+announcing+GPT%2D4%2E1+and+GPT%2D4%2E1+mini%3B+GPT%2D4%2E1+excels+at+coding+and+precise+instruction+following%2C+while+GPT%2D4%2E1+mini+i)
32. OpenAI. 2025b. Introducing GPT-5 for developers. Announces API model sizes including gpt-5-mini. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025b%2E+Introducing+GPT%2D5+for+developers%2E+Announces+API+model+sizes+including+gpt%2D5%2Dmini%2E)
33. OpenAI. 2025c. Openai o4-mini: reasoning model release. Blog post (16 Apr 2025) announcing the o3 and o4-mini models; o4-mini is a small model optimized for fast, cost-efficient reasoning. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025c%2E+Openai+o4%2Dmini%3A+reasoning+model+release%2E+Blog+post+%2816+Apr+2025%29+announcing+the+o3+and+o4%2Dmini+models%3B+o4%2Dmini+is+a+small+model+optimized+for+fast%2C+cost%2Defficient+reasoning%2E)
34. Justice Ou, Tinglin Huang, Yilun Zhao, Ziyang Yu, Peiqing Lu, and Rex Ying. 2025. [Experience retrieval-augmentation with electronic health records enables accurate discharge qa](https://md.research.elamur.ai/free/en/papers/ou-2026-experience-retrieval-augmentation-electronic-health/).
35. Kaichen Ouyang, Zong Ke, Shengwei Fu, Lingjie Liu, Puning Zhao, and Dayu Hu. 2024. Learn from global correlations: Enhancing evolutionary algorithm via spectral gnn. arXiv preprint arXiv:2412.17629. — [arXiv:2412.17629](https://arxiv.org/abs/2412.17629) [find](https://scholar.google.com/scholar?q=Kaichen+Ouyang%2C+Zong+Ke%2C+Shengwei+Fu%2C+Lingjie+Liu%2C+Puning+Zhao%2C+and+Dayu+Hu%2E+2024%2E+Learn+from+global+correlations%3A+Enhancing+evolutionary+algorithm+via+spectral+gnn%2E+arXiv+preprint+arXiv%3A2412%2E17629%2E+%E2%80%94)
36. Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. [Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering](https://md.research.elamur.ai/free/en/papers/pal-2022-medmcqa-large-scale-multi-subject/). Proceedings of the Conference on Health, Inference, and Learning, pages 248–260.
37. Anusri Pampari, Preethi Raghavan, Jennifer Liang, and Jian Peng. 2018. [emrqa: A large corpus for question answering on electronic medical records](https://md.research.elamur.ai/free/en/papers/pampari-2018-emrqa-large-corpus-question-answering/). Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2357–2368.
38. Sithursan Sivasubramaniam, Cedric Osei-Akoto, Yi Zhang, Kurt Stockinger, and Jonathan Fuerst. 2024. SM3-text-to-query: Synthetic multi-model medical text-to-query benchmark. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. [find](https://scholar.google.com/scholar?q=Sithursan+Sivasubramaniam%2C+Cedric+Osei%2DAkoto%2C+Yi+Zhang%2C+Kurt+Stockinger%2C+and+Jonathan+Fuerst%2E+2024%2E+SM3%2Dtext%2Dto%2Dquery%3A+Synthetic+multi%2Dmodel+medical+text%2Dto%2Dquery+benchmark%2E+In+The+Thirty%2Deight+Confer)
39. Siddharth Soni, Kirk Roberts, Daqing Wang, Arvind Subburathinam, and Sivaramakrishnan Long. 2022. Radqa: A question answering dataset to improve comprehension of radiology reports. In Proceedings of the Language Resources and Evaluation Conference, pages 6567–6577. [find](https://scholar.google.com/scholar?q=Siddharth+Soni%2C+Kirk+Roberts%2C+Daqing+Wang%2C+Arvind+Subburathinam%2C+and+Sivaramakrishnan+Long%2E+2022%2E+Radqa%3A+A+question+answering+dataset+to+improve+comprehension+of+radiology+reports%2E+In+Proceedings+of+t)
40. Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, Johannes Heidecke, Amelia Glaese, and Tejal Patwardhan. 2025. [Paperbench: Evaluating ai’s ability to replicate ai research](https://md.research.elamur.ai/free/en/papers/starace-2025-paperbench-evaluating-ai-s/).
41. Hanna Suominen, Liadh Kelly, Lorraine Goeuriot, and Martin Krallinger. 2020. Clef ehealth evaluation lab 2020. In Advances in Information Retrieval: 42nd European Conference on Information Retrieval (ECIR 2020), Part II, volume 12036 of Lecture Notes in Computer Science, pages 587–594. Springer. [find](https://scholar.google.com/scholar?q=Hanna+Suominen%2C+Liadh+Kelly%2C+Lorraine+Goeuriot%2C+and+Martin+Krallinger%2E+2020%2E+Clef+ehealth+evaluation+lab+2020%2E+In+Advances+in+Information+Retrieval%3A+42nd+European+Conference+on+Information+Retrieval+%28)
42. Logesh Kumar Umapathi Wang, Ankit Pal, and Malaikannan Sankarasubbu. 2022. Drugehrqa: A question answering dataset on structured and unstructured electronic health records. In Proceedings of the Language Resources and Evaluation Conference, pages 2789–2797. [find](https://scholar.google.com/scholar?q=Logesh+Kumar+Umapathi+Wang%2C+Ankit+Pal%2C+and+Malaikannan+Sankarasubbu%2E+2022%2E+Drugehrqa%3A+A+question+answering+dataset+on+structured+and+unstructured+electronic+health+records%2E+In+Proceedings+of+the+Langu)
43. Xiaoxuan Wang, Pavan Kapanipathi, Ryan Musa, Mo Yu, Kartik Talamadupula, Ibrahim Abdelaziz, Maria Chang, Achille Fokoue, Bassem Makni, Nicholas Mattei, et al. 2020. Mimicsql: Text-to-sql generation for question answering on electronic medical records. In Proceedings of The Web Conference 2020, pages… [find](https://scholar.google.com/scholar?q=Xiaoxuan+Wang%2C+Pavan+Kapanipathi%2C+Ryan+Musa%2C+Mo+Yu%2C+Kartik+Talamadupula%2C+Ibrahim+Abdelaziz%2C+Maria+Chang%2C+Achille+Fokoue%2C+Bassem+Makni%2C+Nicholas+Mattei%2C+et+al%2E+2020%2E+Mimicsql%3A+Text%2Dto%2Dsql+generation+fo)
44. Yifan Zhang, Jingqing Chen, Qiao Yuan, Zhihao Ding, Huaishao Luo, Kaixiong Pan, Mengzhuo Zhang, Haiyang Yu, Qingyun Chen, Xiangru Wang, et al. 2025. [Medxpertqa: Benchmarking expert-level medical reasoning and understanding](https://md.research.elamur.ai/free/en/papers/zuo-2025-medxpertqa-benchmarking-expert-level-medical/). arXiv preprint arXiv:2501.18362.
45. Victor Zhong, Caiming Xiong, and Richard Socher. 2017. [Seq2sql: Generating structured queries from natural language using reinforcement learning](https://md.research.elamur.ai/free/en/papers/zhong-2017-seq2sql-generating-structured-queries-natural/). In arXiv preprint arXiv:1709.00103.
46. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024. [MMLU-pro: A more robust and challenging multi-task language understanding benchmark](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/)…
47. Fangyu Wei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, et al. 2024. Spider 2.0: Evaluating language models on real-world enterprise text-to-sql workflows. arXiv preprint arXiv:2411.07763. — [arXiv:2411.07763](https://arxiv.org/abs/2411.07763) [find](https://scholar.google.com/scholar?q=Fangyu+Wei%2C+Jixuan+Chen%2C+Yuxiao+Ye%2C+Ruisheng+Cao%2C+Dongchan+Shin%2C+Hongjin+Su%2C+Zhaoqing+Suo%2C+Hongcheng+Gao%2C+Wenjing+Hu%2C+Pengcheng+Yin%2C+et+al%2E+2024%2E+Spider+2%2E0%3A+Evaluating+language+models+on+real%2Dworld+e)
48. Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Ben Feuer, Siddhartha Jain, Ravid Shwartz-Ziv, Neel Jain, Khalid Saifullah, Siddartha Naidu, et al. 2024. Livebench: A challenging, contamination-limited llm benchmark. arXiv preprint arXiv:2406.19314. [find](https://scholar.google.com/scholar?q=Colin+White%2C+Samuel+Dooley%2C+Manley+Roberts%2C+Arka+Pal%2C+Ben+Feuer%2C+Siddhartha+Jain%2C+Ravid+Shwartz%2DZiv%2C+Neel+Jain%2C+Khalid+Saifullah%2C+Siddartha+Naidu%2C+et+al%2E+2024%2E+Livebench%3A+A+challenging%2C+contamination%2D)
49. Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Benjamin Feuer, Siddhartha Jain, Ravid Shwartz-Ziv, Neel Jain, Khalid Saifullah, Sreemanti Dey, Shubh-Agrawal, Sandeep Singh Sandha, Siddartha Venkat Naidu, Chinmay Hegde, Yann LeCun, Tom Goldstein, Willie Neiswanger, and Micah Goldblum. 2025. Li… [find](https://scholar.google.com/scholar?q=Colin+White%2C+Samuel+Dooley%2C+Manley+Roberts%2C+Arka+Pal%2C+Benjamin+Feuer%2C+Siddhartha+Jain%2C+Ravid+Shwartz%2DZiv%2C+Neel+Jain%2C+Khalid+Saifullah%2C+Sreemanti+Dey%2C+Shubh%2DAgrawal%2C+Sandeep+Singh+Sandha%2C+Siddartha+Ven)
50. Zheyuan Yang, Lyuhao Chen, Arman Cohan, and Yilun Zhao. 2025. Table-r1: Inference-time scaling for table reasoning tasks. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20605–20624, Suzhou, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zheyuan+Yang%2C+Lyuhao+Chen%2C+Arman+Cohan%2C+and+Yilun+Zhao%2E+2025%2E+Table%2Dr1%3A+Inference%2Dtime+scaling+for+table+reasoning+tasks%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+Natural+Language)
51. Tao Yu, Rui Zhang, Kai Yang, Michihiko Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. 2018. [Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task](https://md.research.elamur.ai/free/en/papers/yu-2018-spider-large-scale-human-labeled/). In Proceedings of the 2018 Conference on Empirical Me…

*Updated 2 November 2026*