---
title: "Baichuan-M1 — paper by Wang et al. (2025)"
description: "Baichuan-M1 instead trains a model from scratch with medical competence as a design goal. 81 references, cited by 19 papers here."
authors: ["Bingning Wang", "Haizhou Zhao", "Huozhi Zhou", "Liang Song", "Mingyu Xu", "Wei Cheng", "Xiangrong Zeng", "Yupeng Zhang", "Yuqi Huo", "Zecheng Wang", "Zhengyun Zhao", "Da Pan", "Fei Kou", "Fei Li", "Fuzhong Chen", "Guosheng Dong", "Han Liu", "Hongda Zhang", "Jin He", "Jinjie Yang", "Kangxi Wu", "Kegeng Wu", "Lei Su", "Linlin Niu", "Linzhuang Sun", "Mang Wang", "Pengcheng Fan", "Qianli Shen", "Rihui Xin", "Shunya Dang", "Songchi Zhou", "Weipeng Chen", "Wenjing Luo", "Xin Chen", "Xin Men", "Xionghai Lin", "Xuezhen Dong", "Yan Zhang", "Yifei Duan", "Yuyan Zhou", "Zhi Ma", "Zhiying Wu"]
year: 2025
venue: "arXiv:2502.12671"
source: "https://arxiv.org/abs/2502.12671"
doi: "10.48550/arxiv.2502.12671"
arxiv: "2502.12671"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2025-baichuan-m1-pushing-medical/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2025-baichuan-m1-pushing-medical/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/) / Baichuan-M1: Pushing the Medical Capability of Large Language Models

# Baichuan-M1: Pushing the Medical Capability of Large Language Models

**Authors:** [Bingning Wang](https://md.research.elamur.ai/free/en/authors/bingning-wang/), Haizhou Zhao, Huozhi Zhou, Liang Song, [Mingyu Xu](https://md.research.elamur.ai/free/en/authors/mingyu-xu/), [Wei Cheng](https://md.research.elamur.ai/free/en/authors/wei-cheng/), [Xiangrong Zeng](https://md.research.elamur.ai/free/en/authors/xiangrong-zeng/), Yupeng Zhang, [Yuqi Huo](https://md.research.elamur.ai/free/en/authors/yuqi-huo/), Zecheng Wang, Zhengyun Zhao, [Da Pan](https://md.research.elamur.ai/free/en/authors/da-pan/), Fei Kou, [Fei Li](https://md.research.elamur.ai/free/en/authors/fei-li/), Fuzhong Chen, [Guosheng Dong](https://md.research.elamur.ai/free/en/authors/guosheng-dong/), [Han Liu](https://md.research.elamur.ai/free/en/authors/han-liu/), Hongda Zhang, Jin He, Jinjie Yang, Kangxi Wu, Kegeng Wu, Lei Su, Linlin Niu, [Linzhuang Sun](https://md.research.elamur.ai/free/en/authors/linzhuang-sun/), Mang Wang, Pengcheng Fan, Qianli Shen, Rihui Xin, Shunya Dang, Songchi Zhou, [Weipeng Chen](https://md.research.elamur.ai/free/en/authors/weipeng-chen/), Wenjing Luo, [Xin Chen](https://md.research.elamur.ai/free/en/authors/xin-chen/), [Xin Men](https://md.research.elamur.ai/free/en/authors/xin-men/), Xionghai Lin, Xuezhen Dong, [Yan Zhang](https://md.research.elamur.ai/free/en/authors/yan-zhang/), Yifei Duan, [Yuyan Zhou](https://md.research.elamur.ai/free/en/authors/yuyan-zhou/), Zhi Ma and Zhiying Wu · **Year:** 2025 · **Venue:** arXiv:2502.12671 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2502.12671) · [PDF](https://arxiv.org/pdf/2502.12671) · [DOI 10.48550/arxiv.2502.12671](https://doi.org/10.48550/arxiv.2502.12671) · [arXiv 2502.12671](https://arxiv.org/abs/2502.12671) · [OpenAlex](https://openalex.org/W4407759590)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [19](https://md.research.elamur.ai/free/en/papers/wang-2025-baichuan-m1-pushing-medical/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
General-purpose LLMs often lack depth in medicine, where knowledge is specialized and good training data is scarce. Most prior medical models adapt an existing general model through continued pretraining or fine-tuning. Baichuan-M1 instead trains a model from scratch with medical competence as a design goal. The pretraining corpus has about 20T tokens, mixing general text, code and multilingual data with clinical and patient-related material. The data pipeline uses global deduplication with upsampling by duplicate count, multidimensional quality scoring with small models, and domain classification for rebalancing. A three-stage curriculum gradually raises the share and difficulty of medical data while lengthening the context window. More than 100B synthetic medical reasoning tokens supplement it, and the architecture is a modified Transformer. The released Baichuan-M1-14B-Instruct beats Qwen2.5-72B-Instruct on medical benchmarks and trails GPT-4o and Claude-3.5-Sonnet only modestly. The work matters because it gives evidence that a moderately sized model trained from scratch can be competitive in a vertical domain, and because the weights are openly released.

## Key points
- Corpus composition: 12T English, 4T Chinese, 2T multilingual (top 30 languages) and 2T code tokens. Overrepresented web domains are downsampled across 27 categories, and STEM content is upsampled.
- Data ablations:
  - Ten repetitions of the top 10% quality data perform about as well as three repetitions of the top 33%.
  - For code, keeping the top 20% (scored by a fine-tuned Qwen2.5 Coder) with 5x upsampling works best.
  - Adding roughly 10% top-quality multilingual data slightly improves Chinese and English results.
- Open releases: Baichuan-M1-14B-Base and Baichuan-M1-14B-Instruct on GitHub and Hugging Face.
- Evaluation setup:
  - Free-text answers are scored by Qwen2.5-72B-Instruct, with at most 1000 samples per benchmark.
  - Benchmarks include RareBench, MedCalc and MMLU medical subsets.
  - A new NEJMQA set was built from NEJM case challenges covering about 200 patients.
- Base model results versus Qwen2.5-14B: MBPP 74.0 vs 72.8, HumanEval 60.4 vs 56.7, MATH 46.0 vs 45.4. The authors also report that continued pretraining of annealed base models hurt general skills when boosting medical ones.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 241 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 184 works

## Cite

Cite as: Bingning Wang, Haizhou Zhao, Huozhi Zhou, Liang Song, Mingyu Xu, Wei Cheng, Xiangrong Zeng, Yupeng Zhang, Yuqi Huo, Zecheng Wang, Zhengyun Zhao, Da Pan, Fei Kou, Fei Li, Fuzhong Chen, Guosheng Dong, Han Liu, Hongda Zhang, Jin He, Jinjie Yang, Kangxi Wu, Kegeng Wu, Lei Su, Linlin Niu, Linzhuang Sun, Mang Wang, Pengcheng Fan, Qianli Shen, Rihui Xin, Shunya Dang, Songchi Zhou, Weipeng Chen, Wenjing Luo, Xin Chen, Xin Men, Xionghai Lin, Xuezhen Dong, Yan Zhang, Yifei Duan, Yuyan Zhou, Zhi Ma and Zhiying Wu. 2025. Baichuan-M1: Pushing the Medical Capability of Large Language Models. arXiv:2502.12671.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{wang2025baichuan,
  title = {{Baichuan-M1: Pushing the Medical Capability of Large Language Models}},
  author = {Bingning Wang, Haizhou Zhao, Huozhi Zhou, Liang Song, Mingyu Xu, Wei Cheng, Xiangrong Zeng, Yupeng Zhang, Yuqi Huo, Zecheng Wang, Zhengyun Zhao, Da Pan, Fei Kou, Fei Li, Fuzhong Chen, Guosheng Dong, Han Liu, Hongda Zhang, Jin He, Jinjie Yang, Kangxi Wu, Kegeng Wu, Lei Su, Linlin Niu, Linzhuang Sun, Mang Wang, Pengcheng Fan, Qianli Shen, Rihui Xin, Shunya Dang, Songchi Zhou, Weipeng Chen, Wenjing Luo, Xin Chen, Xin Men, Xionghai Lin, Xuezhen Dong, Yan Zhang, Yifei Duan, Yuyan Zhou, Zhi Ma and Zhiying Wu},
  year = {2025},
  doi = {10.48550/arxiv.2502.12671},
  eprint = {2502.12671},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2502.12671}
}
```

- [← previous Aligning LLMs with Individual Preferences via Interaction](https://md.research.elamur.ai/free/en/papers/wu-2025-aligning-llms-individual-preferences-interaction/)
- [next → ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation](https://md.research.elamur.ai/free/en/papers/zhao-2025-chartcoder-advancing-multimodal-large-language/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/)

## References

81 references, 41 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rarearena. [https://github.com/zhao-zy15/RareArena/](https://github.com/zhao-zy15/RareArena/), 2024. [↗](https://github.com/zhao-zy15/RareArena/)
2. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Sali… — [Phi-4 Technical Report](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-4-technical-report/)
3. Ibrahim Adeshola and Adeola Praise Adepoju. The opportunities and challenges of chatgpt in education. Interactive Learning Environments, 32(10):6159–6172, 2024. [find](https://scholar.google.com/scholar?q=Ibrahim+Adeshola+and+Adeola+Praise+Adepoju%2E+The+opportunities+and+challenges+of+chatgpt+in+education%2E+Interactive+Learning+Environments%2C+32%2810%29%3A6159%E2%80%936172%2C+2024%2E)
4. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/), 2021. URL https://arxiv.org/abs/2108.07732. [↗](https://arxiv.org/abs/2108.07732)
5. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. [Qwen technical report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/). arXiv preprint arXiv:2309.16609, 2023.
6. Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xlstm: Extended long short-term memory. arXiv preprint arXiv:2405.04517, 2024. [find](https://scholar.google.com/scholar?q=Maximilian+Beck%2C+Korbinian+P%C3%B6ppel%2C+Markus+Spanring%2C+Andreas+Auer%2C+Oleksandra+Prudnikova%2C+Michael+Kopp%2C+G%C3%BCnter+Klambauer%2C+Johannes+Brandstetter%2C+and+Sepp+Hochreiter%2E+xlstm%3A+Extended+long+short%2Dterm+mem)
7. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 7432–7439, 2020.
8. Hanjie Chen, Zhouxiang Fang, Yash Singla, and Mark Dredze. [Benchmarking large language models on answering and explaining challenging medical questions](https://md.research.elamur.ai/free/en/papers/chen-2025-benchmarking-large-language-models-answering/). arXiv preprint arXiv:2402.18060, 2024a.
9. Hao Chen, Abdul Waheed, Xiang Li, Yidong Wang, Jindong Wang, Bhiksha Raj, and Marah I. Abdin. [On the diversity of synthetic data and its impact on training large language models](https://md.research.elamur.ai/free/en/papers/chen-2024-diversity-synthetic-data-impact-training/), 2024b. URL https://arxiv.org/abs/2410.15226. [↗](https://arxiv.org/abs/2410.15226)
10. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. Huatuogpt-o1, towards medical complex reasoning with llms. arXiv preprint arXiv:2412.18925, 2024c. [find](https://scholar.google.com/scholar?q=Junying+Chen%2C+Zhenyang+Cai%2C+Ke+Ji%2C+Xidong+Wang%2C+Wanlong+Liu%2C+Rongsheng+Wang%2C+Jianye+Hou%2C+and+Benyou+Wang%2E+Huatuogpt%2Do1%2C+towards+medical+complex+reasoning+with+llms%2E+arXiv+preprint+arXiv%3A2412%2E18925%2C+20)
11. Justin Chih-Yao Chen, Zifeng Wang, Hamid Palangi, Rujun Han, Sayna Ebrahimi, Long Le, Vincent Perot, Swaroop Mishra, Mohit Bansal, Chen-Yu Lee, and Tomas Pfister. Reverse thinking makes llms stronger reasoners, 2024d. URL https://arxiv.org/abs/2411. 19865. [↗](https://arxiv.org/abs/2411)
12. Xuanzhong Chen, Xiaohao Mao, Qihan Guo, Lun Wang, Shuyang Zhang, and Ting Chen. Rarebench: Can llms serve as rare diseases specialists? In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4850–4861, 2024e. [find](https://scholar.google.com/scholar?q=Xuanzhong+Chen%2C+Xiaohao+Mao%2C+Qihan+Guo%2C+Lun+Wang%2C+Shuyang+Zhang%2C+and+Ting+Chen%2E+Rarebench%3A+Can+llms+serve+as+rare+diseases+specialists%3F+In+Proceedings+of+the+30th+ACM+SIGKDD+Conference+on+Knowledge+Di)
13. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457, 2018.
14. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, H… [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2C+Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+Damai+Dai%2C+Daya+Guo%2C+Dejian+Yang%2C+Deli+Chen%2C+Dongjie+Ji%2C+Erhan)
15. Longxu Dou, Qian Liu, Guangtao Zeng, Jia Guo, Jiahui Zhou, Wei Lu, and Min Lin. [Sailor: Open language models for south-east asia](https://md.research.elamur.ai/free/en/papers/dou-2024-sailor-open-language-models-south/). arXiv preprint arXiv:2404.03608, 2024.
16. Daniel Y Fu, Tri Dao, Khaled K Saab, Armin W Thomas, Atri Rudra, and Christopher Ré. Hungry hungry hippos: Towards language modeling with state space models. arXiv preprint arXiv:2212.14052, 2022. [find](https://scholar.google.com/scholar?q=Daniel+Y+Fu%2C+Tri+Dao%2C+Khaled+K+Saab%2C+Armin+W+Thomas%2C+Atri+Rudra%2C+and+Christopher+R%C3%A9%2E+Hungry+hungry+hippos%3A+Towards+language+modeling+with+state+space+models%2E+arXiv+preprint+arXiv%3A2212%2E14052%2C+2022%2E)
17. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948, 2025.
18. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300, 2020.
19. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/), 2021.
20. Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, et al. [Minicpm: Unveiling the potential of small language models with scalable training strategies](https://md.research.elamur.ai/free/en/papers/hu-2024-minicpm-unveiling-potential-small-language/). arXiv preprint arXiv:2404.06395, 2024.
21. Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, Yao Fu, Maosong Sun, and Junxian He. [C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models](https://md.research.elamur.ai/free/en/papers/huang-2023-c-eval-multi-level-multi/). In Advances in Neural Information Processing…
22. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin. Qwen… — [Qwen2.5-Coder Technical Report](https://md.research.elamur.ai/free/en/papers/hui-2024-qwen2-5-coder-technical-report/)
23. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. [What disease does this patient have? a large-scale open domain question answering dataset from medical exams](https://md.research.elamur.ai/free/en/papers/jin-2020-disease-this-patient-have-large/). Applied Sciences, 11(14):6421, 2021.
24. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. [Pubmedqa: A dataset for biomedical research question answering](https://md.research.elamur.ai/free/en/papers/jin-2019-pubmedqa-dataset-biomedical-research-question/). arXiv preprint arXiv:1909.06146, 2019.
25. Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. [Mimic-iii, a freely accessible critical care database](https://md.research.elamur.ai/free/en/papers/johnson-2016-mimic-iii-freely-accessible-critical/). Scientific data, 3(1):1–9, 2016.
26. Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, et al. [Mimic-iv, a freely accessible electronic health record dataset](https://md.research.elamur.ai/free/en/papers/johnson-2023-mimic-iv-freely-accessible-electronic/). Scientific data, 10(1):1, 2023.
27. Nikhil Khandekar, Qiao Jin, Guangzhi Xiong, Soren Dunn, Serina S Applebaum, Zain Anwar, Maame Sarfo-Gyamfi, Conrad W Safranek, Abid A Anwar, Andrew Zhang, Aidan Gilson, Maxwell B Singer, Amisha Dave, Andrew Taylor, Aidong Zhang, Qingyu Chen, and Zhiyong Lu. Medcalc-bench: Evaluating large language m… [find](https://scholar.google.com/scholar?q=Nikhil+Khandekar%2C+Qiao+Jin%2C+Guangzhi+Xiong%2C+Soren+Dunn%2C+Serina+S+Applebaum%2C+Zain+Anwar%2C+Maame+Sarfo%2DGyamfi%2C+Conrad+W+Safranek%2C+Abid+A+Anwar%2C+Andrew+Zhang%2C+Aidan+Gilson%2C+Maxwell+B+Singer%2C+Amisha+Dave%2C+)
28. T Kudo. [Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing](https://md.research.elamur.ai/free/en/papers/kudo-2018-sentencepiece-simple-language-independent-subword/). arXiv preprint arXiv:1808.06226, 2018.
29. Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. [Cmmlu: Measuring massive multitask language understanding in chinese](https://md.research.elamur.ai/free/en/papers/li-2023-cmmlu-measuring-massive-multitask-language/), 2024a.
30. Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. Mediq: Question-asking llms for adaptive and reliable medical reasoning. arXiv preprint arXiv:2406.[00922, 2024b](https://md.research.elamur.ai/free/en/papers/li-2024-mediq-question-asking-llms-benchmark/).
31. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. [Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/). Advances in Neural Information Processing Systems, 36, 2024a.
32. Junling Liu, Peilin Zhou, Yining Hua, Dading Chong, Zhongyu Tian, Andrew Liu, Helin Wang, Chenyu You, Zhenhua Guo, Lei Zhu, et al. Benchmarking large language models on cmexam-a comprehensive chinese medical exam dataset. Advances in Neural Information Processing Systems, 36, 2024b. [find](https://scholar.google.com/scholar?q=Junling+Liu%2C+Peilin+Zhou%2C+Yining+Hua%2C+Dading+Chong%2C+Zhongyu+Tian%2C+Andrew+Liu%2C+Helin+Wang%2C+Chenyu+You%2C+Zhenhua+Guo%2C+Lei+Zhu%2C+et+al%2E+Benchmarking+large+language+models+on+cmexam%2Da+comprehensive+chinese+)
33. Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101%2C+2017%2E)
34. Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, and Navdeep Jaitly. Rephrasing the web: A recipe for compute and data-efficient language modeling, 2024. URL https://arxiv.org/abs/2401.16380. [↗](https://arxiv.org/abs/2401.16380)
35. Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. [Shortgpt: Layers in large language models are more redundant than you expect](https://md.research.elamur.ai/free/en/papers/men-2024-shortgpt-layers-large-language-models/). arXiv preprint arXiv:2403.03853, 2024.
36. OpenAI. Introducing chatgpt. Blog post openai.com/blog/chatgpt, 2022. [find](https://scholar.google.com/scholar?q=OpenAI%2E+Introducing+chatgpt%2E+Blog+post+openai%2Ecom%2Fblog%2Fchatgpt%2C+2022%2E)
37. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). ArXiv, abs/2303.08774, 2023.
38. Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. [The fineweb datasets: Decanting the web for the finest text data at scale](https://md.research.elamur.ai/free/en/papers/penedo-2024-fineweb-datasets-decanting-web-finest/), 2024. URL https://arxiv.org/abs/2406.17557. [↗](https://arxiv.org/abs/2406.17557)
39. Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, et al. Rwkv: Reinventing rnns for the transformer era. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 14048–14077, 2023. [find](https://scholar.google.com/scholar?q=Bo+Peng%2C+Eric+Alcaide%2C+Quentin+Anthony%2C+Alon+Albalak%2C+Samuel+Arcadinho%2C+Stella+Biderman%2C+Huanqi+Cao%2C+Xin+Cheng%2C+Michael+Chung%2C+Leon+Derczynski%2C+et+al%2E+Rwkv%3A+Reinventing+rnns+for+the+transformer+era%2E+I)
40. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng… [find](https://scholar.google.com/scholar?q=Qwen%2C+%3A%2C+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+)
41. Alexey Romanov and Chaitanya Shivade. [Lessons from natural language inference in the clinical domain](https://md.research.elamur.ai/free/en/papers/romanov-2018-lessons-natural-language-inference-clinical/). 2018. URL http://arxiv.org/abs/1808.06752. [↗](http://arxiv.org/abs/1808.06752)
42. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106, 2021. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2C+2021%2E)
43. Mohammad-Javad Sanaei, Mehrnaz Sadat Ravari, and Hassan Abolghasemi. Chatgpt in medicine: Opportunity and challenges. Iranian Journal of Blood and Cancer, 15(3):60–67, 2023. [find](https://scholar.google.com/scholar?q=Mohammad%2DJavad+Sanaei%2C+Mehrnaz+Sadat+Ravari%2C+and+Hassan+Abolghasemi%2E+Chatgpt+in+medicine%3A+Opportunity+and+challenges%2E+Iranian+Journal+of+Blood+and+Cancer%2C+15%283%29%3A60%E2%80%9367%2C+2023%2E)
44. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2C+2017%2E)
45. Rico Sennrich. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). arXiv preprint arXiv:1508.07909, 2015.
46. Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202, 2020. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+Glu+variants+improve+transformer%2E+arXiv+preprint+arXiv%3A2002%2E05202%2C+2020%2E)
47. Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, and Raghav Mani. Biomegatron: larger biomedical domain language model. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4700–4706, 2020. [find](https://scholar.google.com/scholar?q=Hoo%2DChang+Shin%2C+Yang+Zhang%2C+Evelina+Bakhturina%2C+Raul+Puri%2C+Mostofa+Patwary%2C+Mohammad+Shoeybi%2C+and+Raghav+Mani%2E+Biomegatron%3A+larger+biomedical+domain+language+model%2E+In+Proceedings+of+the+2020+Conferen)
48. Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, et al. Toward expert-level medical question answering with large language models. Nature Medicine, pp. 1–8, 2025. [find](https://scholar.google.com/scholar?q=Karan+Singhal%2C+Tao+Tu%2C+Juraj+Gottweis%2C+Rory+Sayres%2C+Ellery+Wulczyn%2C+Mohamed+Amin%2C+Le+Hou%2C+Kevin+Clark%2C+Stephen+R+Pfohl%2C+Heather+Cole%2DLewis%2C+et+al%2E+Toward+expert%2Dlevel+medical+question+answering+with+l)
49. David So, Wojciech Mańke, Hanxiao Liu, Zihang Dai, Noam Shazeer, and Quoc V Le. Searching for efficient transformers for language modeling. Advances in neural information processing systems, 34:6010–6022, 2021. [find](https://scholar.google.com/scholar?q=David+So%2C+Wojciech+Man%CC%81ke%2C+Hanxiao+Liu%2C+Zihang+Dai%2C+Noam+Shazeer%2C+and+Quoc+V+Le%2E+Searching+for+efficient+transformers+for+language+modeling%2E+Advances+in+neural+information+processing+systems%2C+34%3A6010%E2%80%93)
50. Kyle Dylan Spurlock, Cagla Acun, Esin Saka, and Olfa Nasraoui. Chatgpt for conversational recommendation: Refining recommendations by reprompting with feedback. arXiv preprint arXiv:2401.03605, 2024. — [arXiv:2401.03605](https://arxiv.org/abs/2401.03605) [find](https://scholar.google.com/scholar?q=Kyle+Dylan+Spurlock%2C+Cagla+Acun%2C+Esin+Saka%2C+and+Olfa+Nasraoui%2E+Chatgpt+for+conversational+recommendation%3A+Refining+recommendations+by+reprompting+with+feedback%2E+arXiv+preprint+arXiv%3A2401%2E03605%2C+2024%2E+)
51. Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. [Roformer: Enhanced transformer with rotary position embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/). Neurocomputing, 568:127063, 2024.
52. Liping Tang, Nikhil Ranjan, Omkar Pangarkar, Xuezhi Liang, Zhen Wang, Li An, Bhaskar Rao, Linghao Jin, Huijuan Wang, Zhoujun Cheng, Suqi Sun, Cun Mu, Victor Miller, Xuezhe Ma, Yue Peng, Zhengzhong Liu, and Eric P. Xing. Txt360: A top-quality llm pre-training dataset requires the perfect blend. https… [find](https://scholar.google.com/scholar?q=Liping+Tang%2C+Nikhil+Ranjan%2C+Omkar+Pangarkar%2C+Xuezhi+Liang%2C+Zhen+Wang%2C+Li+An%2C+Bhaskar+Rao%2C+Linghao+Jin%2C+Huijuan+Wang%2C+Zhoujun+Cheng%2C+Suqi+Sun%2C+Cun+Mu%2C+Victor+Miller%2C+Xuezhe+Ma%2C+Yue+Peng%2C+Zhengzhong+Liu)
53. Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. [Galactica: A large language model for science](https://md.research.elamur.ai/free/en/papers/taylor-2022-galactica-large-language-model-science/). arXiv preprint arXiv:2211.09085, 2022.
54. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+Cassidy+Hardin%2C+Surya+Bhupatiraju%2C+L%C3%A9onard+Hussenot%2C+Thomas+Mesnard%2C+Bobak+Shahriari%2C+Alexandre+Ram%C3%A9%2C+et+al%2E+Gemma+2%3A+Improving+open+la)
55. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/), 2023.
56. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30, 2017.
57. Roger Waleffe, Wonmin Byeon, Duncan Riach, Brandon Norick, Vijay Anand Korthikanti, Tri Dao, Albert Gu, Ali Hatamizadeh, Sudhakar Singh, Deepak Narayanan, Garvit Kulshreshtha, Vartika Singh, Jared Casper, Jan Kautz, Mohammad Shoeybi, and Bryan Catanzaro. An empirical study of mamba-based language mo… [find](https://scholar.google.com/scholar?q=Roger+Waleffe%2C+Wonmin+Byeon%2C+Duncan+Riach%2C+Brandon+Norick%2C+Vijay+Anand+Korthikanti%2C+Tri+Dao%2C+Albert+Gu%2C+Ali+Hatamizadeh%2C+Sudhakar+Singh%2C+Deepak+Narayanan%2C+Garvit+Kulshreshtha%2C+Vartika+Singh%2C+Jared+Cas)
58. Dandan Wang and Shiqing Zhang. Large language models in medical and healthcare fields: applications, advances, and challenges. Artificial Intelligence Review, 57(11):299, 2024. [find](https://scholar.google.com/scholar?q=Dandan+Wang+and+Shiqing+Zhang%2E+Large+language+models+in+medical+and+healthcare+fields%3A+applications%2C+advances%2C+and+challenges%2E+Artificial+Intelligence+Review%2C+57%2811%29%3A299%2C+2024%2E)
59. Guanchu Wang, Junhao Ran, Ruixiang Tang, Chia-Yuan Chang, Yu-Neng Chuang, Zirui Liu, Vladimir Braverman, Zhandong Liu, and Xia Hu. Assessing and enhancing large language models in rare disease question-answering. arXiv preprint arXiv:2408.08422, 2024a. — [arXiv:2408.08422](https://arxiv.org/abs/2408.08422) [find](https://scholar.google.com/scholar?q=Guanchu+Wang%2C+Junhao+Ran%2C+Ruixiang+Tang%2C+Chia%2DYuan+Chang%2C+Yu%2DNeng+Chuang%2C+Zirui+Liu%2C+Vladimir+Braverman%2C+Zhandong+Liu%2C+and+Xia+Hu%2E+Assessing+and+enhancing+large+language+models+in+rare+disease+questio)
60. Haochun Wang, Sendong Zhao, Zewen Qiang, Zijian Li, Chi Liu, Nuwa Xi, Yanrui Du, Bing Qin, and Ting Liu. Knowledge-tuning large language models with structured medical knowledge bases for trustworthy response generation in chinese. ACM Trans. Knowl. Discov. Data, August 2024b. ISSN 1556-4681. doi: 1… — [doi:10.1145/3686807](https://doi.org/10.1145/3686807) [find](https://scholar.google.com/scholar?q=Haochun+Wang%2C+Sendong+Zhao%2C+Zewen+Qiang%2C+Zijian+Li%2C+Chi+Liu%2C+Nuwa+Xi%2C+Yanrui+Du%2C+Bing+Qin%2C+and+Ting+Liu%2E+Knowledge%2Dtuning+large+language+models+with+structured+medical+knowledge+bases+for+trustworthy+)
61. Xidong Wang, Guiming Hardy Chen, Dingjie Song, Zhiyi Zhang, Zhihong Chen, Qingying Xiao, Feng Jiang, Jianquan Li, Xiang Wan, Benyou Wang, et al. [Cmb: A comprehensive medical benchmark in chinese](https://md.research.elamur.ai/free/en/papers/wang-2023-cmb-comprehensive-medical-benchmark/). arXiv preprint arXiv:2308.08833, 2023.
62. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, F. Xia, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. ArXiv, abs/2201.[11903, 2022](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 246411621. [↗](https://api.semanticscholar.org/CorpusID:)
63. Tianwen Wei, Jian Luan, Wei Liu, Shuang Dong, and Bin Wang. [Cmath: Can your language model pass chinese elementary school math test?](https://md.research.elamur.ai/free/en/papers/wei-2023-cmath-language-model-pass/), 2023.
64. Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. [Retrieval head mechanistically explains long-context factuality](https://md.research.elamur.ai/free/en/papers/wu-2024-retrieval-head-mechanistically-explains-long/). arXiv preprint arXiv:2404.15574, 2024.
65. Jingjing Xu, Xu Sun, Zhiyuan Zhang, Guangxiang Zhao, and Junyang Lin. Understanding and improving layer normalization. Advances in neural information processing systems, 32, 2019. [find](https://scholar.google.com/scholar?q=Jingjing+Xu%2C+Xu+Sun%2C+Zhiyuan+Zhang%2C+Guangxiang+Zhao%2C+and+Junyang+Lin%2E+Understanding+and+improving+layer+normalization%2E+Advances+in+neural+information+processing+systems%2C+32%2C+2019%2E)
66. Mingyu Xu, Wei Cheng, Bingning Wang, and Weipeng Chen. Kv shifting attention enhances language modeling. arXiv preprint arXiv:2411.19574, 2024a. — [arXiv:2411.19574](https://arxiv.org/abs/2411.19574) [find](https://scholar.google.com/scholar?q=Mingyu+Xu%2C+Wei+Cheng%2C+Bingning+Wang%2C+and+Weipeng+Chen%2E+Kv+shifting+attention+enhances+language+modeling%2E+arXiv+preprint+arXiv%3A2411%2E19574%2C+2024a%2E+%E2%80%94+%5BarXiv%3A2411%2E19574%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2411%2E19574%29)
67. Mingyu Xu, Xin Men, Bingning Wang, Qingyu Zhang, Hongyu Lin, Xianpei Han, et al. [Base of rope bounds context length](https://md.research.elamur.ai/free/en/papers/men-2024-base-rope-bounds-context/). In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024b.
68. Weixiang Yan, Haitian Liu, Tengxiao Wu, Qian Chen, Wen Wang, Haoyuan Chai, Jiayi Wang, Weishan Zhao, Yixin Zhang, Renjun Zhang, et al. Clinicallab: Aligning agents for multi-departmental clinical diagnostics in the real world. arXiv preprint arXiv:2406.13890, 2024. — [arXiv:2406.13890](https://arxiv.org/abs/2406.13890) [find](https://scholar.google.com/scholar?q=Weixiang+Yan%2C+Haitian+Liu%2C+Tengxiao+Wu%2C+Qian+Chen%2C+Wen+Wang%2C+Haoyuan+Chai%2C+Jiayi+Wang%2C+Weishan+Zhao%2C+Yixin+Zhang%2C+Renjun+Zhang%2C+et+al%2E+Clinicallab%3A+Aligning+agents+for+multi%2Ddepartmental+clinical+diag)
69. Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, et al. [Baichuan 2: Open large-scale language models](https://md.research.elamur.ai/free/en/papers/yang-2023-baichuan-2-open-large-scale/). arXiv preprint arXiv:2309.10305, 2023.
70. Bowen Yang, Bharat Venkitesh, Dwarak Talupuru, Hangyu Lin, David Cairuz, Phil Blunsom, and Acyr Locatelli. Rope to nope and back again: A new hybrid attention strategy. arXiv preprint arXiv:2501.18795, 2025a. — [arXiv:2501.18795](https://arxiv.org/abs/2501.18795) [find](https://scholar.google.com/scholar?q=Bowen+Yang%2C+Bharat+Venkitesh%2C+Dwarak+Talupuru%2C+Hangyu+Lin%2C+David+Cairuz%2C+Phil+Blunsom%2C+and+Acyr+Locatelli%2E+Rope+to+nope+and+back+again%3A+A+new+hybrid+attention+strategy%2E+arXiv+preprint+arXiv%3A2501%2E18795)
71. Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, and Yoon Kim. Parallelizing linear transformers with the delta rule over sequence length. arXiv preprint arXiv:2406.06484, 2024. [find](https://scholar.google.com/scholar?q=Songlin+Yang%2C+Bailin+Wang%2C+Yu+Zhang%2C+Yikang+Shen%2C+and+Yoon+Kim%2E+Parallelizing+linear+transformers+with+the+delta+rule+over+sequence+length%2E+arXiv+preprint+arXiv%3A2406%2E06484%2C+2024%2E)
72. Songlin Yang, Jan Kautz, and Ali Hatamizadeh. Gated delta networks: Improving mamba2 with delta rule. In The Thirteenth International Conference on Learning Representations, 2025b. URL [https://openreview.net/forum?id=r8H7xhYPwz](https://openreview.net/forum?id=r8H7xhYPwz). [↗](https://openreview.net/forum?id=r8H7xhYPwz)
73. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+arXiv+preprint+arXiv%3A1905%2E07830%2C+2019%2E)
74. Yongcheng Zeng, Guoqing Liu, Weiyu Ma, Ning Yang, Haifeng Zhang, and Jun Wang. Token-level direct preference optimization. arXiv preprint arXiv:2404.11999, 2024. — [arXiv:2404.11999](https://arxiv.org/abs/2404.11999) [find](https://scholar.google.com/scholar?q=Yongcheng+Zeng%2C+Guoqing+Liu%2C+Weiyu+Ma%2C+Ning+Yang%2C+Haifeng+Zhang%2C+and+Jun+Wang%2E+Token%2Dlevel+direct+preference+optimization%2E+arXiv+preprint+arXiv%3A2404%2E11999%2C+2024%2E+%E2%80%94+%5BarXiv%3A2404%2E11999%5D%28https%3A%2F%2Farxiv%2Eorg)
75. Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Neural Information Processing Systems, 32, 2019. [find](https://scholar.google.com/scholar?q=Biao+Zhang+and+Rico+Sennrich%2E+Root+mean+square+layer+normalization%2E+Advances+in+Neural+Information+Processing+Systems%2C+32%2C+2019%2E)
76. Hongbo Zhang, Junying Chen, Feng Jiang, Fei Yu, Zhihong Chen, Jianquan Li, Guiming Chen, Xiangbo Wu, Zhiyi Zhang, Qingying Xiao, et al. [Huatuogpt, towards taming language model to be a doctor](https://md.research.elamur.ai/free/en/papers/zhang-2023-huatuogpt-taming-language-model-be/). arXiv preprint arXiv:2305.15075, 2023.
77. Xiaotian Zhang, Chunyang Li, Yi Zong, Zhengyu Ying, Liang He, and Xipeng Qiu. [Evaluating the performance of large language models on gaokao benchmark](https://md.research.elamur.ai/free/en/papers/zhang-2023-evaluating-performance-large-language/), 2024. URL https://arxiv.org/abs/2305.12474. [↗](https://arxiv.org/abs/2305.12474)
78. Xuanyu Zhang and Qing Yang. [Xuanyuan 2.0: A large chinese financial chat model with hundreds of billions parameters](https://md.research.elamur.ai/free/en/papers/zhang-2023-xuanyuan-2-0-large-chinese/). In Proceedings of the 32nd ACM international conference on information and knowledge management, pp. 4435–4439, 2023.
79. Zhengyun Zhao, Qiao Jin, Fangyuan Chen, Tuorui Peng, and Sheng Yu. A large-scale dataset of patient summaries for retrieval-based clinical decision support systems. Scientific data, 10(1):909, 2023. [find](https://scholar.google.com/scholar?q=Zhengyun+Zhao%2C+Qiao+Jin%2C+Fangyuan+Chen%2C+Tuorui+Peng%2C+and+Sheng+Yu%2E+A+large%2Dscale+dataset+of+patient+summaries+for+retrieval%2Dbased+clinical+decision+support+systems%2E+Scientific+data%2C+10%281%29%3A909%2C+2023%2E)
80. Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. Jec-qa: A legal-domain question answering dataset, 2019. URL https://arxiv.org/abs/ 1911.12011. [↗](https://arxiv.org/abs/)
81. Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, et al. [Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions](https://md.research.elamur.ai/free/en/papers/zhuo-2024-bigcodebench-benchmarking-code-generation-diverse/). arXiv preprint arXiv:2406.15877, 2024.

*Updated 1 November 2026*