---
title: "Is Biomedical Specialization Still Worth It? — Mannion 2026"
description: "The authors ask whether continued pre-training on domain text (DAPT) still helps modern general-purpose LLMs."
authors: ["Aidan Mannion", "Cécile Macaire", "Armand Violle", "Stéphane Ohayon", "Xavier Tannier", "Didier Schwab", "Lorraine Goeuriot", "François Portet"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.833/"
doi: "10.63317/5aujtfs5wq6b"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/mannion-2026-is-biomedical-specialization-still-worth/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/mannion-2026-is-biomedical-specialization-still-worth/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/3/) / Is Biomedical Specialization Still Worth It? Insights from Domain-Adaptive Language Modelling with a New French Health Corpus

# Is Biomedical Specialization Still Worth It? Insights from Domain-Adaptive Language Modelling with a New French Health Corpus

**Authors:** Aidan Mannion, Cécile Macaire, Armand Violle, Stéphane Ohayon, [Xavier Tannier](https://md.research.elamur.ai/free/authors/xavier-tannier/), [Didier Schwab](https://md.research.elamur.ai/free/authors/didier-schwab/), Lorraine Goeuriot, François Portet · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.833/) · [PDF](https://aclanthology.org/2026.lrec-1.833.pdf) · [DOI 10.63317/5aujtfs5wq6b](https://doi.org/10.63317/5aujtfs5wq6b) · [OpenAlex](https://openalex.org/W7160092800)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
The authors ask whether continued pre-training on domain text (DAPT) still helps modern general-purpose LLMs. They study this for French biomedical NLP, a setting with few openly licensed resources. Their first contribution is PARCOMED, a French health corpus assembled within the PARTAGES project. Every source's license was checked, and the corpus comes in two versions: one cleared for commercial use and one restricted to research. They then applied causal language modelling to Qwen3-Base models of four sizes and benchmarked checkpoints throughout training on medical and non-medical QA in English and French. Finally, each adapted checkpoint was combined with its original base model using SLERP. Specialization brought scattered gains on French medical tasks, mostly for the smaller models. However, the unmerged models lost ground in other areas, and averaged scores barely moved. Merging removed every statistically significant drop and kept most of the medical gains. The study suggests that broad DAPT offers limited value for strong base models. It is still worth considering for small models where compute is scarce, provided it is followed by merging.

## Key points
- **PARCOMED corpus:**
  - The full version contains 906,489 documents (about 900M words); the commercial-use version, used for training, has 892,343 documents (about 884M words).
  - Scientific text makes up roughly 94% of the words, and HAL alone contributes about 703M.
  - It is published on Hugging Face under HealthDataHub.
- **Models:** Qwen3-Base at 0.6B, 1.7B, 4B and 8B was adapted. The 8B model was chosen after it outperformed Llama, Mistral, Apollo-7B and BioMistral on French biomedical tasks. The adapted models are released as the Qwen3-PDAPT collection.
- **Evaluation:** Tasks were split into four groups, (EN/FR)-MEDICAL and (EN/FR)-OTHER, drawn from MMLU and MMLU-Pro-X. Statistical significance testing was applied.
- **Merging matters:** After SLERP merging, no model showed a significant decline in any task group. In some cases merging even improved results on the targeted French medical tasks.
- **Limitations and next steps:** The study covers only causal language modelling, without SFT, and only the Qwen3 family. The authors plan subtopic-specific pre-training and want to test other merging methods such as DARE and TIES. Code is available at github.com/PARTAGES-dev/partages-llm.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mannion2026is,
  title = {{Is Biomedical Specialization Still Worth It? Insights from Domain-Adaptive Language Modelling with a New French Health Corpus}},
  author = {Mannion, Aidan and Macaire, Cécile and Violle, Armand and Ohayon, Stéphane and Tannier, Xavier and Schwab, Didier and Goeuriot, Lorraine and Portet, François},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5aujtfs5wq6b},
  url = {https://doi.org/10.63317/5aujtfs5wq6b}
}
```

- [← previous Investigating Memorization in Language Models Trained via Knowledge Distillation](https://md.research.elamur.ai/free/papers/macking-2026-investigating-memorization-language-models-trained/)
- [next → Is Cross-Lingual Transfer in Bilingual Models Human-Like? A Study with Overlapping Word Forms in Dutch and English](https://md.research.elamur.ai/free/papers/skrjanec-2026-is-cross-lingual-transfer-bilingual/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/3/)

## References

57 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Project Apertus, Alejandro Hernández-Cano, Alexander Hägele, Allen Hao Huang, Angelika Romanou, Antoni-Joan Solergibert, Barna Pasztor, Bettina Messmer, Dhia Garbaya, Eduard Frank Ďurech, Ido Hakimi, Juan García Giraldo, Mete Ismayilzada, Negar Foroutan, Skander Moalla, Tiancheng Chen, and Vinko Sab… [find](https://scholar.google.com/scholar?q=Project+Apertus%2C+Alejandro+Hern%C3%A1ndez%2DCano%2C+Alexander+H%C3%A4gele%2C+Allen+Hao+Huang%2C+Angelika+Romanou%2C+Antoni%2DJoan+Solergibert%2C+Barna+Pasztor%2C+Bettina+Messmer%2C+Dhia+Garbaya%2C+Eduard+Frank+%C4%8Eurech%2C+Ido+Hakimi%2C+)
2. Elie Bakouch, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Lewis Tunstall, Carlos Miguel Patiño, Edward Beeching, Aymeric Roucher, Aksel Joonas Reedi, Quentin Gallouédec, Kashif Rasul, Nathan Habib, Clémentine Fourrier, Hynek Kydlicek, Guilherme Penedo, Hugo Larcher, Mathieu Morlon, Vaibhav Sriva… [find](https://scholar.google.com/scholar?q=Elie+Bakouch%2C+Loubna+Ben+Allal%2C+Anton+Lozhkov%2C+Nouamane+Tazi%2C+Lewis+Tunstall%2C+Carlos+Miguel+Pati%C3%B1o%2C+Edward+Beeching%2C+Aymeric+Roucher%2C+Aksel+Joonas+Reedi%2C+Quentin+Gallou%C3%A9dec%2C+Kashif+Rasul%2C+Nathan+Habib)
3. SmolLM3: smol, multilingual, long-context reasoner. [https://huggingface.co/blog/smollm3](https://huggingface.co/blog/smollm3). [↗](https://huggingface.co/blog/smollm3)
4. Ikram Belmadani, Benoit Favre, Richard Dufour, Frédéric Bechet, and Carlos Ramisch. 2025. Adaptation des connaissances médicales pour les grands modèles de langue : Stratégies et analyse comparative. In Actes de CORIA-TALN-RJCRI-RECITAL 2025. Actes des 32ème Conférence sur le Traitement Automatique… [find](https://scholar.google.com/scholar?q=Ikram+Belmadani%2C+Benoit+Favre%2C+Richard+Dufour%2C+Fr%C3%A9d%C3%A9ric+Bechet%2C+and+Carlos+Ramisch%2E+2025%2E+Adaptation+des+connaissances+m%C3%A9dicales+pour+les+grands+mod%C3%A8les+de+langue+%3A+Strat%C3%A9gies+et+analyse+comparative%2E+)
5. Elliot Bolton, Abhinav Venigalla, Michihiro Yasunaga, David Hall, Betty Xiong, Tony Lee, Roxana Daneshjou, Jonathan Frankle, Percy Liang, Michael Carbin, and Christopher D. Manning. 2024. Biomedlm: A 2.7b parameter language model trained on biomedical text. [find](https://scholar.google.com/scholar?q=Elliot+Bolton%2C+Abhinav+Venigalla%2C+Michihiro+Yasunaga%2C+David+Hall%2C+Betty+Xiong%2C+Tony+Lee%2C+Roxana+Daneshjou%2C+Jonathan+Frankle%2C+Percy+Liang%2C+Michael+Carbin%2C+and+Christopher+D%2E+Manning%2E+2024%2E+Biomedlm%3A+A+)
6. Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen A. Creel, Jared Davis, Dora Demsz… [find](https://scholar.google.com/scholar?q=Rishi+Bommasani%2C+Drew+A%2E+Hudson%2C+Ehsan+Adeli%2C+Russ+Altman%2C+Simran+Arora%2C+Sydney+von+Arx%2C+Michael+S%2E+Bernstein%2C+Jeannette+Bohg%2C+Antoine+Bosselut%2C+Emma+Brunskill%2C+Erik+Brynjolfsson%2C+S%2E+Buch%2C+Dallas+Card) [our card](https://md.research.elamur.ai/free/papers/bommasani-2021-opportunities-risks-foundation-models/)
7. Jie Chen, Zhipeng Chen, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Yingqian Min, Xin Zhao, Zhicheng Dou, Jiaxin Mao, Yankai Lin, Ruihua Song, Jun Xu, Xu Chen, Rui Yan, Zhewei Wei, Di Hu, Wenbing Huang, and Ji-Rong Wen. 2025. Towards effective and efficient continual pre-training of large langu… [find](https://scholar.google.com/scholar?q=Jie+Chen%2C+Zhipeng+Chen%2C+Jiapeng+Wang%2C+Kun+Zhou%2C+Yutao+Zhu%2C+Jinhao+Jiang%2C+Yingqian+Min%2C+Xin+Zhao%2C+Zhicheng+Dou%2C+Jiaxin+Mao%2C+Yankai+Lin%2C+Ruihua+Song%2C+Jun+Xu%2C+Xu+Chen%2C+Rui+Yan%2C+Zhewei+Wei%2C+Di+Hu%2C+Wenbing)
8. Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, Alexandre Sallinen, Alireza Sakhaeirad, Vinitra Swamy, Igor Krawczuk, Deniz Bayazit, Axel Marmet, Syrielle Montariol, Mary-Anne H… [find](https://scholar.google.com/scholar?q=Zeming+Chen%2C+Alejandro+Hern%C3%A1ndez+Cano%2C+Angelika+Romanou%2C+Antoine+Bonnet%2C+Kyle+Matoba%2C+Francesco+Salvi%2C+Matteo+Pagliardini%2C+Simin+Fan%2C+Andreas+K%C3%B6pf%2C+Amirkeivan+Mohtashami%2C+Alexandre+Sallinen%2C+Alireza+S) [our card](https://md.research.elamur.ai/free/papers/chen-2023-meditron-70b-scaling-medical-pretraining/)
9. Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. Arcee’s MergeKit: A toolkit for merging large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing:… [find](https://scholar.google.com/scholar?q=Charles+Goddard%2C+Shamane+Siriwardhana%2C+Malikeh+Ehghaghi%2C+Luke+Meyers%2C+Vladimir+Karpukhin%2C+Brian+Benedict%2C+Mark+McQuade%2C+and+Jacob+Solawetz%2E+2024%2E+Arcee%E2%80%99s+MergeKit%3A+A+toolkit+for+merging+large+language)
10. Nathan Godey, Wissam Antoun, Rian Touchent, Rachel Bawden, Éric de la Clergerie, Benoît Sagot, and Djamé Seddah. 2025. Gaperon: A peppered english-french generative language model suite. [find](https://scholar.google.com/scholar?q=Nathan+Godey%2C+Wissam+Antoun%2C+Rian+Touchent%2C+Rachel+Bawden%2C+%C3%89ric+de+la+Clergerie%2C+Beno%C3%AEt+Sagot%2C+and+Djam%C3%A9+Seddah%2E+2025%2E+Gaperon%3A+A+peppered+english%2Dfrench+generative+language+model+suite%2E)
11. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, and Arthur Hinsvark. 2024.… [find](https://scholar.google.com/scholar?q=Aaron+Grattafiori%2C+Abhimanyu+Dubey%2C+Abhinav+Jauhri%2C+Abhinav+Pandey%2C+Abhishek+Kadian%2C+Ahmad+Al%2DDahle%2C+Aiesha+Letman%2C+Akhil+Mathur%2C+Alan+Schelten%2C+Alex+Vaughan%2C+Amy+Yang%2C+Angela+Fan%2C+Anirudh+Goyal%2C+Anth)
12. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, Online. As… [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Ana+Marasovi%C4%87%2C+Swabha+Swayamdipta%2C+Kyle+Lo%2C+Iz+Beltagy%2C+Doug+Downey%2C+and+Noah+A%2E+Smith%2E+2020%2E+Don%E2%80%99t+Stop+Pretraining%3A+Adapt+Language+Models+to+Domains+and+Tasks%2E+In+Proceedings+of+t)
13. Tianyu Han, Lisa C. Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander Löser, Daniel Truhn, and Keno K. Bressem. 2023. MedAlpaca – An Open-Source Collection of Medical Conversational AI Models and Training Data. _eprint: 2304.08247. [find](https://scholar.google.com/scholar?q=Tianyu+Han%2C+Lisa+C%2E+Adams%2C+Jens%2DMichalis+Papaioannou%2C+Paul+Grundmann%2C+Tom+Oberhauser%2C+Alexander+L%C3%B6ser%2C+Daniel+Truhn%2C+and+Keno+K%2E+Bressem%2E+2023%2E+MedAlpaca+%E2%80%93+An+Open%2DSource+Collection+of+Medical+Convers) [our card](https://md.research.elamur.ai/free/papers/han-2023-medalpaca-open-source-collection-medical/)
14. Daniel P Jeong, Saurabh Garg, Zachary Chase Lipton, and Michael Oberst. 2024. Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12143–12170, Miami, Florida, USA. As… [find](https://scholar.google.com/scholar?q=Daniel+P+Jeong%2C+Saurabh+Garg%2C+Zachary+Chase+Lipton%2C+and+Michael+Oberst%2E+2024%2E+Medical+Adaptation+of+Large+Language+and+Vision%2DLanguage+Models%3A+Are+We+Making+Progress%3F+In+Proceedings+of+the+2024+Confer)
15. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
16. Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, and Gaël… [find](https://scholar.google.com/scholar?q=Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+Louis+Rouillard%2C+Thomas+Mesnard%2C+Geoffrey+Cideron%2C+Je)
17. Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024. BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains. _eprint: 2402.10373. [find](https://scholar.google.com/scholar?q=Yanis+Labrak%2C+Adrien+Bazoge%2C+Emmanuel+Morin%2C+Pierre%2DAntoine+Gourraud%2C+Mickael+Rouvier%2C+and+Richard+Dufour%2E+2024%2E+BioMistral%3A+A+Collection+of+Open%2DSource+Pretrained+Large+Language+Models+for+Medical+Do) [our card](https://md.research.elamur.ai/free/papers/labrak-2024-biomistral-collection-open-source-pretrained/)
18. Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoit Crabbe, Laurent Besacier, and Didier Schwab. 2020. Flaubert: Unsupervised language model pre-training for french. In LREC. [find](https://scholar.google.com/scholar?q=Hang+Le%2C+Lo%C3%AFc+Vial%2C+Jibril+Frej%2C+Vincent+Segonne%2C+Maximin+Coavoux%2C+Benjamin+Lecouteux%2C+Alexandre+Allauzen%2C+Benoit+Crabbe%2C+Laurent+Besacier%2C+and+Didier+Schwab%2E+2020%2E+Flaubert%3A+Unsupervised+language+mod) [our card](https://md.research.elamur.ai/free/papers/le-2019-flaubert-unsupervised-language-model-pre/)
19. J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. Ho So, and J. Kang. 2020. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. [find](https://scholar.google.com/scholar?q=J%2E+Lee%2C+W%2E+Yoon%2C+S%2E+Kim%2C+D%2E+Kim%2C+S%2E+Kim%2C+C%2E+Ho+So%2C+and+J%2E+Kang%2E+2020%2E+BioBERT%3A+a+pre%2Dtrained+biomedical+language+representation+model+for+biomedical+text+mining%2E) [our card](https://md.research.elamur.ai/free/papers/lee-2019-biobert-pre-trained-biomedical-language/)
20. Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022. BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining. Briefings in Bioinformatics, 23(6). [find](https://scholar.google.com/scholar?q=Renqian+Luo%2C+Liai+Sun%2C+Yingce+Xia%2C+Tao+Qin%2C+Sheng+Zhang%2C+Hoifung+Poon%2C+and+Tie%2DYan+Liu%2E+2022%2E+BioGPT%3A+Generative+Pre%2Dtrained+Transformer+for+Biomedical+Text+Generation+and+Mining%2E+Briefings+in+Bioinfo)
21. Pedro Henrique Martins, João Alves, Patrick Fernandes, Nuno M. Guerreiro, Ricardo Rei, Amin Farajian, Mateusz Klimaszewski, Duarte M. Alves, José Pombal, Nicolas Boizard, Manuel Faysse, Pierre Colombo, François Yvon, Barry Haddow, José G. C. de Souza, Alexandra Birch, and André F. T. Martins. 2025.… [find](https://scholar.google.com/scholar?q=Pedro+Henrique+Martins%2C+Jo%C3%A3o+Alves%2C+Patrick+Fernandes%2C+Nuno+M%2E+Guerreiro%2C+Ricardo+Rei%2C+Amin+Farajian%2C+Mateusz+Klimaszewski%2C+Duarte+M%2E+Alves%2C+Jos%C3%A9+Pombal%2C+Nicolas+Boizard%2C+Manuel+Faysse%2C+Pierre+Colombo)
22. OpenAI. 2025. gpt-oss-120b & gpt-oss-20b model card. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025%2E+gpt%2Doss%2D120b+%26+gpt%2Doss%2D20b+model+card%2E) [our card](https://md.research.elamur.ai/free/papers/agarwal-2025-gpt-oss-120b-gpt-oss/)
23. Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, Justin Chen, Fereshteh Mahvar, Liron Yatziv, Tiffany Chen, Bram Sterling, Stefanie Anna Baby, Susanna Maria Baby, Jeremy Lai, and Samuel Schmidgal… [find](https://scholar.google.com/scholar?q=Andrew+Sellergren%2C+Sahar+Kazemzadeh%2C+Tiam+Jaroensri%2C+Atilla+Kiraly%2C+Madeleine+Traverse%2C+Timo+Kohlberger%2C+Shawn+Xu%2C+Fayaz+Jamil%2C+C%C3%ADan+Hughes%2C+Charles+Lau%2C+Justin+Chen%2C+Fereshteh+Mahvar%2C+Liron+Yatziv%2C+T)
24. Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Abubakr Babiker, Nathanael Schärli, Aakanksha Chowdhery, Philip Mansfield, Dina Demner-Fushm… [find](https://scholar.google.com/scholar?q=Karan+Singhal%2C+Shekoofeh+Azizi%2C+Tao+Tu%2C+S%2E+Sara+Mahdavi%2C+Jason+Wei%2C+Hyung+Won+Chung%2C+Nathan+Scales%2C+Ajay+Tanwani%2C+Heather+Cole%2DLewis%2C+Stephen+Pfohl%2C+Perry+Payne%2C+Martin+Seneviratne%2C+Paul+Gamble%2C+Chris) [our card](https://md.research.elamur.ai/free/papers/singhal-2022-large-language-models-encode-clinical/)
25. Kentaro Ueda, François Portet, Hirohiko Suwa, and Keiichi Yasumoto. 2026. Merging continual pre-training models for domain-specialized llms: A case study in finance. In Proceedings of LREC 2026. [find](https://scholar.google.com/scholar?q=Kentaro+Ueda%2C+Fran%C3%A7ois+Portet%2C+Hirohiko+Suwa%2C+and+Keiichi+Yasumoto%2E+2026%2E+Merging+continual+pre%2Dtraining+models+for+domain%2Dspecialized+llms%3A+A+case+study+in+finance%2E+In+Proceedings+of+LREC+2026%2E)
26. Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. 2023. Ties-merging: Resolving interference when merging models. In Advances in Neural Information Processing Systems, volume 36, pages 7093–7115. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Prateek+Yadav%2C+Derek+Tam%2C+Leshem+Choshen%2C+Colin+A+Raffel%2C+and+Mohit+Bansal%2E+2023%2E+Ties%2Dmerging%3A+Resolving+interference+when+merging+models%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume) [our card](https://md.research.elamur.ai/free/papers/yadav-2023-ties-merging-resolving-interference-merging/)
27. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jin… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Anfeng+Li%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Gao%2C+Chengen+Huang%2C+Chenxu+Lv%2C+Chujie+Zheng%2C+Dayiheng+Liu%2C+Fan+Zhou%2C+Fei+Huang%2C+Feng+Hu%2C+Hao+Ge%2C+Haoran+Wei%2C+Hua) [our card](https://md.research.elamur.ai/free/papers/yang-2025-qwen3-technical-report/)
28. Guorui Zheng, Xidong Wang, Juhao Liang, Nuo Chen, Yuping Zheng, and Benyou Wang. 2024. Efficiently democratizing medical llms for 50 languages via a mixture of language family experts. 10. [find](https://scholar.google.com/scholar?q=Guorui+Zheng%2C+Xidong+Wang%2C+Juhao+Liang%2C+Nuo+Chen%2C+Yuping+Zheng%2C+and+Benyou+Wang%2E+2024%2E+Efficiently+democratizing+medical+llms+for+50+languages+via+a+mixture+of+language+family+experts%2E+10%2E)
29. Language Resource References BDPM. 2013. Base de Données Publique des Médicaments (BDPM). https://base-don nees-publique.medicaments.gouv.fr/. [find](https://scholar.google.com/scholar?q=Language+Resource+References+BDPM%2E+2013%2E+Base+de+Donn%C3%A9es+Publique+des+M%C3%A9dicaments+%28BDPM%29%2E+https%3A%2F%2Fbase%2Ddon+nees%2Dpublique%2Emedicaments%2Egouv%2Efr%2F%2E)
30. Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, et al. 2016. Findings of the 2016 conference on machine translation (wmt16). In First conference on machine translation, pages 131–1… [find](https://scholar.google.com/scholar?q=Ondrej+Bojar%2C+Rajen+Chatterjee%2C+Christian+Federmann%2C+Yvette+Graham%2C+Barry+Haddow%2C+Matthias+Huck%2C+Antonio+Jimeno+Yepes%2C+Philipp+Koehn%2C+Varvara+Logacheva%2C+Christof+Monz%2C+et+al%2E+2016%2E+Findings+of+the+201) [our card](https://md.research.elamur.ai/free/papers/bojar-2016-findings-2016-conference-machine-translation/)
31. CERIMES. 2003. Centre de ressources et d’information sur les multimédias pour l’enseignement supérieur [http://www.cerimes.fr](http://www.cerimes.fr). [↗](http://www.cerimes.fr)
32. CNRS. 2001. HAL (archive ouverte). [https://hal.science](https://hal.science). [↗](https://hal.science)
33. CNRS. 2012. Infrastructure de services pour la fouille de texte (ISTEX). [https://www.iste](https://www.iste) x.fr. [↗](https://www.iste)
34. Clément Dalloux, Vincent Claveau, Natalia Grabar, Lucas Emanuel Silva Oliveira, Claudia Maria Cabral Moro, Yohan Bonescki Gumiel, and Deborah Ribeiro Carvalho. 2021. Supervised learning for the detection of negation and of its scope in french and brazilian portuguese biomedical corpora. Natural Lang… [find](https://scholar.google.com/scholar?q=Cl%C3%A9ment+Dalloux%2C+Vincent+Claveau%2C+Natalia+Grabar%2C+Lucas+Emanuel+Silva+Oliveira%2C+Claudia+Maria+Cabral+Moro%2C+Yohan+Bonescki+Gumiel%2C+and+Deborah+Ribeiro+Carvalho%2E+2021%2E+Supervised+learning+for+the+detect)
35. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Ani… [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci)
36. Natalia Grabar and Rémi Cardon. 2018. CLEAR-Simple Corpus for Medical French. In ATA, Tilburg, Netherlands. [find](https://scholar.google.com/scholar?q=Natalia+Grabar+and+R%C3%A9mi+Cardon%2E+2018%2E+CLEAR%2DSimple+Corpus+for+Medical+French%2E+In+ATA%2C+Tilburg%2C+Netherlands%2E)
37. Natalia Grabar, Vincent Claveau, and Clément Dalloux. 2018. Cas: French corpus with clinical cases. In LOUHI 2018-The Ninth International Workshop on Health Text Mining and Information Analysis, pages 1–7. [find](https://scholar.google.com/scholar?q=Natalia+Grabar%2C+Vincent+Claveau%2C+and+Cl%C3%A9ment+Dalloux%2E+2018%2E+Cas%3A+French+corpus+with+clinical+cases%2E+In+LOUHI+2018%2DThe+Ninth+International+Workshop+on+Health+Text+Mining+and+Information+Analysis%2C+pages)
38. Cyril Grouin, Natalia Grabar, and Gabriel Illouz. 2021. Classification de cas cliniques et évaluation automatique de réponses d’étudiants: présentation de la campagne deft 2021 (clinical cases classification and automatic evaluation of student answers: Presentation of the deft 2021 challenge). In Ac… [find](https://scholar.google.com/scholar?q=Cyril+Grouin%2C+Natalia+Grabar%2C+and+Gabriel+Illouz%2E+2021%2E+Classification+de+cas+cliniques+et+%C3%A9valuation+automatique+de+r%C3%A9ponses+d%E2%80%99%C3%A9tudiants%3A+pr%C3%A9sentation+de+la+campagne+deft+2021+%28clinical+cases+classif)
39. Haute Autorité de Santé (HAS). 2021a. Base sur la qualité et la sécurité des soins (QualiScope). [https://www.data.gouv.fr/datasets/base-sur-la-qualite-et-la-securit](https://www.data.gouv.fr/datasets/base-sur-la-qualite-et-la-securit) e-des-soins-anciennement-scope-san te/informations. [↗](https://www.data.gouv.fr/datasets/base-sur-la-qualite-et-la-securit)
40. Haute Autorité de Santé (HAS). 2021b. Commission nationale d’évaluation des dispositifs médicaux et des technologies de santé (CNEDiMTS). [https://www.data.gouv.fr/datasets/evaluation-des-dispositifs-medicau](https://www.data.gouv.fr/datasets/evaluation-des-dispositifs-medicau) x. [↗](https://www.data.gouv.fr/datasets/evaluation-des-dispositifs-medicau)
41. Haute Autorité de Santé (HAS). 2021c. Haute Autorité de Santé. [https://www.has-sante.f](https://www.has-sante.f) r. [↗](https://www.has-sante.f)
42. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Collin+Burns%2C+Steven+Basart%2C+Andy+Zou%2C+Mantas+Mazeika%2C+Dawn+Song%2C+and+Jacob+Steinhardt%2E+2021%2E+Measuring+massive+multitask+language+understanding%2E+Proceedings+of+the+International+Confer) [our card](https://md.research.elamur.ai/free/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/)
43. Julien Knafou, Luc Mottin, Anaïs Mottaz, Alexandre Flament, and Patrick Ruch. 2025. Trans-BERT: A Framework for Synthetic Translation in Domain-Specific Language Modeling. In The 2025 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Julien+Knafou%2C+Luc+Mottin%2C+Ana%C3%AFs+Mottaz%2C+Alexandre+Flament%2C+and+Patrick+Ruch%2E+2025%2E+Trans%2DBERT%3A+A+Framework+for+Synthetic+Translation+in+Domain%2DSpecific+Language+Modeling%2E+In+The+2025+Conference+on+Em)
44. Ali Can Kocabiyikoglu, François Portet, Prudence Gibert, Hervé Blanchon, Jean-Marc Babouchkine, and Gaëtan Gavazzi. 2022. A spoken drug prescription dataset in french for spoken language understanding. In LREC 2022. [find](https://scholar.google.com/scholar?q=Ali+Can+Kocabiyikoglu%2C+Fran%C3%A7ois+Portet%2C+Prudence+Gibert%2C+Herv%C3%A9+Blanchon%2C+Jean%2DMarc+Babouchkine%2C+and+Ga%C3%ABtan+Gavazzi%2E+2022%2E+A+spoken+drug+prescription+dataset+in+french+for+spoken+language+understanding)
45. Jan A Kors, Simon Clematide, Saber A Akhondi, Erik M Van Mulligen, and Dietrich Rebholz-Schuhmann. 2015. A multilingual gold-standard corpus for biomedical concept recognition: the mantra gsc. Journal of the American Medical Informatics Association, 22(5):948–956. [find](https://scholar.google.com/scholar?q=Jan+A+Kors%2C+Simon+Clematide%2C+Saber+A+Akhondi%2C+Erik+M+Van+Mulligen%2C+and+Dietrich+Rebholz%2DSchuhmann%2E+2015%2E+A+multilingual+gold%2Dstandard+corpus+for+biomedical+concept+recognition%3A+the+mantra+gsc%2E+Journal)
46. Yanis Labrak, Adrien Bazoge, Richard Dufour, Béatrice Daille, Pierre-Antoine Gourraud, Emmanuel Morin, and Mickaël Rouvier. 2022. Frenchmedmcqa: A french multiple-choice question answering dataset for medical domain. In Proceedings of the 13th International Workshop on Health Text Mining and Informa… [find](https://scholar.google.com/scholar?q=Yanis+Labrak%2C+Adrien+Bazoge%2C+Richard+Dufour%2C+B%C3%A9atrice+Daille%2C+Pierre%2DAntoine+Gourraud%2C+Emmanuel+Morin%2C+and+Micka%C3%ABl+Rouvier%2E+2022%2E+Frenchmedmcqa%3A+A+french+multiple%2Dchoice+question+answering+dataset+for)
47. Yanis Labrak, Adrien Bazoge, Richard Dufour, Mickael Rouvier, Emmanuel Morin, Béatrice Daille, and Pierre-Antoine Gourraud. 2023. DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domains. In Proceedings of the 61th Annual Meeting of the Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Yanis+Labrak%2C+Adrien+Bazoge%2C+Richard+Dufour%2C+Mickael+Rouvier%2C+Emmanuel+Morin%2C+B%C3%A9atrice+Daille%2C+and+Pierre%2DAntoine+Gourraud%2E+2023%2E+DrBERT%3A+A+Robust+Pre%2Dtrained+Model+in+French+for+Biomedical+and+Clinic)
48. Fréjus AA Laleye, Gaël de Chalendar, Antonia Blanié, Antoine Brouquet, and Dan Behnamou. 2020. A french medical conversations corpus annotated for a virtual patient dialogue system. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 574–580. [find](https://scholar.google.com/scholar?q=Fr%C3%A9jus+AA+Laleye%2C+Ga%C3%ABl+de+Chalendar%2C+Antonia+Blani%C3%A9%2C+Antoine+Brouquet%2C+and+Dan+Behnamou%2E+2020%2E+A+french+medical+conversations+corpus+annotated+for+a+virtual+patient+dialogue+system%2E+In+Proceedings+of+)
49. Anne-Lyse Minard, Roberto Zanoli, Begoña Altuna, Manuela Speranza, Bernardo Magnini, and Alberto Lavelli. 2021. European Clinical Case Corpus (E3C-Corpus). Dataset. Version 2.0.0. Licensed under CC BY-NC 4.0. [find](https://scholar.google.com/scholar?q=Anne%2DLyse+Minard%2C+Roberto+Zanoli%2C+Bego%C3%B1a+Altuna%2C+Manuela+Speranza%2C+Bernardo+Magnini%2C+and+Alberto+Lavelli%2E+2021%2E+European+Clinical+Case+Corpus+%28E3C%2DCorpus%29%2E+Dataset%2E+Version+2%2E0%2E0%2E+Licensed+under+CC+BY)
50. Aurélie Névéol, Cyril Grouin, Jeremy Leixa, Sophie Rosset, and Pierre Zweigenbaum. 2014. The quaero french medical corpus: A ressource for medical entity recognition and normalization. Proc of BioTextMining Work, pages 24–30. [find](https://scholar.google.com/scholar?q=Aur%C3%A9lie+N%C3%A9v%C3%A9ol%2C+Cyril+Grouin%2C+Jeremy+Leixa%2C+Sophie+Rosset%2C+and+Pierre+Zweigenbaum%2E+2014%2E+The+quaero+french+medical+corpus%3A+A+ressource+for+medical+entity+recognition+and+normalization%2E+Proc+of+BioText)
51. Mariana Neves, Antonio Jimeno Yepes, Aurélie Névéol, Cristian Grozea, Amy Siu, Madeleine Kittner, and Karin Verspoor. 2018. Findings of the WMT 2018 biomedical translation shared task: Evaluation on Medline test sets. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers,… [find](https://scholar.google.com/scholar?q=Mariana+Neves%2C+Antonio+Jimeno+Yepes%2C+Aur%C3%A9lie+N%C3%A9v%C3%A9ol%2C+Cristian+Grozea%2C+Amy+Siu%2C+Madeleine+Kittner%2C+and+Karin+Verspoor%2E+2018%2E+Findings+of+the+WMT+2018+biomedical+translation+shared+task%3A+Evaluation+on+M)
52. Vincent Segonne, Aidan Mannion, Laura Cristina Alonzo Canul, Alexandre Audibert, Xingyu Liu, Cécile Macaire, Adrien Pupier, Yongxin Zhou, Mathilde Aguiar, Felix Herron, Magali Norré, Massih-Reza Amini, Pierrette Bouillon, Iris Eshkol-Taravella, Emmanuelle Esperança-Rodier, Thomas François, Lorraine… [find](https://scholar.google.com/scholar?q=Vincent+Segonne%2C+Aidan+Mannion%2C+Laura+Cristina+Alonzo+Canul%2C+Alexandre+Audibert%2C+Xingyu+Liu%2C+C%C3%A9cile+Macaire%2C+Adrien+Pupier%2C+Yongxin+Zhou%2C+Mathilde+Aguiar%2C+Felix+Herron%2C+Magali+Norr%C3%A9%2C+Massih%2DReza+Amini)
53. Ralf Steinberger, Mohamed Ebrahim, Alexandros Poulis, Manuel Carrasco-Benitez, Patrick Schlüter, Marek Przybyszewski, and Signe Gilbro. 2014. An overview of the european union’s highly multilingual parallel corpora. Language resources and evaluation, 48(4):679–707. [find](https://scholar.google.com/scholar?q=Ralf+Steinberger%2C+Mohamed+Ebrahim%2C+Alexandros+Poulis%2C+Manuel+Carrasco%2DBenitez%2C+Patrick+Schl%C3%BCter%2C+Marek+Przybyszewski%2C+and+Signe+Gilbro%2E+2014%2E+An+overview+of+the+european+union%E2%80%99s+highly+multilingual+pa)
54. Jorg Tiedemann. 2012. Parallel data, tools and interfaces in opus. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 2214–2218, Istanbul, Turkey. [find](https://scholar.google.com/scholar?q=Jorg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+opus%2E+In+Proceedings+of+the+Eighth+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9912%29%2C+pages+2214%E2%80%932218%2C+Istanbul%2C+Turk)
55. Wikipedia Contributors. 2025. Wikipédia, l’encyclopédie libre – Médecine, Pharmacie, Biologie. [https://fr.wikipedia.org](https://fr.wikipedia.org). [↗](https://fr.wikipedia.org)
56. Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, et al. 2025. Mmlu-prox: A multilingual benchmark for advanced large language model evaluation. arXiv preprint arXiv:2503.10497. [find](https://scholar.google.com/scholar?q=Weihao+Xuan%2C+Rui+Yang%2C+Heli+Qi%2C+Qingcheng+Zeng%2C+Yunze+Xiao%2C+Aosong+Feng%2C+Dairui+Liu%2C+Yun+Xing%2C+Junjue+Wang%2C+Fan+Gao%2C+et+al%2E+2025%2E+Mmlu%2Dprox%3A+A+multilingual+benchmark+for+advanced+large+language+model+)
57. Jamil Zaghir, Mina Bjelogrlic, Jean-Philippe Goldman, Soukaïna Aananou, Christophe Gaudet-Blavignac, and Christian Lovis. 2024. Frasimed: A clinical french annotated resource produced through crosslingual bert-based annotation projection. In Proceedings of the 2024 Joint International Conference on… [find](https://scholar.google.com/scholar?q=Jamil+Zaghir%2C+Mina+Bjelogrlic%2C+Jean%2DPhilippe+Goldman%2C+Souka%C3%AFna+Aananou%2C+Christophe+Gaudet%2DBlavignac%2C+and+Christian+Lovis%2E+2024%2E+Frasimed%3A+A+clinical+french+annotated+resource+produced+through+crosslin)