---
title: "Effects of Adaptive Pretraining in Specialized Domains for Named"
description: "The authors offer three recommendations: If adapted models exist, choose the one closest to your domain. 38 references."
authors: ["Jack Lynam", "Sam Henry"]
year: 2026
venue: "BioNLP 2026"
source: "https://aclanthology.org/2026.bionlp-1.18/"
doi: "10.18653/v1/2026.bionlp-1.18"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lynam-2026-effects-adaptive-pretraining-specialized-domains/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lynam-2026-effects-adaptive-pretraining-specialized-domains/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) / Effects of Adaptive Pretraining in Specialized Domains for Named Entity Recognition

# Effects of Adaptive Pretraining in Specialized Domains for Named Entity Recognition

**Authors:** Jack Lynam and [Sam Henry](https://md.research.elamur.ai/free/en/authors/sam-henry/) · **Year:** 2026 · **Venue:** [BioNLP 2026](https://md.research.elamur.ai/free/en/venues/bionlp/) ([BioNLP 2026](https://md.research.elamur.ai/free/en/venues/bionlp/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.bionlp-1.18/) · [PDF](https://aclanthology.org/2026.bionlp-1.18.pdf) · [DOI 10.18653/v1/2026.bionlp-1.18](https://doi.org/10.18653/v1/2026.bionlp-1.18) · [OpenAlex](https://openalex.org/W7166879436)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper tackles a practical question for NER developers in specialized fields: does extra domain pretraining of a language model pay off, and which released model should you pick? Lynam and Henry benchmark nine BERT-family encoders on nine NER datasets from three biomedical sub-domains: clinical, scientific and health-related social media. The models include general BERT-Base and its domain-adapted variants (BioBERT, BioClinicalBERT, BioDischargeSummaryBERT, BlueBERT with and without MIMIC, BERTweet, BioRedditBERT). They also test PubMedBERT, which was trained from scratch on PubMed abstracts. Differences are checked with Friedman, Wilcoxon and cross-validation t-tests. Adaptive pretraining helps, but usually only a little and often without statistical significance. Even the from-scratch PubMedBERT gains just slightly. The only clear outlier is BERTweet, which performs poorly. The authors offer three recommendations:
- If adapted models exist, choose the one closest to your domain.
- In a new domain, prototype with plain BERT.
- Spend budget on annotation or new methods rather than costly pretraining from scratch.

## Key points
- Datasets span the three sub-domains, among them i2b2 2010, n2c2 2018, BC5CDR, NCBI, NLMChem, BC7MED, COMETA and ADEMiner. Pretraining sources include PubMed, PubMed Central, MIMIC-III, Twitter and Reddit.
- Across all datasets, Friedman's test finds a significant difference among models (p = 0.0017). Once BERTweet is excluded, the difference disappears (p = 0.4061).
- Adapted models beat BERT-Base by under 2% on 7 of the 9 datasets. The larger gains are 3.37% on NLMChem and 8.88% on BC7MED.
- Some cross-domain winners appeared, such as PubMedBERT topping the clinical n2c2 2018 set. These were not significantly better than the best in-domain model (e.g., p = 0.0686 against BioDischargeSummaryBERT), so the "pick the closest domain" heuristic holds.
- BioRedditBERT significantly outperforms BERTweet on every social media dataset (p ≤ 0.0073). The authors note limitations: significance tests rely on few samples, and they did no per-model hyperparameter tuning.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

Also on the byline: [Randolph–Macon College](https://ror.org/03zstcc67).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · 200 works

## Cite

Cite as: Jack Lynam and Sam Henry. 2026. Effects of Adaptive Pretraining in Specialized Domains for Named Entity Recognition. BioNLP 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lynam2026effects,
  title = {{Effects of Adaptive Pretraining in Specialized Domains for Named Entity Recognition}},
  author = {Jack Lynam and Sam Henry},
  booktitle = {BioNLP 2026},
  year = {2026},
  doi = {10.18653/v1/2026.bionlp-1.18},
  url = {https://doi.org/10.18653/v1/2026.bionlp-1.18}
}
```

- [← previous Effect of Data Augmentation with Multi-View Perspectives of Signers on the DGS-Fabeln-1 Dataset](https://md.research.elamur.ai/free/en/papers/renner-2026-effect-data-augmentation-multi-view/)
- [next → Efficient Agent Evaluation via Diversity-Guided User Simulation](https://md.research.elamur.ai/free/en/papers/nakash-2026-efficient-agent-evaluation-diversity-guided/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/)

## References

38 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Emily Alsentzer, John R Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew McDermott. 2019. [Publicly available clinical bert embeddings](https://md.research.elamur.ai/free/en/papers/alsentzer-2019-publicly-available-clinical-bert-embeddings/). arXiv preprint arXiv:1904.03323.
2. Rezarta Islamaj, Robert Leaman, Sun Kim, Dongseop Kwon, Chih-Hsuan Wei, Donald C Comeau, Yifan Peng, David Cissel, Cathleen Coss, Carol Fisher, and 1 others. 2021. NLM-Chem, a new resource for chemical entity recognition in PubMed full text literature. Scientific Data, 8(1):1–12. [find](https://scholar.google.com/scholar?q=Rezarta+Islamaj%2C+Robert+Leaman%2C+Sun+Kim%2C+Dongseop+Kwon%2C+Chih%2DHsuan+Wei%2C+Donald+C+Comeau%2C+Yifan+Peng%2C+David+Cissel%2C+Cathleen+Coss%2C+Carol+Fisher%2C+and+1+others%2E+2021%2E+NLM%2DChem%2C+a+new+resource+for+chemica)
3. Marco Basaldella, Fangyu Liu, Ehsan Shareghi, and Nigel Collier. 2020. COMETA: A corpus for medical entity linking in the social media. arXiv preprint arXiv:2010.03295. [find](https://scholar.google.com/scholar?q=Marco+Basaldella%2C+Fangyu+Liu%2C+Ehsan+Shareghi%2C+and+Nigel+Collier%2E+2020%2E+COMETA%3A+A+corpus+for+medical+entity+linking+in+the+social+media%2E+arXiv+preprint+arXiv%3A2010%2E03295%2E)
4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
5. Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. [MIMIC-III, a freely accessible critical care database](https://md.research.elamur.ai/free/en/papers/johnson-2016-mimic-iii-freely-accessible-critical/). Scientific data, 3(1):1–9.
6. Gamal Crichton, Sampo Pyysalo, Billy Chiu, and Anna Korhonen. 2017. A neural network multi-task learning approach to biomedical named entity recognition. BMC bioinformatics, 18(1):1–14. [find](https://scholar.google.com/scholar?q=Gamal+Crichton%2C+Sampo+Pyysalo%2C+Billy+Chiu%2C+and+Anna+Korhonen%2E+2017%2E+A+neural+network+multi%2Dtask+learning+approach+to+biomedical+named+entity+recognition%2E+BMC+bioinformatics%2C+18%281%29%3A1%E2%80%9314%2E)
7. Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine learning research, 7(Jan):1–30. [find](https://scholar.google.com/scholar?q=Janez+Dem%C5%A1ar%2E+2006%2E+Statistical+comparisons+of+classifiers+over+multiple+data+sets%2E+Journal+of+Machine+learning+research%2C+7%28Jan%29%3A1%E2%80%9330%2E)
8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805.
9. Rezarta Islamaj Dogan and Zhiyong Lu. 2012. An improved corpus of disease mentions in PubMed citations. In BioNLP: Proceedings of the 2012 Workshop on Biomedical Natural Language Processing, pages 91–99. [find](https://scholar.google.com/scholar?q=Rezarta+Islamaj+Dogan+and+Zhiyong+Lu%2E+2012%2E+An+improved+corpus+of+disease+mentions+in+PubMed+citations%2E+In+BioNLP%3A+Proceedings+of+the+2012+Workshop+on+Biomedical+Natural+Language+Processing%2C+pages+91%E2%80%93)
10. Kevin Donnelly and 1 others. 2006. SNOMED-CT: The advanced terminology and coding system for eHealth. Studies in health technology and informatics, 121:279. [find](https://scholar.google.com/scholar?q=Kevin+Donnelly+and+1+others%2E+2006%2E+SNOMED%2DCT%3A+The+advanced+terminology+and+coding+system+for+eHealth%2E+Studies+in+health+technology+and+informatics%2C+121%3A279%2E)
11. Graciela Gonzalez, Ari Z Klein, Ivan Flores, Davy Weissenbacher, Arjun Magge, Karen O’Connor, Abeed Sarker, Anne-Lyse Minard, Elena Tutubalina, Zulfat Miftahutdinov, and 1 others. 2020. Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task. In Proceedings of the… [find](https://scholar.google.com/scholar?q=Graciela+Gonzalez%2C+Ari+Z+Klein%2C+Ivan+Flores%2C+Davy+Weissenbacher%2C+Arjun+Magge%2C+Karen+O%E2%80%99Connor%2C+Abeed+Sarker%2C+Anne%2DLyse+Minard%2C+Elena+Tutubalina%2C+Zulfat+Miftahutdinov%2C+and+1+others%2E+2020%2E+Proceedings+of)
12. Yves Grandvalet and Yoshua Bengio. 2006. Hypothesis testing for cross-validation. Montreal Universite de Montreal, Operationnelle DdIeR, 1285. [find](https://scholar.google.com/scholar?q=Yves+Grandvalet+and+Yoshua+Bengio%2E+2006%2E+Hypothesis+testing+for+cross%2Dvalidation%2E+Montreal+Universite+de+Montreal%2C+Operationnelle+DdIeR%2C+1285%2E)
13. Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. [Domain-specific language model pretraining for biomedical natural language processing](https://md.research.elamur.ai/free/en/papers/gu-2020-domain-specific-language-model-pretraining/). ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1–23.
14. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). arXiv preprint arXiv:2006.03654.
15. Sam Henry, Kevin Buchan, Michele Filannino, Amber Stubbs, and Ozlem Uzuner. 2020. 2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records. Journal of the American Medical Informatics Association, 27(1):3–12. [find](https://scholar.google.com/scholar?q=Sam+Henry%2C+Kevin+Buchan%2C+Michele+Filannino%2C+Amber+Stubbs%2C+and+Ozlem+Uzuner%2E+2020%2E+2018+n2c2+shared+task+on+adverse+drug+events+and+medication+extraction+in+electronic+health+records%2E+Journal+of+the+Am)
16. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://md.research.elamur.ai/free/en/papers/lee-2019-biobert-pre-trained-biomedical-language/). Bioinformatics, 36(4):1234–1240.
17. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692.
18. Arjun Magge, Elena Tutubalina, Zulfat Miftahutdinov, Ilseyar Alimova, Anne Dirkson, Suzan Verberne, Davy Weissenbacher, and Graciela Gonzalez-Hernandez. 2021. DeepADEMiner: a deep learning pharmacovigilance pipeline for extraction and normalization of adverse drug event mentions on Twitter. Journal… [find](https://scholar.google.com/scholar?q=Arjun+Magge%2C+Elena+Tutubalina%2C+Zulfat+Miftahutdinov%2C+Ilseyar+Alimova%2C+Anne+Dirkson%2C+Suzan+Verberne%2C+Davy+Weissenbacher%2C+and+Graciela+Gonzalez%2DHernandez%2E+2021%2E+DeepADEMiner%3A+a+deep+learning+pharmacovig)
19. George Michalopoulos, Yuanxin Wang, Hussam Kaka, Helen Chen, and Alexander Wong. 2020. Umlsbert: Clinical domain knowledge augmentation of contextual embeddings using the unified medical language system metathesaurus. arXiv preprint arXiv:2010.10391. [find](https://scholar.google.com/scholar?q=George+Michalopoulos%2C+Yuanxin+Wang%2C+Hussam+Kaka%2C+Helen+Chen%2C+and+Alexander+Wong%2E+2020%2E+Umlsbert%3A+Clinical+domain+knowledge+augmentation+of+contextual+embeddings+using+the+unified+medical+language+syst)
20. Antonio Miranda, Farrokh Mehryary, Jouni Luoma, Sampo Pyysalo, Alfonso Valencia, and Martin Krallinger. 2021. Overview of DrugProt BioCreative VII track: quality evaluation and large scale text mining of drug-gene/protein relations. In Proceedings of the seventh BioCreative challenge evaluation work… [find](https://scholar.google.com/scholar?q=Antonio+Miranda%2C+Farrokh+Mehryary%2C+Jouni+Luoma%2C+Sampo+Pyysalo%2C+Alfonso+Valencia%2C+and+Martin+Krallinger%2E+2021%2E+Overview+of+DrugProt+BioCreative+VII+track%3A+quality+evaluation+and+large+scale+text+mining)
21. Claude Nadeau and Yoshua Bengio. 2003. Inference for the generalization error. Machine Learning, 52. [find](https://scholar.google.com/scholar?q=Claude+Nadeau+and+Yoshua+Bengio%2E+2003%2E+Inference+for+the+generalization+error%2E+Machine+Learning%2C+52%2E)
22. Prakash M Nadkarni, Lucila Ohno-Machado, and Wendy W Chapman. 2011. Natural language processing: an introduction. Journal of the American Medical Informatics Association, 18(5):544–551. [find](https://scholar.google.com/scholar?q=Prakash+M+Nadkarni%2C+Lucila+Ohno%2DMachado%2C+and+Wendy+W+Chapman%2E+2011%2E+Natural+language+processing%3A+an+introduction%2E+Journal+of+the+American+Medical+Informatics+Association%2C+18%285%29%3A544%E2%80%93551%2E)
23. Mark Neumann, Daniel King, Iz Beltagy, and Waleed Ammar. 2019. [ScispaCy: fast and robust models for biomedical natural language processing](https://md.research.elamur.ai/free/en/papers/neumann-2019-scispacy-fast-robust-models-biomedical/). arXiv preprint arXiv:1902.07669.
24. Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020. [BERTweet: A pre-trained language model for English Tweets](https://md.research.elamur.ai/free/en/papers/nguyen-2020-bertweet-pre-trained-language-model/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 9–14.
25. Yifan Peng, Shankai Yan, and Zhiyong Lu. 2019. [Transfer learning in biomedical natural language processing: an evaluation of bert and elmo on ten benchmarking datasets](https://md.research.elamur.ai/free/en/papers/peng-2019-transfer-learning-biomedical-natural-language/). arXiv preprint arXiv:1906.05474.
26. Oona Rainio, Jarmo Teuho, and Riku Klén. 2024. Evaluation metrics and statistical tests for machine learning. Scientific reports, 14(1):6086. [find](https://scholar.google.com/scholar?q=Oona+Rainio%2C+Jarmo+Teuho%2C+and+Riku+Kl%C3%A9n%2E+2024%2E+Evaluation+metrics+and+statistical+tests+for+machine+learning%2E+Scientific+reports%2C+14%281%29%3A6086%2E)
27. Richard J Roberts. 2001. PubMed Central: The Gen-Bank of the published literature. [find](https://scholar.google.com/scholar?q=Richard+J+Roberts%2E+2001%2E+PubMed+Central%3A+The+Gen%2DBank+of+the+published+literature%2E)
28. Guzman Santafe, Iñaki Inza, and Jose A Lozano. 2015. Dealing with the evaluation of supervised classification algorithms. Artificial Intelligence Review, 44(4):467–508. [find](https://scholar.google.com/scholar?q=Guzman+Santafe%2C+I%C3%B1aki+Inza%2C+and+Jose+A+Lozano%2E+2015%2E+Dealing+with+the+evaluation+of+supervised+classification+algorithms%2E+Artificial+Intelligence+Review%2C+44%284%29%3A467%E2%80%93508%2E)
29. Abeed Sarker, Maksim Belousov, Jasper Friedrichs, Kai Hakala, Svetlana Kiritchenko, Farrokh Mehryary, Sifei Han, Tung Tran, Anthony Rios, Ramakanth Kavuluru, and 1 others. 2018. Data and systems for medication-related text classification and concept normalization from Twitter: insights from the Soci… [find](https://scholar.google.com/scholar?q=Abeed+Sarker%2C+Maksim+Belousov%2C+Jasper+Friedrichs%2C+Kai+Hakala%2C+Svetlana+Kiritchenko%2C+Farrokh+Mehryary%2C+Sifei+Han%2C+Tung+Tran%2C+Anthony+Rios%2C+Ramakanth+Kavuluru%2C+and+1+others%2E+2018%2E+Data+and+systems+for+m)
30. Davy Weissenbacher, Abeed Sarker, Arjun Magge, Ashlynn Daughton, Karen O’Connor, Michael Paul, and Graciela Gonzalez. 2019. Overview of the fourth social media mining for health (SMM4H) shared tasks at ACL 2019. In Proceedings of the fourth social media mining for health applications (\# SMM4H) works… [find](https://scholar.google.com/scholar?q=Davy+Weissenbacher%2C+Abeed+Sarker%2C+Arjun+Magge%2C+Ashlynn+Daughton%2C+Karen+O%E2%80%99Connor%2C+Michael+Paul%2C+and+Graciela+Gonzalez%2E+2019%2E+Overview+of+the+fourth+social+media+mining+for+health+%28SMM4H%29+shared+tasks+a)
31. David L Wheeler, Tanya Barrett, Dennis A Benson, Stephen H Bryant, Kathi Canese, Vyacheslav Chetvernin, Deanna M Church, Michael DiCuccio, Ron Edgar, Scott Federhen, and 1 others. 2007. Database resources of the national center for biotechnology information. Nucleic acids research, 35(suppl_1):D5–D1… [find](https://scholar.google.com/scholar?q=David+L+Wheeler%2C+Tanya+Barrett%2C+Dennis+A+Benson%2C+Stephen+H+Bryant%2C+Kathi+Canese%2C+Vyacheslav+Chetvernin%2C+Deanna+M+Church%2C+Michael+DiCuccio%2C+Ron+Edgar%2C+Scott+Federhen%2C+and+1+others%2E+2007%2E+Database+resou)
32. Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometrics bulletin, 1(6):80–83. [find](https://scholar.google.com/scholar?q=Frank+Wilcoxon%2E+1945%2E+Individual+comparisons+by+ranking+methods%2E+Biometrics+bulletin%2C+1%286%29%3A80%E2%80%9383%2E)
33. Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. [Aligning books and movies: Towards story-like visual explanations by watching movies and reading books](https://md.research.elamur.ai/free/en/papers/zhu-2015-aligning-books-movies-towards-story/). In Proceedings of the IEEE international conference on computer vision, pages 19–…
34. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
35. Özlem Uzuner, Brett R South, Shuying Shen, and Scott L DuVall. 2011. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association, 18(5):552–556. [find](https://scholar.google.com/scholar?q=%C3%96zlem+Uzuner%2C+Brett+R+South%2C+Shuying+Shen%2C+and+Scott+L+DuVall%2E+2011%2E+2010+i2b2%2FVA+challenge+on+concepts%2C+assertions%2C+and+relations+in+clinical+text%2E+Journal+of+the+American+Medical+Informatics+Associa)
36. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in neural information processing systems, pages 5998–6008.
37. Chih-Hsuan Wei, Yifan Peng, Robert Leaman, Allan Peter Davis, Carolyn J Mattingly, Jiao Li, Thomas C Wiegers, and Zhiyong Lu. 2016. Assessing the state of the art in biomedical relation extraction: overview of the BioCreative V chemical-disease relation (CDR) task. [find](https://scholar.google.com/scholar?q=Chih%2DHsuan+Wei%2C+Yifan+Peng%2C+Robert+Leaman%2C+Allan+Peter+Davis%2C+Carolyn+J+Mattingly%2C+Jiao+Li%2C+Thomas+C+Wiegers%2C+and+Zhiyong+Lu%2E+2016%2E+Assessing+the+state+of+the+art+in+biomedical+relation+extraction%3A+ov)
38. Davy Weissenbacher, Siddharth Rawal, Arjun Magge, and Graciela Gonzalez-Hernandez. 2021. Addressing extreme imbalance for detecting medications mentioned in twitter user timelines. In International Conference on Artificial Intelligence in Medicine, pages 93–102. Springer. [find](https://scholar.google.com/scholar?q=Davy+Weissenbacher%2C+Siddharth+Rawal%2C+Arjun+Magge%2C+and+Graciela+Gonzalez%2DHernandez%2E+2021%2E+Addressing+extreme+imbalance+for+detecting+medications+mentioned+in+twitter+user+timelines%2E+In+International+Co)

*Updated 1 November 2026*