--- title: "Biomedical Named Entity Recognition using Conditional Random Fields and Rich Feature — Settles 2004" description: "This paper addresses biomedical named entity recognition: finding mentions of PROTEIN, DNA, RNA, CELL-LINE and… 7 references, cited by 69 papers here." authors: ["Burr Settles"] year: 2004 venue: "Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP)" source: "https://aclanthology.org/W04-1221/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/settles-2004-biomedical-named-entity-recognition-conditional/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/settles-2004-biomedical-named-entity-recognition-conditional/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/) / Biomedical Named Entity Recognition using Conditional Random Fields and Rich Feature Sets # Biomedical Named Entity Recognition using Conditional Random Fields and Rich Feature Sets **Authors:** [Burr Settles](https://md.research.elamur.ai/free/en/authors/burr-settles/) · **Year:** 2004 · **Venue:** Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W04-1221/) · [PDF](https://aclanthology.org/W04-1221.pdf) · [OpenAlex](https://openalex.org/W2114361266) **Citations:** 531 citations (OpenAlex, 2026-10-14) · [69](https://md.research.elamur.ai/free/en/papers/settles-2004-biomedical-named-entity-recognition-conditional/cited-by/) in our library, of the 140,884 papers whose bibliographies we read ## Summary This paper addresses biomedical named entity recognition: finding mentions of PROTEIN, DNA, RNA, CELL-LINE and CELL-TYPE in MEDLINE-style abstracts. The work was built for the BioNLP/NLPBA 2004 shared task and uses the organizers' training and evaluation data in IOB format. The author frames the task as sequence labeling with a linear-chain Conditional Random Field, implemented in MALLET and trained with L-BFGS. The paper compares two feature configurations. The first uses only orthographic and contextual cues. The second adds semantic lexicons, both hand-built and drawn from public databases, plus keyword lists derived automatically with a χ² test. The orthography-only model scored an overall F1 of 69.8, which the author considers roughly state of the art. Adding lexicons did not raise the overall score, which fell slightly to 69.5. However, lexicons helped the two rarest classes, RNA and CELL-LINE. The CRF also performed fairly evenly across entity types, which the author attributes to discriminative training. This shows that a mostly surface-feature CRF is a strong baseline for multi-class biomedical NER. ## Key points - Model: a first-order linear-chain CRF with binary feature functions and a Gaussian prior on the weights. It was trained with MALLET's L-BFGS and decoded with a Viterbi-style algorithm. - Orthographic features: - training vocabulary; - 17 regex-based flags, such as HasDash and RomanNumeral; - prefixes and suffixes of 3–5 characters; - "word class" and "brief word class" shape patterns; - neighboring words in a [-1,1] window. - Semantic resources: 17 lexicons in total. - Hand-entered lists cover Greek letters, amino acids, chemical elements, viruses and their abbreviations. - Database-derived lists come from Cancer GeneticsWeb, BBID, SwissProt and the Cell Line Database. - A CELL-TYPE list of about 50 entries was grown with Google Sets. - Results: overall F1 was 69.8 for the orthographic model (88.9 on training data; 230 iterations, about 18 hours). The full model reached 69.5 (86.7 on training data; 152 iterations, about 9 hours). - RNA and CELL-LINE together account for under 10% of mentions, and both gained in precision and recall from the lexicons. All comparisons come from a single split without cross-validation, so the differences are probably not significant. ## Interesting ideas - **Two-level word-shape features.** Each token is mapped to a character-class pattern, with uppercase, lowercase, digit and other characters each replaced by a symbol. A second, compressed version merges repeated symbols. Together these capture naming conventions like "IL5"/"SH3" without memorizing tokens, which makes them useful for any domain with productive, code-like names, such as product IDs or place codes. - **Class-indicative keyword lexicons via χ² testing.** Words inside labeled spans are counted per class. Each class is then tested against all the other classes combined, and words that differ significantly are kept for their majority class. This cheaply produces disambiguating cue lists that can help sparse labels, and it could be reused for any fine-grained tagging scheme. - **Bootstrapping a missing gazetteer from a few seeds.** Where no database existed, a set-expansion tool was given a few training examples and iteratively grew a list of new terms, some absent from the training data. This approach is attractive for low-resource languages or niche categories where curated lists are unavailable. - **Reporting relaxed boundary scores.** Alongside exact-match F1, the paper reports left-boundary and right-boundary matching F1. This separates boundary errors from type errors, which is useful when evaluating any span-based extractor. - **Per-class analysis of added features.** A feature set that slightly hurts overall F1 may still help rare classes. Breaking results down by label, and noting the lack of significance testing on a single split, guards against discarding features too early. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) ## Institutions - [University of Wisconsin–Madison](https://md.research.elamur.ai/free/en/universities/university-of-wisconsin-madison/) — University, Madison, United States · 18 works · 46,745 citations ## Cite Cite as: Burr Settles. 2004. Biomedical Named Entity Recognition using Conditional Random Fields and Rich Feature Sets. Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{settles2004biomedical, title = {{Biomedical Named Entity Recognition using Conditional Random Fields and Rich Feature Sets}}, author = {Burr Settles}, booktitle = {Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP)}, year = {2004}, doi = {10.3115/1567594.1567618}, url = {https://doi.org/10.3115/1567594.1567618} } ``` - [← previous A Linear Programming Formulation for Global Inference in Natural Language Tasks](https://md.research.elamur.ai/free/en/papers/roth-2004-linear-programming-formulation-global-inference/) - [next → Chinese Segmentation and New Word Detection using Conditional Random Fields](https://md.research.elamur.ai/free/en/papers/peng-2004-chinese-segmentation-new-word-detection/) - [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/) ## References 7 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Nigel Collier, Chikashi Nobata, and Jun ichi Tsujii. 2000. Extracting the names of genes and gene products with a hidden markov model. In Proceedings of the International Conference on Computational Linguistics, pages 201–207. Saarbru-ucken, Germany. [find](https://scholar.google.com/scholar?q=Nigel+Collier%2C+Chikashi+Nobata%2C+and+Jun+ichi+Tsujii%2E+2000%2E+Extracting+the+names+of+genes+and+gene+products+with+a+hidden+markov+model%2E+In+Proceedings+of+the+International+Conference+on+Computational+L) 2. Michael Collins. 2002. Ranking algorithms for named-entity extraction: Boosting and the voted perceptron. In Proceedings of the Association for Computational Linguistics Conference, pages 489–496. Philadelphia, USA. [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+2002%2E+Ranking+algorithms+for+named%2Dentity+extraction%3A+Boosting+and+the+voted+perceptron%2E+In+Proceedings+of+the+Association+for+Computational+Linguistics+Conference%2C+pages+489%E2%80%93496%2E+Phi) 3. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the International Conference on Machine Learning. Williamstown, MA, USA. [find](https://scholar.google.com/scholar?q=John+Lafferty%2C+Andrew+McCallum%2C+and+Fernando+Pereira%2E+2001%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E+In+Proceedings+of+the+International+Conference+on) 4. Andrew McCallum and Wei Li. 2003. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In Proceedings of the Conference on Natural Language Learning, pages 188–191. Edmonton, Canada. [our card](https://md.research.elamur.ai/free/en/papers/mccallum-2003-early-results-named-entity-recognition/) 5. Andrew McCallum. 2002. Mallet: A machine learning for language toolkit. [http://mallet.cs.umass.edu](http://mallet.cs.umass.edu). [↗](http://mallet.cs.umass.edu) 6. Ryan McDonald and Fernando Pereira. 2004. Identifying gene and protein mentions in text using conditional random fields. In Proceedings of BioCreative: Critical Assessment for Information Extraction in Biology. Grenada, Spain. [find](https://scholar.google.com/scholar?q=Ryan+McDonald+and+Fernando+Pereira%2E+2004%2E+Identifying+gene+and+protein+mentions+in+text+using+conditional+random+fields%2E+In+Proceedings+of+BioCreative%3A+Critical+Assessment+for+Information+Extraction+i) 7. Fei Sha and Fernando Pereira. 2003. Shallow parsing with conditional random fields. In Proceedings of the Human Language Technology and North American Association for Computational Linguistics Conference. Edmonton, Canada. [find](https://scholar.google.com/scholar?q=Fei+Sha+and+Fernando+Pereira%2E+2003%2E+Shallow+parsing+with+conditional+random+fields%2E+In+Proceedings+of+the+Human+Language+Technology+and+North+American+Association+for+Computational+Linguistics+Confere)