---
title: "Building a Coreference-Annotated — Batista-Navarro 2011"
description: "Biomedical coreference resolution has been held back by a shortage of annotated training data."
authors: ["Riza Theresa Batista-Navarro", "Sophia Ananiadou"]
year: 2011
venue: "Proceedings of BioNLP 2011 Workshop"
source: "https://aclanthology.org/W11-0210/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/navarro-2011-building-coreference-annotated-corpus-domain/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/navarro-2011-building-coreference-annotated-corpus-domain/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2011](https://md.research.elamur.ai/free/en/papers/2011/) / Building a Coreference-Annotated Corpus from the Domain of Biochemistry

# Building a Coreference-Annotated Corpus from the Domain of Biochemistry

**Authors:** Riza Theresa Batista-Navarro, [Sophia Ananiadou](https://md.research.elamur.ai/free/en/authors/sophia-ananiadou/) · **Year:** 2011 · **Venue:** Proceedings of BioNLP 2011 Workshop · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W11-0210/) · [PDF](https://aclanthology.org/W11-0210.pdf)

**Citations:** [6](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Biomedical coreference resolution has been held back by a shortage of annotated training data. Most existing resources, such as MEDSTRACT and the MEDCo abstracts, contain only abstracts. The one downloadable full-text corpus, FlySlip, covers only gene-related entities. To fill this gap, the authors built the HANAPIN corpus from 20 open-access full-text papers in marine natural products chemistry, a biochemistry subfield where many kinds of entities appear together. They wrote an annotation scheme covering four nominal coreference types: sortal, pronominal, numerical and abbreviation. The last two are frequent in chemistry writing and had not been annotated in earlier biomedical corpora. Three chemistry graduates labelled every coreferring mention of chemical compounds, drug targets, organisms, diseases and drug effects over five months. Numerical references turned out to be the most common type, and chemical compounds dominate the chains. Inter-annotator agreement was satisfactory, reaching 84% Krippendorff's alpha with the MASI distance. The planned public release of the corpus and guidelines gives researchers a multi-type, full-text resource for building and testing biomedical coreference systems.

## Key points
- **Source and size:** Articles were sampled at random from the journal Marine Drugs (PubMed Central Open Access subset), published 2003–2009. Experimental sections were left unannotated. The corpus totals 1,027 sentences and 27,358 words.
- **Chains:** There are 395 non-singleton coreference chains.
  - Chemical compounds: 70.89%
  - Drug targets: 12.66% (mostly proteins, cell lines and pathogens)
  - Organisms: 9.87%
  - Drug effects: 3.29%
  - Diseases: 3.29%
- **Links:** Of 760 coreference links, 46% are numerical, 33% sortal, 11% pronominal and 10% abbreviation. Within the sortal type, definite and indefinite noun phrases are the most frequent subtypes.
- **Reliability:** Krippendorff's alpha was 75% using the original set-based distance and 84% using MASI. Both exceed the 67% level usually cited as the threshold for unreliability.
- **Comparison and scope:** The paper compares HANAPIN with MEDSTRACT, MEDCo, FlySlip, CRAFT and DrugNerAr. Annotation was done in XConc with XML output. Verbal and indirect coreference are not covered and are left for future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/) · [Treebanks](https://md.research.elamur.ai/free/en/topics/treebanks/)

## Cited by

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 3 not yet.

- [Coreference Resolution for the Biomedical Domain: A Survey](https://md.research.elamur.ai/free/en/papers/lu-2021-coreference-resolution-biomedical-domain-survey/)
- [SciCorp: A Corpus of English Scientific Articles Annotated for Information Status Analysis](https://md.research.elamur.ai/free/en/papers/roesiger-2016-scicorp-corpus-english-scientific-articles/)
- [Towards Coreference for Literary Text: Analyzing Domain-Specific Phenomena](https://md.research.elamur.ai/free/en/papers/roesiger-2018-coreference-literary-text-analyzing-domain/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{batistanavarro2011building,
  title = {{Building a Coreference-Annotated Corpus from the Domain of Biochemistry}},
  author = {Batista-Navarro, Riza Theresa and Ananiadou, Sophia},
  booktitle = {Proceedings of BioNLP 2011 Workshop},
  year = {2011},
  url = {https://aclanthology.org/W11-0210/}
}
```

- [← previous Bootstrapping coreference resolution using word associations](https://md.research.elamur.ai/free/en/papers/kobdani-2011-bootstrapping-coreference-resolution-word-associations/)
- [next → Chameleons in Imagined Conversations: A New Approach to Understanding Coordination of Linguistic Style in Dialogs](https://md.research.elamur.ai/free/en/papers/mizil-2011-chameleons-imagined-conversations-new-approach/)
- [↑ back to papers of 2011](https://md.research.elamur.ai/free/en/papers/2011/)

## References

16 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ron Artstein and Massimo Poesio. 2004. Inter-Coder Agreement for Computational Linguistics. Computational Linguistics, 34(4):555-596. [find](https://scholar.google.com/scholar?q=Ron+Artstein+and+Massimo+Poesio%2E+2004%2E+Inter%2DCoder+Agreement+for+Computational+Linguistics%2E+Computational+Linguistics%2C+34%284%29%3A555%2D596%2E)
2. José Castaño, Jason Zhang and James Pustejovsky. 2002. Anaphora resolution in biomedical literature. Proceedings of the International Symposium on Reference Resolution for NLP. [find](https://scholar.google.com/scholar?q=Jose%CC%81+Castan%CC%83o%2C+Jason+Zhang+and+James+Pustejovsky%2E+2002%2E+Anaphora+resolution+in+biomedical+literature%2E+Proceedings+of+the+International+Symposium+on+Reference+Resolution+for+NLP%2E)
3. K. Bretonnel Cohen, Philip V. Ogren, Lynne Fox and Lawrence E. Hunter. 2005. Empirical data on corpus design and usage in biomedical natural language processing. AMIA Annual Symposium Proceedings, pages 156-160. [find](https://scholar.google.com/scholar?q=K%2E+Bretonnel+Cohen%2C+Philip+V%2E+Ogren%2C+Lynne+Fox+and+Lawrence+E%2E+Hunter%2E+2005%2E+Empirical+data+on+corpus+design+and+usage+in+biomedical+natural+language+processing%2E+AMIA+Annual+Symposium+Proceedings%2C+pag)
4. K. Bretonnel Cohen, Helen L. Johnson, Karin Verspoor, Christophe Roeder, Lawrence E. Hunter. 2010. The structural and content aspects of abstracts versus bodies of full text journal articles are different. BMC Bioinformatics, 11(1):492. [find](https://scholar.google.com/scholar?q=K%2E+Bretonnel+Cohen%2C+Helen+L%2E+Johnson%2C+Karin+Verspoor%2C+Christophe+Roeder%2C+Lawrence+E%2E+Hunter%2E+2010%2E+The+structural+and+content+aspects+of+abstracts+versus+bodies+of+full+text+journal+articles+are+diffe)
5. K. Bretonnel Cohen, Arrick Lanfranchi, William Corvey, William A. Baumgartner Jr., Christophe Roeder, Philip V. Ogren, Martha Palmer and Lawrence E. Hunter. 2010. Annotation of all coreference in biomedical text: Guideline selection and adaptation. Proceedings of the Second Workshop on Building and… [find](https://scholar.google.com/scholar?q=K%2E+Bretonnel+Cohen%2C+Arrick+Lanfranchi%2C+William+Corvey%2C+William+A%2E+Baumgartner+Jr%2E%2C+Christophe+Roeder%2C+Philip+V%2E+Ogren%2C+Martha+Palmer+and+Lawrence+E%2E+Hunter%2E+2010%2E+Annotation+of+all+coreference+in+biom)
6. Caroline Gasperin, Nikiforos Karamanis and Ruth Seal. 2007. Annotation of anaphoric relations in biomedical full-text articles using a domain-relevant scheme. Proceedings of the 6th Discourse Anaphora and Anaphor Resolution Colloquium (DAARC 2007). [find](https://scholar.google.com/scholar?q=Caroline+Gasperin%2C+Nikiforos+Karamanis+and+Ruth+Seal%2E+2007%2E+Annotation+of+anaphoric+relations+in+biomedical+full%2Dtext+articles+using+a+domain%2Drelevant+scheme%2E+Proceedings+of+the+6th+Discourse+Anaphora)
7. Ralph Grishman and Beth Sundheim. 1995. Design of the MUC-6 Evaluation. MUC ’95: Proceedings of the 6th Message Understanding Conference, pages 1-11. [find](https://scholar.google.com/scholar?q=Ralph+Grishman+and+Beth+Sundheim%2E+1995%2E+Design+of+the+MUC%2D6+Evaluation%2E+MUC+%E2%80%9995%3A+Proceedings+of+the+6th+Message+Understanding+Conference%2C+pages+1%2D11%2E)
8. Lynette Hirschman. 1997. MUC-7 Coreference Task Definition. Message Understanding Conference 7 Proceedings. [find](https://scholar.google.com/scholar?q=Lynette+Hirschman%2E+1997%2E+MUC%2D7+Coreference+Task+Definition%2E+Message+Understanding+Conference+7+Proceedings%2E) [our card](https://md.research.elamur.ai/free/en/papers/hirschman-1998-appendix-f-muc-7-coreference/)
9. Daniel Jurafsky and James H. Martin. 2009. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Prentice-Hall, 2nd edition. [find](https://scholar.google.com/scholar?q=Daniel+Jurafsky+and+James+H%2E+Martin%2E+2009%2E+Speech+and+Language+Processing%3A+An+Introduction+to+Natural+Language+Processing%2C+Computational+Linguistics%2C+and+Speech+Recognition%2E+Prentice%2DHall%2C+2nd+edition)
10. Klaus H. Krippendorff. 1980. Content Analysis: An Introduction to Its Methodology. Beverly Hills, CA: Sage Publications. [find](https://scholar.google.com/scholar?q=Klaus+H%2E+Krippendorff%2E+1980%2E+Content+Analysis%3A+An+Introduction+to+Its+Methodology%2E+Beverly+Hills%2C+CA%3A+Sage+Publications%2E)
11. Tara McIntosh and James R.Curran. 2009. Challenges for automatically extracting molecular interactions from full-text articles. BMC Bioinformatics, 10(1):311. [find](https://scholar.google.com/scholar?q=Tara+McIntosh+and+James+R%2ECurran%2E+2009%2E+Challenges+for+automatically+extracting+molecular+interactions+from+full%2Dtext+articles%2E+BMC+Bioinformatics%2C+10%281%29%3A311%2E)
12. Ruslan Mitkov, Richard Evans, Constantin Orasan, Catalina Barbu, Lisa Jones and Violeta Sotirova. 2005. Coreference and anaphora: developing annotating tools, annotated resources and annotation strategies. Proceedings of the Discourse Anaphora and Anaphora Resolution Colloquium (DAARC 2000), pages 4… [find](https://scholar.google.com/scholar?q=Ruslan+Mitkov%2C+Richard+Evans%2C+Constantin+Orasan%2C+Catalina+Barbu%2C+Lisa+Jones+and+Violeta+Sotirova%2E+2005%2E+Coreference+and+anaphora%3A+developing+annotating+tools%2C+annotated+resources+and+annotation+strate)
13. Rebecca J. Passoneau. 2004. Computing reliability for coreference annotation. Proceedings of the International Conference on Language Resouces (LREC). [find](https://scholar.google.com/scholar?q=Rebecca+J%2E+Passoneau%2E+2004%2E+Computing+reliability+for+coreference+annotation%2E+Proceedings+of+the+International+Conference+on+Language+Resouces+%28LREC%29%2E)
14. M. Schumie, M. Weeber, B. Schijvenaars, E. van Mulligen, C. van der Eijk, R. Jelier, B. Mons and J. Kors. 2004. Distribution of information in biomedical abstracts and full-text publications. Bioinformatics, 20(16):2597-2604. [find](https://scholar.google.com/scholar?q=M%2E+Schumie%2C+M%2E+Weeber%2C+B%2E+Schijvenaars%2C+E%2E+van+Mulligen%2C+C%2E+van+der+Eijk%2C+R%2E+Jelier%2C+B%2E+Mons+and+J%2E+Kors%2E+2004%2E+Distribution+of+information+in+biomedical+abstracts+and+full%2Dtext+publications%2E+Bioinfor)
15. Isabel Segura-Bedmar, Mario Crespo, César de Pablo-Sánchez and Paloma Martı́nez. 2009. Resolving anaphoras for the extraction of drug- drug interactions in pharmacological documents. BMC Bioinformatics, 11(Suppl 2):S1. [find](https://scholar.google.com/scholar?q=Isabel+Segura%2DBedmar%2C+Mario+Crespo%2C+Ce%CC%81sar+de+Pablo%2DSa%CC%81nchez+and+Paloma+Mart%C4%B1%CC%81nez%2E+2009%2E+Resolving+anaphoras+for+the+extraction+of+drug%2D+drug+interactions+in+pharmacological+documents%2E+BMC+Bioinformat)
16. Parantu K. Shah, Carolina Perez-Iratxeta, Peer Bork and Miguel A. Andrade. 2003. Information extraction from full text scientific articles: Where are the keywords? BMC Bioinformatics, 4(1): 20. [find](https://scholar.google.com/scholar?q=Parantu+K%2E+Shah%2C+Carolina+Perez%2DIratxeta%2C+Peer+Bork+and+Miguel+A%2E+Andrade%2E+2003%2E+Information+extraction+from+full+text+scientific+articles%3A+Where+are+the+keywords%3F+BMC+Bioinformatics%2C+4%281%29%3A+20%2E)