---
title: "BioASQ-QA — paper by Krithara et al. (2023) · Scientific Data"
description: "This data descriptor presents BioASQ-QA, the corpus built for Task B of the BioASQ challenge, which… 18 references, cited by 72 papers here."
authors: ["Anastasia Krithara", "Anastasios Nentidis", "Konstantinos Bougiatiotis", "Georgios Paliouras (Γεώργιος Παλιούρας)"]
year: 2023
venue: "Scientific Data"
source: "https://doi.org/10.1038/s41597-023-02068-4"
doi: "10.1038/s41597-023-02068-4"
licence: "CC BY 4.0 (publisher, via OpenAlex)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/krithara-2023-bioasq-qa-manually-curated-corpus/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/krithara-2023-bioasq-qa-manually-curated-corpus/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/5/) / BioASQ-QA: A manually curated corpus for Biomedical Question Answering

# BioASQ-QA: A manually curated corpus for Biomedical Question Answering

**Authors:** Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis and Georgios Paliouras (Γεώργιος Παλιούρας) · **Year:** 2023 · **Venue:** [Scientific Data](https://md.research.elamur.ai/free/en/venues/scientific-data/) ([Scientific Data 2023](https://md.research.elamur.ai/free/en/venues/scientific-data/2023/)) · **Publisher:** [Nature Portfolio](https://md.research.elamur.ai/free/en/publishers/nature-portfolio/) · **Licence:** [CC BY 4.0 (publisher, via OpenAlex)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://doi.org/10.1038/s41597-023-02068-4) · [PDF](https://www.nature.com/articles/s41597-023-02068-4.pdf) · [DOI 10.1038/s41597-023-02068-4](https://doi.org/10.1038/s41597-023-02068-4) · [OpenAlex](https://openalex.org/W4361001766)

**Citations:** 73 citations (OpenAlex, 2026-10-14) · [72](https://md.research.elamur.ai/free/en/papers/krithara-2023-bioasq-qa-manually-curated-corpus/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Biomedical literature grows faster than experts can follow, with PubMed now holding tens of millions of records. Systems that answer precise research questions are therefore needed, but most QA benchmarks are less realistic and provide only short answers. This data descriptor presents BioASQ-QA, the corpus built for Task B of the BioASQ challenge, which has run every year since 2012. European biomedical specialists write English questions based on needs from their own work. They use a dedicated annotation tool to find supporting MEDLINE articles and snippets and to link concepts from resources such as MeSH, Gene Ontology, Disease Ontology, Jochem and UniProt. Each question receives a paragraph-length "ideal" answer that works as a summary and, where the type allows, an exact answer as well. After each challenge round, the experts grade system outputs and use them to correct and expand the gold data. The resource now contains 4721 questions and continues to grow. It supports work on retrieval, QA, multi-document summarization and text generation. It has also served as a common benchmark showing steady progress, with top systems' ideal answers scored above 4 out of 5.

## Key points
- **Size and growth:** 4721 expert-written questions so far, with about 500 added each year. In total, 21 experts contributed, and 7 of them were the most active.
- **Question types:** factoid, yes/no, list and summary. Each question comes with relevant documents, snippets, ontology concepts, an ideal answer and, where applicable, an exact answer. RDF triples from Linked Life Data were used early on but later dropped.
- **Ideal answer grading:** experts score ideal answers on four criteria (including precision, repetition and readability) using a 1–5 scale. Averages for the best systems exceed 80%.
- **Inter-annotator agreement:** experts chose quite different documents and snippets (mean F1 below 50%). Their ideal answers nevertheless agreed closely, meaning they reached the same answers through different evidence.
- **Availability:** the data is released as JSON on Zenodo, and the BioASQ software is open source on GitHub. Around 100 universities and companies have taken part in the challenge.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National Centre of Scientific Research "Demokritos"](https://md.research.elamur.ai/free/en/labs/national-centre-of-scientific-research-demokritos/) — Research institute, Athens, Greece · 4 works · 1,826 citations
- [Institute of Informatics & Telecommunications](https://md.research.elamur.ai/free/en/labs/institute-of-informatics-telecommunications/) — Research institute, Agia Paraskevi, Greece · 2 works · 1,638 citations
- [Aristotle University of Thessaloniki](https://md.research.elamur.ai/free/en/universities/aristotle-university-of-thessaloniki/) — University, Thessaloniki, Greece · 2 works · 193 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works · [🧬 Biochemistry, Genetics and Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/) · 263 works · [🧬 Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/molecular-biology/) · 260 works

🧭 Topics (1): [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 487 works

## Cite

Cite as: Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis and Georgios Paliouras. 2023. BioASQ-QA: A manually curated corpus for Biomedical Question Answering. Scientific Data.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{krithara2023bioasq,
  title = {{BioASQ-QA: A manually curated corpus for Biomedical Question Answering}},
  author = {Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis and Georgios Paliouras},
  booktitle = {Scientific Data},
  year = {2023},
  doi = {10.1038/s41597-023-02068-4},
  url = {https://doi.org/10.1038/s41597-023-02068-4}
}
```

- [← previous Unified Demonstration Retriever for In-Context Learning](https://md.research.elamur.ai/free/en/papers/li-2023-unified-demonstration-retriever-context-learning/)
- [next → Efficient Methods for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/treviso-2023-efficient-methods-natural-language-processing/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/5/)

## References

18 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. National Library of Medicine. Medline pubmed production statistics. [https://www.nlm.nih.gov/bsd/pmresources.html](https://www.nlm.nih.gov/bsd/pmresources.html) (2022). [↗](https://www.nlm.nih.gov/bsd/pmresources.html)
2. Chen, Q., Allot, A. & Lu, Z. LitCovid: an open database of COVID-19 literature. Nucleic Acids Research 49, D1534–D1540, https://doi.org/10.1093/nar/gkaa952 (2020). — [doi:10.1093/nar/gkaa952](https://doi.org/10.1093/nar/gkaa952) [↗](https://doi.org/10.1093/nar/gkaa952)
3. Nentidis, A., Krithara, A. & Paliouras, G. BioASQ website. www.BioASQ.org (2022). [find](https://scholar.google.com/scholar?q=Nentidis%2C+A%2E%2C+Krithara%2C+A%2E+%26+Paliouras%2C+G%2E+BioASQ+website%2E+www%2EBioASQ%2Eorg+%282022%29%2E)
4. National Library of Medicine. The Medical Subject Headings (MeSH) thesaurus. [https://www.nlm.nih.gov/mesh/meshhome.html](https://www.nlm.nih.gov/mesh/meshhome.html) (2022). [↗](https://www.nlm.nih.gov/mesh/meshhome.html)
5. Linked Life Data (LLD). [http://linkedlifedata.com/](http://linkedlifedata.com/) (2012). [↗](http://linkedlifedata.com/)
6. Wasim, M., Mahmood, D. W. & Khan, D. U. G. A survey of datasets for biomedical question answering systems. International Journal of Advanced Computer Science and Applications 8, https://doi.org/10.14569/IJACSA.2017.080767 (2017). — [doi:10.14569/ijacsa.2017.080767](https://doi.org/10.14569/ijacsa.2017.080767) [↗](https://doi.org/10.14569/IJACSA.2017.080767)
7. Jin, Q. et al. Biomedical question answering: A survey of approaches and challenges. ACM Comput. Surv. 55, https://doi.org/10.1145/3490238 (2022). [↗](https://doi.org/10.1145/3490238)
8. Hettne, K. M. et al. A dictionary to identify small molecules and drugs in free text. Bioinformatics 25, 2983–2991, https://doi.org/10.1093/bioinformatics/btp535 (2009). — [doi:10.1093/bioinformatics/btp535](https://doi.org/10.1093/bioinformatics/btp535) [↗](https://doi.org/10.1093/bioinformatics/btp535)
9. Ashburner, M. et al. Gene ontology: tool for the unification of biology. the gene ontology consortium. Nat Genet 25, 25–29, https://doi.org/10.1038/75556 (2000). — [doi:10.1038/75556](https://doi.org/10.1038/75556) [↗](https://doi.org/10.1038/75556)
10. The Gene Ontology Consortium. The Gene Ontology Resource: 20 years and still GOing strong. Nucleic Acids Research 47, D330–D338, https://doi.org/10.1093/nar/gky1055 (2018). — [doi:10.1093/nar/gky1055](https://doi.org/10.1093/nar/gky1055) [↗](https://doi.org/10.1093/nar/gky1055)
11. The UniProt Consortium. UniProt: the Universal Protein Knowledgebase. Nucleic Acids Research 51, D523–D531, https://doi.org/10.1093/nar/gkac1052 (2022). — [doi:10.1093/nar/gkac1052](https://doi.org/10.1093/nar/gkac1052) [↗](https://doi.org/10.1093/nar/gkac1052)
12. Schriml, L. M. et al. Human Disease Ontology 2018 update: classification, content and workflow expansion. Nucleic Acids Research 47, D955–D962, https://doi.org/10.1093/nar/gky1032 (2018). — [doi:10.1093/nar/gky1032](https://doi.org/10.1093/nar/gky1032) [↗](https://doi.org/10.1093/nar/gky1032)
13. Doms, A. & Schroeder, M. GoPubMed: exploring PubMed with the Gene Ontology. Nucleic Acids Research 33, W783–W786, https://doi.org/10.1093/nar/gki470 (2005). — [doi:10.1093/nar/gki470](https://doi.org/10.1093/nar/gki470) [↗](https://doi.org/10.1093/nar/gki470)
14. Malakasiotis, P., Androutsopoulos, I., Almirantis, Y., Polychronopoulos, D. & Pavlopoulos, I. Tutorials and guidelines 2 [http://www.bioasq.org/sites/default/files/PublicDocuments/BioASQ_D3.7-TutorialsGuidelines2ndVersion_final_0.pdf](http://www.bioasq.org/sites/default/files/PublicDocuments/BioASQ_D3.7-TutorialsGuidelines2ndVersion_final_0.pdf) (2013). [↗](http://www.bioasq.org/sites/default/files/PublicDocuments/BioASQ_D3.7-TutorialsGuidelines2ndVersion_final_0.pdf)
15. Krithara, A., Nentidis, A., Bougiatiotis, K. & Paliouras, G. [BioASQ-QA: A manually curated corpus for biomedical question answering](https://md.research.elamur.ai/free/en/papers/krithara-2023-bioasq-qa-manually-curated-corpus/). zenodo https://doi.org/10.5281/zenodo.7655130 (2023). — [doi:10.5281/zenodo.7655130](https://doi.org/10.5281/zenodo.7655130) [↗](https://doi.org/10.5281/zenodo.7655130)
16. Nentidis, A. et al. Overview of BioASQ 2020: The eighth BioASQ challenge on large-scale biomedical semantic indexing and question answering. In 11th International Conference of the CLEF Association, vol. 12260 of Lecture Notes in Computer Science, 194–214, https://doi.org/10.1007/978-3-030-58219-7_1… — [doi:10.1007/978-3-030-58219-7_16](https://doi.org/10.1007/978-3-030-58219-7_16) [↗](https://doi.org/10.1007/978-3-030-58219-7_1…)
17. Nentidis, A. et al. Overview of BioASQ 2021: The Ninth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering. In 12th International Conference of the CLEF Association, vol. 12880 of Lecture Notes in Computer Science, 239–263, https://doi.org/10.1007/978-3-030-85251-1_18… — [doi:10.1007/978-3-030-85251-1_18](https://doi.org/10.1007/978-3-030-85251-1_18) [↗](https://doi.org/10.1007/978-3-030-85251-1_18…)
18. Nentidis, A. et al. Overview of BioASQ 2022: The Tenth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering. In 13th International Conference of the CLEF Association, vol. 13390 of Lecture Notes in Computer Science, 337–361, https://doi.org/10.1007/978-3-031-13643-6_22… — [doi:10.1007/978-3-031-13643-6_22](https://doi.org/10.1007/978-3-031-13643-6_22) [↗](https://doi.org/10.1007/978-3-031-13643-6_22…)

*Updated 2 November 2026*