--- title: "PubMed 200k RCT — paper by Dernoncourt and Lee (2017)" description: "The paper addresses sequential sentence classification: assigning each sentence of a medical abstract… 1 reference, cited by 84 papers here." authors: ["Franck Dernoncourt", "Ji Young Lee"] year: 2017 venue: "Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)" source: "https://aclanthology.org/I17-2052/" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/dernoncourt-2017-pubmed-200k-rct-dataset-sequential/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dernoncourt-2017-pubmed-200k-rct-dataset-sequential/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/) / PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts # PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts **Authors:** [Franck Dernoncourt](https://md.research.elamur.ai/free/en/authors/franck-dernoncourt/) and [Ji Young Lee](https://md.research.elamur.ai/free/en/authors/ji-young-lee/) · **Year:** 2017 · **Venue:** [Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)](https://md.research.elamur.ai/free/en/venues/ijcnlp/) ([IJCNLP 2017](https://md.research.elamur.ai/free/en/venues/ijcnlp/2017/)) · **Publisher:** [Asian Federation of Natural Language Processing](https://md.research.elamur.ai/free/en/publishers/asian-federation-of-natural-language-processing/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/I17-2052/) · [PDF](https://aclanthology.org/I17-2052.pdf) · [OpenAlex](https://openalex.org/W2766801974) **Citations:** 31 citations (OpenAlex, 2026-10-14) · [84](https://md.research.elamur.ai/free/en/papers/dernoncourt-2017-pubmed-200k-rct-dataset-sequential/cited-by/) in our library, of the 141,840 papers whose bibliographies we read ## Summary The paper addresses sequential sentence classification: assigning each sentence of a medical abstract a rhetorical role while taking its neighbours into account. Over half of RCT abstracts in PubMed lack section headings, which slows literature screening for tasks such as systematic reviews. Earlier corpora for this task were small, did not focus on RCTs, or could not be obtained. To fill this gap, the authors built PubMed 200k RCT from the 2016 MEDLINE/PubMed Baseline. They kept articles tagged with the RCT MeSH term and only those abstracts that already have clean section structure. The headings that authors assigned, as normalised by PubMed, then serve as free sentence labels: background, objective, methods, results and conclusions. The result is about 196k abstracts with 2.3M sentences, plus a smaller 20k variant. Four baselines were tested, ranging from a context-free logistic regression to a bi-LSTM+CRF network (bi-ANN), and the bi-ANN performed best with an F1 of up to 91.6. The work provides a large, public benchmark for context-aware sentence labelling and a practical basis for tools that help readers skim abstracts. ## Key points - **Source and filtering:** The data come from the MEDLINE/PubMed Baseline 2016, which has 24,358,442 records. - The MeSH term D016449 selects 399,254 RCT abstracts. - Of these, 195,654 are structured, meaning they have 3–9 sections and none labelled "None", "Unassigned" or empty. - **PubMed 200k RCT splits:** 190,654 abstracts for training, 2,500 for validation and 2,500 for test. Each sentence carries one of 5 labels. - **PubMed 20k RCT splits:** 15k/2.5k/2.5k abstracts, drawn from the most recently published abstracts in the large set. - **Format:** Plain text files with a PMID line followed by lines of the form "LABEL sentence". Sentence and token boundaries come from Stanford CoreNLP. A second copy replaces all digits with "@". - **Test F1 results (20k / 200k):** | Model | 20k | 200k | |---|---|---| | LR | 83.1 | 85.9 | | Forward ANN | 86.1 | 88.4 | | CRF | 89.5 | 91.5 | | bi-ANN | 90.0 | 91.6 | Background and objective are the hardest classes, with F1 of 75.6 and 70.7, and they are frequently mistaken for each other. ## Interesting ideas - **Free labels from existing structure:** Instead of paying annotators, the authors treat structure that authors already created (section headings) as supervision and apply it to sentences. Many other domains have similar latent markup, such as headed documents, forms or legal texts with numbered parts, which could yield large labelled corpora cheaply. - **Strict filtering over maximum coverage:** Abstracts with unusual section counts or unmapped headings were simply dropped, at a cost of only about 0.5% outliers plus the unstructured ones. Trading a bit of volume for label cleanliness is a sensible default when building weakly supervised datasets. - **Two dataset sizes and a digit-masked copy:** A small subset lets people iterate quickly, and replacing digits with "@" shrinks vocabulary noise without extra preprocessing on the user's side. Both choices make a resource easier to adopt. - **Baselines that isolate the value of context:** The lineup runs from a model that ignores context (LR) to one that sees only preceding sentences (Forward ANN) to models that use the full sequence (CRF, bi-ANN). This ladder shows how much surrounding context actually contributes, and the same evaluation design transfers to any sequence-labelling benchmark. - **Reporting runtime next to accuracy in ablations:** The n-gram size and window size tables show diminishing F1 returns against growing cost. This helps practitioners choose a reasonable operating point rather than only the top score. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 159 works · 65,157 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [🧬 Biochemistry, Genetics and Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/) · 116 works · [🧬 Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/molecular-biology/) · 114 works ## Cite Cite as: Franck Dernoncourt and Ji Young Lee. 2017. PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts. Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{dernoncourt2017pubmed, title = {{PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts}}, author = {Franck Dernoncourt and Ji Young Lee}, booktitle = {Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)}, year = {2017}, url = {https://aclanthology.org/I17-2052/} } ``` - [← previous An Incremental Parser for Abstract Meaning Representation](https://md.research.elamur.ai/free/en/papers/damonte-2017-incremental-parser-abstract-meaning-representation/) - [next → Arabic Diacritization: Stats, Rules, and Hacks](https://md.research.elamur.ai/free/en/papers/darwish-2017-arabic-diacritization-stats-rules-hacks/) - [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Iman Amini, David Martinez, and Diego Molla. 2012. Overview of the ALTA 2012 Shared Task. In Australasian Language Technology Association Workshop 2012. volume 7, page 124. \#\# Conclusion [find](https://scholar.google.com/scholar?q=Iman+Amini%2C+David+Martinez%2C+and+Diego+Molla%2E+2012%2E+Overview+of+the+ALTA+2012+Shared+Task%2E+In+Australasian+Language+Technology+Association+Workshop+2012%2E+volume+7%2C+page+124%2E+%5C%23%5C%23+Conclusion) *Updated 24 October 2026*