---
title: "A Dataset of Psychiatric Hospital Notes with — Miller 2026"
description: "Clinical temporal information extraction identifies events and time expressions in notes and links them on a timeline."
authors: ["Timothy A. Miller", "Gaby Dinh", "David Harris", "WonJin Yoon", "Spencer Thomas", "Boyu Ren", "Meihua Hall", "Guergana Savova"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.592/"
doi: "10.63317/4pj6fbqovg3f"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/miller-2026-dataset-psychiatric-hospital-notes-temporal/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/miller-2026-dataset-psychiatric-hospital-notes-temporal/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/) / A Dataset of Psychiatric Hospital Notes with Temporal Information Annotations

# A Dataset of Psychiatric Hospital Notes with Temporal Information Annotations

**Authors:** Timothy A. Miller, Gaby Dinh, David Harris, [WonJin Yoon](https://md.research.elamur.ai/free/authors/wonjin-yoon/), Spencer Thomas, Boyu Ren, Meihua Hall, [Guergana Savova](https://md.research.elamur.ai/free/authors/guergana-savova/) · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.592/) · [PDF](https://aclanthology.org/2026.lrec-1.592.pdf) · [DOI 10.63317/4pj6fbqovg3f](https://doi.org/10.63317/4pj6fbqovg3f) · [OpenAlex](https://openalex.org/W7160128585)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Clinical temporal information extraction identifies events and time expressions in notes and links them on a timeline. Existing resources come mostly from oncology records (e.g., THYME), plus one general-medicine set (i2b2 2012). Psychiatry has had no comparable public data, even though its notes are strongly narrative, rich in psychosocial context, and cover long patient histories.

The authors built the first temporally annotated corpus of English psychiatric inpatient notes. It contains 120 de-identified discharge summaries and progress notes, split evenly between mood-disorder and psychosis patients and selected to match the cohort's demographics. They adapted the THYME guidelines to this domain, for instance by skipping non-narrative sections and tagging frequent psychiatric multi-word terms (such as suicidal ideation) as whole events. Annotation was done in Anafora.

In total, the corpus includes upward of 18,000 events, 13,000 temporal relations, and 2,200 temporal expressions. A baseline trained on non-psychiatric data scores only 0.152 F1 on relation extraction, which shows that out-of-domain models transfer poorly and that in-domain data is essential. The guidelines will be released publicly, and the corpus is available under data use agreements.

## Key points
- **Corpus composition:** 120 notes from a US academic psychiatric hospital.
  - Notes are half discharge summaries and half progress notes, with one note per patient.
  - Patients are half mood disorders (ICD10 F3*) and half psychosis (F2*).
- **Demographic sampling:** The authors drew 1000 random 120-note samples and kept the one whose demographic profile was closest to the full cohort by cosine similarity.
- **De-identification:**
  - Philter was modified to keep dates while removing birthdates.
  - Two rounds of manual review followed, the second by a psychiatry expert.
  - Annotators also reported any PHI they found during annotation.
- **Schema changes to THYME:**
  - Certain sections are ignored.
  - Multi-word event spans are used instead of head words only.
  - A "Do not annotate" layer marks non-narrative sections for an upstream filtering classifier.
- **Release:** Annotation offsets without text are shared via the GitHub repository "thyme-mh". Full text requires a data use agreement. Research was funded by NIMH (R01MH126977).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{miller2026dataset,
  title = {{A Dataset of Psychiatric Hospital Notes with Temporal Information Annotations}},
  author = {Miller, Timothy A. and Dinh, Gaby and Harris, David and Yoon, WonJin and Thomas, Spencer and Ren, Boyu and Hall, Meihua and Savova, Guergana},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/4pj6fbqovg3f},
  url = {https://doi.org/10.63317/4pj6fbqovg3f}
}
```

- [← previous A Dataset for Oral Reading in Young English Readers](https://md.research.elamur.ai/free/papers/rose-2026-dataset-oral-reading-young-english/)
- [next → A Dialogue System for Second Language Learning with Dynamic Adaptation to In-Dialogue Difficulty Changes](https://md.research.elamur.ai/free/papers/morioka-2026-dialogue-system-second-language-learning/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/)

## References

15 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. S Bethard, L Derczynski, and G Savova. 2015. Semeval-2015 task 6: Clinical tempeval. Proc. SemEval. [find](https://scholar.google.com/scholar?q=S+Bethard%2C+L+Derczynski%2C+and+G+Savova%2E+2015%2E+Semeval%2D2015+task+6%3A+Clinical+tempeval%2E+Proc%2E+SemEval%2E)
2. Steven Bethard. 2025. Anafora Tools. [https://github.com/bethard/anaforatools](https://github.com/bethard/anaforatools). Accessed: 2025-09-18. [↗](https://github.com/bethard/anaforatools)
3. Steven Bethard, Guergana Savova, Wei-Te Chen, Leon Derczynski, James Pustejovsky, and Marc Verhagen. 2016. SemEval-2016 Task 12: Clinical TempEval. In Proceedings of the 10th International Conference on Semantic Evaluation (SemEval 2016). [find](https://scholar.google.com/scholar?q=Steven+Bethard%2C+Guergana+Savova%2C+Wei%2DTe+Chen%2C+Leon+Derczynski%2C+James+Pustejovsky%2C+and+Marc+Verhagen%2E+2016%2E+SemEval%2D2016+Task+12%3A+Clinical+TempEval%2E+In+Proceedings+of+the+10th+International+Conference+)
4. Steven Bethard, Guergana Savova, Martha Palmer, and James Pustejovsky. 2017. SemEval-2017 Task 12: Clinical TempEval. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 565–572, Vancouver, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Steven+Bethard%2C+Guergana+Savova%2C+Martha+Palmer%2C+and+James+Pustejovsky%2E+2017%2E+SemEval%2D2017+Task+12%3A+Clinical+TempEval%2E+In+Proceedings+of+the+11th+International+Workshop+on+Semantic+Evaluation+%28SemEval%2D)
5. WT Chen and W Styler. 2013. Anafora: A Web-based General Purpose Annotation Tool. : The 2013 Annual Conference of the \\ldots. [find](https://scholar.google.com/scholar?q=WT+Chen+and+W+Styler%2E+2013%2E+Anafora%3A+A+Web%2Dbased+General+Purpose+Annotation+Tool%2E+%3A+The+2013+Annual+Conference+of+the+%5C%5Cldots%2E)
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+Deep+Bidirectional+Transformers+for+Language+Understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
7. Beau Norgeot, Kathleen Muenzen, Thomas A. Peterson, Xuancheng Fan, Benjamin S. Glicksberg, Gundolf Schenk, Eugenia Rutenberg, Boris Oskotsky, Marina Sirota, Jinoos Yazdany, Gabriela Schmajuk, Dana Ludwig, Theodore Goldstein, and Atul J. Butte. 2020. Protected Health Information filter (Philter): acc… [find](https://scholar.google.com/scholar?q=Beau+Norgeot%2C+Kathleen+Muenzen%2C+Thomas+A%2E+Peterson%2C+Xuancheng+Fan%2C+Benjamin+S%2E+Glicksberg%2C+Gundolf+Schenk%2C+Eugenia+Rutenberg%2C+Boris+Oskotsky%2C+Marina+Sirota%2C+Jinoos+Yazdany%2C+Gabriela+Schmajuk%2C+Dana+Lud)
8. James Pustejovsky, Kiyong Lee, Harry Bunt, and Roser Saur. 2011. ISO-TimeML Primer. pages 1–45. [find](https://scholar.google.com/scholar?q=James+Pustejovsky%2C+Kiyong+Lee%2C+Harry+Bunt%2C+and+Roser+Saur%2E+2011%2E+ISO%2DTimeML+Primer%2E+pages+1%E2%80%9345%2E)
9. James Pustejovsky and Amber Stubbs. 2011. Increasing informativeness in temporal annotation. Proceedings of the 5th Law Workshop (LAW V), (June):152–160. [find](https://scholar.google.com/scholar?q=James+Pustejovsky+and+Amber+Stubbs%2E+2011%2E+Increasing+informativeness+in+temporal+annotation%2E+Proceedings+of+the+5th+Law+Workshop+%28LAW+V%29%2C+%28June%29%3A152%E2%80%93160%2E) [our card](https://md.research.elamur.ai/free/papers/pustejovsky-2011-increasing-informativeness-temporal-annotation/)
10. William F. Styler IV, Steven Bethard, Sean Finan, Martha Palmer, Sameer Pradhan, Piet C de Groen, Brad Erickson, Timothy Miller, Chen Lin, Guergana Savova, and James Pustejovsky. 2014. Temporal Annotation in the Clinical Domain. Transactions of the Association for Computational Linguistics, 2:143–15… [find](https://scholar.google.com/scholar?q=William+F%2E+Styler+IV%2C+Steven+Bethard%2C+Sean+Finan%2C+Martha+Palmer%2C+Sameer+Pradhan%2C+Piet+C+de+Groen%2C+Brad+Erickson%2C+Timothy+Miller%2C+Chen+Lin%2C+Guergana+Savova%2C+and+James+Pustejovsky%2E+2014%2E+Temporal+Annota)
11. Weiyi Sun, Anna Rumshisky, and Ozlem Uzuner. 2013. Evaluating temporal relations in clinical text: 2012 i2b2 Challenge. Journal of the American Medical Informatics Association : JAMIA, 20(5):806–813. Place: England. [find](https://scholar.google.com/scholar?q=Weiyi+Sun%2C+Anna+Rumshisky%2C+and+Ozlem+Uzuner%2E+2013%2E+Evaluating+temporal+relations+in+clinical+text%3A+2012+i2b2+Challenge%2E+Journal+of+the+American+Medical+Informatics+Association+%3A+JAMIA%2C+20%285%29%3A806%E2%80%93813%2E+)
12. Naushad UzZaman and James F Allen. 2011. Temporal Evaluation. The 49th Annual Meeting of the Association for Computational Linguistics Human Language Technologies, 271(5):351–356. [find](https://scholar.google.com/scholar?q=Naushad+UzZaman+and+James+F+Allen%2E+2011%2E+Temporal+Evaluation%2E+The+49th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+Human+Language+Technologies%2C+271%285%29%3A351%E2%80%93356%2E)
13. Natalia Viani, Lucia Yin, Joyce Kam, Ayunni Alawi, André Bittar, Rina Dutta, Rashmi Patel, Robert Stewart, and Sumithra Velupillai. 2018. Time Expressions in Mental Health Records for Symptom Onset Extraction. In Proceedings of the Ninth International Workshop on Health Text Mining and Information A… [find](https://scholar.google.com/scholar?q=Natalia+Viani%2C+Lucia+Yin%2C+Joyce+Kam%2C+Ayunni+Alawi%2C+Andr%C3%A9+Bittar%2C+Rina+Dutta%2C+Rashmi+Patel%2C+Robert+Stewart%2C+and+Sumithra+Velupillai%2E+2018%2E+Time+Expressions+in+Mental+Health+Records+for+Symptom+Onset+Ex)
14. Kristin Wright-Bettner, Chen Lin, Timothy Miller, Steven Bethard, Dmitriy Dligach, Martha Palmer, James H. Martin, and Guergana Savova. 2020. Defining and Learning Refined Temporal Relations in the Clinical Narrative. In Proceedings of the 11th International Workshop on Health Text Mining and Inform… [find](https://scholar.google.com/scholar?q=Kristin+Wright%2DBettner%2C+Chen+Lin%2C+Timothy+Miller%2C+Steven+Bethard%2C+Dmitriy+Dligach%2C+Martha+Palmer%2C+James+H%2E+Martin%2C+and+Guergana+Savova%2E+2020%2E+Defining+and+Learning+Refined+Temporal+Relations+in+the+Cl)
15. Jiarui Yao, Harry Hochheiser, WonJin Yoon, Eli Goldner, and Guergana Savova. 2024. Overview of the 2024 Shared Task on Chemotherapy Treatment Timeline Extraction. In Proceedings of the 6th Clinical Natural Language Processing Workshop, pages 557–569, Mexico City, Mexico. Association for Computationa… [find](https://scholar.google.com/scholar?q=Jiarui+Yao%2C+Harry+Hochheiser%2C+WonJin+Yoon%2C+Eli+Goldner%2C+and+Guergana+Savova%2E+2024%2E+Overview+of+the+2024+Shared+Task+on+Chemotherapy+Treatment+Timeline+Extraction%2E+In+Proceedings+of+the+6th+Clinical+Na)