---
title: "emrQA: A Large Corpus for Question Answering — Pampari 2018"
description: "Physicians often cannot find the information they need in unstructured electronic medical records (EMRs)."
authors: ["Anusri Pampari", "Preethi Raghavan", "Jennifer Liang", "Jian Peng"]
year: 2018
venue: "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D18-1258/"
doi: "10.18653/v1/d18-1258"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pampari-2018-emrqa-large-corpus-question-answering/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pampari-2018-emrqa-large-corpus-question-answering/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / emrQA: A Large Corpus for Question Answering on Electronic Medical Records

# emrQA: A Large Corpus for Question Answering on Electronic Medical Records

**Authors:** Anusri Pampari, Preethi Raghavan, Jennifer Liang, [Jian Peng](https://md.research.elamur.ai/free/en/authors/jian-peng/) · **Year:** 2018 · **Venue:** Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D18-1258/) · [PDF](https://aclanthology.org/D18-1258.pdf) · [DOI 10.18653/v1/d18-1258](https://doi.org/10.18653/v1/d18-1258) · [OpenAlex](https://openalex.org/W2891113091)

**Citations:** 187 citations (openalex-snapshot · 2026-10-14) · [98](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Physicians often cannot find the information they need in unstructured electronic medical records (EMRs). No public, large-scale corpus existed for training patient-specific QA systems, because privacy restrictions and the cost of expert labeling make manual annotation impractical. The authors propose a generation framework that reuses annotations originally made for other clinical NLP tasks.

The framework works in three steps:
- Questions gathered from physicians and medical experts are turned into templates with entity placeholders.
- A domain expert pairs each template with a logical form template.
- Existing i2b2 annotations (entities, relations, coreference) automatically fill the placeholders and supply answer evidence.

Writing logical forms is the only manual work. Applied to several i2b2 datasets, the method produced emrQA. It contains over 400,000 question–answer pairs and about 1 million question–logical form pairs, and it includes arithmetic and temporal reasoning that SQuAD lacks. The baselines are a DrQA reader for evidence extraction, logistic regression for class-type answers, and models for predicting logical forms. They show the corpus is far from trivial. The approach is relevant to any domain where direct QA annotation is infeasible but annotations for related tasks already exist.

## Key points
- emrQA contains 455,837 question–answer pairs and 1,295,814 question–logical form pairs over 2,425 clinical notes. The data comes from five i2b2 sources: Relations, Medications, Heart disease, Obesity and Smoking.
- Source questions came from three places: 976 from Veterans Administration physicians, 5,696 written by medical experts over 71 patient records, and 15 prototypical questions. These were normalized into templates linked to logical forms.
- Average lengths are 8.6 tokens per question, 18.7 per evidence span, 33 per logical form and 3,825 per note. Each note has about 187 questions on average.
- DrQA's exact match falls as logical forms grow more complex: 55.3% for single-attribute, 32.2% for single-relation and 12.6% for multiple-relation templates.
- Generation scripts are released at github.com/panushri25/emrQA. The underlying i2b2 data requires a license agreement. The authors argue the same approach could build open-domain sets, for example from DBpedia plus Wikipedia.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

98 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 91 not yet.

- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://md.research.elamur.ai/free/en/papers/dasigi-2021-dataset-information-seeking-questions-answers/)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/)
- [English Machine Reading Comprehension Datasets: A Survey](https://md.research.elamur.ai/free/en/papers/dzendzik-2021-english-machine-reading-comprehension-datasets/)
- [Towards Mitigating LLM Hallucination via Self Reflection](https://md.research.elamur.ai/free/en/papers/ji-2023-mitigating-llm-hallucination-self-reflection/)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://md.research.elamur.ai/free/en/papers/jin-2019-pubmedqa-dataset-biomedical-research-question/)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://md.research.elamur.ai/free/en/papers/jin-2020-disease-this-patient-have-large/)
- [ExpertQA: Expert-Curated Questions and Attributed Answers](https://md.research.elamur.ai/free/en/papers/malaviya-2024-expertqa-expert-curated-questions-attributed/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pampari2018emrqa,
  title = {{emrQA: A Large Corpus for Question Answering on Electronic Medical Records}},
  author = {Pampari, Anusri and Raghavan, Preethi and Liang, Jennifer and Peng, Jian},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1258},
  url = {https://doi.org/10.18653/v1/d18-1258}
}
```

- [← previous Efficient Low-rank Multimodal Fusion With Modality-Specific Factors](https://md.research.elamur.ai/free/en/papers/liu-2018-efficient-low-rank-multimodal-fusion/)
- [next → End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification](https://md.research.elamur.ai/free/en/papers/libovicky-2018-end-end-non-autoregressive-neural/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/)

## References

58 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alan R Aronson. 2001. Effective mapping of biomedical text to the umls metathesaurus: the metamap program. In Proceedings of the AMIA Symposium, page 17. American Medical Informatics Association. [find](https://scholar.google.com/scholar?q=Alan+R+Aronson%2E+2001%2E+Effective+mapping+of+biomedical+text+to+the+umls+metathesaurus%3A+the+metamap+program%2E+In+Proceedings+of+the+AMIA+Symposium%2C+page+17%2E+American+Medical+Informatics+Association%2E)
2. Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2016. A simple but tough-to-beat baseline for sentence embeddings. [find](https://scholar.google.com/scholar?q=Sanjeev+Arora%2C+Yingyu+Liang%2C+and+Tengyu+Ma%2E+2016%2E+A+simple+but+tough%2Dto%2Dbeat+baseline+for+sentence+embeddings%2E)
3. Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In The semantic web, pages 722–735. Springer. [find](https://scholar.google.com/scholar?q=S%C3%B6ren+Auer%2C+Christian+Bizer%2C+Georgi+Kobilarov%2C+Jens+Lehmann%2C+Richard+Cyganiak%2C+and+Zachary+Ives%2E+2007%2E+Dbpedia%3A+A+nucleus+for+a+web+of+open+data%2E+In+The+semantic+web%2C+pages+722%E2%80%93735%2E+Springer%2E)
4. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+arXiv+preprint+arXiv%3A1409%2E0473%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
5. Jonathan Berant, Vivek Srikumar, Pei-Chun Chen, Abby Vander Linden, Brittany Harding, Brad Huang, Peter Clark, and Christopher D Manning. 2014. Modeling biological processes for reading comprehension. In EMNLP. [find](https://scholar.google.com/scholar?q=Jonathan+Berant%2C+Vivek+Srikumar%2C+Pei%2DChun+Chen%2C+Abby+Vander+Linden%2C+Brittany+Harding%2C+Brad+Huang%2C+Peter+Clark%2C+and+Christopher+D+Manning%2E+2014%2E+Modeling+biological+processes+for+reading+comprehension%2E) [our card](https://md.research.elamur.ai/free/en/papers/berant-2014-modeling-biological-processes-reading-comprehension/)
6. William Boag, Kevin Wacome, Tristan Naumann, and Anna Rumshisky. 2015. Cliner: A lightweight tool for clinical named entity recognition. AMIA Joint Summits on Clinical Research Informatics (poster). [find](https://scholar.google.com/scholar?q=William+Boag%2C+Kevin+Wacome%2C+Tristan+Naumann%2C+and+Anna+Rumshisky%2E+2015%2E+Cliner%3A+A+lightweight+tool+for+clinical+named+entity+recognition%2E+AMIA+Joint+Summits+on+Clinical+Research+Informatics+%28poster%29%2E)
7. Olivier Bodenreider. 2004. The unified medical language system (umls): integrating biomedical terminology. Nucleic acids research, 32(suppl_1):D267– D270. [find](https://scholar.google.com/scholar?q=Olivier+Bodenreider%2E+2004%2E+The+unified+medical+language+system+%28umls%29%3A+integrating+biomedical+terminology%2E+Nucleic+acids+research%2C+32%28suppl%5F1%29%3AD267%E2%80%93+D270%2E)
8. Antoine Bordes, Nicolas Usunier, Sumit Chopra, and Jason Weston. 2015. Large-scale simple question answering with memory networks. arXiv preprint arXiv:1506.02075. [find](https://scholar.google.com/scholar?q=Antoine+Bordes%2C+Nicolas+Usunier%2C+Sumit+Chopra%2C+and+Jason+Weston%2E+2015%2E+Large%2Dscale+simple+question+answering+with+memory+networks%2E+arXiv+preprint+arXiv%3A1506%2E02075%2E)
9. Qingqing Cai and Alexander Yates. 2013. Large-scale semantic parsing via schema matching and lexicon extension. In ACL (1), pages 423–433. [find](https://scholar.google.com/scholar?q=Qingqing+Cai+and+Alexander+Yates%2E+2013%2E+Large%2Dscale+semantic+parsing+via+schema+matching+and+lexicon+extension%2E+In+ACL+%281%29%2C+pages+423%E2%80%93433%2E)
10. Piotr Chabierski, Alessandra Russo, and Mark Law. 2017. Logic-based approach to machine comprehension of text. [find](https://scholar.google.com/scholar?q=Piotr+Chabierski%2C+Alessandra+Russo%2C+and+Mark+Law%2E+2017%2E+Logic%2Dbased+approach+to+machine+comprehension+of+text%2E)
11. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading wikipedia to answer open-domain questions. arXiv preprint arXiv:1704.00051. [find](https://scholar.google.com/scholar?q=Danqi+Chen%2C+Adam+Fisch%2C+Jason+Weston%2C+and+Antoine+Bordes%2E+2017%2E+Reading+wikipedia+to+answer+open%2Ddomain+questions%2E+arXiv+preprint+arXiv%3A1704%2E00051%2E) [our card](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/)
12. David L Chen and William B Dolan. 2011. Collecting highly parallel data for paraphrase evaluation. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 190–200. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=David+L+Chen+and+William+B+Dolan%2E+2011%2E+Collecting+highly+parallel+data+for+paraphrase+evaluation%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%3A+Human+Lan) [our card](https://md.research.elamur.ai/free/en/papers/chen-2011-collecting-highly-parallel-data-paraphrase/)
13. Joachim Daiber, Max Jakob, Chris Hokamp, and Pablo N. Mendes. 2013. Improving efficiency and accuracy in multilingual entity extraction. In Proceedings of the 9th International Conference on Semantic Systems (I-Semantics). [find](https://scholar.google.com/scholar?q=Joachim+Daiber%2C+Max+Jakob%2C+Chris+Hokamp%2C+and+Pablo+N%2E+Mendes%2E+2013%2E+Improving+efficiency+and+accuracy+in+multilingual+entity+extraction%2E+In+Proceedings+of+the+9th+International+Conference+on+Semantic+)
14. Randall Davis, Bruce Buchanan, and Edward Shortliffe. 1977. Production rules as a representation for a knowledge-based consultation program. Artificial intelligence, 8(1):15–45. [find](https://scholar.google.com/scholar?q=Randall+Davis%2C+Bruce+Buchanan%2C+and+Edward+Shortliffe%2E+1977%2E+Production+rules+as+a+representation+for+a+knowledge%2Dbased+consultation+program%2E+Artificial+intelligence%2C+8%281%29%3A15%E2%80%9345%2E)
15. Dina Demner-Fushman, Wendy Webber Chapman, and Clement J. McDonald. 2009. What can natural language processing do for clinical decision support? Journal of Biomedical Informatics, 42(5):760–772. [find](https://scholar.google.com/scholar?q=Dina+Demner%2DFushman%2C+Wendy+Webber+Chapman%2C+and+Clement+J%2E+McDonald%2E+2009%2E+What+can+natural+language+processing+do+for+clinical+decision+support%3F+Journal+of+Biomedical+Informatics%2C+42%285%29%3A760%E2%80%93772%2E)
16. Mary Devereaux. 2013. The use of patient records (ehr) for research. [find](https://scholar.google.com/scholar?q=Mary+Devereaux%2E+2013%2E+The+use+of+patient+records+%28ehr%29+for+research%2E)
17. Bhuwan Dhingra, Kathryn Mazaitis, and William W Cohen. 2017. Quasar: Datasets for question answering by search and reading. arXiv preprint arXiv:1707.03904. — [arXiv:1707.03904](https://arxiv.org/abs/1707.03904) [find](https://scholar.google.com/scholar?q=Bhuwan+Dhingra%2C+Kathryn+Mazaitis%2C+and+William+W+Cohen%2E+2017%2E+Quasar%3A+Datasets+for+question+answering+by+search+and+reading%2E+arXiv+preprint+arXiv%3A1707%2E03904%2E+%E2%80%94+%5BarXiv%3A1707%2E03904%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
18. Minwei Feng, Bing Xiang, Michael R Glass, Lidan Wang, and Bowen Zhou. 2015. Applying deep learning to answer selection: A study and an open task. In Automatic Speech Recognition and Understanding (ASRU), 2015 IEEE Workshop on, pages 813–820. IEEE. [find](https://scholar.google.com/scholar?q=Minwei+Feng%2C+Bing+Xiang%2C+Michael+R+Glass%2C+Lidan+Wang%2C+and+Bowen+Zhou%2E+2015%2E+Applying+deep+learning+to+answer+selection%3A+A+study+and+an+open+task%2E+In+Automatic+Speech+Recognition+and+Understanding+%28ASR)
19. Yansong Feng, Songfang Huang, Dongyan Zhao, et al. 2016. Hybrid question answering over knowledge base and free text. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2397–2407. [find](https://scholar.google.com/scholar?q=Yansong+Feng%2C+Songfang+Huang%2C+Dongyan+Zhao%2C+et+al%2E+2016%2E+Hybrid+question+answering+over+knowledge+base+and+free+text%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Conference+on+Computational+)
20. David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya A Kalyanpur, Adam Lally, J William Murdock, Eric Nyberg, John Prager, et al. 2010. Building watson: An overview of the deepqa project. AI magazine, 31(3):59–79. [find](https://scholar.google.com/scholar?q=David+Ferrucci%2C+Eric+Brown%2C+Jennifer+Chu%2DCarroll%2C+James+Fan%2C+David+Gondek%2C+Aditya+A+Kalyanpur%2C+Adam+Lally%2C+J+William+Murdock%2C+Eric+Nyberg%2C+John+Prager%2C+et+al%2E+2010%2E+Building+watson%3A+An+overview+of+the)
21. Jianfeng Gao, Rangan Majumder, and Bill Dolan. Machine reading for question answering: from symbolic to neural computation. [find](https://scholar.google.com/scholar?q=Jianfeng+Gao%2C+Rangan+Majumder%2C+and+Bill+Dolan%2E+Machine+reading+for+question+answering%3A+from+symbolic+to+neural+computation%2E)
22. Abbas Ghaddar and Phillippe Langlais. 2017. Winer: A wikipedia annotated corpus for named entity recognition. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 413–422. [find](https://scholar.google.com/scholar?q=Abbas+Ghaddar+and+Phillippe+Langlais%2E+2017%2E+Winer%3A+A+wikipedia+annotated+corpus+for+named+entity+recognition%2E+In+Proceedings+of+the+Eighth+International+Joint+Conference+on+Natural+Language+Processing)
23. Y. Guo, R. Gaizauskas, I. Roberts, G. Demetriou, and M. Hepple. 2006. Identifying personal health information using support vector machines. In i2b2 Workshop on Challenges in Natural Language Processing for Clinical Data, pages 10–11. [find](https://scholar.google.com/scholar?q=Y%2E+Guo%2C+R%2E+Gaizauskas%2C+I%2E+Roberts%2C+G%2E+Demetriou%2C+and+M%2E+Hepple%2E+2006%2E+Identifying+personal+health+information+using+support+vector+machines%2E+In+i2b2+Workshop+on+Challenges+in+Natural+Language+Processi)
24. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems, pages 1693–1701. [find](https://scholar.google.com/scholar?q=Karl+Moritz+Hermann%2C+Tomas+Kocisky%2C+Edward+Grefenstette%2C+Lasse+Espeholt%2C+Will+Kay%2C+Mustafa+Suleyman%2C+and+Phil+Blunsom%2E+2015%2E+Teaching+machines+to+read+and+comprehend%2E+In+Advances+in+Neural+Information) [our card](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/)
25. Robin Jia and Percy Liang. 2016. Data recombination for neural semantic parsing. arXiv preprint arXiv:1606.03622. [find](https://scholar.google.com/scholar?q=Robin+Jia+and+Percy+Liang%2E+2016%2E+Data+recombination+for+neural+semantic+parsing%2E+arXiv+preprint+arXiv%3A1606%2E03622%2E)
26. Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. Mimic-iii, a freely accessible critical care database. Scientific data, 3:160035. [find](https://scholar.google.com/scholar?q=Alistair+EW+Johnson%2C+Tom+J+Pollard%2C+Lu+Shen%2C+H+Lehman+Li%2Dwei%2C+Mengling+Feng%2C+Mohammad+Ghassemi%2C+Benjamin+Moody%2C+Peter+Szolovits%2C+Leo+Anthony+Celi%2C+and+Roger+G+Mark%2E+2016%2E+Mimic%2Diii%2C+a+freely+accessibl) [our card](https://md.research.elamur.ai/free/en/papers/johnson-2016-mimic-iii-freely-accessible-critical/)
27. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551. [find](https://scholar.google.com/scholar?q=Mandar+Joshi%2C+Eunsol+Choi%2C+Daniel+S+Weld%2C+and+Luke+Zettlemoyer%2E+2017%2E+Triviaqa%3A+A+large+scale+distantly+supervised+challenge+dataset+for+reading+comprehension%2E+arXiv+preprint+arXiv%3A1705%2E03551%2E) [our card](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/)
28. Harlan M Krumholz, Sharon F Terry, and Joanne Wald-streicher. 2016. Data acquisition, curation, and use for a continuously learning health system. Jama, 316(16):1669–1670. [find](https://scholar.google.com/scholar?q=Harlan+M+Krumholz%2C+Sharon+F+Terry%2C+and+Joanne+Wald%2Dstreicher%2E+2016%2E+Data+acquisition%2C+curation%2C+and+use+for+a+continuously+learning+health+system%2E+Jama%2C+316%2816%29%3A1669%E2%80%931670%2E)
29. Chonho Lee, Zhaojing Luo, Kee Yuan Ngiam, Meihui Zhang, Kaiping Zheng, Gang Chen, Beng Chin Ooi, and Wei Luen James Yip. 2017. Big healthcare data analytics: Challenges and applications. In Handbook of Large-Scale Distributed Computing in Smart Healthcare, pages 11–41. Springer. [find](https://scholar.google.com/scholar?q=Chonho+Lee%2C+Zhaojing+Luo%2C+Kee+Yuan+Ngiam%2C+Meihui+Zhang%2C+Kaiping+Zheng%2C+Gang+Chen%2C+Beng+Chin+Ooi%2C+and+Wei+Luen+James+Yip%2E+2017%2E+Big+healthcare+data+analytics%3A+Challenges+and+applications%2E+In+Handbook+o)
30. Vanessa Lopez, Christina Unger, Philipp Cimiano, and Enrico Motta. 2013. Evaluating question answering over linked data. Web Semantics: Science, Services and Agents on the World Wide Web, 21:3–13. [find](https://scholar.google.com/scholar?q=Vanessa+Lopez%2C+Christina+Unger%2C+Philipp+Cimiano%2C+and+Enrico+Motta%2E+2013%2E+Evaluating+question+answering+over+linked+data%2E+Web+Semantics%3A+Science%2C+Services+and+Agents+on+the+World+Wide+Web%2C+21%3A3%E2%80%9313%2E)
31. Minh-Thang Luong, Eugene Brevdo, and Rui Zhao. 2017. Neural machine translation (seq2seq) tutorial. [https://github.com/tensorflow/nmt](https://github.com/tensorflow/nmt). [↗](https://github.com/tensorflow/nmt)
32. George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39– 41. [find](https://scholar.google.com/scholar?q=George+A+Miller%2E+1995%2E+Wordnet%3A+a+lexical+database+for+english%2E+Communications+of+the+ACM%2C+38%2811%29%3A39%E2%80%93+41%2E) [our card](https://md.research.elamur.ai/free/en/papers/lex-1994-miller-wordnet/)
33. Suphakit Niwattanakul, Jatsada Singthongchai, Ekkachai Naenudorn, and Supachanun Wanapu. 2013. Using of jaccard coefficient for keywords similarity. In Proceedings of the International MultiConference of Engineers and Computer Scientists, volume 1. [find](https://scholar.google.com/scholar?q=Suphakit+Niwattanakul%2C+Jatsada+Singthongchai%2C+Ekkachai+Naenudorn%2C+and+Supachanun+Wanapu%2E+2013%2E+Using+of+jaccard+coefficient+for+keywords+similarity%2E+In+Proceedings+of+the+International+MultiConference)
34. Joel Nothman, James R Curran, and Tara Murphy. 2008. Transforming wikipedia into named entity training data. In Proceedings of the Australasian Language Technology Association Workshop 2008, pages 124–132. [find](https://scholar.google.com/scholar?q=Joel+Nothman%2C+James+R+Curran%2C+and+Tara+Murphy%2E+2008%2E+Transforming+wikipedia+into+named+entity+training+data%2E+In+Proceedings+of+the+Australasian+Language+Technology+Association+Workshop+2008%2C+pages+124)
35. Hamid Palangi, Paul Smolensky, Xiaodong He, and Li Deng. 2018. Question-answering with grammatically-interpretable representations. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, LA. [find](https://scholar.google.com/scholar?q=Hamid+Palangi%2C+Paul+Smolensky%2C+Xiaodong+He%2C+and+Li+Deng%2E+2018%2E+Question%2Danswering+with+grammatically%2Dinterpretable+representations%2E+In+Proceedings+of+the+32nd+AAAI+Conference+on+Artificial+Intelligenc)
36. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+annual+meeting+on+association+for+computat) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
37. Preethi Raghavan and Siddharth Patwardhan. 2016. Question answering on electronic medical records. In Proceedings of the 2016 Summit on Clinical Research Informatics, San Francisco, CA, March 2016. [find](https://scholar.google.com/scholar?q=Preethi+Raghavan+and+Siddharth+Patwardhan%2E+2016%2E+Question+answering+on+electronic+medical+records%2E+In+Proceedings+of+the+2016+Summit+on+Clinical+Research+Informatics%2C+San+Francisco%2C+CA%2C+March+2016%2E)
38. Preethi Raghavan, Siddharth Patwardhan, Jennifer J. Liang, and Murthy V. Devarakonda. 2017. Annotating electronic medical records for question answering. arXiv:1805.06816. — [arXiv:1805.06816](https://arxiv.org/abs/1805.06816) [find](https://scholar.google.com/scholar?q=Preethi+Raghavan%2C+Siddharth+Patwardhan%2C+Jennifer+J%2E+Liang%2C+and+Murthy+V%2E+Devarakonda%2E+2017%2E+Annotating+electronic+medical+records+for+question+answering%2E+arXiv%3A1805%2E06816%2E+%E2%80%94+%5BarXiv%3A1805%2E06816%5D%28https%3A%2F)
39. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250. [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Jian+Zhang%2C+Konstantin+Lopyrev%2C+and+Percy+Liang%2E+2016%2E+Squad%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E+arXiv+preprint+arXiv%3A1606%2E05250%2E) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
40. Matthew Richardson, Christopher JC Burges, and Erin Renshaw. 2013. Mctest: A challenge dataset for the open-domain machine comprehension of text. In EMNLP, volume 3, page 4. [find](https://scholar.google.com/scholar?q=Matthew+Richardson%2C+Christopher+JC+Burges%2C+and+Erin+Renshaw%2E+2013%2E+Mctest%3A+A+challenge+dataset+for+the+open%2Ddomain+machine+comprehension+of+text%2E+In+EMNLP%2C+volume+3%2C+page+4%2E) [our card](https://md.research.elamur.ai/free/en/papers/missing-2013-richardson-mctest-challenge-dataset-open-domain/)
41. Kirk Roberts and Dina Demner-Fushman. 2016. Annotating logical forms for ehr questions. In LREC... International Conference on Language Resources & Evaluation:[proceedings]. International Conference on Language Resources and Evaluation, volume 2016, page 3772. NIH Public Access. [find](https://scholar.google.com/scholar?q=Kirk+Roberts+and+Dina+Demner%2DFushman%2E+2016%2E+Annotating+logical+forms+for+ehr+questions%2E+In+LREC%2E%2E%2E+International+Conference+on+Language+Resources+%26+Evaluation%3A%5Bproceedings%5D%2E+International+Conference+o)
42. Benjamin Shickel, Patrick James Tighe, Azra Bihorac, and Parisa Rashidi. 2017. Deep ehr: A survey of recent advances in deep learning techniques for electronic health record (ehr) analysis. IEEE Journal of Biomedical and Health Informatics. [find](https://scholar.google.com/scholar?q=Benjamin+Shickel%2C+Patrick+James+Tighe%2C+Azra+Bihorac%2C+and+Parisa+Rashidi%2E+2017%2E+Deep+ehr%3A+A+survey+of+recent+advances+in+deep+learning+techniques+for+electronic+health+record+%28ehr%29+analysis%2E+IEEE+Journ)
43. Amber Stubbs and Özlem Uzuner. 2015. Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/uthealth corpus. Journal of biomedical informatics, 58:S20–S29. [find](https://scholar.google.com/scholar?q=Amber+Stubbs+and+%C3%96zlem+Uzuner%2E+2015%2E+Annotating+longitudinal+clinical+narratives+for+de%2Didentification%3A+The+2014+i2b2%2Futhealth+corpus%2E+Journal+of+biomedical+informatics%2C+58%3AS20%E2%80%93S29%2E)
44. Yu Su, Huan Sun, Brian Sadler, Mudhakar Srivatsa, Izzeddin Gur, Zenghui Yan, and Xifeng Yan. 2016. On generating characteristic-rich question sets for qa evaluation. In EMNLP, pages 562–572. [find](https://scholar.google.com/scholar?q=Yu+Su%2C+Huan+Sun%2C+Brian+Sadler%2C+Mudhakar+Srivatsa%2C+Izzeddin+Gur%2C+Zenghui+Yan%2C+and+Xifeng+Yan%2E+2016%2E+On+generating+characteristic%2Drich+question+sets+for+qa+evaluation%2E+In+EMNLP%2C+pages+562%E2%80%93572%2E)
45. Simon Šuster, Stéphan Tulkens, and Walter Daelemans. 2017. A short review of ethical challenges in clinical natural language processing. arXiv preprint arXiv:1703.10090. [find](https://scholar.google.com/scholar?q=Simon+%C5%A0uster%2C+St%C3%A9phan+Tulkens%2C+and+Walter+Daelemans%2E+2017%2E+A+short+review+of+ethical+challenges+in+clinical+natural+language+processing%2E+arXiv+preprint+arXiv%3A1703%2E10090%2E)
46. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+neural+information+processing+systems%2C+pages+3104%E2%80%933112%2E)
47. Paul C Tang, Danielle Fafchamps, and Edward H Shortliffe. 1994. Traditional medical records as a source of clinical data in the outpatient setting. In Proceedings of the Annual Symposium on Computer Application in Medical Care, page 575. American Medical Informatics Association. [find](https://scholar.google.com/scholar?q=Paul+C+Tang%2C+Danielle+Fafchamps%2C+and+Edward+H+Shortliffe%2E+1994%2E+Traditional+medical+records+as+a+source+of+clinical+data+in+the+outpatient+setting%2E+In+Proceedings+of+the+Annual+Symposium+on+Computer+A)
48. Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2016. Newsqa: A machine comprehension dataset. arXiv preprint arXiv:1611.09830. [find](https://scholar.google.com/scholar?q=Adam+Trischler%2C+Tong+Wang%2C+Xingdi+Yuan%2C+Justin+Harris%2C+Alessandro+Sordoni%2C+Philip+Bachman%2C+and+Kaheer+Suleman%2E+2016%2E+Newsqa%3A+A+machine+comprehension+dataset%2E+arXiv+preprint+arXiv%3A1611%2E09830%2E) [our card](https://md.research.elamur.ai/free/en/papers/trischler-2017-newsqa-machine-comprehension-dataset/)
49. George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, et al. 2015. An overview of the bioasq large-scale biomedical semantic indexing and question answeri… [find](https://scholar.google.com/scholar?q=George+Tsatsaronis%2C+Georgios+Balikas%2C+Prodromos+Malakasiotis%2C+Ioannis+Partalas%2C+Matthias+Zschunke%2C+Michael+R+Alvers%2C+Dirk+Weissenborn%2C+Anastasia+Krithara%2C+Sergios+Petridis%2C+Dimitris+Polychronopoulos%2C+)
50. Ricardo Usbeck, Axel-Cyrille Ngonga Ngomo, Lorenz Bühmann, and Christina Unger. 2015. Hawk– hybrid question answering using linked data. In European Semantic Web Conference, pages 353– 368. Springer. [find](https://scholar.google.com/scholar?q=Ricardo+Usbeck%2C+Axel%2DCyrille+Ngonga+Ngomo%2C+Lorenz+B%C3%BChmann%2C+and+Christina+Unger%2E+2015%2E+Hawk%E2%80%93+hybrid+question+answering+using+linked+data%2E+In+European+Semantic+Web+Conference%2C+pages+353%E2%80%93+368%2E+Springer%2E)
51. Özlem Uzuner. 2009. Recognizing obesity and comorbidities in sparse data. Journal of the American Medical Informatics Association, 16(4):561–570. [find](https://scholar.google.com/scholar?q=%C3%96zlem+Uzuner%2E+2009%2E+Recognizing+obesity+and+comorbidities+in+sparse+data%2E+Journal+of+the+American+Medical+Informatics+Association%2C+16%284%29%3A561%E2%80%93570%2E)
52. Ozlem Uzuner, Andreea Bodnari, Shuying Shen, Tyler Forbush, John Pestian, and Brett R South. 2012. Evaluating the state of the art in coreference resolution for electronic medical records. Journal of the American Medical Informatics Association, 19(5):786–791. [find](https://scholar.google.com/scholar?q=Ozlem+Uzuner%2C+Andreea+Bodnari%2C+Shuying+Shen%2C+Tyler+Forbush%2C+John+Pestian%2C+and+Brett+R+South%2E+2012%2E+Evaluating+the+state+of+the+art+in+coreference+resolution+for+electronic+medical+records%2E+Journal+of+)
53. Özlem Uzuner, Ira Goldstein, Yuan Luo, and Isaac Kohane. 2008. Identifying patient smoking status from medical discharge records. Journal of the American Medical Informatics Association, 15(1):14–24. [find](https://scholar.google.com/scholar?q=%C3%96zlem+Uzuner%2C+Ira+Goldstein%2C+Yuan+Luo%2C+and+Isaac+Kohane%2E+2008%2E+Identifying+patient+smoking+status+from+medical+discharge+records%2E+Journal+of+the+American+Medical+Informatics+Association%2C+15%281%29%3A14%E2%80%9324%2E)
54. Özlem Uzuner, Imre Solti, and Eithon Cadag. 2010a. Extracting medication information from clinical text. Journal of the American Medical Informatics Association, 17(5):514–518. [find](https://scholar.google.com/scholar?q=%C3%96zlem+Uzuner%2C+Imre+Solti%2C+and+Eithon+Cadag%2E+2010a%2E+Extracting+medication+information+from+clinical+text%2E+Journal+of+the+American+Medical+Informatics+Association%2C+17%285%29%3A514%E2%80%93518%2E)
55. Özlem Uzuner, Imre Solti, Fei Xia, and Eithon Cadag. 2010b. Community annotation experiment for ground truth generation for the i2b2 medication challenge. Journal of the American Medical Informatics Association, 17(5):519–523. [find](https://scholar.google.com/scholar?q=%C3%96zlem+Uzuner%2C+Imre+Solti%2C+Fei+Xia%2C+and+Eithon+Cadag%2E+2010b%2E+Community+annotation+experiment+for+ground+truth+generation+for+the+i2b2+medication+challenge%2E+Journal+of+the+American+Medical+Informatics+A)
56. Özlem Uzuner, Brett R South, Shuying Shen, and Scott L DuVall. 2011. 2010 i2b2/va challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association, 18(5):552–556. [find](https://scholar.google.com/scholar?q=%C3%96zlem+Uzuner%2C+Brett+R+South%2C+Shuying+Shen%2C+and+Scott+L+DuVall%2E+2011%2E+2010+i2b2%2Fva+challenge+on+concepts%2C+assertions%2C+and+relations+in+clinical+text%2E+Journal+of+the+American+Medical+Informatics+Associa)
57. Alfredo Vellido, José David Martín-Guerrero, and Paulo JG Lisboa. 2012. Making machine learning models interpretable. In ESANN, volume 12, pages 163–172. Citeseer. [find](https://scholar.google.com/scholar?q=Alfredo+Vellido%2C+Jos%C3%A9+David+Mart%C3%ADn%2DGuerrero%2C+and+Paulo+JG+Lisboa%2E+2012%2E+Making+machine+learning+models+interpretable%2E+In+ESANN%2C+volume+12%2C+pages+163%E2%80%93172%2E+Citeseer%2E)
58. Ellen M Voorhees et al. 1999. The trec-8 question answering track report. In Trec, volume 99, pages 77– 82. [find](https://scholar.google.com/scholar?q=Ellen+M+Voorhees+et+al%2E+1999%2E+The+trec%2D8+question+answering+track+report%2E+In+Trec%2C+volume+99%2C+pages+77%E2%80%93+82%2E) [our card](https://md.research.elamur.ai/free/en/papers/voorhees-2000-trec-8-question-answering-track/)