---
title: "Lessons from Natural Language Inference in — Romanov 2018"
description: "Neural NLI models perform well on open-domain benchmarks such as SNLI, but they generalize poorly to specialized fields where labeled data is scarce and…"
authors: ["Alexey Romanov", "Chaitanya Shivade"]
year: 2018
venue: "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D18-1187/"
doi: "10.18653/v1/d18-1187"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/romanov-2018-lessons-natural-language-inference-clinical/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/romanov-2018-lessons-natural-language-inference-clinical/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / Lessons from Natural Language Inference in the Clinical Domain

# Lessons from Natural Language Inference in the Clinical Domain

**Authors:** [Alexey Romanov](https://md.research.elamur.ai/free/en/authors/alexey-romanov/), Chaitanya Shivade · **Year:** 2018 · **Venue:** Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D18-1187/) · [PDF](https://aclanthology.org/D18-1187.pdf) · [DOI 10.18653/v1/d18-1187](https://doi.org/10.18653/v1/d18-1187) · [OpenAlex](https://openalex.org/W2888120268)

**Citations:** 212 citations (OpenAlex, 2026-10-14) · [131](https://md.research.elamur.ai/free/en/papers/romanov-2018-lessons-natural-language-inference-clinical/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Neural NLI models perform well on open-domain benchmarks such as SNLI, but they generalize poorly to specialized fields where labeled data is scarce and reasoning depends on expert knowledge. Romanov and Shivade address this for medicine by building MedNLI. Its premises are sentences drawn from the Past Medical History sections of MIMIC-III clinical notes. Following the SNLI protocol, clinicians wrote an entailed, a neutral and a contradicting hypothesis for each premise.

The authors benchmark several systems on the new dataset:
* a gradient-boosting model with hand-crafted and UMLS-based features,
* BOW,
* InferSent,
* ESIM.

They then test two ways to improve these baselines. The first is transfer learning from open-domain corpora. The second is injecting domain knowledge, either through embeddings trained on clinical text or through the SNOMED-CT/UMLS ontology (retrofitting and a proposed knowledge-directed attention). Both strategies yield gains, with domain-specific embeddings giving the clearest improvement. Ontology integration helps only modestly.

The work matters because it supplies the first expert-annotated clinical NLI benchmark with public baselines. It also shows that medical inference typically requires multi-step knowledge that open-domain models lack.

## Key points
- **MedNLI size and split:** four clinicians annotated 4,683 premises over six weeks, producing 14,049 sentence pairs. These are split 11,232 / 1,395 / 1,422 (train/dev/test), with no premise shared across splits.
- **Annotation quality:** a pilot of 552 pairs between two radiologists reached Cohen's κ = 0.78. Most disagreements involved "definitely true" versus "maybe true." The final data is single-annotated and distributed via the MIMIC-III derived data repository.
- **Ontology methods:** retrofitting embeddings with ontology relations lowered accuracy by about 1.7–2.7 points. Knowledge-directed attention gave small gains of up to 0.3 points, and about 60% of those gains came from the neutral class.
- **Path lengths:** shortest UMLS paths between premise and hypothesis concepts are often long, unlike WordNet paths in SNLI. This points to multi-step reasoning in MedNLI.
- **Errors and artifacts:** neutral is the hardest class. Typical errors involve numerical lab values, abbreviations, negation, and world or medical knowledge. A hypothesis-only classifier still reaches 61.9 F1, compared with 67.0 on SNLI and 53.9 on MultiNLI, which reveals annotation artifacts in MedNLI.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{romanov2018lessons,
  title = {{Lessons from Natural Language Inference in the Clinical Domain}},
  author = {Romanov, Alexey and Shivade, Chaitanya},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1187},
  url = {https://doi.org/10.18653/v1/d18-1187}
}
```

- [← previous Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://md.research.elamur.ai/free/en/papers/li-2018-delete-retrieve-generate-simple-approach/)
- [next → A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss](https://md.research.elamur.ai/free/en/papers/hsu-2018-unified-model-extractive-abstractive-summarization/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/)

## References

39 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Yossi+Adi%2C+Einat+Kermany%2C+Yonatan+Belinkov%2C+Ofer+Lavi%2C+and+Yoav+Goldberg%2E+2017%2E+Fine%2Dgrained+analysis+of+sentence+embeddings+using+auxiliary+prediction+tasks%2E+In+Proceedings+of+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/adi-2016-fine-grained-analysis-sentence-embeddings/)
2. Alan R Aronson and François-Michel Lang. 2010. An overview of metamap: historical perspective and recent advances. Journal of the American Medical Informatics Association, 17(3):229–236. [find](https://scholar.google.com/scholar?q=Alan+R+Aronson+and+Franc%CC%A7ois%2DMichel+Lang%2E+2010%2E+An+overview+of+metamap%3A+historical+perspective+and+recent+advances%2E+Journal+of+the+American+Medical+Informatics+Association%2C+17%283%29%3A229%E2%80%93236%2E)
3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+Proceedings+of+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
4. Olivier Bodenreider. 2004. The Unified Medical Language System (UMLS): Integrating biomedical terminology. Nucleic Acids Research, 32(suppl 1):D267–D270. [find](https://scholar.google.com/scholar?q=Olivier+Bodenreider%2E+2004%2E+The+Unified+Medical+Language+System+%28UMLS%29%3A+Integrating+biomedical+terminology%2E+Nucleic+Acids+Research%2C+32%28suppl+1%29%3AD267%E2%80%93D270%2E)
5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5. [find](https://scholar.google.com/scholar?q=Piotr+Bojanowski%2C+Edouard+Grave%2C+Armand+Joulin%2C+and+Tomas+Mikolov%2E+2017%2E+Enriching+word+vectors+with+subword+information%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+5%2E)
6. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+In+Proceedings+of+EMNLP%2E)
7. Anita Burgun and Olivier Bodenreider. 2001. Comparing terms, concepts and semantic classes in Word-Net and the Unified Medical Language System. In Proceedings of the NAACL Workshop: WordNet and other lexical resources: Applications, extensions and customizations., pages 77–82. [find](https://scholar.google.com/scholar?q=Anita+Burgun+and+Olivier+Bodenreider%2E+2001%2E+Comparing+terms%2C+concepts+and+semantic+classes+in+Word%2DNet+and+the+Unified+Medical+Language+System%2E+In+Proceedings+of+the+NAACL+Workshop%3A+WordNet+and+other+)
8. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced lstm for natural language inference. In Procedings of ACL. [find](https://scholar.google.com/scholar?q=Qian+Chen%2C+Xiaodan+Zhu%2C+Zhen%2DHua+Ling%2C+Si+Wei%2C+Hui+Jiang%2C+and+Diana+Inkpen%2E+2017%2E+Enhanced+lstm+for+natural+language+inference%2E+In+Procedings+of+ACL%2E)
9. Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+Van+Merrie%CC%88nboer%2C+Dzmitry+Bahdanau%2C+and+Yoshua+Bengio%2E+2014%2E+On+the+properties+of+neural+machine+translation%3A+Encoder%2Ddecoder+approaches%2E+In+Proceedings+of+SSST%2D8%2C+Eighth+Workshop+) [our card](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/)
10. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Douwe+Kiela%2C+Holger+Schwenk%2C+Loic+Barrault%2C+and+Antoine+Bordes%2E+2017%2E+Supervised+learning+of+universal+sentence+representations+from+natural+language+inference+data%2E+In+Proceedings+of+) [our card](https://md.research.elamur.ai/free/en/papers/conneau-2017-supervised-learning-universal-sentence-representations/)
11. Ido Dagan, Bill Dolan, Bernardo Magnini, and Dan Roth. 2009. Recognizing textual entailment: Rational, evaluation and approaches. Natural Language Engineering, 15(4):i–xvii. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Bill+Dolan%2C+Bernardo+Magnini%2C+and+Dan+Roth%2E+2009%2E+Recognizing+textual+entailment%3A+Rational%2C+evaluation+and+approaches%2E+Natural+Language+Engineering%2C+15%284%29%3Ai%E2%80%93xvii%2E)
12. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The pascal recognising textual entailment challenge. In Machine learning challenges. evaluating predictive uncertainty, visual object classification, and recognising tectual entailment, pages 177– 190. Springer. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2006%2E+The+pascal+recognising+textual+entailment+challenge%2E+In+Machine+learning+challenges%2E+evaluating+predictive+uncertainty%2C+visual+object+classificati)
13. Manaal Faruqui, Jesse Dodge, Sujay K. Jauhar, Chris Dyer, Eduard Hovy, and Noah A. Smith. 2015. Retrofitting word vectors to semantic lexicons. In Proceedings of NAACL. [find](https://scholar.google.com/scholar?q=Manaal+Faruqui%2C+Jesse+Dodge%2C+Sujay+K%2E+Jauhar%2C+Chris+Dyer%2C+Eduard+Hovy%2C+and+Noah+A%2E+Smith%2E+2015%2E+Retrofitting+word+vectors+to+semantic+lexicons%2E+In+Proceedings+of+NAACL%2E)
14. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. Bradford Books. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2E+1998%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+Bradford+Books%2E)
15. Carol Friedman, Pauline Kra, and Andrey Rzhetsky. 2002. Two biomedical sublanguages: a description based on the theories of Zellig Harris. Journal of Biomedical Informatics, 35(4):222–235. [find](https://scholar.google.com/scholar?q=Carol+Friedman%2C+Pauline+Kra%2C+and+Andrey+Rzhetsky%2E+2002%2E+Two+biomedical+sublanguages%3A+a+description+based+on+the+theories+of+Zellig+Harris%2E+Journal+of+Biomedical+Informatics%2C+35%284%29%3A222%E2%80%93235%2E)
16. Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Deep sparse rectifier neural networks. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 315–323. [find](https://scholar.google.com/scholar?q=Xavier+Glorot%2C+Antoine+Bordes%2C+and+Yoshua+Bengio%2E+2011%2E+Deep+sparse+rectifier+neural+networks%2E+In+Proceedings+of+the+International+Conference+on+Artificial+Intelligence+and+Statistics%2C+pages+315%E2%80%93323%2E)
17. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R Bowman, and Noah A Smith. 2018. Annotation artifacts in natural language inference data. Proceedings of NAACL. [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Swabha+Swayamdipta%2C+Omer+Levy%2C+Roy+Schwartz%2C+Samuel+R+Bowman%2C+and+Noah+A+Smith%2E+2018%2E+Annotation+artifacts+in+natural+language+inference+data%2E+Proceedings+of+NAACL%2E)
18. Paul KJ Han, William MP Klein, and Neeraj K Arora. 2011. Varieties of uncertainty in health care: a conceptual taxonomy. Medical Decision Making, 31(6):828–838. [find](https://scholar.google.com/scholar?q=Paul+KJ+Han%2C+William+MP+Klein%2C+and+Neeraj+K+Arora%2E+2011%2E+Varieties+of+uncertainty+in+health+care%3A+a+conceptual+taxonomy%2E+Medical+Decision+Making%2C+31%286%29%3A828%E2%80%93838%2E)
19. Alistair EW Johnson, Tom J Pollard, Lu Shen, Liwei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific Data, 3. [find](https://scholar.google.com/scholar?q=Alistair+EW+Johnson%2C+Tom+J+Pollard%2C+Lu+Shen%2C+Liwei+H+Lehman%2C+Mengling+Feng%2C+Mohammad+Ghassemi%2C+Benjamin+Moody%2C+Peter+Szolovits%2C+Leo+Anthony+Celi%2C+and+Roger+G+Mark%2E+2016%2E+MIMIC%2DIII%2C+a+freely+accessible) [our card](https://md.research.elamur.ai/free/en/papers/johnson-2016-mimic-iii-freely-accessible-critical/)
20. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Diederik+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+Proceedings+of+ICLR%2E)
21. Ji Young Lee, Franck Dernoncourt, and Peter Szolovits. 2018. Transfer learning for named-entity recognition with neural networks. Proceedings of LREC. [find](https://scholar.google.com/scholar?q=Ji+Young+Lee%2C+Franck+Dernoncourt%2C+and+Peter+Szolovits%2E+2018%2E+Transfer+learning+for+named%2Dentity+recognition+with+neural+networks%2E+Proceedings+of+LREC%2E)
22. Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer, et al. 2015. Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web, 6(2):167–195. [find](https://scholar.google.com/scholar?q=Jens+Lehmann%2C+Robert+Isele%2C+Max+Jakob%2C+Anja+Jentzsch%2C+Dimitris+Kontokostas%2C+Pablo+N+Mendes%2C+Sebastian+Hellmann%2C+Mohamed+Morsey%2C+Patrick+Van+Kleef%2C+So%CC%88ren+Auer%2C+et+al%2E+2015%2E+Dbpedia%E2%80%93a+large%2Dscale%2C+mult)
23. Lingpipe. 2008. LingPipe 4.1.0. [find](https://scholar.google.com/scholar?q=Lingpipe%2E+2008%2E+LingPipe+4%2E1%2E0%2E)
24. Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In Proceedings of NIPS. [find](https://scholar.google.com/scholar?q=Bryan+McCann%2C+James+Bradbury%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+2017%2E+Learned+in+translation%3A+Contextualized+word+vectors%2E+In+Proceedings+of+NIPS%2E) [our card](https://md.research.elamur.ai/free/en/papers/mccann-2017-learned-translation-contextualized-word-vectors/)
25. Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica, 22(3):276–282. [find](https://scholar.google.com/scholar?q=Mary+L+McHugh%2E+2012%2E+Interrater+reliability%3A+the+kappa+statistic%2E+Biochemia+medica%2C+22%283%29%3A276%E2%80%93282%2E)
26. Stephane M. Meystre, Guergana K. Savova, Karin C. Kipper-Schuler, and John F. Hurdle. 2008. Extracting information from textual documents in the electronic health record: a review of recent research. Yearbook of Medical Informatics, 35:128–44. [find](https://scholar.google.com/scholar?q=Stephane+M%2E+Meystre%2C+Guergana+K%2E+Savova%2C+Karin+C%2E+Kipper%2DSchuler%2C+and+John+F%2E+Hurdle%2E+2008%2E+Extracting+information+from+textual+documents+in+the+electronic+health+record%3A+a+review+of+recent+research%2E+)
27. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111–3119. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+S+Corrado%2C+and+Jeff+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Advances+in+Neural+Information+Processing+)
28. Ankur P Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Ankur+P+Parikh%2C+Oscar+Ta%CC%88ckstro%CC%88m%2C+Dipanjan+Das%2C+and+Jakob+Uszkoreit%2E+2016%2E+A+decomposable+attention+model+for+natural+language+inference%2E+In+Proceedings+of+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/parikh-2016-decomposable-attention-model-natural-language/)
29. Adina Williams, Nikita Nangia, and Samuel R Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of NAACL. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+R+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+NAACL%2E)
30. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Barret+Zoph%2C+Deniz+Yuret%2C+Jonathan+May%2C+and+Kevin+Knight%2E+2016%2E+Transfer+learning+for+low%2Dresource+neural+machine+translation%2E+In+Proceedings+of+EMNLP%2E)
31. Ted Pedersen, Serguei VS Pakhomov, Siddharth Patwardhan, and Christopher G Chute. 2007. Measures of semantic similarity and relatedness in the biomedical domain. Journal of Biomedical Informatics, 40(3):288–299. [find](https://scholar.google.com/scholar?q=Ted+Pedersen%2C+Serguei+VS+Pakhomov%2C+Siddharth+Patwardhan%2C+and+Christopher+G+Chute%2E+2007%2E+Measures+of+semantic+similarity+and+relatedness+in+the+biomedical+domain%2E+Journal+of+Biomedical+Informatics%2C+40%28)
32. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D%2E+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Proceedings+of+EMNLP%2E)
33. Pranav Rajpurkar, Jeremy Irvin, Kaylie Zhu, Brandon Yang, Hershel Mehta, Tony Duan, Daisy Ding, Aarti Bagul, Curtis Langlotz, Katie Shpanskaya, et al. 2017. Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225. — [arXiv:1711.05225](https://arxiv.org/abs/1711.05225) [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Jeremy+Irvin%2C+Kaylie+Zhu%2C+Brandon+Yang%2C+Hershel+Mehta%2C+Tony+Duan%2C+Daisy+Ding%2C+Aarti+Bagul%2C+Curtis+Langlotz%2C+Katie+Shpanskaya%2C+et+al%2E+2017%2E+Chexnet%3A+Radiologist%2Dlevel+pneumonia+detect)
34. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiskỳ, and Phil Blunsom. 2016. Reasoning about entailment with neural attention. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Tim+Rockta%CC%88schel%2C+Edward+Grefenstette%2C+Karl+Moritz+Hermann%2C+Toma%CC%81s%CC%8C+Koc%CC%8Cisky%CC%80%2C+and+Phil+Blunsom%2E+2016%2E+Reasoning+about+entailment+with+neural+attention%2E+In+Proceedings+of+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/rocktaschel-2015-reasoning-about-entailment-neural-attention/)
35. Sunil Kumar Sahu and Ashish Anand. 2017. What matters in a transferable neural network model for relation classification in the biomedical domain? arXiv preprint arXiv:1708.03446. — [arXiv:1708.03446](https://arxiv.org/abs/1708.03446) [find](https://scholar.google.com/scholar?q=Sunil+Kumar+Sahu+and+Ashish+Anand%2E+2017%2E+What+matters+in+a+transferable+neural+network+model+for+relation+classification+in+the+biomedical+domain%3F+arXiv+preprint+arXiv%3A1708%2E03446%2E+%E2%80%94+%5BarXiv%3A1708%2E03446%5D)
36. Chaitanya Shivade, Courtney Hebert, Marcelo Lopetegui, Marie-Catherine De Marneffe, Eric Fosler-Lussier, and Albert M Lai. 2015. Textual inference for eligibility criteria resolution in clinical trials. Journal of Biomedical Informatics, 58:S211–S218. [find](https://scholar.google.com/scholar?q=Chaitanya+Shivade%2C+Courtney+Hebert%2C+Marcelo+Lopetegui%2C+Marie%2DCatherine+De+Marneffe%2C+Eric+Fosler%2DLussier%2C+and+Albert+M+Lai%2E+2015%2E+Textual+inference+for+eligibility+criteria+resolution+in+clinical+trial)
37. Gabriel Stanovsky, Daniel Gruhl, and Pablo Mendes. 2017. Recognizing mentions of adverse drug reaction in social media using knowledge-infused recurrent models. In Proceedings of EACL. [find](https://scholar.google.com/scholar?q=Gabriel+Stanovsky%2C+Daniel+Gruhl%2C+and+Pablo+Mendes%2E+2017%2E+Recognizing+mentions+of+adverse+drug+reaction+in+social+media+using+knowledge%2Dinfused+recurrent+models%2E+In+Proceedings+of+EACL%2E)
38. George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, et al. 2015. An overview of the bioasq large-scale biomedical semantic indexing and question answeri… [find](https://scholar.google.com/scholar?q=George+Tsatsaronis%2C+Georgios+Balikas%2C+Prodromos+Malakasiotis%2C+Ioannis+Partalas%2C+Matthias+Zschunke%2C+Michael+R+Alvers%2C+Dirk+Weissenborn%2C+Anastasia+Krithara%2C+Sergios+Petridis%2C+Dimitris+Polychronopoulos%2C+)
39. Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Advances in Neural Information Processing Systems, pages 2773–2781. [find](https://scholar.google.com/scholar?q=Oriol+Vinyals%2C+%C5%81ukasz+Kaiser%2C+Terry+Koo%2C+Slav+Petrov%2C+Ilya+Sutskever%2C+and+Geoffrey+Hinton%2E+2015%2E+Grammar+as+a+foreign+language%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+2773%E2%80%932781%2E)