---
title: "Modeling Biological Processes for Reading — Berant 2014"
description: "Most machine reading research harvests facts from huge, redundant web collections."
authors: ["Jonathan Berant", "Vivek Srikumar", "Pei-Chun Chen", "Abby Vander Linden", "Brittany Harding", "Brad Huang", "Peter Clark", "Christopher D. Manning"]
year: 2014
venue: "Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)"
source: "https://aclanthology.org/D14-1159/"
doi: "10.3115/v1/d14-1159"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/berant-2014-modeling-biological-processes-reading-comprehension/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/berant-2014-modeling-biological-processes-reading-comprehension/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/) / Modeling Biological Processes for Reading Comprehension

# Modeling Biological Processes for Reading Comprehension

**Authors:** [Jonathan Berant](https://md.research.elamur.ai/free/en/authors/jonathan-berant/), [Vivek Srikumar](https://md.research.elamur.ai/free/en/authors/vivek-srikumar/), Pei-Chun Chen, Abby Vander Linden, Brittany Harding, Brad Huang, [Peter Clark](https://md.research.elamur.ai/free/en/authors/peter-clark/), [Christopher D. Manning](https://md.research.elamur.ai/free/en/authors/christopher-d-manning/) · **Year:** 2014 · **Venue:** Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/D14-1159/) · [PDF](https://aclanthology.org/D14-1159.pdf) · [DOI 10.3115/v1/d14-1159](https://doi.org/10.3115/v1/d14-1159) · [OpenAlex](https://openalex.org/W2252016937)

**Citations:** 195 citations (openalex-snapshot · 2026-10-14) · [93](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Most machine reading research harvests facts from huge, redundant web collections. This paper instead targets micro-reading: answering questions about one paragraph, where the answer depends on how events and entities in a described process relate to each other. The authors build ProcessBank, a set of 200 biology textbook paragraphs. Each paragraph is annotated with a process graph and paired with two-option multiple-choice questions written by a biologist. Their system works in two stages. First, a supervised predictor extracts event triggers, arguments, semantic roles and event-event relations across sentences, using structural constraints during inference. Second, a rule-based procedure converts each question plus candidate answer into a graph query and runs it over the predicted structure, which acts as a tiny knowledge base. Reasoning over these predicted structures answers questions substantially more accurately than baselines built on shallower signals such as text or syntactic proximity. This shows that explicit structure helps on comprehension questions that lexical matching cannot solve.

## Key points
- **ProcessBank data:**
  - 200 paragraphs from Campbell and Reece's *Biology*, extending the 148 identified by Scaria et al. (2013).
  - 585 questions in total.
  - A second annotator checked 326 of them and agreed on the answer in 98.1% of cases.
- **Process graph:**
  - Triggers are marked T-YES or T-NO, depending on whether the event happens.
  - Arguments receive roles: AGENT, THEME, SOURCE, DESTINATION, LOCATION, RESULT, OTHER.
  - Events connect through eight relations: CAUSE, ENABLE and PREVENT, their -OR (disjunctive) variants, SUPER (sub-event) and SAME (event coreference).
- **Annotation:**
  - Three biologists annotated structures in the brat tool, with one annotating and another validating each paragraph.
  - A paragraph averages 7.0 triggers, 11.3 arguments and 7.9 relations.
  - Questions were written independently of the structure annotation.
- **Relation to Scaria et al. (2013):** temporal relations are dropped, on the argument that causal links already imply the ordering that matters.
- **Error analysis:** the authors sampled 20 errors from the system using gold structures (GOLD) and 20 from the system using predicted structures (ProRead). Structure-prediction mistakes cause over half of ProRead's errors. GOLD's errors are spread across alignment, missing annotation, entity coreference, lexical variability and query-pattern coverage.

## Interesting ideas
- **Balanced distractors:** both answer options are built to overlap the paragraph lexically to a similar degree, and they reuse the paragraph's exact wording. This blocks word-matching shortcuts and isolates genuine comprehension, a useful recipe for any QA benchmark meant to test reasoning.
- **Answering as querying:** each question-answer pair becomes a query over a structure extracted from the document. This splits the problem into extraction and querying, so you can measure how much error each part contributes, for example by running the same queries on gold versus predicted structures.
- **Conjunction versus disjunction in relation labels:** having -OR versions of the causal relations lets the graph express that an event needs several causes, or leads to only one of several outcomes. This is a cheap way to add logical nuance to an event-relation schema.
- **Constraint-aware inference:** joint prediction enforces structural consistency, for instance forbidding mutual causation between two events. Such global constraints can help any graph-extraction task where labels on different edges depend on each other.
- **Separate annotators for questions and structures:** because the question writer never saw the structure annotation, the evaluation is not biased toward the schema. It also exposes gaps in the annotation, which is a good practice when building paired resources.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

93 papers of our library cite it (of the 140884 whose bibliographies we read); 21 of them have a page here, 72 not yet.

- [Joint Event Trigger Identification and Event Coreference Resolution with Structured Perceptron](https://md.research.elamur.ai/free/en/papers/araki-2015-joint-event-trigger-identification-event/)
- [A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task](https://md.research.elamur.ai/free/en/papers/chen-2016-thorough-examination-cnn-daily-mail/)
- [Resolving Event Coreference with Supervised Representation Learning and Clustering-Oriented Regularization](https://md.research.elamur.ai/free/en/papers/dean-2018-resolving-event-coreference-supervised-representation/)
- [MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension](https://md.research.elamur.ai/free/en/papers/fisch-2019-mrqa-2019-shared-task-evaluating/)
- [Out-of-domain FrameNet Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/hartmann-2017-out-domain-framenet-semantic-role/)
- [DEGREE: A Data-Efficient Generation-Based Event Extraction Model](https://md.research.elamur.ai/free/en/papers/hsu-2022-degree-data-efficient-generation-based/)
- [Towards Mitigating LLM Hallucination via Self Reflection](https://md.research.elamur.ai/free/en/papers/ji-2023-mitigating-llm-hallucination-self-reflection/)
- [How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks](https://md.research.elamur.ai/free/en/papers/kaushik-2018-how-much-reading-does-reading/)
- [Event Extraction as Machine Reading Comprehension](https://md.research.elamur.ai/free/en/papers/liu-2020-event-extraction-as-machine-reading/)
- [Machine Comprehension with Discourse Relations](https://md.research.elamur.ai/free/en/papers/narasimhan-2015-machine-comprehension-discourse-relations/)
- [emrQA: A Large Corpus for Question Answering on Electronic Medical Records](https://md.research.elamur.ai/free/en/papers/pampari-2018-emrqa-large-corpus-question-answering/)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
- [Solving General Arithmetic Word Problems](https://md.research.elamur.ai/free/en/papers/roy-2015-solving-general-arithmetic-word-problems/)
- [DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension](https://md.research.elamur.ai/free/en/papers/saha-2018-duorc-complex-language-understanding-paraphrased/)
- [Linguistically-Informed Self-Attention for Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/strubell-2018-linguistically-informed-self-attention-semantic/)
- [MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension](https://md.research.elamur.ai/free/en/papers/talmor-2019-multiqa-empirical-investigation-generalization-transfer/)
- [Revisiting the Evaluation for Cross Document Event Coreference](https://md.research.elamur.ai/free/en/papers/upadhyay-2016-revisiting-evaluation-cross-document-event/)
- [Gated Self-Matching Networks for Reading Comprehension and Question Answering](https://md.research.elamur.ai/free/en/papers/wang-2017-gated-self-matching-networks-reading/)
- [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://md.research.elamur.ai/free/en/papers/weston-2015-towards-ai-complete-question-answering/)
- [Dynamic Coattention Networks For Question Answering](https://md.research.elamur.ai/free/en/papers/xiong-2016-dynamic-coattention-networks-question-answering/)
- [Joint Extraction of Events and Entities within a Document Context](https://md.research.elamur.ai/free/en/papers/yang-2016-joint-extraction-events-entities-within/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{berant2014modeling,
  title = {{Modeling Biological Processes for Reading Comprehension}},
  author = {Berant, Jonathan and Srikumar, Vivek and Chen, Pei-Chun and Linden, Abby Vander and Harding, Brittany and Huang, Brad and Clark, Peter and Manning, Christopher D.},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2014},
  doi = {10.3115/v1/d14-1159},
  url = {https://doi.org/10.3115/v1/d14-1159}
}
```

- [← previous Microsoft COCO: Common Objects in Context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/)
- [next → Modeling Joint Entity and Relation Extraction with Table Representation](https://md.research.elamur.ai/free/en/papers/miwa-2014-modeling-joint-entity-relation-extraction/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/)

## References

37 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Jonathan+Berant%2C+Andrew+Chou%2C+Roy+Frostig%2C+and+Percy+Liang%2E+2013%2E+Semantic+parsing+on+Freebase+from+question%2Danswer+pairs%2E+In+Proceedings+of+EMNLP%2E)
2. Neil Campbell and Jane Reece. 2005. Biology. Benjamin Cummings. [find](https://scholar.google.com/scholar?q=Neil+Campbell+and+Jane+Reece%2E+2005%2E+Biology%2E+Benjamin+Cummings%2E)
3. Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka Jr., and Tom M. Mitchell. 2010. Toward an architecture for never-ending language learning. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Andrew+Carlson%2C+Justin+Betteridge%2C+Bryan+Kisiel%2C+Burr+Settles%2C+Estevam+R%2E+Hruschka+Jr%2E%2C+and+Tom+M%2E+Mitchell%2E+2010%2E+Toward+an+architecture+for+never%2Dending+language+learning%2E+In+Proceedings+of+AAAI%2E)
4. Nathanael Chambers and Daniel Jurafsky. 2008. Jointly combining implicit constraints improves temporal ordering. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Nathanael+Chambers+and+Daniel+Jurafsky%2E+2008%2E+Jointly+combining+implicit+constraints+improves+temporal+ordering%2E+In+Proceedings+of+EMNLP%2E)
5. Wen-Chi Chou, Richard Tzong-Han Tsai, Ying-Shan Su, Wei Ku, Ting-Yi Sung, and Wen-Lian Hsu. 2006. A semi-automatic method for annotating a biomedical proposition bank. In Proceedings of the Workshop on Frontiers in Linguistically Annotated Corpora 2006, July. [find](https://scholar.google.com/scholar?q=Wen%2DChi+Chou%2C+Richard+Tzong%2DHan+Tsai%2C+Ying%2DShan+Su%2C+Wei+Ku%2C+Ting%2DYi+Sung%2C+and+Wen%2DLian+Hsu%2E+2006%2E+A+semi%2Dautomatic+method+for+annotating+a+biomedical+proposition+bank%2E+In+Proceedings+of+the+Workshop+o)
6. James Clarke, Dan Goldwasser, Ming-Wei Chang, and Dan Roth. 2010. Driving semantic parsing from the world’s response. In Proceedings of CoNLL. [find](https://scholar.google.com/scholar?q=James+Clarke%2C+Dan+Goldwasser%2C+Ming%2DWei+Chang%2C+and+Dan+Roth%2E+2010%2E+Driving+semantic+parsing+from+the+world%E2%80%99s+response%2E+In+Proceedings+of+CoNLL%2E)
7. James Clarke, Vivek Srikumar, Mark Sammons, and Dan Roth. 2012. An NLP Curator (or: How I Learned to Stop Worrying and Love NLP Pipelines). In Proceedings of LREC. [find](https://scholar.google.com/scholar?q=James+Clarke%2C+Vivek+Srikumar%2C+Mark+Sammons%2C+and+Dan+Roth%2E+2012%2E+An+NLP+Curator+%28or%3A+How+I+Learned+to+Stop+Worrying+and+Love+NLP+Pipelines%29%2E+In+Proceedings+of+LREC%2E)
8. Michael Collins. 2002. Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+2002%2E+Discriminative+training+methods+for+hidden+markov+models%3A+Theory+and+experiments+with+perceptron+algorithms%2E+In+Proceedings+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/collins-2002-discriminative-training-methods-hidden-markov/)
9. Quang Do, Wei Lu, and Dan Roth. 2012. Joint inference for event timeline construction. In Proceedings of EMNLP-CoNLL. [find](https://scholar.google.com/scholar?q=Quang+Do%2C+Wei+Lu%2C+and+Dan+Roth%2E+2012%2E+Joint+inference+for+event+timeline+construction%2E+In+Proceedings+of+EMNLP%2DCoNLL%2E)
10. George R. Doddington, Alexis Mitchell, Mark A. Przybocki, Lance A. Ramshaw, Stephanie Strassel, and Ralph M. Weischedel. 2004. The Automatic Content Extraction (ACE) Program-Tasks, Data, and Evaluation. In Proceedings of LREC. [find](https://scholar.google.com/scholar?q=George+R%2E+Doddington%2C+Alexis+Mitchell%2C+Mark+A%2E+Przybocki%2C+Lance+A%2E+Ramshaw%2C+Stephanie+Strassel%2C+and+Ralph+M%2E+Weischedel%2E+2004%2E+The+Automatic+Content+Extraction+%28ACE%29+Program%2DTasks%2C+Data%2C+and+Evaluatio) [our card](https://md.research.elamur.ai/free/en/papers/doddington-2004-automatic-content-extraction-ace-program/)
11. Oren Etzioni, Michele Banko, and Michael J. Cafarella. 2006. Machine reading. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Oren+Etzioni%2C+Michele+Banko%2C+and+Michael+J%2E+Cafarella%2E+2006%2E+Machine+reading%2E+In+Proceedings+of+AAAI%2E)
12. Anthony Fader, Stephen Soderland, and Oren Etzioni. 2011. Identifying relations for open information extraction. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Anthony+Fader%2C+Stephen+Soderland%2C+and+Oren+Etzioni%2E+2011%2E+Identifying+relations+for+open+information+extraction%2E+In+Proceedings+of+EMNLP%2E)
13. Anthony Fader, Luke Zettlemoyer, and Oren Etzioni. 2013. Paraphrase-Driven Learning for Open Question Answering. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Anthony+Fader%2C+Luke+Zettlemoyer%2C+and+Oren+Etzioni%2E+2013%2E+Paraphrase%2DDriven+Learning+for+Open+Question+Answering%2E+In+Proceedings+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/fader-2013-paraphrase-driven-learning-open-question/)
14. Christiane Fellbaum, editor. 1998. WordNet: An electronic lexical database. MIT Press. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2C+editor%2E+1998%2E+WordNet%3A+An+electronic+lexical+database%2E+MIT+Press%2E)
15. Lynette Hirschman, Marc Light, Eric Breck, and John D. Burger. 1999. Deep read: A reading comprehension system. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Lynette+Hirschman%2C+Marc+Light%2C+Eric+Breck%2C+and+John+D%2E+Burger%2E+1999%2E+Deep+read%3A+A+reading+comprehension+system%2E+In+Proceedings+of+ACL%2E)
16. Jin-Dong Kim, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Junichi Tsujii. 2009. Overview of BioNLP 09 shared task on event extraction. In Proceedings of BioNLP. [find](https://scholar.google.com/scholar?q=Jin%2DDong+Kim%2C+Tomoko+Ohta%2C+Sampo+Pyysalo%2C+Yoshinobu+Kano%2C+and+Junichi+Tsujii%2E+2009%2E+Overview+of+BioNLP+09+shared+task+on+event+extraction%2E+In+Proceedings+of+BioNLP%2E)
17. Jin-Dong Kim, Sampo Pyysalo, Tomoko Ohta, Robert Bossy, and Junichi Tsujii. 2011. Overview of BioNLP shared task 2011. In Proceedings of BioNLP. [find](https://scholar.google.com/scholar?q=Jin%2DDong+Kim%2C+Sampo+Pyysalo%2C+Tomoko+Ohta%2C+Robert+Bossy%2C+and+Junichi+Tsujii%2E+2011%2E+Overview+of+BioNLP+shared+task+2011%2E+In+Proceedings+of+BioNLP%2E)
18. Jayant Krishnamurthy and Thomas Kollar. 2013. Jointly learning to parse and perceive: Connecting natural language to the physical world. TACL, 1:193–206. [find](https://scholar.google.com/scholar?q=Jayant+Krishnamurthy+and+Thomas+Kollar%2E+2013%2E+Jointly+learning+to+parse+and+perceive%3A+Connecting+natural+language+to+the+physical+world%2E+TACL%2C+1%3A193%E2%80%93206%2E)
19. Nate Kushman, Yoav Artzi, Luke Zettlemoyer, and Regina Barzilay. 2014. Learning to automatically solve algebra word problems. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Nate+Kushman%2C+Yoav+Artzi%2C+Luke+Zettlemoyer%2C+and+Regina+Barzilay%2E+2014%2E+Learning+to+automatically+solve+algebra+word+problems%2E+In+Proceedings+of+ACL%2E)
20. Beth Levin. 1993. English verb classes and alternations: A preliminary investigation. University of Chicago Press. [find](https://scholar.google.com/scholar?q=Beth+Levin%2E+1993%2E+English+verb+classes+and+alternations%3A+A+preliminary+investigation%2E+University+of+Chicago+Press%2E)
21. Catherine Macleod, Ralph Grishman, Adam Meyers, Leslie Barrett, and Ruth Reeves. 1998. Nomlex: A lexicon of nominalizations. In Proceedings of EU-RALEX. [find](https://scholar.google.com/scholar?q=Catherine+Macleod%2C+Ralph+Grishman%2C+Adam+Meyers%2C+Leslie+Barrett%2C+and+Ruth+Reeves%2E+1998%2E+Nomlex%3A+A+lexicon+of+nominalizations%2E+In+Proceedings+of+EU%2DRALEX%2E)
22. Thomas L. Magnanti and Laurence A. Wolsey. 1995. Optimal trees. Handbooks in operations research and management science, 7:503–615. [find](https://scholar.google.com/scholar?q=Thomas+L%2E+Magnanti+and+Laurence+A%2E+Wolsey%2E+1995%2E+Optimal+trees%2E+Handbooks+in+operations+research+and+management+science%2C+7%3A503%E2%80%93615%2E)
23. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of ACL: System Demonstrations. [find](https://scholar.google.com/scholar?q=Christopher+D%2E+Manning%2C+Mihai+Surdeanu%2C+John+Bauer%2C+Jenny+Finkel%2C+Steven+J%2E+Bethard%2C+and+David+Mc%2DClosky%2E+2014%2E+The+Stanford+CoreNLP+natural+language+processing+toolkit%2E+In+Proceedings+of+ACL%3A+System+) [our card](https://md.research.elamur.ai/free/en/papers/manning-2014-stanford-corenlp-natural-language-processing/)
24. André L. Martins, Noah A. Smith, and Eric P. Xing. 2009. Concise integer linear programming formulations for dependency parsing. In Proceedings of ACL/IJCNLP. [find](https://scholar.google.com/scholar?q=Andre%CC%81+L%2E+Martins%2C+Noah+A%2E+Smith%2C+and+Eric+P%2E+Xing%2E+2009%2E+Concise+integer+linear+programming+formulations+for+dependency+parsing%2E+In+Proceedings+of+ACL%2FIJCNLP%2E)
25. David McClosky and Christopher D. Manning. 2012. Learning constraints for consistent timeline extraction. In Proceedings of EMNLP-CoNLL. [find](https://scholar.google.com/scholar?q=David+McClosky+and+Christopher+D%2E+Manning%2E+2012%2E+Learning+constraints+for+consistent+timeline+extraction%2E+In+Proceedings+of+EMNLP%2DCoNLL%2E)
26. Dan Moldovan, Christine Clark, Sanda Harabagiu, and Steve Maiorano. 2003. Cogex: A logic prover for question answering. In Proceedings of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Dan+Moldovan%2C+Christine+Clark%2C+Sanda+Harabagiu%2C+and+Steve+Maiorano%2E+2003%2E+Cogex%3A+A+logic+prover+for+question+answering%2E+In+Proceedings+of+NAACL%2DHLT%2E)
27. Martha Palmer, Paul Kingsbury, and Daniel Gildea. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71–106. [find](https://scholar.google.com/scholar?q=Martha+Palmer%2C+Paul+Kingsbury%2C+and+Daniel+Gildea%2E+2005%2E+The+proposition+bank%3A+An+annotated+corpus+of+semantic+roles%2E+Computational+Linguistics%2C+31%281%29%3A71%E2%80%93106%2E) [our card](https://md.research.elamur.ai/free/en/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/)
28. Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2008. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics, 34(2). [find](https://scholar.google.com/scholar?q=Vasin+Punyakanok%2C+Dan+Roth%2C+and+Wen%2Dtau+Yih%2E+2008%2E+The+importance+of+syntactic+parsing+and+inference+in+semantic+role+labeling%2E+Computational+Linguistics%2C+34%282%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/punyakanok-2008-importance-syntactic-parsing-inference-semantic/)
29. Matthew Richardson, Christopher JC Burges, and Erin Renshaw. 2013. MCTest: A challenge dataset for the open-domain machine comprehension of text. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Matthew+Richardson%2C+Christopher+JC+Burges%2C+and+Erin+Renshaw%2E+2013%2E+MCTest%3A+A+challenge+dataset+for+the+open%2Ddomain+machine+comprehension+of+text%2E+In+Proceedings+of+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/missing-2013-richardson-mctest-challenge-dataset-open-domain/)
30. Sebastian Riedel and Andrew McCallum. 2011. Fast and robust joint models for biomedical event extraction. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Sebastian+Riedel+and+Andrew+McCallum%2E+2011%2E+Fast+and+robust+joint+models+for+biomedical+event+extraction%2E+In+Proceedings+of+EMNLP%2E)
31. Dan Roth and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of CoNLL. [find](https://scholar.google.com/scholar?q=Dan+Roth+and+Wen%2Dtau+Yih%2E+2004%2E+A+linear+programming+formulation+for+global+inference+in+natural+language+tasks%2E+In+Proceedings+of+CoNLL%2E) [our card](https://md.research.elamur.ai/free/en/papers/roth-2004-linear-programming-formulation-global-inference/)
32. Josef Ruppenhofer, Michael Ellsworth, Miriam RL Petruck, Christopher R. Johnson, and Jan Scheffczyk. 2006. FrameNet II: Extended theory and practice. Berkeley FrameNet Release, 1. [find](https://scholar.google.com/scholar?q=Josef+Ruppenhofer%2C+Michael+Ellsworth%2C+Miriam+RL+Petruck%2C+Christopher+R%2E+Johnson%2C+and+Jan+Scheffczyk%2E+2006%2E+FrameNet+II%3A+Extended+theory+and+practice%2E+Berkeley+FrameNet+Release%2C+1%2E)
33. Aju Thalappillil Scaria, Jonathan Berant, Mengqiu Wang, Peter Clark, Justin Lewis, Brittany Harding, and Christopher D. Manning. 2013. Learning biological processes with global constraints. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Aju+Thalappillil+Scaria%2C+Jonathan+Berant%2C+Mengqiu+Wang%2C+Peter+Clark%2C+Justin+Lewis%2C+Brittany+Harding%2C+and+Christopher+D%2E+Manning%2E+2013%2E+Learning+biological+processes+with+global+constraints%2E+In+Proceed)
34. Pontus Stenetorp, Sampo Pyysalo, Goran Topić, Tomoko Ohta, Sophia Ananiadou, and Jun’ichi Tsujii. 2012. Brat: a web-based tool for NLP-assisted text annotation. In Proceedings of the demonstrations at EACL. [find](https://scholar.google.com/scholar?q=Pontus+Stenetorp%2C+Sampo+Pyysalo%2C+Goran+Topic%CC%81%2C+Tomoko+Ohta%2C+Sophia+Ananiadou%2C+and+Jun%E2%80%99ichi+Tsujii%2E+2012%2E+Brat%3A+a+web%2Dbased+tool+for+NLP%2Dassisted+text+annotation%2E+In+Proceedings+of+the+demonstrations+a) [our card](https://md.research.elamur.ai/free/en/papers/stenetorp-2012-brat-web-based-tool-nlp/)
35. Yuk Wah Wong and Raymond J. Mooney. 2006. Learning for semantic parsing with statistical machine translation. In Proceedings of HLT-NAACL. [find](https://scholar.google.com/scholar?q=Yuk+Wah+Wong+and+Raymond+J%2E+Mooney%2E+2006%2E+Learning+for+semantic+parsing+with+statistical+machine+translation%2E+In+Proceedings+of+HLT%2DNAACL%2E)
36. Katsumasa Yoshikawa, Sebastian Riedel, Masayuki Asahara, and Yuji Matsumoto. 2009. Jointly identifying temporal relations with Markov logic. In Proceedings of ACL/IJCNLP. [find](https://scholar.google.com/scholar?q=Katsumasa+Yoshikawa%2C+Sebastian+Riedel%2C+Masayuki+Asahara%2C+and+Yuji+Matsumoto%2E+2009%2E+Jointly+identifying+temporal+relations+with+Markov+logic%2E+In+Proceedings+of+ACL%2FIJCNLP%2E)
37. Luke S. Zettlemoyer and Michael Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Proceedings of UAI. [find](https://scholar.google.com/scholar?q=Luke+S%2E+Zettlemoyer+and+Michael+Collins%2E+2005%2E+Learning+to+map+sentences+to+logical+form%3A+Structured+classification+with+probabilistic+categorial+grammars%2E+In+Proceedings+of+UAI%2E)