--- title: "Using Predicate-Argument Structures for — Surdeanu 2003" description: "The paper asks whether event-level information extraction (IE) can avoid hand-built, domain-specific pattern grammars." authors: ["Mihai Surdeanu", "Sanda Harabagiu", "John Williams", "Paul Aarseth"] year: 2003 venue: "Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics" source: "https://aclanthology.org/P03-1002/" doi: "10.3115/1075096.1075098" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/surdeanu-2003-predicate-argument-structures-information-extraction/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/surdeanu-2003-predicate-argument-structures-information-extraction/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2003](https://md.research.elamur.ai/free/en/papers/2003/) / Using Predicate-Argument Structures for Information Extraction # Using Predicate-Argument Structures for Information Extraction **Authors:** [Mihai Surdeanu](https://md.research.elamur.ai/free/en/authors/mihai-surdeanu/), [Sanda Harabagiu](https://md.research.elamur.ai/free/en/authors/sanda-harabagiu/), John Williams, Paul Aarseth · **Year:** 2003 · **Venue:** Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/P03-1002/) · [PDF](https://aclanthology.org/P03-1002.pdf) · [DOI 10.3115/1075096.1075098](https://doi.org/10.3115/1075096.1075098) · [OpenAlex](https://openalex.org/W2145310422) **Citations:** 364 citations (openalex-snapshot · 2026-10-14) · [127](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary The paper asks whether event-level information extraction (IE) can avoid hand-built, domain-specific pattern grammars. The authors propose to build it on automatically recognized predicate-argument structures instead. In their design, a domain is described by two things: the predicates that matter for it and a mapping from those predicates' arguments to templette slots. Argument labeling is learned from PropBank, the Wall Street Journal part of Penn Treebank 2 annotated with verb arguments. The authors replace the backoff-lattice statistical labeler of Gildea and Palmer (2002) with C5 decision trees. These trees can take a much larger feature set, including content words, part-of-speech tags, named-entity information and phrasal-verb statistics. The new labeler beats the statistical one by about 17% F-score, and this improvement carries over to up to 14% better extraction. They test on two HUB Event-99 domains, "market change" and "death". There the predicate-argument approach reaches roughly 90% of a cascaded FSA system's F-measure. It needs only a couple of hours of slot mapping per domain, compared with about ten person-days of pattern writing. This makes IE much cheaper to port to new domains. ## Key points - The argument labeler is trained on PropBank, a one-million-word corpus. Its numbered roles run from Arg0 to Arg5, and it also has ArgM functional tags such as ArgM-LOC, ArgM-TMP and ArgM-DIR. - The method uses two C5 classifiers, one that finds argument constituents and one that assigns their roles. It reaches over 88% F-measure on identification and over 83% accuracy on role assignment. Predicate lexical features drive fewer than 5% of the tree's branching decisions. - Templette F-measure, comparing the statistical labeler, the inductive labeler and the FSA system: - Market change: 68.9%, 82.8% and 91.3%. - Death: 58.4%, 67.0% and 72.7%. - Correctly matched event structures were 26, 33 and 38 for the three systems, and missed ones were 16, 9 and 4. For the comparison, coreference and templette merging were switched off in both architectures. - The "death" domain relies on verbs that are rare in training; for example, "die" occurs only 5 times in PropBank. The inductive model still held up there. ## Interesting ideas - **Separating language analysis from the domain.** A generic semantic-role layer is paired with a thin, per-domain table that maps arguments to slots. This turns domain porting into a small mapping task that could be learned automatically. The same split could work for any extraction schema, such as trade events or geographic relations. - **Choosing a learner that makes feature testing easy.** The authors moved away from a backoff lattice, whose structure grows awkward with each added feature. With decision trees they could quickly try many features and see how each one mattered. This is a useful reminder that model choice can speed up research iteration, not only change final accuracy. - **A "content word" next to the syntactic head.** Simple rules pick the informative word inside a phrase, for example the noun inside a PP or the verb inside an SBAR or infinitive VP. Heads like prepositions or "that" carry little meaning. The trick could transfer to other parsing-based features where function-word heads hide the semantics. - **Backing off sparse lexical features to coarser ones.** The authors noticed that head-word and content-word features were rarely used because of sparsity. Their fix was to add the POS tags of those words and named-entity flags as generalizations. This is cheap and applicable to low-resource settings. - **Evaluating in two kinds of domains on purpose.** They compared a domain whose verbs are common in training with one whose verbs are rare. They also reported both slot-level F-measure and counts of matched events. This shows how a component generalizes to unseen predicates, not just its average score. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Semantic role labeling](https://md.research.elamur.ai/free/en/topics/semantic-role-labeling/) · [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) ## Cited by Cited by 127 papers of our library cite it (of the 140884 whose bibliographies we read); 16 of them have a page here, 111 not yet. - [UTD-SRL: A Pipeline Architecture for Extracting Frame Semantic Structures](https://md.research.elamur.ai/free/en/papers/bejan-2007-utd-srl-pipeline-architecture-extracting/) - [Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/carreras-2005-introduction-conll-2005-shared-task/) - [Probabilistic Frame-Semantic Parsing](https://md.research.elamur.ai/free/en/papers/das-2010-probabilistic-frame-semantic-parsing/) - [Frame-Semantic Parsing](https://md.research.elamur.ai/free/en/papers/das-2014-frame-semantic-parsing/) - [The Hebrew FrameNet Project](https://md.research.elamur.ai/free/en/papers/frame-2016-hayoun-hebrew-framenet-project/) - [Dependency-based Semantic Role Labeling of PropBank](https://md.research.elamur.ai/free/en/papers/johansson-2008-dependency-based-semantic-role-labeling/) - [A Comparative Study on Generalization of Semantic Roles in FrameNet](https://md.research.elamur.ai/free/en/papers/matsubayashi-2009-comparative-study-generalization-semantic-roles/) - [The Proposition Bank: An Annotated Corpus of Semantic Roles](https://md.research.elamur.ai/free/en/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/) - [Evaluating FrameNet-style semantic parsing: the role of coverage gaps in FrameNet](https://md.research.elamur.ai/free/en/papers/palmer-2010-evaluating-framenet-style-semantic-parsing/) - [Shallow Semantic Parsing using Support Vector Machines](https://md.research.elamur.ai/free/en/papers/pradhan-2004-shallow-semantic-parsing-support-vector/) - [Towards Robust Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/pradhan-2008-robust-semantic-role-labeling/) - [The Importance of Syntactic Parsing and Inference in Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/punyakanok-2008-importance-syntactic-parsing-inference-semantic/) - [Deep Semantic Role Labeling with Self-Attention](https://md.research.elamur.ai/free/en/papers/tan-2017-deep-semantic-role-labeling-self/) - [Evaluating Universal Dependency Parser Recovery of Predicate Argument Structure via CompChain Analysis](https://md.research.elamur.ai/free/en/papers/ud-2021-indurkhya-evaluating-universal-dependency-parser-recovery/) - [A Joint Sequential and Relational Model for Frame-Semantic Parsing](https://md.research.elamur.ai/free/en/papers/yang-2017-joint-sequential-relational-model-frame/) - [End-to-end learning of semantic role labeling using recurrent neural networks](https://md.research.elamur.ai/free/en/papers/zhou-2015-end-end-learning-semantic-role/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{surdeanu2003using, title = {{Using Predicate-Argument Structures for Information Extraction}}, author = {Surdeanu, Mihai and Harabagiu, Sanda and Williams, John and Aarseth, Paul}, booktitle = {Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics}, year = {2003}, doi = {10.3115/1075096.1075098}, url = {https://doi.org/10.3115/1075096.1075098} } ``` - [← previous Undisclosed Speech: Patterns of Communication in the Book of Isaiah](https://md.research.elamur.ai/free/en/papers/hbo-2003-liss-undisclosed-speech-patterns-communication-book/) - [next → Using the Web in Machine Learning for Other-Anaphora Resolution](https://md.research.elamur.ai/free/en/papers/modjeska-2003-web-machine-learning-other-anaphora/) - [↑ back to papers of 2003](https://md.research.elamur.ai/free/en/papers/2003/) ## References 14 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet Project. In Proceedings of COL-ING/ACL ’98:86-90,. Montreal, Canada. [find](https://scholar.google.com/scholar?q=Collin+F%2E+Baker%2C+Charles+J%2E+Fillmore%2C+and+John+B%2E+Lowe%2E+1998%2E+The+Berkeley+FrameNet+Project%2E+In+Proceedings+of+COL%2DING%2FACL+%E2%80%9998%3A86%2D90%2C%2E+Montreal%2C+Canada%2E) [our card](https://md.research.elamur.ai/free/en/papers/lex-1998-baker-berkeley-framenet/) 2. Eric Brill. 1995. Transformation-Based Error-Driven Learning and Natural Language Processing: A Case Study in Part of Speech Tagging. Computational Linguistics. [find](https://scholar.google.com/scholar?q=Eric+Brill%2E+1995%2E+Transformation%2DBased+Error%2DDriven+Learning+and+Natural+Language+Processing%3A+A+Case+Study+in+Part+of+Speech+Tagging%2E+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/brill-1995-transformation-based-error-driven-learning/) 3. Michael Collins. 1997. Three Generative, Lexicalized Models for Statistical Parsing. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics (ACL 1997):16-23, Madrid, Spain. [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+1997%2E+Three+Generative%2C+Lexicalized+Models+for+Statistical+Parsing%2E+In+Proceedings+of+the+35th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL+1997%29%3A16%2D23%2C+Madrid) 4. Daniel Gildea and Daniel Jurafsky. 2002. Automatic Labeling of Semantic Roles. Computational Linguistics, 28(3):245-288. [find](https://scholar.google.com/scholar?q=Daniel+Gildea+and+Daniel+Jurafsky%2E+2002%2E+Automatic+Labeling+of+Semantic+Roles%2E+Computational+Linguistics%2C+28%283%29%3A245%2D288%2E) [our card](https://md.research.elamur.ai/free/en/papers/gildea-2002-automatic-labeling-semantic-roles/) 5. Daniel Gildea and Martha Palmer. 2002. The Necessity of Parsing for Predicate Argument Recognition. In Proceedings of the 40th Meeting of the Association for Computational Linguistics (ACL 2002):239-246, Philadelphia, PA. [find](https://scholar.google.com/scholar?q=Daniel+Gildea+and+Martha+Palmer%2E+2002%2E+The+Necessity+of+Parsing+for+Predicate+Argument+Recognition%2E+In+Proceedings+of+the+40th+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL+2002%29%3A239%2D2) 6. Lynette Hirschman, Patricia Robinson, Lisa Ferro, Nancy Chinchor, Erica Brown, Ralph Grishman, Beth Sundheim 1999. Hub-4 Event99 General Guidelines and Templettes. [find](https://scholar.google.com/scholar?q=Lynette+Hirschman%2C+Patricia+Robinson%2C+Lisa+Ferro%2C+Nancy+Chinchor%2C+Erica+Brown%2C+Ralph+Grishman%2C+Beth+Sundheim+1999%2E+Hub%2D4+Event99+General+Guidelines+and+Templettes%2E) 7. Jerry R. Hobbs, Douglas Appelt, John Bear, David Israel, Megumi Kameyama, Mark E. Stickel, and Mabry Tyson. 1997. FASTUS: A Cascaded Finite-State Transducer for Extracting Information from Natural-Language Text. In Finite-State Language Processing, pages 383-406, MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Jerry+R%2E+Hobbs%2C+Douglas+Appelt%2C+John+Bear%2C+David+Israel%2C+Megumi+Kameyama%2C+Mark+E%2E+Stickel%2C+and+Mabry+Tyson%2E+1997%2E+FASTUS%3A+A+Cascaded+Finite%2DState+Transducer+for+Extracting+Information+from+Natural%2DLan) 8. Paul Kingsbury, Martha Palmer, and Mitch Marcus. 2002. Adding Semantic Annotation to the Penn TreeBank. In Proceedings of the Human Language Technology Conference (HLT 2002):252-256, San Diego, California. [find](https://scholar.google.com/scholar?q=Paul+Kingsbury%2C+Martha+Palmer%2C+and+Mitch+Marcus%2E+2002%2E+Adding+Semantic+Annotation+to+the+Penn+TreeBank%2E+In+Proceedings+of+the+Human+Language+Technology+Conference+%28HLT+2002%29%3A252%2D256%2C+San+Diego%2C+Califo) 9. Beth Levin. 1993. English Verb Classes and Alternations a Preliminary Investigation. University of Chicago Press. [find](https://scholar.google.com/scholar?q=Beth+Levin%2E+1993%2E+English+Verb+Classes+and+Alternations+a+Preliminary+Investigation%2E+University+of+Chicago+Press%2E) 10. Grace Ngai and Radu Florian. 2001. Transformation-Based Learning in The Fast Lane. In Proceedings of the North American Association for Computational Linguistics (NAACL 2001):40-47. [find](https://scholar.google.com/scholar?q=Grace+Ngai+and+Radu+Florian%2E+2001%2E+Transformation%2DBased+Learning+in+The+Fast+Lane%2E+In+Proceedings+of+the+North+American+Association+for+Computational+Linguistics+%28NAACL+2001%29%3A40%2D47%2E) 11. Ross Quinlan. 2002. Data Mining Tools See5 and C5.0. [http://www.rulequest.com/see5-info.html](http://www.rulequest.com/see5-info.html). [↗](http://www.rulequest.com/see5-info.html) 12. Ellen Riloff and Rosie Jones. 1996. Automatically Generating Extraction Patterns from Untagged Text. In Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI-96)):1044-1049. [find](https://scholar.google.com/scholar?q=Ellen+Riloff+and+Rosie+Jones%2E+1996%2E+Automatically+Generating+Extraction+Patterns+from+Untagged+Text%2E+In+Proceedings+of+the+Thirteenth+National+Conference+on+Artificial+Intelligence+%28AAAI%2D96%29%29%3A1044%2D104) 13. Mihai Surdeanu and Sanda Harabagiu. 2002. Infrastructure for Open-Domain Information Extraction In Proceedings of the Human Language Technology Conference (HLT 2002):325-330. [find](https://scholar.google.com/scholar?q=Mihai+Surdeanu+and+Sanda+Harabagiu%2E+2002%2E+Infrastructure+for+Open%2DDomain+Information+Extraction+In+Proceedings+of+the+Human+Language+Technology+Conference+%28HLT+2002%29%3A325%2D330%2E) 14. Roman Yangarber, Ralph Grishman, Pasi Tapainen and Silja Huttunen, 2000. Automatic Acquisition of Domain Knowledge for Information Extraction. In Proceedings of the 18th International Conference on Computational Linguistics (COLING-2000): 940-946, Saarbrucken, Germany. [find](https://scholar.google.com/scholar?q=Roman+Yangarber%2C+Ralph+Grishman%2C+Pasi+Tapainen+and+Silja+Huttunen%2C+2000%2E+Automatic+Acquisition+of+Domain+Knowledge+for+Information+Extraction%2E+In+Proceedings+of+the+18th+International+Conference+on+Co)