---
title: "Joint Event Extraction via Structured Prediction — Li 2013"
description: "The paper addresses sentence-level event extraction in the ACE setting, which has 8 types and 33 subtypes."
authors: ["Qi Li", "Heng Ji", "Liang Huang"]
year: 2013
venue: "Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P13-1008/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/li-2013-joint-event-extraction-structured-prediction/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/li-2013-joint-event-extraction-structured-prediction/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2013](https://md.research.elamur.ai/free/en/papers/2013/) / Joint Event Extraction via Structured Prediction with Global Features

# Joint Event Extraction via Structured Prediction with Global Features

**Authors:** [Qi Li](https://md.research.elamur.ai/free/en/authors/qi-li/), [Heng Ji](https://md.research.elamur.ai/free/en/authors/heng-ji/), [Liang Huang](https://md.research.elamur.ai/free/en/authors/liang-huang/) · **Year:** 2013 · **Venue:** Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P13-1008/) · [PDF](https://aclanthology.org/P13-1008.pdf) · [OpenAlex](https://openalex.org/W2165962657)

**Citations:** 446 citations (openalex-snapshot · 2026-10-14) · [235](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper addresses sentence-level event extraction in the ACE setting, which has 8 types and 33 subtypes. A system must find trigger words, assign their subtypes, and link entity mentions, time expressions and values to them as role-labelled arguments. Typical systems chain independent classifiers, so trigger errors spread to argument labelling and cannot be fixed later. The authors model the whole sentence output (triggers plus typed edges to argument candidates) as a single structure. They train it with a structured perceptron. Exact inference is intractable, so decoding uses beam search: for each token, the decoder labels the trigger and then its argument edges. Training uses early update, and test-time decoding uses averaged weights. On top of local features, they add global features that span several triggers and arguments at once. With gold argument candidates, the joint model reaches 67.5 F1 on trigger classification and 52.7 F1 on argument roles. This beats a pipeline and even a prior system that used cross-entity evidence beyond the sentence. The result shows that joint inference pays off for IE.

## Key points
- The task follows ACE: triggers are labelled with one of 33 subtypes or "not a trigger." Argument roles per subtype are restricted by the entity-type constraints from the ACE guidelines (e.g., an Attacker must be PER, ORG or GPE).
- With gold entities, timex and values, the results compare as follows:
  - Staged MaxEnt baseline: 65.9 trigger and 43.9 argument-role F1.
  - Joint model with local features: 65.7 and 46.5.
  - Joint model with local and global features: 67.5 and 52.7.
  - Hong et al. (2011), sentence-level: 59.7 and 36.5.
  - Hong et al. (2011), cross-entity: 68.3 and 48.3.
- With predicted argument candidates, the full joint model drops to 65.6 trigger F1 (−1.9) and 41.8 argument F1 (−10.9).
- Early update beats standard update. With global features and beam size 1, test argument F1 is 52.1 versus 49.8. At beam size 4 the gap shrinks (52.7 vs 51.4).
- Standard update produces many "invalid updates," especially with small beams and with global features. This motivates the early-update variant.

## Interesting ideas
- **Interleaved trigger-then-edges beam decoding.** The decoder labels a token's trigger and then immediately scores its edges to every candidate argument, all inside one beam. This lets argument evidence (e.g., "tank" as an Instrument) rescue an ambiguous trigger. The same layout suits any task where node labels and edge labels constrain each other, such as relation extraction.
- **Global features over partial structures.** Beam search makes arbitrary non-local features cheap to score. Examples are propagating a shared argument between co-occurring events (Victim of Die → Target of Attack) and penalising implausible counts, such as two Attackers for one event. This is a practical alternative to separate re-rankers when exact dynamic programming is impossible.
- **Early update for inexact search.** Training stops and updates as soon as the gold prefix falls out of the beam. This avoids updates that do not actually fix a violation. Counting invalid updates is also a useful diagnostic for whether a search problem has become too hard for standard perceptron training.
- **Type constraints to prune the label space.** Allowed argument roles per event subtype are restricted using the annotation guideline's entity-type rules. This cheaply shrinks the search space and removes impossible outputs. The trick transfers to any schema-driven IE annotation.
- **Gold vs predicted candidate evaluation.** Reporting both settings exposes how much of the remaining error comes from upstream entity recognition. Here, argument F1 drops by about 11 points with predicted candidates. This framing is useful when building pipelines for new languages.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

235 papers of our library cite it (of the 140884 whose bibliographies we read); 25 of them have a page here, 210 not yet.

- [Joint Event Trigger Identification and Event Coreference Resolution with Structured Perceptron](https://md.research.elamur.ai/free/en/papers/araki-2015-joint-event-trigger-identification-event/)
- [Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/chen-2015-event-extraction-dynamic-multi-pooling/)
- [Event Extraction by Answering (Almost) Natural Questions](https://md.research.elamur.ai/free/en/papers/du-2020-event-extraction-answering-almost-natural/)
- [A Language-Independent Neural Network for Event Detection](https://md.research.elamur.ai/free/en/papers/feng-2016-language-independent-neural-network-event/)
- [DEGREE: A Data-Efficient Generation-Based Event Extraction Model](https://md.research.elamur.ai/free/en/papers/hsu-2022-degree-data-efficient-generation-based/)
- [Zero-Shot Transfer Learning for Event Extraction](https://md.research.elamur.ai/free/en/papers/huang-2018-zero-shot-transfer-learning-event/)
- [QANom: Question-Answer driven SRL for Nominalizations](https://md.research.elamur.ai/free/en/papers/klein-2020-qanom-question-answer-driven-srl/)
- [Incremental Joint Extraction of Entity Mentions and Relations](https://md.research.elamur.ai/free/en/papers/li-2014-incremental-joint-extraction-entity-mentions/)
- [Event Extraction as Multi-turn Question Answering](https://md.research.elamur.ai/free/en/papers/li-2020-event-extraction-as-multi-turn/)
- [Leveraging FrameNet to Improve Automatic Event Detection](https://md.research.elamur.ai/free/en/papers/liu-2016-leveraging-framenet-improve-automatic-event/)
- [Exploiting Argument Information to Improve Event Detection via Supervised Attention Mechanisms](https://md.research.elamur.ai/free/en/papers/liu-2017-exploiting-argument-information-improve-event/)
- [Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation](https://md.research.elamur.ai/free/en/papers/liu-2018-jointly-multiple-events-extraction-attention/)
- [Event Extraction as Machine Reading Comprehension](https://md.research.elamur.ai/free/en/papers/liu-2020-event-extraction-as-machine-reading/)
- [Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction](https://md.research.elamur.ai/free/en/papers/lu-2021-text2event-controllable-sequence-structure-generation/)
- [Prune-and-Score: Learning for Greedy Coreference Resolution](https://md.research.elamur.ai/free/en/papers/ma-2014-prune-score-learning-greedy-coreference/)
- [Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction](https://md.research.elamur.ai/free/en/papers/ma-2022-prompt-extraction-paie-prompting-argument/)
- [Event Detection and Domain Adaptation with Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/nguyen-2015-event-detection-domain-adaptation-convolutional/)
- [Joint Event Extraction via Recurrent Neural Networks](https://md.research.elamur.ai/free/en/papers/nguyen-2016-joint-event-extraction-recurrent-neural/)
- [Event Detection and Co-reference with Minimal Supervision](https://md.research.elamur.ai/free/en/papers/peng-2016-event-detection-co-reference-minimal/)
- [Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging](https://md.research.elamur.ai/free/en/papers/reimers-2017-reporting-score-distributions-makes-difference/)
- [Exploiting Document Structures and Cluster Consistencies for Event Coreference Resolution](https://md.research.elamur.ai/free/en/papers/tran-2021-exploiting-document-structures-cluster-consistencies/)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://md.research.elamur.ai/free/en/papers/wadden-2019-entity-relation-event-extraction-contextualized/)
- [MAVEN: A Massive General Domain Event Detection Dataset](https://md.research.elamur.ai/free/en/papers/wang-2020-maven-massive-general-domain-event/)
- [Joint Extraction of Events and Entities within a Document Context](https://md.research.elamur.ai/free/en/papers/yang-2016-joint-extraction-events-entities-within/)
- [Exploring Pre-trained Language Models for Event Extraction and Generation](https://md.research.elamur.ai/free/en/papers/yang-2019-exploring-pre-trained-language-models/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{li2013joint,
  title = {{Joint Event Extraction via Structured Prediction with Global Features}},
  author = {Li, Qi and Ji, Heng and Huang, Liang},
  booktitle = {Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2013},
  url = {https://aclanthology.org/P13-1008/}
}
```

- [← previous Joint Coreference Resolution and Named-Entity Linking with Multi-Pass Sieves](https://md.research.elamur.ai/free/en/papers/hajishirzi-2013-joint-coreference-resolution-named-entity/)
- [next → Latent Anaphora Resolution for Cross-Lingual Pronoun Prediction](https://md.research.elamur.ai/free/en/papers/hardmeier-2013-latent-anaphora-resolution-cross-lingual/)
- [↑ back to papers of 2013](https://md.research.elamur.ai/free/en/papers/2013/)

## References

22 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Ahn. 2006. The stages of event extraction. In Proceedings of the Workshop on Annotating and Reasoning about Time and Events, pages 1–8. [find](https://scholar.google.com/scholar?q=David+Ahn%2E+2006%2E+The+stages+of+event+extraction%2E+In+Proceedings+of+the+Workshop+on+Annotating+and+Reasoning+about+Time+and+Events%2C+pages+1%E2%80%938%2E) [our card](https://md.research.elamur.ai/free/en/papers/ahn-2006-stages-event-extraction/)
2. Peter F Brown, Peter V Desouza, Robert L Mercer, Vincent J Della Pietra, and Jenifer C Lai. 1992. Class-based n-gram models of natural language. Computational linguistics, 18(4):467–479. [find](https://scholar.google.com/scholar?q=Peter+F+Brown%2C+Peter+V+Desouza%2C+Robert+L+Mercer%2C+Vincent+J+Della+Pietra%2C+and+Jenifer+C+Lai%2E+1992%2E+Class%2Dbased+n%2Dgram+models+of+natural+language%2E+Computational+linguistics%2C+18%284%29%3A467%E2%80%93479%2E)
3. Zheng Chen and Heng Ji. 2009. Language specific issue and feature exploration in chinese event extraction. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Papers, pages 2… [find](https://scholar.google.com/scholar?q=Zheng+Chen+and+Heng+Ji%2E+2009%2E+Language+specific+issue+and+feature+exploration+in+chinese+event+extraction%2E+In+Proceedings+of+Human+Language+Technologies%3A+The+2009+Annual+Conference+of+the+North+Americ)
4. Chen Chen and Vincent Ng. 2012. Joint modeling for chinese event extraction with rich linguistic features. In COLING, pages 529–544. [find](https://scholar.google.com/scholar?q=Chen+Chen+and+Vincent+Ng%2E+2012%2E+Joint+modeling+for+chinese+event+extraction+with+rich+linguistic+features%2E+In+COLING%2C+pages+529%E2%80%93544%2E)
5. Michael Collins and Brian Roark. 2004. Incremental parsing with the perceptron algorithm. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, page 111. [find](https://scholar.google.com/scholar?q=Michael+Collins+and+Brian+Roark%2E+2004%2E+Incremental+parsing+with+the+perceptron+algorithm%2E+In+Proceedings+of+the+42nd+Annual+Meeting+on+Association+for+Computational+Linguistics%2C+page+111%2E) [our card](https://md.research.elamur.ai/free/en/papers/collins-2004-incremental-parsing-perceptron-algorithm/)
6. Michael Collins. 2002. Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10, pages 1–8. [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+2002%2E+Discriminative+training+methods+for+hidden+markov+models%3A+Theory+and+experiments+with+perceptron+algorithms%2E+In+Proceedings+of+the+ACL%2D02+conference+on+Empirical+methods+in+natu) [our card](https://md.research.elamur.ai/free/en/papers/collins-2002-discriminative-training-methods-hidden-markov/)
7. Marie-Catherine De Marneffe, Bill MacCartney, and Christopher D Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of LREC, volume 6, pages 449–454. [find](https://scholar.google.com/scholar?q=Marie%2DCatherine+De+Marneffe%2C+Bill+MacCartney%2C+and+Christopher+D+Manning%2E+2006%2E+Generating+typed+dependency+parses+from+phrase+structure+parses%2E+In+Proceedings+of+LREC%2C+volume+6%2C+pages+449%E2%80%93454%2E) [our card](https://md.research.elamur.ai/free/en/papers/map-2006-marneffe-generating-typed-dependency-parses-phrase/)
8. Ralph Grishman, David Westbrook, and Adam Meyers. 2005. Nyu’s english ace 2005 system description. In Proceedings of ACE 2005 Evaluation Workshop. Washington. [find](https://scholar.google.com/scholar?q=Ralph+Grishman%2C+David+Westbrook%2C+and+Adam+Meyers%2E+2005%2E+Nyu%E2%80%99s+english+ace+2005+system+description%2E+In+Proceedings+of+ACE+2005+Evaluation+Workshop%2E+Washington%2E)
9. Yu Hong, Jianfeng Zhang, Bin Ma, Jian-Min Yao, Guodong Zhou, and Qiaoming Zhu. 2011. Using cross-entity inference to improve event extraction. In Proceedings of ACL, pages 1127–1136. [find](https://scholar.google.com/scholar?q=Yu+Hong%2C+Jianfeng+Zhang%2C+Bin+Ma%2C+Jian%2DMin+Yao%2C+Guodong+Zhou%2C+and+Qiaoming+Zhu%2E+2011%2E+Using+cross%2Dentity+inference+to+improve+event+extraction%2E+In+Proceedings+of+ACL%2C+pages+1127%E2%80%931136%2E) [our card](https://md.research.elamur.ai/free/en/papers/hong-2011-cross-entity-inference-improve-event/)
10. Liang Huang, Suphan Fayong, and Yang Guo. 2012. Structured perceptron with inexact search. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 142–151. [find](https://scholar.google.com/scholar?q=Liang+Huang%2C+Suphan+Fayong%2C+and+Yang+Guo%2E+2012%2E+Structured+perceptron+with+inexact+search%2E+In+Proceedings+of+the+2012+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Ling) [our card](https://md.research.elamur.ai/free/en/papers/huang-2012-structured-perceptron-inexact-search/)
11. Heng Ji and Ralph Grishman. 2008. Refining event extraction through cross-document inference. In Proceedings of ACL, pages 254–262. [find](https://scholar.google.com/scholar?q=Heng+Ji+and+Ralph+Grishman%2E+2008%2E+Refining+event+extraction+through+cross%2Ddocument+inference%2E+In+Proceedings+of+ACL%2C+pages+254%E2%80%93262%2E) [our card](https://md.research.elamur.ai/free/en/papers/ji-2008-refining-event-extraction-through-cross/)
12. Peifeng Li, Guodong Zhou, Qiaoming Zhu, and Libin Hou. 2012a. Employing compositional semantics and discourse consistency in chinese event extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 10… [find](https://scholar.google.com/scholar?q=Peifeng+Li%2C+Guodong+Zhou%2C+Qiaoming+Zhu%2C+and+Libin+Hou%2E+2012a%2E+Employing+compositional+semantics+and+discourse+consistency+in+chinese+event+extraction%2E+In+Proceedings+of+the+2012+Joint+Conference+on+Em)
13. Qi Li, Haibo Li, Heng Ji, Wen Wang, Jing Zheng, and Fei Huang. 2012b. Joint bilingual name tagging for parallel corpora. In Proceedings of the 21st ACM international conference on Information and knowledge management, pages 1727–1731. [find](https://scholar.google.com/scholar?q=Qi+Li%2C+Haibo+Li%2C+Heng+Ji%2C+Wen+Wang%2C+Jing+Zheng%2C+and+Fei+Huang%2E+2012b%2E+Joint+bilingual+name+tagging+for+parallel+corpora%2E+In+Proceedings+of+the+21st+ACM+international+conference+on+Information+and+know)
14. Shasha Liao and Ralph Grishman. 2010. Using document level cross-event inference to improve event extraction. In Proceedings of ACL, pages 789–797. [find](https://scholar.google.com/scholar?q=Shasha+Liao+and+Ralph+Grishman%2E+2010%2E+Using+document+level+cross%2Devent+inference+to+improve+event+extraction%2E+In+Proceedings+of+ACL%2C+pages+789%E2%80%93797%2E) [our card](https://md.research.elamur.ai/free/en/papers/liao-2010-document-level-cross-event-inference/)
15. Catherine Macleod, Ralph Grishman, Adam Meyers, Leslie Barrett, and Ruth Reeves. 1998. Nomlex: A lexicon of nominalizations. In Proceedings of EU-RALEX, volume 98, pages 187–193. [find](https://scholar.google.com/scholar?q=Catherine+Macleod%2C+Ralph+Grishman%2C+Adam+Meyers%2C+Leslie+Barrett%2C+and+Ruth+Reeves%2E+1998%2E+Nomlex%3A+A+lexicon+of+nominalizations%2E+In+Proceedings+of+EU%2DRALEX%2C+volume+98%2C+pages+187%E2%80%93193%2E)
16. David McClosky, Mihai Surdeanu, and Christopher D. Manning. 2011. Event extraction as dependency parsing. In Proceedings of ACL, pages 1626–1635. [find](https://scholar.google.com/scholar?q=David+McClosky%2C+Mihai+Surdeanu%2C+and+Christopher+D%2E+Manning%2E+2011%2E+Event+extraction+as+dependency+parsing%2E+In+Proceedings+of+ACL%2C+pages+1626%E2%80%931635%2E)
17. Scott Miller, Jethran Guinness, and Alex Zamanian. 2004. Name tagging with word clusters and discriminative training. In Proceedings of HLT-NAACL, volume 4, pages 337–342. [find](https://scholar.google.com/scholar?q=Scott+Miller%2C+Jethran+Guinness%2C+and+Alex+Zamanian%2E+2004%2E+Name+tagging+with+word+clusters+and+discriminative+training%2E+In+Proceedings+of+HLT%2DNAACL%2C+volume+4%2C+pages+337%E2%80%93342%2E) [our card](https://md.research.elamur.ai/free/en/papers/miller-2004-name-tagging-word-clusters-discriminative/)
18. Siddharth Patwardhan and Ellen Riloff. 2009. A unified model of phrasal and sentential evidence for information extraction. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1-Volume 1, pages 151– 160. [find](https://scholar.google.com/scholar?q=Siddharth+Patwardhan+and+Ellen+Riloff%2E+2009%2E+A+unified+model+of+phrasal+and+sentential+evidence+for+information+extraction%2E+In+Proceedings+of+the+2009+Conference+on+Empirical+Methods+in+Natural+Langua)
19. Sebastian Riedel and Andrew McCallum. 2011a. Fast and robust joint models for biomedical event extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1– 12. [find](https://scholar.google.com/scholar?q=Sebastian+Riedel+and+Andrew+McCallum%2E+2011a%2E+Fast+and+robust+joint+models+for+biomedical+event+extraction%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+1)
20. Sebastian Riedel and Andrew McCallum. 2011b. Robust biomedical event extraction with dual decomposition and minimal domain adaptation. In Proceedings of the BioNLP Shared Task 2011 Workshop, pages 46–50. [find](https://scholar.google.com/scholar?q=Sebastian+Riedel+and+Andrew+McCallum%2E+2011b%2E+Robust+biomedical+event+extraction+with+dual+decomposition+and+minimal+domain+adaptation%2E+In+Proceedings+of+the+BioNLP+Shared+Task+2011+Workshop%2C+pages+46%E2%80%93)
21. Sebastian Riedel, Hong-Woo Chun, Toshihisa Takagi, and Jun’ichi Tsujii. 2009. A markov logic approach to bio-molecular event extraction. In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing: Shared Task, pages 41–49. [find](https://scholar.google.com/scholar?q=Sebastian+Riedel%2C+Hong%2DWoo+Chun%2C+Toshihisa+Takagi%2C+and+Jun%E2%80%99ichi+Tsujii%2E+2009%2E+A+markov+logic+approach+to+bio%2Dmolecular+event+extraction%2E+In+Proceedings+of+the+Workshop+on+Current+Trends+in+Biomedical+)
22. Ang Sun, Ralph Grishman, and Satoshi Sekine. 2011. Semi-supervised relation extraction with large-scale word clustering. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 521–529. [find](https://scholar.google.com/scholar?q=Ang+Sun%2C+Ralph+Grishman%2C+and+Satoshi+Sekine%2E+2011%2E+Semi%2Dsupervised+relation+extraction+with+large%2Dscale+word+clustering%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Association+for+Computational+)