--- title: "Refining Event Extraction through Cross-Document — Ji 2008" description: "Sentence-level event extractors judge each event mention in isolation." authors: ["Heng Ji", "Ralph Grishman"] year: 2008 venue: "Proceedings of ACL-08: HLT" source: "https://aclanthology.org/P08-1030/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ji-2008-refining-event-extraction-through-cross/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ji-2008-refining-event-extraction-through-cross/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2008](https://md.research.elamur.ai/free/en/papers/2008/) / Refining Event Extraction through Cross-Document Inference # Refining Event Extraction through Cross-Document Inference **Authors:** [Heng Ji](https://md.research.elamur.ai/free/en/authors/heng-ji/), [Ralph Grishman](https://md.research.elamur.ai/free/en/authors/ralph-grishman/) · **Year:** 2008 · **Venue:** Proceedings of ACL-08: HLT · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/P08-1030/) · [PDF](https://aclanthology.org/P08-1030.pdf) · [OpenAlex](https://openalex.org/W2165516035) **Citations:** 386 citations (openalex-snapshot · 2026-10-14) · [191](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Sentence-level event extractors judge each event mention in isolation. As a result, they miss triggers that appear in unfamiliar contexts and accept spurious ones that happen to match a pattern. Ji and Grishman apply Yarowsky's "One Sense Per Discourse" idea to ACE event extraction and widen the "discourse" from one document to a cluster of topically related documents. The pipeline works in four steps: - A baseline English IE system (Grishman et al., 2005), which combines patterns with Maximum Entropy classifiers, tags each test document and assigns confidences. - An INDRI query built from the detected triggers and arguments retrieves related documents from unlabeled corpora. - The same baseline tags those related documents. - Weighted frequency and margin statistics are pooled within the document and across the cluster. A cascade of threshold rules then deletes weak mentions, spreads the dominant event type or role, and adds missed triggers and arguments. Without any additional labeled data, the method raises F-measure by 7.6% for trigger labeling and by 6% for argument labeling. The work shows that redundancy across related texts can act as free, dynamically built background knowledge for IE. ## Key points - The task is ACE 2005 event extraction, with 8 event types and 33 subtypes treated as 33 separate types. Event mention coreference is not handled. - The baseline extracts events sentence by sentence. It uses trigger-to-argument patterns plus MaxEnt classifiers for trigger labeling, argument detection, role assignment and reportable-event decisions. - For each test text, the top N=25 related documents are retrieved with INDRI. The query weights triggers and arguments by local confidence and frequency, and it is expanded with coreferent or ACE-related names. - Error analysis motivates the method. For example, "fire" is an End-Position trigger in only 7% of 81 training occurrences but in 100% of the occurrences in test plus related documents. Two hypotheses follow: "one trigger sense per cluster" and "one argument role per cluster". - Inference uses nine rules with thresholds δ1–δ13. The rules first act within the document and then across the cluster. ## Interesting ideas - **Retrieve your own context at test time.** Fetching documents on the same topic and running the same tagger over them creates a topic-specific distribution of labels. This kind of self-supervision could help any low-resource extractor, for example a Ukrainian NER or event tagger with little annotated data. - **Let easy mentions help hard ones.** A phrasing that matches a known pattern with high confidence in one document can justify labeling an ambiguous phrasing of the same event elsewhere. The confident evidence can also veto a locally confident but globally rare label. - **Use margins as well as counts.** The method propagates a label only when the top type or role clearly beats the runner-up. This guards against spreading errors when the evidence is split, and the same safeguard suits any label-propagation or majority-vote scheme. - **Cascade from local to global.** Rules apply first within the sentence and document, and then across the cluster, with cluster-level decisions overriding document-level ones on conflict. This ordering is a simple, interpretable alternative to joint inference. - **Reuse high-confidence output.** The authors suggest using consistent, high-confidence cluster results as bootstrapped training data. The same aggregation could also produce event-driven summaries or coreference hints. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 191 papers of our library cite it (of the 140884 whose bibliographies we read); 23 of them have a page here, 168 not yet. - [Joint Event Trigger Identification and Event Coreference Resolution with Structured Perceptron](https://md.research.elamur.ai/free/en/papers/araki-2015-joint-event-trigger-identification-event/) - [A Pairwise Event Coreference Model, Feature Impact and Evaluation for Event Coreference Resolution](https://md.research.elamur.ai/free/en/papers/chen-2009-pairwise-event-coreference-model-feature/) - [Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/chen-2015-event-extraction-dynamic-multi-pooling/) - [Event Extraction by Answering (Almost) Natural Questions](https://md.research.elamur.ai/free/en/papers/du-2020-event-extraction-answering-almost-natural/) - [A Language-Independent Neural Network for Event Detection](https://md.research.elamur.ai/free/en/papers/feng-2016-language-independent-neural-network-event/) - [Dual Cache for Long Document Neural Coreference Resolution](https://md.research.elamur.ai/free/en/papers/guo-2023-dual-cache-long-document-neural/) - [Using Cross-Entity Inference to Improve Event Extraction](https://md.research.elamur.ai/free/en/papers/hong-2011-cross-entity-inference-improve-event/) - [DEGREE: A Data-Efficient Generation-Based Event Extraction Model](https://md.research.elamur.ai/free/en/papers/hsu-2022-degree-data-efficient-generation-based/) - [Zero-Shot Transfer Learning for Event Extraction](https://md.research.elamur.ai/free/en/papers/huang-2018-zero-shot-transfer-learning-event/) - [Joint Event Extraction via Structured Prediction with Global Features](https://md.research.elamur.ai/free/en/papers/li-2013-joint-event-extraction-structured-prediction/) - [Event Extraction as Multi-turn Question Answering](https://md.research.elamur.ai/free/en/papers/li-2020-event-extraction-as-multi-turn/) - [Document-Level Event Argument Extraction by Conditional Generation](https://md.research.elamur.ai/free/en/papers/li-2021-document-level-event-argument-extraction/) - [Using Document Level Cross-Event Inference to Improve Event Extraction](https://md.research.elamur.ai/free/en/papers/liao-2010-document-level-cross-event-inference/) - [Leveraging FrameNet to Improve Automatic Event Detection](https://md.research.elamur.ai/free/en/papers/liu-2016-leveraging-framenet-improve-automatic-event/) - [Exploiting Argument Information to Improve Event Detection via Supervised Attention Mechanisms](https://md.research.elamur.ai/free/en/papers/liu-2017-exploiting-argument-information-improve-event/) - [Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation](https://md.research.elamur.ai/free/en/papers/liu-2018-jointly-multiple-events-extraction-attention/) - [Event Extraction as Machine Reading Comprehension](https://md.research.elamur.ai/free/en/papers/liu-2020-event-extraction-as-machine-reading/) - [Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction](https://md.research.elamur.ai/free/en/papers/lu-2021-text2event-controllable-sequence-structure-generation/) - [Event Detection and Domain Adaptation with Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/nguyen-2015-event-detection-domain-adaptation-convolutional/) - [Joint Event Extraction via Recurrent Neural Networks](https://md.research.elamur.ai/free/en/papers/nguyen-2016-joint-event-extraction-recurrent-neural/) - [Event Detection and Co-reference with Minimal Supervision](https://md.research.elamur.ai/free/en/papers/peng-2016-event-detection-co-reference-minimal/) - [MAVEN: A Massive General Domain Event Detection Dataset](https://md.research.elamur.ai/free/en/papers/wang-2020-maven-massive-general-domain-event/) - [Joint Extraction of Events and Entities within a Document Context](https://md.research.elamur.ai/free/en/papers/yang-2016-joint-extraction-events-entities-within/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{ji2008refining, title = {{Refining Event Extraction through Cross-Document Inference}}, author = {Ji, Heng and Grishman, Ralph}, booktitle = {Proceedings of ACL-08: HLT}, year = {2008}, url = {https://aclanthology.org/P08-1030/} } ``` - [← previous Phrase-Based and Deep Syntactic English-to-Czech Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/bojar-2008-phrase-based-deep-syntactic-english/) - [next → Reusable Tagset Conversion Using Tagset Drivers](https://md.research.elamur.ai/free/en/papers/zeman-2008-reusable-tagset-conversion-tagset-drivers/) - [↑ back to papers of 2008](https://md.research.elamur.ai/free/en/papers/2008/) ## References 14 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. David Ahn. 2006. The stages of event extraction. Proc. COLING/ACL 2006 Workshop on Annotating and Reasoning about Time and Events. Sydney, Australia. [find](https://scholar.google.com/scholar?q=David+Ahn%2E+2006%2E+The+stages+of+event+extraction%2E+Proc%2E+COLING%2FACL+2006+Workshop+on+Annotating+and+Reasoning+about+Time+and+Events%2E+Sydney%2C+Australia%2E) [our card](https://md.research.elamur.ai/free/en/papers/ahn-2006-stages-event-extraction/) 2. Ralph Grishman, David Westbrook and Adam Meyers. 2005. [find](https://scholar.google.com/scholar?q=Ralph+Grishman%2C+David+Westbrook+and+Adam+Meyers%2E+2005%2E) 3. NYU’s English ACE 2005 System Description. [find](https://scholar.google.com/scholar?q=NYU%E2%80%99s+English+ACE+2005+System+Description%2E) 4. Proc. ACE 2005 Evaluation Workshop. Washington, US. [find](https://scholar.google.com/scholar?q=Proc%2E+ACE+2005+Evaluation+Workshop%2E+Washington%2C+US%2E) 5. Hilda Hardy, Vika Kanchakouskaya and Tomek Strzalkowski. 2006. Automatic Event Classification Using Surface Text Features. Proc. AAAI06 Workshop on Event Extraction and Synthesis. Boston, Massachusetts. US. [find](https://scholar.google.com/scholar?q=Hilda+Hardy%2C+Vika+Kanchakouskaya+and+Tomek+Strzalkowski%2E+2006%2E+Automatic+Event+Classification+Using+Surface+Text+Features%2E+Proc%2E+AAAI06+Workshop+on+Event+Extraction+and+Synthesis%2E+Boston%2C+Massachusett) 6. Gideon Mann. 2007. Multi-document Relationship Fusion via Constraints on Probabilistic Databases. Proc. HLT/NAACL 2007. Rochester, NY, US. [find](https://scholar.google.com/scholar?q=Gideon+Mann%2E+2007%2E+Multi%2Ddocument+Relationship+Fusion+via+Constraints+on+Probabilistic+Databases%2E+Proc%2E+HLT%2FNAACL+2007%2E+Rochester%2C+NY%2C+US%2E) 7. Siddharth Patwardhan and Ellen Riloff. 2007. Effective Information Extraction with Semantic Affinity Patterns and Relevant Regions. [find](https://scholar.google.com/scholar?q=Siddharth+Patwardhan+and+Ellen+Riloff%2E+2007%2E+Effective+Information+Extraction+with+Semantic+Affinity+Patterns+and+Relevant+Regions%2E) 8. Proc. EMNLP 2007. Prague, Czech Republic. [find](https://scholar.google.com/scholar?q=Proc%2E+EMNLP+2007%2E+Prague%2C+Czech+Republic%2E) 9. Trevor Strohman, Donald Metzler, Howard Turtle and W. Bruce Croft. 2005. Indri: A Language-model based Search Engine for Complex Queries (extended version). Technical Report IR-407, CIIR, Umass Amherst, US. [find](https://scholar.google.com/scholar?q=Trevor+Strohman%2C+Donald+Metzler%2C+Howard+Turtle+and+W%2E+Bruce+Croft%2E+2005%2E+Indri%3A+A+Language%2Dmodel+based+Search+Engine+for+Complex+Queries+%28extended+version%29%2E+Technical+Report+IR%2D407%2C+CIIR%2C+Umass+Amhers) 10. Roman Yangarber, Clive Best, Peter von Etter, Flavio Fuart, David Horby and Ralf Steinberger. 2007. Combining Information about Epidemic Threats from Multiple Sources. [find](https://scholar.google.com/scholar?q=Roman+Yangarber%2C+Clive+Best%2C+Peter+von+Etter%2C+Flavio+Fuart%2C+David+Horby+and+Ralf+Steinberger%2E+2007%2E+Combining+Information+about+Epidemic+Threats+from+Multiple+Sources%2E) 11. Proc. RANLP 2007 workshop on Multi-source, Multilingual Information Extraction and Summarization. Borovets, Bulgaria. [find](https://scholar.google.com/scholar?q=Proc%2E+RANLP+2007+workshop+on+Multi%2Dsource%2C+Multilingual+Information+Extraction+and+Summarization%2E+Borovets%2C+Bulgaria%2E) 12. Roman Yangarber. 2006. Verification of Facts across Document Boundaries. Proc. International Workshop on Intelligent Information Access. Helsinki, Finland. [find](https://scholar.google.com/scholar?q=Roman+Yangarber%2E+2006%2E+Verification+of+Facts+across+Document+Boundaries%2E+Proc%2E+International+Workshop+on+Intelligent+Information+Access%2E+Helsinki%2C+Finland%2E) 13. Roman Yangarber and Lauri Jokipii. 2005. Redundancy-based Correction of Automatically Extracted Facts. Proc. HLT/EMNLP 2005. Vancouver, Canada. [find](https://scholar.google.com/scholar?q=Roman+Yangarber+and+Lauri+Jokipii%2E+2005%2E+Redundancy%2Dbased+Correction+of+Automatically+Extracted+Facts%2E+Proc%2E+HLT%2FEMNLP+2005%2E+Vancouver%2C+Canada%2E) 14. Unsupervised Word Sense Disambiguation Rivaling Supervised Methods. Proc. ACL 1995. Cambridge, MA, US. [find](https://scholar.google.com/scholar?q=Unsupervised+Word+Sense+Disambiguation+Rivaling+Supervised+Methods%2E+Proc%2E+ACL+1995%2E+Cambridge%2C+MA%2C+US%2E) [our card](https://md.research.elamur.ai/free/en/papers/yarowsky-1995-unsupervised-word-sense-disambiguation-rivaling/)