--- title: "Cross-Document Event Coreference — Bagga 1999" description: "Bagga and Baldwin extend their earlier cross-document person coreference system so that it handles events." authors: ["Amit Bagga", "Breck Baldwin"] year: 1999 venue: "Coreference and Its Applications" source: "https://aclanthology.org/W99-0201/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/bagga-1999-cross-document-event-coreference-annotations/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/bagga-1999-cross-document-event-coreference-annotations/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [1999](https://md.research.elamur.ai/free/papers/1999/) / Cross-Document Event Coreference: Annotations, Experiments, and Observations # Cross-Document Event Coreference: Annotations, Experiments, and Observations **Authors:** [Amit Bagga](https://md.research.elamur.ai/free/authors/amit-bagga/), [Breck Baldwin](https://md.research.elamur.ai/free/authors/breck-baldwin/) · **Year:** 1999 · **Venue:** Coreference and Its Applications · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W99-0201/) · [PDF](https://aclanthology.org/W99-0201.pdf) · [OpenAlex](https://openalex.org/W7095400930) **Citations:** 0 citations (openalex-snapshot · 2026-10-14) · [29](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Bagga and Baldwin extend their earlier cross-document person coreference system so that it handles events. Given only a coarse event class, such as resignations, elections or espionage, the system groups documents into equivalence classes. Two documents share a class when they describe the same real-world event, meaning the same actors, organization and time. The authors argue this differs from Information Extraction and TDT because the input is minimal: candidate documents are found simply by matching a keyword substring. CAMP does not build within-document chains for events. So the system creates an event "summary" from each document's sentences that contain the event verb or its nominalizations. Optionally it also uses a few chosen synonyms. A vector space model then compares these summaries and merges those whose similarity exceeds a threshold. Output is scored with both the MUC scorer and the authors' B-CUBED metric. F-measures range from about 45% to 90%. Elections proved hardest because separate elections share places, offices, people and issues. A two-annotator study showed high human agreement, which suggests the task is well defined and gives an upper bound for systems. ## Key points - **Pipeline:** within-document coreference from Penn's CAMP, then a SentenceExtractor that builds per-document summaries, then VSM-Disambiguate, which applies vector space similarity with a threshold. - **Data sets:** the earlier "John Smith" person data set, plus event data sets for "resign", "elections" and "espionage". - **Results:** F-measure is roughly 45–90% across data sets and scorers. - Adding synonym sentences improved the elections results by about 3 F-points. - One chain about the 1996 US elections covered 36 of the 73 election articles. - **Scoring:** the authors map cross-document scoring onto a within-document problem using a "meta document". They also introduce the B-CUBED scorer to address weaknesses of the MUC scorer. - **Interannotator agreement on elections:** - MUC scorer: 87% precision and recall when annotated independently, rising to 95% after adjudication. - B-CUBED scorer: 80% independent, 93% after adjudication. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) Topics: [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/) ## Cited by Cited by 29 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 16 not yet. - [Detecting Subevent Structure for Event Coreference Resolution](https://md.research.elamur.ai/free/papers/araki-2014-detecting-subevent-structure-event-coreference/) - [Unsupervised Event Coreference Resolution with Rich Linguistic Features](https://md.research.elamur.ai/free/papers/bejan-2010-unsupervised-event-coreference-resolution-rich/) - [Unsupervised Event Coreference Resolution](https://md.research.elamur.ai/free/papers/bejan-2014-unsupervised-event-coreference-resolution/) - [Graph-based Event Coreference Resolution](https://md.research.elamur.ai/free/papers/chen-2009-graph-based-event-coreference-resolution/) - [A Pairwise Event Coreference Model, Feature Impact and Evaluation for Event Coreference Resolution](https://md.research.elamur.ai/free/papers/chen-2009-pairwise-event-coreference-model-feature/) - [Resolving Event Coreference with Supervised Representation Learning and Clustering-Oriented Regularization](https://md.research.elamur.ai/free/papers/dean-2018-resolving-event-coreference-supervised-representation/) - [NPs for Events: Experiments in Coreference Annotation](https://md.research.elamur.ai/free/papers/hasler-2006-nps-events-experiments-coreference-annotation/) - [Joint Entity and Event Coreference Resolution across Documents](https://md.research.elamur.ai/free/papers/lee-2012-joint-entity-event-coreference-resolution/) - [Supervised Within-Document Event Coreference using Information Propagation](https://md.research.elamur.ai/free/papers/liu-2014-supervised-within-document-event-coreference/) - [Multimodal Cross-Document Event Coreference Resolution Using Linear Semantic Transfer and Mixed-Modality Ensembles](https://md.research.elamur.ai/free/papers/nath-2024-multimodal-cross-document-event-coreference/) - [Event Detection and Co-reference with Minimal Supervision](https://md.research.elamur.ai/free/papers/peng-2016-event-detection-co-reference-minimal/) - [Revisiting the Evaluation for Cross Document Event Coreference](https://md.research.elamur.ai/free/papers/upadhyay-2016-revisiting-evaluation-cross-document-event/) - [A Discriminative Hierarchical Model for Fast Coreference at Large Scale](https://md.research.elamur.ai/free/papers/wick-2012-discriminative-hierarchical-model-fast-coreference/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{bagga1999cross, title = {{Cross-Document Event Coreference: Annotations, Experiments, and Observations}}, author = {Bagga, Amit and Baldwin, Breck}, booktitle = {Coreference and Its Applications}, year = {1999}, url = {https://aclanthology.org/W99-0201/} } ``` - [← previous Corpus-Based Identification of Non-Anaphoric Noun Phrases](https://md.research.elamur.ai/free/papers/bean-1999-corpus-based-identification-non-anaphoric/) - [next → Discourse Relations: A Structural and Presuppositional Account Using Lexicalised TAG](https://md.research.elamur.ai/free/papers/webber-1999-discourse-relations-structural-presuppositional-account/) - [↑ back to papers of 1999](https://md.research.elamur.ai/free/papers/1999/) ## References 7 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Bagga, Amit, and Breck Baldwin. Algorithms for Scoring Coreference Chains. Proceedings of the Linguistic Coreference Workshop at The First International Conference on Language Resources and Evaluation, May 1998. [find](https://scholar.google.com/scholar?q=Bagga%2C+Amit%2C+and+Breck+Baldwin%2E+Algorithms+for+Scoring+Coreference+Chains%2E+Proceedings+of+the+Linguistic+Coreference+Workshop+at+The+First+International+Conference+on+Language+Resources+and+Evaluation) [our card](https://md.research.elamur.ai/free/papers/bagga-1998-algorithms-scoring-coreference-chains/) 2. Bagga, Amit, and Breck Baldwin. How Much Processing is Required for Cross-Document Coreference? Proceedings of the Linguistic Coreference Workshop at The First International Conference on Language Resources and Evaluation, May 1998. [find](https://scholar.google.com/scholar?q=Bagga%2C+Amit%2C+and+Breck+Baldwin%2E+How+Much+Processing+is+Required+for+Cross%2DDocument+Coreference%3F+Proceedings+of+the+Linguistic+Coreference+Workshop+at+The+First+International+Conference+on+Language+Res) 3. Bagga, Amit, and Breck Baldwin. Entity-Based Cross-Document Coreferencing Using the Vector Space Model. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Con-ferende on Computational Linguistics (COLING-ACL'98), pp. 79-85, August 19… [find](https://scholar.google.com/scholar?q=Bagga%2C+Amit%2C+and+Breck+Baldwin%2E+Entity%2DBased+Cross%2DDocument+Coreferencing+Using+the+Vector+Space+Model%2E+In+Proceedings+of+the+36th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+t) 4. Bagga, Amit, and Breck Baldwin. Coreference as the Foundations for Link Analysis Over Free Text Databases. In Proceedings of the COLING-A CL'98 Content Visualization and Intermedia Representations Workshop (CVIR'98), pp. 19-24, August 1998. [find](https://scholar.google.com/scholar?q=Bagga%2C+Amit%2C+and+Breck+Baldwin%2E+Coreference+as+the+Foundations+for+Link+Analysis+Over+Free+Text+Databases%2E+In+Proceedings+of+the+COLING%2DA+CL%2798+Content+Visualization+and+Intermedia+Representations+Wor) 5. Grishman, Ralph. Whither Written Language Evaluation?, Proceedings of the Human Language Technology Workshop, pp. 120-125, March 1994, San Francisco: Morgan Kaufmann. [find](https://scholar.google.com/scholar?q=Grishman%2C+Ralph%2E+Whither+Written+Language+Evaluation%3F%2C+Proceedings+of+the+Human+Language+Technology+Workshop%2C+pp%2E+120%2D125%2C+March+1994%2C+San+Francisco%3A+Morgan+Kaufmann%2E) 6. Salton, G e r a r d . AUtomatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer, 1989, Reading, MA: Addison-Wesley. number of correct elements in the output chain containing entity~ number of elements in the output chain containing entityi Precisioni = number… [find](https://scholar.google.com/scholar?q=Salton%2C+G+e+r+a+r+d+%2E+AUtomatic+Text+Processing%3A+The+Transformation%2C+Analysis%2C+and+Retrieval+of+Information+by+Computer%2C+1989%2C+Reading%2C+MA%3A+Addison%2DWesley%2E+number+of+correct+elements+in+the+output+cha) 7. John Smith resign elections espionage 197 219 135 184 Figure 7: Analysis of the Data Sets Optimal Threshold 0.15 0.15 0.20 0.20 0.08 0.10 0.25 0.25 Data Set Scorer MUC B-CUBED MUC B-CUBED MUC B-CUBED MUC B-CUBED John Smith resign elections espionage Figure 16: Analysis of the Data Sets avg \# of summ… [find](https://scholar.google.com/scholar?q=John+Smith+resign+elections+espionage+197+219+135+184+Figure+7%3A+Analysis+of+the+Data+Sets+Optimal+Threshold+0%2E15+0%2E15+0%2E20+0%2E20+0%2E08+0%2E10+0%2E25+0%2E25+Data+Set+Scorer+MUC+B%2DCUBED+MUC+B%2DCUBED+MUC+B%2DCUBED+)