---
title: "New Insights into Cross-Document Event — Cremisini 2020"
description: "Cross-Document Event Coreference (CDEC) links mentions of the same real-world event across different documents."
authors: ["Andres Cremisini", "Mark Finlayson"]
year: 2020
venue: "Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events"
source: "https://aclanthology.org/2020.nuse-1.1/"
doi: "10.18653/v1/2020.nuse-1.1"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/cremisini-2020-new-insights-into-cross-document/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/cremisini-2020-new-insights-into-cross-document/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2020](https://md.research.elamur.ai/free/papers/2020/page/3/) / New Insights into Cross-Document Event Coreference: Systematic Comparison and a Simplified Approach

# New Insights into Cross-Document Event Coreference: Systematic Comparison and a Simplified Approach

**Authors:** Andres Cremisini, [Mark Finlayson](https://md.research.elamur.ai/free/authors/mark-finlayson/) · **Year:** 2020 · **Venue:** Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.nuse-1.1/) · [PDF](https://aclanthology.org/2020.nuse-1.1.pdf) · [DOI 10.18653/v1/2020.nuse-1.1](https://doi.org/10.18653/v1/2020.nuse-1.1) · [OpenAlex](https://openalex.org/W3045916626)

**Citations:** 8 citations (openalex-snapshot · 2026-10-14) · [5](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Cross-Document Event Coreference (CDEC) links mentions of the same real-world event across different documents. It is usually evaluated on the ECB+ corpus. The authors reviewed prior systems and found them hard to compare for three reasons: some use gold labels and others computed ones, document clustering is handled differently, and test sets are chosen arbitrarily. They then built a deliberately simple and explainable pipeline modelled on the Bag of Events system by Vossen and Cybulska (2018). It uses only event trigger annotations, a compact pairwise feature set, document clustering that does not rely on gold annotations, and a new event clustering step. Every pipeline stage was evaluated separately. The system matches or slightly surpasses the best trigger-only results on ECB+ despite being much simpler. The work also offers concrete evaluation practices, so future CDEC results can be compared more fairly.

## Key points
- The pairwise event coreference classifier uses 4 features. Bag of Events uses 19, of which 5 come from triggers and 14 from entities.
- Experiments use the verified ECB+ subset, which has 1,840 sentences, 5,726 events and 897 coreference chains with a mean length of 5.5. The full corpus covers 982 documents across 43 topics.
- This is the first cross-validated CDEC evaluation on ECB+, using 5-fold CV. It is also the first explicit evaluation of the pairwise classification step.
- Better document clustering can raise CDEC scores by at most about 3 points. This may reflect how easy it is to cluster ECB+ documents.
- The authors recommend reporting results both with and without gold document clusters. They also note that tuning the pairwise classifier for F0.8 rather than F1 gives the best downstream CDEC performance.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/)

## Cited by

Cited by

5 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 1 not yet.

- [Generalizing Cross-Document Event Coreference Resolution Across Multiple Corpora](https://md.research.elamur.ai/free/papers/bugert-2021-generalizing-cross-document-event-coreference/)
- [Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference](https://md.research.elamur.ai/free/papers/held-2021-focus-what-matters-applying-discourse/)
- [Contrastive Representation Learning for Cross-Document Coreference Resolution of Events and Entities](https://md.research.elamur.ai/free/papers/hsu-2022-contrastive-representation-learning-cross-document/)
- [Okay, Let's Do This! Modeling Event Coreference with Generated Rationales and Knowledge Distillation](https://md.research.elamur.ai/free/papers/nath-2024-okay-let-s-do-this/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{cremisini2020new,
  title = {{New Insights into Cross-Document Event Coreference: Systematic Comparison and a Simplified Approach}},
  author = {Cremisini, Andres and Finlayson, Mark},
  booktitle = {Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events},
  year = {2020},
  doi = {10.18653/v1/2020.nuse-1.1},
  url = {https://doi.org/10.18653/v1/2020.nuse-1.1}
}
```

- [← previous Neural Syntactic Preordering for Controlled Paraphrase Generation](https://md.research.elamur.ai/free/papers/goyal-2020-neural-syntactic-preordering-controlled-paraphrase/)
- [next → NoEl: An Annotated Corpus for Noun Ellipsis in English](https://md.research.elamur.ai/free/papers/gram-2020-khullar-noel-annotated-corpus-noun-ellipsis/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/papers/2020/page/3/)

## References

18 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shany Barhom, Vered Shwartz, Alon Eirew, Michael Bugert, Nils Reimers, and Ido Dagan. 2019. Revisiting Joint Modeling of Cross-document Entity and Event Coreference Resolution. arXiv, 2. [find](https://scholar.google.com/scholar?q=Shany+Barhom%2C+Vered+Shwartz%2C+Alon+Eirew%2C+Michael+Bugert%2C+Nils+Reimers%2C+and+Ido+Dagan%2E+2019%2E+Revisiting+Joint+Modeling+of+Cross%2Ddocument+Entity+and+Event+Coreference+Resolution%2E+arXiv%2C+2%2E) [our card](https://md.research.elamur.ai/free/papers/barhom-2019-revisiting-joint-modeling-cross-document/)
2. Jing Lu and Vincent Ng. 2017. Joint Learning for Event Coreference Resolution. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 90–101. [find](https://scholar.google.com/scholar?q=Jing+Lu+and+Vincent+Ng%2E+2017%2E+Joint+Learning+for+Event+Coreference+Resolution%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers%29%2C+pages)
3. Cosmin Bejan and Sanda Harabagiu. 2010. Unsupervised Event Coreference Resolution with Rich Linguistic Features. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1412–1422. [find](https://scholar.google.com/scholar?q=Cosmin+Bejan+and+Sanda+Harabagiu%2E+2010%2E+Unsupervised+Event+Coreference+Resolution+with+Rich+Linguistic+Features%2E+Proceedings+of+the+48th+Annual+Meeting+of+the+Association+for+Computational+Linguistics) [our card](https://md.research.elamur.ai/free/papers/bejan-2010-unsupervised-event-coreference-resolution-rich/)
4. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25–32, Vancouver, Canada. [find](https://scholar.google.com/scholar?q=Xiaoqiang+Luo%2E+2005%2E+On+coreference+resolution+performance+metrics%2E+In+Proceedings+of+Human+Language+Technology+Conference+and+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+25%E2%80%93) [our card](https://md.research.elamur.ai/free/papers/luo-2005-coreference-resolution-performance-metrics/)
5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606. [find](https://scholar.google.com/scholar?q=Piotr+Bojanowski%2C+Edouard+Grave%2C+Armand+Joulin%2C+and+Tomas+Mikolov%2E+2016%2E+Enriching+word+vectors+with+subword+information%2E+arXiv+preprint+arXiv%3A1607%2E04606%2E)
6. Nathanael Chambers, Taylor Cassidy, Bill McDowell, and Steven Bethard. 2014. Dense Event Ordering with a Multi-Pass Architecture. Transactions of the Association for Computational Linguistics, 2:273– 284. [find](https://scholar.google.com/scholar?q=Nathanael+Chambers%2C+Taylor+Cassidy%2C+Bill+McDowell%2C+and+Steven+Bethard%2E+2014%2E+Dense+Event+Ordering+with+a+Multi%2DPass+Architecture%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+2%3A273%E2%80%93+)
7. Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J Bethard, and David Mc-Closky. 2014. The {Stanford} {CoreNLP} Natural Language Processing Toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60. [find](https://scholar.google.com/scholar?q=Christopher+D+Manning%2C+Mihai+Surdeanu%2C+John+Bauer%2C+Jenny+Finkel%2C+Steven+J+Bethard%2C+and+David+Mc%2DClosky%2E+2014%2E+The+%7BStanford%7D+%7BCoreNLP%7D+Natural+Language+Processing+Toolkit%2E+In+Association+for+Computati) [our card](https://md.research.elamur.ai/free/papers/manning-2014-stanford-corenlp-natural-language-processing/)
8. Zheng Chen and H Ji. 2009. Graph-based event coreference resolution. Proceedings of the 2009 Workshop on Graph-based Methods for Natural Language Processing, pages 54–57. [find](https://scholar.google.com/scholar?q=Zheng+Chen+and+H+Ji%2E+2009%2E+Graph%2Dbased+event+coreference+resolution%2E+Proceedings+of+the+2009+Workshop+on+Graph%2Dbased+Methods+for+Natural+Language+Processing%2C+pages+54%E2%80%9357%2E) [our card](https://md.research.elamur.ai/free/papers/chen-2009-graph-based-event-coreference-resolution/)
9. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. CoRR, abs/1802.05365. [find](https://scholar.google.com/scholar?q=Matthew+E%2E+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E+CoRR%2C+abs%2F1802%2E05365%2E)
10. Prafulla Kumar Choubey and Ruihong Huang. 2017. Event Coreference Resolution by Iteratively Unfolding Inter-dependencies among Events. arXiv. [find](https://scholar.google.com/scholar?q=Prafulla+Kumar+Choubey+and+Ruihong+Huang%2E+2017%2E+Event+Coreference+Resolution+by+Iteratively+Unfolding+Inter%2Ddependencies+among+Events%2E+arXiv%2E) [our card](https://md.research.elamur.ai/free/papers/choubey-2017-event-coreference-resolution-iteratively-unfolding/)
11. Scikit-Learn. 2019. Scikit-learn probability calibration. [https://scikit-learn.org/stable/](https://scikit-learn.org/stable/) modules/calibration.html . Accessed: 2019-11-23. [↗](https://scikit-learn.org/stable/)
12. Agata Cybulska and Piek Vossen. 2014. Using a sledgehammer to crack a nut? Lexical diversity and event coreference resolution. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4545– 4552. [find](https://scholar.google.com/scholar?q=Agata+Cybulska+and+Piek+Vossen%2E+2014%2E+Using+a+sledgehammer+to+crack+a+nut%3F+Lexical+diversity+and+event+coreference+resolution%2E+Proceedings+of+the+Ninth+International+Conference+on+Language+Resources+a) [our card](https://md.research.elamur.ai/free/papers/cybulska-2014-sledgehammer-crack-nut-lexical-diversity/)
13. Emma Strubell, Pat Verga, David Belanger, and Andrew McCallum. 2017. Fast and accurate entity recognition with iterated dilated convolutions. In EMNLP. [find](https://scholar.google.com/scholar?q=Emma+Strubell%2C+Pat+Verga%2C+David+Belanger%2C+and+Andrew+McCallum%2E+2017%2E+Fast+and+accurate+entity+recognition+with+iterated+dilated+convolutions%2E+In+EMNLP%2E)
14. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extraction (ace) program-tasks, data, and evaluation. Proceedings of LREC, 2. [find](https://scholar.google.com/scholar?q=George+Doddington%2C+Alexis+Mitchell%2C+Mark+Przybocki%2C+Lance+Ramshaw%2C+Stephanie+Strassel%2C+and+Ralph+Weischedel%2E+2004%2E+The+automatic+content+extraction+%28ace%29+program%2Dtasks%2C+data%2C+and+evaluation%2E+Proceedin) [our card](https://md.research.elamur.ai/free/papers/doddington-2004-automatic-content-extraction-ace-program/)
15. Piek Vossen and Agata Cybulska. 2018. Identity and Granularity of Events in Text. In Lecture Notes in Computer Science, volume 9624 LNCS, pages 501– 522. Springer-Verlag. [find](https://scholar.google.com/scholar?q=Piek+Vossen+and+Agata+Cybulska%2E+2018%2E+Identity+and+Granularity+of+Events+in+Text%2E+In+Lecture+Notes+in+Computer+Science%2C+volume+9624+LNCS%2C+pages+501%E2%80%93+522%2E+Springer%2DVerlag%2E)
16. Brendan J. Frey and Delbert Dueck. 2007. Clustering by passing messages between data points. Science, 315(5814):972–976. [find](https://scholar.google.com/scholar?q=Brendan+J%2E+Frey+and+Delbert+Dueck%2E+2007%2E+Clustering+by+passing+messages+between+data+points%2E+Science%2C+315%285814%29%3A972%E2%80%93976%2E)
17. Bishan Yang, Claire Cardie, and Peter Frazier. 2015. A Hierarchical Distance-dependent Bayesian Model for Event Coreference Resolution. arXiv. [find](https://scholar.google.com/scholar?q=Bishan+Yang%2C+Claire+Cardie%2C+and+Peter+Frazier%2E+2015%2E+A+Hierarchical+Distance%2Ddependent+Bayesian+Model+for+Event+Coreference+Resolution%2E+arXiv%2E)
18. Sen Yang, Dawei Feng, Linbo Qiao, Zhigang Kan, and Dongsheng Li. 2019. Exploring Pre-trained Language Models for Event Extraction and Generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5284–5294, Florence, Italy. Association for Computationa… [find](https://scholar.google.com/scholar?q=Sen+Yang%2C+Dawei+Feng%2C+Linbo+Qiao%2C+Zhigang+Kan%2C+and+Dongsheng+Li%2E+2019%2E+Exploring+Pre%2Dtrained+Language+Models+for+Event+Extraction+and+Generation%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Assoc)