---
title: "Revisiting the Evaluation for Cross Document — Upadhyay 2016"
description: "Cross document event coreference (CDEC) asks whether event mentions in different documents describe the same real-world event."
authors: ["Shyam Upadhyay", "Nitish Gupta", "Christos Christodoulopoulos", "Dan Roth"]
year: 2016
venue: "Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers"
source: "https://aclanthology.org/C16-1183/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/upadhyay-2016-revisiting-evaluation-cross-document-event/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/upadhyay-2016-revisiting-evaluation-cross-document-event/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2016](https://md.research.elamur.ai/free/papers/2016/page/3/) / Revisiting the Evaluation for Cross Document Event Coreference

# Revisiting the Evaluation for Cross Document Event Coreference

**Authors:** [Shyam Upadhyay](https://md.research.elamur.ai/free/authors/shyam-upadhyay/), [Nitish Gupta](https://md.research.elamur.ai/free/authors/nitish-gupta/), [Christos Christodoulopoulos](https://md.research.elamur.ai/free/authors/christos-christodoulopoulos/), [Dan Roth](https://md.research.elamur.ai/free/authors/dan-roth/) · **Year:** 2016 · **Venue:** Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/C16-1183/) · [PDF](https://aclanthology.org/C16-1183.pdf) · [OpenAlex](https://openalex.org/W2578696587)

**Citations:** 18 citations (openalex-snapshot · 2026-10-14) · [12](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Cross document event coreference (CDEC) asks whether event mentions in different documents describe the same real-world event. The authors examine how CDEC has been evaluated on ECB+, the only public corpus with cross document event coreference annotations. They find that earlier evaluation protocols are inconsistent and too lenient. The B&H setting scores each topic separately, so wrong links across topics go unpunished. The YCF setting splits evaluation by sub-topic and also drops singleton mentions. Both depend on the corpus's topic layout, which a real system never sees at test time.

The paper proposes two complementary settings, SIMPLE-CDEC and PURE-CDEC. The second isolates decisions made across documents from those made inside a single document. Under these settings, a baseline that only links mentions within documents no longer looks deceptively strong, and Lemma-δ turns out to be a better baseline than plain Lemma matching. Keeping singletons in the evaluation also exposed many annotation gaps in ECB+. This matters because unreliable metrics and noisy gold data make it impossible to judge real progress on corpus-level event aggregation.

## Key points
- ECB+ statistics: 982 documents, 43 topics, 86 sub-topics and 14,874 event mentions, of which 7,605 are singletons. The authors use the train/dev/test splits of Yang et al. (2015).
- Prior settings ignore errors across topics (B&H) or across sub-topics and singletons (YCF). The authors recommend always reporting both SIMPLE-CDEC and PURE-CDEC.
- In PURE-CDEC, only systems that actually make cross document links (Lemma, Lemma-δ, SAC) get non-zero MUC and Blanc-PW scores. Lemma-WD loses about 10% average F, versus roughly 6% for Lemma-δ and SAC.
- SAC and Lemma-δ perform best overall. Lemma-δ is proposed as the standard baseline in place of Lemma.
- An anchored SVM (following Goldberg and Elhadad, 2007) with an anchor-weight threshold of 0.95, followed by manual review, uncovered over 300 missing singleton links in ECB+. The authors also note partially annotated documents and released the error list.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/)

## Cited by

Cited by

12 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 5 not yet.

- [Revisiting Joint Modeling of Cross-document Entity and Event Coreference Resolution](https://md.research.elamur.ai/free/papers/barhom-2019-revisiting-joint-modeling-cross-document/)
- [Event Coreference Data (Almost) for Free: Mining Hyperlinks from Online News](https://md.research.elamur.ai/free/papers/bugert-2021-event-coreference-data-almost-free/)
- [Generalizing Cross-Document Event Coreference Resolution Across Multiple Corpora](https://md.research.elamur.ai/free/papers/bugert-2021-generalizing-cross-document-event-coreference/)
- [Resolving Event Coreference with Supervised Representation Learning and Clustering-Oriented Regularization](https://md.research.elamur.ai/free/papers/dean-2018-resolving-event-coreference-supervised-representation/)
- [WEC: Deriving a Large-scale Cross-document Event Coreference dataset from Wikipedia](https://md.research.elamur.ai/free/papers/eirew-2021-wec-deriving-large-scale-cross/)
- [Contrastive Representation Learning for Cross-Document Coreference Resolution of Events and Entities](https://md.research.elamur.ai/free/papers/hsu-2022-contrastive-representation-learning-cross-document/)
- [What happens before and after: Multi-Event Commonsense in Event Coreference Resolution](https://md.research.elamur.ai/free/papers/ravi-2023-what-happens-before-after-multi/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{upadhyay2016revisiting,
  title = {{Revisiting the Evaluation for Cross Document Event Coreference}},
  author = {Upadhyay, Shyam and Gupta, Nitish and Christodoulopoulos, Christos and Roth, Dan},
  booktitle = {Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers},
  year = {2016},
  url = {https://aclanthology.org/C16-1183/}
}
```

- [← previous Results of the WMT16 Metrics Shared Task](https://md.research.elamur.ai/free/papers/bojar-2016-results-wmt16-metrics-shared-task/)
- [next → Richer Event Description: Integrating event coreference with temporal, causal and bridging annotation](https://md.research.elamur.ai/free/papers/gorman-2016-richer-event-description-integrating-event/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/papers/2016/page/3/)

## References

39 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. James Allan, Jaime G Carbonell, George Doddington, Jonathan Yamron, and Yiming Yang. 1998. Topic detection and tracking pilot study final report. [find](https://scholar.google.com/scholar?q=James+Allan%2C+Jaime+G+Carbonell%2C+George+Doddington%2C+Jonathan+Yamron%2C+and+Yiming+Yang%2E+1998%2E+Topic+detection+and+tracking+pilot+study+final+report%2E)
2. Jun Araki, Zhengzhong Liu, Eduard H Hovy, and Teruko Mitamura. 2014. Detecting subevent structure for event coreference resolution. In LREC. [find](https://scholar.google.com/scholar?q=Jun+Araki%2C+Zhengzhong+Liu%2C+Eduard+H+Hovy%2C+and+Teruko+Mitamura%2E+2014%2E+Detecting+subevent+structure+for+event+coreference+resolution%2E+In+LREC%2E) [our card](https://md.research.elamur.ai/free/papers/araki-2014-detecting-subevent-structure-event-coreference/)
3. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In The first international conference on language resources and evaluation workshop on linguistics coreference. [find](https://scholar.google.com/scholar?q=Amit+Bagga+and+Breck+Baldwin%2E+1998%2E+Algorithms+for+scoring+coreference+chains%2E+In+The+first+international+conference+on+language+resources+and+evaluation+workshop+on+linguistics+coreference%2E) [our card](https://md.research.elamur.ai/free/papers/bagga-1998-algorithms-scoring-coreference-chains/)
4. Amit Bagga and Breck Baldwin. 1999. Cross-document event coreference: Annotations, experiments, and observations. In the Workshop on Coreference and its Applications. [find](https://scholar.google.com/scholar?q=Amit+Bagga+and+Breck+Baldwin%2E+1999%2E+Cross%2Ddocument+event+coreference%3A+Annotations%2C+experiments%2C+and+observations%2E+In+the+Workshop+on+Coreference+and+its+Applications%2E) [our card](https://md.research.elamur.ai/free/papers/bagga-1999-cross-document-event-coreference-annotations/)
5. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In ACL-COLING. [find](https://scholar.google.com/scholar?q=Collin+F%2E+Baker%2C+Charles+J%2E+Fillmore%2C+and+John+B%2E+Lowe%2E+1998%2E+The+Berkeley+FrameNet+project%2E+In+ACL%2DCOLING%2E) [our card](https://md.research.elamur.ai/free/papers/lex-1998-baker-berkeley-framenet/)
6. Cosmin Bejan and Sanda Harabagiu. 2010. Unsupervised event coreference resolution with rich linguistic features. In ACL. [find](https://scholar.google.com/scholar?q=Cosmin+Bejan+and+Sanda+Harabagiu%2E+2010%2E+Unsupervised+event+coreference+resolution+with+rich+linguistic+features%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/bejan-2010-unsupervised-event-coreference-resolution-rich/)
7. Cosmin Adrian Bejan and Sanda Harabagiu. 2014. Unsupervised event coreference resolution. Computational Linguistics. [find](https://scholar.google.com/scholar?q=Cosmin+Adrian+Bejan+and+Sanda+Harabagiu%2E+2014%2E+Unsupervised+event+coreference+resolution%2E+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/papers/bejan-2014-unsupervised-event-coreference-resolution/)
8. E. Bengtson and D. Roth. 2008. Understanding the value of features for coreference resolution. In EMNLP. [find](https://scholar.google.com/scholar?q=E%2E+Bengtson+and+D%2E+Roth%2E+2008%2E+Understanding+the+value+of+features+for+coreference+resolution%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/bengtson-2008-understanding-value-features-coreference-resolution/)
9. Jonathan Berant, Vivek Srikumar, Pei-Chun Chen, Abby Vander Linden, Brittany Harding, Brad Huang Peter Clark, and Christopher D. Manning. 2014. Modeling biological processes for reading comprehension. In EMNLP. [find](https://scholar.google.com/scholar?q=Jonathan+Berant%2C+Vivek+Srikumar%2C+Pei%2DChun+Chen%2C+Abby+Vander+Linden%2C+Brittany+Harding%2C+Brad+Huang+Peter+Clark%2C+and+Christopher+D%2E+Manning%2E+2014%2E+Modeling+biological+processes+for+reading+comprehension%2E)
10. Nathanael Chambers and Dan Jurafsky. 2008. Unsupervised learning of narrative event chains. In ACL. [find](https://scholar.google.com/scholar?q=Nathanael+Chambers+and+Dan+Jurafsky%2E+2008%2E+Unsupervised+learning+of+narrative+event+chains%2E+In+ACL%2E)
11. Kai-Wei Chang, Shyam Upadhyay, Ming-Wei Chang, Vivek Srikumar, and Dan Roth. 2015. IllinoisSL: A JAVA library for structured prediction. In Arxiv Preprint, volume abs/1509.07179. — [arXiv:1509.07179](https://arxiv.org/abs/1509.07179) [find](https://scholar.google.com/scholar?q=Kai%2DWei+Chang%2C+Shyam+Upadhyay%2C+Ming%2DWei+Chang%2C+Vivek+Srikumar%2C+and+Dan+Roth%2E+2015%2E+IllinoisSL%3A+A+JAVA+library+for+structured+prediction%2E+In+Arxiv+Preprint%2C+volume+abs%2F1509%2E07179%2E+%E2%80%94+%5BarXiv%3A1509%2E07179%5D%28)
12. Agata Cybulska and Piek Vossen. 2014. Using a sledgehammer to crack a nut? lexical diversity and event coreference resolution. In LREC. [find](https://scholar.google.com/scholar?q=Agata+Cybulska+and+Piek+Vossen%2E+2014%2E+Using+a+sledgehammer+to+crack+a+nut%3F+lexical+diversity+and+event+coreference+resolution%2E+In+LREC%2E) [our card](https://md.research.elamur.ai/free/papers/cybulska-2014-sledgehammer-crack-nut-lexical-diversity/)
13. Agata Cybulska and Piek Vossen. 2015. Translating granularity of event slots into features for event coreference resolution. In Workshop on Events. [find](https://scholar.google.com/scholar?q=Agata+Cybulska+and+Piek+Vossen%2E+2015%2E+Translating+granularity+of+event+slots+into+features+for+event+coreference+resolution%2E+In+Workshop+on+Events%2E) [our card](https://md.research.elamur.ai/free/papers/cybulska-2015-translating-granularity-event-slots-into/)
14. Naomi Daniel, Dragomir Radev, and Timothy Allison. 2003. Sub-event based multi-document summarization. In the HLT-NAACL 03 on Text summarization workshop-Volume 5. [find](https://scholar.google.com/scholar?q=Naomi+Daniel%2C+Dragomir+Radev%2C+and+Timothy+Allison%2E+2003%2E+Sub%2Devent+based+multi%2Ddocument+summarization%2E+In+the+HLT%2DNAACL+03+on+Text+summarization+workshop%2DVolume+5%2E)
15. Dipanjan Das and Noah Smith. 2011. Semi-supervised frame-semantic parsing for unknown predicates. In ACL. [find](https://scholar.google.com/scholar?q=Dipanjan+Das+and+Noah+Smith%2E+2011%2E+Semi%2Dsupervised+frame%2Dsemantic+parsing+for+unknown+predicates%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/das-2011-semi-supervised-frame-semantic-parsing/)
16. Q. Do, Y. Chan, and D. Roth. 2011. Minimally supervised event causality identification. In EMNLP. [find](https://scholar.google.com/scholar?q=Q%2E+Do%2C+Y%2E+Chan%2C+and+D%2E+Roth%2E+2011%2E+Minimally+supervised+event+causality+identification%2E+In+EMNLP%2E)
17. Yoav Goldberg and Michael Elhadad. 2007. SVM model tampering and anchored learning: A case study in hebrew NP chunking. In ACL. [find](https://scholar.google.com/scholar?q=Yoav+Goldberg+and+Michael+Elhadad%2E+2007%2E+SVM+model+tampering+and+anchored+learning%3A+A+case+study+in+hebrew+NP+chunking%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/aclbulk-2007-goldberg-svm-model-tampering-anchored-learning/)
18. Ben Hachey, Joel Nothman, and Will Radford. 2014. Cheap and easy entity evaluation. In ACL. [find](https://scholar.google.com/scholar?q=Ben+Hachey%2C+Joel+Nothman%2C+and+Will+Radford%2E+2014%2E+Cheap+and+easy+entity+evaluation%2E+In+ACL%2E)
19. Kevin Humphreys, Robert Gaizauskas, and Saliha Azzam. 1997. Event coreference for information extraction. In a Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts. [find](https://scholar.google.com/scholar?q=Kevin+Humphreys%2C+Robert+Gaizauskas%2C+and+Saliha+Azzam%2E+1997%2E+Event+coreference+for+information+extraction%2E+In+a+Workshop+on+Operational+Factors+in+Practical%2C+Robust+Anaphora+Resolution+for+Unrestricted) [our card](https://md.research.elamur.ai/free/papers/humphreys-1997-event-coreference-information-extraction/)
20. Heeyoung Lee, Marta Recasens, Angel Chang, Mihai Surdeanu, and Dan Jurafsky. 2012. Joint entity and event coreference resolution across documents. In EMNLP-CoNLL. [find](https://scholar.google.com/scholar?q=Heeyoung+Lee%2C+Marta+Recasens%2C+Angel+Chang%2C+Mihai+Surdeanu%2C+and+Dan+Jurafsky%2E+2012%2E+Joint+entity+and+event+coreference+resolution+across+documents%2E+In+EMNLP%2DCoNLL%2E) [our card](https://md.research.elamur.ai/free/papers/lee-2012-joint-entity-event-coreference-resolution/)
21. Zhengzhong Liu, Jun Araki, Eduard H Hovy, and Teruko Mitamura. 2014. Supervised within-document event coreference using information propagation. In LREC. [find](https://scholar.google.com/scholar?q=Zhengzhong+Liu%2C+Jun+Araki%2C+Eduard+H+Hovy%2C+and+Teruko+Mitamura%2E+2014%2E+Supervised+within%2Ddocument+event+coreference+using+information+propagation%2E+In+LREC%2E) [our card](https://md.research.elamur.ai/free/papers/liu-2014-supervised-within-document-event-coreference/)
22. Xiaoqiang Luo, Sameer Pradhan, Marta Recasens, and Eduard Hovy. 2014. An extension of BLANC to system mentions. [find](https://scholar.google.com/scholar?q=Xiaoqiang+Luo%2C+Sameer+Pradhan%2C+Marta+Recasens%2C+and+Eduard+Hovy%2E+2014%2E+An+extension+of+BLANC+to+system+mentions%2E)
23. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In EMNLP. [find](https://scholar.google.com/scholar?q=Xiaoqiang+Luo%2E+2005%2E+On+coreference+resolution+performance+metrics%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/luo-2005-coreference-resolution-performance-metrics/)
24. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In ACL: System Demonstrations. [find](https://scholar.google.com/scholar?q=Christopher+D%2E+Manning%2C+Mihai+Surdeanu%2C+John+Bauer%2C+Jenny+Finkel%2C+Steven+J%2E+Bethard%2C+and+David+McClosky%2E+2014%2E+The+Stanford+CoreNLP+natural+language+processing+toolkit%2E+In+ACL%3A+System+Demonstrations%2E) [our card](https://md.research.elamur.ai/free/papers/manning-2014-stanford-corenlp-natural-language-processing/)
25. James Mayfield, David Alexander, Bonnie J Dorr, Jason Eisner, Tamer Elsayed, Tim Finin, Clayton Fink, Marjorie Freedman, Nikesh Garera, Paul McNamee, et al. 2009. Cross-document coreference resolution: A key technology for learning by reading. In AAAI Spring Symposium: Learning by Reading and Learni… [find](https://scholar.google.com/scholar?q=James+Mayfield%2C+David+Alexander%2C+Bonnie+J+Dorr%2C+Jason+Eisner%2C+Tamer+Elsayed%2C+Tim+Finin%2C+Clayton+Fink%2C+Marjorie+Freedman%2C+Nikesh+Garera%2C+Paul+McNamee%2C+et+al%2E+2009%2E+Cross%2Ddocument+coreference+resolution)
26. Srini Narayanan and Sanda Harabagiu. 2004. Question answering based on semantic structures. In COLING. [find](https://scholar.google.com/scholar?q=Srini+Narayanan+and+Sanda+Harabagiu%2E+2004%2E+Question+answering+based+on+semantic+structures%2E+In+COLING%2E)
27. V. Ng and C. Cardie. 2002. Identifying anaphoric and non-anaphoric noun phrases to improve coreference resolution. In COLING. [find](https://scholar.google.com/scholar?q=V%2E+Ng+and+C%2E+Cardie%2E+2002%2E+Identifying+anaphoric+and+non%2Danaphoric+noun+phrases+to+improve+coreference+resolution%2E+In+COLING%2E)
28. Haoruo Peng, Kai-Wei Chang, and Dan Roth. 2015. A joint framework for coreference resolution and mention head detection. In CoNLL. [find](https://scholar.google.com/scholar?q=Haoruo+Peng%2C+Kai%2DWei+Chang%2C+and+Dan+Roth%2E+2015%2E+A+joint+framework+for+coreference+resolution+and+mention+head+detection%2E+In+CoNLL%2E) [our card](https://md.research.elamur.ai/free/papers/peng-2015-joint-framework-coreference-resolution-mention/)
29. Haoruo Peng, Yangqiu Song, and Dan Roth. 2016. Event detection and co-reference with minimal supervision. In EMNLP, page 11. ACL. [find](https://scholar.google.com/scholar?q=Haoruo+Peng%2C+Yangqiu+Song%2C+and+Dan+Roth%2E+2016%2E+Event+detection+and+co%2Dreference+with+minimal+supervision%2E+In+EMNLP%2C+page+11%2E+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/peng-2016-event-detection-co-reference-minimal/)
30. Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. Conll-2011 shared task: Modeling unrestricted coreference in ontonotes. In CoNLL. [find](https://scholar.google.com/scholar?q=Sameer+Pradhan%2C+Lance+Ramshaw%2C+Mitchell+Marcus%2C+Martha+Palmer%2C+Ralph+Weischedel%2C+and+Nianwen+Xue%2E+2011%2E+Conll%2D2011+shared+task%3A+Modeling+unrestricted+coreference+in+ontonotes%2E+In+CoNLL%2E) [our card](https://md.research.elamur.ai/free/papers/pradhan-2011-conll-2011-shared-task-modeling/)
31. Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, and Michael Strube. 2014. Scoring coreference partitions of predicted mentions: A reference implementation. In ACL. [find](https://scholar.google.com/scholar?q=Sameer+Pradhan%2C+Xiaoqiang+Luo%2C+Marta+Recasens%2C+Eduard+Hovy%2C+Vincent+Ng%2C+and+Michael+Strube%2E+2014%2E+Scoring+coreference+partitions+of+predicted+mentions%3A+A+reference+implementation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/pradhan-2014-scoring-coreference-partitions-predicted-mentions/)
32. V. Punyakanok, D. Roth, and W. Yih. 2008. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics, 34(2). [find](https://scholar.google.com/scholar?q=V%2E+Punyakanok%2C+D%2E+Roth%2C+and+W%2E+Yih%2E+2008%2E+The+importance+of+syntactic+parsing+and+inference+in+semantic+role+labeling%2E+Computational+Linguistics%2C+34%282%29%2E) [our card](https://md.research.elamur.ai/free/papers/punyakanok-2008-importance-syntactic-parsing-inference-semantic/)
33. William M Rand. 1971. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical association. [find](https://scholar.google.com/scholar?q=William+M+Rand%2E+1971%2E+Objective+criteria+for+the+evaluation+of+clustering+methods%2E+Journal+of+the+American+Statistical+association%2E)
34. Marta Recasens and Eduard Hovy. 2011. BLANC: Implementing the rand index for coreference evaluation. Natural Language Engineering. [find](https://scholar.google.com/scholar?q=Marta+Recasens+and+Eduard+Hovy%2E+2011%2E+BLANC%3A+Implementing+the+rand+index+for+coreference+evaluation%2E+Natural+Language+Engineering%2E)
35. Rui Sun, Yue Zhang, Meishan Zhang, and Donghong Ji. 2015. Event-driven headline generation. In ACL. [find](https://scholar.google.com/scholar?q=Rui+Sun%2C+Yue+Zhang%2C+Meishan+Zhang%2C+and+Donghong+Ji%2E+2015%2E+Event%2Ddriven+headline+generation%2E+In+ACL%2E)
36. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In the 6th conference on Message understanding. [find](https://scholar.google.com/scholar?q=Marc+Vilain%2C+John+Burger%2C+John+Aberdeen%2C+Dennis+Connolly%2C+and+Lynette+Hirschman%2E+1995%2E+A+model%2Dtheoretic+coreference+scoring+scheme%2E+In+the+6th+conference+on+Message+understanding%2E) [our card](https://md.research.elamur.ai/free/papers/vilain-1995-model-theoretic-coreference-scoring-scheme/)
37. Travis Wolfe, Mark Dredze, and Benjamin Van Durme. 2015. Predicate argument alignment using a global coherence model. In NAACL. [find](https://scholar.google.com/scholar?q=Travis+Wolfe%2C+Mark+Dredze%2C+and+Benjamin+Van+Durme%2E+2015%2E+Predicate+argument+alignment+using+a+global+coherence+model%2E+In+NAACL%2E)
38. Bishan Yang, Claire Cardie, and Peter Frazier. 2015. A hierarchical distance-dependent bayesian model for event coreference resolution. TACL. [find](https://scholar.google.com/scholar?q=Bishan+Yang%2C+Claire+Cardie%2C+and+Peter+Frazier%2E+2015%2E+A+hierarchical+distance%2Ddependent+bayesian+model+for+event+coreference+resolution%2E+TACL%2E)
39. Tongtao Zhang, Hongzhi Li, Heng Ji, and Shih-Fu Chang. 2015. Cross-document event coreference resolution based on cross-media features. In EMNLP. [find](https://scholar.google.com/scholar?q=Tongtao+Zhang%2C+Hongzhi+Li%2C+Heng+Ji%2C+and+Shih%2DFu+Chang%2E+2015%2E+Cross%2Ddocument+event+coreference+resolution+based+on+cross%2Dmedia+features%2E+In+EMNLP%2E)