---
title: "Annotating Near-Identity from Coreference Disagreements — Recasens 2012"
description: "Standard coreference corpora treat coreference as a yes/no relation. 26 references, cited by 15 papers here."
authors: ["Marta Recasens", "M. Antònia Martí", "Constantin Orasan"]
year: 2012
venue: "Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)"
source: "https://aclanthology.org/L12-1391/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/recasens-2012-annotating-near-identity-coreference-disagreements/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/recasens-2012-annotating-near-identity-coreference-disagreements/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2012](https://md.research.elamur.ai/free/en/papers/2012/) / Annotating Near-Identity from Coreference Disagreements

# Annotating Near-Identity from Coreference Disagreements

**Authors:** [Marta Recasens](https://md.research.elamur.ai/free/en/authors/marta-recasens/), [M. Antònia Martí](https://md.research.elamur.ai/free/en/authors/maria-antonia-marti/) and [Constantin Orasan](https://md.research.elamur.ai/free/en/authors/constantin-orasan/) · **Year:** 2012 · **Venue:** Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/L12-1391/) · [PDF](https://aclanthology.org/L12-1391.pdf) · [OpenAlex](https://openalex.org/W1009530365)

**Citations:** 21 citations (OpenAlex, 2026-10-14) · [15](https://md.research.elamur.ai/free/en/papers/recasens-2012-annotating-near-identity-coreference-disagreements/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
Standard coreference corpora treat coreference as a yes/no relation. Borderline cases, where two mentions refer to "almost" the same entity, therefore cause annotator disagreement and add noise for resolution systems. The authors extend two existing coreference resources, the English NP4E and the Catalan AnCora-CA, with near-identity labels on 50K-token subsets of each. In a first attempt, annotators marked an explicit identity degree (total, strong near-identity, weak near-identity) on existing links. Only 2–6% of relations received a near-identity label, which suggests people are not consciously aware of the phenomenon. The second attempt was indirect. Five annotators made only a binary coreferent/non-coreferent decision, and a merging algorithm turned their level of agreement into identity degrees. This approach classified 12–16% of relations as near-identical. The released corpora, NIdent-EN and NIdent-CA, are presented as the first resources annotated for near-identity. They can support coreference systems that separate clear identity from borderline cases, as well as linguistic research on reference.

## Key points
- Source data: NP4E contains Reuters newswire on terrorism and security across five topics. AnCora-CA is a 400K-word Catalan news corpus, of which a 50K-token subset was used. Both were converted to the AnCora-CA entity-ID format.
- Merging algorithm: each link is scored by the share of the five annotators who marked it, and links with a score above 0.4 (at least 2 annotators) are kept. Mention scores above 0.5 count as clear coreference, scores of 0.25–0.5 as identity degree 2, and scores up to 0.25 as degree 1.
- Implicit-method distribution:
  - NP4E: 88.19% total identity, 10.86% degree 2, 0.95% degree 1.
  - AnCora-CA: 83.75% total identity, 14.75% degree 2, 1.50% degree 1.
- Near-identity rose from 6% to 12% in English and from 2% to 16% in Catalan. The authors relate the higher Catalan rate to Romance languages' preference for anaphors whose head differs from the previous mention.
- NIdent-EN and NIdent-CA are freely available at http://clic.ub.edu/corpus/nident.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [🔗 Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/)

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 248 works · 214,804 citations
- [University of Wolverhampton](https://md.research.elamur.ai/free/en/universities/university-of-wolverhampton/) — University, Wolverhampton, United Kingdom · 27 works · 5,947 citations
- [Universitat de Barcelona](https://md.research.elamur.ai/free/en/universities/universitat-de-barcelona/) — University, Barcelona, Spain · 7 works · 1,246 citations

## Cite

Cite as: Marta Recasens, M. Antònia Martí and Constantin Orasan. 2012. Annotating Near-Identity from Coreference Disagreements. Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{recasens2012annotating,
  title = {{Annotating Near-Identity from Coreference Disagreements}},
  author = {Marta Recasens, M. Antònia Martí and Constantin Orasan},
  booktitle = {Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)},
  year = {2012},
  doi = {10.63317/4bz9jeo9esow},
  url = {https://doi.org/10.63317/4bz9jeo9esow}
}
```

- [← previous The Rocky Road towards a Swedish FrameNet - Creating SweFN](https://md.research.elamur.ai/free/en/papers/heppin-2012-rocky-road-swedish-framenet-creating/)
- [next → Classifying Dialogue Acts in Multi-party Live Chats](https://md.research.elamur.ai/free/en/papers/kim-2012-classifying-dialogue-acts-multi-party/)
- [↑ back to papers of 2012](https://md.research.elamur.ai/free/en/papers/2012/)

## References

26 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Chris Brockett. 2007. Aligning the RTE 2006 Corpus. Technical Report MSR-TR-2007-77, Microsoft Research. [find](https://scholar.google.com/scholar?q=Chris+Brockett%2E+2007%2E+Aligning+the+RTE+2006+Corpus%2E+Technical+Report+MSR%2DTR%2D2007%2D77%2C+Microsoft+Research%2E)
2. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The Automatic Content Extraction (ACE) program – tasks, data, and evaluation. In Proceedings of LREC 2004, pages 837–840. [our card](https://md.research.elamur.ai/free/en/papers/doddington-2004-automatic-content-extraction-ace-program/)
3. Aria Haghighi and Dan Klein. 2010. Coreference resolution in a modular, entity-centered model. In Proceedings of HLT-NAACL 2010, pages 385–393. [our card](https://md.research.elamur.ai/free/en/papers/haghighi-2010-coreference-resolution-modular-entity-centered/)
4. Laura Hasler and Constantin Orasan. 2009. Do coreferential arguments make event mentions coreferential? In Proceedings of DAARC 2009, pages 151–163. [find](https://scholar.google.com/scholar?q=Laura+Hasler+and+Constantin+Orasan%2E+2009%2E+Do+coreferential+arguments+make+event+mentions+coreferential%3F+In+Proceedings+of+DAARC+2009%2C+pages+151%E2%80%93163%2E)
5. Laura Hasler, Constantin Orasan, and Karin Naumann. 2006. NPs for Events: Experiments in coreference annotation. In Proceedings of LREC 2006, pages 1167–1172. [our card](https://md.research.elamur.ai/free/en/papers/hasler-2006-nps-events-experiments-coreference-annotation/)
6. Erhard Hinrichs, Sandra Kübler, and Karin Naumann. 2005. A unified representation for morphological, syntactic, semantic and referential annotations. In Proceedings of the ACL Workshop on Frontiers in Corpus Annotation II: Pie in the Sky, pages 13–20. [find](https://scholar.google.com/scholar?q=Erhard+Hinrichs%2C+Sandra+Ku%CC%88bler%2C+and+Karin+Naumann%2E+2005%2E+A+unified+representation+for+morphological%2C+syntactic%2C+semantic+and+referential+annotations%2E+In+Proceedings+of+the+ACL+Workshop+on+Frontiers+i)
7. Lynette Hirschman and Nancy Chinchor. 1997. MUC-7 Coreference Task Definition – Version 3.0. In Proceedings of MUC-7. [our card](https://md.research.elamur.ai/free/en/papers/hirschman-1998-appendix-f-muc-7-coreference/)
8. Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 Shared Task. In Proceedings of CoNLL: Shared Task, pages 28–34. [our card](https://md.research.elamur.ai/free/en/papers/lee-2011-stanford-s-multi-pass-sieve/)
9. Lita Lundquist. 2007. Lexical anaphors in Danish and French. In Monika Schwarz-Friesel, Manfred Consten, and Mareile Knees, editors, Anaphors in Text: Cognitive, formal and applied approaches to anaphoric reference, pages 37–48. John Benjamins, Amsterdam, Netherlands. [find](https://scholar.google.com/scholar?q=Lita+Lundquist%2E+2007%2E+Lexical+anaphors+in+Danish+and+French%2E+In+Monika+Schwarz%2DFriesel%2C+Manfred+Consten%2C+and+Mareile+Knees%2C+editors%2C+Anaphors+in+Text%3A+Cognitive%2C+formal+and+applied+approaches+to+anaph)
10. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of HLT-EMNLP 2005, pages 25–32. [our card](https://md.research.elamur.ai/free/en/papers/luo-2005-coreference-resolution-performance-metrics/)
11. Rebecca Passonneau. 2004. Computing reliability for coreference annotation. In Proceedings of LREC 2004, pages 1503–1506. [find](https://scholar.google.com/scholar?q=Rebecca+Passonneau%2E+2004%2E+Computing+reliability+for+coreference+annotation%2E+In+Proceedings+of+LREC+2004%2C+pages+1503%E2%80%931506%2E)
12. Massimo Poesio and Ron Artstein. 2008. Anaphoric annotation in the ARRAU corpus. In Proceedings of LREC 2008, pages 1170–1174. [our card](https://md.research.elamur.ai/free/en/papers/poesio-2008-anaphoric-annotation-arrau-corpus/)
13. Massimo Poesio. 2004. The MATE/GNOME proposals for anaphoric annotation, revisited. In Proceedings of SIG-DIAL, pages 154–162. [our card](https://md.research.elamur.ai/free/en/papers/poesio-2004-mate-gnome-proposals-anaphoric-annotation/)
14. Sameer Pradhan, Eduard Hovy, Mitch Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2007. OntoNotes: A Unified Relational Semantic Representation. In Proceedings of the International Conference on Semantic Computing (ICSC), pages 517–526. [find](https://scholar.google.com/scholar?q=Sameer+Pradhan%2C+Eduard+Hovy%2C+Mitch+Marcus%2C+Martha+Palmer%2C+Lance+Ramshaw%2C+and+Ralph+Weischedel%2E+2007%2E+OntoNotes%3A+A+Unified+Relational+Semantic+Representation%2E+In+Proceedings+of+the+International+Confer)
15. Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes. In Proceedings of CoNLL: Shared Task, pages 1–27. [our card](https://md.research.elamur.ai/free/en/papers/pradhan-2011-conll-2011-shared-task-modeling/)
16. Marta Recasens and M. Antònia Martı́. 2010. AnCora-CO: Coreferentially annotated corpora for Spanish and Catalan. Language Resources and Evaluation, 44(4):315– 345. [find](https://scholar.google.com/scholar?q=Marta+Recasens+and+M%2E+Anto%CC%80nia+Mart%C4%B1%CC%81%2E+2010%2E+AnCora%2DCO%3A+Coreferentially+annotated+corpora+for+Spanish+and+Catalan%2E+Language+Resources+and+Evaluation%2C+44%284%29%3A315%E2%80%93+345%2E)
17. Marta Recasens, Eduard Hovy, and M. Antònia Martı́. 2010. A typology of near-identity relations for coreference (NIDENT). In Proceedings of LREC 2010, pages 149–156. [our card](https://md.research.elamur.ai/free/en/papers/recasens-2010-typology-near-identity-relations-coreference/)
18. Marta Recasens, Eduard Hovy, and M. Antònia Martı́. 2011. Identity, Non-Identity, and Near-Identity: Addressing the complexity of coreference. Lingua, 121(6):1138–1152. [find](https://scholar.google.com/scholar?q=Marta+Recasens%2C+Eduard+Hovy%2C+and+M%2E+Anto%CC%80nia+Mart%C4%B1%CC%81%2E+2011%2E+Identity%2C+Non%2DIdentity%2C+and+Near%2DIdentity%3A+Addressing+the+complexity+of+coreference%2E+Lingua%2C+121%286%29%3A1138%E2%80%931152%2E)
19. Marta Recasens, Liliana Tolchinsky, and M. Antònia Martı́. in prep. Coreference is not always either/or: Psycholinguistic evidence of near-identity. [find](https://scholar.google.com/scholar?q=Marta+Recasens%2C+Liliana+Tolchinsky%2C+and+M%2E+Anto%CC%80nia+Mart%C4%B1%CC%81%2E+in+prep%2E+Coreference+is+not+always+either%2For%3A+Psycholinguistic+evidence+of+near%2Didentity%2E)
20. Tony Rose, Mark Stevenson, and Miles Whitehead. 2002. The Reuters Corpus Volume 1 - from Yesterday’s News to Tomorrow’s Language Resources. In Proceedings of LREC 2002, pages 827–833. [find](https://scholar.google.com/scholar?q=Tony+Rose%2C+Mark+Stevenson%2C+and+Miles+Whitehead%2E+2002%2E+The+Reuters+Corpus+Volume+1+%2D+from+Yesterday%E2%80%99s+News+to+Tomorrow%E2%80%99s+Language+Resources%2E+In+Proceedings+of+LREC+2002%2C+pages+827%E2%80%93833%2E)
21. Pasi Tapanainen and Timo Järvinen. 1997. A non-projective dependency parser. In Proceedings of ANLP 1997, pages 64–71. [our card](https://md.research.elamur.ai/free/en/papers/tapanainen-1997-non-projective-dependency-parser/)
22. Mariona Taulé, M. Antònia Martı́, and Marta Recasens. 2008. AnCora: Multilevel annotated corpora for Catalan and Spanish. In Proceedings of LREC 2008, pages 96–101. [find](https://scholar.google.com/scholar?q=Mariona+Taule%CC%81%2C+M%2E+Anto%CC%80nia+Mart%C4%B1%CC%81%2C+and+Marta+Recasens%2E+2008%2E+AnCora%3A+Multilevel+annotated+corpora+for+Catalan+and+Spanish%2E+In+Proceedings+of+LREC+2008%2C+pages+96%E2%80%93101%2E)
23. Kees van Deemter and Rodger Kibble. 2000. On coreferring: Coreference in MUC and related annotation schemes. Computational Linguistics, 26(4):629–637. [our card](https://md.research.elamur.ai/free/en/papers/deemter-2000-coreferring-coreference-muc-related-annotation/)
24. Kees van Deemter. 2010. Not Exactly: In Praise of Vagueness. Oxford University Press, USA. [find](https://scholar.google.com/scholar?q=Kees+van+Deemter%2E+2010%2E+Not+Exactly%3A+In+Praise+of+Vagueness%2E+Oxford+University+Press%2C+USA%2E)
25. Yannick Versley. 2008. Vagueness and referential ambiguity in a large-scale annotated corpus. Research on Language and Computation, 6:333–353. [find](https://scholar.google.com/scholar?q=Yannick+Versley%2E+2008%2E+Vagueness+and+referential+ambiguity+in+a+large%2Dscale+annotated+corpus%2E+Research+on+Language+and+Computation%2C+6%3A333%E2%80%93353%2E)
26. Annie Zaenen. 2006. Mark-up barking up the wrong tree. Computational Linguistics, 32(4):577–580. [find](https://scholar.google.com/scholar?q=Annie+Zaenen%2E+2006%2E+Mark%2Dup+barking+up+the+wrong+tree%2E+Computational+Linguistics%2C+32%284%29%3A577%E2%80%93580%2E)