---
title: "A Model-Theoretic Coreference Scoring Scheme — Vilain 1995"
description: "The paper defines the scoring metric used for the coreference task of MUC-6, now known as the MUC metric. Coreference chains are viewed as equivalence…"
authors: ["Marc Vilain", "John Burger", "John Aberdeen", "Dennis Connolly", "Lynette Hirschman"]
year: 1995
venue: "Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995"
source: "https://aclanthology.org/M95-1005/"
doi: "10.3115/1072399.1072405"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/vilain-1995-model-theoretic-coreference-scoring-scheme/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/vilain-1995-model-theoretic-coreference-scoring-scheme/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1995](https://md.research.elamur.ai/free/en/papers/1995/) / A Model-Theoretic Coreference Scoring Scheme

# A Model-Theoretic Coreference Scoring Scheme

**Authors:** Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, Lynette Hirschman · **Year:** 1995 · **Venue:** Sixth Message Understanding Conference (MUC-6), Columbia, Maryland **Link:** [aclanthology.org/M95-1005](https://aclanthology.org/M95-1005/) (ACL Anthology M95-1005) **License of the paper:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) (ACL Anthology, before 2016) — [creativecommons.org/licenses/by-nc-sa/3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/)

[**683** citations](https://openalex.org/works?filter=cites:W1965693266) OpenAlex · 2026-10-14 · [**372** citations in our library](https://md.research.elamur.ai/free/en/papers/vilain-1995-model-theoretic-coreference-scoring-scheme/cited-by/) of the 140673 papers whose bibliographies we read · [Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995](https://aclanthology.org/volumes/M95-1/) · [licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/M95-1005/)
- [PDF](https://aclanthology.org/M95-1005.pdf)
- [DOI 10.3115/1072399.1072405](https://doi.org/10.3115/1072399.1072405)
- [OpenAlex](https://openalex.org/W1965693266)

**Authors (5)**

1. [Marc Vilain](https://openalex.org/A5039994909) [Semantic Scholar](https://www.semanticscholar.org/author/2200382) · h 15 · 2670 citations · 50 works
2. John Burger [Semantic Scholar](https://www.semanticscholar.org/author/5416402)
3. [John Aberdeen](https://openalex.org/A5015172675) [ORCID](https://orcid.org/0000-0002-0112-1037) · [Semantic Scholar](https://www.semanticscholar.org/author/2757028) · h 21 · 2575 citations · 58 works
4. [Dennis Connolly](https://openalex.org/A5087362365) [Semantic Scholar](https://www.semanticscholar.org/author/143838856) · h 5 · 736 citations · 10 works
5. [Lynette Hirschman](https://openalex.org/A5068115280) [ORCID](https://orcid.org/0000-0001-8442-0720) · [Semantic Scholar](https://www.semanticscholar.org/author/145148360) · h 46 · 9346 citations · 264 works

Citations and author metrics: OpenAlex, 2026-10-14; authors with the same name are not always told apart.

## Summary

The paper defines the scoring metric used for the coreference task of MUC-6, now known as the MUC metric. Coreference chains are viewed as equivalence classes, and scoring counts links rather than individual pairwise decisions. Recall for a key chain is the number of links needed to connect it minus the number of pieces the response splits it into, normalized by the minimal number of links; precision is computed symmetrically with key and response swapped. This makes the score independent of which particular links a system outputs, as long as the resulting partition is the same. The metric is simple and intuitive, but it is known to ignore singleton entities and to reward over-merging, which later metrics (B³, CEAF) were designed to address.

## How Mova uses it

- `coref/src/score.rs`: the MUC metric is implemented in Rust as one of the coreference scores, alongside B³, CEAF-e/CEAF-m, and CoNLL F1 (the average of MUC, B³ and CEAF-e).
- Mentions are matched by head word (as in the CRAC shared tasks), with and without singletons.
- The scorer is checked against the worked example of Pradhan et al. 2014 (MUC 0.40 / 0.40) and by negative controls: one giant cluster gives MUC recall 1 but CoNLL \< 0.7; all-singletons gives MUC 0.

## Effectiveness in Mova

It is an evaluation metric, not a modelling idea, so there is no gain to measure. It is a component of the reported results, e.g. on GUM test (UD 2.18, gold trees, with singletons) the default sieve system scores CoNLL F1 73.04 with MUC F1 69.18, B³ 75.26, CEAF-e 74.68.

---

👨‍🔬💥

Topics: [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.3115/1072399.1072405](https://doi.org/10.3115/1072399.1072405)

```bibtex
@inproceedings{vilain1995model,
  title = {{A Model-Theoretic Coreference Scoring Scheme}},
  author = {Vilain, Marc and Burger, John and Aberdeen, John and Connolly, Dennis and Hirschman, Lynette},
  booktitle = {Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995},
  year = {1995},
  doi = {10.3115/1072399.1072405},
  url = {https://doi.org/10.3115/1072399.1072405}
}
```

- [next → Anaphora Resolution in Machine Translation](https://md.research.elamur.ai/free/en/papers/mitkov-1995-anaphora-resolution-machine-translation/)
- [↑ back to papers of 1995](https://md.research.elamur.ai/free/en/papers/1995/)