---
title: "Machine Learning for Coreference Resolution — Ng 2005"
description: "Most learning-based coreference resolvers decide pairwise whether two NPs corefer and then greedily cluster those decisions."
authors: ["Vincent Ng"]
year: 2005
venue: "Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)"
source: "https://aclanthology.org/P05-1020/"
doi: "10.3115/1219840.1219860"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/ng-2005-machine-learning-coreference-resolution-local/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/ng-2005-machine-learning-coreference-resolution-local/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2005](https://md.research.elamur.ai/free/papers/2005/) / Machine Learning for Coreference Resolution: From Local Classification to Global Ranking

# Machine Learning for Coreference Resolution: From Local Classification to Global Ranking

**Authors:** [Vincent Ng](https://md.research.elamur.ai/free/authors/vincent-ng/) · **Year:** 2005 · **Venue:** Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P05-1020/) · [PDF](https://aclanthology.org/P05-1020.pdf) · [DOI 10.3115/1219840.1219860](https://doi.org/10.3115/1219840.1219860) · [OpenAlex](https://openalex.org/W2104619957)

**Citations:** 75 citations (openalex-snapshot · 2026-10-14) · [28](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Most learning-based coreference resolvers decide pairwise whether two NPs corefer and then greedily cluster those decisions. This pipeline has three weaknesses: it depends on ad-hoc design choices, it explores only one partition, and it never directly optimizes clustering-level accuracy. Ng reframes the task as choosing among complete partitions. Many learning-based coreference systems are trained on part of the training data and each proposes a partition of a document's NPs. An SVM ranker, trained on a held-out subset, then picks the best candidate. The ranker uses global, partition-based features and is trained so that partitions scoring higher under the target coreference metric receive higher ranks.

The approach was evaluated on three ACE data sets with the MUC and B-CUBED scorers. It compared favorably with two state-of-the-art baselines that follow the standard pipeline (Soon et al.; Ng and Cardie), with gains of up to 4–7% under one of the two metrics. The work matters because it offers a flexible framework that can combine heterogeneous resolvers. It also reduces reliance on manual design decisions.

## Key points
- The candidate pool contains 54 coreference systems, built by combining:
  - three learners (C4.5, RIPPER, maximum entropy);
  - three training-instance creation methods (McCarthy and Lehnert's, Soon et al.'s, Ng and Cardie's);
  - two feature sets (Soon et al.'s 12 features, Ng and Cardie's 53);
  - three clustering algorithms (closest-first, best-first, aggressive-merge).
- The ranker is trained with Joachims' (2002) SVM package on partitions generated for held-out texts.
- B-CUBED F-measure on BNEWS / NPAPER / NWIRE:

  | System | BNEWS | NPAPER | NWIRE |
  |---|---|---|---|
  | Ranking framework | 65.6 | 66.7 | 66.4 |
  | Soon et al. baseline | 63.5 | 65.6 | 64.2 |
  | Ng and Cardie baseline | 65.5 | 63.3 | 64.4 |
  | Perfect ranker (upper bound) | 70.0 | 69.1 | 69.1 |

  The gains here are modest but consistent, which indicates robustness across data sets.
- Under the MUC scorer, the same resolver performed best on all three test sets: C4.5 with McCarthy and Lehnert instances, Ng and Cardie features, and aggressive-merge clustering. Under B-CUBED, the best resolvers varied across test sets.
- The two scorers rank candidates differently for two reasons. B-CUBED also credits correct non-coreference decisions, and it penalizes wrong merges of large clusters more heavily than MUC does.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/)

## Cited by

Cited by

28 papers of our library cite it (of the 140673 whose bibliographies we read); 16 of them have a page here, 12 not yet.

- [First-Order Probabilistic Models for Coreference Resolution](https://md.research.elamur.ai/free/papers/culotta-2007-first-order-probabilistic-models-coreference/)
- [Semantic Relations between Events and their Time, Locations and Participants for Event Coreference Resolution](https://md.research.elamur.ai/free/papers/cybulska-2013-semantic-relations-between-events-their/)
- [Joint Determination of Anaphoricity and Coreference Resolution using Integer Programming](https://md.research.elamur.ai/free/papers/denis-2007-joint-determination-anaphoricity-coreference-resolution/)
- [Specialized Models and Ranking for Coreference Resolution](https://md.research.elamur.ai/free/papers/denis-2008-specialized-models-ranking-coreference-resolution/)
- [Coreference Resolution in a Modular, Entity-Centered Model](https://md.research.elamur.ai/free/papers/haghighi-2010-coreference-resolution-modular-entity-centered/)
- [Aiding Pronoun Translation with Co-Reference Resolution](https://md.research.elamur.ai/free/papers/nagard-2010-aiding-pronoun-translation-co-reference/)
- [BESTCUT: A Graph Algorithm for Coreference Resolution](https://md.research.elamur.ai/free/papers/nicolae-2006-bestcut-graph-algorithm-coreference-resolution/)
- [Joint Unsupervised Coreference Resolution with Markov Logic](https://md.research.elamur.ai/free/papers/poon-2008-joint-unsupervised-coreference-resolution-markov/)
- [A Constraint-Based Hypergraph Partitioning Approach to Coreference Resolution](https://md.research.elamur.ai/free/papers/sapena-2013-constraint-based-hypergraph-partitioning-approach/)
- [A Fully-Lexicalized Probabilistic Model for Japanese Zero Anaphora Resolution](https://md.research.elamur.ai/free/papers/sasano-2008-fully-lexicalized-probabilistic-model-japanese/)
- [Large-Scale Cross-Document Coreference Using Distributed Inference and Hierarchical Models](https://md.research.elamur.ai/free/papers/singh-2011-large-scale-cross-document-coreference/)
- [A Discriminative Hierarchical Model for Fast Coreference at Large Scale](https://md.research.elamur.ai/free/papers/wick-2012-discriminative-hierarchical-model-fast-coreference/)
- [Coreference Resolution Using Semantic Relatedness Information from Automatically Discovered Patterns](https://md.research.elamur.ai/free/papers/yang-2007-coreference-resolution-semantic-relatedness-information/)
- [An Entity-Mention Model for Coreference Resolution with Inductive Logic Programming](https://md.research.elamur.ai/free/papers/yang-2008-entity-mention-model-coreference-resolution/)
- [A Twin-Candidate Model for Learning-Based Anaphora Resolution](https://md.research.elamur.ai/free/papers/yang-2008-twin-candidate-model-learning-based/)
- [Domain Adaptation for Coreference Resolution: An Adaptive Ensemble Approach](https://md.research.elamur.ai/free/papers/yang-2012-domain-adaptation-coreference-resolution-adaptive/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ng2005machine,
  title = {{Machine Learning for Coreference Resolution: From Local Classification to Global Ranking}},
  author = {Ng, Vincent},
  booktitle = {Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)},
  year = {2005},
  doi = {10.3115/1219840.1219860},
  url = {https://doi.org/10.3115/1219840.1219860}
}
```

- [← previous Learning to Rank using Gradient Descent](https://md.research.elamur.ai/free/papers/burges-2005-learning-rank-using-gradient/)
- [next → Machine Translation Using Probabilistic Synchronous Dependency Insertion Grammars](https://md.research.elamur.ai/free/papers/ding-2005-machine-translation-probabilistic-synchronous-dependency/)
- [↑ back to papers of 2005](https://md.research.elamur.ai/free/papers/2005/)

## References

26 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. C. Aone and S. W. Bennett. 1995. Evaluating automated and manual acquisition of anaphora resolution strategies. In Proc. of the ACL, pages 122–129. [find](https://scholar.google.com/scholar?q=C%2E+Aone+and+S%2E+W%2E+Bennett%2E+1995%2E+Evaluating+automated+and+manual+acquisition+of+anaphora+resolution+strategies%2E+In+Proc%2E+of+the+ACL%2C+pages+122%E2%80%93129%2E) [our card](https://md.research.elamur.ai/free/papers/aone-1995-evaluating-automated-manual-acquisition-anaphora/)
2. A. Bagga and B. Baldwin. 1998. Entity-based cross-document coreferencing using the vector space model. In Proc. of COLING-ACL, pages 79–85. [find](https://scholar.google.com/scholar?q=A%2E+Bagga+and+B%2E+Baldwin%2E+1998%2E+Entity%2Dbased+cross%2Ddocument+coreferencing+using+the+vector+space+model%2E+In+Proc%2E+of+COLING%2DACL%2C+pages+79%E2%80%9385%2E)
3. N. Bansal, A. Blum, and S. Chawla. 2002. Correlation clustering. In Proc. of FOCS, pages 238–247. [find](https://scholar.google.com/scholar?q=N%2E+Bansal%2C+A%2E+Blum%2C+and+S%2E+Chawla%2E+2002%2E+Correlation+clustering%2E+In+Proc%2E+of+FOCS%2C+pages+238%E2%80%93247%2E)
4. D. Bean and E. Riloff. 2004. Unsupervised learning of contextual role knowledge for coreference resolution. In Proc. of HLT/NAACL, pages 297–304. [find](https://scholar.google.com/scholar?q=D%2E+Bean+and+E%2E+Riloff%2E+2004%2E+Unsupervised+learning+of+contextual+role+knowledge+for+coreference+resolution%2E+In+Proc%2E+of+HLT%2FNAACL%2C+pages+297%E2%80%93304%2E) [our card](https://md.research.elamur.ai/free/papers/bean-2004-unsupervised-learning-contextual-role-knowledge/)
5. A. Berger, S. Della Pietra, and V. Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–71. [find](https://scholar.google.com/scholar?q=A%2E+Berger%2C+S%2E+Della+Pietra%2C+and+V%2E+Della+Pietra%2E+1996%2E+A+maximum+entropy+approach+to+natural+language+processing%2E+Computational+Linguistics%2C+22%281%29%3A39%E2%80%9371%2E) [our card](https://md.research.elamur.ai/free/papers/berger-1996-maximum-entropy-approach-natural-language/)
6. C. Cardie and K. Wagstaff. 1999. Noun phrase coreference as clustering. In Proc. of EMNLP/VLC, pages 82–89. [find](https://scholar.google.com/scholar?q=C%2E+Cardie+and+K%2E+Wagstaff%2E+1999%2E+Noun+phrase+coreference+as+clustering%2E+In+Proc%2E+of+EMNLP%2FVLC%2C+pages+82%E2%80%9389%2E) [our card](https://md.research.elamur.ai/free/papers/cardie-1999-noun-phrase-coreference-as-clustering/)
7. W. Cohen. 1995. Fast effective rule induction. In Proc. of ICML, pages 115–123. [find](https://scholar.google.com/scholar?q=W%2E+Cohen%2E+1995%2E+Fast+effective+rule+induction%2E+In+Proc%2E+of+ICML%2C+pages+115%E2%80%93123%2E)
8. M. Collins. 2002. Discriminative training methods for Hidden Markov Models: Theory and experiments with perceptron algorithms. In Proc. of EMNLP, pages 1–8. [find](https://scholar.google.com/scholar?q=M%2E+Collins%2E+2002%2E+Discriminative+training+methods+for+Hidden+Markov+Models%3A+Theory+and+experiments+with+perceptron+algorithms%2E+In+Proc%2E+of+EMNLP%2C+pages+1%E2%80%938%2E) [our card](https://md.research.elamur.ai/free/papers/collins-2002-discriminative-training-methods-hidden-markov/)
9. S. Harabagiu, R. Bunescu, and S. Maiorano. 2001. Text and knowledge mining for coreference resolution. In Proc. of NAACL, pages 55–62. [find](https://scholar.google.com/scholar?q=S%2E+Harabagiu%2C+R%2E+Bunescu%2C+and+S%2E+Maiorano%2E+2001%2E+Text+and+knowledge+mining+for+coreference+resolution%2E+In+Proc%2E+of+NAACL%2C+pages+55%E2%80%9362%2E)
10. R. Iida, K. Inui, H. Takamura, and Y. Matsumoto. 2003. Incorporating contextual cues in trainable models for coreference resolution. In Proc. of the EACL Workshop on The Computational Treatment of Anaphora. [find](https://scholar.google.com/scholar?q=R%2E+Iida%2C+K%2E+Inui%2C+H%2E+Takamura%2C+and+Y%2E+Matsumoto%2E+2003%2E+Incorporating+contextual+cues+in+trainable+models+for+coreference+resolution%2E+In+Proc%2E+of+the+EACL+Workshop+on+The+Computational+Treatment+of+Ana)
11. T. Joachims. 2002. Optimizing search engines using clickthrough data. In Proc. of KDD, pages 133–142. [find](https://scholar.google.com/scholar?q=T%2E+Joachims%2E+2002%2E+Optimizing+search+engines+using+clickthrough+data%2E+In+Proc%2E+of+KDD%2C+pages+133%E2%80%93142%2E)
12. A. Kehler. 1997. Probabilistic coreference in information extraction. In Proc. of EMNLP, pages 163–173. [find](https://scholar.google.com/scholar?q=A%2E+Kehler%2E+1997%2E+Probabilistic+coreference+in+information+extraction%2E+In+Proc%2E+of+EMNLP%2C+pages+163%E2%80%93173%2E) [our card](https://md.research.elamur.ai/free/papers/kehler-1997-probabilistic-coreference-information-extraction/)
13. X. Luo, A. Ittycheriah, H. Jing, N. Kambhatla, and S. Roukos. 2004. A mention-synchronous coreference resolution algorithm based on the Bell tree. In Proc. of the ACL, pages 136–143. [find](https://scholar.google.com/scholar?q=X%2E+Luo%2C+A%2E+Ittycheriah%2C+H%2E+Jing%2C+N%2E+Kambhatla%2C+and+S%2E+Roukos%2E+2004%2E+A+mention%2Dsynchronous+coreference+resolution+algorithm+based+on+the+Bell+tree%2E+In+Proc%2E+of+the+ACL%2C+pages+136%E2%80%93143%2E) [our card](https://md.research.elamur.ai/free/papers/luo-2004-mention-synchronous-coreference-resolution-algorithm/)
14. Coreference System Feature Set Learner Ng and Cardie’s C4.5 Ng and Cardie’s C4.5 Ng and Cardie’s C4.5 Soon et al.’s RIPPER Ng and Cardie’s C4.5 Ng and Cardie’s MaxEnt Clustering Algorithm aggressive-merge aggressive-merge aggressive-merge closest-first aggressive-merge closest-first A. McCallum and… [find](https://scholar.google.com/scholar?q=Coreference+System+Feature+Set+Learner+Ng+and+Cardie%E2%80%99s+C4%2E5+Ng+and+Cardie%E2%80%99s+C4%2E5+Ng+and+Cardie%E2%80%99s+C4%2E5+Soon+et+al%2E%E2%80%99s+RIPPER+Ng+and+Cardie%E2%80%99s+C4%2E5+Ng+and+Cardie%E2%80%99s+MaxEnt+Clustering+Algorithm+aggressive%2Dm)
15. J. McCarthy and W. Lehnert. 1995. Using decision trees for coreference resolution. In Proc. of the IJCAI, pages 1050–1055. [find](https://scholar.google.com/scholar?q=J%2E+McCarthy+and+W%2E+Lehnert%2E+1995%2E+Using+decision+trees+for+coreference+resolution%2E+In+Proc%2E+of+the+IJCAI%2C+pages+1050%E2%80%931055%2E)
16. T. Morton. 2000. Coreference for NLP applications. In Proc. of the ACL. [find](https://scholar.google.com/scholar?q=T%2E+Morton%2E+2000%2E+Coreference+for+NLP+applications%2E+In+Proc%2E+of+the+ACL%2E)
17. C. Müller, S. Rapp, and M. Strube. 2002. Applying co-training to reference resolution. In Proc. of the ACL, pages 352–359. [find](https://scholar.google.com/scholar?q=C%2E+Mu%CC%88ller%2C+S%2E+Rapp%2C+and+M%2E+Strube%2E+2002%2E+Applying+co%2Dtraining+to+reference+resolution%2E+In+Proc%2E+of+the+ACL%2C+pages+352%E2%80%93359%2E)
18. V. Ng and C. Cardie. 2002a. Combining sample selection and error-driven pruning for machine learning of coreference rules. In Proc. of EMNLP, pages 55–62. [find](https://scholar.google.com/scholar?q=V%2E+Ng+and+C%2E+Cardie%2E+2002a%2E+Combining+sample+selection+and+error%2Ddriven+pruning+for+machine+learning+of+coreference+rules%2E+In+Proc%2E+of+EMNLP%2C+pages+55%E2%80%9362%2E) [our card](https://md.research.elamur.ai/free/papers/ng-2002-combining-sample-selection-error-driven/)
19. V. Ng and C. Cardie. 2002b. Improving machine learning approaches to coreference resolution. In Proc. of the ACL, pages 104–111. [find](https://scholar.google.com/scholar?q=V%2E+Ng+and+C%2E+Cardie%2E+2002b%2E+Improving+machine+learning+approaches+to+coreference+resolution%2E+In+Proc%2E+of+the+ACL%2C+pages+104%E2%80%93111%2E) [our card](https://md.research.elamur.ai/free/papers/ng-2002-improving-machine-learning-approaches-coreference/)
20. J. R. Quinlan. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann. [find](https://scholar.google.com/scholar?q=J%2E+R%2E+Quinlan%2E+1993%2E+C4%2E5%3A+Programs+for+Machine+Learning%2E+Morgan+Kaufmann%2E) [our card](https://md.research.elamur.ai/free/papers/quinlan-1993-c45-programs-machine-learning/)
21. W. M. Soon, H. T. Ng, and D. Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521–544. [find](https://scholar.google.com/scholar?q=W%2E+M%2E+Soon%2C+H%2E+T%2E+Ng%2C+and+D%2E+Lim%2E+2001%2E+A+machine+learning+approach+to+coreference+resolution+of+noun+phrases%2E+Computational+Linguistics%2C+27%284%29%3A521%E2%80%93544%2E) [our card](https://md.research.elamur.ai/free/papers/soon-2001-machine-learning-approach-coreference-resolution/)
22. M. Strube and C. Müller. 2003. A machine learning approach to pronoun resolution in spoken dialogue. In Proc. of the ACL, pages 168–175. [find](https://scholar.google.com/scholar?q=M%2E+Strube+and+C%2E+Mu%CC%88ller%2E+2003%2E+A+machine+learning+approach+to+pronoun+resolution+in+spoken+dialogue%2E+In+Proc%2E+of+the+ACL%2C+pages+168%E2%80%93175%2E) [our card](https://md.research.elamur.ai/free/papers/strube-2003-machine-learning-approach-pronoun-resolution/)
23. M. Strube, S. Rapp, and C. Müller. 2002. The influence of minimum edit distance on reference resolution. In Proc. of EMNLP, pages 312–319. [find](https://scholar.google.com/scholar?q=M%2E+Strube%2C+S%2E+Rapp%2C+and+C%2E+Mu%CC%88ller%2E+2002%2E+The+influence+of+minimum+edit+distance+on+reference+resolution%2E+In+Proc%2E+of+EMNLP%2C+pages+312%E2%80%93319%2E)
24. M. Vilain, J. Burger, J. Aberdeen, D. Connolly, and L. Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proc. of the Sixth Message Understanding Conference (MUC-6), pages 45–52. [find](https://scholar.google.com/scholar?q=M%2E+Vilain%2C+J%2E+Burger%2C+J%2E+Aberdeen%2C+D%2E+Connolly%2C+and+L%2E+Hirschman%2E+1995%2E+A+model%2Dtheoretic+coreference+scoring+scheme%2E+In+Proc%2E+of+the+Sixth+Message+Understanding+Conference+%28MUC%2D6%29%2C+pages+45%E2%80%9352%2E) [our card](https://md.research.elamur.ai/free/papers/vilain-1995-model-theoretic-coreference-scoring-scheme/)
25. X. Yang, G. D. Zhou, J. Su, and C. L. Tan. 2003. Coreference resolution using competitive learning approach. In Proc. of the ACL, pages 176–183. [find](https://scholar.google.com/scholar?q=X%2E+Yang%2C+G%2E+D%2E+Zhou%2C+J%2E+Su%2C+and+C%2E+L%2E+Tan%2E+2003%2E+Coreference+resolution+using+competitive+learning+approach%2E+In+Proc%2E+of+the+ACL%2C+pages+176%E2%80%93183%2E)
26. D. Zelenko, C. Aone, and J. Tibbetts. 2004. Coreference resolution for information extraction. In Proc. of the ACL Workshop on Reference Resolution and its Applications, pages 9–16. [find](https://scholar.google.com/scholar?q=D%2E+Zelenko%2C+C%2E+Aone%2C+and+J%2E+Tibbetts%2E+2004%2E+Coreference+resolution+for+information+extraction%2E+In+Proc%2E+of+the+ACL+Workshop+on+Reference+Resolution+and+its+Applications%2C+pages+9%E2%80%9316%2E) [our card](https://md.research.elamur.ai/free/papers/zelenko-2004-coreference-resolution-information-extraction/)