---
title: "A Discriminative Hierarchical Model for Fast — Wick 2012"
description: "Most coreference systems decide, for each pair of mentions, whether the two refer to the same entity."
authors: ["Michael Wick", "Sameer Singh", "Andrew McCallum"]
year: 2012
venue: "Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P12-1040/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/wick-2012-discriminative-hierarchical-model-fast-coreference/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/wick-2012-discriminative-hierarchical-model-fast-coreference/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2012](https://md.research.elamur.ai/free/papers/2012/) / A Discriminative Hierarchical Model for Fast Coreference at Large Scale

# A Discriminative Hierarchical Model for Fast Coreference at Large Scale

**Authors:** [Michael Wick](https://md.research.elamur.ai/free/authors/michael-wick/), [Sameer Singh](https://md.research.elamur.ai/free/authors/sameer-singh/), [Andrew McCallum](https://md.research.elamur.ai/free/authors/andrew-mccallum/) · **Year:** 2012 · **Venue:** Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P12-1040/) · [PDF](https://aclanthology.org/P12-1040.pdf) · [OpenAlex](https://openalex.org/W2109215439)

**Citations:** 48 citations (openalex-snapshot · 2026-10-14) · [9](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary

Most coreference systems decide, for each pair of mentions, whether the two refer to the same entity. This scales quadratically and cannot capture properties of whole entities. Wick, Singh and McCallum replace this design with a discriminative hierarchical factor graph, a conditional random field in which each entity is a tree of latent sub-entities with mentions as leaves. Each internal node summarizes its children, so a few upper-level nodes can stand in for huge groups of near-identical mentions. Trees can grow to any depth and width. Compatibility factors link only child–parent pairs, so attaching or detaching a subtree means evaluating one factor rather than comparing against every mention in the entity. Inference uses Metropolis-Hastings MCMC. The authors test the model on author disambiguation using bibliographic records from DBLP and BibTeX. Against a pairwise CRF baseline, it reaches a given F1 accuracy one to two orders of magnitude faster, and the gap widens as the data grows. It resolves nearly six million author mentions on a single CPU in under four hours. This shows that rich entity-level coreference can run at web scale.

## Key points

- **Speed on 145k mentions:** the hierarchical model reached 60% F1 accuracy 19× faster than the pairwise CRF and 90% accuracy 31× faster. It needed more samples to converge, but each sample was much cheaper.
- **Speed on 1.3 million mentions:** it hit 60% accuracy 72× faster than the pairwise model and reached 90% in 2,350 seconds.
- **DBLP at scale (about 5 million mentions):** it reached 80% accuracy in 6,700 s (under two hours) and over 90% in 10,600 s.
- **Combined DBLP + BibTeX (5,803,811 mentions):** it reached 90% accuracy in about 13,500 s on one CPU.
- **Comparison with earlier work:** unlike the fixed, shallow sub-entity/super-entity hierarchy of Singh et al. (2011), the trees here are learned and arbitrarily deep. Unlike greedy streaming clustering (Rao et al., 2010), the model can revisit earlier errors.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/)

## Cited by

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 7 not yet.

- [Prune-and-Score: Learning for Greedy Coreference Resolution](https://md.research.elamur.ai/free/papers/ma-2014-prune-score-learning-greedy-coreference/)
- [Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution](https://md.research.elamur.ai/free/papers/wiseman-2015-learning-anaphoricity-antecedent-ranking-features/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wick2012discriminative,
  title = {{A Discriminative Hierarchical Model for Fast Coreference at Large Scale}},
  author = {Wick, Michael and Singh, Sameer and McCallum, Andrew},
  booktitle = {Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2012},
  url = {https://aclanthology.org/P12-1040/}
}
```

- [← previous A Corpus for Research on Deliberation and Debate](https://md.research.elamur.ai/free/papers/walker-2012-corpus-research-deliberation-debate/)
- [next → A Fully Coreference-annotated Corpus of Scholarly Papers from the ACL Anthology](https://md.research.elamur.ai/free/papers/schafer-2012-fully-coreference-annotated-corpus-scholarly/)
- [↑ back to papers of 2012](https://md.research.elamur.ai/free/papers/2012/)

## References

31 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Amit Bagga and Breck Baldwin. 1999. Cross-document event coreference: annotations, experiments, and observations. In Proceedings of the Workshop on Coreference and its Applications, CorefApp ’99, pages 1–8, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Amit+Bagga+and+Breck+Baldwin%2E+1999%2E+Cross%2Ddocument+event+coreference%3A+annotations%2C+experiments%2C+and+observations%2E+In+Proceedings+of+the+Workshop+on+Coreference+and+its+Applications%2C+CorefApp+%E2%80%9999%2C+page) [our card](https://md.research.elamur.ai/free/papers/bagga-1999-cross-document-event-coreference-annotations/)
2. Eric Bengston and Dan Roth. 2008. Understanding the value of features for coreference resolution. In Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Eric+Bengston+and+Dan+Roth%2E+2008%2E+Understanding+the+value+of+features+for+coreference+resolution%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2E) [our card](https://md.research.elamur.ai/free/papers/bengtson-2008-understanding-value-features-coreference-resolution/)
3. Indrajit Bhattacharya and Lise Getoor. 2006. A latent Dirichlet model for unsupervised entity resolution. In SDM. [find](https://scholar.google.com/scholar?q=Indrajit+Bhattacharya+and+Lise+Getoor%2E+2006%2E+A+latent+Dirichlet+model+for+unsupervised+entity+resolution%2E+In+SDM%2E)
4. Mikhail Bilenko, Beena Kamath, and Raymond J. Mooney. 2006. Adaptive blocking: Learning to scale up record linkage. In Proceedings of the Sixth International Conference on Data Mining, ICDM ’06, pages 87–96, Washington, DC, USA. IEEE Computer Society. [find](https://scholar.google.com/scholar?q=Mikhail+Bilenko%2C+Beena+Kamath%2C+and+Raymond+J%2E+Mooney%2E+2006%2E+Adaptive+blocking%3A+Learning+to+scale+up+record+linkage%2E+In+Proceedings+of+the+Sixth+International+Conference+on+Data+Mining%2C+ICDM+%E2%80%9906%2C+pages)
5. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet allocation. Journal on Machine Learning Research, 3:993–1022. [find](https://scholar.google.com/scholar?q=David+M%2E+Blei%2C+Andrew+Y%2E+Ng%2C+and+Michael+I%2E+Jordan%2E+2003%2E+Latent+Dirichlet+allocation%2E+Journal+on+Machine+Learning+Research%2C+3%3A993%E2%80%931022%2E) [our card](https://md.research.elamur.ai/free/papers/blei-2003-latent-dirichlet-allocation/)
6. Aron Culotta and Andrew McCallum. 2006. Practical Markov logic containing first-order quantifiers with application to identity uncertainty. In Human Language Technology Workshop on Computationally Hard Problems and Joint Inference in Speech and Language Processing (HLT/NAACL), June. [find](https://scholar.google.com/scholar?q=Aron+Culotta+and+Andrew+McCallum%2E+2006%2E+Practical+Markov+logic+containing+first%2Dorder+quantifiers+with+application+to+identity+uncertainty%2E+In+Human+Language+Technology+Workshop+on+Computationally+Har)
7. Aron Culotta, Michael Wick, and Andrew McCallum. 2007. First-order probabilistic models for coreference resolution. In North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL HLT). [find](https://scholar.google.com/scholar?q=Aron+Culotta%2C+Michael+Wick%2C+and+Andrew+McCallum%2E+2007%2E+First%2Dorder+probabilistic+models+for+coreference+resolution%2E+In+North+American+Chapter+of+the+Association+for+Computational+Linguistics+%2D+Human+L) [our card](https://md.research.elamur.ai/free/papers/culotta-2007-first-order-probabilistic-models-coreference/)
8. Aria Haghighi and Dan Klein. 2007. Unsupervised coreference resolution in a nonparametric bayesian model. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 848–855. [find](https://scholar.google.com/scholar?q=Aria+Haghighi+and+Dan+Klein%2E+2007%2E+Unsupervised+coreference+resolution+in+a+nonparametric+bayesian+model%2E+In+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2C+pages+848%E2%80%93855%2E) [our card](https://md.research.elamur.ai/free/papers/haghighi-2007-unsupervised-coreference-resolution-nonparametric-bayesian/)
9. Aria Haghighi and Dan Klein. 2010. Coreference resolution in a modular, entity-centered model. In North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL HLT), pages 385–393. [find](https://scholar.google.com/scholar?q=Aria+Haghighi+and+Dan+Klein%2E+2010%2E+Coreference+resolution+in+a+modular%2C+entity%2Dcentered+model%2E+In+North+American+Chapter+of+the+Association+for+Computational+Linguistics+%2D+Human+Language+Technologies+) [our card](https://md.research.elamur.ai/free/papers/haghighi-2010-coreference-resolution-modular-entity-centered/)
10. Mauricio A. Hernández and Salvatore J. Stolfo. 1995. The merge/purge problem for large databases. In Proceedings of the 1995 ACM SIGMOD international conference on Management of data, SIGMOD ’95, pages 127–138, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Mauricio+A%2E+Herna%CC%81ndez+and+Salvatore+J%2E+Stolfo%2E+1995%2E+The+merge%2Fpurge+problem+for+large+databases%2E+In+Proceedings+of+the+1995+ACM+SIGMOD+international+conference+on+Management+of+data%2C+SIGMOD+%E2%80%9995%2C+pag)
11. Jun S. Liu, Faming Liang, and Wing Hung Wong. 2000. The multiple-try method and local optimization in metropolis sampling. Journal of the American Statistical Association, 96(449):121–134. [find](https://scholar.google.com/scholar?q=Jun+S%2E+Liu%2C+Faming+Liang%2C+and+Wing+Hung+Wong%2E+2000%2E+The+multiple%2Dtry+method+and+local+optimization+in+metropolis+sampling%2E+Journal+of+the+American+Statistical+Association%2C+96%28449%29%3A121%E2%80%93134%2E)
12. Andrew McCallum and Ben Wellner. 2004. Conditional models of identity uncertainty with application to noun coreference. In Neural Information Processing Systems (NIPS). [find](https://scholar.google.com/scholar?q=Andrew+McCallum+and+Ben+Wellner%2E+2004%2E+Conditional+models+of+identity+uncertainty+with+application+to+noun+coreference%2E+In+Neural+Information+Processing+Systems+%28NIPS%29%2E)
13. Andrew McCallum, Kamal Nigam, and Lyle Ungar. 2000. Efficient clustering of high-dimensional data sets with application to reference matching. In International Conference on Knowledge Discovery and Data Mining (KDD), pages 169–178. [find](https://scholar.google.com/scholar?q=Andrew+McCallum%2C+Kamal+Nigam%2C+and+Lyle+Ungar%2E+2000%2E+Efficient+clustering+of+high%2Ddimensional+data+sets+with+application+to+reference+matching%2E+In+International+Conference+on+Knowledge+Discovery+and+Da)
14. Andrew McCallum, Karl Schultz, and Sameer Singh. 2009. FACTORIE: Probabilistic programming via imperatively defined factor graphs. In Neural Information Processing Systems (NIPS). [find](https://scholar.google.com/scholar?q=Andrew+McCallum%2C+Karl+Schultz%2C+and+Sameer+Singh%2E+2009%2E+FACTORIE%3A+Probabilistic+programming+via+imperatively+defined+factor+graphs%2E+In+Neural+Information+Processing+Systems+%28NIPS%29%2E)
15. Brian Milch, Bhaskara Marthi, and Stuart Russell. 2006. BLOG: Relational Modeling with Unknown Objects. Ph.D. thesis, University of California, Berkeley. [find](https://scholar.google.com/scholar?q=Brian+Milch%2C+Bhaskara+Marthi%2C+and+Stuart+Russell%2E+2006%2E+BLOG%3A+Relational+Modeling+with+Unknown+Objects%2E+Ph%2ED%2E+thesis%2C+University+of+California%2C+Berkeley%2E)
16. Radford Neal. 2000. Slice sampling. Annals of Statistics, 31:705–767. [find](https://scholar.google.com/scholar?q=Radford+Neal%2E+2000%2E+Slice+sampling%2E+Annals+of+Statistics%2C+31%3A705%E2%80%93767%2E)
17. Vincent Ng. 2005. Machine learning for coreference resolution: From local classification to global ranking. In Annual Meeting of the Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Vincent+Ng%2E+2005%2E+Machine+learning+for+coreference+resolution%3A+From+local+classification+to+global+ranking%2E+In+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2E) [our card](https://md.research.elamur.ai/free/papers/ng-2005-machine-learning-coreference-resolution-local/)
18. Hoifung Poon and Pedro Domingos. 2007. Joint inference in information extraction. In AAAI Conference on Artificial Intelligence, pages 913–918. [find](https://scholar.google.com/scholar?q=Hoifung+Poon+and+Pedro+Domingos%2E+2007%2E+Joint+inference+in+information+extraction%2E+In+AAAI+Conference+on+Artificial+Intelligence%2C+pages+913%E2%80%93918%2E)
19. Altaf Rahman and Vincent Ng. 2009. Supervised models for coreference resolution. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2 - Volume 2, EMNLP ’09, pages 968–977, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Altaf+Rahman+and+Vincent+Ng%2E+2009%2E+Supervised+models+for+coreference+resolution%2E+In+Proceedings+of+the+2009+Conference+on+Empirical+Methods+in+Natural+Language+Processing%3A+Volume+2+%2D+Volume+2%2C+EMNLP+%E2%80%99) [our card](https://md.research.elamur.ai/free/papers/rahman-2009-supervised-models-coreference-resolution/)
20. Delip Rao, Paul McNamee, and Mark Dredze. 2010. Streaming cross document entity coreference resolution. In International Conference on Computational Linguistics (COLING), pages 1050–1058, Beijing, China, August. [find](https://scholar.google.com/scholar?q=Delip+Rao%2C+Paul+McNamee%2C+and+Mark+Dredze%2E+2010%2E+Streaming+cross+document+entity+coreference+resolution%2E+In+International+Conference+on+Computational+Linguistics+%28COLING%29%2C+pages+1050%E2%80%931058%2C+Beijing%2C+Chi) [our card](https://md.research.elamur.ai/free/papers/rao-2010-streaming-cross-document-entity-coreference/)
21. Coling 2010 Organizing Committee. [find](https://scholar.google.com/scholar?q=Coling+2010+Organizing+Committee%2E)
22. Yael Ravin and Zunaid Kazi. 1999. Is Hillary Rodham Clinton the president? disambiguating names across documents. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 9–16. [find](https://scholar.google.com/scholar?q=Yael+Ravin+and+Zunaid+Kazi%2E+1999%2E+Is+Hillary+Rodham+Clinton+the+president%3F+disambiguating+names+across+documents%2E+In+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2C+pages+9%E2%80%9316%2E)
23. Matthew Richardson and Pedro Domingos. 2006. Markov logic networks. Machine Learning, 62(1-2):107–136. [find](https://scholar.google.com/scholar?q=Matthew+Richardson+and+Pedro+Domingos%2E+2006%2E+Markov+logic+networks%2E+Machine+Learning%2C+62%281%2D2%29%3A107%E2%80%93136%2E)
24. Sameer Singh, Michael L. Wick, and Andrew McCallum. 2010. Distantly labeling data for large scale cross-document coreference. Computing Research Repository (CoRR), abs/1005.4298. — [arXiv:1005.4298](https://arxiv.org/abs/1005.4298) [find](https://scholar.google.com/scholar?q=Sameer+Singh%2C+Michael+L%2E+Wick%2C+and+Andrew+McCallum%2E+2010%2E+Distantly+labeling+data+for+large+scale+cross%2Ddocument+coreference%2E+Computing+Research+Repository+%28CoRR%29%2C+abs%2F1005%2E4298%2E+%E2%80%94+%5BarXiv%3A1005%2E4298%5D%28h)
25. Sameer Singh, Amarnag Subramanya, Fernando Pereira, and Andrew McCallum. 2011. Large-scale cross-document coreference using distributed inference and hierarchical models. In Association for Computational Linguistics: Human Language Technologies (ACL HLT). [find](https://scholar.google.com/scholar?q=Sameer+Singh%2C+Amarnag+Subramanya%2C+Fernando+Pereira%2C+and+Andrew+McCallum%2E+2011%2E+Large%2Dscale+cross%2Ddocument+coreference+using+distributed+inference+and+hierarchical+models%2E+In+Association+for+Computatio) [our card](https://md.research.elamur.ai/free/papers/singh-2011-large-scale-cross-document-coreference/)
26. Parag Singla and Pedro Domingos. 2005. Discriminative training of Markov logic networks. In AAAI, Pittsburgh, PA. [find](https://scholar.google.com/scholar?q=Parag+Singla+and+Pedro+Domingos%2E+2005%2E+Discriminative+training+of+Markov+logic+networks%2E+In+AAAI%2C+Pittsburgh%2C+PA%2E)
27. Parag Singla and Pedro Domingos. 2008. Lifted first-order belief propagation. In Proceedings of the 23rd national conference on Artificial intelligence - Volume 2, AAAI’08, pages 1094–1099. AAAI Press. [find](https://scholar.google.com/scholar?q=Parag+Singla+and+Pedro+Domingos%2E+2008%2E+Lifted+first%2Dorder+belief+propagation%2E+In+Proceedings+of+the+23rd+national+conference+on+Artificial+intelligence+%2D+Volume+2%2C+AAAI%E2%80%9908%2C+pages+1094%E2%80%931099%2E+AAAI+Press)
28. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Comput. Linguist., 27(4):521–544. [find](https://scholar.google.com/scholar?q=Wee+Meng+Soon%2C+Hwee+Tou+Ng%2C+and+Daniel+Chung+Yong+Lim%2E+2001%2E+A+machine+learning+approach+to+coreference+resolution+of+noun+phrases%2E+Comput%2E+Linguist%2E%2C+27%284%29%3A521%E2%80%93544%2E) [our card](https://md.research.elamur.ai/free/papers/soon-2001-machine-learning-approach-coreference-resolution/)
29. R.H. Swendsen and J.S. Wang. 1987. Nonuniversal critical dynamics in MC simulations. Phys. Rev. Lett., 58(2):68–88. [find](https://scholar.google.com/scholar?q=R%2EH%2E+Swendsen+and+J%2ES%2E+Wang%2E+1987%2E+Nonuniversal+critical+dynamics+in+MC+simulations%2E+Phys%2E+Rev%2E+Lett%2E%2C+58%282%29%3A68%E2%80%9388%2E)
30. Michael Wick, Aron Culotta, Khashayar Rohanimanesh, and Andrew McCallum. 2009. An entity-based model for coreference resolution. In SIAM International Conference on Data Mining (SDM). [find](https://scholar.google.com/scholar?q=Michael+Wick%2C+Aron+Culotta%2C+Khashayar+Rohanimanesh%2C+and+Andrew+McCallum%2E+2009%2E+An+entity%2Dbased+model+for+coreference+resolution%2E+In+SIAM+International+Conference+on+Data+Mining+%28SDM%29%2E)
31. Xiaofeng Yang, Jian Su, Jun Lang, Chew Lim Tan, Ting Liu, and Sheng Li. 2008. An entity-mention model for coreference resolution with inductive logic programming. In Association for Computational Linguistics, pages 843–851. [find](https://scholar.google.com/scholar?q=Xiaofeng+Yang%2C+Jian+Su%2C+Jun+Lang%2C+Chew+Lim+Tan%2C+Ting+Liu%2C+and+Sheng+Li%2E+2008%2E+An+entity%2Dmention+model+for+coreference+resolution+with+inductive+logic+programming%2E+In+Association+for+Computational+Ling) [our card](https://md.research.elamur.ai/free/papers/yang-2008-entity-mention-model-coreference-resolution/)