---
title: "An Expectation Maximization Approach to — Cherry 2005"
description: "The paper asks whether third-person pronouns can be linked to their antecedents without annotated training data."
authors: ["Colin Cherry", "Shane Bergsma"]
year: 2005
venue: "Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005)"
source: "https://aclanthology.org/W05-0612/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/cherry-2005-expectation-maximization-approach-pronoun-resolution/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/cherry-2005-expectation-maximization-approach-pronoun-resolution/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2005](https://md.research.elamur.ai/free/papers/2005/) / An Expectation Maximization Approach to Pronoun Resolution

# An Expectation Maximization Approach to Pronoun Resolution

**Authors:** [Colin Cherry](https://md.research.elamur.ai/free/authors/colin-cherry/), [Shane Bergsma](https://md.research.elamur.ai/free/authors/shane-bergsma/) · **Year:** 2005 · **Venue:** Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W05-0612/) · [PDF](https://aclanthology.org/W05-0612.pdf) · [OpenAlex](https://openalex.org/W2126795320)

**Citations:** 32 citations (openalex-snapshot · 2026-10-14) · [17](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper asks whether third-person pronouns can be linked to their antecedents without annotated training data. Cherry and Bergsma build a fully automatic pipeline. A dependency-parsed news-domain Question Answering corpus supplies candidate nouns from the current and previous sentence. Linguistic modules handle pleonastic, cataphoric and other special cases before learning. Expectation Maximization, modeled on the word-alignment approach of IBM Model 1, then treats the true antecedent as a hidden variable. It jointly estimates four lexicalized distributions:
- gender/number affinity of a word,
- the pronoun's syntactic context given the antecedent,
- the prior likelihood of a word being an antecedent,
- a distance ("jump") preference.

Seeding EM with a gender/number model collected from unambiguous cases helps. Reweighting the learned models with maximum entropy helps further. On a news test set with non-anaphoric pronouns removed, the unsupervised system comes within a small margin of a supervised SVM. The semi-supervised extension matches the SVM statistically. The work shows that unlabeled text alone can yield competitive pronoun resolution. It also shows that automatically induced gender/number statistics are an especially strong cue.

## Key points
- The candidate window (current plus preceding sentence) contained a correct antecedent in 97% of lists on a labeled development text.
- Accuracy on all pronouns rose from 0.397 for the previous-noun baseline to 0.610 for uniform-start EM. The initializer alone reached 0.628, EM with the initializer 0.632, and the maxent extension 0.669, against an upper bound of 0.838.
- The maxent weights favored the pronoun (gender/number) model heavily (λ = 0.931). The context, antecedent-prior and jump models received weights between 0.056 and 0.167.
- The comparison SVM used 52 features and 1398 labeled pronouns and scored 0.714. EM with the initializer scored 0.664, a 7% relative drop. The maxent extension scored 0.708, which is not significantly different from the SVM (McNemar's test).
- Sentences containing quotations were excluded from training because their candidate lists often lack the true antecedent (upper bound only 0.754 on these). Missing antecedents were mainly caused by:
  - pleonastic detection errors (31%),
  - non-noun referents (25%),
  - the limited window (21%).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/)

## Cited by

Cited by

17 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 6 not yet.

- [Bootstrapping Path-Based Pronoun Resolution](https://md.research.elamur.ai/free/papers/bergsma-2006-bootstrapping-path-based-pronoun-resolution/)
- [EM Works for Pronoun Anaphora Resolution](https://md.research.elamur.ai/free/papers/charniak-2009-em-works-pronoun-anaphora-resolution/)
- [Chinese Zero Pronoun Resolution: An Unsupervised Probabilistic Model Rivaling Supervised Resolvers](https://md.research.elamur.ai/free/papers/chen-2014-chinese-zero-pronoun-resolution-unsupervised/)
- [Chinese Zero Pronoun Resolution: A Joint Unsupervised Discourse-Aware Model Rivaling State-of-the-Art Resolvers](https://md.research.elamur.ai/free/papers/chen-2015-chinese-zero-pronoun-resolution-joint/)
- [The Same-Head Heuristic for Coreference](https://md.research.elamur.ai/free/papers/elsner-2010-same-head-heuristic-coreference/)
- [Latent Anaphora Resolution for Cross-Lingual Pronoun Prediction](https://md.research.elamur.ai/free/papers/hardmeier-2013-latent-anaphora-resolution-cross-lingual/)
- [Dependency-driven Anaphoricity Determination for Coreference Resolution](https://md.research.elamur.ai/free/papers/kong-2010-dependency-driven-anaphoricity-determination-coreference/)
- [Unsupervised Ranking Model for Entity Coreference Resolution](https://md.research.elamur.ai/free/papers/ma-2016-unsupervised-ranking-model-entity-coreference/)
- [Unsupervised Models for Coreference Resolution](https://md.research.elamur.ai/free/papers/ng-2008-unsupervised-models-coreference-resolution/)
- [Supervised Noun Phrase Coreference Research: The First Fifteen Years](https://md.research.elamur.ai/free/papers/ng-2010-supervised-noun-phrase-coreference-research/)
- [Global Learning of Noun Phrase Anaphoricity in Coreference Resolution via Label Propagation](https://md.research.elamur.ai/free/papers/zhou-2009-global-learning-noun-phrase-anaphoricity/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{cherry2005expectation,
  title = {{An Expectation Maximization Approach to Pronoun Resolution}},
  author = {Cherry, Colin and Bergsma, Shane},
  booktitle = {Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005)},
  year = {2005},
  doi = {10.3115/1706543.1706559},
  url = {https://doi.org/10.3115/1706543.1706559}
}
```

- [← previous Adaptation of the translation model for statistical machine translation based on information retrieval](https://md.research.elamur.ai/free/papers/hildebrand-2005-adaptation-translation-model-statistical-machine/)
- [next → Annotating Discourse Connectives in the Chinese Treebank](https://md.research.elamur.ai/free/papers/xue-2005-annotating-discourse-connectives-chinese-treebank/)
- [↑ back to papers of 2005](https://md.research.elamur.ai/free/papers/2005/)

## References

23 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David L. Bean and Ellen Riloff. 2004. Unsupervised learning of contextual role knowledge for coreference resolution. In HLT-NAACL, pages 297–304. [find](https://scholar.google.com/scholar?q=David+L%2E+Bean+and+Ellen+Riloff%2E+2004%2E+Unsupervised+learning+of+contextual+role+knowledge+for+coreference+resolution%2E+In+HLT%2DNAACL%2C+pages+297%E2%80%93304%2E) [our card](https://md.research.elamur.ai/free/papers/bean-2004-unsupervised-learning-contextual-role-knowledge/)
2. Adam L. Berger, Stephen A. Della Pietra, and Vincent J. Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–71. [find](https://scholar.google.com/scholar?q=Adam+L%2E+Berger%2C+Stephen+A%2E+Della+Pietra%2C+and+Vincent+J%2E+Della+Pietra%2E+1996%2E+A+maximum+entropy+approach+to+natural+language+processing%2E+Computational+Linguistics%2C+22%281%29%3A39%E2%80%9371%2E) [our card](https://md.research.elamur.ai/free/papers/berger-1996-maximum-entropy-approach-natural-language/)
3. Shane Bergsma. 2005a. Automatic acquisition of gender information for anaphora resolution. In Proceedings of the 18th Conference of the Canadian Society for Computational Intelligence (Canadian AI 2005), pages 342–353, Victoria, BC. [find](https://scholar.google.com/scholar?q=Shane+Bergsma%2E+2005a%2E+Automatic+acquisition+of+gender+information+for+anaphora+resolution%2E+In+Proceedings+of+the+18th+Conference+of+the+Canadian+Society+for+Computational+Intelligence+%28Canadian+AI+200)
4. Shane Bergsma. 2005b. Corpus-based learning for pronominal anaphora resolution. Master’s thesis, Department of Computing Science, University of Alberta, Edmonton. [http://www.cs.ualberta.ca/˜bergsma/Pubs/thesis.pdf](http://www.cs.ualberta.ca/˜bergsma/Pubs/thesis.pdf). [↗](http://www.cs.ualberta.ca/˜bergsma/Pubs/thesis.pdf)
5. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–312. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Stephen+A%2E+Della+Pietra%2C+Vincent+J%2E+Della+Pietra%2C+and+Robert+L%2E+Mercer%2E+1993%2E+The+mathematics+of+statistical+machine+translation%3A+Parameter+estimation%2E+Computational+Linguistics%2C+19%282%29)
6. Claire Cardie and Kiri Wagstaff. 1999. Noun phrase coreference as clustering. In Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 82–89. [find](https://scholar.google.com/scholar?q=Claire+Cardie+and+Kiri+Wagstaff%2E+1999%2E+Noun+phrase+coreference+as+clustering%2E+In+Proceedings+of+the+1999+Joint+SIGDAT+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+Very+Large+Corp) [our card](https://md.research.elamur.ai/free/papers/cardie-1999-noun-phrase-coreference-as-clustering/)
7. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1):1–38. [find](https://scholar.google.com/scholar?q=A%2E+P%2E+Dempster%2C+N%2E+M%2E+Laird%2C+and+D%2E+B%2E+Rubin%2E+1977%2E+Maximum+likelihood+from+incomplete+data+via+the+EM+algorithm%2E+Journal+of+the+Royal+Statistical+Society%2C+39%281%29%3A1%E2%80%9338%2E)
8. Niyu Ge, John Hale, and Eugene Charniak. 1998. A statistical approach to anaphora resolution. In Proceedings of the Sixth Workshop on Very Large Corpora, pages 161–171. [find](https://scholar.google.com/scholar?q=Niyu+Ge%2C+John+Hale%2C+and+Eugene+Charniak%2E+1998%2E+A+statistical+approach+to+anaphora+resolution%2E+In+Proceedings+of+the+Sixth+Workshop+on+Very+Large+Corpora%2C+pages+161%E2%80%93171%2E) [our card](https://md.research.elamur.ai/free/papers/ge-1998-statistical-approach-anaphora-resolution/)
9. L. Haegeman. 1994. Introduction to Government & Binding theory: Second Edition. Basil Blackwell, Cambridge, UK. [find](https://scholar.google.com/scholar?q=L%2E+Haegeman%2E+1994%2E+Introduction+to+Government+%26+Binding+theory%3A+Second+Edition%2E+Basil+Blackwell%2C+Cambridge%2C+UK%2E)
10. Donald Hindle and Mats Rooth. 1993. Structural ambiguity and lexical relations. Computational Linguistics, 19(1):103– 120. [find](https://scholar.google.com/scholar?q=Donald+Hindle+and+Mats+Rooth%2E+1993%2E+Structural+ambiguity+and+lexical+relations%2E+Computational+Linguistics%2C+19%281%29%3A103%E2%80%93+120%2E)
11. Harold Jeffreys, 1961. Theory of Probability, chapter 3.23. Oxford: Clarendon Press, 3rd edition. [find](https://scholar.google.com/scholar?q=Harold+Jeffreys%2C+1961%2E+Theory+of+Probability%2C+chapter+3%2E23%2E+Oxford%3A+Clarendon+Press%2C+3rd+edition%2E)
12. Andrew Kehler. 1997. Probabilistic coreference in information extraction. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing, pages 163–173. [find](https://scholar.google.com/scholar?q=Andrew+Kehler%2E+1997%2E+Probabilistic+coreference+in+information+extraction%2E+In+Proceedings+of+the+Second+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+163%E2%80%93173%2E) [our card](https://md.research.elamur.ai/free/papers/kehler-1997-probabilistic-coreference-information-extraction/)
13. Christopher Kennedy and Branimir Boguraev. 1996. Anaphora for everyone: Pronominal anaphora resolution without a parser. In Proceedings of the 16th Conference on Computational Linguistics, pages 113–118. [find](https://scholar.google.com/scholar?q=Christopher+Kennedy+and+Branimir+Boguraev%2E+1996%2E+Anaphora+for+everyone%3A+Pronominal+anaphora+resolution+without+a+parser%2E+In+Proceedings+of+the+16th+Conference+on+Computational+Linguistics%2C+pages+113%E2%80%931) [our card](https://md.research.elamur.ai/free/papers/kennedy-1996-anaphora-everyone-pronominal-anaphora-resolution/)
14. Shalom Lappin and Herbert J. Leass. 1994. An algorithm for pronominal anaphora resolution. Computational Linguistics, 20(4):535–561. [find](https://scholar.google.com/scholar?q=Shalom+Lappin+and+Herbert+J%2E+Leass%2E+1994%2E+An+algorithm+for+pronominal+anaphora+resolution%2E+Computational+Linguistics%2C+20%284%29%3A535%E2%80%93561%2E) [our card](https://md.research.elamur.ai/free/papers/lappin-1994-algorithm-pronominal-anaphora-resolution/)
15. Dekang Lin. 1994. Principar - an efficient, broad-coverage, principle-based parser. In Proceedings of COLING-94, pages 42–48, Kyoto, Japan. [find](https://scholar.google.com/scholar?q=Dekang+Lin%2E+1994%2E+Principar+%2D+an+efficient%2C+broad%2Dcoverage%2C+principle%2Dbased+parser%2E+In+Proceedings+of+COLING%2D94%2C+pages+42%E2%80%9348%2C+Kyoto%2C+Japan%2E)
16. Robert Malouf. 2002. A comparison of algorithms for maximum entropy parameter estimation. In Proceedings of the Sixth Conference on Natural Language Learning (CoNLL-2002), pages 49–55. [find](https://scholar.google.com/scholar?q=Robert+Malouf%2E+2002%2E+A+comparison+of+algorithms+for+maximum+entropy+parameter+estimation%2E+In+Proceedings+of+the+Sixth+Conference+on+Natural+Language+Learning+%28CoNLL%2D2002%29%2C+pages+49%E2%80%9355%2E) [our card](https://md.research.elamur.ai/free/papers/malouf-2002-comparison-algorithms-maximum-entropy-parameter/)
17. Ruslan Mitkov, Richard Evans, and Constantin Orasan. 2002. A new, fully automatic version of Mitkov’s knowledge-poor pronoun resolution method. In Proceedings of the Third International Conference on Computational Linguistics and Intelligent Text Processing, pages 168–186. [find](https://scholar.google.com/scholar?q=Ruslan+Mitkov%2C+Richard+Evans%2C+and+Constantin+Orasan%2E+2002%2E+A+new%2C+fully+automatic+version+of+Mitkov%E2%80%99s+knowledge%2Dpoor+pronoun+resolution+method%2E+In+Proceedings+of+the+Third+International+Conference+on+)
18. Christoph Müller, Stefan Rapp, and Michael Strube. 2002. Applying co-training to reference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 352–359. [find](https://scholar.google.com/scholar?q=Christoph+Mu%CC%88ller%2C+Stefan+Rapp%2C+and+Michael+Strube%2E+2002%2E+Applying+co%2Dtraining+to+reference+resolution%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+page)
19. Vincent Ng and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 104–111. [find](https://scholar.google.com/scholar?q=Vincent+Ng+and+Claire+Cardie%2E+2002%2E+Improving+machine+learning+approaches+to+coreference+resolution%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+1) [our card](https://md.research.elamur.ai/free/papers/ng-2002-improving-machine-learning-approaches-coreference/)
20. Vincent Ng and Claire Cardie. 2003. Weakly supervised natural language learning without redundant views. In HLT-NAACL 2003: Proceedings of the Main Conference, pages 173–180. [find](https://scholar.google.com/scholar?q=Vincent+Ng+and+Claire+Cardie%2E+2003%2E+Weakly+supervised+natural+language+learning+without+redundant+views%2E+In+HLT%2DNAACL+2003%3A+Proceedings+of+the+Main+Conference%2C+pages+173%E2%80%93180%2E)
21. Franz J. Och and Hermann Ney. 2002. Discriminative training and maximum entropy models for statistical machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 295–302, Philadelphia, PA, July. [find](https://scholar.google.com/scholar?q=Franz+J%2E+Och+and+Hermann+Ney%2E+2002%2E+Discriminative+training+and+maximum+entropy+models+for+statistical+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Computation) [our card](https://md.research.elamur.ai/free/papers/och-2002-discriminative-training-maximum-entropy-models/)
22. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521– 544. [find](https://scholar.google.com/scholar?q=Wee+Meng+Soon%2C+Hwee+Tou+Ng%2C+and+Daniel+Chung+Yong+Lim%2E+2001%2E+A+machine+learning+approach+to+coreference+resolution+of+noun+phrases%2E+Computational+Linguistics%2C+27%284%29%3A521%E2%80%93+544%2E) [our card](https://md.research.elamur.ai/free/papers/soon-2001-machine-learning-approach-coreference-resolution/)
23. Ellen Vorhees. 2002. Overview of the TREC 2002 question answering track. In Proceedings of the Eleventh Text REtrieval Conference (TREC). [find](https://scholar.google.com/scholar?q=Ellen+Vorhees%2E+2002%2E+Overview+of+the+TREC+2002+question+answering+track%2E+In+Proceedings+of+the+Eleventh+Text+REtrieval+Conference+%28TREC%29%2E)