--- title: "Probabilistic Coreference in Information — Kehler 1997" description: "Most information extraction systems commit to a single answer when deciding which entity descriptions refer to the same thing." authors: ["Andrew Kehler"] year: 1997 venue: "Second Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/W97-0319/" arxiv: "cmp-lg/9706012" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/kehler-1997-probabilistic-coreference-information-extraction/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/kehler-1997-probabilistic-coreference-information-extraction/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [1997](https://md.research.elamur.ai/free/papers/1997/) / Probabilistic Coreference in Information Extraction # Probabilistic Coreference in Information Extraction **Authors:** [Andrew Kehler](https://md.research.elamur.ai/free/authors/andrew-kehler/) · **Year:** 1997 · **Venue:** Second Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W97-0319/) · [PDF](https://aclanthology.org/W97-0319.pdf) · [arXiv cmp-lg/9706012](https://arxiv.org/abs/cmp-lg/9706012) · [OpenAlex](https://openalex.org/W1710827551) **Citations:** 52 citations (openalex-snapshot · 2026-10-14) · [22](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Most information extraction systems commit to a single answer when deciding which entity descriptions refer to the same thing. That leaves a downstream data-fusion component with no sense of how reliable the output is. Kehler instead treats template merging in SRI's FASTUS as a probabilistic problem. A "coreference set" is a group of templates that might corefer. The goal is a probability distribution over all ways of partitioning that set ("coreference configurations"), with known incompatibilities, such as clashing facility types, ruled out in advance. Hand-coded training data is scarce and covers few large sets, so the approach has two stages: - A maximum entropy model (Berger et al., 1996) estimates coreference for individual template pairs. - Two strategies, "merging decision" and "evidential", combine the pairwise scores into a distribution over full configurations. On held-out data, both strategies beat uniform and greedy baselines in cross-entropy, and the evidential method did best. The work matters because calibrated probabilities let fusion systems weigh IE output against other sources. The same machinery could also replace deterministic merging inside an IE system. ## Key points - The setting is FASTUS applied to short military-style messages. In a worked example, four depot templates yield seven admissible configurations once type conflicts are excluded. - Pairwise coreference is learned with maximum entropy over binary features (e.g., identical slot values, phrase form, textual distance), trained via improved iterative scaling. - The pairwise models trained on merging-decision data reached a training cross-entropy of about 0.70, clearly lower than the evidential pairwise model. The paper attributes this to cleaner training data. - Evaluation cross-entropy (Table 2) followed the same ranking on all three training/test divisions: | Method | Test Set 1 | Test Set 2 | Third division (more test data) | |---|---|---|---| | Uniform | 2.12 | 1.76 | 2.01 | | Greedy | 1.50 | 1.30 | 1.41 | | Merging decision | 1.32 | 1.13 | 1.27 | | Evidential | 1.10 | 0.89 | 1.00 | - Suggested future work includes richer context features (e.g., subject lines and tables, separate treatment of pronouns). Pruning and smoothing will be needed for longer texts with much larger coreference sets. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) Topics: [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/) ## Cited by Cited by 22 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 9 not yet. - [Corpus-Based Identification of Non-Anaphoric Noun Phrases](https://md.research.elamur.ai/free/papers/bean-1999-corpus-based-identification-non-anaphoric/) - [Unsupervised Learning of Contextual Role Knowledge for Coreference Resolution](https://md.research.elamur.ai/free/papers/bean-2004-unsupervised-learning-contextual-role-knowledge/) - [An Expectation Maximization Approach to Pronoun Resolution](https://md.research.elamur.ai/free/papers/cherry-2005-expectation-maximization-approach-pronoun-resolution/) - [Entity-Centric Coreference Resolution with Model Stacking](https://md.research.elamur.ai/free/papers/clark-2015-entity-centric-coreference-resolution-model/) - [Joint Determination of Anaphoricity and Coreference Resolution using Integer Programming](https://md.research.elamur.ai/free/papers/denis-2007-joint-determination-anaphoricity-coreference-resolution/) - [Multilingual Coreference Resolution](https://md.research.elamur.ai/free/papers/harabagiu-2000-multilingual-coreference-resolution/) - [Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules](https://md.research.elamur.ai/free/papers/lee-2013-deterministic-coreference-resolution-based-entity/) - [A Mention-Synchronous Coreference Resolution Algorithm Based On the Bell Tree](https://md.research.elamur.ai/free/papers/luo-2004-mention-synchronous-coreference-resolution-algorithm/) - [Combining Sample Selection and Error-Driven Pruning for Machine Learning of Coreference Rules](https://md.research.elamur.ai/free/papers/ng-2002-combining-sample-selection-error-driven/) - [Machine Learning for Coreference Resolution: From Local Classification to Global Ranking](https://md.research.elamur.ai/free/papers/ng-2005-machine-learning-coreference-resolution-local/) - [Supervised Noun Phrase Coreference Research: The First Fifteen Years](https://md.research.elamur.ai/free/papers/ng-2010-supervised-noun-phrase-coreference-research/) - [A Machine Learning Approach to Coreference Resolution of Noun Phrases](https://md.research.elamur.ai/free/papers/soon-2001-machine-learning-approach-coreference-resolution/) - [A Twin-Candidate Model for Learning-Based Anaphora Resolution](https://md.research.elamur.ai/free/papers/yang-2008-twin-candidate-model-learning-based/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{kehler1997probabilistic, title = {{Probabilistic Coreference in Information Extraction}}, author = {Kehler, Andrew}, booktitle = {Second Conference on Empirical Methods in Natural Language Processing}, year = {1997}, eprint = {cmp-lg/9706012}, archivePrefix = {arXiv}, url = {https://arxiv.org/abs/cmp-lg/9706012} } ``` - [← previous On representing language-specific information in interlingua](https://md.research.elamur.ai/free/papers/farwell-1997-representing-language-specific-information-interlingua/) - [next → Resolving bridging references in unrestricted text](https://md.research.elamur.ai/free/papers/poesio-1997-resolving-bridging-references-unrestricted-text/) - [↑ back to papers of 1997](https://md.research.elamur.ai/free/papers/1997/) ## References 10 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Aone, Chinatsu and Scott William Bennett. 1995. Evaluating automated and manual acquisition of anaphora resolution strategies. In Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics (ACL-95), pages 122-129, Cambridge, MA, June. [find](https://scholar.google.com/scholar?q=Aone%2C+Chinatsu+and+Scott+William+Bennett%2E+1995%2E+Evaluating+automated+and+manual+acquisition+of+anaphora+resolution+strategies%2E+In+Proceedings+of+the+33rd+Annual+Meeting+of+the+Association+for+Computat) [our card](https://md.research.elamur.ai/free/papers/aone-1995-evaluating-automated-manual-acquisition-anaphora/) 2. Berger, Adam, Stephen A. Della Pietra, and Vincent J. Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39-71. [find](https://scholar.google.com/scholar?q=Berger%2C+Adam%2C+Stephen+A%2E+Della+Pietra%2C+and+Vincent+J%2E+Della+Pietra%2E+1996%2E+A+maximum+entropy+approach+to+natural+language+processing%2E+Computational+Linguistics%2C+22%281%29%3A39%2D71%2E) [our card](https://md.research.elamur.ai/free/papers/berger-1996-maximum-entropy-approach-natural-language/) 3. Connolly, Dennis, John D. Burger, and David S. Day. 1994. A machine learning approach to anaphoric reference. In Proceedings of the International Conference on New Methods in Language Processing (NeMLaP). [find](https://scholar.google.com/scholar?q=Connolly%2C+Dennis%2C+John+D%2E+Burger%2C+and+David+S%2E+Day%2E+1994%2E+A+machine+learning+approach+to+anaphoric+reference%2E+In+Proceedings+of+the+International+Conference+on+New+Methods+in+Language+Processing+%28NeML) 4. Dagan, Ido and Alon Itai. 1990. Automatic acquisition of constraints for the resolution of anaphora references and syntactic ambiguities. In Proceedings of the 13th International Conference on Computational Linguistics (COLING-90), pages 330-332. [find](https://scholar.google.com/scholar?q=Dagan%2C+Ido+and+Alon+Itai%2E+1990%2E+Automatic+acquisition+of+constraints+for+the+resolution+of+anaphora+references+and+syntactic+ambiguities%2E+In+Proceedings+of+the+13th+International+Conference+on+Computa) 5. Dagan, Ido, John Justenson, Shalom Lappin, Herbert Leass, and Amnon Ribak. 1995. Syntax and lexical statistics in anaphora resolution. Applied Artificial Intelligence, 9(6):633-644, Nov/Dec. [find](https://scholar.google.com/scholar?q=Dagan%2C+Ido%2C+John+Justenson%2C+Shalom+Lappin%2C+Herbert+Leass%2C+and+Amnon+Ribak%2E+1995%2E+Syntax+and+lexical+statistics+in+anaphora+resolution%2E+Applied+Artificial+Intelligence%2C+9%286%29%3A633%2D644%2C+Nov%2FDec%2E) 6. Dempster, Arthur P. 1968. A generalization of Bayesian inference. Journal of the Royal Statistical Society, 30:205-247. [find](https://scholar.google.com/scholar?q=Dempster%2C+Arthur+P%2E+1968%2E+A+generalization+of+Bayesian+inference%2E+Journal+of+the+Royal+Statistical+Society%2C+30%3A205%2D247%2E) 7. Hobbs, Jerry R., Douglas E. Appelt, John Bear, David Israel, Megumi Kameyama, Mark Stickel, and Mabry Tyson. 1996. FASTUS: A cascaded finite-state transducer for extracting information from natural-language text. In Finite State Devices for Natural Language Processing. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Hobbs%2C+Jerry+R%2E%2C+Douglas+E%2E+Appelt%2C+John+Bear%2C+David+Israel%2C+Megumi+Kameyama%2C+Mark+Stickel%2C+and+Mabry+Tyson%2E+1996%2E+FASTUS%3A+A+cascaded+finite%2Dstate+transducer+for+extracting+information+from+natural%2Dla) 8. Kennedy, Christopher and Branimir Boguraev. 1996a. Anaphora for everyone: Pronominal anaphora resolution without a parser. In Proceedings of the 16th International Conference on Computational Linguistics (COLING-96). [find](https://scholar.google.com/scholar?q=Kennedy%2C+Christopher+and+Branimir+Boguraev%2E+1996a%2E+Anaphora+for+everyone%3A+Pronominal+anaphora+resolution+without+a+parser%2E+In+Proceedings+of+the+16th+International+Conference+on+Computational+Linguist) [our card](https://md.research.elamur.ai/free/papers/kennedy-1996-anaphora-everyone-pronominal-anaphora-resolution/) 9. Kennedy, Christopher and Branimir Boguraev. 1996b. Anaphora in a wider context: Tracking discourse referents. In Proceedings of the 12th European Conference on Artificial Intelligence (ECAI-96). [find](https://scholar.google.com/scholar?q=Kennedy%2C+Christopher+and+Branimir+Boguraev%2E+1996b%2E+Anaphora+in+a+wider+context%3A+Tracking+discourse+referents%2E+In+Proceedings+of+the+12th+European+Conference+on+Artificial+Intelligence+%28ECAI%2D96%29%2E) 10. Lappin, Shalom and Herbert Leass. 1994. An algorithm for pronominal anaphora resolution. Computational Linguistics, 20(4):535-561. [find](https://scholar.google.com/scholar?q=Lappin%2C+Shalom+and+Herbert+Leass%2E+1994%2E+An+algorithm+for+pronominal+anaphora+resolution%2E+Computational+Linguistics%2C+20%284%29%3A535%2D561%2E) [our card](https://md.research.elamur.ai/free/papers/lappin-1994-algorithm-pronominal-anaphora-resolution/)