--- title: "Understanding the Value of Features for — Bengtson 2008" description: "Most coreference research before this paper focused on new model structures and usually compared them against weak pairwise baselines." authors: ["Eric Bengtson", "Dan Roth"] year: 2008 venue: "Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/D08-1031/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/bengtson-2008-understanding-value-features-coreference-resolution/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bengtson-2008-understanding-value-features-coreference-resolution/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2008](https://md.research.elamur.ai/free/en/papers/2008/) / Understanding the Value of Features for Coreference Resolution # Understanding the Value of Features for Coreference Resolution **Authors:** Eric Bengtson, [Dan Roth](https://md.research.elamur.ai/free/en/authors/dan-roth/) · **Year:** 2008 · **Venue:** Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/D08-1031/) · [PDF](https://aclanthology.org/D08-1031.pdf) · [OpenAlex](https://openalex.org/W2050273484) **Citations:** 254 citations (openalex-snapshot · 2026-10-14) · [98](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Most coreference research before this paper focused on new model structures and usually compared them against weak pairwise baselines. Bengtson and Roth argue that carefully engineered features may matter as much as the model. They build a deliberately simple pairwise classifier: an averaged perceptron, trained in Learning Based Java, scores pairs of mentions. The Best-Link strategy then attaches each mention to its highest-scoring preceding candidate if that score clears a threshold. Pronouns are not allowed to serve as antecedents for non-pronominal mentions. The features cover string matching, semantic agreement based on WordNet and word lists, apposition, distance, aligned modifiers, memorization, and predicted entity types, and all pairwise conjunctions of these are added. Trained and tested on ACE data, the system beats the more complex cluster-level model of Culotta et al. (2007) by 1.5 B-Cubed F points. The authors offer it as a strong baseline for checking whether structural modeling innovations still pay off. An ablation study and an end-to-end version that runs on raw text are also included. ## Key points - On the development set, adding feature groups one at a time raises B-Cubed F from 75.76 to 81.79. Distance, apposition and entity-type features give the largest statistically significant gains. - Predicted entity types are highly discriminative. When types agree, 13% of mention pairs are coreferent, against about 6% overall. Pairs that fire the relative-pronoun feature are positive 93% of the time. - For training, each mention is paired with only its closest preceding coreferent mention as a positive example. Every earlier non-coreferent mention gives a negative example. - The end-to-end pipeline has a chunking-based head detector (about 90% F1), an extent classifier and a mention-type classifier. On unseen ACE test data it reaches P 84.91, R 72.53 and B-Cubed F 78.24. - The full feature package and the LBJ code were released publicly by the Cognitive Computation Group at UIUC. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/) ## Cited by Cited by 98 papers of our library cite it (of the 140673 whose bibliographies we read); 51 of them have a page here, 47 not yet. - [Antecedent Selection for Sluicing: Structure and Content](https://md.research.elamur.ai/free/en/papers/aclbulk-2016-anand-antecedent-selection-sluicing-structure-content/) - [Detecting Subevent Structure for Event Coreference Resolution](https://md.research.elamur.ai/free/en/papers/araki-2014-detecting-subevent-structure-event-coreference/) - [Coreference Semantics from Web Features](https://md.research.elamur.ai/free/en/papers/bansal-2012-coreference-semantics-web-features/) - [Learning Structured Perceptrons for Coreference Resolution with Latent Antecedents and Non-local Features](https://md.research.elamur.ai/free/en/papers/bjorkelund-2014-learning-structured-perceptrons-coreference-resolution/) - [End-to-End Coreference Resolution via Hypergraph Partitioning](https://md.research.elamur.ai/free/en/papers/cai-2010-end-end-coreference-resolution-hypergraph/) - [Evaluation Metrics For End-to-End Coreference Resolution Systems](https://md.research.elamur.ai/free/en/papers/cai-2010-evaluation-metrics-end-end-coreference/) - [Inference Protocols for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/chang-2011-inference-protocols-coreference-resolution/) - [A Constrained Latent Variable Model for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/chang-2013-constrained-latent-variable-model-coreference/) - [Entity-Centric Coreference Resolution with Model Stacking](https://md.research.elamur.ai/free/en/papers/clark-2015-entity-centric-coreference-resolution-model/) - [Improving Coreference Resolution by Learning Entity-Level Distributed Representations](https://md.research.elamur.ai/free/en/papers/clark-2016-improving-coreference-resolution-learning-entity/) - [Decentralized Entity-Level Modeling for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/durrett-2013-decentralized-entity-level-modeling-coreference/) - [Easy Victories and Uphill Battles in Coreference Resolution](https://md.research.elamur.ai/free/en/papers/durrett-2013-easy-victories-uphill-battles-coreference/) - [Latent Trees for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/fernandes-2014-latent-trees-coreference-resolution/) - [Removing the Training Wheels: A Coreference Dataset that Entertains Humans and Challenges Computers](https://md.research.elamur.ai/free/en/papers/guha-2015-removing-training-wheels-coreference-dataset/) - [BERT for Coreference Resolution: Baselines and Analysis](https://md.research.elamur.ai/free/en/papers/joshi-2019-bert-coreference-resolution-baselines-analysis/) - [Using Type Information to Improve Entity Coreference Resolution](https://md.research.elamur.ai/free/en/papers/khosla-2020-type-information-improve-entity-coreference/) - [Error-Driven Analysis of Challenges in Coreference Resolution](https://md.research.elamur.ai/free/en/papers/kummerfeld-2013-error-driven-analysis-challenges-coreference/) - [Neural Cross-Lingual Coreference Resolution And Its Application To Entity Linking](https://md.research.elamur.ai/free/en/papers/kundu-2018-neural-cross-lingual-coreference-resolution/) - [Improving pairwise coreference models through feature space hierarchy learning](https://md.research.elamur.ai/free/en/papers/lassalle-2013-improving-pairwise-coreference-models-through/) - [Stanford's Multi-Pass Sieve Coreference Resolution System at the CoNLL-2011 Shared Task](https://md.research.elamur.ai/free/en/papers/lee-2011-stanford-s-multi-pass-sieve/) - [Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules](https://md.research.elamur.ai/free/en/papers/lee-2013-deterministic-coreference-resolution-based-entity/) - [Higher-Order Coreference Resolution with Coarse-to-Fine Inference](https://md.research.elamur.ai/free/en/papers/lee-2018-higher-order-coreference-resolution-coarse/) - [Prune-and-Score: Learning for Greedy Coreference Resolution](https://md.research.elamur.ai/free/en/papers/ma-2014-prune-score-learning-greedy-coreference/) - [Transferring Coreference Resolvers with Posterior Regularization](https://md.research.elamur.ai/free/en/papers/martins-2015-transferring-coreference-resolvers-posterior-regularization/) - [Recall Error Analysis for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/martschat-2014-recall-error-analysis-coreference-resolution/) - [Latent Structures for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/martschat-2015-latent-structures-coreference-resolution/) - [Using Linguistic Features to Improve the Generalization Capability of Neural Coreference Resolvers](https://md.research.elamur.ai/free/en/papers/moosavi-2018-linguistic-features-improve-generalization-capability/) - [LingMess: Linguistically Informed Multi Expert Scorers for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/otmazgin-2023-lingmess-linguistically-informed-multi-expert/) - [A Joint Framework for Coreference Resolution and Mention Head Detection](https://md.research.elamur.ai/free/en/papers/peng-2015-joint-framework-coreference-resolution-mention/) - [Solving Hard Coreference Problems](https://md.research.elamur.ai/free/en/papers/peng-2015-solving-hard-coreference-problems/) - [Event Detection and Co-reference with Minimal Supervision](https://md.research.elamur.ai/free/en/papers/peng-2016-event-detection-co-reference-minimal/) - [A Multi-Pass Sieve for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/raghunathan-2010-multi-pass-sieve-coreference-resolution/) - [Learning-based Multi-Sieve Co-reference Resolution with Knowledge](https://md.research.elamur.ai/free/en/papers/ratinov-2012-learning-based-multi-sieve-co/) - [Coreference Resolution across Corpora: Languages, Coding Schemes, and Preprocessing Information](https://md.research.elamur.ai/free/en/papers/recasens-2010-coreference-resolution-across-corpora-languages/) - [A Constraint-Based Hypergraph Partitioning Approach to Coreference Resolution](https://md.research.elamur.ai/free/en/papers/sapena-2013-constraint-based-hypergraph-partitioning-approach/) - [A Fully Coreference-annotated Corpus of Scholarly Papers from the ACL Anthology](https://md.research.elamur.ai/free/en/papers/schafer-2012-fully-coreference-annotated-corpus-scholarly/) - [Large-Scale Cross-Document Coreference Using Distributed Inference and Hierarchical Models](https://md.research.elamur.ai/free/en/papers/singh-2011-large-scale-cross-document-coreference/) - [Conundrums in Noun Phrase Coreference Resolution: Making Sense of the State-of-the-Art](https://md.research.elamur.ai/free/en/papers/stoyanov-2009-conundrums-noun-phrase-coreference-resolution/) - [Coreference Resolution with Reconcile](https://md.research.elamur.ai/free/en/papers/stoyanov-2010-coreference-resolution-reconcile/) - [Easy-first Coreference Resolution](https://md.research.elamur.ai/free/en/papers/stoyanov-2012-easy-first-coreference-resolution/) - [CREAD: Combined Resolution of Ellipses and Anaphora in Dialogues](https://md.research.elamur.ai/free/en/papers/tseng-2021-cread-combined-resolution-ellipses-anaphora/) - [Revisiting the Evaluation for Cross Document Event Coreference](https://md.research.elamur.ai/free/en/papers/upadhyay-2016-revisiting-evaluation-cross-document-event/) - [The More Antecedents, the Merrier: Resolving Multi-Antecedent Anaphors](https://md.research.elamur.ai/free/en/papers/vala-2016-more-antecedents-merrier-resolving-multi/) - [Limited memory incremental coreference resolution](https://md.research.elamur.ai/free/en/papers/webster-2014-limited-memory-incremental-coreference-resolution/) - [Using Coreference Links to Improve Spanish-to-English Machine Translation](https://md.research.elamur.ai/free/en/papers/werlen-2017-coreference-links-improve-spanish-english/) - [A Discriminative Hierarchical Model for Fast Coreference at Large Scale](https://md.research.elamur.ai/free/en/papers/wick-2012-discriminative-hierarchical-model-fast-coreference/) - [Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/wiseman-2015-learning-anaphoricity-antecedent-ranking-features/) - [Learning Global Features for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/wiseman-2016-learning-global-features-coreference-resolution/) - [Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering](https://md.research.elamur.ai/free/en/papers/zhang-2018-neural-coreference-resolution-deep-biaffine/) - [PCR4ALL: A Comprehensive Evaluation Benchmark for Pronoun Coreference Resolution in English](https://md.research.elamur.ai/free/en/papers/zhao-2022-pcr4all-comprehensive-evaluation-benchmark-pronoun/) - [Dynamic Knowledge-Base Alignment for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/zheng-2013-dynamic-knowledge-base-alignment-coreference/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{bengtson2008understanding, title = {{Understanding the Value of Features for Coreference Resolution}}, author = {Bengtson, Eric and Roth, Dan}, booktitle = {Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing}, year = {2008}, doi = {10.3115/1613715.1613756}, url = {https://doi.org/10.3115/1613715.1613756} } ``` - [← previous Tree-distance and some other variants of evalb](https://md.research.elamur.ai/free/en/papers/lingbulk-2008-emms-tree-distance-some-other-variants/) - [next → Unsupervised Models for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/ng-2008-unsupervised-models-coreference-resolution/) - [↑ back to papers of 2008](https://md.research.elamur.ai/free/en/papers/2008/) ## References 15 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. A. Bagga and B. Baldwin. 1998. Algorithms for scoring coreference chains. In MUC7. [find](https://scholar.google.com/scholar?q=A%2E+Bagga+and+B%2E+Baldwin%2E+1998%2E+Algorithms+for+scoring+coreference+chains%2E+In+MUC7%2E) [our card](https://md.research.elamur.ai/free/en/papers/bagga-1998-algorithms-scoring-coreference-chains/) 2. A. Culotta, M. Wick, R. Hall, and A. McCallum. 2007. First-order probabilistic models for coreference resolution. In HLT/NAACL, pages 81–88. [find](https://scholar.google.com/scholar?q=A%2E+Culotta%2C+M%2E+Wick%2C+R%2E+Hall%2C+and+A%2E+McCallum%2E+2007%2E+First%2Dorder+probabilistic+models+for+coreference+resolution%2E+In+HLT%2FNAACL%2C+pages+81%E2%80%9388%2E) [our card](https://md.research.elamur.ai/free/en/papers/culotta-2007-first-order-probabilistic-models-coreference/) 3. P. Denis and J. Baldridge. 2007. Joint determination of anaphoricity and coreference resolution using integer programming. In HLT/NAACL, pages 236–243, Rochester, New York. [find](https://scholar.google.com/scholar?q=P%2E+Denis+and+J%2E+Baldridge%2E+2007%2E+Joint+determination+of+anaphoricity+and+coreference+resolution+using+integer+programming%2E+In+HLT%2FNAACL%2C+pages+236%E2%80%93243%2C+Rochester%2C+New+York%2E) [our card](https://md.research.elamur.ai/free/en/papers/denis-2007-joint-determination-anaphoricity-coreference-resolution/) 4. C. Fellbaum. 1998. WordNet: An Electronic Lexical Database. MIT Press. [find](https://scholar.google.com/scholar?q=C%2E+Fellbaum%2E+1998%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+MIT+Press%2E) 5. Y. Freund and R. E. Schapire. 1998. Large margin classification using the Perceptron algorithm. In COLT, pages 209–217. [↗](https://doi.org/10.1023/a:1007662407062) 6. H. Ji, D. Westbrook, and R. Grishman. 2005. Using semantic relations to refine coreference decisions. In EMNLP/HLT, pages 17–24, Vancouver, British Columbia, Canada. [find](https://scholar.google.com/scholar?q=H%2E+Ji%2C+D%2E+Westbrook%2C+and+R%2E+Grishman%2E+2005%2E+Using+semantic+relations+to+refine+coreference+decisions%2E+In+EMNLP%2FHLT%2C+pages+17%E2%80%9324%2C+Vancouver%2C+British+Columbia%2C+Canada%2E) [our card](https://md.research.elamur.ai/free/en/papers/ji-2005-semantic-relations-refine-coreference-decisions/) 7. X. Luo and I. Zitouni. 2005. Multi-lingual coreference resolution with syntactic features. In HLT/EMNLP, pages 660–667, Vancouver, British Columbia, Canada. [find](https://scholar.google.com/scholar?q=X%2E+Luo+and+I%2E+Zitouni%2E+2005%2E+Multi%2Dlingual+coreference+resolution+with+syntactic+features%2E+In+HLT%2FEMNLP%2C+pages+660%E2%80%93667%2C+Vancouver%2C+British+Columbia%2C+Canada%2E) [our card](https://md.research.elamur.ai/free/en/papers/luo-2005-multi-lingual-coreference-resolution-syntactic/) 8. X. Luo, A. Ittycheriah, H. Jing, N. Kambhatla, and S. Roukos. 2004. A mention-synchronous coreference resolution algorithm based on the bell tree. In ACL, page 135, Morristown, NJ, USA. [find](https://scholar.google.com/scholar?q=X%2E+Luo%2C+A%2E+Ittycheriah%2C+H%2E+Jing%2C+N%2E+Kambhatla%2C+and+S%2E+Roukos%2E+2004%2E+A+mention%2Dsynchronous+coreference+resolution+algorithm+based+on+the+bell+tree%2E+In+ACL%2C+page+135%2C+Morristown%2C+NJ%2C+USA%2E) [our card](https://md.research.elamur.ai/free/en/papers/luo-2004-mention-synchronous-coreference-resolution-algorithm/) 9. V. Ng and C. Cardie. 2002a. Identifying anaphoric and non-anaphoric noun phrases to improve coreference resolution. In COLING-2002. [find](https://scholar.google.com/scholar?q=V%2E+Ng+and+C%2E+Cardie%2E+2002a%2E+Identifying+anaphoric+and+non%2Danaphoric+noun+phrases+to+improve+coreference+resolution%2E+In+COLING%2D2002%2E) 10. V. Ng and C. Cardie. 2002b. Improving machine learning approaches to coreference resolution. In ACL. [find](https://scholar.google.com/scholar?q=V%2E+Ng+and+C%2E+Cardie%2E+2002b%2E+Improving+machine+learning+approaches+to+coreference+resolution%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/ng-2002-improving-machine-learning-approaches-coreference/) 11. NIST. 2004. The ace evaluation plan. www.nist.gov/speech/tests/ace/index.htm. [find](https://scholar.google.com/scholar?q=NIST%2E+2004%2E+The+ace+evaluation+plan%2E+www%2Enist%2Egov%2Fspeech%2Ftests%2Face%2Findex%2Ehtm%2E) 12. V. Punyakanok and D. Roth. 2001. The use of classifiers in sequential inference. In The Conference on Advances in Neural Information Processing Systems (NIPS), pages 995–1001. MIT Press. [↗](https://doi.org/10.48550/arxiv.cs/0111003) 13. N. Rizzolo and D. Roth. 2007. Modeling Discriminative Global Inference. In Proceedings of the First International Conference on Semantic Computing (ICSC), pages 597–604, Irvine, California. [find](https://scholar.google.com/scholar?q=N%2E+Rizzolo+and+D%2E+Roth%2E+2007%2E+Modeling+Discriminative+Global+Inference%2E+In+Proceedings+of+the+First+International+Conference+on+Semantic+Computing+%28ICSC%29%2C+pages+597%E2%80%93604%2C+Irvine%2C+California%2E) 14. W. M. Soon, H. T. Ng, and C. Y. Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521– 544. [↗](https://doi.org/10.1162/089120101753342653) [our card](https://md.research.elamur.ai/free/en/papers/soon-2001-machine-learning-approach-coreference-resolution/) 15. M. Vilain, J. Burger, J. Aberdeen, D. Connolly, and L. Hirschman. 1995. A model-theoretic coreference scoring scheme. In MUC6, pages 45–52. [find](https://scholar.google.com/scholar?q=M%2E+Vilain%2C+J%2E+Burger%2C+J%2E+Aberdeen%2C+D%2E+Connolly%2C+and+L%2E+Hirschman%2E+1995%2E+A+model%2Dtheoretic+coreference+scoring+scheme%2E+In+MUC6%2C+pages+45%E2%80%9352%2E) [our card](https://md.research.elamur.ai/free/en/papers/vilain-1995-model-theoretic-coreference-scoring-scheme/)