--- title: "Higher-Order Coreference Resolution with — Lee 2018" description: "First-order coreference systems score only pairs of mentions." authors: ["Kenton Lee", "Luheng He", "Luke Zettlemoyer"] year: 2018 venue: "Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)" source: "https://aclanthology.org/N18-2108/" doi: "10.18653/v1/n18-2108" arxiv: "1804.05392" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/lee-2018-higher-order-coreference-resolution-coarse/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lee-2018-higher-order-coreference-resolution-coarse/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / Higher-Order Coreference Resolution with Coarse-to-Fine Inference # Higher-Order Coreference Resolution with Coarse-to-Fine Inference **Authors:** [Kenton Lee](https://md.research.elamur.ai/free/en/authors/kenton-lee/), [Luheng He](https://md.research.elamur.ai/free/en/authors/luheng-he/), [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/) · **Year:** 2018 · **Venue:** Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/N18-2108/) · [PDF](https://aclanthology.org/N18-2108.pdf) · [DOI 10.18653/v1/n18-2108](https://doi.org/10.18653/v1/n18-2108) · [arXiv 1804.05392](https://arxiv.org/abs/1804.05392) · [OpenAlex](https://openalex.org/W2797051334) **Citations:** 371 citations (openalex-snapshot · 2026-10-14) · [240](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary First-order coreference systems score only pairs of mentions. Each link is decided on its own, so a cluster can be made of links that each look plausible but don't fit together. In the paper's example, an ambiguous "you" connects a singular "I" with a plural "all of you". The authors extend the end-to-end span-ranking model of Lee et al. (2017) with a differentiable approximation of higher-order inference. The predicted antecedent distribution acts as attention and produces an expected antecedent vector. A learned gate merges this vector into each span's representation, and the spans are then rescored. Repeating this step is expensive, so they add a cheap bilinear "coarse" factor. It ranks every candidate antecedent and passes only the best few to the costly "fine" scorer, and the whole model is trained with a single objective. On the English CoNLL-2012 (OntoNotes) data the system reaches 73.0 average F1, a new state of the art at the time, while scoring far fewer antecedents per span. The work shows that a model can enforce consistent clusters and still be efficient. ## Key points - Baseline: Lee et al. (2017) with ELMo embeddings and tuned hyperparameters, scoring 72.3 average F1 (MUC, B³, CEAFφ4) on the test set. Adding coarse-to-fine pruning gives 72.6, and adding second-order inference gives 73.0. - The coarse-to-fine model sends only 50 antecedents per span to the fine scorer, compared with the 250 nearest used by the distance heuristic. On the dev set, cutting K from 250 to 50 costs under 0.2 F1 with coarse-to-fine pruning but almost 5 F1 with the heuristic. - Using the bilinear score on its own instead of the full pairwise scorer loses more than 3 F1. It works as a pre-filter, not as the only scorer. - Second-order inference adds 0.8 F1 on the dev set, and a third iteration adds just 0.1. The gain comes mainly from higher precision, while coarse-to-fine pruning mainly raises recall. - The code is public at github.com/kentonl/e2e-coref. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/) ## Cited by Cited by 240 papers of our library cite it (of the 140673 whose bibliographies we read); 82 of them have a page here, 158 not yet. - [Neural Coreference Resolution for Arabic](https://md.research.elamur.ai/free/en/papers/aloraini-2020-neural-coreference-resolution-arabic/) - [Anaphora Resolution in Dialogue: Description of the DFKI-TalkingRobots System for the CODI-CRAC 2021 Shared-Task](https://md.research.elamur.ai/free/en/papers/anikina-2021-anaphora-resolution-dialogue-description-dfki/) - [Rewarding Coreference Resolvers for Being Consistent with World Knowledge](https://md.research.elamur.ai/free/en/papers/aralikatte-2019-rewarding-coreference-resolvers-being-consistent/) - [CorefInst: Leveraging LLMs for Multilingual Coreference Resolution](https://md.research.elamur.ai/free/en/papers/arslan-2026-corefinst-leveraging-llms-multilingual-coreference/) - [Deep Enhanced Representation for Implicit Discourse Relation Recognition](https://md.research.elamur.ai/free/en/papers/bai-2018-deep-enhanced-representation-implicit-discourse/) - [An Annotated Dataset of Coreference in English Literature](https://md.research.elamur.ai/free/en/papers/bamman-2020-annotated-dataset-coreference-english-literature/) - [Revisiting Joint Modeling of Cross-document Entity and Event Coreference Resolution](https://md.research.elamur.ai/free/en/papers/barhom-2019-revisiting-joint-modeling-cross-document/) - [Character Coreference Resolution in Movie Screenplays](https://md.research.elamur.ai/free/en/papers/baruah-2023-character-coreference-resolution-movie-screenplays/) - [Recognition of They/Them as Singular Personal Pronouns in Coreference Resolution](https://md.research.elamur.ai/free/en/papers/baumler-2022-recognition-they-them-as-singular/) - [Coreference Resolution through a seq2seq Transition-Based System](https://md.research.elamur.ai/free/en/papers/bohnet-2023-coreference-resolution-through-seq2seq-transition/) - [Investigating Multilingual Coreference Resolution by Universal Annotations](https://md.research.elamur.ai/free/en/papers/chai-2023-investigating-multilingual-coreference-resolution-universal/) - [Tackling Zero Pronoun Resolution and Non-Zero Coreference Resolution Jointly](https://md.research.elamur.ai/free/en/papers/chen-2021-tackling-zero-pronoun-resolution-non/) - [Neural End-to-end Coreference Resolution for German in Different Domains](https://md.research.elamur.ai/free/en/papers/coref--schroder-neural-end-end-coreference-resolution/) - [Do UD Trees Match Mention Spans in Coreference Annotations?](https://md.research.elamur.ai/free/en/papers/coref-2021-popel-do-ud-trees-match-mention/) - [NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation](https://md.research.elamur.ai/free/en/papers/dhole-2023-nl-augmenter-framework-task-sensitive/) - [Word-Level Coreference Resolution](https://md.research.elamur.ai/free/en/papers/dobrovolskii-2021-word-level-coreference-resolution/) - [Multi-Sentence Argument Linking](https://md.research.elamur.ai/free/en/papers/ebner-2020-multi-sentence-argument-linking/) - [Cross-document Event Coreference Search: Task, Dataset and Modeling](https://md.research.elamur.ai/free/en/papers/eirew-2022-cross-document-event-coreference-search/) - [The KnowRef Coreference Corpus: Removing Gender and Number Cues for Difficult Pronominal Anaphora Resolution](https://md.research.elamur.ai/free/en/papers/emami-2019-knowref-coreference-corpus-removing-gender/) - [ChEMU-Ref: A Corpus for Modeling Anaphora Resolution in the Chemical Domain](https://md.research.elamur.ai/free/en/papers/fang-2021-chemu-ref-corpus-modeling-anaphora/) - [What does it take to bake a cake? The RecipeRef corpus and anaphora resolution in procedural text](https://md.research.elamur.ai/free/en/papers/fang-2022-what-does-it-take-bake/) - [End-to-end Deep Reinforcement Learning Based Coreference Resolution](https://md.research.elamur.ai/free/en/papers/fei-2019-end-end-deep-reinforcement-learning/) - [Annotating Mentions Alone Enables Efficient Domain Adaptation for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/gandhi-2023-annotating-mentions-alone-enables-efficient/) - [Neural Coreference Resolution with Limited Lexical Context and Explicit Mention Detection for Oral French](https://md.research.elamur.ai/free/en/papers/grobol-2019-neural-coreference-resolution-limited-lexical/) - [Coreference for Discourse Parsing: A Neural Approach](https://md.research.elamur.ai/free/en/papers/guz-2020-coreference-discourse-parsing-neural-approach/) - [FantasyCoref: Coreference Resolution on Fantasy Literature Through Omniscient Writer's Point of View](https://md.research.elamur.ai/free/en/papers/han-2021-fantasycoref-coreference-resolution-fantasy-literature/) - [Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/he-2018-jointly-predicting-predicates-arguments-neural/) - [Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference](https://md.research.elamur.ai/free/en/papers/held-2021-focus-what-matters-applying-discourse/) - [NARC – Norwegian Anaphora Resolution Corpus](https://md.research.elamur.ai/free/en/papers/hlum-2022-narc-norwegian-anaphora-resolution-corpus/) - [Incorporating Syntax and Semantics in Coreference Resolution with Heterogeneous Graph Attention Network](https://md.research.elamur.ai/free/en/papers/jiang-2021-incorporating-syntax-semantics-coreference-resolution/) - [Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction](https://md.research.elamur.ai/free/en/papers/jie-2022-learning-reason-deductively-math-word/) - [BERT for Coreference Resolution: Baselines and Analysis](https://md.research.elamur.ai/free/en/papers/joshi-2019-bert-coreference-resolution-baselines-analysis/) - [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://md.research.elamur.ai/free/en/papers/joshi-2019-spanbert-improving-pre-training-representing/) - [Coreference Resolution with Entity Equalization](https://md.research.elamur.ai/free/en/papers/kantor-2019-coreference-resolution-entity-equalization/) - [Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation](https://md.research.elamur.ai/free/en/papers/levy-2021-collecting-large-scale-gender-bias/) - [AMALGUM – A Free, Balanced, Multilayer English Web Corpus](https://md.research.elamur.ai/free/en/papers/lingbulk-2020-gessler-amalgum-free-balanced-multilayer-english/) - [Coreference-Aware Dialogue Summarization](https://md.research.elamur.ai/free/en/papers/liu-2021-coreference-aware-dialogue-summarization/) - [Conundrums in Entity Coreference Resolution: Making Sense of the State of the Art](https://md.research.elamur.ai/free/en/papers/lu-2020-conundrums-entity-coreference-resolution-making/) - [Constrained Multi-Task Learning for Event Coreference Resolution](https://md.research.elamur.ai/free/en/papers/lu-2021-constrained-multi-task-learning-event/) - [Conundrums in Event Coreference Resolution: Making Sense of the State of the Art](https://md.research.elamur.ai/free/en/papers/lu-2021-conundrums-event-coreference-resolution-making/) - [Coreference Resolution for the Biomedical Domain: A Survey](https://md.research.elamur.ai/free/en/papers/lu-2021-coreference-resolution-biomedical-domain-survey/) - [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://md.research.elamur.ai/free/en/papers/luan-2018-multi-task-identification-entities-relations/) - [Maverick: Efficient and Accurate Coreference Resolution Defying Recent Trends](https://md.research.elamur.ai/free/en/papers/martinelli-2024-maverick-efficient-accurate-coreference-resolution/) - [Exploring Pre-Trained Transformers and Bilingual Transfer Learning for Arabic Coreference Resolution](https://md.research.elamur.ai/free/en/papers/min-2021-exploring-pre-trained-transformers-bilingual/) - [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/min-2021-recent-advances-natural-language-processing/) - [Multilingual Coreference Resolution in Low-resource South Asian Languages](https://md.research.elamur.ai/free/en/papers/mishra-2024-multilingual-coreference-resolution-low-resource/) - [Using Automatically Extracted Minimum Spans to Disentangle Coreference Evaluation from Boundary Detection](https://md.research.elamur.ai/free/en/papers/moosavi-2019-automatically-extracted-minimum-spans-disentangle/) - [Automated evaluation of written discourse coherence using GPT-4](https://md.research.elamur.ai/free/en/papers/naismith-2023-automated-evaluation-written-discourse-coherence/) - [F-coref: Fast, Accurate and Easy to Use Coreference Resolution](https://md.research.elamur.ai/free/en/papers/otmazgin-2022-f-coref-fast-accurate-easy/) - [LingMess: Linguistically Informed Multi Expert Scorers for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/otmazgin-2023-lingmess-linguistically-informed-multi-expert/) - [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://md.research.elamur.ai/free/en/papers/pan-2023-unifying-large-language-models-knowledge/) - [Scoring Coreference Chains with Split-Antecedent Anaphors](https://md.research.elamur.ai/free/en/papers/paun-2023-scoring-coreference-chains-split-antecedent/) - [A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation](https://md.research.elamur.ai/free/en/papers/poesio-2019-crowdsourced-corpus-multiple-judgments-disagreement/) - [A Benchmark of Rule-Based and Neural Coreference Resolution in Dutch Novels and News](https://md.research.elamur.ai/free/en/papers/poot-2020-benchmark-rule-based-neural-coreference/) - [A Controlled Reevaluation of Coreference Resolution Models](https://md.research.elamur.ai/free/en/papers/porada-2024-controlled-reevaluation-coreference-resolution-models/) - [GECOR: An End-to-End Generative Ellipsis and Co-reference Resolution Model for Task-Oriented Dialogue](https://md.research.elamur.ai/free/en/papers/quan-2019-gecor-end-end-generative-ellipsis/) - [ÚFAL CorPipe at CRAC 2022: Effectivity of Multilingual Models for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/straka-2022-fal-corpipe-crac-2022-effectivity/) - [ÚFAL CorPipe at CRAC 2023: Larger Context Improves Multilingual Coreference Resolution](https://md.research.elamur.ai/free/en/papers/straka-2023-fal-corpipe-crac-2023-larger/) - [Mitigating Gender Bias in Natural Language Processing: Literature Review](https://md.research.elamur.ai/free/en/papers/sun-2019-mitigating-gender-bias-natural-language/) - [What do you learn from context? Probing for sentence structure in contextualized word representations](https://md.research.elamur.ai/free/en/papers/tenney-2019-learn-context-probing-sentence-structure/) - [Scaling Within Document Coreference to Long Texts](https://md.research.elamur.ai/free/en/papers/thirukovalluru-2021-scaling-within-document-coreference-long/) - [Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks](https://md.research.elamur.ai/free/en/papers/toshniwal-2020-learning-ignore-long-document-coreference/) - [On Generalization in Coreference Resolution](https://md.research.elamur.ai/free/en/papers/toshniwal-2021-generalization-coreference-resolution/) - [Deep Cross-Lingual Coreference Resolution for Less-Resourced Languages: The Case of Basque](https://md.research.elamur.ai/free/en/papers/urbizu-2019-deep-cross-lingual-coreference-resolution/) - [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://md.research.elamur.ai/free/en/papers/wadden-2019-entity-relation-event-extraction-contextualized/) - [CorefQA: Coreference Resolution as Query-based Span Prediction](https://md.research.elamur.ai/free/en/papers/wu-2020-corefqa-coreference-resolution-as-query/) - [Understanding Mention Detector-Linker Interaction in Neural Coreference Resolution](https://md.research.elamur.ai/free/en/papers/wu-2021-understanding-mention-detector-linker-interaction/) - [Incremental Neural Coreference Resolution in Constant Memory](https://md.research.elamur.ai/free/en/papers/xia-2020-incremental-neural-coreference-resolution-constant/) - [Revealing the Myth of Higher-Order Inference in Coreference Resolution](https://md.research.elamur.ai/free/en/papers/xu-2020-revealing-myth-higher-order-inference/) - [What You See is What You Get: Visual Pronoun Coreference Resolution in Dialogues](https://md.research.elamur.ai/free/en/papers/yu-2019-what-you-see-is-what/) - [A Cluster Ranking Model for Full Anaphora Resolution](https://md.research.elamur.ai/free/en/papers/yu-2020-cluster-ranking-model-full-anaphora/) - [Free the Plural: Unrestricted Split-Antecedent Anaphora Resolution](https://md.research.elamur.ai/free/en/papers/yu-2020-free-plural-unrestricted-split-antecedent/) - [Multitask Learning-Based Neural Bridging Reference Resolution](https://md.research.elamur.ai/free/en/papers/yu-2020-multitask-learning-based-neural-bridging/) - [Stay Together: A System for Single and Split-antecedent Anaphora Resolution](https://md.research.elamur.ai/free/en/papers/yu-2021-stay-together-system-single-split/) - [The CODI-CRAC 2022 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue](https://md.research.elamur.ai/free/en/papers/yu-2022-codi-crac-2022-shared-task/) - [The Universal Anaphora Scorer](https://md.research.elamur.ai/free/en/papers/yu-2022-universal-anaphora-scorer/) - [Adapting Coreference Resolution Models through Active Learning](https://md.research.elamur.ai/free/en/papers/yuan-2022-adapting-coreference-resolution-models-through/) - [Seq2seq is All You Need for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/zhang-2023-seq2seq-is-all-you-need/) - [PCR4ALL: A Comprehensive Evaluation Benchmark for Pronoun Coreference Resolution in English](https://md.research.elamur.ai/free/en/papers/zhao-2022-pcr4all-comprehensive-evaluation-benchmark-pronoun/) - [Multilingual Coreference Resolution in Multiparty Dialogue](https://md.research.elamur.ai/free/en/papers/zheng-2023-multilingual-coreference-resolution-multiparty-dialogue/) - [Anatomy of OntoGUM—Adapting GUM to the OntoNotes Scheme to Evaluate Robustness of SOTA Coreference Algorithms](https://md.research.elamur.ai/free/en/papers/zhu-2021-anatomy-ontogum-adapting-gum-ontonotes/) - [OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genres](https://md.research.elamur.ai/free/en/papers/zhu-2021-ontogum-evaluating-contextualized-sota-coreference/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{lee2018higher, title = {{Higher-Order Coreference Resolution with Coarse-to-Fine Inference}}, author = {Lee, Kenton and He, Luheng and Zettlemoyer, Luke}, booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)}, year = {2018}, doi = {10.18653/v1/n18-2108}, eprint = {1804.05392}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/n18-2108} } ``` - [← previous Hate Speech Dataset from a White Supremacy Forum](https://md.research.elamur.ai/free/en/papers/gibert-2018-hate-speech-dataset-white-supremacy/) - [next → HotFlip: White-Box Adversarial Examples for Text Classification](https://md.research.elamur.ai/free/en/papers/ebrahimi-2018-hotflip-white-box-adversarial-examples/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) ## References 2 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Eric Bengtson and Dan Roth. 2008. Understanding the value of features for coreference resolution. In EMNLP. [find](https://scholar.google.com/scholar?q=Eric+Bengtson+and+Dan+Roth%2E+2008%2E+Understanding+the+value+of+features+for+coreference+resolution%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/bengtson-2008-understanding-value-features-coreference-resolution/) 2. Anders Björkelund and Jonas Kuhn. 2014. Learning structured perceptrons for coreference resolution with latent antecedents and non-local features. In ACL. [find](https://scholar.google.com/scholar?q=Anders+Bjo%CC%88rkelund+and+Jonas+Kuhn%2E+2014%2E+Learning+structured+perceptrons+for+coreference+resolution+with+latent+antecedents+and+non%2Dlocal+features%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/bjorkelund-2014-learning-structured-perceptrons-coreference-resolution/)