--- title: "They Exist! Introducing Plural Mentions to — Zhou 2018" description: "Most coreference resolution and entity linking systems assume that each mention points to a single entity." authors: ["Ethan Zhou", "Jinho D. Choi"] year: 2018 venue: "Proceedings of the 27th International Conference on Computational Linguistics" source: "https://aclanthology.org/C18-1003/" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/zhou-2018-they-exist-introducing-plural-mentions/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/zhou-2018-they-exist-introducing-plural-mentions/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2018](https://md.research.elamur.ai/free/papers/2018/page/3/) / They Exist! Introducing Plural Mentions to Coreference Resolution and Entity Linking # They Exist! Introducing Plural Mentions to Coreference Resolution and Entity Linking **Authors:** [Ethan Zhou](https://md.research.elamur.ai/free/authors/ethan-zhou/), [Jinho D. Choi](https://md.research.elamur.ai/free/authors/jinho-d-choi/) · **Year:** 2018 · **Venue:** Proceedings of the 27th International Conference on Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/C18-1003/) · [PDF](https://aclanthology.org/C18-1003.pdf) · [OpenAlex](https://openalex.org/W2820779948) **Citations:** 25 citations (openalex-snapshot · 2026-10-14) · [30](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Most coreference resolution and entity linking systems assume that each mention points to a single entity. Plural expressions like "we" or "they" often refer to several people, and these systems handle them poorly. The CoNLL'12 scheme makes this worse: a plural can only attach to one noun phrase, so its link to the individual referents is lost. The authors address the gap using the character identification corpus built from *Friends* transcripts, the dataset used in SemEval 2018. They extend it to cover four complete seasons, clean up mismatched speaker and entity labels, and add crowdsourced annotation in which every plural mention is mapped to a set of entities. For coreference, they propose a transition-based algorithm that decides selectively when to create clusters for singular and plural mentions. They also adapt the evaluation metrics so that plurals are scored fairly. For entity linking, they build a neural multi-task model that resolves both mention types together. Both models beat singular-only baselines, which shows that plural reference can be learned. The corpus and code are released publicly. ## Key points - **Corpus:** The expanded corpus has 47,367 mentions, about 2.5× the earlier 18,608, covering Seasons 1–4 of *Friends* from the Character Mining project. - About 9% of mentions (4,349) are plural. - The entity labels are known characters, GENERIC, GENERAL and OTHER. The old COLLECTIVE tag was dropped. - **Annotation quality:** Plural mentions were double-annotated, with a Cohen's kappa of 56.88%, roughly 20% below the agreement for singulars. - When annotators disagreed, the union of their entity sets was used as a pseudo-adjudication. - **Coreference results:** With metrics adjusted for plurals, the new model outperformed the CZC baseline on F1 across all three measures: | Metric | New model | CZC baseline | |---|---|---| | B³ | 74.4 | 70.6 | | CEAFφ4 | 59.0 | 55.4 | | BLANC | 78.0 | 75.9 | - **Entity linking results:** The multi-task model raised micro-average F1 on plural mentions from 29.8 to 41.7, with essentially no loss on singulars (72.8 for both). - Overall micro F1 rose from 66.2 to 67.4. - Overall macro F1 rose from 53.2 to 55.5. - **Resources:** The data and code are available at github.com/emorynlp/character-identification. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) Topics: [Named entity recognition](https://md.research.elamur.ai/free/topics/named-entity-recognition/) · [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/) ## Cited by Cited by 30 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 20 not yet. - [FantasyCoref: Coreference Resolution on Fantasy Literature Through Omniscient Writer's Point of View](https://md.research.elamur.ai/free/papers/han-2021-fantasycoref-coreference-resolution-fantasy-literature/) - [Scoring Coreference Chains with Split-Antecedent Anaphors](https://md.research.elamur.ai/free/papers/paun-2023-scoring-coreference-chains-split-antecedent/) - [Challenges to Evaluating the Generalization of Coreference Resolution Models: A Measurement Modeling Perspective](https://md.research.elamur.ai/free/papers/porada-2024-challenges-evaluating-generalization-coreference-resolution/) - [On Generalization in Coreference Resolution](https://md.research.elamur.ai/free/papers/toshniwal-2021-generalization-coreference-resolution/) - [Free the Plural: Unrestricted Split-Antecedent Anaphora Resolution](https://md.research.elamur.ai/free/papers/yu-2020-free-plural-unrestricted-split-antecedent/) - [Stay Together: A System for Single and Split-antecedent Anaphora Resolution](https://md.research.elamur.ai/free/papers/yu-2021-stay-together-system-single-split/) - [The CODI-CRAC 2022 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue](https://md.research.elamur.ai/free/papers/yu-2022-codi-crac-2022-shared-task/) - [The Universal Anaphora Scorer](https://md.research.elamur.ai/free/papers/yu-2022-universal-anaphora-scorer/) - [The Universal Anaphora Scorer 2.0](https://md.research.elamur.ai/free/papers/yu-2023-universal-anaphora-scorer-2-0/) - [Multilingual Coreference Resolution in Multiparty Dialogue](https://md.research.elamur.ai/free/papers/zheng-2023-multilingual-coreference-resolution-multiparty-dialogue/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{zhou2018they, title = {{They Exist! Introducing Plural Mentions to Coreference Resolution and Entity Linking}}, author = {Zhou, Ethan and Choi, Jinho D.}, booktitle = {Proceedings of the 27th International Conference on Computational Linguistics}, year = {2018}, url = {https://aclanthology.org/C18-1003/} } ``` - [← previous The semantic transparency of English compound nouns](https://md.research.elamur.ai/free/papers/schafer-2018-semantic-transparency-english-compound/) - [next → THU_NGN at SemEval-2018 Task 1: Fine-grained Tweet Sentiment Intensity Analysis with Attention CNN-LSTM](https://md.research.elamur.ai/free/papers/wu-2018-thu-ngn-semeval-2018-task/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/papers/2018/page/3/) ## References 15 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Amit Bagga and Breck Baldwin. 1998. Algorithms for Scoring Coreference Chains. In The first international conference on language resources and evaluation workshop on linguistics coreference, pages 563–566. [find](https://scholar.google.com/scholar?q=Amit+Bagga+and+Breck+Baldwin%2E+1998%2E+Algorithms+for+Scoring+Coreference+Chains%2E+In+The+first+international+conference+on+language+resources+and+evaluation+workshop+on+linguistics+coreference%2C+pages+563) [our card](https://md.research.elamur.ai/free/papers/bagga-1998-algorithms-scoring-coreference-chains/) 2. Henry Yu-Hsin Chen and Jinho D. Choi. 2016. Character Identification on Multiparty Conversation: Identifying Mentions of Characters in TV Shows. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL’16, pages 90–100. [find](https://scholar.google.com/scholar?q=Henry+Yu%2DHsin+Chen+and+Jinho+D%2E+Choi%2E+2016%2E+Character+Identification+on+Multiparty+Conversation%3A+Identifying+Mentions+of+Characters+in+TV+Shows%2E+In+Proceedings+of+the+17th+Annual+Meeting+of+the+Specia) 3. Henry Yu-Hsin Chen, Ethan Zhou, and Jinho D. Choi. 2017. Robust Coreference Resolution and Entity Linking on Dialogues: Character Identification on TV Show Transcripts. In Proceedings of the 21st Conference on Computational Natural Language Learning, CoNLL’17, pages 216–225, Vancouver, Canada. [find](https://scholar.google.com/scholar?q=Henry+Yu%2DHsin+Chen%2C+Ethan+Zhou%2C+and+Jinho+D%2E+Choi%2E+2017%2E+Robust+Coreference+Resolution+and+Entity+Linking+on+Dialogues%3A+Character+Identification+on+TV+Show+Transcripts%2E+In+Proceedings+of+the+21st+Conf) [our card](https://md.research.elamur.ai/free/papers/chen-2017-robust-coreference-resolution-entity-linking/) 4. Kevin Clark and Christopher D. Manning. 2016. Deep reinforcement learning for mention-ranking coreference models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP’16, pages 2256–2262. [find](https://scholar.google.com/scholar?q=Kevin+Clark+and+Christopher+D%2E+Manning%2E+2016%2E+Deep+reinforcement+learning+for+mention%2Dranking+coreference+models%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Process) [our card](https://md.research.elamur.ai/free/papers/clark-2016-deep-reinforcement-learning-mention-ranking/) 5. The Character Mining project provides a superset of the corpus presented in this paper for several other tasks: [https://github.com/emorynlp/character-mining](https://github.com/emorynlp/character-mining). [↗](https://github.com/emorynlp/character-mining) 6. Greg Durrett, David Hall, and Dan Klein. 2013. Decentralized entity-level modeling for coreference resolution. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 114–124, Sofia, Bulgaria, August. Association for Computational Lin… [find](https://scholar.google.com/scholar?q=Greg+Durrett%2C+David+Hall%2C+and+Dan+Klein%2E+2013%2E+Decentralized+entity%2Dlevel+modeling+for+coreference+resolution%2E+In+Proceedings+of+the+51st+Annual+Meeting+of+the+Association+for+Computational+Linguistic) [our card](https://md.research.elamur.ai/free/papers/durrett-2013-decentralized-entity-level-modeling-coreference/) 7. Matthew Francis-Landau, Greg Durrett, and Dan Klein. 2016. Capturing semantic similarity for entity linking with convolutional neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 125… [find](https://scholar.google.com/scholar?q=Matthew+Francis%2DLandau%2C+Greg+Durrett%2C+and+Dan+Klein%2E+2016%2E+Capturing+semantic+similarity+for+entity+linking+with+convolutional+neural+networks%2E+In+Proceedings+of+the+2016+Conference+of+the+North+Ameri) 8. Nitish Gupta, Sameer Singh, and Dan Roth. 2017. Entity linking via joint encoding of types, descriptions, and context. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2681–2690, Copenhagen, Denmark, September. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Nitish+Gupta%2C+Sameer+Singh%2C+and+Dan+Roth%2E+2017%2E+Entity+linking+via+joint+encoding+of+types%2C+descriptions%2C+and+context%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Pr) 9. Prateek Jain, Manav Ratan Mital, Sumit Kumar, Amitabha Mukerjee, and Achla M. Raina. 2004. Anaphora resolution in multi-person dialogues. In Michael Strube and Candy Sidner, editors, Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue, pages 47–50, Cambridge, Massachusetts, USA, April… [find](https://scholar.google.com/scholar?q=Prateek+Jain%2C+Manav+Ratan+Mital%2C+Sumit+Kumar%2C+Amitabha+Mukerjee%2C+and+Achla+M%2E+Raina%2E+2004%2E+Anaphora+resolution+in+multi%2Dperson+dialogues%2E+In+Michael+Strube+and+Candy+Sidner%2C+editors%2C+Proceedings+of+th) 10. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark, September. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kenton+Lee%2C+Luheng+He%2C+Mike+Lewis%2C+and+Luke+Zettlemoyer%2E+2017%2E+End%2Dto%2Dend+neural+coreference+resolution%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+page) [our card](https://md.research.elamur.ai/free/papers/lee-2017-end-end-neural-coreference-resolution/) 11. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25–32, Vancouver, British Columbia, Canada, October. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xiaoqiang+Luo%2E+2005%2E+On+coreference+resolution+performance+metrics%2E+In+Proceedings+of+Human+Language+Technology+Conference+and+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+25%E2%80%93) [our card](https://md.research.elamur.ai/free/papers/luo-2005-coreference-resolution-performance-metrics/) 12. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes. In Proceedings of the Sixteenth Conference on Computational Natural Language Learning: Shared Task, CoNLL’12, pages 1–40. [find](https://scholar.google.com/scholar?q=Sameer+Pradhan%2C+Alessandro+Moschitti%2C+Nianwen+Xue%2C+Olga+Uryupina%2C+and+Yuchen+Zhang%2E+2012%2E+CoNLL%2D2012+Shared+Task%3A+Modeling+Multilingual+Unrestricted+Coreference+in+OntoNotes%2E+In+Proceedings+of+the+Six) [our card](https://md.research.elamur.ai/free/papers/pradhan-2012-conll-2012-shared-task-modeling/) 13. Marta Recasens and Eduard H. Hovy. 2011. BLANC: Implementing the Rand index for coreference evaluation. Natural Language Engineering, 17(4):485–510. [find](https://scholar.google.com/scholar?q=Marta+Recasens+and+Eduard+H%2E+Hovy%2E+2011%2E+BLANC%3A+Implementing+the+Rand+index+for+coreference+evaluation%2E+Natural+Language+Engineering%2C+17%284%29%3A485%E2%80%93510%2E) 14. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A Model-theoretic Coreference Scoring Scheme. In Proceedings of the 6th Conference on Message Understanding, MUC6 ’95, pages 45–52, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Marc+Vilain%2C+John+Burger%2C+John+Aberdeen%2C+Dennis+Connolly%2C+and+Lynette+Hirschman%2E+1995%2E+A+Model%2Dtheoretic+Coreference+Scoring+Scheme%2E+In+Proceedings+of+the+6th+Conference+on+Message+Understanding%2C+MUC6) [our card](https://md.research.elamur.ai/free/papers/vilain-1995-model-theoretic-coreference-scoring-scheme/) 15. Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning global features for coreference resolution. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 994–1004, San Diego, California,… [find](https://scholar.google.com/scholar?q=Sam+Wiseman%2C+Alexander+M%2E+Rush%2C+and+Stuart+M%2E+Shieber%2E+2016%2E+Learning+global+features+for+coreference+resolution%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+Association) [our card](https://md.research.elamur.ai/free/papers/wiseman-2016-learning-global-features-coreference-resolution/)