--- title: "Learning to Ignore — Toshniwal 2020" description: "Leading coreference systems need memory and runtime that grow quadratically with document length, which makes book-scale texts impractical." authors: ["Shubham Toshniwal", "Sam Wiseman", "Allyson Ettinger", "Karen Livescu", "Kevin Gimpel"] year: 2020 venue: "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)" source: "https://aclanthology.org/2020.emnlp-main.685/" doi: "10.18653/v1/2020.emnlp-main.685" arxiv: "2010.02807" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/toshniwal-2020-learning-ignore-long-document-coreference/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/toshniwal-2020-learning-ignore-long-document-coreference/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/) / Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks # Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks **Authors:** [Shubham Toshniwal](https://md.research.elamur.ai/free/en/authors/shubham-toshniwal/), [Sam Wiseman](https://md.research.elamur.ai/free/en/authors/sam-wiseman/), [Allyson Ettinger](https://md.research.elamur.ai/free/en/authors/allyson-ettinger/), [Karen Livescu](https://md.research.elamur.ai/free/en/authors/karen-livescu/), [Kevin Gimpel](https://md.research.elamur.ai/free/en/authors/kevin-gimpel/) · **Year:** 2020 · **Venue:** Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2020.emnlp-main.685/) · [PDF](https://aclanthology.org/2020.emnlp-main.685.pdf) · [DOI 10.18653/v1/2020.emnlp-main.685](https://doi.org/10.18653/v1/2020.emnlp-main.685) · [arXiv 2010.02807](https://arxiv.org/abs/2010.02807) · [OpenAlex](https://openalex.org/W3092075664) **Citations:** 37 citations (openalex-snapshot · 2026-10-14) · [43](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Leading coreference systems need memory and runtime that grow quadratically with document length, which makes book-scale texts impractical. Entity-mention models that store only entity representations are cheaper, but they still keep every entity, and that set grows with the document. The authors observe that few entities are "active" (between their first and last mention) at any given token. Building on this, they design a quasi-online model that holds only a fixed number of entity slots, which keeps runtime linear in document length. It uses frozen SpanBERT-large encodings, a pretrained mention proposer, and sequential clustering of candidate spans. When memory is full, a learned module (LB-MEM) decides whether to evict a tracked entity, skip the mention, or mark it invalid. This is compared with an unbounded variant (U-MEM) and an LRU-based rule (RB-MEM) on LitBank and OntoNotes. With 20 slots, LB-MEM nearly matches U-MEM and clearly beats LRU eviction when memory is tight. It also lowers training memory. This shows that coreference models can scale to long documents without tracking every entity. ## Key points - Entity statistics: the maximum number of entities active at one token is far below the total per document (LitBank 18 vs 199; OntoNotes 24 vs 94). - LitBank test CoNLL F1: - U-MEM reaches 75.9, a new state of the art (previous best, Bamman et al. 2020: 68.1). - LB-MEM with 5/10/20 cells scores 69.5/74.9/75.7. - RB-MEM with 5/10/20 cells scores 66.7/71.8/72.6. - OntoNotes test CoNLL F1: - U-MEM scores 77.4 and LB-MEM with 20 cells scores 77.3, both close to Xia et al. 2020 (78.2). - Both remain below Joshi et al. 2020 (79.6) and Wu et al. 2020 (83.1). - On a LitBank split, peak training memory drops from 11.6 GB (U-MEM) to about 8.0–9.1 GB, and inference is slightly faster. Peak inference memory is about the same (3.1 vs 3.2 GB) because the SpanBERT encoder dominates it. - LB-MEM ignores far fewer mentions than RB-MEM (e.g., 18.3 vs 83.2 on LitBank with 5 cells). This suggests it learns which entities will be mentioned again. Code is available at github.com/shtoshni92/long-doc-coref. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/) ## Cited by Cited by 43 papers of our library cite it (of the 140673 whose bibliographies we read); 15 of them have a page here, 28 not yet. - [Findings of the Second Shared Task on Multilingual Coreference Resolution](https://md.research.elamur.ai/free/en/papers/abokrtsky-2023-findings-second-shared-task-multilingual/) - [Large language models are few-shot clinical information extractors](https://md.research.elamur.ai/free/en/papers/agrawal-2022-large-language-models-are-few/) - [Character Coreference Resolution in Movie Screenplays](https://md.research.elamur.ai/free/en/papers/baruah-2023-character-coreference-resolution-movie-screenplays/) - [Neural End-to-end Coreference Resolution for German in Different Domains](https://md.research.elamur.ai/free/en/papers/coref--schroder-neural-end-end-coreference-resolution/) - [Dual Cache for Long Document Neural Coreference Resolution](https://md.research.elamur.ai/free/en/papers/guo-2023-dual-cache-long-document-neural/) - [Coreference Resolution without Span Representations](https://md.research.elamur.ai/free/en/papers/kirstain-2021-coreference-resolution-without-span-representations/) - [Major Entity Identification: A Generalizable Alternative to Coreference Resolution](https://md.research.elamur.ai/free/en/papers/manikantan-2024-major-entity-identification-generalizable-alternative/) - [Multilingual Coreference Resolution in Low-resource South Asian Languages](https://md.research.elamur.ai/free/en/papers/mishra-2024-multilingual-coreference-resolution-low-resource/) - [Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities](https://md.research.elamur.ai/free/en/papers/novak-2026-findings-fifth-shared-task-multilingual/) - [F-coref: Fast, Accurate and Easy to Use Coreference Resolution](https://md.research.elamur.ai/free/en/papers/otmazgin-2022-f-coref-fast-accurate-easy/) - [LingMess: Linguistically Informed Multi Expert Scorers for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/otmazgin-2023-lingmess-linguistically-informed-multi-expert/) - [Scaling Within Document Coreference to Long Texts](https://md.research.elamur.ai/free/en/papers/thirukovalluru-2021-scaling-within-document-coreference-long/) - [On Generalization in Coreference Resolution](https://md.research.elamur.ai/free/en/papers/toshniwal-2021-generalization-coreference-resolution/) - [Seq2seq is All You Need for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/zhang-2023-seq2seq-is-all-you-need/) - [Multilingual Coreference Resolution in Multiparty Dialogue](https://md.research.elamur.ai/free/en/papers/zheng-2023-multilingual-coreference-resolution-multiparty-dialogue/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{toshniwal2020learning, title = {{Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks}}, author = {Toshniwal, Shubham and Wiseman, Sam and Ettinger, Allyson and Livescu, Karen and Gimpel, Kevin}, booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)}, year = {2020}, doi = {10.18653/v1/2020.emnlp-main.685}, eprint = {2010.02807}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/2020.emnlp-main.685} } ``` - [← previous Learning to Faithfully Rationalize by Construction](https://md.research.elamur.ai/free/en/papers/jain-2020-learning-faithfully-rationalize-construction/) - [next → LEDGAR: A Large-Scale Multi-label Corpus for Text Classification of Legal Provisions in Contracts](https://md.research.elamur.ai/free/en/papers/tuggener-2020-ledgar-large-scale-multi-label/) - [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/) ## References 23 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Alan Baddeley. 1986. Working Memory. Oxford University Press. [find](https://scholar.google.com/scholar?q=Alan+Baddeley%2E+1986%2E+Working+Memory%2E+Oxford+University+Press%2E) 2. David Bamman, Olivia Lewke, and Anya Mansoor. 2020. An Annotated Dataset of Coreference in English Literature. In LREC. [↗](https://doi.org/10.48550/arxiv.1912.01140) [our card](https://md.research.elamur.ai/free/en/papers/bamman-2020-annotated-dataset-coreference-english-literature/) 3. Kevin Clark and Christopher D. Manning. 2015. Entity-Centric Coreference Resolution with Model Stacking. In ACL. [find](https://scholar.google.com/scholar?q=Kevin+Clark+and+Christopher+D%2E+Manning%2E+2015%2E+Entity%2DCentric+Coreference+Resolution+with+Model+Stacking%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/clark-2015-entity-centric-coreference-resolution-model/) 4. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. SpanBERT: Improving Pre-training by Representing and Predicting Spans. TACL, 8. [↗](https://doi.org/10.1162/tacl_a_00300) [our card](https://md.research.elamur.ai/free/en/papers/joshi-2019-spanbert-improving-pre-training-representing/) 5. Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel Weld. 2019. BERT for Coreference Resolution: Baselines and Analysis. In EMNLP. [find](https://scholar.google.com/scholar?q=Mandar+Joshi%2C+Omer+Levy%2C+Luke+Zettlemoyer%2C+and+Daniel+Weld%2E+2019%2E+BERT+for+Coreference+Resolution%3A+Baselines+and+Analysis%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/joshi-2019-bert-coreference-resolution-baselines-analysis/) 6. Frank Keller. 2010. Cognitively Plausible Models of Human Language Processing. In ACL. [find](https://scholar.google.com/scholar?q=Frank+Keller%2E+2010%2E+Cognitively+Plausible+Models+of+Human+Language+Processing%2E+In+ACL%2E) 7. Jack W. Rae, Jonathan J. Hunt, Ivo Danihelka, Timothy Harley, Andrew W. Senior, Gregory Wayne, Alex Graves, and Tim Lillicrap. 2016. Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes. In NeurIPS. [↗](https://doi.org/10.48550/arxiv.1610.09027) 8. Altaf Rahman and Vincent Ng. 2011. Narrowing the modeling gap: a cluster-ranking approach to coreference resolution. Journal of Artificial Intelligence Research, 40:469–521. [↗](https://doi.org/10.1613/jair.3120) 9. Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy P. Lillicrap. 2016. One-shot Learning with Memory-Augmented Neural Networks. In ICML. [↗](https://doi.org/10.48550/arxiv.1605.06065) 10. Veselin Stoyanov and Jason Eisner. 2012. Easy-first Coreference Resolution. In COLING. [find](https://scholar.google.com/scholar?q=Veselin+Stoyanov+and+Jason+Eisner%2E+2012%2E+Easy%2Dfirst+Coreference+Resolution%2E+In+COLING%2E) [our card](https://md.research.elamur.ai/free/en/papers/stoyanov-2012-easy-first-coreference-resolution/) 11. MK Tanenhaus, MJ Spivey-Knowlton, KM Eberhard, and JC Sedivy. 1995. Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217). [↗](https://doi.org/10.1126/science.7777863) 12. Shubham Toshniwal, Allyson Ettinger, Kevin Gimpel, and Karen Livescu. 2020. PeTra: A Sparsely Supervised Memory Model for People Tracking. In ACL. [find](https://scholar.google.com/scholar?q=Shubham+Toshniwal%2C+Allyson+Ettinger%2C+Kevin+Gimpel%2C+and+Karen+Livescu%2E+2020%2E+PeTra%3A+A+Sparsely+Supervised+Memory+Model+for+People+Tracking%2E+In+ACL%2E) 13. Kellie Webster and James R. Curran. 2014. Limited memory incremental coreference resolution. In COLING. [find](https://scholar.google.com/scholar?q=Kellie+Webster+and+James+R%2E+Curran%2E+2014%2E+Limited+memory+incremental+coreference+resolution%2E+In+COLING%2E) [our card](https://md.research.elamur.ai/free/en/papers/webster-2014-limited-memory-incremental-coreference-resolution/) 14. Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning Global Features for Coreference Resolution. In NAACL. [find](https://scholar.google.com/scholar?q=Sam+Wiseman%2C+Alexander+M%2E+Rush%2C+and+Stuart+M%2E+Shieber%2E+2016%2E+Learning+Global+Features+for+Coreference+Resolution%2E+In+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/wiseman-2016-learning-global-features-coreference-resolution/) 15. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In ICLR. [↗](https://doi.org/10.48550/arxiv.1412.6980) 16. Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. Coreference Resolution as Query-based Span Prediction. In ACL. [↗](https://doi.org/10.48550/arxiv.1911.01746) 17. Jonathan K. Kummerfeld and Dan Klein. 2013. Error-Driven Analysis of Challenges in Coreference Resolution. In EMNLP. [find](https://scholar.google.com/scholar?q=Jonathan+K%2E+Kummerfeld+and+Dan+Klein%2E+2013%2E+Error%2DDriven+Analysis+of+Challenges+in+Coreference+Resolution%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/kummerfeld-2013-error-driven-analysis-challenges-coreference/) 18. Patrick Xia, João Sedoc, and Benjamin Van Durme. 2020. Revisiting Memory-Efficient Incremental Coreference Resolution. In EMNLP. [find](https://scholar.google.com/scholar?q=Patrick+Xia%2C+Jo%C3%A3o+Sedoc%2C+and+Benjamin+Van+Durme%2E+2020%2E+Revisiting+Memory%2DEfficient+Incremental+Coreference+Resolution%2E+In+EMNLP%2E) 19. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end Neural Coreference Resolution. In EMNLP. [find](https://scholar.google.com/scholar?q=Kenton+Lee%2C+Luheng+He%2C+Mike+Lewis%2C+and+Luke+Zettlemoyer%2E+2017%2E+End%2Dto%2Dend+Neural+Coreference+Resolution%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/lee-2017-end-end-neural-coreference-resolution/) 20. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-Order Coreference Resolution with Coarse-to-Fine Inference. In NAACL-HLT. [find](https://scholar.google.com/scholar?q=Kenton+Lee%2C+Luheng+He%2C+and+Luke+Zettlemoyer%2E+2018%2E+Higher%2DOrder+Coreference+Resolution+with+Coarse%2Dto%2DFine+Inference%2E+In+NAACL%2DHLT%2E) [our card](https://md.research.elamur.ai/free/en/papers/lee-2018-higher-order-coreference-resolution-coarse/) 21. Fei Liu, Luke Zettlemoyer, and Jacob Eisenstein. 2019. The Referential Reader: A Recurrent Entity Network for Anaphora Resolution. In ACL. [find](https://scholar.google.com/scholar?q=Fei+Liu%2C+Luke+Zettlemoyer%2C+and+Jacob+Eisenstein%2E+2019%2E+The+Referential+Reader%3A+A+Recurrent+Entity+Network+for+Anaphora+Resolution%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/liu-2019-referential-reader-recurrent-entity-network/) 22. Xiaoqiang Luo, Abe Ittycheriah, Hongyan Jing, Nanda Kambhatla, and Salim Roukos. 2004. A Mention-Synchronous Coreference Resolution Algorithm Based On the Bell Tree. In ACL. [find](https://scholar.google.com/scholar?q=Xiaoqiang+Luo%2C+Abe+Ittycheriah%2C+Hongyan+Jing%2C+Nanda+Kambhatla%2C+and+Salim+Roukos%2E+2004%2E+A+Mention%2DSynchronous+Coreference+Resolution+Algorithm+Based+On+the+Bell+Tree%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/luo-2004-mention-synchronous-coreference-resolution-algorithm/) 23. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL - Shared Task, CoNLL ’12. *Figure 1: Histograms of Maximum Active Entities for documents in LitBank and OntoNotes.* *Figure 2: Histograms of Entity Spread as fraction of document length for LitBank and OntoNotes.* [find](https://scholar.google.com/scholar?q=Sameer+Pradhan%2C+Alessandro+Moschitti%2C+Nianwen+Xue%2C+Olga+Uryupina%2C+and+Yuchen+Zhang%2E+2012%2E+CoNLL%2D2012+Shared+Task%3A+Modeling+Multilingual+Unrestricted+Coreference+in+OntoNotes%2E+In+Joint+Conference+on+EM) [our card](https://md.research.elamur.ai/free/en/papers/pradhan-2012-conll-2012-shared-task-modeling/)