---
title: "Seq2seq is All You Need for Coreference — Zhang 2023"
description: "Coreference resolution has usually depended on task-specific architectures, such as span-ranking models, pointer networks like ASP, or transition systems."
authors: ["Wenzheng Zhang", "Sam Wiseman", "Karl Stratos"]
year: 2023
venue: "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2023.emnlp-main.704/"
doi: "10.18653/v1/2023.emnlp-main.704"
arxiv: "2310.13774"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2023-seq2seq-is-all-you-need/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2023-seq2seq-is-all-you-need/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/5/) / Seq2seq is All You Need for Coreference Resolution

# Seq2seq is All You Need for Coreference Resolution

**Authors:** Wenzheng Zhang, [Sam Wiseman](https://md.research.elamur.ai/free/en/authors/sam-wiseman/), Karl Stratos · **Year:** 2023 · **Venue:** Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.emnlp-main.704/) · [PDF](https://aclanthology.org/2023.emnlp-main.704.pdf) · [DOI 10.18653/v1/2023.emnlp-main.704](https://doi.org/10.18653/v1/2023.emnlp-main.704) · [arXiv 2310.13774](https://arxiv.org/abs/2310.13774) · [OpenAlex](https://openalex.org/W4389519415)

**Citations:** 7 citations (openalex-snapshot · 2026-10-14) · [25](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Coreference resolution has usually depended on task-specific architectures, such as span-ranking models, pointer networks like ASP, or transition systems. Earlier attempts to treat it as plain sequence-to-sequence generation performed noticeably worse. The authors fine-tune unmodified pretrained encoder-decoders (T5, FLAN-T5, T0). The model reads a document and outputs a linearized form of its coreference annotation: each mention is wrapped in special markers and followed by an integer cluster ID.

Two output formats are studied:
- **Full linearization** reproduces the whole document. It uses a compact "copy" action space, and constrained beam search guarantees a valid clustering.
- **Partial linearization** emits only the tagged mentions. It decodes faster but needs a separate step to align mentions back to the source.

With T0, this simple setup matches or beats specialized systems on English OntoNotes, PreCo and LitBank. The results suggest that bespoke coreference models are not required for state-of-the-art accuracy. Model scale, the amount of training data and the design of the target sequence turn out to be the decisive factors.

## Key points
- **OntoNotes test average F1:**
  - T0 3B scores 82.9, ahead of ASP built on the same base model (82.3).
  - T0 11B scores 83.2, edging past CorefQA (83.1) and close to the best reported result of 83.3, which uses a 13B model.
- **PreCo and LitBank:**
  - On PreCo the model reaches 88.5, above Toshniwal et al. (87.8).
  - On LitBank alone it trails Toshniwal et al. (78.3 vs 79.2 cross-validation F1).
  - Trained jointly on LitBank, OntoNotes and PreCo, it reaches 81.2 split-0 F1, well above their 78.2.
- **Ablations on OntoNotes dev:**
  - Putting the cluster integer after the mention works better than putting it before (82.6 vs 82.3).
  - Linking mentions via antecedent strings, as TANL does, drops performance to 79.5.
  - For partial linearization, sentence markers matter (81.1 vs 79.9 without them).
- **Error analysis:** mention detection is the main bottleneck (89.2 F1). Clustering of correctly detected mentions reaches 95.8 F1.
- **Code:** released at github.com/WenzhengZhang/Seq2seqCoref.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/)

## Cited by

Cited by

25 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 19 not yet.

- [CorefInst: Leveraging LLMs for Multilingual Coreference Resolution](https://md.research.elamur.ai/free/en/papers/arslan-2026-corefinst-leveraging-llms-multilingual-coreference/)
- [Lions: 1 and Tigers: 2 and Bears: 3, Oh My! Literary Coreference Annotation with LLMs](https://md.research.elamur.ai/free/en/papers/hicke-2024-lions-1-tigers-2-bears/)
- [Dialogue Discourse Parsing as Generation: A Sequence-to-Sequence LLM-based Approach](https://md.research.elamur.ai/free/en/papers/li-2024-dialogue-discourse-parsing-as-generation/)
- [Major Entity Identification: A Generalizable Alternative to Coreference Resolution](https://md.research.elamur.ai/free/en/papers/manikantan-2024-major-entity-identification-generalizable-alternative/)
- [Maverick: Efficient and Accurate Coreference Resolution Defying Recent Trends](https://md.research.elamur.ai/free/en/papers/martinelli-2024-maverick-efficient-accurate-coreference-resolution/)
- [Multilingual coreference resolution as text generation](https://md.research.elamur.ai/free/en/papers/skachkova-2024-multilingual-coreference-resolution-as-text/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2023seq2seq,
  title = {{Seq2seq is All You Need for Coreference Resolution}},
  author = {Zhang, Wenzheng and Wiseman, Sam and Stratos, Karl},
  booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.emnlp-main.704},
  eprint = {2310.13774},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.emnlp-main.704}
}
```

- [← previous Sentiment Analysis in the Era of Large Language Models: A Reality Check](https://md.research.elamur.ai/free/en/papers/zhang-2023-sentiment-analysis-era-large-language/)
- [next → SGLang: Efficient Execution of Structured Language Model Programs](https://md.research.elamur.ai/free/en/papers/zheng-2023-sglang-efficient-execution-structured-language/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/5/)

## References

31 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Bamman, Olivia Lewke, and Anya Mansoor. 2020. An annotated dataset of coreference in English literature. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 44–54, Marseille, France. European Language Resources Association. [↗](https://doi.org/10.48550/arxiv.1912.01140) [our card](https://md.research.elamur.ai/free/en/papers/bamman-2020-annotated-dataset-coreference-english-literature/)
2. Bernd Bohnet, Chris Alberti, and Michael Collins. 2023. Coreference Resolution through a seq2seq Transition-Based System. Transactions of the Association for Computational Linguistics, 11:212–226. [↗](https://doi.org/10.1162/tacl_a_00543) [our card](https://md.research.elamur.ai/free/en/papers/bohnet-2023-coreference-resolution-through-seq2seq-transition/)
3. Hong Chen, Zhenhua Fan, Hao Lu, Alan Yuille, and Shu Rong. 2018. PreCo: A large-scale dataset in preschool vocabulary for coreference resolution. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 172– 181, Brussels, Belgium. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d18-1016) [our card](https://md.research.elamur.ai/free/en/papers/chen-2018-preco-large-scale-dataset-preschool/)
4. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416. [↗](https://arxiv.org/abs/2210.11416)
5. Angel Daza and Anette Frank. 2018. A sequence-to-sequence model for semantic role labeling. In Proceedings of The Third Workshop on Representation Learning for NLP, pages 207–216. [↗](https://doi.org/10.48550/arxiv.1807.03006)
6. Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2021. Autoregressive entity retrieval. In International Conference on Learning Representations. [↗](https://doi.org/10.48550/arxiv.2010.00904)
7. Vladimir Dobrovolskii. 2021. Word-level coreference resolution. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7670–7675. coreference in ontonotes. In Joint conference on EMNLP and CoNLL-shared task, pages 1–40. [↗](https://doi.org/10.48448/3d5h-p136) [our card](https://md.research.elamur.ai/free/en/papers/dobrovolskii-2021-word-level-coreference-resolution/)
8. Osamu Gotoh. 1982. An improved algorithm for matching biological sequences. Journal of molecular biology, 162(3):705–708. [↗](https://doi.org/10.1016/0022-2836(82)90398-9)
9. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551. [↗](https://doi.org/10.48550/arxiv.1910.10683)
10. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S Weld, Luke Zettlemoyer, and Omer Levy. 2020. Spanbert: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77. [↗](https://doi.org/10.1162/tacl_a_00300) [our card](https://md.research.elamur.ai/free/en/papers/joshi-2019-spanbert-improving-pre-training-representing/)
11. Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel S Weld. 2019. Bert for coreference resolution: Baselines and analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5803–5808. [find](https://scholar.google.com/scholar?q=Mandar+Joshi%2C+Omer+Levy%2C+Luke+Zettlemoyer%2C+and+Daniel+S+Weld%2E+2019%2E+Bert+for+coreference+resolution%3A+Baselines+and+analysis%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Langu) [our card](https://md.research.elamur.ai/free/en/papers/joshi-2019-bert-coreference-resolution-baselines-analysis/)
12. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR (Poster). [↗](https://doi.org/10.48550/arxiv.1412.6980)
13. Yuval Kirstain, Ori Ram, and Omer Levy. 2021. Coreference resolution without span representations. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 14–19, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.acl-short.3) [our card](https://md.research.elamur.ai/free/en/papers/kirstain-2021-coreference-resolution-without-span-representations/)
14. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d17-1018) [our card](https://md.research.elamur.ai/free/en/papers/lee-2017-end-end-neural-coreference-resolution/)
15. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–692, New Orleans, Louisiana. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n18-2108) [our card](https://md.research.elamur.ai/free/en/papers/lee-2018-higher-order-coreference-resolution-coarse/)
16. Tianyu Liu, Yuchen Eleanor Jiang, Nicholas Monath, Ryan Cotterell, and Mrinmaya Sachan. 2022. Autoregressive structured prediction with language models. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 993–1005, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. [↗](https://doi.org/10.3929/ethz-b-000588610)
17. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1– 16. IEEE. [find](https://scholar.google.com/scholar?q=Samyam+Rajbhandari%2C+Jeff+Rasley%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+2020%2E+Zero%3A+Memory+optimizations+toward+training+trillion+parameter+models%2E+In+SC20%3A+International+Conference+for+High+Performance+Com)
18. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 3505–3506. [find](https://scholar.google.com/scholar?q=Jeff+Rasley%2C+Samyam+Rajbhandari%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+2020%2E+Deepspeed%3A+System+optimizations+enable+training+deep+learning+models+with+over+100+billion+parameters%2E+In+Proceedings+of+the+26t)
19. Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. 2022. Multi-task prompted training enables zero-shot task generalization. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Albert+Webson%2C+Colin+Raffel%2C+Stephen+Bach%2C+Lintang+Sutawika%2C+Zaid+Alyafeai%2C+Antoine+Chaffin%2C+Arnaud+Stiegler%2C+Arun+Raja%2C+Manan+Dey%2C+M+Saiful+Bari%2C+Canwen+Xu%2C+Urmish+Thakker%2C+Shanya+Sharma) [our card](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
20. Raghuveer Thirukovalluru, Nicholas Monath, Kumar Shridhar, Manzil Zaheer, Mrinmaya Sachan, and Andrew McCallum. 2021. Scaling within document coreference to long texts. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3921–3931. [find](https://scholar.google.com/scholar?q=Raghuveer+Thirukovalluru%2C+Nicholas+Monath%2C+Kumar+Shridhar%2C+Manzil+Zaheer%2C+Mrinmaya+Sachan%2C+and+Andrew+McCallum%2E+2021%2E+Scaling+within+document+coreference+to+long+texts%2E+In+Findings+of+the+Association+) [our card](https://md.research.elamur.ai/free/en/papers/thirukovalluru-2021-scaling-within-document-coreference-long/)
21. Shubham Toshniwal, Sam Wiseman, Allyson Ettinger, Karen Livescu, and Kevin Gimpel. 2020. Learning to ignore: Long document coreference with bounded memory neural networks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8519–8526. [find](https://scholar.google.com/scholar?q=Shubham+Toshniwal%2C+Sam+Wiseman%2C+Allyson+Ettinger%2C+Karen+Livescu%2C+and+Kevin+Gimpel%2E+2020%2E+Learning+to+ignore%3A+Long+document+coreference+with+bounded+memory+neural+networks%2E+In+Proceedings+of+the+2020+C) [our card](https://md.research.elamur.ai/free/en/papers/toshniwal-2020-learning-ignore-long-document-coreference/)
22. Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, RISHITA ANUBHAI, Cicero Nogueira dos Santos, Bing Xiang, and Stefano Soatto. 2021. Structured prediction as translation between augmented natural languages. In International Conference on Learning Representations. [↗](https://doi.org/10.48550/arxiv.2101.05779)
23. Shubham Toshniwal, Patrick Xia, Sam Wiseman, Karen Livescu, and Kevin Gimpel. 2021. On generalization in coreference resolution. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 111–120, Punta Cana, Dominican Republic. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.crac-1.12) [our card](https://md.research.elamur.ai/free/en/papers/toshniwal-2021-generalization-coreference-resolution/)
24. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. Conll-2012 shared task: Modeling multilingual unrestricted Gorka Urbizu, Ander Soraluze, and Olatz Arregi. 2020. Sequence to sequence coreference resolution. In Proceedings of the Third Workshop on Computational 2. Inside Mention: Open mentions exist (i.e., the count of separation tokens | is less than that of mention start tokens \<m>), but not in Inside Cluster Identity state. Models of Reference, Anaphora and Coreference, pages 39–46, Barcelona, Spain (online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sameer+Pradhan%2C+Alessandro+Moschitti%2C+Nianwen+Xue%2C+Olga+Uryupina%2C+and+Yuchen+Zhang%2E+2012%2E+Conll%2D2012+shared+task%3A+Modeling+multilingual+unrestricted+Gorka+Urbizu%2C+Ander+Soraluze%2C+and+Olatz+Arregi%2E+202)
25. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771. [↗](https://arxiv.org/abs/1910.03771) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
26. Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. Corefqa: Coreference resolution as query-based span prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6953–6963. 3. Outside: Not in Inside Mention or Inside Cluster Identity states. Token Action. Depending on the current state, different tokens are permitted: • Outside: The next token from the input source and the mention start token \<m> are allowed. • Inside Mention: The next token from the input source, the mention start token \<m>, and the separation token | are allowed. [find](https://scholar.google.com/scholar?q=Wei+Wu%2C+Fei+Wang%2C+Arianna+Yuan%2C+Fei+Wu%2C+and+Jiwei+Li%2E+2020%2E+Corefqa%3A+Coreference+resolution+as+query%2Dbased+span+prediction%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computation) [our card](https://md.research.elamur.ai/free/en/papers/wu-2020-corefqa-coreference-resolution-as-query/)
27. Zhaofeng Wu and Matt Gardner. 2021. Understanding mention detector-linker interaction in neural coreference resolution. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 150–157. [↗](https://doi.org/10.48448/9cw4-g040) [our card](https://md.research.elamur.ai/free/en/papers/wu-2021-understanding-mention-detector-linker-interaction/)
28. Patrick Xia, João Sedoc, and Benjamin Van Durme. 2020. Incremental neural coreference resolution in constant memory. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8617–8624. [find](https://scholar.google.com/scholar?q=Patrick+Xia%2C+Jo%C3%A3o+Sedoc%2C+and+Benjamin+Van+Durme%2E+2020%2E+Incremental+neural+coreference+resolution+in+constant+memory%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Proc) [our card](https://md.research.elamur.ai/free/en/papers/xia-2020-incremental-neural-coreference-resolution-constant/)
29. Patrick Xia and Benjamin Van Durme. 2021. Moving on from ontonotes: Coreference resolution model transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5241–5256. [find](https://scholar.google.com/scholar?q=Patrick+Xia+and+Benjamin+Van+Durme%2E+2021%2E+Moving+on+from+ontonotes%3A+Coreference+resolution+model+transfer%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pa)
30. Liyan Xu and Jinho D. Choi. 2020. Revealing the myth of higher-order inference in coreference resolution. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8527–8533, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.emnlp-main.686) [our card](https://md.research.elamur.ai/free/en/papers/xu-2020-revealing-myth-higher-order-inference/)
31. Juntao Yu, Alexandra Uma, and Massimo Poesio. 2020. A cluster ranking model for full anaphora resolution. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 11–20. [↗](https://doi.org/10.48550/arxiv.1911.09532) [our card](https://md.research.elamur.ai/free/en/papers/yu-2020-cluster-ranking-model-full-anaphora/)