--- title: "The CODI-CRAC 2021 Shared Task on Anaphora — Li 2021" description: "CODI-CRAC 2021 was the first shared task devoted entirely to anaphora resolution in dialogue." authors: ["Shengjie Li", "Hideo Kobayashi", "Vincent Ng"] year: 2021 venue: "Proceedings of the CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue" source: "https://aclanthology.org/2021.codi-sharedtask.8/" doi: "10.18653/v1/2021.codi-sharedtask.8" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/li-2021-codi-crac-2021-shared-task/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/li-2021-codi-crac-2021-shared-task/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2021](https://md.research.elamur.ai/free/papers/2021/page/4/) / The CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis Resolution in Dialogue: A Cross-Team Analysis # The CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis Resolution in Dialogue: A Cross-Team Analysis **Authors:** Shengjie Li, [Hideo Kobayashi](https://md.research.elamur.ai/free/authors/hideo-kobayashi/), [Vincent Ng](https://md.research.elamur.ai/free/authors/vincent-ng/) · **Year:** 2021 · **Venue:** Proceedings of the CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2021.codi-sharedtask.8/) · [PDF](https://aclanthology.org/2021.codi-sharedtask.8.pdf) · [DOI 10.18653/v1/2021.codi-sharedtask.8](https://doi.org/10.18653/v1/2021.codi-sharedtask.8) · [OpenAlex](https://openalex.org/W3212286796) **Citations:** 2 citations (openalex-snapshot · 2026-10-14) · 5 in our library, of the 140673 papers whose bibliographies we read ## Summary CODI-CRAC 2021 was the first shared task devoted entirely to anaphora resolution in dialogue. It had three tracks: entity coreference, bridging, and discourse deixis. The last two are rarely studied, especially for conversational data. This paper compares the participating systems across teams, combining statistics computed from their outputs with manual inspection of their errors. For entity coreference, the authors break mention extraction down by pronoun type and compare official scores on four dialogue corpora. For discourse deixis, they argue that structural limits on antecedents shrink the search space, so the real bottleneck is deciding whether a word like "that" or "this" is deictic at all. The authors stress that deep error analysis is hard because teams submitted clusters rather than links, models are tightly integrated, and teams were free to use external data. They recommend that future shared tasks collect system links, run diagnostic tests with gold mentions or anaphors, and check annotation quality. ## Key points - Four teams were analysed in the entity coreference track: Emory, UTD, KU and DFKI. They were evaluated on the LIGHT, AMI, Persuasion and Switchboard corpora using MUC, B³, CEAFe and CoNLL scores. Emory scored best overall, and AMI was the hardest corpus. - Every system except DFKI averaged above 85% mention-extraction F-score. Emory and KU favoured recall, while UTD had the highest precision. - About half of the 19,051 gold mentions are pronouns, reflecting dialogue style. Roughly 80% of wrongly extracted mentions are non-pronominal phrases, and only about 4% of gold mentions were missed by all systems. - Discourse deixis antecedents are limited to utterances within one turn and tend to appear close to the anaphor. Recognising the anaphor is the main difficulty, and DFKI over-labelled "this" and "that" as deictic. - The Gold phase supplied a confusing union of mentions from all three tracks. In the deixis track, these gold mentions also omitted the antecedents. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) Topics: [Discourse analysis](https://md.research.elamur.ai/free/topics/discourse-analysis/) · [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{li2021codi, title = {{The CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis Resolution in Dialogue: A Cross-Team Analysis}}, author = {Li, Shengjie and Kobayashi, Hideo and Ng, Vincent}, booktitle = {Proceedings of the CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue}, year = {2021}, doi = {10.18653/v1/2021.codi-sharedtask.8}, url = {https://doi.org/10.18653/v1/2021.codi-sharedtask.8} } ``` - [← previous The CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue](https://md.research.elamur.ai/free/papers/khosla-2021-codi-crac-2021-shared-task/) - [next → The Curious Case of Hallucinations in Neural Machine Translation](https://md.research.elamur.ai/free/papers/raunak-2021-curious-case-hallucinations-neural-machine/) - [↑ back to papers of 2021](https://md.research.elamur.ai/free/papers/2021/page/4/) ## References 1 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Tatiana Anikina, Cennet Oguz, Natalia Skachkova, Siyu Tao, Sharmila Upadhyaya, and Ivana Kruijff-Korbayova. 2021. Anaphora resolution in dialogue: Description of the DFKI-TalkingRobots system for the CODI-CRAC 2021 shared-task. In Proceedings of the CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue, Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tatiana+Anikina%2C+Cennet+Oguz%2C+Natalia+Skachkova%2C+Siyu+Tao%2C+Sharmila+Upadhyaya%2C+and+Ivana+Kruijff%2DKorbayova%2E+2021%2E+Anaphora+resolution+in+dialogue%3A+Description+of+the+DFKI%2DTalkingRobots+system+for+the+) [our card](https://md.research.elamur.ai/free/papers/anikina-2021-anaphora-resolution-dialogue-description-dfki/)