---
title: "ÚFAL CorPipe at CRAC 2022 — Straka 2022"
description: "This paper presents CorPipe, the ÚFAL system that took first place in the shared task on resolving coreference across many languages organized at CRAC…"
authors: ["Milan Straka", "Jana Straková"]
year: 2022
venue: "Proceedings of the CRAC 2022 Shared Task on Multilingual Coreference Resolution"
source: "https://aclanthology.org/2022.crac-mcr.4/"
arxiv: "2209.07278"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/straka-2022-fal-corpipe-crac-2022-effectivity/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/straka-2022-fal-corpipe-crac-2022-effectivity/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2022](https://md.research.elamur.ai/free/papers/2022/page/4/) / ÚFAL CorPipe at CRAC 2022: Effectivity of Multilingual Models for Coreference Resolution

# ÚFAL CorPipe at CRAC 2022: Effectivity of Multilingual Models for Coreference Resolution

**Authors:** [Milan Straka](https://md.research.elamur.ai/free/authors/milan-straka/), [Jana Straková](https://md.research.elamur.ai/free/authors/jana-strakova/) · **Year:** 2022 · **Venue:** Proceedings of the CRAC 2022 Shared Task on Multilingual Coreference Resolution · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.crac-mcr.4/) · [PDF](https://aclanthology.org/2022.crac-mcr.4.pdf) · [arXiv 2209.07278](https://arxiv.org/abs/2209.07278) · [OpenAlex](https://openalex.org/W4297847400)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14) · [12](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
This paper presents CorPipe, the ÚFAL system that took first place in the shared task on resolving coreference across many languages organized at CRAC 2022. The task covered 13 datasets in 10 languages from CorefUD 1.0, with all datasets weighted equally.

CorPipe sits between a classic pipeline and an end-to-end model. It first detects mentions, then links each detected mention to its most probable antecedent. Both steps are trained together on top of a single fine-tuned Transformer encoder, and each step has its own small task-specific layers. This design avoids scoring every possible span. Empty nodes, which matter in pro-drop languages, are inserted as explicit marked tokens so the model can handle them.

The authors compared many multilingual and language-specific pretrained encoders, as well as several ways of sampling data across corpora. A single large multilingual model trained on all the data, with RemBERT working best, beat individually trained models on every dataset. The gains were not limited to the small or typologically related ones. This matters for low-resource languages: one shared large model can be a stronger default than separate monolingual systems.

## Key points
- Dataset sizes ranged from 457 training sentences (de/en parcorfull) to nearly 40K (cs pcedt, cs pdt), so the macro-averaged score made performance balancing across corpora important.
- Mention detection is framed as token tagging with an extended BIO scheme that encodes stack-like PUSH/POP operations, decoded with a linear-chain CRF.
- Coreference linking uses masked dot-product attention over earlier mentions. A mention can point to itself, which handles singletons. Training targets all gold antecedents, while prediction keeps only the top one.
- Encoders tested include RemBERT, XLM-R base/large, and mBERT, plus monolingual models such as RobeCzech, HerBERT, RuBERT, SpanBERT, and CamemBERT. Zero-shot transfer to unseen corpora performed far worse than multilingual training.
- Trimming predicted mentions to their syntactic heads improved partial-match scores; the code is released at github.com/ufal/crac2022-corpipe.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/)

## Cited by

Cited by

12 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 9 not yet.

- [Findings of the Second Shared Task on Multilingual Coreference Resolution](https://md.research.elamur.ai/free/papers/abokrtsky-2023-findings-second-shared-task-multilingual/)
- [Investigating Multilingual Coreference Resolution by Universal Annotations](https://md.research.elamur.ai/free/papers/chai-2023-investigating-multilingual-coreference-resolution-universal/)
- [ÚFAL CorPipe at CRAC 2023: Larger Context Improves Multilingual Coreference Resolution](https://md.research.elamur.ai/free/papers/straka-2023-fal-corpipe-crac-2023-larger/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{straka2022ufal,
  title = {{ÚFAL CorPipe at CRAC 2022: Effectivity of Multilingual Models for Coreference Resolution}},
  author = {Straka, Milan and Straková, Jana},
  booktitle = {Proceedings of the CRAC 2022 Shared Task on Multilingual Coreference Resolution},
  year = {2022},
  eprint = {2209.07278},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2209.07278}
}
```

- [← previous ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers](https://md.research.elamur.ai/free/papers/yao-2022-zeroquant-efficient-affordable-post-training/)
- [next → ♫ MuSiQue: Multihop Questions via Single-hop Question Composition](https://md.research.elamur.ai/free/papers/trivedi-2022-musique-multihop-questions-single-hop/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/papers/2022/page/4/)

## References

20 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jordi Armengol-Estapé, Casimiro Pio Carrino, Carlos Rodriguez-Penagos, Ona de Gibert Bonet, Carme Armentano-Oller, Aitor Gonzalez-Agirre, Maite Melero, and Marta Villegas. 2021. Are multilingual models the best choice for moderately under-resourced languages? A comprehensive assessment for Catalan.… [find](https://scholar.google.com/scholar?q=Jordi+Armengol%2DEstap%C3%A9%2C+Casimiro+Pio+Carrino%2C+Carlos+Rodriguez%2DPenagos%2C+Ona+de+Gibert+Bonet%2C+Carme+Armentano%2DOller%2C+Aitor+Gonzalez%2DAgirre%2C+Maite+Melero%2C+and+Marta+Villegas%2E+2021%2E+Are+multilingual+model)
2. Ben Kantor and Amir Globerson. 2019. Coreference resolution with entity equalization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 673–677, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ben+Kantor+and+Amir+Globerson%2E+2019%2E+Coreference+resolution+with+entity+equalization%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+673%E2%80%93677%2C+Florenc) [our card](https://md.research.elamur.ai/free/papers/kantor-2019-coreference-resolution-entity-equalization/)
3. José Cañete, Gabriel Chaperon, Rodrigo Fuentes, Jou-Hui Ho, Hojin Kang, and Jorge Pérez. 2020. Spanish pre-trained bert model and evaluation data. In PML4DC at ICLR 2020. [find](https://scholar.google.com/scholar?q=Jos%C3%A9+Ca%C3%B1ete%2C+Gabriel+Chaperon%2C+Rodrigo+Fuentes%2C+Jou%2DHui+Ho%2C+Hojin+Kang%2C+and+Jorge+P%C3%A9rez%2E+2020%2E+Spanish+pre%2Dtrained+bert+model+and+evaluation+data%2E+In+PML4DC+at+ICLR+2020%2E)
4. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR (Poster), CoRR abs/1412.6980. [find](https://scholar.google.com/scholar?q=Diederik+P%2E+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+ICLR+%28Poster%29%2C+CoRR+abs%2F1412%2E6980%2E)
5. Yuri Kuratov and Mikhail Y. Arkhipov. 2019. Adaptation of deep bidirectional multilingual transformers for russian language. CoRR, abs/1905.07213. [find](https://scholar.google.com/scholar?q=Yuri+Kuratov+and+Mikhail+Y%2E+Arkhipov%2E+2019%2E+Adaptation+of+deep+bidirectional+multilingual+transformers+for+russian+language%2E+CoRR%2C+abs%2F1905%2E07213%2E)
6. Branden Chan, Stefan Schweter, and Timo Möller. 2020. German’s next language model. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6788–6796, Barcelona, Spain (Online). International Committee on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Branden+Chan%2C+Stefan+Schweter%2C+and+Timo+M%C3%B6ller%2E+2020%2E+German%E2%80%99s+next+language+model%2E+In+Proceedings+of+the+28th+International+Conference+on+Computational+Linguistics%2C+pages+6788%E2%80%936796%2C+Barcelona%2C+Spain+) [our card](https://md.research.elamur.ai/free/papers/chan-2020-german-s-next-language-model/)
7. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Milan Straka, Jakub Náplava, Jana Straková, and David Samuel. 2021. Robeczech: Czech roberta, a monolingual contextualized language representation model.… [find](https://scholar.google.com/scholar?q=Kenton+Lee%2C+Luheng+He%2C+Mike+Lewis%2C+and+Luke+Zettlemoyer%2E+2017%2E+End%2Dto%2Dend+neural+coreference+resolution%2E+In+Proceedings+of+the+2017+Conference+on+Milan+Straka%2C+Jakub+N%C3%A1plava%2C+Jana+Strakov%C3%A1%2C+and+David+) [our card](https://md.research.elamur.ai/free/papers/lee-2017-end-end-neural-coreference-resolution/)
8. Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Empirical+Methods+in+Natural+Language+Processing%2C+pages+188%E2%80%93197%2C+Copenhagen%2C+Denmark%2E+Association+for+Computational+Linguistics%2E)
9. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–… [find](https://scholar.google.com/scholar?q=Kenton+Lee%2C+Luheng+He%2C+and+Luke+Zettlemoyer%2E+2018%2E+Higher%2Dorder+coreference+resolution+with+coarse%2Dto%2Dfine+inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter+of+the+Associa) [our card](https://md.research.elamur.ai/free/papers/lee-2018-higher-order-coreference-resolution-coarse/)
10. Matej Ulčar and Marko Robnik-Šikonja. 2021. Training dataset and dictionary sizes matter in bert models: the case of baltic languages. [find](https://scholar.google.com/scholar?q=Matej+Ulc%CC%8Car+and+Marko+Robnik%2D%C5%A0ikonja%2E+2021%2E+Training+dataset+and+dictionary+sizes+matter+in+bert+models%3A+the+case+of+baltic+languages%2E)
11. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81+ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+Neural+Information+Process)
12. Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, and Benoît Sagot. 2020. CamemBERT: a tasty French language model. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7203– 7219,… [find](https://scholar.google.com/scholar?q=Louis+Martin%2C+Benjamin+Muller%2C+Pedro+Javier+Ortiz+Su%C3%A1rez%2C+Yoann+Dupont%2C+Laurent+Romary%2C+%C3%89ric+de+la+Clergerie%2C+Djam%C3%A9+Seddah%2C+and+Beno%C3%AEt+Sagot%2E+2020%2E+CamemBERT%3A+a+tasty+French+language+model%2E+In+Proceed) [our card](https://md.research.elamur.ai/free/papers/martin-2019-camembert-tasty-french-language-model/)
13. Liyan Xu and Jinho D. Choi. 2020. Revealing the myth of higher-order inference in coreference resolution. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8527–8533, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Liyan+Xu+and+Jinho+D%2E+Choi%2E+2020%2E+Revealing+the+myth+of+higher%2Dorder+inference+in+coreference+resolution%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMN) [our card](https://md.research.elamur.ai/free/papers/xu-2020-revealing-myth-higher-order-inference/)
14. Robert Mroczkowski, Piotr Rybak, Alina Wróblewska, and Ireneusz Gawlik. 2021. HerBERT: Efficiently pretrained transformer-based language model for Polish. In Proceedings of the 8th Workshop on Balto-Slavic Natural Language Processing, pages 1–10, Kiyv, Ukraine. Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=Robert+Mroczkowski%2C+Piotr+Rybak%2C+Alina+Wr%C3%B3blewska%2C+and+Ireneusz+Gawlik%2E+2021%2E+HerBERT%3A+Efficiently+pretrained+transformer%2Dbased+language+model+for+Polish%2E+In+Proceedings+of+the+8th+Workshop+on+Balto%2DS)
15. Zdeněk Žabokrtský, Miloslav Konopík, Anna Nedoluzhko, Michal Novák, Maciej Ogrodniczuk, Martin Popel, Ondřej Pražák, Jakub Sido, Daniel Zeman, and Yilun Zhu. 2022. Findings of the Shared Task on Multilingual Coreference Resolution. In Proceedings of the CRAC 2022 Shared Task on Multilingual Corefe… [find](https://scholar.google.com/scholar?q=Zdene%CC%8Ck+%C5%BDabokrtsk%C3%BD%2C+Miloslav+Konop%C3%ADk%2C+Anna+Nedoluzhko%2C+Michal+Nov%C3%A1k%2C+Maciej+Ogrodniczuk%2C+Martin+Popel%2C+Ondr%CC%8Cej+Pra%C5%BE%C3%A1k%2C+Jakub+Sido%2C+Daniel+Zeman%2C+and+Yilun+Zhu%2E+2022%2E+Findings+of+the+Shared+Task+on+Mul)
16. Anna Nedoluzhko, Michal Novák, Martin Popel, Zdeněk Žabokrtský, Amir Zeldes, Daniel Zeman, Peter Bourgonje, Silvie Cinková, Jan Hajič, Christian Hardmeier, Pauline Krielke, Frédéric Landragin, Ekaterina Lapshinova-Koltunski, M. Antònia Martí, Marie Mikulová, Maciej Ogrodniczuk, Marta Recasens, Man… [find](https://scholar.google.com/scholar?q=Anna+Nedoluzhko%2C+Michal+Nov%C3%A1k%2C+Martin+Popel%2C+Zdene%CC%8Ck+%C5%BDabokrtsk%C3%BD%2C+Amir+Zeldes%2C+Daniel+Zeman%2C+Peter+Bourgonje%2C+Silvie+Cinkov%C3%A1%2C+Jan+Hajic%CC%8C%2C+Christian+Hardmeier%2C+Pauline+Krielke%2C+Fr%C3%A9d%C3%A9ric+Landragin%2C+Ekate)
17. Anna Nedoluzhko, Michal Novák, Martin Popel, Zdeněk Žabokrtský, and Daniel Zeman. 2021. Coreference in universal dependencies 0.1 (CorefUD 0.1). LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. [find](https://scholar.google.com/scholar?q=Anna+Nedoluzhko%2C+Michal+Nov%C3%A1k%2C+Martin+Popel%2C+Zdene%CC%8Ck+%C5%BDabokrtsk%C3%BD%2C+and+Daniel+Zeman%2E+2021%2E+Coreference+in+universal+dependencies+0%2E1+%28CorefUD+0%2E1%29%2E+LINDAT%2FCLARIAH%2DCZ+digital+library+at+the+Institute+of+)
18. Dávid Márk Nemeskey. 2020. Natural Language Processing Methods for Language Modeling. Ph.D. thesis, Eötvös Loránd University. [find](https://scholar.google.com/scholar?q=D%C3%A1vid+M%C3%A1rk+Nemeskey%2E+2020%2E+Natural+Language+Processing+Methods+for+Language+Modeling%2E+Ph%2ED%2E+thesis%2C+E%C3%B6tv%C3%B6s+Lor%C3%A1nd+University%2E)
19. Ondřej Pražák, Miloslav Konopík, and Jakub Sido. 2021. Multilingual coreference resolution with harmonized annotations. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1119–1123. [find](https://scholar.google.com/scholar?q=Ondr%CC%8Cej+Pra%C5%BE%C3%A1k%2C+Miloslav+Konop%C3%ADk%2C+and+Jakub+Sido%2E+2021%2E+Multilingual+coreference+resolution+with+harmonized+annotations%2E+In+Proceedings+of+the+International+Conference+on+Recent+Advances+in+Natural+La) [our card](https://md.research.elamur.ai/free/papers/prazak-2021-multilingual-coreference-resolution-harmonized-annotations/)
20. Ondřej Pražák and Miloslav Konopik. 2022. End-to-end Multilingual Coreference Resolution with Mention Head Prediction. In Proceedings of the CRAC 2022 Shared Task on Multilingual Coreference Resolution. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ondr%CC%8Cej+Pra%C5%BE%C3%A1k+and+Miloslav+Konopik%2E+2022%2E+End%2Dto%2Dend+Multilingual+Coreference+Resolution+with+Mention+Head+Prediction%2E+In+Proceedings+of+the+CRAC+2022+Shared+Task+on+Multilingual+Coreference+Resoluti) [our card](https://md.research.elamur.ai/free/papers/prazak-2022-end-end-multilingual-coreference-resolution/)