---
title: "UDapter — paper by Üstün et al. (2022) — summary and ideas"
description: "Multilingual parsers built on pre-trained Transformers help low-resource languages… 58.0 accuracy. 23 references, cited by 16 papers here."
authors: ["Ahmet Üstün", "Arianna Bisazza", "Gosse Bouma", "Gertjan van Noord"]
year: 2022
venue: "Computational Linguistics"
source: "https://aclanthology.org/2022.cl-3.3/"
doi: "10.1162/coli_a_00443"
licence: "CC BY-NC-ND 4.0 (Computational Linguistics, MIT Press)"
licence_url: "https://creativecommons.org/licenses/by-nc-nd/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ustun-2022-udapter-typology-based-language-adapters/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ustun-2022-udapter-typology-based-language-adapters/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/) / UDapter: Typology-based Language Adapters for Multilingual Dependency Parsing and Sequence Labeling

# UDapter: Typology-based Language Adapters for Multilingual Dependency Parsing and Sequence Labeling

**Authors:** [Ahmet Üstün](https://md.research.elamur.ai/free/en/authors/ahmet-ustun/), [Arianna Bisazza](https://md.research.elamur.ai/free/en/authors/arianna-bisazza/), [Gosse Bouma](https://md.research.elamur.ai/free/en/authors/gosse-bouma/) and [Gertjan van Noord](https://md.research.elamur.ai/free/en/authors/gertjan-van-noord/) · **Year:** 2022 · **Venue:** [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/) ([Computational Linguistics 2022](https://md.research.elamur.ai/free/en/venues/cl/2022/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY-NC-ND 4.0 (Computational Linguistics, MIT Press)](https://creativecommons.org/licenses/by-nc-nd/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.cl-3.3/) · [PDF](https://aclanthology.org/2022.cl-3.3.pdf) · [DOI 10.1162/coli\_a\_00443](https://doi.org/10.1162/coli_a_00443) · [OpenAlex](https://openalex.org/W4225909405)

**Citations:** 15 citations (OpenAlex, 2026-10-14) · [16](https://md.research.elamur.ai/free/en/papers/ustun-2022-udapter-typology-based-language-adapters/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Multilingual parsers built on pre-trained Transformers help low-resource languages, but they often lose accuracy on high-resource ones. This happens because languages compete for shared capacity and interfere with each other. UDapter tackles this by adding adapter modules whose weights are generated from a language embedding through contextual parameter generation. That embedding is projected from URIEL typological features rather than learned from scratch, so the model can produce adapters even for languages it never saw during fine-tuning. Many typological values are missing, so an extension randomly masks features during training and learns to predict them alongside the main task. This removes the need for an external imputation tool. UDapter was trained on 13 typologically diverse high-resource UD treebanks and tested on those languages plus 30 low-resource (zero-shot) languages. It covers dependency parsing, POS tagging, morphological tagging and NER. In parsing it beats monolingual and multilingual UDify baselines on most languages. In sequence labeling it helps high-resource languages and stays competitive in zero-shot settings. The work shows how linguistic typology can guide parameter sharing in multilingual NLP.

## Key points
- The backbone is an mBERT-based UDify-style parser. Adapters are generated per language from typology vectors drawn from URIEL, which covers 3,000+ languages. Code is available at github.com/ahmetustun/udapter.
- On the 30 zero-shot languages, UDapter reaches an average LAS of 36.5. This compares with 35.3 for multi-udify and 34.1 for the original UDify.
- Zero-shot sequence labeling results are mixed:
  - POS tagging: 58.4 vs 58.0 accuracy.
  - Morphological tagging: 44.3 vs 44.4 F1.
  - NER: 65.6 vs 67.4 F1.
- Jointly predicting masked typological features works well:
  - Accuracy is about 90% for training languages and about 82% for zero-shot languages.
  - Majority-class baselines reach about 68% and 65%, respectively.
- Ablations indicate that typology-driven adapter generation is the main source of the gains. Parsing quality with jointly predicted features is close to that obtained with KNN-imputed features.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Groningen](https://md.research.elamur.ai/free/en/universities/university-of-groningen/) — University, Groningen, Netherlands · 70 works · 5,142 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works · [🌳 Dependency parsing](https://md.research.elamur.ai/free/en/topics/dependency-parsing/) · 131 works

## Cite

Cite as: Ahmet Üstün, Arianna Bisazza, Gosse Bouma and Gertjan van Noord. 2022. UDapter: Typology-based Language Adapters for Multilingual Dependency Parsing and Sequence Labeling. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{ustun2022udapter,
  title = {{UDapter: Typology-based Language Adapters for Multilingual Dependency Parsing and Sequence Labeling}},
  author = {Ahmet Üstün, Arianna Bisazza, Gosse Bouma and Gertjan {van Noord}},
  journal = {Computational Linguistics},
  year = {2022},
  doi = {10.1162/coli\_a\_00443},
  url = {https://doi.org/10.1162/coli_a_00443}
}
```

- [← previous Pairwise Representation Learning for Event Coreference](https://md.research.elamur.ai/free/en/papers/yu-2022-pairwise-representation-learning-event-coreference/)
- [next → A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language Model](https://md.research.elamur.ai/free/en/papers/lakim-2022-holistic-assessment-carbon-footprint-noor/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/)

## References

23 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aharoni, Roee, Melvin Johnson, and Orhan Firat. 2019. [Massively multilingual neural machine translation](https://md.research.elamur.ai/free/en/papers/ammar-2016-many-languages-one-parser/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3874–388…
2. Arivazhagan, Naveen, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George Foster, Colin Cherry, Wolfgang Macherey, Zhifeng Chen, and Yonghui Wu. 2019. [Massively multilingual neural machine translation in the wild: Findings and challenges](https://md.research.elamur.ai/free/en/papers/arivazhagan-2019-massively-multilingual-neural-machine-translation/). CoRR, abs/1…
3. Artetxe, Mikel, Sebastian Ruder, and Dani Yogatama. 2020. [On the cross-lingual transferability of monolingual representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637. https://doi.org/10.18653/v1/2020 .acl-main.421 Chu, Yoeng Jin. 1… [↗](https://doi.org/10.18653/v1/2020)
4. Dong, Daxiang, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015. [Multi-task learning for multiple language translation](https://md.research.elamur.ai/free/en/papers/dong-2015-multi-task-learning-multiple-language/). In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Lo…
5. Duong, Long, Trevor Cohn, Steven Bird, and Paul Cook. 2015. A neural network model for low-resource Universal Dependency parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 339–348. https://doi.org/10.18653/v1/D15 -1040 Edmonds, Jack. 1967. Optim… [↗](https://doi.org/10.18653/v1/D15)
6. Firat, Orhan, Kyunghyun Cho, and Yoshua Bengio. 2016. [Multi-way, multilingual neural machine translation with a shared attention mechanism](https://md.research.elamur.ai/free/en/papers/firat-2016-multi-way-multilingual-neural-machine/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 866–875…
7. Ha, Thanh-Le, Jan Niehues, and Alex Waibel. 2016. [Toward multilingual neural machine translation with universal encoder and decoder](https://md.research.elamur.ai/free/en/papers/ha-2016-multilingual-neural-machine-translation-universal/). In Proceedings of the 13th International Conference on Spoken Language Translation. International Workshop on Spoken Language Translation. https://aclanthology.org/20… [↗](https://aclanthology.org/20…)
8. Hendrycks, Dan and Kevin Gimpel. 2016. Bridging nonlinearities and stochastic regularizers with Gaussian Error Linear Units. International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hendrycks%2C+Dan+and+Kevin+Gimpel%2E+2016%2E+Bridging+nonlinearities+and+stochastic+regularizers+with+Gaussian+Error+Linear+Units%2E+International+Conference+on+Learning+Representations%2E)
9. Houlsby, Neil, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning, pages 2790–2799. [find](https://scholar.google.com/scholar?q=Houlsby%2C+Neil%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+De+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+NLP%2E+In)
10. Howard, Jeremy and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328–339. https://doi.org/10.18653/v1/P18-1031 Inoue, Go, Hiroyuki Shindo,… [↗](https://doi.org/10.18653/v1/P18-1031)
11. Dallas, Texas: SIL International. Lin, Yu Hsiang, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, and Graham Neubig. 2019. [Choosing transfer languages for cross-lingual learning](https://md.research.elamur.ai/free/en/papers/malaviya-2017-learning-language-representations-typology-prediction/). In Proceed…
12. Mulcaire, Phoebe, Jungo Kasai, and Noah A. Smith. 2019. Polyglot contextual representations improve crosslingual transfer. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Paper… [find](https://scholar.google.com/scholar?q=Mulcaire%2C+Phoebe%2C+Jungo+Kasai%2C+and+Noah+A%2E+Smith%2E+2019%2E+Polyglot+contextual+representations+improve+crosslingual+transfer%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+As)
13. Neubig, Graham and Junjie Hu. 2018. [Rapid adaptation of neural machine translation to new languages](https://md.research.elamur.ai/free/en/papers/neubig-2018-rapid-adaptation-neural-machine-translation/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 875–880. https://doi.org/10 .18653/v1/D18-1103 Nivre, Joakim, Mitchell Abrams, Željko Agić, Lars Ahr…
14. Rahimi, Afshin, Yuan Li, and Trevor Cohn. 2019. [Massively multilingual transfer for NER](https://md.research.elamur.ai/free/en/papers/rahimi-2019-massively-multilingual-transfer-ner/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 151–164. https://doi.org/10.18653/v1/P19-1015 Rebuffi, Sylvestre Alvise, Hakan Bilen, and Andrea Vedaldi. 2018.… [↗](https://doi.org/10.18653/v1/P19-1015)
15. Stickland, Asa Cooper and Iain Murray. 2019. Bert and pals: Projected attention layers for efficient adaptation in multi-task learning. In International Conference on Machine Learning, pages 5986–5995. [find](https://scholar.google.com/scholar?q=Stickland%2C+Asa+Cooper+and+Iain+Murray%2E+2019%2E+Bert+and+pals%3A+Projected+attention+layers+for+efficient+adaptation+in+multi%2Dtask+learning%2E+In+International+Conference+on+Machine+Learning%2C+pages+5986%E2%80%935995)
16. Straka, Milan. 2018. [UDPipe 2.0 prototype at CoNLL 2018 UD shared task](https://md.research.elamur.ai/free/en/papers/straka-2018-udpipe-2-0-prototype-conll/). In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 197–207. Täckström, Oscar, Ryan McDonald, and Joakim Nivre. 2013. Target language adaptation of discriminative t…
17. Tran, Ke and Arianna Bisazza. 2019. Zero-shot dependency parsing with pre-trained multilingual sentence representations. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 281–288. https://doi.org/10 .18653/v1/D19-6132 Tsai, Henry, Jason Riesa, M… [find](https://scholar.google.com/scholar?q=Tran%2C+Ke+and+Arianna+Bisazza%2E+2019%2E+Zero%2Dshot+dependency+parsing+with+pre%2Dtrained+multilingual+sentence+representations%2E+In+Proceedings+of+the+2nd+Workshop+on+Deep+Learning+Approaches+for+Low%2DResource)
18. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems, pages 5998–6008. Vilares, David, Carlos Gómez-Rodrı́guez, and Miguel A. Alonso. 2016.…
19. Wang, Xinyu and Kewei Tu. 2020. Second-order neural dependency parsing with message passing and end-to-end training. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Proce… [find](https://scholar.google.com/scholar?q=Wang%2C+Xinyu+and+Kewei+Tu%2E+2020%2E+Second%2Dorder+neural+dependency+parsing+with+message+passing+and+end%2Dto%2Dend+training%2E+In+Proceedings+of+the+1st+Conference+of+the+Asia%2DPacific+Chapter+of+the+Association)
20. Wu, Shijie and Mark Dredze. 2019. [Beto, Bentz, Becas: The surprising cross-lingual effectiveness of BERT](https://md.research.elamur.ai/free/en/papers/wu-2019-beto-bentz-becas-surprising-cross/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 833–844.…
21. Yang, Songlin and Kewei Tu. 2021. Headed span-based projective dependency parsing. arXiv preprint arXiv:2108.04750. [find](https://scholar.google.com/scholar?q=Yang%2C+Songlin+and+Kewei+Tu%2E+2021%2E+Headed+span%2Dbased+projective+dependency+parsing%2E+arXiv+preprint+arXiv%3A2108%2E04750%2E)
22. Zeman, Daniel and Philip Resnik. 2008. [Cross-language parser adaptation between related languages](https://md.research.elamur.ai/free/en/papers/zeman-2008-cross-language-parser-adaptation-between/). In Proceedings of the IJCNLP-08 Workshop on NLP for Less Privileged Languages, pages 35–42.
23. Zhang, Yuan and Regina Barzilay. 2015. [Hierarchical low-rank tensors for multilingual transfer parsing](https://md.research.elamur.ai/free/en/papers/zhang-2015-hierarchical-low-rank-tensors-multilingual/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1857–1867. https://doi.org/10.18653/v1/D15-1213 [↗](https://doi.org/10.18653/v1/D15-1213)

*Updated 30 October 2026*