---
title: "MAD-G — paper by Ansell et al. (2021) — summary and ideas"
description: "MAD-G replaces per-language training with one shared hyper-network. 44 references, cited by 73 papers here."
authors: ["Alan Ansell", "Edoardo Maria Ponti", "Jonas Pfeiffer", "Sebastian Ruder", "Goran Glavaš", "Ivan Vulić", "Anna Korhonen"]
year: 2021
venue: "Findings of the Association for Computational Linguistics: EMNLP 2021"
source: "https://aclanthology.org/2021.findings-emnlp.410/"
doi: "10.18653/v1/2021.findings-emnlp.410"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ansell-2021-mad-g-multilingual-adapter-generation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ansell-2021-mad-g-multilingual-adapter-generation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/) / MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer

# MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer

**Authors:** Alan Ansell, [Edoardo Maria Ponti](https://md.research.elamur.ai/free/en/authors/edoardo-maria-ponti/), [Jonas Pfeiffer](https://md.research.elamur.ai/free/en/authors/jonas-pfeiffer/), [Sebastian Ruder](https://md.research.elamur.ai/free/en/authors/sebastian-ruder/), [Goran Glavaš](https://md.research.elamur.ai/free/en/authors/goran-glavas/), [Ivan Vulić](https://md.research.elamur.ai/free/en/authors/ivan-vulic/) and [Anna Korhonen](https://md.research.elamur.ai/free/en/authors/anna-korhonen/) · **Year:** 2021 · **Venue:** [Findings of the Association for Computational Linguistics: EMNLP 2021](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2021](https://md.research.elamur.ai/free/en/venues/findings/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.findings-emnlp.410/) · [PDF](https://aclanthology.org/2021.findings-emnlp.410.pdf) · [DOI 10.18653/v1/2021.findings-emnlp.410](https://doi.org/10.18653/v1/2021.findings-emnlp.410) · [OpenAlex](https://openalex.org/W3214578205)

**Citations:** 56 citations (OpenAlex, 2026-10-14) · [73](https://md.research.elamur.ai/free/en/papers/ansell-2021-mad-g-multilingual-adapter-generation/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Massively multilingual transformers such as mBERT transfer poorly to languages that are typologically distant or thinly represented in pretraining. The usual remedy is to train a separate language adapter for each language with masked language modeling (MLM), but that requires sizeable monolingual corpora and compute that most languages lack. MAD-G replaces per-language training with one shared hyper-network. Given a language's typological vector from the URIEL database, it outputs the weights of a task-agnostic bottleneck language adapter for mBERT. The generator is trained with MLM on Wikipedia text in 95 diverse languages. A variant also conditions on Transformer layer position, which cuts parameters further. Because only typology is needed, adapters can be produced even for languages with no text at all. In zero-shot transfer for POS tagging, UD dependency parsing and NER, MAD-G is competitive with MAD-X while costing about 50 times less per target language. It is especially helpful for low-resource African NER. This makes MAD-G a cheap way to support under-resourced languages.

## Key points
- The generator learns from 95 Wikipedias and takes URIEL feature vectors as input. Compared with language-specific adapter training (MAD-X), it is roughly 50× more efficient and needs fewer parameters.
- Training task adapters on 20 source languages, with the total number of examples held fixed, brought large gains for POS and DP. The biggest jump came when moving from one to two source languages.
- Unseen evaluation languages were grouped by genus: mBERT-genus (21), MAD-G-genus (4) and unseen-genus (8). MAD-G helped only the MAD-G-genus group, which shows that typological diversity in the training languages is crucial.
- For 14 unseen UD languages, generated adapters were further fine-tuned on samples of 1,000 to 100,000 Wikipedia words. This beat random initialization, with a modest margin for POS and a clearer one for DP that persisted even at 100k tokens.
- The code is released in the AdapterHub `adapter-transformers` repository.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 155 works · 115,562 citations
- [Technische Universität Darmstadt](https://md.research.elamur.ai/free/en/universities/technische-universitat-darmstadt/) — University, Darmstadt, Germany · 76 works · 35,008 citations
- [Charles University](https://md.research.elamur.ai/free/en/universities/charles-university/) — University, Prague, Czechia · 108 works · 20,329 citations
- [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) — Company, London, United Kingdom · 35 works · 75,886 citations
- [McGill University](https://md.research.elamur.ai/free/en/universities/mcgill-university/) — University, Montreal, Canada · 51 works · 10,476 citations
- [Mila - Quebec Artificial Intelligence Institute](https://md.research.elamur.ai/free/en/labs/mila-quebec-artificial-intelligence-institute/) — Research institute, Montreal, Canada · 31 works · 13,061 citations
- [University of Mannheim](https://md.research.elamur.ai/free/en/universities/university-of-mannheim/) — University, Mannheim, Germany · 18 works · 6,984 citations
- [Digital Research Infrastructure for Language Technologies, Arts and Humanities](https://md.research.elamur.ai/free/en/labs/digital-research-infrastructure-for-language-technologies-arts-and-humanities/) — Research institute, Prague, Czechia · 3 works · 136 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 467 works

## Cite

Cite as: Alan Ansell, Edoardo Maria Ponti, Jonas Pfeiffer, Sebastian Ruder, Goran Glavaš, Ivan Vulić and Anna Korhonen. 2021. MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer. Findings of the Association for Computational Linguistics: EMNLP 2021.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ansell2021mad,
  title = {{MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer}},
  author = {Alan Ansell, Edoardo Maria Ponti, Jonas Pfeiffer, Sebastian Ruder, Goran Glavaš, Ivan Vulić and Anna Korhonen},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.410},
  url = {https://doi.org/10.18653/v1/2021.findings-emnlp.410}
}
```

- [← previous GO FIGURE: A Meta Evaluation of Factuality in Summarization](https://md.research.elamur.ai/free/en/papers/gabriel-2021-go-figure-meta-evaluation-factuality/)
- [next → Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmark](https://md.research.elamur.ai/free/en/papers/niklaus-2021-swiss-judgment-prediction-multilingual-legal/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/)

## References

44 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. [David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D’souza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsuddeen Muhammad, Chris Chinenye Emezue, Joyce Nakatumba-Nabende, Perez Ogayo, Anuolu…](https://md.research.elamur.ai/free/en/papers/adelani-2021-masakhaner-named-entity-recognition-african/)
2. Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George F. Foster, Colin Cherry, Wolfgang Macherey, Zhifeng Chen, and Yonghui Wu. 2019. [Massively multilingual neural machine translation in the wild: Findings and challenges](https://md.research.elamur.ai/free/en/papers/arivazhagan-2019-massively-multilingual-neural-machine-translation/). arXiv pre…
3. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018. [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://md.research.elamur.ai/free/en/papers/artetxe-2018-vecmap-unsupervised/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1:…
4. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. [On the cross-lingual transferability of monolingual representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637, Online. Association for Computational Linguistics.
5. Ankur Bapna and Orhan Firat. 2019. Non-parametric adaptation for neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1921–1931, Minneapol… [find](https://scholar.google.com/scholar?q=Ankur+Bapna+and+Orhan+Firat%2E+2019%2E+Non%2Dparametric+adaptation+for+neural+machine+translation%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Li)
6. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlingu)
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
8. Timothy Dozat and Christopher D. Manning. 2017. [Deep biaffine attention for neural dependency parsing](https://md.research.elamur.ai/free/en/papers/dozat-2016-deep-biaffine-attention-neural-dependency/). In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. Open-Review.net.
9. Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. 2016. [Multi-way, multilingual neural machine translation with a shared attention mechanism](https://md.research.elamur.ai/free/en/papers/firat-2016-multi-way-multilingual-neural-machine/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 866–875,…
10. Goran Glavaš, Robert Litschko, Sebastian Ruder, and Ivan Vulić. 2019. How to (properly) evaluate cross-lingual word embeddings: On strong baselines, comparative analyses, and some misconceptions. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 710–7… [find](https://scholar.google.com/scholar?q=Goran+Glava%C5%A1%2C+Robert+Litschko%2C+Sebastian+Ruder%2C+and+Ivan+Vulic%CC%81%2E+2019%2E+How+to+%28properly%29+evaluate+cross%2Dlingual+word+embeddings%3A+On+strong+baselines%2C+comparative+analyses%2C+and+some+misconceptions%2E+In+)
11. Goran Glavaš and Ivan Vulić. 2021. Is supervised syntactic parsing beneficial for language understanding tasks? an empirical investigation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3090–3104, Online. Associati… [find](https://scholar.google.com/scholar?q=Goran+Glava%C5%A1+and+Ivan+Vulic%CC%81%2E+2021%2E+Is+supervised+syntactic+parsing+beneficial+for+language+understanding+tasks%3F+an+empirical+investigation%2E+In+Proceedings+of+the+16th+Conference+of+the+European+Chapt)
12. David Ha, Andrew M. Dai, and Quoc V. Le. 2017. Hypernetworks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. [find](https://scholar.google.com/scholar?q=David+Ha%2C+Andrew+M%2E+Dai%2C+and+Quoc+V%2E+Le%2E+2017%2E+Hypernetworks%2E+In+5th+International+Conference+on+Learning+Representations%2C+ICLR+2017%2C+Toulon%2C+France%2C+April+24%2D26%2C+2017%2C+Conference+Track+Proceedings%2E+O)
13. Harald Hammarström, Robert Forkel, and Martin Haspelmath. 2017. Glottolog 3.0. Max Planck Institute for the Science of Human History. [find](https://scholar.google.com/scholar?q=Harald+Hammarstr%C3%B6m%2C+Robert+Forkel%2C+and+Martin+Haspelmath%2E+2017%2E+Glottolog+3%2E0%2E+Max+Planck+Institute+for+the+Science+of+Human+History%2E)
14. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+de+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+NLP%2E)
15. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 2790–2799. PMLR. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+36th+International+Conference+on+Machine+Learning%2C+ICML+2019%2C+9%2D15+June+2019%2C+Long+Beach%2C+California%2C+USA%2C+volume+97+of+Proceedings+of+Machine+Learning+Research%2C+pages+2790%E2%80%932799%2E)
16. Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. XTREME: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of… [find](https://scholar.google.com/scholar?q=Junjie+Hu%2C+Sebastian+Ruder%2C+Aditya+Siddhant%2C+Graham+Neubig%2C+Orhan+Firat%2C+and+Melvin+Johnson%2E+2020%2E+XTREME%3A+A+massively+multilingual+multi%2Dtask+benchmark+for+evaluating+cross%2Dlingual+generalisation%2E+In)
17. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. [The state and fate of linguistic diversity and inclusion in the NLP world](https://md.research.elamur.ai/free/en/papers/joshi-2020-state-fate-linguistic-diversity-inclusion/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Comp…
18. Anne Lauscher, Vinit Ravishankar, Ivan Vulić, and Goran Glavaš. 2020. [From zero to hero: On the limitations of zero-shot language transfer with multilingual Transformers](https://md.research.elamur.ai/free/en/papers/lauscher-2020-zero-hero-limitations-zero-shot/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4483–4499, Online. Asso…
19. Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. 2017. [URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors](https://md.research.elamur.ai/free/en/papers/littell-2017-uriel-lang2vec-representing-languages-as/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational…
20. Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2016. [Multi-task sequence to sequence learning](https://md.research.elamur.ai/free/en/papers/luong-2015-multi-task-sequence-sequence-learning/). In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings. OpenReview.net.
21. Michael McCloskey and Neal J Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109–165. Elsevier. [find](https://scholar.google.com/scholar?q=Michael+McCloskey+and+Neal+J+Cohen%2E+1989%2E+Catastrophic+interference+in+connectionist+networks%3A+The+sequential+learning+problem%2E+In+Psychology+of+learning+and+motivation%2C+volume+24%2C+pages+109%E2%80%93165%2E+Else)
22. Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013. Exploiting similarities among languages for machine translation. arXiv preprint. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Quoc+V+Le%2C+and+Ilya+Sutskever%2E+2013%2E+Exploiting+similarities+among+languages+for+machine+translation%2E+arXiv+preprint%2E)
23. Nafise Sadat Moosavi, Angela Fan, Vered Shwartz, Goran Glavaš, Shafiq Joty, Alex Wang, and Thomas Wolf, editors. 2020. Proceedings of SustaiNLP: Workshop on Simple and Efficient Natural Language Processing. Association for Computational Linguistics, Online. [find](https://scholar.google.com/scholar?q=Nafise+Sadat+Moosavi%2C+Angela+Fan%2C+Vered+Shwartz%2C+Goran+Glava%C5%A1%2C+Shafiq+Joty%2C+Alex+Wang%2C+and+Thomas+Wolf%2C+editors%2E+2020%2E+Proceedings+of+SustaiNLP%3A+Workshop+on+Simple+and+Efficient+Natural+Language+Proce)
24. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021a. [AdapterFusion: Non-destructive task composition for transfer learning](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterfusion-non-destructive-task-composition/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487–50…
25. Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020a. [AdapterHub: A framework for adapting transformers](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterhub-framework-adapting-transformers/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,…
26. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020b. [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-mad-x-adapter-based-framework/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7654–7673, Online. Association for Computational…
27. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2021b. [UNKs Everywhere: Adapting Multilingual Language Models to New Scripts](https://md.research.elamur.ai/free/en/papers/pfeiffer-2021-unks-everywhere-adapting-multilingual-language/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Online, November , 2021.
28. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. [How multilingual is multilingual BERT?](https://md.research.elamur.ai/free/en/papers/pires-2019-how-multilingual-is-multilingual-bert/) In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996– 5001, Florence, Italy. Association for Computational Linguistics.
29. Emmanouil Antonios Platanios, Mrinmaya Sachan, Graham Neubig, and Tom Mitchell. 2018a. [Contextual parameter generation for universal neural machine translation](https://md.research.elamur.ai/free/en/papers/platanios-2018-contextual-parameter-generation-universal-neural/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 425–435, Brussels, Belgium. Association fo…
30. Emmanouil Antonios Platanios, Mrinmaya Sachan, Graham Neubig, and Tom Mitchell. 2018b. [Contextual parameter generation for universal neural machine translation](https://md.research.elamur.ai/free/en/papers/platanios-2018-contextual-parameter-generation-universal-neural/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 425–435, Brussels, Belgium. Association fo…
31. Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, and Anna Korhonen. 2020. [XCOPA: A multilingual dataset for causal commonsense reasoning](https://md.research.elamur.ai/free/en/papers/ponti-2020-xcopa-multilingual-dataset-causal-commonsense/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2362–2376, Online. Association…
32. Edoardo Maria Ponti, Helen O’Horan, Yevgeni Berzak, Ivan Vulić, Roi Reichart, Thierry Poibeau, Ekaterina Shutova, and Anna Korhonen. 2019a. [Modeling language variation and universals: A survey on typological linguistics for natural language processing](https://md.research.elamur.ai/free/en/papers/ponti-2019-modeling-language-variation-universals-survey/). Computational Linguistics, 45(3):559–601.
33. Edoardo Maria Ponti, Ivan Vulić, Ryan Cotterell, Roi Reichart, and Anna Korhonen. 2019b. Towards zero-shot language modeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJ… [find](https://scholar.google.com/scholar?q=Edoardo+Maria+Ponti%2C+Ivan+Vulic%CC%81%2C+Ryan+Cotterell%2C+Roi+Reichart%2C+and+Anna+Korhonen%2E+2019b%2E+Towards+zero%2Dshot+language+modeling%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Lan)
34. Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. [Learning multiple visual domains with residual adapters](https://md.research.elamur.ai/free/en/papers/rebuffi-2017-learning-multiple-visual-domains-residual/). In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 506–516.
35. Andreas Rücklé, Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, and Iryna Gurevych. 2021. [AdapterDrop: On the Efficiency of Adapters in Transformers](https://md.research.elamur.ai/free/en/papers/ruckle-2021-adapterdrop-efficiency-adapters-transformers/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Online, November , 2021…
36. Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, and Iryna Gurevych. 2021. [How good is your tokenizer? on the monolingual performance of multilingual language models](https://md.research.elamur.ai/free/en/papers/rust-2021-how-good-is-your-tokenizer/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, ACL 2021, Online, August 1-6,…
37. Edoardo Ponti, Ivan Vulić, Ryan Cotterell, Marinela Parovic, Roi Reichart, and Anna Korhonen. 2021a. Parameter space factorization for zero-shot learning across tasks and languages. Transactions of the Association for Computational Linguistics, 9(0):410– 428. [find](https://scholar.google.com/scholar?q=Edoardo+Ponti%2C+Ivan+Vulic%CC%81%2C+Ryan+Cotterell%2C+Marinela+Parovic%2C+Roi+Reichart%2C+and+Anna+Korhonen%2E+2021a%2E+Parameter+space+factorization+for+zero%2Dshot+learning+across+tasks+and+languages%2E+Transactions+of+t)
38. Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016. Meta-learning with memory-augmented neural networks. In Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1842– 1850, New Yor… [find](https://scholar.google.com/scholar?q=Adam+Santoro%2C+Sergey+Bartunov%2C+Matthew+Botvinick%2C+Daan+Wierstra%2C+and+Timothy+Lillicrap%2E+2016%2E+Meta%2Dlearning+with+memory%2Daugmented+neural+networks%2E+In+Proceedings+of+The+33rd+International+Conference+o)
39. Edoardo Maria Ponti, Rahul Aralikatte, Disha Shrivastava, Siva Reddy, and Anders Søgaard. 2021b. Minimax and neyman–Pearson meta-learning for outlier Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. [Energy and policy considerations for deep learning in NLP](https://md.research.elamur.ai/free/en/papers/strubell-2019-energy-policy-considerations-deep-learning/). In Proceedings of the 57th Annual…
40. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. [Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition](https://md.research.elamur.ai/free/en/papers/sang-2003-introduction-conll-2003-shared-task/). In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142–147.
41. Ahmet Üstün, Arianna Bisazza, Gosse Bouma, and Gertjan van Noord. 2020. UDapter: Language adaptation for truly Universal Dependency parsing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2302–2315, Online. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Ahmet+%C3%9Cst%C3%BCn%2C+Arianna+Bisazza%2C+Gosse+Bouma%2C+and+Gertjan+van+Noord%2E+2020%2E+UDapter%3A+Language+adaptation+for+truly+Universal+Dependency+parsing%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+)
42. Shijie Wu and Mark Dredze. 2019. [Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT](https://md.research.elamur.ai/free/en/papers/wu-2019-beto-bentz-becas-surprising-cross/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 833–844,…
43. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. [mT5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling…
44. Daniel Zeman, Joakim Nivre, Mitchell Abrams, Elia Ackermann, Noëmi Aepli, Hamid Aghaei, Željko Agić, Amir Ahmadi, Lars Ahrenberg, Chika Kennedy Ajede, Gabrielė Aleksandravičiūtė, Ika Alfina, Lene Antonsen, Katya Aplonova, Angelina Aquino, Carolina Aragon, Maria Jesus Aran- zabe, Hórunn Arnardó… [find](https://scholar.google.com/scholar?q=Daniel+Zeman%2C+Joakim+Nivre%2C+Mitchell+Abrams%2C+Elia+Ackermann%2C+No%C3%ABmi+Aepli%2C+Hamid+Aghaei%2C+%C5%BDeljko+Agic%CC%81%2C+Amir+Ahmadi%2C+Lars+Ahrenberg%2C+Chika+Kennedy+Ajede%2C+Gabriele%CC%87+Aleksandravic%CC%8Ciu%CC%84te%CC%87%2C+Ika+Alfina%2C+Lene)

*Updated 24 October 2026*