---
title: "Cascading Adaptors to Leverage English Data to Improve"
description: "The paper asks how far multilingual transformers can push extractive question answering for… 19 references, cited by 5 papers here."
authors: ["Hariom Pandya", "Bhavik Ardeshna", "Brijesh Bhatt"]
year: 2021
venue: "Proceedings of the 18th International Conference on Natural Language Processing (ICON)"
source: "https://aclanthology.org/2021.icon-main.66/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pandya-2021-cascading-adaptors-leverage-english-data/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pandya-2021-cascading-adaptors-leverage-english-data/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/) / Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages

# Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages

**Authors:** Hariom Pandya, Bhavik Ardeshna and Brijesh Bhatt · **Year:** 2021 · **Venue:** [Proceedings of the 18th International Conference on Natural Language Processing (ICON)](https://md.research.elamur.ai/free/en/venues/icon/) ([ICON 2021](https://md.research.elamur.ai/free/en/venues/icon/2021/)) · **Publisher:** [NLP Association of India (NLPAI)](https://md.research.elamur.ai/free/en/publishers/nlp-association-of-india-nlpai/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.icon-main.66/) · [PDF](https://aclanthology.org/2021.icon-main.66.pdf) · [OpenAlex](https://openalex.org/W4225763334)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · 5 in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks how far multilingual transformers can push extractive question answering for languages with little QA training data, and whether small adapter modules help more than plain fine-tuning. The authors had no training split for the non-English languages, so they built one. They combined the XQuAD data with the MLQA test portion to form training data and reserved the MLQA dev set as a shared test set for seven languages: English, German, Spanish, Arabic, Hindi, Vietnamese and Simplified Chinese.

They compared four setups:
- **A:** full fine-tuning of mBERT and XLM-RoBERTa.
- **B:** task adapters only (Houlsby vs. Pfeiffer).
- **C:** language adapters, alone and stacked under a task adapter.
- **D:** MAD-X-style zero-shot transfer. A task adapter trained over the English language adapter is reused after swapping in the target-language adapter.

XLM-RoBERTa large gave the strongest scores. Among base-size variants, language adapters gave small gains for several languages. Zero-shot MAD-X came close to supervised results for Hindi and German. The work matters because it shows how to reuse English QA data cheaply through modular, swappable components.

## Key points
- **Data split:** train = MLQA test + XQuAD; test = MLQA dev.
  - Sizes for most languages are roughly 5.7k–6.9k train and about 500 test (Hindi 6854/507).
  - English has 12780/1148.
- **Setup A (F1/EM):**
  - Hindi: mBERT 56.25/39.45, XLM-R base 64.49/48.32, XLM-R large 73.37/56.02.
  - English with XLM-R large reaches 85.98/74.39.
  - Chinese is weakest for all models (about 42–50 F1).
- **Setup C, language adapter only:** beats task adapters for Hindi (66.14/49.11), Spanish (70.25/49.6), German, Chinese and English.
  - Stacking a task adapter on top lowered scores for most languages; Arabic was the exception.
- **Task adapter architecture:** the choice between Houlsby and Pfeiffer had negligible effect for most languages.
- **Setup D, zero-shot MAD-X from English:**
  - Hindi 65.24/48.91 and German 60.42/43.35.
  - Arabic drops to 50.12/31.33.
  - Jaccard and WER are reported alongside F1/EM for all setups.

## Interesting ideas
- Keeping the task skill and the language knowledge in separate plug-in modules lets one task module be trained once on English and reused by swapping only the language part; this is useful wherever one capability must serve several input varieties.
- When a benchmark ships only evaluation splits, the authors carve their own training data from them but hold one fixed split as the test set for every language, so comparisons stay consistent.
- Freezing the backbone and updating only a small adapter makes the trained component cheap to store, share and compare.
- Scoring answers with several overlap metrics (EM, F1, Jaccard, WER) shows partial-credit behaviour that exact match alone hides.
- The tables show that stacking more modules does not automatically help (language adapter alone often beat language + task), which argues for always comparing against the simpler configuration.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): The same split could keep a parser's core weights fixed while small swappable modules handle different UD dialects or genres, rather than retraining gparse per treebank.
- **I2** (learn): Our evaluation sets for fables and school tests could follow the same discipline: one frozen test portion shared by all experiments, with everything else free for silver training.
- **I3** (learn): Repair rules or teacher-tier corrections could likewise be packaged as small add-ons over a frozen base parser, so each one's effect is measurable and reversible.
- **I4** (answer): QA readers on tales could report exact and token-overlap scores together, separating "wrong span" from "slightly too long span" errors.
- **I5** (learn): Every added component in the pipeline should be evaluated against the version without it, as a built-in negative control.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Dharmsinh Desai University](https://md.research.elamur.ai/free/en/universities/dharmsinh-desai-university/) — University, Nadiād, India · 2 works · 92 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (4): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works · [🌱 Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) · 267 works

## Cite

Cite as: Hariom Pandya, Bhavik Ardeshna and Brijesh Bhatt. 2021. Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages. Proceedings of the 18th International Conference on Natural Language Processing (ICON).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pandya2021cascading,
  title = {{Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages}},
  author = {Hariom Pandya, Bhavik Ardeshna and Brijesh Bhatt},
  booktitle = {Proceedings of the 18th International Conference on Natural Language Processing (ICON)},
  year = {2021},
  url = {https://aclanthology.org/2021.icon-main.66/}
}
```

- [← previous ArCorona: Analyzing Arabic Tweets in the Early Days of Coronavirus (COVID-19) Pandemic](https://md.research.elamur.ai/free/en/papers/mubarak-2021-arcorona-analyzing-arabic-tweets-early/)
- [next → Controlling Prosody in End-to-End TTS: A Case Study on Contrastive Focus Generation](https://md.research.elamur.ai/free/en/papers/latif-2021-controlling-prosody-end-end-tts/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/)

## References

19 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. [On the cross-lingual transferability of monolingual representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637, Online. Association for Computational Linguistics.
2. Ankur Bapna and Orhan Firat. 2019. [Simple, scalable adaptation for neural machine translation](https://md.research.elamur.ai/free/en/papers/bapna-2019-simple-scalable-adaptation-neural-machine/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1538– 1548, Hong Kong, China. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1909.08478)
3. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.747)
4. Pieter Delobelle, Thomas Winters, and Bettina Berendt. 2020. [RobBERT: a Dutch RoBERTa-based Language Model](https://md.research.elamur.ai/free/en/papers/delobelle-2020-robbert-dutch-roberta-based-language/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3255–3265, Online. Association for Computational Linguistics.
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
6. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2790–2799. PMLR. [↗](https://doi.org/10.48550/arxiv.1902.00751)
7. Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, and Pratyush Kumar. 2020. [IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages](https://md.research.elamur.ai/free/en/papers/kakwani-2020-indicnlpsuite-monolingual-corpora-evaluation-benchmarks/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4948– 4961, Online. Association for Computational Linguistics.
8. Patrick Lewis, Barlas Oguz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. 2020. [MLQA: Evaluating cross-lingual extractive question answering](https://md.research.elamur.ai/free/en/papers/lewis-2019-mlqa-evaluating-cross-lingual-extractive/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7315– 7330, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.653)
9. Rudra Murthy, Anoop Kunchukuttan, and Pushpak Bhattacharyya. 2019. Addressing word-order divergence in multilingual neural machine translation for extremely low resource languages. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3868–3873, Minneapolis, Minnesota. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rudra+Murthy%2C+Anoop+Kunchukuttan%2C+and+Pushpak+Bhattacharyya%2E+2019%2E+Addressing+word%2Dorder+divergence+in+multilingual+neural+machine+translation+for+extremely+low+resource+languages%2E+In+Proceedings+of+t)
10. HA Pandya and BS Bhatt. 2021. Question answering survey: Directions, challenges, datasets, evaluation matrices. Journal of Xidian University, 15(4):152– 168. [↗](https://doi.org/10.20944/preprints202112.0136.v1)
11. Youngja Park, Siddharth Patwardhan, Karthik Visweswariah, and Stephen Gates. 2008. An empirical analysis of word error rate and keyword error rate. pages 2070–2073. [find](https://scholar.google.com/scholar?q=Youngja+Park%2C+Siddharth+Patwardhan%2C+Karthik+Visweswariah%2C+and+Stephen+Gates%2E+2008%2E+An+empirical+analysis+of+word+error+rate+and+keyword+error+rate%2E+pages+2070%E2%80%932073%2E)
12. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. [AdapterFusion: Non-destructive task composition for transfer learning](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterfusion-non-destructive-task-composition/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487–503, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.eacl-main.39)
13. Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020a. [AdapterHub: A framework for adapting transformers](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterhub-framework-adapting-transformers/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 46–54, Online. Association for Computational Linguistics.
14. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020b. [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-mad-x-adapter-based-framework/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7654–7673, Online. Association for Computational Linguistics.
15. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. [How multilingual is multilingual BERT?](https://md.research.elamur.ai/free/en/papers/pires-2019-how-multilingual-is-multilingual-bert/) In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996–5001, Florence, Italy. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/p19-1493)
16. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21(140):1–67. [↗](https://doi.org/10.48550/arxiv.1910.10683)
17. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [SQuAD: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d16-1264)
18. Tulika Saha, Dhawal Gupta, Sriparna Saha, and Pushpak Bhattacharyya. 2021. A unified dialogue management strategy for multi-intent dialogue conversations in multiple languages. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 20(6). [↗](https://doi.org/10.1145/3461763)
19. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. [Xlnet: Generalized autoregressive pretraining for language understanding](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/). In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. [↗](https://doi.org/10.48550/arxiv.1906.08237)

*Updated 1 November 2026*