---
title: "TADA : Task Agnostic Dialect Adapters for English (2023)"
description: "The authors instead train one dialect module per dialect that works with any task. 0.3 points. 57 references, cited by 10 papers here."
authors: ["William Held", "Caleb Ziems", "Diyi Yang"]
year: 2023
venue: "Findings of the Association for Computational Linguistics: ACL 2023"
source: "https://aclanthology.org/2023.findings-acl.51/"
doi: "10.18653/v1/2023.findings-acl.51"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/held-2023-tada-task-agnostic-dialect-adapters/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/held-2023-tada-task-agnostic-dialect-adapters/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) / TADA : Task Agnostic Dialect Adapters for English

# TADA : Task Agnostic Dialect Adapters for English

**Authors:** [William Held](https://md.research.elamur.ai/free/en/authors/william-held/), [Caleb Ziems](https://md.research.elamur.ai/free/en/authors/caleb-ziems/) and [Diyi Yang](https://md.research.elamur.ai/free/en/authors/diyi-yang/) · **Year:** 2023 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2023](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2023](https://md.research.elamur.ai/free/en/venues/findings/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.findings-acl.51/) · [PDF](https://aclanthology.org/2023.findings-acl.51.pdf) · [DOI 10.18653/v1/2023.findings-acl.51](https://doi.org/10.18653/v1/2023.findings-acl.51) · [OpenAlex](https://openalex.org/W4385570931)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · [10](https://md.research.elamur.ai/free/en/papers/held-2023-tada-task-agnostic-dialect-adapters/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Encoder models such as RoBERTa perform worse when the input is in an English dialect other than Standard American English (SAE). Earlier fixes, such as Multi-VALUE augmentation, have to be repeated for every combination of dialect and task. The authors instead train one dialect module per dialect that works with any task.

They used Multi-VALUE's rule-based morphosyntactic rewrites on 1,000 WiC sentences, producing SAE/dialect sentence pairs. On these pairs they trained invertible adapters with two objectives:
- an L2 loss that pulls the dialect CLS vector toward the frozen SAE CLS vector;
- an adversarial loss, from a small transformer critic, aimed at token-level morphosyntactic differences.

At inference, the TADA module sits in front of ordinary SAE task adapters, and no task data in the dialect is needed. On VALUE-transformed GLUE, TADA raises the mean score for all four dialects tested. Task-specific dialect training still scores higher. This matters because a single cheap module (about 30 minutes on one GPU) can serve an open-ended set of downstream tasks.

## Key points
- **Training data:** 1,000 synthetic parallel pairs built from WiC sentences with Multi-VALUE, plus 100 held-out pairs used for early stopping on the L2 loss. Training runs 30 epochs, batch 16, learning rate 5e-4, in about 30 minutes on an RTX 2080 Ti.
- **AAE results with RoBERTa base (mean over 7 GLUE tasks):**

  | Method | Mean | Parameters |
  |---|---|---|
  | SAE finetuning | 75.1 | – |
  | SAE adapters | 74.7 | – |
  | TADA | 77.5 | one 895K module for all tasks |
  | VALUE task-specific finetuning | 77.5 | T × 110M |
  | VALUE task-specific adapters | 80.8 | T × 895K |

- **Biggest single gain:** CoLA under AAE rose from 14.1 to 29.5.
- **Cross-dialect mean gains:** +2.8 for AAE, +0.3 for Indian English, +0.4 for Nigerian English and +3.9 for Singaporean English. For Singaporean English, CoLA went from -0.4 to 20.3.
- **Ablation:**
  - Dropping the adversarial loss costs only 0.3 points on average.
  - Dropping the L2 sequence loss makes the adversary unstable and collapse, and the mean falls to 32.6 (-44.9).
- **Limitations noted by the authors:** evaluation uses only synthetic morphosyntactic perturbations, so register and topic shifts are untested. Only encoder models were evaluated.

## Interesting ideas
- Instead of retraining each task model for a dialect, a small front module maps dialect input into the representation space the frozen SAE task modules already expect, so one fix covers every downstream task.
- A rule-based rewriter turns a small set of clean, lexically varied sentences into parallel data; the small size is deliberate, so native speakers or linguists could later replace it with curated minimal pairs.
- The anchored-versus-unanchored ablation shows that an adversarial objective collapses without a simple supervised anchor, and only the full ablation exposes this.
- The authors suggest reporting results across both tasks and dialects, because a fix that helps one dialect a lot can barely move another (+3.9 versus +0.3).

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): gparse could gain one dialect-alignment adapter trained on SAE/dialect minimal pairs, with the existing UD parsing head left untouched, so a single module serves all dialects rather than one retrained parser per dialect.
- **I2** (learn): Mova could build tiny dialect/standard pair sets from clean sources such as dictionary example sentences, and use them both as alignment training data and as auditable test items.
- **I3** (learn): any adversarial or self-play objective in Mova's training should be paired with a plain distance or agreement anchor and checked by ablation, as a negative control against silent collapse.
- **I4** (answer): Mova's school-test and fable-QA evaluations could be reported as a task × dialect grid, so a gain on one variety is not mistaken for general robustness.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 434 works · 314,419 citations
- [Georgia Institute of Technology](https://md.research.elamur.ai/free/en/universities/georgia-institute-of-technology/) — University, Atlanta, United States · 133 works · 24,497 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: William Held, Caleb Ziems and Diyi Yang. 2023. TADA : Task Agnostic Dialect Adapters for English. Findings of the Association for Computational Linguistics: ACL 2023.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{held2023tada,
  title = {{TADA : Task Agnostic Dialect Adapters for English}},
  author = {William Held, Caleb Ziems and Diyi Yang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.51},
  url = {https://doi.org/10.18653/v1/2023.findings-acl.51}
}
```

- [← previous Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2023-skills-context-prompting-unlocking-compositionality/)
- [next → ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mind](https://md.research.elamur.ai/free/en/papers/ma-2023-tomchallenges-principle-guided-dataset-diverse/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/)

## References

57 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Oliver Adams, Adam Makarucha, Graham Neubig, Steven Bird, and Trevor Cohn. 2017. Cross-lingual word embeddings for low-resource language modeling. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 937–947, Valencia, Spain. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Oliver+Adams%2C+Adam+Makarucha%2C+Graham+Neubig%2C+Steven+Bird%2C+and+Trevor+Cohn%2E+2017%2E+Cross%2Dlingual+word+embeddings+for+low%2Dresource+language+modeling%2E+In+Proceedings+of+the+15th+Conference+of+the+European)
2. Anmol Agarwal, Jigar Gupta, Rahul Goel, Shyam Upadhyay, Pankaj Joshi, and Rengarajan Aravamudhan. 2022. Cst5: Data augmentation for code-switched semantic parsing. arXiv preprint arXiv:2211.07514. [↗](https://arxiv.org/abs/2211.07514)
3. Orevaoghene Ahia, Julia Kreutzer, and Sara Hooker. 2021. The low-resource double bind: An empirical study of pruning for low-resource machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3316–3333, Punta Cana, Dominican Republic. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.findings-emnlp.282)
4. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018. [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://md.research.elamur.ai/free/en/papers/artetxe-2018-vecmap-unsupervised/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 789–798, Melbourne, Australia. Association for Computational Linguistics.
5. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 610–623. [↗](https://doi.org/10.1145/3442188.3445922)
6. Steven Bird. 2022. Local languages, third spaces, and other high-resource scenarios. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7817–7829, Dublin, Ireland. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2022.acl-long.539)
7. Terra Blevins, Robert Kwiatkowski, Jamie MacBeth, Kathleen McKeown, Desmond Patton, and Owen Rambow. 2016. Automatically processing tweets from gang-involved youth: Towards detecting loss and aggression. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2196–2206, Osaka, Japan. [find](https://scholar.google.com/scholar?q=Terra+Blevins%2C+Robert+Kwiatkowski%2C+Jamie+MacBeth%2C+Kathleen+McKeown%2C+Desmond+Patton%2C+and+Owen+Rambow%2E+2016%2E+Automatically+processing+tweets+from+gang%2Dinvolved+youth%3A+Towards+detecting+loss+and+aggressi)
8. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
9. Su Lin Blodgett, Johnny Wei, and Brendan O’Connor. 2018. [Twitter Universal Dependency parsing for African-American and mainstream American English](https://md.research.elamur.ai/free/en/papers/blodgett-2018-twitter-universal-dependency-parsing-african/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1415–1425, Melbourne, Australia. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/p18-1131)
10. Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. 2021. [On the opportunities and risks of foundation models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/). [↗](https://doi.org/10.48550/arxiv.2108.07258)
11. Shuguang Chen, Gustavo Aguilar, Anirudh Srinivasan, Mona Diab, and Thamar Solorio. 2022. Calcs 2021 shared task: Machine translation for code-switched data. arXiv preprint arXiv:2202.09625. [↗](https://arxiv.org/abs/2202.09625)
12. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.747)
13. Alexis Conneau and Guillaume Lample. 2019. Cross-lingual language model pretraining. Advances in neural information processing systems, 32. [find](https://scholar.google.com/scholar?q=Alexis+Conneau+and+Guillaume+Lample%2E+2019%2E+Cross%2Dlingual+language+model+pretraining%2E+Advances+in+neural+information+processing+systems%2C+32%2E)
14. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475–2485, Brussels, Belgium. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d18-1269)
15. Marta R. Costa-jussà. 2017. Why Catalan-Spanish neural machine translation? analysis, comparison and combination with standard rule and phrase-based technologies. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 55–62, Valencia, Spain. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Marta+R%2E+Costa%2Djuss%C3%A0%2E+2017%2E+Why+Catalan%2DSpanish+neural+machine+translation%3F+analysis%2C+comparison+and+combination+with+standard+rule+and+phrase%2Dbased+technologies%2E+In+Proceedings+of+the+Fourth+Workshop)
16. Marta R. Costa-jussà, Marcos Zampieri, and Santanu Pal. 2018. A neural approach to language variety translation. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 275–282, Santa Fe, New Mexico, USA. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1807.00651)
17. Dorottya Demszky, Devyani Sharma, Jonathan Clark, Vinodkumar Prabhakaran, and Jacob Eisenstein. 2021. Learning to recognize dialect features. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2315–2338, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.naacl-main.184)
18. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
19. Mona Diab, Nizar Habash, Owen Rambow, Mohamed Altantawy, and Yassine Benajiba. 2010. Colaba: Arabic dialect annotation and processing. In Lrec workshop on semitic language processing, pages 66–74. Citeseer. [find](https://scholar.google.com/scholar?q=Mona+Diab%2C+Nizar+Habash%2C+Owen+Rambow%2C+Mohamed+Altantawy%2C+and+Yassine+Benajiba%2E+2010%2E+Colaba%3A+Arabic+dialect+annotation+and+processing%2E+In+Lrec+workshop+on+semitic+language+processing%2C+pages+66%E2%80%9374%2E+Cit)
20. Long Duong, Hiroshi Kanayama, Tengfei Ma, Steven Bird, and Trevor Cohn. 2016. Learning crosslingual word embeddings without bilingual corpora. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1285– 1295, Austin, Texas. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Long+Duong%2C+Hiroshi+Kanayama%2C+Tengfei+Ma%2C+Steven+Bird%2C+and+Trevor+Cohn%2E+2016%2E+Learning+crosslingual+word+embeddings+without+bilingual+corpora%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Method)
21. Kawin Ethayarajh and Dan Jurafsky. 2020. Utility is in the eye of the user: A critique of NLP leaderboards. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4846–4853, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh+and+Dan+Jurafsky%2E+2020%2E+Utility+is+in+the+eye+of+the+user%3A+A+critique+of+NLP+leaderboards%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28E)
22. Edouard Grave, Armand Joulin, and Quentin Berthet. 2019. Unsupervised alignment of embeddings with wasserstein procrustes. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1880–1890. PMLR. [↗](https://doi.org/10.48550/arxiv.1805.11222)
23. Suchin Gururangan, Dallas Card, Sarah K. Dreier, Emily K. Gade, Leroy Z. Wang, Zeyu Wang, Luke Zettlemoyer, and Noah A. Smith. 2022. Whose language counts as high quality? measuring language ideologies in text data selection. [↗](https://doi.org/10.48550/arxiv.2201.10474)
24. Injy Hamed, Nizar Habash, Slim Abdennadher, and Ngoc Thang Vu. 2022. [ArzEn-ST: A three-way speech translation corpus for code-switched Egyptian Arabic-English](https://md.research.elamur.ai/free/en/papers/hamed-2022-arzen-st-three-way-speech/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 119–130, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. [↗](https://doi.org/10.60692/m3b03-g1s60)
25. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pages 2790–2799. PMLR. [↗](https://doi.org/10.48550/arxiv.1902.00751)
26. Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2016. [Learning a POS tagger for AAVE-like language](https://md.research.elamur.ai/free/en/papers/jorgensen-2016-learning-pos-tagger-aave-like/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1115–1120, San Diego, California. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n16-1130)
27. David Jurgens, Yulia Tsvetkov, and Dan Jurafsky. 2017. Incorporating dialectal variability for socially equitable language identification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 51–57, Vancouver, Canada. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/p17-2009)
28. Filip Klubička, Gema Ramírez-Sánchez, and Nikola Ljubešić. 2016. Collaborative development of a rule-based machine translator between Croatian and Serbian. In Proceedings of the 19th Annual Conference of the European Association for Machine Translation, pages 361–367. [find](https://scholar.google.com/scholar?q=Filip+Klubic%CC%8Cka%2C+Gema+Ram%C3%ADrez%2DS%C3%A1nchez%2C+and+Nikola+Ljube%C5%A1ic%CC%81%2E+2016%2E+Collaborative+development+of+a+rule%2Dbased+machine+translator+between+Croatian+and+Serbian%2E+In+Proceedings+of+the+19th+Annual+Conferen)
29. Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14):7684–7689. [↗](https://doi.org/10.1073/pnas.1915768117)
30. Guillaume Lample, Alexis Conneau, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018. Word translation without parallel data. In International Conference on Learning Representations. [↗](https://doi.org/10.48550/arxiv.1710.04087)
31. Rebecca Qian, Candace Ross, Jude Fernandes, Eric Smith, Douwe Kiela, and Adina Williams. 2022. Perturbation augmentation for fairer nlp. arXiv preprint arXiv:2205.12586. [↗](https://arxiv.org/abs/2205.12586)
32. Zhaojiang Lin, Andrea Madotto, and Pascale Fung. 2020. [Exploring versatile generative language model via parameter-efficient transfer learning](https://md.research.elamur.ai/free/en/papers/lin-2020-exploring-versatile-generative-language-model/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 441–459, Online. Association for Computational Linguistics.
33. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). J. Mach. Learn. Res., 21(140):1–67. [↗](https://doi.org/10.48550/arxiv.1910.10683)
34. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692. [↗](https://arxiv.org/abs/1907.11692)
35. George A. Miller. 1994. [WordNet: A lexical database for English](https://md.research.elamur.ai/free/en/papers/miller-1994-wordnet/). In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.
36. Mehrad Moradshahi, Giovanni Campagna, Sina Semnani, Silei Xu, and Monica Lam. 2020. Localizing open-ontology QA semantic parsers in a day using machine translation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5970–5983, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mehrad+Moradshahi%2C+Giovanni+Campagna%2C+Sina+Semnani%2C+Silei+Xu%2C+and+Monica+Lam%2E+2020%2E+Localizing+open%2Dontology+QA+semantic+parsers+in+a+day+using+machine+translation%2E+In+Proceedings+of+the+2020+Conferen)
37. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. [AdapterFusion: Non-destructive task composition for transfer learning](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterfusion-non-destructive-task-composition/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487–503, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.eacl-main.39)
38. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020. [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-mad-x-adapter-based-framework/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7654–7673, Online. Association for Computational Linguistics.
39. Mohammad Taher Pilehvar and Jose Camacho-Collados. 2019. [WiC: the word-in-context dataset for evaluating context-sensitive meaning representations](https://md.research.elamur.ai/free/en/papers/pilehvar-2019-wic-word-context-dataset-evaluating/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1267–1273, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1808.09121)
40. Maja Popović, Alberto Poncelas, Marija Brkić, and Andy Way. 2020. Neural machine translation for translating into croatian and serbian. In Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects, pages 102–113. [find](https://scholar.google.com/scholar?q=Maja+Popovic%CC%81%2C+Alberto+Poncelas%2C+Marija+Brkic%CC%81%2C+and+Andy+Way%2E+2020%2E+Neural+machine+translation+for+translating+into+croatian+and+serbian%2E+In+Proceedings+of+the+7th+Workshop+on+NLP+for+Similar+Language)
41. Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. [Learning multiple visual domains with residual adapters](https://md.research.elamur.ai/free/en/papers/rebuffi-2017-learning-multiple-visual-domains-residual/). In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. [↗](https://doi.org/10.48550/arxiv.1705.08045)
42. Wael Salloum and Nizar Habash. 2013. [Dialectal Arabic to English machine translation: Pivoting through Modern Standard Arabic](https://md.research.elamur.ai/free/en/papers/salloum-2013-dialectal-arabic-english-machine-translation/). In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 348–358, Atlanta, Georgia. Association for Computational Linguistics.
43. Karin Kipper Schuler. 2005. VerbNet: A broad-coverage, comprehensive verb lexicon. University of Pennsylvania. [↗](https://doi.org/10.5281/zenodo.8100881)
44. Asa Cooper Stickland and Iain Murray. 2019. Bert and pals: Projected attention layers for efficient adaptation in multi-task learning. In International Conference on Machine Learning, pages 5986–5995. PMLR. [↗](https://doi.org/10.48550/arxiv.1902.02671)
45. Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7167–7176. [↗](https://doi.org/10.48550/arxiv.1702.05464)
46. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in neural information processing systems, pages 5998–6008.
47. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018. [Glue: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In BlackboxNLP@EMNLP. [↗](https://doi.org/10.18653/v1/w18-5446)
48. Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, and Colin Raffel. 2022. What language model architecture and pretraining objective work best for zero-shot generalization? arXiv preprint arXiv:2204.05832. [↗](https://arxiv.org/abs/2204.05832)
49. Zirui Wang, Zachary C. Lipton, and Yulia Tsvetkov. 2020. [On negative interference in multilingual models: Findings and a meta-learning treatment](https://md.research.elamur.ai/free/en/papers/wang-2020-negative-interference-multilingual-models-findings/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4438–4450, Online. Association for Computational Linguistics.
50. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. [mT5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483–498, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.naacl-main.41)
51. Qian Yang, Jina Suh, Nan-Chen Chen, and Gonzalo Ramos. 2018. Grounding interactive machine learning tool design in how non-experts actually build models. In Proceedings of the 2018 Designing Interactive Systems Conference, DIS ’18, page 573–584, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Qian+Yang%2C+Jina+Suh%2C+Nan%2DChen+Chen%2C+and+Gonzalo+Ramos%2E+2018%2E+Grounding+interactive+machine+learning+tool+design+in+how+non%2Dexperts+actually+build+models%2E+In+Proceedings+of+the+2018+Designing+Interacti)
52. Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017. Earth mover’s distance minimization for unsupervised bilingual lexicon induction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1934–1945, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Meng+Zhang%2C+Yang+Liu%2C+Huanbo+Luan%2C+and+Maosong+Sun%2E+2017%2E+Earth+mover%E2%80%99s+distance+minimization+for+unsupervised+bilingual+lexicon+induction%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+i)
53. Caleb Ziems, Jiaao Chen, Camille Harris, Jessica Anderson, and Diyi Yang. 2022. [VALUE: Understanding dialect disparity in NLU](https://md.research.elamur.ai/free/en/papers/ziems-2022-value-understanding-dialect-disparity-nlu/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3701–3720, Dublin, Ireland. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2022.acl-long.258)
54. Caleb Ziems, William Held, Jingfeng Yang, Jwala Dhamala, Rahul Gupta, and Diyi Yang. 2023. [Multi-VALUE: A framework for cross-dialectal English NLP](https://md.research.elamur.ai/free/en/papers/ziems-2023-multi-value-framework-cross-dialectal/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, Canada. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2023.acl-long.44)
55. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1568–1575. Approach N/A N/A VALUE VALUE TADA Dialect Adaptation Details Method Task-Agnostic Dialect Params. Finetuning ✓ 0 Adapters ✓ 0 Finetuning ✗ T × 110M Adapters ✗ T × 895K Adapters ✓ 895K COLA 36.0 31.4 36.2 36.3 38.3 MNLI 79.6 80.8 83.0 82.0 81.5 AAE Glue Performance QNLI RTE QQP SST2 89.2 65.3 86.2 89.7 89.2 62.1 86.0 89.8 89.7 61.4 88.6 89.6 89.5 66.8 85.6 88.8 89.0 62.1 87.0 90.0 STS-B 87.4 86.9 88.2 88.5 88.0 Mean 76.2 75.1 76.7 76.8 76.6 [find](https://scholar.google.com/scholar?q=Barret+Zoph%2C+Deniz+Yuret%2C+Jonathan+May%2C+and+Kevin+Knight%2E+2016%2E+Transfer+learning+for+low%2Dresource+neural+machine+translation%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Lan)
56. Table A1: Dialect Adaptation GLUE results of BERT Base (Devlin et al., 2019) for the 7 GLUE Tasks (Matthew’s Corr. for CoLA; Pearson-Spearman Corr. for STS-B; Accuracy for all others). T is the number of target tasks for dialect adaptation. [find](https://scholar.google.com/scholar?q=Table+A1%3A+Dialect+Adaptation+GLUE+results+of+BERT+Base+%28Devlin+et+al%2E%2C+2019%29+for+the+7+GLUE+Tasks+%28Matthew%E2%80%99s+Corr%2E+for+CoLA%3B+Pearson%2DSpearman+Corr%2E+for+STS%2DB%3B+Accuracy+for+all+others%29%2E+T+is+the+number)
57. ACL 2023 Responsible NLP Checklist [find](https://scholar.google.com/scholar?q=ACL+2023+Responsible+NLP+Checklist)

*Updated 1 November 2026*