---
title: "A Parameter-Efficient Learning Approach — Radhakrishnan 2023"
description: "The paper addresses spoken Arabic dialect identification (ADI)."
authors: ["Srijith Radhakrishnan", "Chao-Han Huck Yang", "Sumeer Ahmad Khan", "Narsis A. Kiani", "David Gomez-Cabrero", "Jesper N. Tegner"]
year: 2023
venue: "arXiv:2305.11244"
source: "https://arxiv.org/abs/2305.11244"
doi: "10.21437/interspeech.2023-1407"
arxiv: "2305.11244"
licence: "CC BY-SA 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/radhakrishnan-2023-parameter-efficient-learning-approach-arabic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/radhakrishnan-2023-parameter-efficient-learning-approach-arabic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/) / A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model

# A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model

**Authors:** Srijith Radhakrishnan, Chao-Han Huck Yang, Sumeer Ahmad Khan, Narsis A. Kiani, David Gomez-Cabrero, Jesper N. Tegner · **Year:** 2023 · **Venue:** arXiv:2305.11244 · **Licence:** [CC BY-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-sa/4.0/)

**Links:** [Publisher's page](https://doi.org/10.21437/interspeech.2023-1407) · [PDF](https://arxiv.org/pdf/2305.11244) · [DOI 10.21437/interspeech.2023-1407](https://doi.org/10.21437/interspeech.2023-1407) · [arXiv 2305.11244](https://arxiv.org/abs/2305.11244) · [OpenAlex](https://openalex.org/W4385823097)

**Citations:** 13 citations (openalex-snapshot · 2026-10-14) · [4](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper addresses spoken Arabic dialect identification (ADI). This task is harder than ordinary language identification because dialects of one language differ only in fine details of pronunciation and accent. It also lacks large pre-trained models built specifically for it. The authors take Whisper Base, a general-purpose multilingual speech recognizer, and adapt it to classify 17 dialects from the ADI-17 corpus.

They compare two families of approaches:
- **Conventional fine-tuning** of the whole network or of its parts, with new dialect tokens added to the tokenizer.
- **Parameter-efficient learning**, where Whisper stays frozen and only small extras are trained: BitFit, a trainable additive input signal (input reprogramming), and residual bottleneck adapters in the encoder.

A random many-to-one mapping from Whisper's existing language tokens to dialects lets the frozen decoder produce class scores. Full fine-tuning on under a third of the training data sets a new state of the art on the official dev and test sets. The best adapter setup comes within 1.86% accuracy of it while training only about 2.5% extra parameters. This shows that dialect identification is achievable with little data and modest compute.

## Key points
- **Data:** ADI-17 (MGB5 challenge) contains YouTube audio for 17 Arabic dialects, with 3033 training hours and 58 hours for dev and test. It is heavily skewed: IRQ has 815.8 h, while JOR has only 25.9 h.
- **Experimental setup:** Only utterances longer than 20 s are used. Training subsets range from 0.5k to 10k samples per class, and the largest subset (53.1 h per class) equals 30.95% of ADI-17.
- **Trainable parameter counts:**
  - Full fine-tuning: 71.8M.
  - Encoder-only fine-tuning: 18.9M.
  - BitFit: about 32K–76K.
  - Input reprogramming: 240K (0.33%).
  - Adapters-64, -128 and -256: 642K, 1M and 1.8M (2.50%).
- **Baselines:** Prior systems use 13.1M (the ADI-17 CNN), 64.3M (CNN+Transformer) and over 50M (supervised clustering) parameters.
- **Frozen baseline:** Whisper with only the label mapping scores just 2.52–4.44%, so some adaptation is clearly necessary.

## Interesting ideas
- Keeping a large pre-trained network frozen and inserting small down-project/GELU/up-project adapters with a residual path recovers nearly all of full fine-tuning accuracy at a tiny parameter cost. — For Mova (tree): gparse could get a separate small adapter per UD treebank or dialect conversion, so new genres are added without retraining or forking the whole transformer parser.
- Instead of a new output head, each target class borrows several existing output tokens chosen at random, and their logits are summed before softmax; notably, a learned mapping performed worse than this fixed one. — For Mova (tokens): when the tagger needs a new coarse label set, scores of existing fine tags could be pooled into the new classes through a fixed, inspectable mapping, which keeps the decision explainable.
- Reporting the frozen model with only the label mapping as a reference row shows how much of the score the adaptation itself contributes. — For Mova (learn): every trained component could be evaluated next to its untouched-backbone variant as a negative control, making clear whether training, rather than the pre-trained model, explains the gains.
- Accuracy is measured across a ladder of training sizes (0.5k to 10k per class) for each adaptation method, comparing data efficiency rather than one final number. — For Mova (learn): silver-data experiments could report learning curves per method, revealing when cheap adapters suffice and when more agreed data is needed.
- Class imbalance is handled by repeatedly sampling random fixed-length windows from long clips of rare dialects. — For Mova (learn): rare constructions in silver corpora could be oversampled by drawing varied spans or sentences around them, instead of duplicating identical examples.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cited by

Cited by

4 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 3 not yet.

- [Jenin at AraGenEval Shared Task: Parameter-Efficient Fine-Tuning and Layer-Wise Analysis of Arabic LLMs for Authorship Style Transfer and Classification](https://md.research.elamur.ai/free/en/papers/malhis-2025-jenin-arageneval-shared-task-parameter/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{radhakrishnan2023parameter,
  title = {{A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model}},
  author = {Radhakrishnan, Srijith and Yang, Chao-Han Huck and Khan, Sumeer Ahmad and Kiani, Narsis A. and Gomez-Cabrero, David and Tegner, Jesper N.},
  year = {2023},
  doi = {10.21437/interspeech.2023-1407},
  eprint = {2305.11244},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.21437/interspeech.2023-1407}
}
```

- [← previous A New Direction in Stance Detection: Target-Stance Extraction in the Wild](https://md.research.elamur.ai/free/en/papers/li-2023-new-direction-stance-detection-target/)
- [next → A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://md.research.elamur.ai/free/en/papers/white-2023-prompt-pattern-catalog-enhance-prompt/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/)

## References

35 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. A. Zissman, T. P. Gleason, D. M. Rekart, and B. L. Losiewicz, “Automatic dialect identification of extemporaneous conversational, latin american spanish speech,” in Proc. of ICASSP, vol. 2. IEEE, 1996, pp. 777–780. [find](https://scholar.google.com/scholar?q=M%2E+A%2E+Zissman%2C+T%2E+P%2E+Gleason%2C+D%2E+M%2E+Rekart%2C+and+B%2E+L%2E+Losiewicz%2C+%E2%80%9CAutomatic+dialect+identification+of+extemporaneous+conversational%2C+latin+american+spanish+speech%2C%E2%80%9D+in+Proc%2E+of+ICASSP%2C+vol%2E+2%2E+IEEE%2C+1)
2. E. Ambikairajah, H. Li, L. Wang, B. Yin, and V. Sethu, “Language identification: A tutorial,” IEEE Circuits and Systems Magazine, vol. 11, no. 2, pp. 82–108, 2011. [find](https://scholar.google.com/scholar?q=E%2E+Ambikairajah%2C+H%2E+Li%2C+L%2E+Wang%2C+B%2E+Yin%2C+and+V%2E+Sethu%2C+%E2%80%9CLanguage+identification%3A+A+tutorial%2C%E2%80%9D+IEEE+Circuits+and+Systems+Magazine%2C+vol%2E+11%2C+no%2E+2%2C+pp%2E+82%E2%80%93108%2C+2011%2E)
3. S. Watanabe, T. Hori, and J. R. Hershey, “Language independent end-to-end architecture for joint language identification and speech recognition,” in Proc. ASRU. IEEE, 2017, pp. 265–271. [find](https://scholar.google.com/scholar?q=S%2E+Watanabe%2C+T%2E+Hori%2C+and+J%2E+R%2E+Hershey%2C+%E2%80%9CLanguage+independent+end%2Dto%2Dend+architecture+for+joint+language+identification+and+speech+recognition%2C%E2%80%9D+in+Proc%2E+ASRU%2E+IEEE%2C+2017%2C+pp%2E+265%E2%80%93271%2E)
4. H. Al-Haj, R. Hsiao, I. Lane, A. W. Black, and A. Waibel, “Pronunciation modeling for dialectal arabic speech recognition,” in 2009 IEEE Workshop on Automatic Speech Recognition and Understanding, 2009, pp. 525–528. [find](https://scholar.google.com/scholar?q=H%2E+Al%2DHaj%2C+R%2E+Hsiao%2C+I%2E+Lane%2C+A%2E+W%2E+Black%2C+and+A%2E+Waibel%2C+%E2%80%9CPronunciation+modeling+for+dialectal+arabic+speech+recognition%2C%E2%80%9D+in+2009+IEEE+Workshop+on+Automatic+Speech+Recognition+and+Understanding%2C+200)
5. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” 2022. [find](https://scholar.google.com/scholar?q=A%2E+Radford%2C+J%2E+W%2E+Kim%2C+T%2E+Xu%2C+G%2E+Brockman%2C+C%2E+McLeavey%2C+and+I%2E+Sutskever%2C+%E2%80%9CRobust+speech+recognition+via+large%2Dscale+weak+supervision%2C%E2%80%9D+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/)
6. C.-H. H. Yang, B. Li, Y. Zhang, N. Chen, R. Prabhavalkar, T. N. Sainath, and T. Strohman, “From English to more languages: Parameter-efficient model reprogramming for cross-lingual speech recognition,” Proc. of ICASSP, 2023. [find](https://scholar.google.com/scholar?q=C%2E%2DH%2E+H%2E+Yang%2C+B%2E+Li%2C+Y%2E+Zhang%2C+N%2E+Chen%2C+R%2E+Prabhavalkar%2C+T%2E+N%2E+Sainath%2C+and+T%2E+Strohman%2C+%E2%80%9CFrom+English+to+more+languages%3A+Parameter%2Defficient+model+reprogramming+for+cross%2Dlingual+speech+recognition%2C)
7. J. He, C. Zhou, X. Ma, T. Berg-Kirkpatrick, and G. Neubig, “Towards a unified view of parameter-efficient transfer learning,” in International Conference on Learning Representations, 2022. [find](https://scholar.google.com/scholar?q=J%2E+He%2C+C%2E+Zhou%2C+X%2E+Ma%2C+T%2E+Berg%2DKirkpatrick%2C+and+G%2E+Neubig%2C+%E2%80%9CTowards+a+unified+view+of+parameter%2Defficient+transfer+learning%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/he-2021-towards-unified-view-parameter-efficient/)
8. Z.-C. Chen, Y.-S. Sung, and H.-y. Lee, “Chapter: Exploiting convolutional neural network adapters for self-supervised speech models,” in 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW). IEEE, 2023, pp. 1–5. [find](https://scholar.google.com/scholar?q=Z%2E%2DC%2E+Chen%2C+Y%2E%2DS%2E+Sung%2C+and+H%2E%2Dy%2E+Lee%2C+%E2%80%9CChapter%3A+Exploiting+convolutional+neural+network+adapters+for+self%2Dsupervised+speech+models%2C%E2%80%9D+in+2023+IEEE+International+Conference+on+Acoustics%2C+Speech%2C+and+Si)
9. C.-L. Fu, Z.-C. Chen, Y.-R. Lee, and H.-y. Lee, “Adapterbias: Parameter-efficient token-dependent representation shift for adapters in nlp tasks,” Proc. of NAACL Findings, 2022. [find](https://scholar.google.com/scholar?q=C%2E%2DL%2E+Fu%2C+Z%2E%2DC%2E+Chen%2C+Y%2E%2DR%2E+Lee%2C+and+H%2E%2Dy%2E+Lee%2C+%E2%80%9CAdapterbias%3A+Parameter%2Defficient+token%2Ddependent+representation+shift+for+adapters+in+nlp+tasks%2C%E2%80%9D+Proc%2E+of+NAACL+Findings%2C+2022%2E)
10. O. F. Zaidan and C. Callison-Burch, “Arabic dialect identification,” Computational Linguistics, vol. 40, no. 1, pp. 171–202, 2014. [find](https://scholar.google.com/scholar?q=O%2E+F%2E+Zaidan+and+C%2E+Callison%2DBurch%2C+%E2%80%9CArabic+dialect+identification%2C%E2%80%9D+Computational+Linguistics%2C+vol%2E+40%2C+no%2E+1%2C+pp%2E+171%E2%80%93202%2C+2014%2E) [our card](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/)
11. S. Shon, A. Ali, Y. Samih, H. Mubarak, and J. Glass, “Adi17: A fine-grained arabic dialect identification dataset,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 8244–8248. [find](https://scholar.google.com/scholar?q=S%2E+Shon%2C+A%2E+Ali%2C+Y%2E+Samih%2C+H%2E+Mubarak%2C+and+J%2E+Glass%2C+%E2%80%9CAdi17%3A+A+fine%2Dgrained+arabic+dialect+identification+dataset%2C%E2%80%9D+in+ICASSP+2020+%2D+2020+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+) [our card](https://md.research.elamur.ai/free/en/papers/salameh-2018-fine-grained-arabic-dialect-identification/)
12. C.-H. H. Yang, Y.-Y. Tsai, and P.-Y. Chen, “Voice2series: Reprogramming acoustic models for time series classification,” in International Conference on Machine Learning. PMLR, 2021, pp. 11 808–11 819. [find](https://scholar.google.com/scholar?q=C%2E%2DH%2E+H%2E+Yang%2C+Y%2E%2DY%2E+Tsai%2C+and+P%2E%2DY%2E+Chen%2C+%E2%80%9CVoice2series%3A+Reprogramming+acoustic+models+for+time+series+classification%2C%E2%80%9D+in+International+Conference+on+Machine+Learning%2E+PMLR%2C+2021%2C+pp%2E+11+808%E2%80%9311+819%2E)
13. I. A. El-khair, “1.5 billion words arabic corpus,” 2016. [find](https://scholar.google.com/scholar?q=I%2E+A%2E+El%2Dkhair%2C+%E2%80%9C1%2E5+billion+words+arabic+corpus%2C%E2%80%9D+2016%2E)
14. W. Zaghouani and A. Charfi, “Arap-tweet: A large multi-dialect Twitter corpus for gender, age and language variety identification,” in Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). Miyazaki, Japan: European Language Resources Association (ELRA), May 2018. [find](https://scholar.google.com/scholar?q=W%2E+Zaghouani+and+A%2E+Charfi%2C+%E2%80%9CArap%2Dtweet%3A+A+large+multi%2Ddialect+Twitter+corpus+for+gender%2C+age+and+language+variety+identification%2C%E2%80%9D+in+Proceedings+of+the+Eleventh+International+Conference+on+Language+)
15. A. Ahmed, N. Ali, M. Alzubaidi, W. Zaghouani, A. A. Abd-alrazaq, and M. Househ, “Freely available arabic corpora: A scoping review,” Computer Methods and Programs in Biomedicine Update, vol. 2, p. 100049, 2022. [find](https://scholar.google.com/scholar?q=A%2E+Ahmed%2C+N%2E+Ali%2C+M%2E+Alzubaidi%2C+W%2E+Zaghouani%2C+A%2E+A%2E+Abd%2Dalrazaq%2C+and+M%2E+Househ%2C+%E2%80%9CFreely+available+arabic+corpora%3A+A+scoping+review%2C%E2%80%9D+Computer+Methods+and+Programs+in+Biomedicine+Update%2C+vol%2E+2%2C+p%2E+100)
16. A. Ali, N. Dehak, P. Cardinal, S. Khurana, S. H. Yella, J. Glass, P. Bell, and S. Renals, “Automatic Dialect Detection in Arabic Broadcast Speech,” in Proc. Interspeech 2016, 2016, pp. 2934– 2938. [find](https://scholar.google.com/scholar?q=A%2E+Ali%2C+N%2E+Dehak%2C+P%2E+Cardinal%2C+S%2E+Khurana%2C+S%2E+H%2E+Yella%2C+J%2E+Glass%2C+P%2E+Bell%2C+and+S%2E+Renals%2C+%E2%80%9CAutomatic+Dialect+Detection+in+Arabic+Broadcast+Speech%2C%E2%80%9D+in+Proc%2E+Interspeech+2016%2C+2016%2C+pp%2E+2934%E2%80%93+2938%2E)
17. M. A. Hearst, S. T. Dumais, E. Osuna, J. Platt, and B. Scholkopf, “Support vector machines,” IEEE Intelligent Systems and their applications, vol. 13, no. 4, pp. 18–28, 1998. [find](https://scholar.google.com/scholar?q=M%2E+A%2E+Hearst%2C+S%2E+T%2E+Dumais%2C+E%2E+Osuna%2C+J%2E+Platt%2C+and+B%2E+Scholkopf%2C+%E2%80%9CSupport+vector+machines%2C%E2%80%9D+IEEE+Intelligent+Systems+and+their+applications%2C+vol%2E+13%2C+no%2E+4%2C+pp%2E+18%E2%80%9328%2C+1998%2E)
18. S. Shon, A. Ali, and J. Glass, “Mit-qcri arabic dialect identification system for the 2017 multi-genre broadcast challenge,” in 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2017, pp. 374–380. [find](https://scholar.google.com/scholar?q=S%2E+Shon%2C+A%2E+Ali%2C+and+J%2E+Glass%2C+%E2%80%9CMit%2Dqcri+arabic+dialect+identification+system+for+the+2017+multi%2Dgenre+broadcast+challenge%2C%E2%80%9D+in+2017+IEEE+Automatic+Speech+Recognition+and+Understanding+Workshop+%28ASRU%29)
19. N. Houlsby, A. Giurgiu, S. Jastrzebski et al., “Parameter-efficient transfer learning for NLP,” in Proc. of ICML, ser. Proceedings of Machine Learning Research, vol. 97. PMLR, 09–15 Jun 2019, pp. 2790–2799. [find](https://scholar.google.com/scholar?q=N%2E+Houlsby%2C+A%2E+Giurgiu%2C+S%2E+Jastrzebski+et+al%2E%2C+%E2%80%9CParameter%2Defficient+transfer+learning+for+NLP%2C%E2%80%9D+in+Proc%2E+of+ICML%2C+ser%2E+Proceedings+of+Machine+Learning+Research%2C+vol%2E+97%2E+PMLR%2C+09%E2%80%9315+Jun+2019%2C+pp%2E+2790)
20. S.-A. Rebuffi, H. Bilen, and A. Vedaldi, “Learning multiple visual domains with residual adapters,” in Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017. [find](https://scholar.google.com/scholar?q=S%2E%2DA%2E+Rebuffi%2C+H%2E+Bilen%2C+and+A%2E+Vedaldi%2C+%E2%80%9CLearning+multiple+visual+domains+with+residual+adapters%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+I%2E+Guyon%2C+U%2E+V%2E+Luxburg%2C+S%2E+Bengio%2C+H%2E+Wallach) [our card](https://md.research.elamur.ai/free/en/papers/rebuffi-2017-learning-multiple-visual-domains-residual/)
21. K. Tomanek, V. Zayats et al., “Residual adapters for parameter-efficient ASR adaptation to atypical and accented speech,” in Proc. of EMNLP. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 6751–6760. [find](https://scholar.google.com/scholar?q=K%2E+Tomanek%2C+V%2E+Zayats+et+al%2E%2C+%E2%80%9CResidual+adapters+for+parameter%2Defficient+ASR+adaptation+to+atypical+and+accented+speech%2C%E2%80%9D+in+Proc%2E+of+EMNLP%2E+Online+and+Punta+Cana%2C+Dominican+Republic%3A+Association+for+)
22. K.-W. Chang, W.-C. Tseng, S.-W. Li, and H.-y. Lee, “An exploration of prompt tuning on generative spoken language model for speech processing tasks,” Proc. of Interspeech, 2022. [find](https://scholar.google.com/scholar?q=K%2E%2DW%2E+Chang%2C+W%2E%2DC%2E+Tseng%2C+S%2E%2DW%2E+Li%2C+and+H%2E%2Dy%2E+Lee%2C+%E2%80%9CAn+exploration+of+prompt+tuning+on+generative+spoken+language+model+for+speech+processing+tasks%2C%E2%80%9D+Proc%2E+of+Interspeech%2C+2022%2E)
23. H. Yen, P.-J. Ku, C.-H. H. Yang, H. Hu, S. M. Siniscalchi, P.-Y. Chen, and Y. Tsao, “Neural model reprogramming with similarity based mapping for low-resource spoken command classification,” arXiv preprint arXiv:2110.03894, 2021. [↗](https://arxiv.org/abs/2110.03894)
24. G. F. Elsayed, I. Goodfellow, and J. Sohl-Dickstein, “Adversarial reprogramming of neural networks,” 2018. [find](https://scholar.google.com/scholar?q=G%2E+F%2E+Elsayed%2C+I%2E+Goodfellow%2C+and+J%2E+Sohl%2DDickstein%2C+%E2%80%9CAdversarial+reprogramming+of+neural+networks%2C%E2%80%9D+2018%2E)
25. E. Ben Zaken, Y. Goldberg, and S. Ravfogel, “BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Dublin, Ireland: Association for Computational Linguistics, May 2022. [find](https://scholar.google.com/scholar?q=E%2E+Ben+Zaken%2C+Y%2E+Goldberg%2C+and+S%2E+Ravfogel%2C+%E2%80%9CBitFit%3A+Simple+parameter%2Defficient+fine%2Dtuning+for+transformer%2Dbased+masked+language%2Dmodels%2C%E2%80%9D+in+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+) [our card](https://md.research.elamur.ai/free/en/papers/ben-2021-zaken-bitfit-simple-parameter-efficient-fine/)
26. E. J. Hu, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022. [find](https://scholar.google.com/scholar?q=E%2E+J%2E+Hu%2C+P%2E+Wallis%2C+Z%2E+Allen%2DZhu%2C+Y%2E+Li%2C+S%2E+Wang%2C+L%2E+Wang%2C+W%2E+Chen+et+al%2E%2C+%E2%80%9CLora%3A+Low%2Drank+adaptation+of+large+language+models%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2022%2E)
27. D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” arXiv preprint arXiv:1606.08415, 2016. [↗](https://arxiv.org/abs/1606.08415)
28. Z. Alhakeem, Y. Kwon, and H.-G. Kang, “Disentangled representations for arabic dialect identification based on supervised clustering with triplet loss,” in 2021 29th European Signal Processing Conference (EUSIPCO), 2021, pp. 526–530. [find](https://scholar.google.com/scholar?q=Z%2E+Alhakeem%2C+Y%2E+Kwon%2C+and+H%2E%2DG%2E+Kang%2C+%E2%80%9CDisentangled+representations+for+arabic+dialect+identification+based+on+supervised+clustering+with+triplet+loss%2C%E2%80%9D+in+2021+29th+European+Signal+Processing+Confere)
29. W. Lin, M. Madhavi, R. K. Das, and H. Li, “Transformer-based arabic dialect identification,” in 2020 International Conference on Asian Language Processing (IALP). IEEE, 2020, pp. 192–196. [find](https://scholar.google.com/scholar?q=W%2E+Lin%2C+M%2E+Madhavi%2C+R%2E+K%2E+Das%2C+and+H%2E+Li%2C+%E2%80%9CTransformer%2Dbased+arabic+dialect+identification%2C%E2%80%9D+in+2020+International+Conference+on+Asian+Language+Processing+%28IALP%29%2E+IEEE%2C+2020%2C+pp%2E+192%E2%80%93196%2E)
30. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. [find](https://scholar.google.com/scholar?q=A%2E+Vaswani%2C+N%2E+Shazeer%2C+N%2E+Parmar%2C+J%2E+Uszkoreit%2C+L%2E+Jones%2C+A%2E+N%2E+Gomez%2C+%C5%81%2E+Kaiser%2C+and+I%2E+Polosukhin%2C+%E2%80%9CAttention+is+all+you+need%2C%E2%80%9D+Advances+in+neural+information+processing+systems%2C+vol%2E+30%2C+2017%2E)
31. G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2261–2269. [find](https://scholar.google.com/scholar?q=G%2E+Huang%2C+Z%2E+Liu%2C+L%2E+Van+Der+Maaten%2C+and+K%2E+Q%2E+Weinberger%2C+%E2%80%9CDensely+connected+convolutional+networks%2C%E2%80%9D+in+2017+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2C+2017%2C+pp%2E+2261%E2%80%932269%2E)
32. R. C. Fong and A. Vedaldi, “Interpretable explanations of black boxes by meaningful perturbation,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 3429–3437. [find](https://scholar.google.com/scholar?q=R%2E+C%2E+Fong+and+A%2E+Vedaldi%2C+%E2%80%9CInterpretable+explanations+of+black+boxes+by+meaningful+perturbation%2C%E2%80%9D+in+Proceedings+of+the+IEEE+international+conference+on+computer+vision%2C+2017%2C+pp%2E+3429%E2%80%933437%2E)
33. R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 618– 626. [find](https://scholar.google.com/scholar?q=R%2E+R%2E+Selvaraju%2C+M%2E+Cogswell%2C+A%2E+Das%2C+R%2E+Vedantam%2C+D%2E+Parikh%2C+and+D%2E+Batra%2C+%E2%80%9CGrad%2Dcam%3A+Visual+explanations+from+deep+networks+via+gradient%2Dbased+localization%2C%E2%80%9D+in+2017+IEEE+International+Conference+on) [our card](https://md.research.elamur.ai/free/en/papers/selvaraju-2016-grad-cam-visual-explanations-deep/)
34. T. Reitmaier, E. Wallington, D. Kalarikalayil Raju et al., “Opportunities and challenges of automatic speech recognition systems for low-resource language speakers,” in Proc. of CHI, ser. CHI ’22. New York, NY, USA: ACM, 2022. [find](https://scholar.google.com/scholar?q=T%2E+Reitmaier%2C+E%2E+Wallington%2C+D%2E+Kalarikalayil+Raju+et+al%2E%2C+%E2%80%9COpportunities+and+challenges+of+automatic+speech+recognition+systems+for+low%2Dresource+language+speakers%2C%E2%80%9D+in+Proc%2E+of+CHI%2C+ser%2E+CHI+%E2%80%9922%2E+New)
35. B. Labrador, G. Zhao, I. L. Moreno, A. S. Scarpati, L. Fowl, and Q. Wang, “Exploring sequence-to-sequence transformer-transducer models for keyword spotting,” in Proc. of ICASSP. IEEE, 2023, pp. 1–5. [find](https://scholar.google.com/scholar?q=B%2E+Labrador%2C+G%2E+Zhao%2C+I%2E+L%2E+Moreno%2C+A%2E+S%2E+Scarpati%2C+L%2E+Fowl%2C+and+Q%2E+Wang%2C+%E2%80%9CExploring+sequence%2Dto%2Dsequence+transformer%2Dtransducer+models+for+keyword+spotting%2C%E2%80%9D+in+Proc%2E+of+ICASSP%2E+IEEE%2C+2023%2C+pp%2E+1%E2%80%935%2E)