---
title: "ArFake: A Robust Framework for Multi-Dialect — Elsetohy 2025"
description: "Neural text-to-speech systems now produce voices that are hard to tell apart from real ones."
authors: ["Mohamed Elsetohy", "Alhassan Ehab", "Ali Mekky", "Besher Hassan", "Shady Shehata"]
year: 2025
venue: "arXiv:2509.22808"
source: "https://arxiv.org/abs/2509.22808"
doi: "10.48550/arxiv.2509.22808"
arxiv: "2509.22808"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/elsetohy-2025-arfake-robust-framework-multi-dialect/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/elsetohy-2025-arfake-robust-framework-multi-dialect/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/) / ArFake: A Robust Framework for Multi-Dialect Arabic Speech Spoofing Detection Benchmark

# ArFake: A Robust Framework for Multi-Dialect Arabic Speech Spoofing Detection Benchmark

**Authors:** Mohamed Elsetohy, Alhassan Ehab, [Ali Mekky](https://md.research.elamur.ai/free/en/authors/ali-mekky/), [Besher Hassan](https://md.research.elamur.ai/free/en/authors/besher-hassan/), [Shady Shehata](https://md.research.elamur.ai/free/en/authors/shady-shehata/) · **Year:** 2025 · **Venue:** arXiv:2509.22808 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2509.22808) · [PDF](https://arxiv.org/pdf/2509.22808) · [DOI 10.48550/arxiv.2509.22808](https://doi.org/10.48550/arxiv.2509.22808) · [arXiv 2509.22808](https://arxiv.org/abs/2509.22808) · [OpenAlex](https://openalex.org/W4415331443)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Neural text-to-speech systems now produce voices that are hard to tell apart from real ones. Anti-spoofing benchmarks such as ASVspoof mostly cover English, so Arabic dialects have had almost no dedicated test resources. ArFake addresses this with a pipeline for both creating and detecting fake Arabic speech across eight dialect regions.

The authors take real utterances from the Casablanca corpus and resynthesize them with four TTS/voice-cloning systems. They judge the fakes three ways: how easily classifiers separate them from real audio, Whisper ASR word error rate, and human MOS ratings. From the bona fide audio and three of the generators they build a mixed corpus. On it they train detectors of two kinds: small heads on frozen speech-model embeddings, and classical classifiers on MFCC features.

Robustness is tested in three settings: in-domain, against a generator never seen in training, and against a dialect never seen in training. The reported in-domain and unseen-generator results reach about 96% and 97%. The released dataset and protocols form a first reference point for securing Arabic speech against deepfakes.

## Key points
- Source data is Casablanca: eight Arabic dialect regions, roughly six hours each. The spoofs come from XTTS-v2, FishSpeech, ArTST and VITS, and each generator is treated as a separate attack family.
- The final corpus has 54,413 utterances, with 31,302 for train+validation and 23,111 for test. Training is about 30% bona fide and 70% spoofed, with per-generator sampling rates that differ. VITS is fully held out for the cross-generator (LOGO) test.
- When a detector is trained per generator, FishSpeech fakes are the hardest to catch. Whisper-large reaches 6.92% EER on them, against 14.92% for the RawNet2 baseline. HuBERT-base gets 0.07% EER on XTTS-v2. ArTST and VITS fakes are separated almost perfectly, even by MFCC-based SVMs.
- Detector heads are two feed-forward layers with ReLU and dropout 0.5, trained with Adam at lr 1e-3. They sit on top of HuBERT-base, wav2vec2, Whisper-small or Whisper-large embeddings.
- The human evaluation used 12 native Arabic speakers rating realism on a 1–5 scale. Each generator contributed eight clips, one per dialect.

## Interesting ideas
- The paper uses how easily a classifier tells synthetic output from genuine data as a measure of distance from the real distribution: the easier the separation, the worse the generator. — For Mova (back): Train a simple discriminator between human-written sentences and sentences regenerated from the meaning graph, and track its accuracy as a naturalness score for generation.
- Under Leave-One-Generator-Out, an entire source of data is withheld from training and used only at test time, which mimics meeting a brand-new system in deployment. — For Mova (learn): Drop one witness parser or one LLM teacher tier when building silver data, then check whether the trained parser still holds up on that source's style of output.
- Under Leave-One-Dialect-Out, training covers all varieties but one, and testing uses the excluded one, so robustness to linguistic shift is measured directly. — For Mova (tree): Train gparse with one UD treebank or genre held out, to see how well the parser and the UD dialect conversion transfer to unseen text varieties.
- The authors note that near-perfect scores on weak generators probably come from obvious artifacts rather than real detection skill, so suspiciously high numbers deserve suspicion. — For Mova (learn): Treat sudden perfect scores as a signal to run negative controls, checking for shortcut cues before trusting the result.
- ASR word error rate against the original transcript is used as a diagnostic and gating signal for generated audio, not as a measure of realism. — For Mova (back): In the text → graph → text compressor, reparse each regenerated sentence and compare its graph to the source graph, rejecting outputs that drift.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{elsetohy2025arfake,
  title = {{ArFake: A Robust Framework for Multi-Dialect Arabic Speech Spoofing Detection Benchmark}},
  author = {Elsetohy, Mohamed and Ehab, Alhassan and Mekky, Ali and Hassan, Besher and Shehata, Shady},
  year = {2025},
  doi = {10.48550/arxiv.2509.22808},
  eprint = {2509.22808},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2509.22808}
}
```

- [← previous Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models](https://md.research.elamur.ai/free/en/papers/askari-2025-are-babylms-deaf-gricean-maxims/)
- [next → ArVoice: A Multi-Speaker Dataset for Arabic Speech Synthesis](https://md.research.elamur.ai/free/en/papers/toyin-2025-arvoice-multi-speaker-dataset-arabic/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/)

## References

21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. B. Talafha, K. Kadaoui, S. M. Magdy, M. Habiboullah, C. M. Chafei, A. O. El-Shangiti, H. Zayed, R. Alhamouri, R. Assi, A. Alraeesi et al., “Casablanca: Data and models for multidialectal arabic speech recognition,” arXiv preprint arXiv:2410.04527, 2024. [↗](https://arxiv.org/abs/2410.04527)
2. A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” 2016. [Online]. Available: https://arxiv.org/abs/1609.03499 [↗](https://arxiv.org/abs/1609.03499) [our card](https://md.research.elamur.ai/free/en/papers/oord-2016-wavenet-generative-model-raw-audio/)
3. Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. Le, Y. Agiomyrgiannakis, R. Clark, and R. A. Saurous, “Tacotron: Towards end-to-end speech synthesis,” 2017. [Online]. Available: https://arxiv.org/abs/1703.10135 [↗](https://arxiv.org/abs/1703.10135)
4. C. Wang, S. Chen, Y. Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y. Liu, H. Wang, J. Li, L. He, S. Zhao, and F. Wei, “Neural codec language models are zero-shot text to speech synthesizers,” 2023. [Online]. Available: https://arxiv.org/abs/2301.02111 [↗](https://arxiv.org/abs/2301.02111) [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-neural-codec-language-models-zero/)
5. M. Le, A. Vyas, B. Shi, B. Karrer, L. Sari, R. Moritz, M. Williamson, V. Manohar, Y. Adi, J. Mahadeokar, and W.-N. Hsu, “Voicebox: Text-guided multilingual universal speech generation at scale,” 2023. [Online]. Available: https://arxiv.org/abs/2306.15687 [↗](https://arxiv.org/abs/2306.15687)
6. E. Casanova, K. Davis, E. Gölge, G. Göknar, I. Gulea, L. Hart, A. Aljafari, J. Meyer, R. Morais, S. Olayemi, and J. Weber, “Xtts: a massively multilingual zero-shot text-to-speech model,” 2024. [Online]. Available: https://arxiv.org/abs/2406.04904 [↗](https://arxiv.org/abs/2406.04904)
7. S. Liao, Y. Wang, T. Li, Y. Cheng, R. Zhang, R. Zhou, and Y. Xing, “Fish-speech: Leveraging large language models for advanced multilingual text-to-speech synthesis,” 2024. [Online]. Available: https://arxiv.org/abs/2411.01156 [↗](https://arxiv.org/abs/2411.01156)
8. X. Wang, H. Delgado, H. Tak, J. weon Jung, H. jin Shim, M. Todisco, I. Kukanov, X. Liu, M. Sahidullah, T. Kinnunen, N. Evans, K. A. Lee, and J. Yamagishi, “Asvspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale,” 2024. [Online]. Available: https://arxiv.org/abs/2408.08739 [↗](https://arxiv.org/abs/2408.08739)
9. J. Yamagishi, X. Wang, M. Todisco, M. Sahidullah, J. Patino, A. Nautsch, X. Liu, K. A. Lee, T. Kinnunen, N. Evans, and H. Delgado, “Asvspoof 2021: accelerating progress in spoofed and deepfake speech detection,” 2021. [Online]. Available: https://arxiv.org/abs/2109.00537 [↗](https://arxiv.org/abs/2109.00537)
10. A. Nautsch, X. Wang, N. Evans, T. H. Kinnunen, V. Vestman, M. Todisco, H. Delgado, M. Sahidullah, J. Yamagishi, and K. A. Lee, “Asvspoof 2019: Spoofing countermeasures for the detection of synthesized, converted and replayed speech,” IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 3, no. 2, p. 252–265, Apr. 2021. [Online]. Available: http://dx.doi.org/10.1109/TBIOM.2021.3059479 [↗](https://doi.org/10.1109/tbiom.2021.3059479)
11. N. M. Müller, P. Czempin, F. Dieckmann, A. Froghyar, and K. Böttinger, “Does audio deepfake detection generalize?” 2024. [Online]. Available: https://arxiv.org/abs/2203.16263 [↗](https://arxiv.org/abs/2203.16263)
12. X. Liu, X. Wang, M. Sahidullah, J. Patino, H. Delgado, T. Kinnunen, M. Todisco, J. Yamagishi, N. Evans, A. Nautsch, and K. A. Lee, “Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, p. 2507–2522, 2023. [Online]. Available: http://dx.doi.org/10.1109/TASLP.2023.3285283 [↗](https://doi.org/10.1109/taslp.2023.3285283)
13. L. Zhang, X. Wang, E. Cooper, N. Evans, and J. Yamagishi, “The partialspoof database and countermeasures for the detection of short fake speech segments embedded in an utterance,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, p. 813–825, 2023. [Online]. Available: http://dx.doi.org/10.1109/TASLP.2022.3233236 [↗](https://doi.org/10.1109/taslp.2022.3233236)
14. Z. M. Almutairi and H. Elgibreen, “Detecting fake audio of arabic speakers using self-supervised deep learning,” IEEE Access, vol. 11, pp. 72 134–72 147, 2023. [find](https://scholar.google.com/scholar?q=Z%2E+M%2E+Almutairi+and+H%2E+Elgibreen%2C+%E2%80%9CDetecting+fake+audio+of+arabic+speakers+using+self%2Dsupervised+deep+learning%2C%E2%80%9D+IEEE+Access%2C+vol%2E+11%2C+pp%2E+72+134%E2%80%9372+147%2C+2023%2E)
15. R. Awaad, A. Hedya, M. Fadel, and A. B. Zaky, “Real-time deepfake detection for ai-generated arabic speech,” in 2024 6th Novel Intelligent and Leading Emerging Sciences Conference (NILES), 2024, pp. 198–201. [find](https://scholar.google.com/scholar?q=R%2E+Awaad%2C+A%2E+Hedya%2C+M%2E+Fadel%2C+and+A%2E+B%2E+Zaky%2C+%E2%80%9CReal%2Dtime+deepfake+detection+for+ai%2Dgenerated+arabic+speech%2C%E2%80%9D+in+2024+6th+Novel+Intelligent+and+Leading+Emerging+Sciences+Conference+%28NILES%29%2C+2024%2C+pp%2E+1)
16. N. M. Müller, P. Kawa, W. H. Choong, E. Casanova, E. Gölge, T. Müller, P. Syga, P. Sperl, and K. Böttinger, “Mlaad: The multi-language audio anti-spoofing dataset,” 2025. [Online]. Available: https://arxiv.org/abs/2401.09512 [↗](https://arxiv.org/abs/2401.09512)
17. K. Kuckreja, P. Gupta, I. Hamed, T. Solorio, M. H. Khan, and A. Dhall, “Tell me habibi, is it real or fake?” 2025. [Online]. Available: https://arxiv.org/abs/2505.22581 [↗](https://arxiv.org/abs/2505.22581)
18. A. Khan, K. M. Malik, and S. Nawaz, “Frame-to-utterance convergence: A spectra-temporal approach for unified spoofing detection,” in ICASSP 2024 - IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 10 761–10 765. [Online]. Available: https://arxiv.org/pdf/2309.09837 [↗](https://arxiv.org/abs/2309.09837)
19. H. Toyin, A. Djanibekov, A. Kulkarni, and H. Aldarmaki, “Artst: Arabic text and speech transformer,” in Proceedings of Arabic-NLP 2023, 2023, pp. 41–51. [find](https://scholar.google.com/scholar?q=H%2E+Toyin%2C+A%2E+Djanibekov%2C+A%2E+Kulkarni%2C+and+H%2E+Aldarmaki%2C+%E2%80%9CArtst%3A+Arabic+text+and+speech+transformer%2C%E2%80%9D+in+Proceedings+of+Arabic%2DNLP+2023%2C+2023%2C+pp%2E+41%E2%80%9351%2E)
20. J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” 2021. [Online]. Available: https://arxiv.org/abs/2106.06103 [↗](https://arxiv.org/abs/2106.06103)
21. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” 2022. [Online]. Available: https://arxiv.org/abs/2212.04356 [↗](https://arxiv.org/abs/2212.04356) [our card](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/)