---
title: "Automatic Dialect Detection in Arabic Broadcast Speech (2015)"
description: "The authors compare three utterance representations: sparse count vectors of phone… 100% accuracy. 24 references, cited by 48 papers here."
authors: ["Ahmed Ali", "Najim Dehak", "Patrick Cardinal", "Sameer Khurana", "Sree Harsha Yella", "James Glass", "Peter Bell", "Steve Renals"]
year: 2015
venue: "arXiv:1509.06928"
source: "https://arxiv.org/abs/1509.06928"
doi: "10.48550/arxiv.1509.06928"
arxiv: "1509.06928"
licence: "CC0 1.0 (arXiv)"
licence_url: "http://creativecommons.org/publicdomain/zero/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ali-2015-automatic-dialect-detection-arabic-broadcast/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ali-2015-automatic-dialect-detection-arabic-broadcast/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/3/) / Automatic Dialect Detection in Arabic Broadcast Speech

# Automatic Dialect Detection in Arabic Broadcast Speech

**Authors:** [Ahmed Ali](https://md.research.elamur.ai/free/en/authors/ahmed-m-abdelatty-ali/), [Najim Dehak](https://md.research.elamur.ai/free/en/authors/najim-dehak/), [Patrick Cardinal](https://md.research.elamur.ai/free/en/authors/patrick-cardinal/), [Sameer Khurana](https://md.research.elamur.ai/free/en/authors/sameer-khurana/), Sree Harsha Yella, [James Glass](https://md.research.elamur.ai/free/en/authors/james-glass/), [Peter Bell](https://md.research.elamur.ai/free/en/authors/peter-bell/) and [Steve Renals](https://md.research.elamur.ai/free/en/authors/steve-renals/) · **Year:** 2015 · **Venue:** [arXiv:1509.06928](https://md.research.elamur.ai/free/en/venues/interspeech-usb/) ([Interspeech 2015](https://md.research.elamur.ai/free/en/venues/interspeech-usb/2015/)) · **Licence:** [CC0 1.0 (arXiv)](http://creativecommons.org/publicdomain/zero/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1509.06928) · [PDF](https://arxiv.org/pdf/1509.06928) · [DOI 10.48550/arxiv.1509.06928](https://doi.org/10.48550/arxiv.1509.06928) · [arXiv 1509.06928](https://arxiv.org/abs/1509.06928) · [OpenAlex](https://openalex.org/W2248508985)

**Citations:** 11 citations (OpenAlex, 2026-10-14) · [48](https://md.research.elamur.ai/free/en/papers/ali-2015-automatic-dialect-detection-arabic-broadcast/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles automatic identification of Arabic dialects from broadcast speech. It treats this as a harder variant of language identification, since the varieties belong to one language yet differ in lexicon and sound. The authors compare three utterance representations:
- sparse count vectors of phone n-grams ("senones", n ≤ 4) taken from a phoneme recognizer;
- word and word-trigram count vectors taken from an ASR system;
- i-vectors built on bottleneck (BN) features from a two-stage DNN.

They test generative and discriminative classifiers and fuse the feature types with a multi-class SVM. Training data is broadcast news in EGY, GLF, LAV, NOR and MSA. The test set comes from a different recording channel (Al Jazeera, via QATS) and was labeled through crowdsourcing. Both binary tasks, Arabic vs English and MSA vs dialect, reach 100% accuracy. The five-way task reaches about 59–60%: the abstract reports 59.2% and the conclusion reports 60.2% for the combined system. Many errors trace back to speakers switching between dialect and MSA. The data and the system are released publicly, which gives the field a reproducible benchmark.

## Key points
- Five target classes: Egyptian (EGY), Gulf (GLF), Levantine (LAV), North African (NOR) and MSA, plus English for the language-ID sanity check.
- On lexical vectors (55K vocabulary, no dimensionality reduction), a linear SVM did best with 45.2% accuracy. The n-gram language model reached 40.4%, Maximum Entropy about 40% and Naive Bayes 37.9%.
- Feature setup for each representation:
  - Senone vectors use 36 phonemes.
  - The word-trigram dictionary holds 580k entries.
  - The i-vector system uses a 2048-component UBM, 400-dimensional i-vectors and LDA plus WCCN.
  - The BN extractor was trained on 60 hours of Al-Jazeera MSA.
- Test labels were collected on CrowdFlower. Each file needed at least 3 judges, with collection stopping at 9 judges or at 75% agreement.
- Code and data are released at github.com/Qatar-Computing-Research-Institute/dialectID and on the QCRI portal.

## Interesting ideas
- Counts of recognizer output units (phone n-grams, words) are reweighted with tf.idf, so units that appear in nearly every utterance stop dominating the vector and each feature stays inspectable.
- The test data was deliberately recorded through a different channel than the training data, so a classifier cannot score well by memorizing recording quirks.
- Labeling used an adaptive stopping rule: start with a few judges and add more only until agreement crosses a threshold or a cap is reached.
- The classifier's confusions (dialect segments predicted as MSA) were compared with the confusions in human labels, which shows that some "errors" reflect genuine code-switching.
- A second network refines the first network's posteriors using a wider time window (31 frames), a cheap two-pass correction design.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): The same transparent n-gram-over-labels vectors could detect which UD treebank dialect or genre a text resembles before choosing a conversion path.
- **I2** (learn): Evaluation sets could be drawn from a different source or genre than training text, serving as a control against shortcuts tied to corpus style.
- **I3** (learn): Silver data could call extra witness parsers or a higher LLM teacher tier only for sentences where the first witnesses disagree, saving cost on easy cases.
- **I4** (learn): Error analysis could check whether a parser mistake falls where human annotators also disagree, separating real bugs from inherently ambiguous input.
- **I5** (tree): gparse could add a light second pass that revises first-pass arc and label scores using wider sentence context, logging each change so it stays explainable.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Johns Hopkins University](https://md.research.elamur.ai/free/en/universities/johns-hopkins-university/) — University, Baltimore, United States · 240 works · 96,487 citations
- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 211 works · 69,000 citations
- [Hamad bin Khalifa University](https://md.research.elamur.ai/free/en/universities/hamad-bin-khalifa-university/) — University, Doha, Qatar · 141 works · 14,175 citations
- [École de Technologie Supérieure](https://md.research.elamur.ai/free/en/universities/ecole-de-technologie-superieure/) — University, Montreal, Canada · 2 works · 60 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 452 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Ahmed Ali, Najim Dehak, Patrick Cardinal, Sameer Khurana, Sree Harsha Yella, James Glass, Peter Bell and Steve Renals. 2015. Automatic Dialect Detection in Arabic Broadcast Speech. arXiv:1509.06928.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ali2015automatic,
  title = {{Automatic Dialect Detection in Arabic Broadcast Speech}},
  author = {Ahmed Ali, Najim Dehak, Patrick Cardinal, Sameer Khurana, Sree Harsha Yella, James Glass, Peter Bell and Steve Renals},
  year = {2015},
  doi = {10.48550/arxiv.1509.06928},
  eprint = {1509.06928},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1509.06928}
}
```

- [← previous A Hierarchy with, of, and for Preposition Supersenses](https://md.research.elamur.ai/free/en/papers/schneider-2015-hierarchy-preposition-supersenses/)
- [next → Improved Arabic Dialect Classification with Social Media Data](https://md.research.elamur.ai/free/en/papers/huang-2015-improved-arabic-dialect-classification-social/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/3/)

## References

24 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. G. Liu, Y. Lei, and J. H. Hansen, “Dialect identification: Impact of differences between read versus spontaneous speech,” EUSIPCO-2010: European Signal Processing Conference, Aalborg, Denmark, 2010. [find](https://scholar.google.com/scholar?q=G%2E+Liu%2C+Y%2E+Lei%2C+and+J%2E+H%2E+Hansen%2C+%E2%80%9CDialect+identification%3A+Impact+of+differences+between+read+versus+spontaneous+speech%2C%E2%80%9D+EUSIPCO%2D2010%3A+European+Signal+Processing+Conference%2C+Aalborg%2C+Denmark%2C+2010%2E)
2. D. A. Reynolds, W. M. Campbell, W. Shen, and E. Singer, “Automatic language recognition via spectral and token based approaches,” in Springer Handbook of Speech Processing, J. Benesty, M. M. Sondhi, and Y. Huang, Eds. Springer, 2008. [↗](https://doi.org/10.1007/978-3-540-49127-9_41)
3. M. Zissman, “Comparison of four approaches to automatic language identification of telephone speech,” IEEE Transactions on Speech and Audio Processing, vol. 4, no. 1, pp. 31–44, 1996. [↗](https://doi.org/10.1109/tsa.1996.481450)
4. D. Martı́nez, L. Burget, L. Ferrer, and N. Scheffer, “ivector-based prosodic system for language identification,” in ICASSP, 2012, pp. 4861–4864. [find](https://scholar.google.com/scholar?q=D%2E+Mart%C4%B1%CC%81nez%2C+L%2E+Burget%2C+L%2E+Ferrer%2C+and+N%2E+Scheffer%2C+%E2%80%9Civector%2Dbased+prosodic+system+for+language+identification%2C%E2%80%9D+in+ICASSP%2C+2012%2C+pp%2E+4861%E2%80%934864%2E)
5. O. Plchot, M. Diez, M. Soufifar, and L. Burget, “Pllr features in language recognition system for rats,” in Fifteenth Annual Conference of the International Speech Communication Association, 2014. [find](https://scholar.google.com/scholar?q=O%2E+Plchot%2C+M%2E+Diez%2C+M%2E+Soufifar%2C+and+L%2E+Burget%2C+%E2%80%9CPllr+features+in+language+recognition+system+for+rats%2C%E2%80%9D+in+Fifteenth+Annual+Conference+of+the+International+Speech+Communication+Association%2C+2014%2E)
6. M. H. Bahari, N. Dehak, L. Burget, A. Ali, J. Glass et al., “Non-negative factor analysis for gmm weight adaptation,” IEEE Transactions on Audio Speech and Language Processing, 2014. [find](https://scholar.google.com/scholar?q=M%2E+H%2E+Bahari%2C+N%2E+Dehak%2C+L%2E+Burget%2C+A%2E+Ali%2C+J%2E+Glass+et+al%2E%2C+%E2%80%9CNon%2Dnegative+factor+analysis+for+gmm+weight+adaptation%2C%E2%80%9D+IEEE+Transactions+on+Audio+Speech+and+Language+Processing%2C+2014%2E)
7. H. Soltau, L. Mangu, and F. Biadsy, “From modern standard arabic to levantine asr: Leveraging gale for dialects,” in ASRU, 2011, pp. 266–271. [find](https://scholar.google.com/scholar?q=H%2E+Soltau%2C+L%2E+Mangu%2C+and+F%2E+Biadsy%2C+%E2%80%9CFrom+modern+standard+arabic+to+levantine+asr%3A+Leveraging+gale+for+dialects%2C%E2%80%9D+in+ASRU%2C+2011%2C+pp%2E+266%E2%80%93271%2E)
8. M. Soufifar, S. Cumani, L. Burget, and J. Černocky, “Discriminative classifiers for phonotactic language recognition with ivectors,” in ICASSP, 2012, pp. 4853–4856. [find](https://scholar.google.com/scholar?q=M%2E+Soufifar%2C+S%2E+Cumani%2C+L%2E+Burget%2C+and+J%2E+C%CC%8Cernocky%2C+%E2%80%9CDiscriminative+classifiers+for+phonotactic+language+recognition+with+ivectors%2C%E2%80%9D+in+ICASSP%2C+2012%2C+pp%2E+4853%E2%80%934856%2E)
9. A. Ali, Y. Zhang, P. Cardinal, N. Dahak, S. Vogel, and J. Glass, “A complete kaldi recipe for building arabic speech recognition systems,” in SLT, 2014. [find](https://scholar.google.com/scholar?q=A%2E+Ali%2C+Y%2E+Zhang%2C+P%2E+Cardinal%2C+N%2E+Dahak%2C+S%2E+Vogel%2C+and+J%2E+Glass%2C+%E2%80%9CA+complete+kaldi+recipe+for+building+arabic+speech+recognition+systems%2C%E2%80%9D+in+SLT%2C+2014%2E)
10. G. Salton, a. Wong, and C. S. Yang, “A Vector Space Model for Automatic Indexing,” Magazine Communications of the ACM, vol. 18, no. 11, pp. 613–620, 1975. [↗](https://doi.org/10.1145/361219.361220)
11. W. Lowe, S. Mcdonald, W. Lowe, and S. Mcdonald, “Division of Informatics , University of Edinburgh Institute for Adaptive and Neural Computation The Direct Route : Mediated Priming in Semantic Space by The Direct Route : Mediated Priming in Semantic Space,” no. April, 2000. [find](https://scholar.google.com/scholar?q=W%2E+Lowe%2C+S%2E+Mcdonald%2C+W%2E+Lowe%2C+and+S%2E+Mcdonald%2C+%E2%80%9CDivision+of+Informatics+%2C+University+of+Edinburgh+Institute+for+Adaptive+and+Neural+Computation+The+Direct+Route+%3A+Mediated+Priming+in+Semantic+Space+b)
12. S. Padó and M. Lapata, “[Dependency-Based Construction of Semantic Space Models](https://md.research.elamur.ai/free/en/papers/pado-2007-dependency-based-construction-semantic-space/),” Computational Linguistics, vol. 33, no. December 2004, pp. 161–199, 2007. [↗](https://doi.org/10.1162/coli.2007.33.2.161)
13. N. Dehak, P. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, “Front-end factor analysis for speaker verification,” Audio, Speech, and Language Processing, IEEE Transactions on, pp. 788–798, 2011. [↗](https://doi.org/10.1109/tasl.2010.2064307)
14. N. Dehak, P. A. Torres-Carrasquillo, D. A. Reynolds, and R. Dehak, “Language recognition via i-vectors and dimensionality reduction,” in INTERSPEECH, 2011, pp. 857–860. [find](https://scholar.google.com/scholar?q=N%2E+Dehak%2C+P%2E+A%2E+Torres%2DCarrasquillo%2C+D%2E+A%2E+Reynolds%2C+and+R%2E+Dehak%2C+%E2%80%9CLanguage+recognition+via+i%2Dvectors+and+dimensionality+reduction%2C%E2%80%9D+in+INTERSPEECH%2C+2011%2C+pp%2E+857%E2%80%93860%2E)
15. P. Kenny, P. Ouellet, N. Dehak, V. Gupta, and P. Dumouchel, “A study of interspeaker variability in speaker verification,” Audio, Speech, and Language Processing, IEEE Transactions on, pp. 980–988, 2008. [↗](https://doi.org/10.1109/tasl.2008.925147)
16. A. Ali, Y. Zhang, and S. Vogel, “QCRI advanced transcription ssystem (QATS),” in SLT, 2014. [find](https://scholar.google.com/scholar?q=A%2E+Ali%2C+Y%2E+Zhang%2C+and+S%2E+Vogel%2C+%E2%80%9CQCRI+advanced+transcription+ssystem+%28QATS%29%2C%E2%80%9D+in+SLT%2C+2014%2E)
17. S. Wray and A. Ali, “Crowdsource a little to label a lot: Labeling a speech corpus of dialectal arabic,” in INTERSPEECH, 2015. [find](https://scholar.google.com/scholar?q=S%2E+Wray+and+A%2E+Ali%2C+%E2%80%9CCrowdsource+a+little+to+label+a+lot%3A+Labeling+a+speech+corpus+of+dialectal+arabic%2C%E2%80%9D+in+INTERSPEECH%2C+2015%2E)
18. H. Drucker, D. Wu, and V. N. Vapnik, “Support vector machines for spam categorization,” Neural Networks, IEEE Transactions on, vol. 10, no. 5, pp. 1048–1054, 1999. [↗](https://doi.org/10.1109/72.788645)
19. S. Meignier and T. Merlin, “Lium spkdiarization: an open source toolkit for diarization,” in CMU SPUD Workshop, 2010. [find](https://scholar.google.com/scholar?q=S%2E+Meignier+and+T%2E+Merlin%2C+%E2%80%9CLium+spkdiarization%3A+an+open+source+toolkit+for+diarization%2C%E2%80%9D+in+CMU+SPUD+Workshop%2C+2010%2E)
20. C.-h. Wu, H.-p. Shen, and C.-s. Hsu, “Code-Switching Event Detection by Using a Latent Language Space Model and the Delta-Bayesian,” vol. 23, no. 11, pp. 1892–1903, 2015. [find](https://scholar.google.com/scholar?q=C%2E%2Dh%2E+Wu%2C+H%2E%2Dp%2E+Shen%2C+and+C%2E%2Ds%2E+Hsu%2C+%E2%80%9CCode%2DSwitching+Event+Detection+by+Using+a+Latent+Language+Space+Model+and+the+Delta%2DBayesian%2C%E2%80%9D+vol%2E+23%2C+no%2E+11%2C+pp%2E+1892%E2%80%931903%2C+2015%2E)
21. P. Cardinal, N. Dehak, Y. Zhang, and J. Glass, “Speaker Adaptation Using the I-Vector Technique for Bottleneck Features,” in INTERSPEECH, 2015. [find](https://scholar.google.com/scholar?q=P%2E+Cardinal%2C+N%2E+Dehak%2C+Y%2E+Zhang%2C+and+J%2E+Glass%2C+%E2%80%9CSpeaker+Adaptation+Using+the+I%2DVector+Technique+for+Bottleneck+Features%2C%E2%80%9D+in+INTERSPEECH%2C+2015%2E)
22. F. Richardson, D. Reynolds, and N. Dehak, “A Unified Deep Neural Network for Speaker and Language Recognition,” in INTER-SPEECH, 2015. [find](https://scholar.google.com/scholar?q=F%2E+Richardson%2C+D%2E+Reynolds%2C+and+N%2E+Dehak%2C+%E2%80%9CA+Unified+Deep+Neural+Network+for+Speaker+and+Language+Recognition%2C%E2%80%9D+in+INTER%2DSPEECH%2C+2015%2E)
23. Y. Song, B. Jiang, Y. Bao, S. Wei, and L. R. Dai, “I-vector representation based on bottleneck features for language identification,” IEEE Electronics Letters, 2013. [↗](https://doi.org/10.1049/el.2013.1721)
24. P. Matejka, L. Zhang, T. Ng, H. S. Mallidi, O. Glembek, J. Ma, and B. Zhang, “Neural network bottleneck features for language identification,” in Proc. of Odyssey, 2014. This figure "DialectDetectionSpace.png" is available in "png" format from: [http://arxiv.org/ps/1509.06928v2](http://arxiv.org/ps/1509.06928v2) This figure "bnf.png" is available in "png" format from: http://arxiv.org/ps/1509.06928v2 [↗](http://arxiv.org/ps/1509.06928v2)

*Updated 1 November 2026*