---
title: "Classifying ASR Transcriptions According to Arabic Dialect (2016)"
description: "The paper reports one team's entries in the Arabic dialect track of the 2016 DSL… 0.4279 accuracy. 24 references, cited by 4 papers here."
authors: ["Abualsoud Hanani", "Aziz Qaroush", "Stephen Taylor"]
year: 2016
venue: "Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3)"
source: "https://aclanthology.org/W16-4817/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hanani-2016-classifying-asr-transcriptions-according-arabic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hanani-2016-classifying-asr-transcriptions-according-arabic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/5/) / Classifying ASR Transcriptions According to Arabic Dialect

# Classifying ASR Transcriptions According to Arabic Dialect

**Authors:** [Abualsoud Hanani](https://md.research.elamur.ai/free/en/authors/abualsoud-hanani/), [Aziz Qaroush](https://md.research.elamur.ai/free/en/authors/aziz-qaroush/) and [Stephen Taylor](https://md.research.elamur.ai/free/en/authors/stephen-taylor/) · **Year:** 2016 · **Venue:** [Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3)](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2016](https://md.research.elamur.ai/free/en/venues/coling/2016/)) · **Publisher:** [International Committee on Computational Linguistics (ICCL)](https://md.research.elamur.ai/free/en/publishers/international-committee-on-computational-linguistics-iccl/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W16-4817/) · [PDF](https://aclanthology.org/W16-4817.pdf) · [OpenAlex](https://openalex.org/W2786568652)

**Citations:** 11 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/hanani-2016-classifying-asr-transcriptions-according-arabic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper reports one team's entries in the Arabic dialect track of the 2016 DSL shared task. Systems had to assign short Buckwalter-transcribed ASR segments to one of five classes: EGY, GLF, LAV, MSA or NOR. The training data had 7619 segments derived from the speech corpus of Ali et al. (2016), and the test set had 1540 segments. The authors built several systems:
- SVMs over character n-gram probability vectors, with LLR-style weighting, upper and lower weight thresholds, and entropy/information-gain feature selection
- a word-unigram system that flags words whose frequency in a dialect is unusually high or low
- LSTM models, combined with the word system through a neural network

The best entry, a trigram SVM, reached 0.4279 accuracy on the test set. That is roughly double chance but about seven standard deviations below the leading teams. Its accuracy on held-out training data was 57%, so the authors spend much of the paper explaining the gap. Their candidate causes are mismatched segment-length distributions, topical words that pass for dialect markers, small validation sets, and data splits that were neither random nor shared across systems.

## Key points
- **Test set results.** The best run, a WEKA SVM on character trigrams, scored 0.4279 accuracy with a weighted F1 of 0.4264. Baselines were 0.20 for chance and 0.2279 for always choosing the majority class. The strongest team scored around 0.51.
- **Other submitted runs.** The unigram "outliers" word system scored 0.3948. The combination of the unigram system with the LSTMs scored 0.4091.
- **Post-task SVMs.** The authors later trained SVMs on bigrams, trigrams and 4-grams. After zero-count components were removed, the feature dimensions were 2601, 13,656 and 82,790. Accuracies were 0.3344, 0.3877 and 0.4214 respectively, so longer n-grams helped.
- **Segment lengths differed between the sets.**
  - In training, about 20% of segments were under 40 characters, and one segment ran to 18,017 characters.
  - In the test data, fewer than 8% were that short, the maximum was 2454 characters and the mean was 239.
- **Significance estimate.** Treating the test scores as binomial (n = 1540) gives a standard deviation of about 1.25% accuracy. On that basis the teams fall into cohorts, and the top three are statistically indistinguishable.

## Interesting ideas
- Turn a raw score into a binomial standard deviation before claiming that one system beats another; this groups systems into cohorts that are effectively tied rather than ranking them by noise.
- Weight features by their inverse corpus-wide probability, then clip the weights at both ends, because Zipf's law says the useful features sit in the middle band: very frequent ones are everywhere and very rare ones are unreliable.
- Score a word by how many standard deviations its frequency in a class sits above or below expectation; the paper warns that a large deviation on a common word can still carry little evidence, and suggests combining the evidence through normal CDF log-probabilities rather than adding raw deviations.
- Words that look strongly typical of a class may simply be the topic of a few stories, as with "AlflsTynyp" in MSA, and such topical leakage can explain much of a drop from development to test.
- Sorted training data split without randomisation, and a different split for each system, leaked shared prefixes and made it impossible to combine models cleanly; one random split used by every system avoids both problems.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Our evaluation reports could attach a binomial error bar to every accuracy figure, so that a small gain on a test set of a few hundred items is not taken as real progress.
- **I2** (tokens): When choosing which suffix or character features the tagger may use, a weight capped at both ends could stop a few rare forms from dominating the decision.
- **I3** (frames): Cues for speech acts or irony mined from labelled text could be scored this way, with the CDF correction, so that frequent hedge words are not over-trusted.
- **I4** (learn): Cue rules learned from silver data could be checked against negative controls that hold the topic fixed, so that topical words are not promoted into general markers.
- **I5** (learn): Every witness parser and the student models should draw on a single shuffled, frozen split, so that their agreements and ensembles are measured on data none of them was trained on.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Birzeit University](https://md.research.elamur.ai/free/en/universities/birzeit-university/) — University, Birzeit, Palestine · 38 works · 798 citations
- [Fitchburg State University](https://md.research.elamur.ai/free/en/universities/fitchburg-state-university/) — University, Fitchburg, United States · 2 works · 39 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 519 works

## Cite

Cite as: Abualsoud Hanani, Aziz Qaroush and Stephen Taylor. 2016. Classifying ASR Transcriptions According to Arabic Dialect. Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hanani2016classifying,
  title = {{Classifying ASR Transcriptions According to Arabic Dialect}},
  author = {Abualsoud Hanani, Aziz Qaroush and Stephen Taylor},
  booktitle = {Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3)},
  year = {2016},
  url = {https://aclanthology.org/W16-4817/}
}
```

- [← previous Arabizi Identification in Twitter Data](https://md.research.elamur.ai/free/en/papers/tobaili-2016-arabizi-identification-twitter-data/)
- [next → Creating Linked Data Morphological Language Resources with MMoOn - The Hebrew Morpheme Inventory](https://md.research.elamur.ai/free/en/papers/klimek-2016-creating-linked-data-morphological-language/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/5/)

## References

24 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hassan Al-Haj, Roger Hsiao, Ian Lane, Alan W. Black, and Alex Waibel. 2009. Pronunciation modeling for dialectal Arabic speech recognition. In Automatic Speech Recognition and Understanding Conference. [↗](https://doi.org/10.5445/ir/1000166347)
2. Mohammed Ali, Moustafa Elshafei, Mansour Al-Ghamdi, Husni Al-Muhtaseb, and Atef Al-Najjar. 2008. Generation of Arabic phonetic dictionaries for speech recognition. In International Conference on Innovations in Information Technology, 2008., pages 59–63. IEEE. [find](https://scholar.google.com/scholar?q=Mohammed+Ali%2C+Moustafa+Elshafei%2C+Mansour+Al%2DGhamdi%2C+Husni+Al%2DMuhtaseb%2C+and+Atef+Al%2DNajjar%2E+2008%2E+Generation+of+Arabic+phonetic+dictionaries+for+speech+recognition%2E+In+International+Conference+on+Innov)
3. Ahmed Ali, Najim Dehak, Patrick Cardinal, Sameer Khurana, Sree Harsha Yella, James Glass, Peter Bell, and Steve Renals. 2016. [Automatic dialect detection in Arabic broadcast speech](https://md.research.elamur.ai/free/en/papers/ali-2015-automatic-dialect-detection-arabic-broadcast/). In Proceedings of Interspeech 2016, pages 2934–2938.
4. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. [A multidialectal parallel corpus of Arabic](https://md.research.elamur.ai/free/en/papers/bouamor-2014-multidialectal-parallel-corpus-arabic/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC14). European Language Resources Association (ELRA), May.
5. Tim Buckwalter. 2002. Aramorph 1.0 program. [find](https://scholar.google.com/scholar?q=Tim+Buckwalter%2E+2002%2E+Aramorph+1%2E0+program%2E)
6. William M. Campbell, Joseph P. Campbell, Terry P. Gleason, Douglas A. Reynolds, and Wade Shen. 2007. Speaker verification using support vector machines and high-level features. IEEE Transactions on Audio, Speech, and Language Processing, 15(7):2085–2094. [↗](https://doi.org/10.1109/tasl.2007.902874)
7. Ronan Collobert, Koray Kavukcuoglu, and Clément Farabet. 2011. Torch7: A MATLAB-like environment for machine learning. In BigLearn, Neural Information Processing Systems Workshop. [find](https://scholar.google.com/scholar?q=Ronan+Collobert%2C+Koray+Kavukcuoglu%2C+and+Cle%CC%81ment+Farabet%2E+2011%2E+Torch7%3A+A+MATLAB%2Dlike+environment+for+machine+learning%2E+In+BigLearn%2C+Neural+Information+Processing+Systems+Workshop%2E)
8. Heba ElFardy and Mona Diab. 2013. [Sentence level dialect identification in Arabic](https://md.research.elamur.ai/free/en/papers/elfardy-2013-sentence-level-dialect-identification-arabic/). In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL), pages 456–461.
9. Mohamed Elmahdy, Rainer Gruhn, Wolfgang Minker, and S. Abdennadher. 2010. Cross-Lingual Acoustic Modeling for Dialectal Arabic Speech Recognition. In International Conference on Speech and Language Processing (Interspeech), September. [find](https://scholar.google.com/scholar?q=Mohamed+Elmahdy%2C+Rainer+Gruhn%2C+Wolfgang+Minker%2C+and+S%2E+Abdennadher%2E+2010%2E+Cross%2DLingual+Acoustic+Modeling+for+Dialectal+Arabic+Speech+Recognition%2E+In+International+Conference+on+Speech+and+Language+Pr)
10. Mohamed Elmahdy, Mark Hasegawa-Johnson, and Eiman Mustafawi. 2012. A baseline speech recognition system for Levantine colloquial Arabic. In 12th ESOLEC conference on Language Engineering. [find](https://scholar.google.com/scholar?q=Mohamed+Elmahdy%2C+Mark+Hasegawa%2DJohnson%2C+and+Eiman+Mustafawi%2E+2012%2E+A+baseline+speech+recognition+system+for+Levantine+colloquial+Arabic%2E+In+12th+ESOLEC+conference+on+Language+Engineering%2E)
11. Mohamed Elmahdy, Mark Hasegawa-Johnson, and Eiman Mustafawi. 2013. A transfer learning approach for under-resourced Arabic dialects speech recognition. In The 6th Language and Technology Conference. [find](https://scholar.google.com/scholar?q=Mohamed+Elmahdy%2C+Mark+Hasegawa%2DJohnson%2C+and+Eiman+Mustafawi%2E+2013%2E+A+transfer+learning+approach+for+under%2Dresourced+Arabic+dialects+speech+recognition%2E+In+The+6th+Language+and+Technology+Conference%2E)
12. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The WEKA data mining software: An update. SIGKDD Explorations, 11(1). [↗](https://doi.org/10.1145/1656274.1656278)
13. Abualsoud Hanani, Martin J. Russell, and Michael J. Carey. 2013. Human and computer recognition of regional accents and ethnic groups from British english speech. Computer Speech and Language, 27(1):5974. [↗](https://doi.org/10.1016/j.csl.2012.01.003)
14. Abualsoud Hanani, Hanna Basha, Yasmeen Sharaf, and Stephen Taylor. 2015. Palestinian Arabic regional accent recognition. In The 8th International Conference on Speech Technology and Human-Computer Dialogue. [find](https://scholar.google.com/scholar?q=Abualsoud+Hanani%2C+Hanna+Basha%2C+Yasmeen+Sharaf%2C+and+Stephen+Taylor%2E+2015%2E+Palestinian+Arabic+regional+accent+recognition%2E+In+The+8th+International+Conference+on+Speech+Technology+and+Human%2DComputer+Dia)
15. Andrej Karpathy, Justin Johnson, and Li Fei-Fei. 2016. [Visualizing and understanding recurrent networks](https://md.research.elamur.ai/free/en/papers/karpathy-2015-visualizing-understanding-recurrent-networks/). In 5th International Conference on Learning Representations. [↗](https://doi.org/10.48550/arxiv.1506.02078)
16. Shervin Malmasi and Mark Dras. 2015. Language identification using classifier ensembles. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (LT4VarDial), pages 35–43, Hissar, Bulgaria. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi+and+Mark+Dras%2E+2015%2E+Language+identification+using+classifier+ensembles%2E+In+Proceedings+of+the+Joint+Workshop+on+Language+Technology+for+Closely+Related+Languages%2C+Varieties+and+Dialec)
17. Shervin Malmasi, Eshrag Refaee, and Mark Dras. 2015. Arabic dialect identification using a parallel multidialectal corpus. In Proceedings of the 14th Conference of the Pacific Association for Computational Linguistics (PACLING 2015), pages 209–217, Bali, Indonesia, May. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi%2C+Eshrag+Refaee%2C+and+Mark+Dras%2E+2015%2E+Arabic+dialect+identification+using+a+parallel+multidialectal+corpus%2E+In+Proceedings+of+the+14th+Conference+of+the+Pacific+Association+for+Computat)
18. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. 2016. [Discriminating between similar languages and Arabic dialect identification: A report on the third DSL shared task](https://md.research.elamur.ai/free/en/papers/malmasi-2016-discriminating-between-similar-languages/). In Proceedings of the 3rd Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (VarDial), Osaka, Japan.
19. Maryam Najafian, Andrea DeMarco, Stephen Cox, and Martin Russell. 2014. Unsupervised model selection for recognition of regional accented speech. In Proceedings of Interspeech 2014. [↗](https://doi.org/10.13140/2.1.5176.8965)
20. Scott Novotney, Rich Schwartz, and Sanjeev Khudanpur. 2011. Unsupervised Arabic dialect adaptation with self-training. In Proceedings of Interspeech 2011, pages 1–4. [find](https://scholar.google.com/scholar?q=Scott+Novotney%2C+Rich+Schwartz%2C+and+Sanjeev+Khudanpur%2E+2011%2E+Unsupervised+Arabic+dialect+adaptation+with+self%2Dtraining%2E+In+Proceedings+of+Interspeech+2011%2C+pages+1%E2%80%934%2E)
21. Jordi Porta and José-Luis Sancho. 2014. Using maximum entropy models to discriminate between similar languages and varieties. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects (VarDial), pages 120–128, Dublin, Ireland. [find](https://scholar.google.com/scholar?q=Jordi+Porta+and+Jose%CC%81%2DLuis+Sancho%2E+2014%2E+Using+maximum+entropy+models+to+discriminate+between+similar+languages+and+varieties%2E+In+Proceedings+of+the+First+Workshop+on+Applying+NLP+Tools+to+Similar+Lan)
22. Dimitra Vergyri, Katrin Kirchhoff, Venkata Raman Rao Gadde, Andreas Stolcke, and Jing Zheng. 2005. Development of a conversational telephone speech recognizer for Levantine Arabic. In Proceedings of Interspeech 2005, pages 1613–1616. [find](https://scholar.google.com/scholar?q=Dimitra+Vergyri%2C+Katrin+Kirchhoff%2C+Venkata+Raman+Rao+Gadde%2C+Andreas+Stolcke%2C+and+Jing+Zheng%2E+2005%2E+Development+of+a+conversational+telephone+speech+recognizer+for+Levantine+Arabic%2E+In+Proceedings+of+I)
23. Omar F. Zaidan and Chris Callison-Burch. 2011. [The Arabic Online Commentary dataset: An annotated dataset of informal Arabic with high dialectal content](https://md.research.elamur.ai/free/en/papers/zaidan-2011-arabic-online-commentary-dataset-annotated/). In Proceedings of ACL, pages 37–41.
24. Omar F. Zaidan and Chris Callison-Burch. 2014. [Arabic dialect identification](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/). Computational Linguistics, 40(1):171–202. [↗](https://doi.org/10.1162/coli_a_00169)

*Updated 1 November 2026*