---
title: "The MGB-2 Challenge — Ali 2016"
description: "MGB-2 is a shared evaluation for automatic recognition of Arabic broadcast speech, held for SLT-2016."
authors: ["Ahmed Ali", "Peter Bell", "James Glass", "Yacine Messaoui", "Hamdy Mubarak", "Steve Renals", "Yifan Zhang"]
year: 2016
venue: "arXiv:1609.05625"
source: "https://arxiv.org/abs/1609.05625"
doi: "10.48550/arxiv.1609.05625"
arxiv: "1609.05625"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/ali-2016-mgb-2-challenge-arabic-multi/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/ali-2016-mgb-2-challenge-arabic-multi/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2016](https://md.research.elamur.ai/free/papers/2016/page/3/) / The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition

# The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition

**Authors:** [Ahmed Ali](https://md.research.elamur.ai/free/authors/ahmed-ali/), Peter Bell, [James Glass](https://md.research.elamur.ai/free/authors/james-glass/), Yacine Messaoui, [Hamdy Mubarak](https://md.research.elamur.ai/free/authors/hamdy-mubarak/), [Steve Renals](https://md.research.elamur.ai/free/authors/steve-renals/), Yifan Zhang · **Year:** 2016 · **Venue:** arXiv:1609.05625 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1609.05625) · [PDF](https://arxiv.org/pdf/1609.05625) · [DOI 10.48550/arxiv.1609.05625](https://doi.org/10.48550/arxiv.1609.05625) · [arXiv 1609.05625](https://arxiv.org/abs/1609.05625) · [OpenAlex](https://openalex.org/W2963077089)

**Citations:** 87 citations (openalex-snapshot · 2026-10-14) · [67](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
MGB-2 is a shared evaluation for automatic recognition of Arabic broadcast speech, held for SLT-2016. The earlier English MGB focused on variety across TV genres; this edition focuses on variety across Arabic dialects. The organisers collected about 1,200 hours of Aljazeera TV recordings from 2005–2015, drawn from 19 programme series of conversations, interviews and reports. The broadcaster's transcripts were edited, not verbatim, and had no timestamps. To make them usable, the organisers decoded the audio with the QCRI Arabic LVCSR system and matched the output against the text using Smith–Waterman local alignment. They kept per-segment quality scores instead of discarding imperfect data. Participants also received over 110M words of Aljazeera.net text for language modelling, a grapheme lexicon, a phoneme lexicon, and an open-source baseline on GitHub. The challenge had two tasks: transcription and word-level alignment. Of 13 teams, eight submitted final systems. The best system (QCRI) reached 14.7% WER, compared with a 34% baseline. Nobody entered the alignment task. The result is a large, closed-condition benchmark for dialectal Arabic ASR.

## Key points
- **Training data:** about 1,200 hours split into more than 550,000 segments.
  - By genre: conversation 63%, interview 19%, report 18%.
  - By language: over 70% is MSA; the rest is EGY, GLF, LEV and NOR dialects.
  - About 300 hours of short highlight clips were removed.
- **Alignment quality:** measured with AnchorRate, the share of transcript words matched to ASR output.
  - Results: 48% exact match, 15% approximate match, 37% no match.
  - Each segment's metadata includes AWD, PMER and WMER.
- **Lexicons:** a grapheme lexicon with over 900K entries, and a V2P-based phoneme lexicon with over 500K words averaging about four pronunciations per word.
- **Evaluation sets:** dev and eval are 10 hours each, taken from the final month of 2015. They were transcribed verbatim and segmented by hand. Four WER variants were reported; the official one uses the GLM and excludes overlapped speech.
- **Official results:**
  - Transcription WER: QCRI 14.7%, LIUM 16.7%, MIT 17.3%, NDSC 18.2%.
  - Alignment baseline, with a 100 ms tolerance window: precision 0.83, recall 0.70, F 0.76.

## Interesting ideas
- **Light supervision from approximate transcripts:** decode the audio, align the result to edited human text using both exact and fuzzy matching, and fill unmatched spans by linear time interpolation. This turns existing subtitles, parliament records or news transcripts into ASR training data without new annotation.
- **Release quality scores instead of a filtered set:** each segment ships with WMER, PMER and AWD, so users set their own thresholds. For example, NDSC kept segments with zero WMER and AWD between 0.3 and 0.7 s, leaving about 680 hours. Average word duration is also a cheap signal for spotting misaligned segments in any corpus.
- **Layered WER scoring:** the organisers scored raw text, then text without punctuation and diacritics, then GLM-mapped text, then text with normalised letter variants. This separates spelling-convention mismatches from real recognition errors, which helps for any language with inconsistent orthography.
- **Comparability by design:** all teams trained under a closed condition on the same data. Dev and eval came from the most recent month, so they could not overlap with training. Overlapped speech was scored but kept out of the main ranking.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

67 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 64 not yet.

- [Beyond Orthography: Automatic Recovery of Short Vowels and Dialectal Sounds in Arabic](https://md.research.elamur.ai/free/papers/acl-2024-kheir-beyond-orthography-automatic-recovery-short/)
- [A Bilingual Bimodal Benchmark for Arabic-English NLP across Grammatical Correction, Essay Scoring, Morphological Tagging, and Speech Recognition](https://md.research.elamur.ai/free/papers/alhafni-2026-bilingual-bimodal-benchmark-arabic-english/)
- [Speech Recognition Challenge in the Wild: Arabic MGB-3](https://md.research.elamur.ai/free/papers/ali-2017-speech-recognition-challenge-wild-arabic/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ali2016mgb,
  title = {{The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition}},
  author = {Ali, Ahmed and Bell, Peter and Glass, James and Messaoui, Yacine and Mubarak, Hamdy and Renals, Steve and Zhang, Yifan},
  year = {2016},
  doi = {10.48550/arxiv.1609.05625},
  eprint = {1609.05625},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1609.05625}
}
```

- [← previous The LAMBADA dataset: Word prediction requiring a broad discourse context](https://md.research.elamur.ai/free/papers/paperno-2016-lambada-dataset-word-prediction-requiring/)
- [next → The More Antecedents, the Merrier: Resolving Multi-Antecedent Anaphors](https://md.research.elamur.ai/free/papers/vala-2016-more-antecedents-merrier-resolving-multi/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/papers/2016/page/3/)

## References

11 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. P. J. Bell, M. J. F. Gales, T. Hain, J. Kilgour, P. Lanchantin, X. Liu, A. McParland, S. Renals, O. Saz, M. Wester, and P. C. Woodland, “The MGB challenge: Evaluating multi-genre broadcast media recognition,” in ASRU, 2015. [find](https://scholar.google.com/scholar?q=P%2E+J%2E+Bell%2C+M%2E+J%2E+F%2E+Gales%2C+T%2E+Hain%2C+J%2E+Kilgour%2C+P%2E+Lanchantin%2C+X%2E+Liu%2C+A%2E+McParland%2C+S%2E+Renals%2C+O%2E+Saz%2C+M%2E+Wester%2C+and+P%2E+C%2E+Woodland%2C+%E2%80%9CThe+MGB+challenge%3A+Evaluating+multi%2Dgenre+broadcast+media+recog)
2. A. Ali, Y. Zhang, and S. Vogel, “QCRI advanced transcription system (QATS),” in SLT, 2014. [find](https://scholar.google.com/scholar?q=A%2E+Ali%2C+Y%2E+Zhang%2C+and+S%2E+Vogel%2C+%E2%80%9CQCRI+advanced+transcription+system+%28QATS%29%2C%E2%80%9D+in+SLT%2C+2014%2E)
3. A. Ali, Y. Zhang, P. Cardinal, N. Dahak, S. Vogel, and J. Glass, “A complete Kaldi recipe for building Arabic speech recognition systems,” in SLT, 2014. [find](https://scholar.google.com/scholar?q=A%2E+Ali%2C+Y%2E+Zhang%2C+P%2E+Cardinal%2C+N%2E+Dahak%2C+S%2E+Vogel%2C+and+J%2E+Glass%2C+%E2%80%9CA+complete+Kaldi+recipe+for+building+Arabic+speech+recognition+systems%2C%E2%80%9D+in+SLT%2C+2014%2E)
4. N. Braunschweiler, M. J. F. Gales, and S Buchholz, “Lightly supervised recognition for automatic alignment of large coherent speech recordings.,” in Interspeech, 2010. [find](https://scholar.google.com/scholar?q=N%2E+Braunschweiler%2C+M%2E+J%2E+F%2E+Gales%2C+and+S+Buchholz%2C+%E2%80%9CLightly+supervised+recognition+for+automatic+alignment+of+large+coherent+speech+recordings%2E%2C%E2%80%9D+in+Interspeech%2C+2010%2E)
5. T. Smith and M. Waterman, “Identification of common molecular subsequences,” Journal of molecular biology, vol. 147, no. 1, 1981. [↗](https://doi.org/10.1016/0022-2836(81)90087-5)
6. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, J. Silovsky,̀ G. Stemmer, and K. Vesely, “The Kaldi speech recognition toolkit,” in ASRU, 2011. [find](https://scholar.google.com/scholar?q=D%2E+Povey%2C+A%2E+Ghoshal%2C+G%2E+Boulianne%2C+L%2E+Burget%2C+O%2E+Glembek%2C+N%2E+Goel%2C+M%2E+Hannemann%2C+P%2E+Motlicek%2C+Y%2E+Qian%2C+P%2E+Schwarz%2C+J%2E+Silovsky%2C%CC%80+G%2E+Stemmer%2C+and+K%2E+Vesely%2C+%E2%80%9CThe+Kaldi+speech+recognition+toolkit%2C%E2%80%9D+in+)
7. Andreas Stolcke et al., “SRILM - an extensible language modeling toolkit.,” in Interspeech, 2002. [find](https://scholar.google.com/scholar?q=Andreas+Stolcke+et+al%2E%2C+%E2%80%9CSRILM+%2D+an+extensible+language+modeling+toolkit%2E%2C%E2%80%9D+in+Interspeech%2C+2002%2E)
8. S. Khurana and A. Ali, “QCRI advanced transcription system (QATS) for the arabic multi-dialect broadcast media recognition: MGB-2 challenge,” in SLT, 2016. [find](https://scholar.google.com/scholar?q=S%2E+Khurana+and+A%2E+Ali%2C+%E2%80%9CQCRI+advanced+transcription+system+%28QATS%29+for+the+arabic+multi%2Ddialect+broadcast+media+recognition%3A+MGB%2D2+challenge%2C%E2%80%9D+in+SLT%2C+2016%2E)
9. N. Tomashenko, K. Vythelingum, A. Rousseau, and Y. Estève, “LIUM ASR systems for the 2016 multi-genre broadcast Arabic challenge,” in SLT, 2016. [find](https://scholar.google.com/scholar?q=N%2E+Tomashenko%2C+K%2E+Vythelingum%2C+A%2E+Rousseau%2C+and+Y%2E+Este%CC%80ve%2C+%E2%80%9CLIUM+ASR+systems+for+the+2016+multi%2Dgenre+broadcast+Arabic+challenge%2C%E2%80%9D+in+SLT%2C+2016%2E)
10. T. AlHanai, W. Hsu, and J. Glass, “Development of the MIT ASR system for the 2016 Arabic multi-genre broadcast challenge,” in SLT, 2016. [find](https://scholar.google.com/scholar?q=T%2E+AlHanai%2C+W%2E+Hsu%2C+and+J%2E+Glass%2C+%E2%80%9CDevelopment+of+the+MIT+ASR+system+for+the+2016+Arabic+multi%2Dgenre+broadcast+challenge%2C%E2%80%9D+in+SLT%2C+2016%2E)
11. X. Yang, D. Qu, W. Zhang, and W. Zhang, “The NDSC transcription system for the 2016 multi-genre broadcast challenge,” in SLT, 2016. Fig. 3. Distribution of Average Word Duration (AWD) Fig. 4. Cumulative Average Word Duration Fig. 5. Cumulative Duration for Word (red line), and Grapheme (blue line) Matching Error Rate (MER) [find](https://scholar.google.com/scholar?q=X%2E+Yang%2C+D%2E+Qu%2C+W%2E+Zhang%2C+and+W%2E+Zhang%2C+%E2%80%9CThe+NDSC+transcription+system+for+the+2016+multi%2Dgenre+broadcast+challenge%2C%E2%80%9D+in+SLT%2C+2016%2E+Fig%2E+3%2E+Distribution+of+Average+Word+Duration+%28AWD%29+Fig%2E+4%2E+Cumulati)