---
title: "Arabic Speech Recognition by End-to-End — Hussein 2021"
description: "The paper asks whether machines now match people at transcribing Arabic, a morphologically rich language with mostly undiacritized text and many dialects…"
authors: ["Amir Hussein", "Shinji Watanabe", "Ahmed Ali"]
year: 2021
venue: "arXiv:2101.08454"
source: "https://arxiv.org/abs/2101.08454"
doi: "10.48550/arxiv.2101.08454"
arxiv: "2101.08454"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hussein-2021-arabic-speech-recognition-end-end/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hussein-2021-arabic-speech-recognition-end-end/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/) / Arabic Speech Recognition by End-to-End, Modular Systems and Human

# Arabic Speech Recognition by End-to-End, Modular Systems and Human

**Authors:** [Amir Hussein](https://md.research.elamur.ai/free/en/authors/amir-hussein/), [Shinji Watanabe](https://md.research.elamur.ai/free/en/authors/shinji-watanabe/), [Ahmed Ali](https://md.research.elamur.ai/free/en/authors/ahmed-ali/) · **Year:** 2021 · **Venue:** arXiv:2101.08454 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2101.08454) · [PDF](https://arxiv.org/pdf/2101.08454) · [DOI 10.48550/arxiv.2101.08454](https://doi.org/10.48550/arxiv.2101.08454) · [arXiv 2101.08454](https://arxiv.org/abs/2101.08454) · [OpenAlex](https://openalex.org/W3124014937)

**Citations:** 62 citations (openalex-snapshot · 2026-10-14) · [14](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper asks whether machines now match people at transcribing Arabic, a morphologically rich language with mostly undiacritized text and many dialects that lack standard spelling. Earlier human-versus-machine comparisons covered only English and modular HMM-DNN systems. The authors built what they describe as the first end-to-end transformer ASR for Arabic in ESPnet. It uses sub-word units to limit out-of-vocabulary words. They compared it head to head with a strong single HMM-DNN system and with human transcribers, both expert linguists and lay native speakers. For this comparison they recorded a new hidden 3-hour test set mixing news and conversation in MSA and dialects. The transformer set new best results on MGB2, MGB3 and MGB5. It arguably beats lay natives, but expert linguists still lead by about 3.5% absolute WER on average. The authors also found that long, unsegmented audio badly hurts the transformer, and they designed a VAD pipeline that caps segment length. The work matters because it gives a clear benchmark and public recipe for Arabic and its dialects.

## Key points
- The E2E transformer reached 12.5% WER on MGB2, 27.5% on MGB3 and 33.8% on MGB5. The earlier best on MGB2 was Aalto's 13.2%, which combined more than 30 systems.
- On dialect tests, AV-WER fell from 37.5 to 34.4 on MGB3 and from 59.4 (RDI-CU) to 56.2 on MGB5 when adaptation and dev data were both used for finetuning.
- In the human study, machine errors closely resembled those of expert linguists. The remaining gap to experts averages 3.5% absolute WER on raw transcripts.
- The VAD pipeline merges InaSpeechSegmenter's speech detection with the length cap of an energy-based VAD.
  - Plain InaSeg left 43.6% of segments longer than 30 s, against 7.9% in training data, and the model broke down beyond roughly 30 s.
  - The improved segmentation keeps every segment under 30 s.
- For the dialect sets, the MGB2 model was finetuned with a lower learning rate (0.1) and no warm-up. Transcripts from four annotators per utterance were pooled, giving four times the training text.

## Interesting ideas
- The authors built a fresh, unpublished test set because public benchmarks may already have shaped earlier systems, which keeps the comparison honest. — For Mova (learn): Keep a private held-out set of texts and questions that no teacher tier or tuning loop ever sees, and use it as the final check alongside negative controls.
- They used two levels of human baseline (expert and lay native) and analysed how machine errors correlate with each, which shows the kind of errors made, not just their number. — For Mova (answer): Benchmark the readers on school tests against both strong and average human answers, and compare error overlap to see whether failures look like plausible human mistakes or like system bugs.
- They explained a sudden accuracy collapse by comparing input length distributions in training and test data, then fixed it by capping length at the source. — For Mova (tree): Track sentence-length distributions for gparse training treebanks against incoming text, and split or flag overlong inputs before parsing instead of trusting the model past its seen range.
- When several annotators gave different but valid transcripts, each one was kept as a separate training target and scored with a multi-reference metric. — For Mova (learn): When witness parsers or annotators disagree on equally valid analyses, keep the alternatives as multiple acceptable references instead of forcing one silver answer.
- They pretrained on the standard variety and then finetuned gently, with a lower rate and no warm-up, on small dialect sets, a simple recipe for low-resource varieties. — For Mova (tree): Train on large standard-English UD treebanks first, then lightly adapt to small treebanks of other registers or UD dialects after conversion.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cited by

Cited by

14 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 10 not yet.

- [Unsupervised Data Selection for TTS: Using Arabic Broadcast News as a Case Study](https://md.research.elamur.ai/free/en/papers/baali-2023-unsupervised-data-selection-tts-using/)
- [Towards One Model to Rule All: Multilingual Strategy for Dialectal Code-Switching Arabic ASR](https://md.research.elamur.ai/free/en/papers/chowdhury-2021-towards-one-model-rule-all/)
- [Open Automatic Speech Recognition Models for Classical and Modern Standard Arabic](https://md.research.elamur.ai/free/en/papers/grigoryan-2025-open-automatic-speech-recognition-models/)
- [N-Shot Benchmarking of Whisper on Diverse Arabic Speech Recognition](https://md.research.elamur.ai/free/en/papers/talafha-2023-n-shot-benchmarking-whisper-diverse/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{hussein2021arabic,
  title = {{Arabic Speech Recognition by End-to-End, Modular Systems and Human}},
  author = {Hussein, Amir and Watanabe, Shinji and Ali, Ahmed},
  year = {2021},
  doi = {10.48550/arxiv.2101.08454},
  eprint = {2101.08454},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2101.08454}
}
```

- [← previous Arabic Offensive Language on Twitter: Analysis and Experiments](https://md.research.elamur.ai/free/en/papers/mubarak-2021-arabic-offensive-language-twitter-analysis/)
- [next → AraCOVID19-MFH: Arabic COVID-19 Multi-label Fake News and Hate Speech Detection Dataset](https://md.research.elamur.ai/free/en/papers/ameur-2021-aracovid19-mfh-arabic-covid-19/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/)

## References

43 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. D. Amodei, S. Ananthanarayanan, R. Anubhai, J. Bai, E. Battenberg, C. Case, J. Casper, B. Catanzaro, Q. Cheng, G. Chen, et al., Deep speech 2: End-to-end speech recognition in English and Mandarin, in: International conference on machine learning, 2016, pp. 173–182. [find](https://scholar.google.com/scholar?q=D%2E+Amodei%2C+S%2E+Ananthanarayanan%2C+R%2E+Anubhai%2C+J%2E+Bai%2C+E%2E+Battenberg%2C+C%2E+Case%2C+J%2E+Casper%2C+B%2E+Catanzaro%2C+Q%2E+Cheng%2C+G%2E+Chen%2C+et+al%2E%2C+Deep+speech+2%3A+End%2Dto%2Dend+speech+recognition+in+English+and+Mandarin%2C+in) [our card](https://md.research.elamur.ai/free/en/papers/amodei-2015-deep-speech-2-end-end/)
2. A. Graves, N. Jaitly, Towards end-to-end speech recognition with recurrent neural networks, in: International conference on machine learning, 2014, pp. 1764–1772. [find](https://scholar.google.com/scholar?q=A%2E+Graves%2C+N%2E+Jaitly%2C+Towards+end%2Dto%2Dend+speech+recognition+with+recurrent+neural+networks%2C+in%3A+International+conference+on+machine+learning%2C+2014%2C+pp%2E+1764%E2%80%931772%2E)
3. W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke, D. Yu, G. Zweig, Achieving human parity in conversational speech recognition, arXiv preprint arXiv:1610.05256. [↗](https://arxiv.org/abs/1610.05256)
4. G. Saon, G. Kurata, T. Sercu, K. Audhkhasi, S. Thomas, D. Dimitriadis, X. Cui, B. Ramabhadran, M. Picheny, L.-L. Lim, et al., English conversational telephone speech recognition by humans and machines, Proc. Interspeech (2017) 132–136. [find](https://scholar.google.com/scholar?q=G%2E+Saon%2C+G%2E+Kurata%2C+T%2E+Sercu%2C+K%2E+Audhkhasi%2C+S%2E+Thomas%2C+D%2E+Dimitriadis%2C+X%2E+Cui%2C+B%2E+Ramabhadran%2C+M%2E+Picheny%2C+L%2E%2DL%2E+Lim%2C+et+al%2E%2C+English+conversational+telephone+speech+recognition+by+humans+and+machines)
5. P. Michalowski, The lives of the sumerian language, Margins of writing, origins of cultures (2006) 159–84. [find](https://scholar.google.com/scholar?q=P%2E+Michalowski%2C+The+lives+of+the+sumerian+language%2C+Margins+of+writing%2C+origins+of+cultures+%282006%29+159%E2%80%9384%2E)
6. P. Smit, S. R. Gangireddy, S. Enarvi, S. Virpioja, M. Kurimo, Aalto system for the 2017 Arabic multi-genre broadcast challenge, in: IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), IEEE, 2017, pp. 338–345. [find](https://scholar.google.com/scholar?q=P%2E+Smit%2C+S%2E+R%2E+Gangireddy%2C+S%2E+Enarvi%2C+S%2E+Virpioja%2C+M%2E+Kurimo%2C+Aalto+system+for+the+2017+Arabic+multi%2Dgenre+broadcast+challenge%2C+in%3A+IEEE+Automatic+Speech+Recognition+and+Understanding+Workshop+%28ASRU%29%2C)
7. H. Mubarak, A. Abdelali, H. Sajjad, Y. Samih, K. Darwish, Highly effective arabic diacritization using sequence to sequence modeling, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 2019, pp. 2390–2395. [find](https://scholar.google.com/scholar?q=H%2E+Mubarak%2C+A%2E+Abdelali%2C+H%2E+Sajjad%2C+Y%2E+Samih%2C+K%2E+Darwish%2C+Highly+effective+arabic+diacritization+using+sequence+to+sequence+modeling%2C+in%3A+Proceedings+of+the+2019+Conference+of+the+North+American+Chapt)
8. A. Das, M. Hasegawa-Johnson, Cross-lingual transfer learning during supervised training in low resource scenarios, in: Sixteenth Annual Conference of the International Speech Communication Association, 2015. [find](https://scholar.google.com/scholar?q=A%2E+Das%2C+M%2E+Hasegawa%2DJohnson%2C+Cross%2Dlingual+transfer+learning+during+supervised+training+in+low+resource+scenarios%2C+in%3A+Sixteenth+Annual+Conference+of+the+International+Speech+Communication+Association)
9. S. Khurana, A. Ali, J. Glass, Darts: Dialectal Arabic transcription system, arXiv preprint arXiv:1909.12163. [↗](https://arxiv.org/abs/1909.12163) [our card](https://md.research.elamur.ai/free/en/papers/khurana-2019-darts-dialectal-arabic-transcription-system/)
10. A. Ahmed, Y. Hifny, K. Shaalan, S. Toral, End-to-end lexicon free Arabic speech recognition using recurrent neural networks, Computational Linguistics, Speech And Image Processing For Arabic Language 4 (2018) 231. [find](https://scholar.google.com/scholar?q=A%2E+Ahmed%2C+Y%2E+Hifny%2C+K%2E+Shaalan%2C+S%2E+Toral%2C+End%2Dto%2Dend+lexicon+free+Arabic+speech+recognition+using+recurrent+neural+networks%2C+Computational+Linguistics%2C+Speech+And+Image+Processing+For+Arabic+Language+)
11. T. Kudo, Subword regularization: Improving neural network translation models with multiple subword candidates, arXiv preprint arXiv:1804.10959. [↗](https://arxiv.org/abs/1804.10959)
12. S. Khurana, A. Ali, Qcri advanced transcription system (QATS) for the Arabic multi-dialect broadcast media recognition: MGB-2 challenge, in: IEEE Spoken Language Technology Workshop (SLT), IEEE, 2016, pp. 292– 298. [find](https://scholar.google.com/scholar?q=S%2E+Khurana%2C+A%2E+Ali%2C+Qcri+advanced+transcription+system+%28QATS%29+for+the+Arabic+multi%2Ddialect+broadcast+media+recognition%3A+MGB%2D2+challenge%2C+in%3A+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2C+IEEE%2C+2016%2C)
13. D. Doukhan, E. Lechapt, M. Evrard, J. Carrive, Ina’s mirex 2018 music and speech detection system, in: Music Information Retrieval Evaluation eXchange (MIREX 2018), 2018. [find](https://scholar.google.com/scholar?q=D%2E+Doukhan%2C+E%2E+Lechapt%2C+M%2E+Evrard%2C+J%2E+Carrive%2C+Ina%E2%80%99s+mirex+2018+music+and+speech+detection+system%2C+in%3A+Music+Information+Retrieval+Evaluation+eXchange+%28MIREX+2018%29%2C+2018%2E)
14. T. Giannakopoulos, pyaudioanalysis: An open-source python library for audio signal analysis, PloS one 10 (12) (2015) e0144610. [find](https://scholar.google.com/scholar?q=T%2E+Giannakopoulos%2C+pyaudioanalysis%3A+An+open%2Dsource+python+library+for+audio+signal+analysis%2C+PloS+one+10+%2812%29+%282015%29+e0144610%2E)
15. G. E. Dahl, D. Yu, L. Deng, A. Acero, Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition, IEEE Transactions on audio, speech, and language processing 20 (1) (2011) 30–42. [find](https://scholar.google.com/scholar?q=G%2E+E%2E+Dahl%2C+D%2E+Yu%2C+L%2E+Deng%2C+A%2E+Acero%2C+Context%2Ddependent+pre%2Dtrained+deep+neural+networks+for+large%2Dvocabulary+speech+recognition%2C+IEEE+Transactions+on+audio%2C+speech%2C+and+language+processing+20+%281%29+%2820)
16. A. Graves, A.-r. Mohamed, G. Hinton, Speech recognition with deep recurrent neural networks, in: IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, 2013, pp. 6645–6649. [find](https://scholar.google.com/scholar?q=A%2E+Graves%2C+A%2E%2Dr%2E+Mohamed%2C+G%2E+Hinton%2C+Speech+recognition+with+deep+recurrent+neural+networks%2C+in%3A+IEEE+international+conference+on+acoustics%2C+speech+and+signal+processing+%28ICASSP%29%2C+IEEE%2C+2013%2C+pp%2E+6645)
17. Y. Hifny, Unified acoustic modeling using deep conditional random fields, Transactions on Machine Learning and Artificial Intelligence 3 (2) (2015) 65–65. [find](https://scholar.google.com/scholar?q=Y%2E+Hifny%2C+Unified+acoustic+modeling+using+deep+conditional+random+fields%2C+Transactions+on+Machine+Learning+and+Artificial+Intelligence+3+%282%29+%282015%29+65%E2%80%9365%2E)
18. V. Peddinti, D. Povey, S. Khudanpur, A time delay neural network architecture for efficient modeling of long temporal contexts, in: Sixteenth Annual Conference of the International Speech Communication Association, 2015. [find](https://scholar.google.com/scholar?q=V%2E+Peddinti%2C+D%2E+Povey%2C+S%2E+Khudanpur%2C+A+time+delay+neural+network+architecture+for+efficient+modeling+of+long+temporal+contexts%2C+in%3A+Sixteenth+Annual+Conference+of+the+International+Speech+Communicatio)
19. A. Ali, S. Vogel, S. Renals, Speech recognition challenge in the wild: Arabic MGB-3, in: IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), IEEE, 2017, pp. 316–322. [find](https://scholar.google.com/scholar?q=A%2E+Ali%2C+S%2E+Vogel%2C+S%2E+Renals%2C+Speech+recognition+challenge+in+the+wild%3A+Arabic+MGB%2D3%2C+in%3A+IEEE+Automatic+Speech+Recognition+and+Understanding+Workshop+%28ASRU%29%2C+IEEE%2C+2017%2C+pp%2E+316%E2%80%93322%2E) [our card](https://md.research.elamur.ai/free/en/papers/ali-2017-speech-recognition-challenge-wild-arabic/)
20. D. Wang, X. Wang, S. Lv, An overview of end-to-end automatic speech recognition, Symmetry 11 (8) (2019) 1018. [find](https://scholar.google.com/scholar?q=D%2E+Wang%2C+X%2E+Wang%2C+S%2E+Lv%2C+An+overview+of+end%2Dto%2Dend+automatic+speech+recognition%2C+Symmetry+11+%288%29+%282019%29+1018%2E)
21. W. Chan, N. Jaitly, Q. Le, O. Vinyals, Listen, attend and spell: A neural network for large vocabulary conversational speech recognition, in: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2016, pp. 4960–4964. [find](https://scholar.google.com/scholar?q=W%2E+Chan%2C+N%2E+Jaitly%2C+Q%2E+Le%2C+O%2E+Vinyals%2C+Listen%2C+attend+and+spell%3A+A+neural+network+for+large+vocabulary+conversational+speech+recognition%2C+in%3A+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Sig)
22. J. K. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho, Y. Bengio, Attention-based models for speech recognition, Advances in neural information processing systems 28 (2015) 577–585. [find](https://scholar.google.com/scholar?q=J%2E+K%2E+Chorowski%2C+D%2E+Bahdanau%2C+D%2E+Serdyuk%2C+K%2E+Cho%2C+Y%2E+Bengio%2C+Attention%2Dbased+models+for+speech+recognition%2C+Advances+in+neural+information+processing+systems+28+%282015%29+577%E2%80%93585%2E) [our card](https://md.research.elamur.ai/free/en/papers/chorowski-2015-attention-based-models-speech-recognition/)
23. S. Watanabe, T. Hori, S. Kim, J. R. Hershey, T. Hayashi, Hybrid ctc/attention architecture for end-to-end speech recognition, IEEE Journal of Selected Topics in Signal Processing 11 (8) (2017) 1240–1253. [find](https://scholar.google.com/scholar?q=S%2E+Watanabe%2C+T%2E+Hori%2C+S%2E+Kim%2C+J%2E+R%2E+Hershey%2C+T%2E+Hayashi%2C+Hybrid+ctc%2Fattention+architecture+for+end%2Dto%2Dend+speech+recognition%2C+IEEE+Journal+of+Selected+Topics+in+Signal+Processing+11+%288%29+%282017%29+1240%E2%80%9312)
24. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, . Kaiser, I. Polosukhin, Attention is all you need, in: Advances in neural information processing systems, 2017, pp. 5998–6008. [find](https://scholar.google.com/scholar?q=A%2E+Vaswani%2C+N%2E+Shazeer%2C+N%2E+Parmar%2C+J%2E+Uszkoreit%2C+L%2E+Jones%2C+A%2E+N%2E+Gomez%2C+%2E+Kaiser%2C+I%2E+Polosukhin%2C+Attention+is+all+you+need%2C+in%3A+Advances+in+neural+information+processing+systems%2C+2017%2C+pp%2E+5998%E2%80%936008%2E)
25. S. Karita, N. Chen, T. Hayashi, T. Hori, H. Inaguma, Z. Jiang, M. Someki, N. E. Y. Soplin, R. Yamamoto, X. Wang, et al., A comparative study on transformer vs rnn in speech applications, in: IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), IEEE, 2019, pp. 449– 456. [find](https://scholar.google.com/scholar?q=S%2E+Karita%2C+N%2E+Chen%2C+T%2E+Hayashi%2C+T%2E+Hori%2C+H%2E+Inaguma%2C+Z%2E+Jiang%2C+M%2E+Someki%2C+N%2E+E%2E+Y%2E+Soplin%2C+R%2E+Yamamoto%2C+X%2E+Wang%2C+et+al%2E%2C+A+comparative+study+on+transformer+vs+rnn+in+speech+applications%2C+in%3A+IEEE+Auto)
26. Y. Wang, A. Mohamed, D. Le, C. Liu, A. Xiao, J. Mahadeokar, H. Huang, A. Tjandra, X. Zhang, F. Zhang, et al., Transformer-based acoustic modeling for hybrid speech recognition, in: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2020, pp. 6874– 6878. [find](https://scholar.google.com/scholar?q=Y%2E+Wang%2C+A%2E+Mohamed%2C+D%2E+Le%2C+C%2E+Liu%2C+A%2E+Xiao%2C+J%2E+Mahadeokar%2C+H%2E+Huang%2C+A%2E+Tjandra%2C+X%2E+Zhang%2C+F%2E+Zhang%2C+et+al%2E%2C+Transformer%2Dbased+acoustic+modeling+for+hybrid+speech+recognition%2C+in%3A+IEEE+International+)
27. G. Synnaeve, Q. Xu, J. Kahn, T. Likhomanenko, E. Grave, V. Pratap, A. Sriram, V. Liptchinsky, R. Collobert, End-to-end ASR: from supervised to semi-supervised learning with modern architectures, arXiv preprint arXiv:1911.08460. [↗](https://arxiv.org/abs/1911.08460)
28. A. Stolcke, J. Droppo, Comparing human and machine errors in conversational speech transcription, arXiv preprint arXiv:1708.08615. [↗](https://arxiv.org/abs/1708.08615)
29. A. Ali, P. Bell, J. Glass, Y. Messaoui, H. Mubarak, S. Renals, Y. Zhang, The mgb-2 challenge: Arabic multi-dialect broadcast media recognition, in: IEEE Spoken Language Technology Workshop (SLT), IEEE, 2016, pp. 279–284. [find](https://scholar.google.com/scholar?q=A%2E+Ali%2C+P%2E+Bell%2C+J%2E+Glass%2C+Y%2E+Messaoui%2C+H%2E+Mubarak%2C+S%2E+Renals%2C+Y%2E+Zhang%2C+The+mgb%2D2+challenge%3A+Arabic+multi%2Ddialect+broadcast+media+recognition%2C+in%3A+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2C+IEEE) [our card](https://md.research.elamur.ai/free/en/papers/ali-2016-mgb-2-challenge-arabic-multi/)
30. D. Povey, V. Peddinti, D. Galvez, P. Ghahremani, V. Manohar, X. Na, Y. Wang, S. Khudanpur, Purely sequence-trained neural networks for ASR based on lattice-free MMI, in: Proc. Interspeech, 2016, pp. 2751–2755. [find](https://scholar.google.com/scholar?q=D%2E+Povey%2C+V%2E+Peddinti%2C+D%2E+Galvez%2C+P%2E+Ghahremani%2C+V%2E+Manohar%2C+X%2E+Na%2C+Y%2E+Wang%2C+S%2E+Khudanpur%2C+Purely+sequence%2Dtrained+neural+networks+for+ASR+based+on+lattice%2Dfree+MMI%2C+in%3A+Proc%2E+Interspeech%2C+2016%2C+pp%2E+2)
31. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, et al., The kaldi speech recognition toolkit, in: IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), no. CONF, IEEE Signal Processing Society, 2011. [find](https://scholar.google.com/scholar?q=D%2E+Povey%2C+A%2E+Ghoshal%2C+G%2E+Boulianne%2C+L%2E+Burget%2C+O%2E+Glembek%2C+N%2E+Goel%2C+M%2E+Hannemann%2C+P%2E+Motlicek%2C+Y%2E+Qian%2C+P%2E+Schwarz%2C+et+al%2E%2C+The+kaldi+speech+recognition+toolkit%2C+in%3A+IEEE+Automatic+Speech+Recognition+)
32. D. Povey, H. Hadian, P. Ghahremani, K. Li, S. Khudanpur, A time-restricted self-attention layer for ASR, in: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2018, pp. 5874–5878. [find](https://scholar.google.com/scholar?q=D%2E+Povey%2C+H%2E+Hadian%2C+P%2E+Ghahremani%2C+K%2E+Li%2C+S%2E+Khudanpur%2C+A+time%2Drestricted+self%2Dattention+layer+for+ASR%2C+in%3A+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+IEEE%2C+20)
33. P. Ghahremani, B. BabaAli, D. Povey, K. Riedhammer, J. Trmal, S. Khudanpur, A pitch extraction algorithm tuned for automatic speech recognition, in: IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, 2014, pp. 2494–2498. [find](https://scholar.google.com/scholar?q=P%2E+Ghahremani%2C+B%2E+BabaAli%2C+D%2E+Povey%2C+K%2E+Riedhammer%2C+J%2E+Trmal%2C+S%2E+Khudanpur%2C+A+pitch+extraction+algorithm+tuned+for+automatic+speech+recognition%2C+in%3A+IEEE+international+conference+on+acoustics%2C+speech+)
34. J. L. Ba, J. R. Kiros, G. E. Hinton, Layer normalization, arXiv preprint arXiv:1607.06450. [↗](https://arxiv.org/abs/1607.06450) [our card](https://md.research.elamur.ai/free/en/papers/ba-2016-layer-normalization/)
35. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778. [find](https://scholar.google.com/scholar?q=K%2E+He%2C+X%2E+Zhang%2C+S%2E+Ren%2C+J%2E+Sun%2C+Deep+residual+learning+for+image+recognition%2C+in%3A+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+2016%2C+pp%2E+770%E2%80%93778%2E)
36. A. Ali, S. Shon, Y. Samih, H. Mubarak, A. Abdelali, J. Glass, S. Renals, K. Choukri, The MGB-5 challenge: Recognition and dialect identification of dialectal Arabic speech, in: IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), IEEE, 2019, pp. 1026–1033. [find](https://scholar.google.com/scholar?q=A%2E+Ali%2C+S%2E+Shon%2C+Y%2E+Samih%2C+H%2E+Mubarak%2C+A%2E+Abdelali%2C+J%2E+Glass%2C+S%2E+Renals%2C+K%2E+Choukri%2C+The+MGB%2D5+challenge%3A+Recognition+and+dialect+identification+of+dialectal+Arabic+speech%2C+in%3A+IEEE+Automatic+Speech+R)
37. A. Ali, W. Magdy, P. Bell, S. Renais, Multi-reference wer for evaluating asr for languages with no orthographic rules, in: IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), IEEE, 2015, pp. 576– 580. [find](https://scholar.google.com/scholar?q=A%2E+Ali%2C+W%2E+Magdy%2C+P%2E+Bell%2C+S%2E+Renais%2C+Multi%2Dreference+wer+for+evaluating+asr+for+languages+with+no+orthographic+rules%2C+in%3A+IEEE+Automatic+Speech+Recognition+and+Understanding+Workshop+%28ASRU%29%2C+IEEE%2C+20)
38. S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y. Unno, N.-E. Y. Soplin, J. Heymann, M. Wiesner, N. Chen, et al., Espnet: End-to-end speech processing toolkit, Proc. Interspeech (2018) 2207–2211. [find](https://scholar.google.com/scholar?q=S%2E+Watanabe%2C+T%2E+Hori%2C+S%2E+Karita%2C+T%2E+Hayashi%2C+J%2E+Nishitoba%2C+Y%2E+Unno%2C+N%2E%2DE%2E+Y%2E+Soplin%2C+J%2E+Heymann%2C+M%2E+Wiesner%2C+N%2E+Chen%2C+et+al%2E%2C+Espnet%3A+End%2Dto%2Dend+speech+processing+toolkit%2C+Proc%2E+Interspeech+%282018%29+220)
39. A. Ali, Y. Zhang, P. Cardinal, N. Dahak, S. Vogel, J. Glass, A complete kaldi recipe for building arabic speech recognition systems, in: IEEE spoken language technology workshop (SLT), IEEE, 2014, pp. 525–529. [find](https://scholar.google.com/scholar?q=A%2E+Ali%2C+Y%2E+Zhang%2C+P%2E+Cardinal%2C+N%2E+Dahak%2C+S%2E+Vogel%2C+J%2E+Glass%2C+A+complete+kaldi+recipe+for+building+arabic+speech+recognition+systems%2C+in%3A+IEEE+spoken+language+technology+workshop+%28SLT%29%2C+IEEE%2C+2014%2C+pp%2E)
40. T. Ko, V. Peddinti, D. Povey, S. Khudanpur, Audio augmentation for speech recognition, in: Sixteenth Annual Conference of the International Speech Communication Association, 2015. [find](https://scholar.google.com/scholar?q=T%2E+Ko%2C+V%2E+Peddinti%2C+D%2E+Povey%2C+S%2E+Khudanpur%2C+Audio+augmentation+for+speech+recognition%2C+in%3A+Sixteenth+Annual+Conference+of+the+International+Speech+Communication+Association%2C+2015%2E)
41. D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, Q. V. Le, Specaugment: A simple data augmentation method for automatic speech recognition, Proc. Interspeech (2019) 2613–2617. [find](https://scholar.google.com/scholar?q=D%2E+S%2E+Park%2C+W%2E+Chan%2C+Y%2E+Zhang%2C+C%2E%2DC%2E+Chiu%2C+B%2E+Zoph%2C+E%2E+D%2E+Cubuk%2C+Q%2E+V%2E+Le%2C+Specaugment%3A+A+simple+data+augmentation+method+for+automatic+speech+recognition%2C+Proc%2E+Interspeech+%282019%29+2613%E2%80%932617%2E) [our card](https://md.research.elamur.ai/free/en/papers/park-2019-specaugment-simple-data-augmentation-method/)
42. R. Pappagari, P. Zelasko, J. Villalba, Y. Carmiel, N. Dehak, Hierarchical transformers for long document classification, in: IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), IEEE, 2019, pp. 838– 844. [find](https://scholar.google.com/scholar?q=R%2E+Pappagari%2C+P%2E+Zelasko%2C+J%2E+Villalba%2C+Y%2E+Carmiel%2C+N%2E+Dehak%2C+Hierarchical+transformers+for+long+document+classification%2C+in%3A+IEEE+Automatic+Speech+Recognition+and+Understanding+Workshop+%28ASRU%29%2C+IEEE%2C+)
43. A. Zeyer, K. Irie, R. Schlüter, H. Ney, Improved training of end-to-end attention models for speech recognition, arXiv preprint arXiv:1805.03294. [↗](https://arxiv.org/abs/1805.03294)