---
title: "HebDB: a Weakly Supervised Dataset for — Turetzky 2024"
description: "HebDB is a speech corpus of roughly 2,500 hours of spontaneous Hebrew: podcasts on many topics plus testimonies of World War II survivors, released both…"
authors: ["Arnon Turetzky", "Or Tal", "Yael Segal-Feldman", "Yehoshua Dissen", "Ella Zeldes", "Amit Roth", "Eyal Cohen", "Yosi Shrem", "Bronya R. Chernyak", "Olga Seleznova", "Joseph Keshet", "Yossi Adi"]
year: 2024
venue: "arXiv:2407.07566"
source: "https://arxiv.org/abs/2407.07566"
arxiv: "2407.07566"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/langstream-2024-turetzky-hebdb-weakly-supervised-dataset-hebrew/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/langstream-2024-turetzky-hebdb-weakly-supervised-dataset-hebrew/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/) / HebDB: a Weakly Supervised Dataset for Hebrew Speech Processing

# HebDB: a Weakly Supervised Dataset for Hebrew Speech Processing

**Authors:** [Arnon Turetzky](https://md.research.elamur.ai/free/en/authors/arnon-turetzky/); [Or Tal](https://md.research.elamur.ai/free/en/authors/or-tal/); Yael Segal-Feldman; Yehoshua Dissen; [Ella Zeldes](https://md.research.elamur.ai/free/en/authors/ella-zeldes/); [Amit Roth](https://md.research.elamur.ai/free/en/authors/amit-roth/); Eyal Cohen; Yosi Shrem; Bronya R. Chernyak; Olga Seleznova; Joseph Keshet; [Yossi Adi](https://md.research.elamur.ai/free/en/authors/yossi-adi/) · **Year:** 2024 · **Venue:** arXiv:2407.07566 **Link:** [arxiv.org/abs/2407.07566](https://arxiv.org/abs/2407.07566) · library `langstream-2024-turetzky-hebdb-weakly-supervised-dataset-hebrew` · **Licence:** [arXiv — non-exclusive license to distribute](https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html) (scope: readonly) — summary only

[**2** citations](https://openalex.org/works?filter=cites:W4402112439) OpenAlex · 2026-10-14 · [**4** citations in our library](https://md.research.elamur.ai/free/en/papers/langstream-2024-turetzky-hebdb-weakly-supervised-dataset-hebrew/cited-by/) of the 140673 papers whose bibliographies we read · [arXiv:2407.07566](https://openalex.org/S4393918409) · [licence](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

- [Publisher's page](https://arxiv.org/abs/2407.07566)
- [PDF](https://arxiv.org/pdf/2407.07566)
- [arXiv 2407.07566](https://arxiv.org/abs/2407.07566)
- [OpenAlex](https://openalex.org/W4402112439)

**Authors (12)**

1. [Arnon Turetzky](https://openalex.org/A5057757909) [Semantic Scholar](https://www.semanticscholar.org/author/14055369) · h 2 · 10 citations · 6 works
2. [Or Tal](https://openalex.org/A5013091852) [Semantic Scholar](https://www.semanticscholar.org/author/2070494536) · h 4 · 168 citations · 15 works
3. Yael Segal-Feldman
4. [Yehoshua Dissen](https://openalex.org/A5035675810) [ORCID](https://orcid.org/0000-0002-9380-8169) · [Semantic Scholar](https://www.semanticscholar.org/author/8505790) · h 4 · 63 citations · 12 works
5. [Ella Zeldes](https://openalex.org/A5104431190) [Semantic Scholar](https://www.semanticscholar.org/author/2310607864) · h 1 · 2 citations · 2 works
6. [Amit Roth](https://openalex.org/A5107234710) [Semantic Scholar](https://www.semanticscholar.org/author/2310611793) · h 2 · 11 citations · 6 works
7. [Eyal Cohen](https://openalex.org/A5018697006) [ORCID](https://orcid.org/0000-0001-7501-3517) · [Semantic Scholar](https://www.semanticscholar.org/author/2161210076) · h 2 · 28 citations · 7 works
8. [Yosi Shrem](https://openalex.org/A5076089159) [Semantic Scholar](https://www.semanticscholar.org/author/1390053733) · h 3 · 21 citations · 11 works
9. [Bronya R. Chernyak](https://openalex.org/A5089375146) [Semantic Scholar](https://www.semanticscholar.org/author/2053819200) · h 3 · 40 citations · 12 works
10. [Olga Seleznova](https://openalex.org/A5093613381) [Semantic Scholar](https://www.semanticscholar.org/author/2284056962) · h 2 · 29 citations · 3 works
11. [Joseph Keshet](https://openalex.org/A5008847407) [ORCID](https://orcid.org/0000-0003-2332-5783) · [Semantic Scholar](https://www.semanticscholar.org/author/2265181863) · h 30 · 4322 citations · 146 works
12. [Yossi Adi](https://openalex.org/A5005191803) [ORCID](https://orcid.org/0000-0003-2237-3898) · [Semantic Scholar](https://www.semanticscholar.org/author/2727584) · h 31 · 4543 citations · 160 works

Citations and author metrics: OpenAlex, 2026-10-14; authors with the same name are not always told apart.

## What it is about

HebDB is a speech corpus of roughly 2,500 hours of spontaneous Hebrew: podcasts on many topics plus testimonies of World War II survivors, released both raw and pre-processed. The pipeline resamples to 16 kHz mono, cuts speech with a voice activity detector, and transcribes each segment with Whisper large-v2; text with non-Hebrew characters is dropped, numbers and dates are spelled out, and final letters are folded. A forced aligner (MMS, on romanised and diacritised text) gives each utterance a confidence score; a threshold of 0.3 keeps about 1,470 of 1,690 hours. Two baselines are trained on it, HuBERT (95M) and a Conformer (110M). On the FLEURS Hebrew test set, the filtered HuBERT with a language model reaches 33.6 WER, the Conformer 41.9, Whisper 1.55B without a language model 27.1, and MMS (1,107 languages) with one 40.0.

## Ideas

- I1. Keep a confidence score per item and train only on items above a threshold. Any weakly labelled Hebrew source that Mova might use (automatic transcripts, auto-parsed text) can be gated the same way, and the gate is tested on a gold subset. The gain here is small (HuBERT 33.9 to 33.6, Conformer 43.0 to 41.9 with LM), so the gate must be measured, not assumed.
- I2. Deterministic normalisation before the labels are used: spell out numbers and dates, drop items with off-script characters (about 1% of samples in this paper), and fold final letters to their regular forms. The paper found the final-letter folding helpful. Each rule should be a counted test: how many items it changes on a sample, and whether the change helps.
- I3. Report a yield table per source (hours kept against hours recorded). In this corpus one source fell from 492 h to 67.4 h after processing, which shows that source quality varies a lot. Mova should log the same yield for every new data source, so that a bad source is visible before it reaches training.

## For Mova

I1 is a rule for the weak-data side of [languages/he/11-ud-analysis.md](https://mova.elamur.ai/free/en/docs/languages/he/11-ud-analysis/) (how automatically produced trees are trusted). I2 goes into [languages/he/02-script-and-spelling.md](https://mova.elamur.ai/free/en/docs/languages/he/02-script-and-spelling/) (final-letter folding as a test) and into the script filter of `tools/langseg`. I3 is a reporting habit for every corpus we build. The paper's speech models are outside Mova's scope, which parses text only.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [arXiv 2407.07566](https://arxiv.org/abs/2407.07566)

```bibtex
@misc{turetzky2024hebdb,
  title = {{HebDB: a Weakly Supervised Dataset for Hebrew Speech Processing}},
  author = {Turetzky, Arnon and Tal, Or and Segal-Feldman, Yael and Dissen, Yehoshua and Zeldes, Ella and Roth, Amit and Cohen, Eyal and Shrem, Yosi and Chernyak, Bronya R. and Seleznova, Olga and Keshet, Joseph and Adi, Yossi},
  year = {2024},
  eprint = {2407.07566},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.07566}
}
```

- [← previous HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://md.research.elamur.ai/free/en/papers/mazeika-2024-harmbench-standardized-evaluation-framework-automated/)
- [next → HeSum: a Novel Dataset for Abstractive Text Summarization in Hebrew](https://md.research.elamur.ai/free/en/papers/findings-2024-argaman-hesum-novel-dataset-abstractive-text/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/)

## References

36 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12 449–12 460, 2020. [find](https://scholar.google.com/scholar?q=A%2E+Baevski%2C+Y%2E+Zhou%2C+A%2E+Mohamed%2C+and+M%2E+Auli%2C+%E2%80%9Cwav2vec+2%2E0%3A+A+framework+for+self%2Dsupervised+learning+of+speech+representations%2C%E2%80%9D+Advances+in+neural+information+processing+systems%2C+vol%2E+33%2C+pp%2E+12+449%E2%80%93) [our card](https://md.research.elamur.ai/free/en/papers/baevski-2020-wav2vec-2-0-framework-self/)
2. P. K. Rubenstein, C. Asawaroengchai, D. D. Nguyen, A. Bapna, Z. Borsos, F. d. C. Quitry, P. Chen, D. E. Badawy, W. Han, E. Kharitonov et al., “Audiopalm: A large language model that can speak and listen,” arXiv preprint arXiv:2306.12925, 2023. [↗](https://arxiv.org/abs/2306.12925) [our card](https://md.research.elamur.ai/free/en/papers/rubenstein-2023-audiopalm-large-language-model-that/)
3. C. Wang, Y. Wu, Y. Qian, K. Kumatani, S. Liu, F. Wei, M. Zeng, and X. Huang, “Unispeech: Unified speech representation learning with labeled and unlabeled data,” in International Conference on Machine Learning. PMLR, 2021, pp. 10 937–10 947. [↗](https://doi.org/10.48550/arxiv.2101.07597)
4. V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2015, pp. 5206–5210. [find](https://scholar.google.com/scholar?q=V%2E+Panayotov%2C+G%2E+Chen%2C+D%2E+Povey%2C+and+S%2E+Khudanpur%2C+%E2%80%9CLibrispeech%3A+an+asr+corpus+based+on+public+domain+audio+books%2C%E2%80%9D+in+2015+IEEE+international+conference+on+acoustics%2C+speech+and+signal+processing+%28IC)
5. J. Kahn, M. Rivière, W. Zheng, E. Kharitonov, Q. Xu, P.-E. Mazaré, J. Karadayi, V. Liptchinsky, R. Collobert, C. Fuegen et al., “Libri-light: A benchmark for asr with limited or no supervision,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 7669–7673. [find](https://scholar.google.com/scholar?q=J%2E+Kahn%2C+M%2E+Rivie%CC%80re%2C+W%2E+Zheng%2C+E%2E+Kharitonov%2C+Q%2E+Xu%2C+P%2E%2DE%2E+Mazare%CC%81%2C+J%2E+Karadayi%2C+V%2E+Liptchinsky%2C+R%2E+Collobert%2C+C%2E+Fuegen+et+al%2E%2C+%E2%80%9CLibri%2Dlight%3A+A+benchmark+for+asr+with+limited+or+no+supervision%2C%E2%80%9D+in+)
6. C. Wang, M. Riviere, A. Lee, A. Wu, C. Talnikar, D. Haziza, M. Williamson, J. Pino, and E. Dupoux, “Voxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,” arXiv preprint arXiv:2101.00390, 2021. [↗](https://arxiv.org/abs/2101.00390) [our card](https://md.research.elamur.ai/free/en/papers/wang-2021-voxpopuli-large-scale-multilingual-speech/)
7. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in International Conference on Machine Learning. PMLR, 2023, pp. 28 492–28 518. [↗](https://doi.org/10.48550/arxiv.2212.04356) [our card](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/)
8. V. Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi et al., “Scaling speech technology to 1,000+ languages,” arXiv preprint arXiv:2305.13516, 2023. [↗](https://arxiv.org/abs/2305.13516) [our card](https://md.research.elamur.ai/free/en/papers/pratap-2023-scaling-speech-technology-1-000/)
9. Y. Zhang, W. Han, J. Qin, Y. Wang, A. Bapna, Z. Chen, N. Chen, B. Li, V. Axelrod, G. Wang et al., “Google usm: Scaling automatic speech recognition beyond 100 languages,” arXiv preprint arXiv:2303.01037, 2023. [↗](https://arxiv.org/abs/2303.01037)
10. C. Wang, S. Chen, Y. Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y. Liu, H. Wang, J. Li et al., “Neural codec language models are zero-shot text to speech synthesizers,” arXiv preprint arXiv:2301.02111, 2023. [↗](https://arxiv.org/abs/2301.02111) [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-neural-codec-language-models-zero/)
11. M. Le, A. Vyas, B. Shi, B. Karrer, L. Sari, R. Moritz, M. Williamson, V. Manohar, Y. Adi, J. Mahadeokar et al., “Voicebox: Text-guided multilingual universal speech generation at scale,” Advances in neural information processing systems, vol. 36, 2024. [find](https://scholar.google.com/scholar?q=M%2E+Le%2C+A%2E+Vyas%2C+B%2E+Shi%2C+B%2E+Karrer%2C+L%2E+Sari%2C+R%2E+Moritz%2C+M%2E+Williamson%2C+V%2E+Manohar%2C+Y%2E+Adi%2C+J%2E+Mahadeokar+et+al%2E%2C+%E2%80%9CVoicebox%3A+Text%2Dguided+multilingual+universal+speech+generation+at+scale%2C%E2%80%9D+Advances+in+n)
12. C. K. Reddy, H. Dubey, V. Gopal, R. Cutler, S. Braun, H. Gamper, R. Aichner, and S. Srinivasan, “Icassp 2021 deep noise suppression challenge,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 6623–6627. [find](https://scholar.google.com/scholar?q=C%2E+K%2E+Reddy%2C+H%2E+Dubey%2C+V%2E+Gopal%2C+R%2E+Cutler%2C+S%2E+Braun%2C+H%2E+Gamper%2C+R%2E+Aichner%2C+and+S%2E+Srinivasan%2C+%E2%80%9CIcassp+2021+deep+noise+suppression+challenge%2C%E2%80%9D+in+ICASSP+2021%2D2021+IEEE+International+Conference+on+Aco)
13. A. Defossez, G. Synnaeve, and Y. Adi, “Real time speech enhancement in the waveform domain,” arXiv preprint arXiv:2006.12847, 2020. [↗](https://arxiv.org/abs/2006.12847)
14. Y. Dissen, F. Kreuk, and J. Keshet, “Self-supervised Speaker Diarization,” in Proc. Interspeech 2022, 2022, pp. 4013–4017. [find](https://scholar.google.com/scholar?q=Y%2E+Dissen%2C+F%2E+Kreuk%2C+and+J%2E+Keshet%2C+%E2%80%9CSelf%2Dsupervised+Speaker+Diarization%2C%E2%80%9D+in+Proc%2E+Interspeech+2022%2C+2022%2C+pp%2E+4013%E2%80%934017%2E)
15. H. Yadav and S. Sitaram, “A survey of multilingual models for automatic speech recognition,” arXiv preprint arXiv:2202.12576, 2022. [↗](https://arxiv.org/abs/2202.12576)
16. T. Likhomanenko, Q. Xu, V. Pratap, P. Tomasello, J. Kahn, G. Avidov, R. Collobert, and G. Synnaeve, “Rethinking evaluation in asr: Are our models robust enough?” arXiv preprint arXiv:2010.11745, 2020. [↗](https://arxiv.org/abs/2010.11745)
17. L. Campbell, “Ethnologue: Languages of the world,” 2008. [find](https://scholar.google.com/scholar?q=L%2E+Campbell%2C+%E2%80%9CEthnologue%3A+Languages+of+the+world%2C%E2%80%9D+2008%2E)
18. A. Petrov, E. La Malfa, P. Torr, and A. Bibi, “Language model tokenizers introduce unfairness between languages,” Advances in Neural Information Processing Systems, vol. 36, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Petrov%2C+E%2E+La+Malfa%2C+P%2E+Torr%2C+and+A%2E+Bibi%2C+%E2%80%9CLanguage+model+tokenizers+introduce+unfairness+between+languages%2C%E2%80%9D+Advances+in+Neural+Information+Processing+Systems%2C+vol%2E+36%2C+2024%2E)
19. A. W. Black, “Cmu wilderness multilingual speech dataset,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 5971– 5975. [find](https://scholar.google.com/scholar?q=A%2E+W%2E+Black%2C+%E2%80%9CCmu+wilderness+multilingual+speech+dataset%2C%E2%80%9D+in+ICASSP+2019%2D2019+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2E+IEEE%2C+2019%2C+pp%2E+5971%E2%80%93+5975%2E)
20. V. Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “Mls: A large-scale multilingual dataset for speech research,” arXiv preprint arXiv:2012.03411, 2020. [↗](https://arxiv.org/abs/2012.03411) [our card](https://md.research.elamur.ai/free/en/papers/pratap-2020-mls-large-scale-multilingual-dataset/)
21. A. Conneau, M. Ma, S. Khanuja, Y. Zhang, V. Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna, “Fleurs: Few-shot learning evaluation of universal representations of speech,” in 2022 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2023, pp. 798–805. [find](https://scholar.google.com/scholar?q=A%2E+Conneau%2C+M%2E+Ma%2C+S%2E+Khanuja%2C+Y%2E+Zhang%2C+V%2E+Axelrod%2C+S%2E+Dalmia%2C+J%2E+Riesa%2C+C%2E+Rivera%2C+and+A%2E+Bapna%2C+%E2%80%9CFleurs%3A+Few%2Dshot+learning+evaluation+of+universal+representations+of+speech%2C%E2%80%9D+in+2022+IEEE+Spoken+La) [our card](https://md.research.elamur.ai/free/en/papers/conneau-2022-fleurs-few-shot-learning-evaluation/)
22. S. Izre’el, B. Hary, and G. Rahav, “Designing cosih: the corpus of spoken israeli hebrew,” International Journal of Corpus Linguistics, vol. 6, no. 2, pp. 171–197, 2001. [↗](https://doi.org/10.1075/ijcl.6.2.01izr)
23. J. Azogui, A. Lerner, and V. Silber-Varod, “The open university of israel map task corpus (matacop),” 2016. [find](https://scholar.google.com/scholar?q=J%2E+Azogui%2C+A%2E+Lerner%2C+and+V%2E+Silber%2DVarod%2C+%E2%80%9CThe+open+university+of+israel+map+task+corpus+%28matacop%29%2C%E2%80%9D+2016%2E)
24. M. Marmorstein and N. Matalon, “The huji corpus of spoken hebrew: An interaction-oriented design of a corpus,” 2022. [↗](https://doi.org/10.21203/rs.3.rs-2111907/v1)
25. O. Sharoni, R. Shenberg, and E. Cooper, “Saspeech: A hebrew single speaker dataset for text to speech and voice conversion,” in Proc. Interspeech, 2023. [find](https://scholar.google.com/scholar?q=O%2E+Sharoni%2C+R%2E+Shenberg%2C+and+E%2E+Cooper%2C+%E2%80%9CSaspeech%3A+A+hebrew+single+speaker+dataset+for+text+to+speech+and+voice+conversion%2C%E2%80%9D+in+Proc%2E+Interspeech%2C+2023%2E)
26. Y. Marmor, K. Misgav, and Y. Lifshitz, “ivrit. ai: A comprehensive dataset of hebrew speech for ai research and development,” arXiv preprint arXiv:2307.08720, 2023. [↗](https://arxiv.org/abs/2307.08720) [our card](https://md.research.elamur.ai/free/en/papers/langstream-2023-marmor-ivrit-ai-comprehensive-dataset-hebrew/)
27. S. Team, “Silero vad: pre-trained enterprise-grade voice activity detector (vad), number detector and language classifier,” 2021. [find](https://scholar.google.com/scholar?q=S%2E+Team%2C+%E2%80%9CSilero+vad%3A+pre%2Dtrained+enterprise%2Dgrade+voice+activity+detector+%28vad%29%2C+number+detector+and+language+classifier%2C%E2%80%9D+2021%2E)
28. J. Sohn, N. S. Kim, and W. Sung, “A statistical model-based voice activity detection,” IEEE signal processing letters, vol. 6, no. 1, pp. 1–3, 1999. [↗](https://doi.org/10.1109/97.736233)
29. V. Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi et al., “Scaling speech technology to 1,000+ languages,” arXiv preprint arXiv:2305.13516, 2023. [↗](https://arxiv.org/abs/2305.13516) [our card](https://md.research.elamur.ai/free/en/papers/pratap-2023-scaling-speech-technology-1-000/)
30. W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3451–3460, 2021. [↗](https://doi.org/10.1109/taslp.2021.3122291) [our card](https://md.research.elamur.ai/free/en/papers/hsu-2021-hubert-self-supervised-speech-representation/)
31. A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu et al., “Conformer: Convolution-augmented transformer for speech recognition,” arXiv preprint arXiv:2005.08100, 2020. [↗](https://arxiv.org/abs/2005.08100) [our card](https://md.research.elamur.ai/free/en/papers/gulati-2020-conformer-convolution-augmented-transformer-speech/)
32. M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli, “fairseq: A fast, extensible toolkit for sequence modeling,” arXiv preprint arXiv:1904.01038, 2019. [↗](https://arxiv.org/abs/1904.01038) [our card](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
33. A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Proceedings of the 23rd international conference on Machine learning, 2006, pp. 369–376. [find](https://scholar.google.com/scholar?q=A%2E+Graves%2C+S%2E+Ferna%CC%81ndez%2C+F%2E+Gomez%2C+and+J%2E+Schmidhuber%2C+%E2%80%9CConnectionist+temporal+classification%3A+labelling+unsegmented+sequence+data+with+recurrent+neural+networks%2C%E2%80%9D+in+Proceedings+of+the+23rd+internat)
34. A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, and R. Pang, “Conformer: Convolution-augmented Transformer for Speech Recognition,” in Proc. Interspeech 2020, 2020, pp. 5036–5040. [find](https://scholar.google.com/scholar?q=A%2E+Gulati%2C+J%2E+Qin%2C+C%2E%2DC%2E+Chiu%2C+N%2E+Parmar%2C+Y%2E+Zhang%2C+J%2E+Yu%2C+W%2E+Han%2C+S%2E+Wang%2C+Z%2E+Zhang%2C+Y%2E+Wu%2C+and+R%2E+Pang%2C+%E2%80%9CConformer%3A+Convolution%2Daugmented+Transformer+for+Speech+Recognition%2C%E2%80%9D+in+Proc%2E+Interspeech+20) [our card](https://md.research.elamur.ai/free/en/papers/gulati-2020-conformer-convolution-augmented-transformer-speech/)
35. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. [find](https://scholar.google.com/scholar?q=A%2E+Vaswani%2C+N%2E+Shazeer%2C+N%2E+Parmar%2C+J%2E+Uszkoreit%2C+L%2E+Jones%2C+A%2E+N%2E+Gomez%2C+%C5%81%2E+Kaiser%2C+and+I%2E+Polosukhin%2C+%E2%80%9CAttention+is+all+you+need%2C%E2%80%9D+Advances+in+neural+information+processing+systems%2C+vol%2E+30%2C+2017%2E)
36. C. Kim, D. Gowda, D. Lee, J. Kim, A. Kumar, S. Kim, A. Garg, and C. Han, “A review of on-device fully neural end-to-end automatic speech recognition algorithms,” in 2020 54th Asilomar Conference on Signals, Systems, and Computers. IEEE, 2020, pp. 277–283. [find](https://scholar.google.com/scholar?q=C%2E+Kim%2C+D%2E+Gowda%2C+D%2E+Lee%2C+J%2E+Kim%2C+A%2E+Kumar%2C+S%2E+Kim%2C+A%2E+Garg%2C+and+C%2E+Han%2C+%E2%80%9CA+review+of+on%2Ddevice+fully+neural+end%2Dto%2Dend+automatic+speech+recognition+algorithms%2C%E2%80%9D+in+2020+54th+Asilomar+Conference+on+Si)