---
title: "Octopus: Towards Building the Arabic Speech LLM Suite (2025)"
description: "The authors build Octopus, a family of Arabic-centric speech LLMs that handle ASR (Arabic, English… 29 references, cited by 2 papers here."
authors: ["Sara Althubaiti", "Vasista Sai Lodagala", "Tjad Clark", "Yousseif Ahmed Elshahawy", "Daniel Izham", "Abdullah Alrajeh", "Aljawahrah Bin Tamran", "Ahmed Ali"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2025.arabicnlp-main.35/"
doi: "10.18653/v1/2025.arabicnlp-main.35"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/althubaiti-2025-octopus-building-arabic-speech-llm/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/althubaiti-2025-octopus-building-arabic-speech-llm/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/14/) / Octopus: Towards Building the Arabic Speech LLM Suite

# Octopus: Towards Building the Arabic Speech LLM Suite

**Authors:** Sara Althubaiti, Vasista Sai Lodagala, Tjad Clark, [Yousseif Ahmed Elshahawy](https://md.research.elamur.ai/free/en/authors/yousseif-ahmed-elshahawy/), [Daniel Izham](https://md.research.elamur.ai/free/en/authors/daniel-izham/), Abdullah Alrajeh, Aljawahrah Bin Tamran and [Ahmed Ali](https://md.research.elamur.ai/free/en/authors/ahmed-m-abdelatty-ali/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-main.35/) · [PDF](https://aclanthology.org/2025.arabicnlp-main.35.pdf) · [DOI 10.18653/v1/2025.arabicnlp-main.35](https://doi.org/10.18653/v1/2025.arabicnlp-main.35) · [OpenAlex](https://openalex.org/W4416037838)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/althubaiti-2025-octopus-building-arabic-speech-llm/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic speech is hard for general audio-language models because of its many dialects, rich morphology, inconsistent spelling and frequent mixing with English. The authors build Octopus, a family of Arabic-centric speech LLMs that handle ASR (Arabic, English and code-switched), Arabic-to-English speech translation, and dialect identification over 17 dialects. The design follows SALMONN. Two frozen audio encoders feed a frozen LLM: Whisper-V3 supplies semantic features and BEATs supplies acoustic-event features. The LLM is ALLaM-13B, LLaMA or DeepSeek. Only a Q-Former, a linear projection and LoRA adapters are trained, using a progressive multitask schedule that starts with ASR. TinyOctopus is a smaller version that pairs Distil-Whisper with LLaMA3-1B or DeepSeek-1.5B. Translation targets were produced by GPT-4o from transcripts and then filtered. The best translation model, Trans_Octopus, beats Whisper-large-v3 and SeamlessM4T on CoVoST2 and FLEURS. The small models reach close to 89% dialect-ID accuracy on ADI17. The work shows that compact, modular systems with frozen components can serve under-resourced speech settings. The models and code are released.

## Key points
- Only the Q-Former, projection layer and LoRA weights are updated during training; the Whisper/BEATs encoders and the base LLM stay frozen.
- The full Octopus models saw about 13,400 hours of in-house Arabic speech, while TinyOctopus saw about 1,900 hours from QASR. Adding the in-house data to TinyOctopus_LLaMA3-1B cut long-form WER | CER from 44.3 | 29.1 to 24.9 | 13.1.
- When GPT-4o translations were filtered for untranslated Arabic leftovers and hallucinations, about 43.25% of in-house segments were dropped, compared with only 1.17% of QASR segments.
- Trans_Octopus reaches BLEU 38.6 (CoVoST2) and 23.2 (FLEURS), with BERTScore F1 of 0.64 and 0.58. On ADI17 the TinyOctopus variants reach 87.4% (LLaMA3-1B) and 88.7% (DeepSeek-1.5B).
- The test suite covers:
  - MGB2, LibriSpeech, TEDLIUM, ESCWA and Mixat-All;
  - Mixat-CS, a 3.15 h pure code-switched subset of Mixat-All;
  - a 25.33 h in-house set of long recordings, 8–10 minutes each.

## Interesting ideas
- Large pretrained parts stay frozen, and only a thin bridge (query transformer, projection, low-rank adapters) is trained between them, so new tasks or backbones can be swapped in cheaply.
- Two complementary encoders, one for meaning and one for low-level signal, are fused through the same alignment module instead of relying on a single representation.
- Synthetic labels from an LLM are checked for specific failure types (untranslated residue, unrelated hallucinated content), and the rejection rate is reported per source corpus, which exposes how reliable each source is.
- When references are themselves synthetic, systems are compared pairwise, with each one's output used in turn as the reference for the others.
- Training is progressive: it starts with the most plentiful and most monotonic task (ASR) before harder, non-monotonic ones like translation are added. A pure code-switched slice is also cut out of a mixed test set to isolate the hard phenomenon.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): gparse could keep its transformer encoder fixed and train only small adapters per treebank or UD dialect, which would make adding a new source cheap and easy to compare.
- **I2** (tokens): the tagger and lemmatizer could combine a word-level view with a separate character/morphology view through an explicit, inspectable merge step.
- **I3** (learn): the LLM teacher tiers could log per-corpus rejection rates of silver frames or trees, so that sources that confuse the teacher get flagged.
- **I4** (learn): when there is no gold data, a witness-parser agreement matrix built this way could show which parser is the outlier before votes are counted for silver data.
- **I5** (answer): the readers could be trained on simple literal QA first, then on tasks that need inference, and scored separately on extracted subsets of hard constructions such as irony or multi-step arithmetic.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 519 works

## Cite

Cite as: Sara Althubaiti, Vasista Sai Lodagala, Tjad Clark, Yousseif Ahmed Elshahawy, Daniel Izham, Abdullah Alrajeh, Aljawahrah Bin Tamran and Ahmed Ali. 2025. Octopus: Towards Building the Arabic Speech LLM Suite. Proceedings of The Third Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{althubaiti2025octopus,
  title = {{Octopus: Towards Building the Arabic Speech LLM Suite}},
  author = {Sara Althubaiti, Vasista Sai Lodagala, Tjad Clark, Yousseif Ahmed Elshahawy, Daniel Izham, Abdullah Alrajeh, Aljawahrah {Bin Tamran} and Ahmed Ali},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-main.35},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-main.35}
}
```

- [← previous NU_Internship team at ImageEval 2025: From Zero-Shot to Ensembles: Enhancing Grounded Arabic Image Captioning](https://md.research.elamur.ai/free/en/papers/gaber-2025-nu-internship-team-imageeval-2025/)
- [next → Optimizing RAG Pipelines for Arabic: A Systematic Analysis of Core Components](https://md.research.elamur.ai/free/en/papers/alsubhi-2025-optimizing-rag-pipelines-arabic-systematic/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/14/)

## References

29 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Milad Abdollahzadeh, Touba Malekzadeh, and Ngai-Man Man Cheung. 2021. Revisit multimodal meta-learning through the lens of multi-task learning. Advances in Neural Information Processing Systems, 34:14632–14644. [↗](https://doi.org/10.48550/arxiv.2110.14202)
2. Ahmed Ali, Peter Bell, James Glass, Yacine Messaoui, Hamdy Mubarak, Steve Renals, and Yifan Zhang. 2016. [The mgb-2 challenge: Arabic multi-dialect broadcast media recognition](https://md.research.elamur.ai/free/en/papers/ali-2016-mgb-2-challenge-arabic-multi/). In 2016 IEEE Spoken Language Technology Workshop (SLT), pages 279– 284. IEEE.
3. Maryam Al Ali and Hanan Aldarmaki. 2024. [Mixat: A data set of bilingual emirati-english speech](https://md.research.elamur.ai/free/en/papers/ali-2024-mixat-data-set-bilingual-emirati/). arXiv preprint arXiv:2405.02578. [↗](https://arxiv.org/abs/2405.02578)
4. Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. [Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes](https://md.research.elamur.ai/free/en/papers/hsieh-2023-distilling-step-step-outperforming-larger/). arXiv preprint arXiv:2305.02301. [↗](https://arxiv.org/abs/2305.02301)
5. M Saiful Bari, Yazeed Alnumay, Norah A Alzahrani, Nouf M Alotaibi, Hisham A Alyahya, Sultan Al-Rashed, Faisal A Mirza, Shaykhah Z Alsubaie, Hassan A Alahmed, Ghadah Alabduljabbar, and 1 others. 2024. [Allam: Large language models for arabic and english](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/). arXiv preprint arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390)
6. Zhifeng Kong, Arushi Goel, Rohan Badlani, Wei Ping, Rafael Valle, and Bryan Catanzaro. 2024. Audio flamingo: A novel audio language model with few-shot learning and dialogue abilities. Proceedings of the 41st International Conference on Machine Learning (ICML). [↗](https://doi.org/10.48550/arxiv.2402.01831)
7. Chen Chen, Yuchen Hu, Siyin Wang, Helin Wang, Zhehuai Chen, Chao Zhang, Chao-Han Huck Yang, and Eng Siong Chng. 2025. Audio large language models can be descriptive speech quality evaluators. arXiv preprint arXiv:2501.17126. [↗](https://arxiv.org/abs/2501.17126)
8. Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Daniel Tompkins, Zhuo Chen, and Furu Wei. 2022. Beats: Audio pre-training with acoustic tokenizers. arXiv preprint arXiv:2212.09058. [↗](https://arxiv.org/abs/2212.09058)
9. Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna. 2022. [Fleurs: Few-shot learning evaluation of universal representations of speech](https://md.research.elamur.ai/free/en/papers/conneau-2022-fleurs-few-shot-learning-evaluation/). arXiv preprint arXiv:2205.12446. [↗](https://arxiv.org/abs/2205.12446)
10. Yassir Fathullah, Chunyang Wu, Egor Lakomkin, Ke Li, Junteng Jia, Yuan Shangguan, Jay Mahadeokar, Ozlem Kalinli, Christian Fuegen, and Mike Seltzer. 2024. Audiochatllama: Towards general-purpose speech abilities for llms. NAACL. [↗](https://doi.org/10.48550/arxiv.2311.06753)
11. Sanchit Gandhi, Patrick von Platen, and Alexander M Rush. 2023. Distil-whisper: Robust knowledge distillation via large-scale pseudo labelling. arXiv preprint arXiv:2311.00430. [↗](https://arxiv.org/abs/2311.00430)
12. Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, S Sakshi, Oriol Nieto, and Ramani Duraiswami. 2024. [GAMA: A large audio-language model with advanced audio understanding and complex reasoning abilities](https://md.research.elamur.ai/free/en/papers/ghosh-2024-gama-large-audio-language-model/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6288–6313. [↗](https://doi.org/10.48550/arxiv.2406.11768)
13. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783. [↗](https://arxiv.org/abs/2407.21783)
14. François Hernandez, Vincent Nguyen, Sahar Ghannay, Natalia Tomashenko, and Yannick Esteve. 2018. Ted-lium 3: Twice as much data and corpus repartition for experiments on speaker adaptation. In Speech and Computer: 20th International Conference, SPECOM 2018, Leipzig, Germany, September 18–22, 2018, Proceedings 20, pages 198–208. Springer. [find](https://scholar.google.com/scholar?q=Fran%C3%A7ois+Hernandez%2C+Vincent+Nguyen%2C+Sahar+Ghannay%2C+Natalia+Tomashenko%2C+and+Yannick+Esteve%2E+2018%2E+Ted%2Dlium+3%3A+Twice+as+much+data+and+corpus+repartition+for+experiments+on+speaker+adaptation%2E+In+Speech+)
15. Rao Ma, Mengjie Qian, Yassir Fathullah, Siyuan Tang, Mark Gales, and Kate Knill. 2024. Cross-lingual transfer learning for speech translation. arXiv preprint arXiv:2407.01130. [↗](https://arxiv.org/abs/2407.01130)
16. Hamdy Mubarak, Amir Hussein, Shammur Absar Chowdhury, and Ahmed Ali. 2021. [Qasr: Qcri aljazeera speech resource–a large scale annotated arabic speech corpus](https://md.research.elamur.ai/free/en/papers/mubarak-2021-qasr-qcri-aljazeera-speech-resource/). arXiv preprint arXiv:2106.13000. [↗](https://arxiv.org/abs/2106.13000)
17. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. [Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/). In International conference on machine learning, pages 28492–28518. PMLR. [↗](https://doi.org/10.48550/arxiv.2212.04356)
18. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. [Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/). arXiv preprint arXiv:1910.01108. [↗](https://arxiv.org/abs/1910.01108)
19. Weiqiao Shan, Yuang Li, Yuhao Zhang, Yingfeng Luo, Chen Xu, Xiaofeng Zhao, Long Meng, Yunfei Lu, Min Zhang, Hao Yang, Tong Xiao, and Jingbo Zhu. 2025. Enhancing speech large language models with prompt-aware mixture of audio encoders. arXiv preprint arXiv:2502.10098. [↗](https://arxiv.org/abs/2502.10098)
20. Suwon Shon, Ahmed Ali, Younes Samih, Hamdy Mubarak, and James Glass. 2020. Adi17: A fine-grained arabic dialect identification dataset. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8244–8248. IEEE. [find](https://scholar.google.com/scholar?q=Suwon+Shon%2C+Ahmed+Ali%2C+Younes+Samih%2C+Hamdy+Mubarak%2C+and+James+Glass%2E+2020%2E+Adi17%3A+A+fine%2Dgrained+arabic+dialect+identification+dataset%2E+In+ICASSP+2020%2D2020+IEEE+International+Conference+on+Acoustics%2C+)
21. Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. 2023. [Salmonn: Towards generic hearing abilities for large language models](https://md.research.elamur.ai/free/en/papers/tang-2023-salmonn-towards-generic-hearing-abilities/). arXiv preprint arXiv:2310.13289. [↗](https://arxiv.org/abs/2310.13289)
22. Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020. Multilingual translation with extensible multilingual pretraining and finetuning. arXiv preprint arXiv:2008.00401. [↗](https://arxiv.org/abs/2008.00401)
23. Silero Team. 2021. Silero vad: pre-trained enterprise-grade voice activity detector (vad), number detector and language classifier. [https://github.com/](https://github.com/) snakers4/silero-vad. [↗](https://github.com/)
24. Hawau Olamide Toyin, Amirbek Djanibekov, Ajinkya Kulkarni, and Hanan Aldarmaki. 2023. [Artst: Arabic text and speech transformer](https://md.research.elamur.ai/free/en/papers/toyin-2023-artst-arabic-text-speech-transformer/). arXiv preprint arXiv:2310.16621. [↗](https://arxiv.org/abs/2310.16621)
25. Changhan Wang, Anne Wu, and Juan Pino. 2020. [Covost 2: A massively multilingual speech-to-text translation corpus](https://md.research.elamur.ai/free/en/papers/wang-2020-covost-2-massively-multilingual-speech/). Preprint, arXiv:2007.10310. [↗](https://arxiv.org/abs/2007.10310)
26. Dingdong Wang, Jincenzi Wu, Junan Li, Dongchao Yang, Xueyuan Chen, Tianhua Zhang, and Helen Meng. 2025. MMSU: A massive multi-task spoken language understanding and reasoning benchmark. arXiv preprint arXiv:2506.04779. [↗](https://arxiv.org/abs/2506.04779)
27. Chih-Kai Yang, Neo S Ho, and Hung-yi Lee. 2025. [Towards holistic evaluation of large audio-language models: A comprehensive survey](https://md.research.elamur.ai/free/en/papers/yang-2025-holistic-evaluation-large-audio-language/). arXiv preprint arXiv:2505.15957. [↗](https://arxiv.org/abs/2505.15957)
28. Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. 2024. [Tinyllama: An open-source small language model](https://md.research.elamur.ai/free/en/papers/zhang-2024-tinyllama-open-source-small-language/). arXiv preprint arXiv:2401.02385. [↗](https://arxiv.org/abs/2401.02385)
29. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. arXiv preprint arXiv:1604.02201. [↗](https://arxiv.org/abs/1604.02201)

*Updated 2 November 2026*