---
title: "Fine-Tashkeel at KSAA-2026 — paper by Barmandah et al. (2026)"
description: "The paper presents Fine-Tashkeel, a system submitted to the KSAA-2026 shared task under Task 2. 10.56% WER. 17 references."
authors: ["Hassan Barmandah", "Fatimah Emad Eldin", "Omer Nacar", "Wareef Alzubaidi"]
year: 2026
venue: "The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks"
source: "https://aclanthology.org/2026.osact-1.31/"
doi: "10.63317/4rgtj4c7thyj"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/barmandah-2026-fine-tashkeel-ksaa-2026-comprehensive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/barmandah-2026-fine-tashkeel-ksaa-2026-comprehensive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/) / Fine-Tashkeel at KSAA-2026: A Comprehensive Evaluation of Seq2Seq and Multimodal Approaches for Automatic Diacritization of Arabic Speech Dictation

# Fine-Tashkeel at KSAA-2026: A Comprehensive Evaluation of Seq2Seq and Multimodal Approaches for Automatic Diacritization of Arabic Speech Dictation

**Authors:** [Hassan Barmandah](https://md.research.elamur.ai/free/en/authors/hassan-barmandah/), [Fatimah Emad Eldin](https://md.research.elamur.ai/free/en/authors/fatimah-emad-eldin/), [Omer Nacar](https://md.research.elamur.ai/free/en/authors/omer-nacar/) and Wareef Alzubaidi · **Year:** 2026 · **Venue:** [The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks](https://md.research.elamur.ai/free/en/venues/osact/) ([Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks 2026](https://md.research.elamur.ai/free/en/venues/osact/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.osact-1.31/) · [PDF](https://aclanthology.org/2026.osact-1.31.pdf) · [DOI 10.63317/4rgtj4c7thyj](https://doi.org/10.63317/4rgtj4c7thyj) · [OpenAlex](https://openalex.org/W7166515333)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper presents Fine-Tashkeel, a system submitted to the KSAA-2026 shared task under Task 2. The task is restoring full diacritics to Arabic text transcribed from spoken dictation. This is hard because speech transcripts mix MSA with regional dialects and lack punctuation, and an undiacritized word can have several valid readings. The data is the VoiceWall corpus, about five hours of speech from nine nationalities. The rules allowed no external data and no large language models.

The authors frame diacritization as character-level translation: a Seq2Seq Transformer (Al-Rfooh et al., 2023) generates the complete voweled string with greedy decoding. They benchmarked 18 systems in four groups: text-only, classifier-based, ASR+text, and fine-tuned. Their best run used the pretrained model zero-shot. It reached DER 10.56%, WER 34.47% and SER 79.88%, placing 5th of 7 teams.

Off-the-shelf speech models did far worse. The authors attribute this to a mismatch between transcription-oriented decoders and the diacritization task, not to audio being useless. Error analysis shows that dialect coverage, case endings and short-vowel ambiguity drive most mistakes.

## Key points
- The dataset splits are 2,327 train, 260 dev and 328 test samples, about 4.5h, 0.5h and 0.7h of audio. Egyptian speakers supply 1,026 of the training items.
- The zero-shot Fine-Tashkeel DER of 10.56% trails the winning team (6.87%) by 3.69 pp. It comes close to the organizers' fine-tuned Text+ASR baseline (9.91%).
- Off-the-shelf multimodal systems scored much worse on DER:
  - ArTST: 42.82%
  - Seamless M4T: 43.01%
  - Whisper variants: 84.71–97.36%
- Several attempts failed to beat the untrained baseline: fine-tuning (which hit convergence problems), curriculum learning, LoRA, ensemble voting and post-processing rules.
- Dev-set DER varies strongly by dialect: Egyptian 3.70%, Qatar 8.74%, Saudi 10.49%, Bahrain 11.17%, Algeria 13.73%.
- Across sentences, 33.1% come out fully correct and only 3.1% exceed 30% DER. Fine-Tashkeel showed the smallest dev-to-test drop, at 2.46 pp.

## Interesting ideas
- Generating the whole output string, instead of tagging each input character, lets the model fix upstream normalization slips as a side effect.
- The scores are reported under four settings, toggling the hard positions (case endings) and the trivial ones (slots needing no mark), so easy cases cannot inflate the headline number.
- Error rates are broken down per dialect and set beside each group's share of training data, which shows how much of the error is a coverage problem.
- The authors argue that the audio models failed because their decoders were built for a different task, not because the modality lacks signal. This is a careful reading of a negative result.
- The gap between dev and test scores served as a stability criterion alongside raw accuracy when choosing the final system.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): A small generative normalizer before tokenization could repair transcript or OCR noise. Its edits would be logged so the change stays explainable.
- **I2** (learn): Parser evaluation could similarly report LAS with and without punctuation and other near-deterministic relations, isolating real attachment skill.
- **I3** (tree): Per-source LAS breakdowns for gparse, set against each treebank's share of training data, would show where silver data is most needed.
- **I4** (learn): When a witness parser or LLM teacher tier underperforms, first check output-format or objective mismatch before discarding that source.
- **I5** (learn): Mova could track the dev-test gap per checkpoint and prefer stable models when promoting repair rules or new parser versions.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [King Abdulaziz University](https://md.research.elamur.ai/free/en/universities/king-abdulaziz-university/) — University, Jeddah, Saudi Arabia · 31 works · 508 citations
- [Umm al-Qura University](https://md.research.elamur.ai/free/en/universities/umm-al-qura-university/) — University, Makkah al Mukarramah, Saudi Arabia · 21 works · 274 citations
- [Tuwaiq Academy](https://md.research.elamur.ai/free/en/universities/tuwaiq-academy/) — University, Riyadh, Saudi Arabia · 11 works · 7 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Hassan Barmandah, Fatimah Emad Eldin, Omer Nacar and Wareef Alzubaidi. 2026. Fine-Tashkeel at KSAA-2026: A Comprehensive Evaluation of Seq2Seq and Multimodal Approaches for Automatic Diacritization of Arabic Speech Dictation. The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{barmandah2026fine,
  title = {{Fine-Tashkeel at KSAA-2026: A Comprehensive Evaluation of Seq2Seq and Multimodal Approaches for Automatic Diacritization of Arabic Speech Dictation}},
  author = {Hassan Barmandah, Fatimah Emad Eldin, Omer Nacar and Wareef Alzubaidi},
  booktitle = {The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks},
  year = {2026},
  doi = {10.63317/4rgtj4c7thyj},
  url = {https://doi.org/10.63317/4rgtj4c7thyj}
}
```

- [← previous Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management](https://md.research.elamur.ai/free/en/papers/ma-2026-fine-mem-fine-grained-feedback/)
- [next → Fine-tuning Whisper Across 81 Languages](https://md.research.elamur.ai/free/en/papers/singh-2026-fine-tuning-whisper-across-81/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/)

## References

17 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bashar Al-Rfooh, Gheith Abandah, and Rami Al-Rfou. 2023. [Fine-tashkeel: Finetuning byte-level models for accurate arabic text diacritization](https://md.research.elamur.ai/free/en/papers/al-rfooh-2023-fine-tashkeel-finetuning-byte-level/). ArXiv preprint arXiv:2303.14588. [↗](https://arxiv.org/abs/2303.14588)
2. Asma Al Wazrah, Waad Alshammari, Rawan Al-matham, Raghad Al-Rasheed, Afrah Altamimi, Rufael Marew, Sawsan Alqahtani, Hanan Aldarmaki, Abdullah Alharbi, Abdulrahman Alshehri, Mohamed Assar, Amal Almazrua, and Abdulrahman AlOsaimy. 2026. [KSAA-2026 Shared Task on Arabic Speech Dictation with Automatic Diacritization](https://md.research.elamur.ai/free/en/papers/wazrah-2026-ksaa-2026-shared-task-arabic/). In Proceedings of the 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7).
3. Faris Alasmary, Orjuwan Zaafarani, and Ahmad Ghannam. 2024. [CATT: Character-based Arabic tashkeel transformer](https://md.research.elamur.ai/free/en/papers/alasmary-2024-catt-character-based-arabic-tashkeel/). In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), pages 250– 257, Bangkok, Thailand. Association for Computational Linguistics.
4. Ahmad Barqawi and Taha Zerrouki. 2017. Shakkala: Arabic text vocalization. [https://github.com/Barqawiz/Shakkala](https://github.com/Barqawiz/Shakkala). [↗](https://github.com/Barqawiz/Shakkala)
5. Bisher. 2025. qwen-1p5-diacritization. [https://huggingface.co/Bisher/qwen-1p5-diacritization](https://huggingface.co/Bisher/qwen-1p5-diacritization). Accessed: March 2026. [↗](https://huggingface.co/Bisher/qwen-1p5-diacritization)
6. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25:1– 53. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+et+al%2E+2024%2E+Scaling+instruction%2Dfinetuned+language+models%2E+Journal+of+Machine+Learning+Research%2C+25%3A1%E2%80%93+53%2E)
7. Etherll. 2025. Tashkeel-700m model. [https://huggingface.co/Etherll/Tashkeel-700M](https://huggingface.co/Etherll/Tashkeel-700M). Accessed: March 2026. [↗](https://huggingface.co/Etherll/Tashkeel-700M)
8. Ali Fadel, Ibraheem Tuffaha, Bara’ Al-Jawarneh, and Mahmoud Al-Ayyoub. 2019. [Neural Arabic text diacritization: State of the art results and a novel approach for machine translation](https://md.research.elamur.ai/free/en/papers/fadel-2019-neural-arabic-text-diacritization-state/). In Proceedings of the 6th Workshop on Asian Translation, pages 215–225, Hong Kong, China. Association for Computational Linguistics. glonor. 2024. byt5-arabic-diacritization. https://huggingface.co/glonor/byt5-arabic-diacritization. Accessed: March 2026. [↗](https://doi.org/10.60692/wrdx8-hg164)
9. MaddoggProduction. 2026. turbo-quran-lora-dataset-mix. whisper-l-v3-https://huggingface.co/MaddoggProduction/ whisper-l-v3-turbo-quran-lora-dataset-mix. Accessed: March 2026. [find](https://scholar.google.com/scholar?q=MaddoggProduction%2E+2026%2E+turbo%2Dquran%2Dlora%2Ddataset%2Dmix%2E+whisper%2Dl%2Dv3%2Dhttps%3A%2F%2Fhuggingface%2Eco%2FMaddoggProduction%2F+whisper%2Dl%2Dv3%2Dturbo%2Dquran%2Dlora%2Ddataset%2Dmix%2E+Accessed%3A+March+2026%2E)
10. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Alexander Erdmann, and Nizar Habash. 2020. [CAMeL tools: An open source python toolkit for Arabic natural language processing](https://md.research.elamur.ai/free/en/papers/obeid-2020-camel-tools-open-source-python/). In Proceedings of the 12th Language Resources and Evaluation Conference, pages 7022–7032. European Language Resources Association.
11. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. [Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/). In Proceedings of the 40th International Conference on Machine Learning, volume 202 of PMLR, pages 28492–28518. [↗](https://doi.org/10.48550/arxiv.2212.04356)
12. Seamless Communication, Loïc Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, John Hoffman, Min-Jae Hwang, Hirofumi Inaguma, Christopher Klaiber, Ilia Kulikov, Pengwei Li, Daniel Licht, Jean Maillard, Ruslan Mavlyutov, Alice Rakotoarison, Kaushik Ram Sadagopan, Abinesh Ramakrishnan, Tuan Tran, Guillaume Wenzek, Yilin Yang, Ethan Ye, Ivan Evtimov, Pierre Fernandez, Cynthia Gao, Prangthip Hansanti, Elahe Kalbassi, Amanda Kallet, Artyom Kozhevnikov, Gabriel Mejia, Robin San Roman, Christophe Touret, Corinne Wong, Carleigh Wood, Bokai Yu, Pierre Andrews, Can Balioglu, Peng-Jen Chen, Marta R. Costa-jussà, Maha Elbayad, Hongyu Gong, Francisco Guzmán, Kevin Heffernan, Somya Jain, Justine Kao, Ann Lee, Xutai Ma, Alex Mourachko, Benjamin Peloquin, Juan Pino, Sravya Popuri, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, Anna Sun, Paden Tomasello, Changhan Wang, Jeff Wang, Skyler Wang, and Mary Williamson. 2023. [Seamless: Multilingual expressive and streaming speech translation](https://md.research.elamur.ai/free/en/papers/communication-2023-seamless-multilingual-expressive-streaming-speech/). In ArXiv preprint arXiv:2312.05187. [↗](https://arxiv.org/abs/2312.05187)
13. Sara Shatnawi, Sawsan Alqahtani, and Hanan Aldarmaki. 2024. Automatic restoration of diacritics for speech data sets. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4166–4176. Association for Computational Linguistics. tarteel-ai. 2022. whisper-base-ar-quran. https://huggingface.co/tarteel-ai/whisper-base-ar-quran. March 2026. Accessed: Hawau Toyin, Amirbek Djanibekov, Ajinkya Kulkarni, and Hanan Aldarmaki. 2023. [ArTST: Arabic text and speech transformer](https://md.research.elamur.ai/free/en/papers/toyin-2023-artst-arabic-text-speech-transformer/). In Proceedings of ArabicNLP 2023, pages 41–51, Singapore (Hybrid). Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2311.10771)
14. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in neural information processing systems, pages 6000–6010.
15. WajeehAzeemX. 2024. whisper-tiny-ar-tashkeel. [https://huggingface.co/WajeehAzeemX/whisper-tiny-ar-tashkeel](https://huggingface.co/WajeehAzeemX/whisper-tiny-ar-tashkeel). Accessed: March 2026. [↗](https://huggingface.co/WajeehAzeemX/whisper-tiny-ar-tashkeel)
16. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. [mt5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483–498. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2010.11934)
17. Taha Zerrouki. 2020. Towards An Open Platform For Arabic Language Processing. Phd thesis, Ecole Nationale Supérieure d’informatique, Alger, Algérie. 1. Evaluation Metrics System performance is evaluated using three complementary metrics. Let N c , N w , and N s represent the total number of characters, words, and sentences in the reference text, respectively. 1. Diacritic Error Rate (DER): Measures the proportion of incorrectly predicted diacritics. S c + D c + I c DER = N c 2. Word Error Rate (WER): The primary metric. A word is marked incorrect if it contains at least one diacritic error, making it highly sensitive to case endings (i’rab). Incorrect Words N w WER = 3. Sentence Error Rate (SER): A strict metric where a sentence is incorrect if any diacritic within it is flawed. Incorrect Sentences N s SER = These metrics are computed under four settings by toggling two binary conditions: (a) whether case endings (i’rab) are considered, and (b) whether character positions with no diacritic are included in the count. The official leaderboard uses “With case ending, Including no diacritic” (WCE, Incl. 0). 2. Full Internal Results 3. Organizer Baseline Results 4. Development Set Analysis Fine-Tashkeel achieves the best dev DER (8.10%) among all models with reliable dev evaluation, followed by Shakkelha (10.26%) and Shakkala (11.64%). Fine-Tashkeel also shows the most consistent dev-to-test behavior, with a modest gap of only 2.46 percentage points (8.10% → 10.56%), indicating robust generalization. 5. Per-Nationality Detailed Analysis Table 8 presents the full per-nationality breakdown of DER, WER, and SER on the development set, with and without case ending evaluation. 5.1. Gender Analysis The model shows a modest gender gap: male speakers achieve 7.78% DER (WCE) compared to 8.62% for female speakers, a difference of 0.84 percentage points. This is likely due to the higher proportion of Egyptian male speakers in the dataset rather than an inherent gender bias in the model. 6. Error Distribution Analysis [find](https://scholar.google.com/scholar?q=Taha+Zerrouki%2E+2020%2E+Towards+An+Open+Platform+For+Arabic+Language+Processing%2E+Phd+thesis%2C+Ecole+Nationale+Sup%C3%A9rieure+d%E2%80%99informatique%2C+Alger%2C+Alg%C3%A9rie%2E+1%2E+Evaluation+Metrics+System+performance+is+evaluat)

*Updated 1 November 2026*