--- title: "Abjad AI at NADI 2025 — paper by Ghannam et al. (2025)" description: "The paper addresses restoring missing diacritics in Arabic text. 3 references, cited by 6 papers here." authors: ["Ahmad Ghannam", "Naif Alharthi", "Faris Alasmary", "Kholood Al Tabash", "Shouq Sadah", "Lahouari Ghouti"] year: 2025 venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks" source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.104/" doi: "10.18653/v1/2025.arabicnlp-sharedtasks.104" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ghannam-2025-abjad-ai-nadi-2025-catt/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ghannam-2025-abjad-ai-nadi-2025-catt/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/10/) / Abjad AI at NADI 2025: CATT-Whisper: Multimodal Diacritic Restoration Using Text and Speech Representations # Abjad AI at NADI 2025: CATT-Whisper: Multimodal Diacritic Restoration Using Text and Speech Representations **Authors:** [Ahmad Ghannam](https://md.research.elamur.ai/free/en/authors/ahmad-ghannam/), Naif Alharthi, [Faris Alasmary](https://md.research.elamur.ai/free/en/authors/faris-alasmary/), [Kholood Al Tabash](https://md.research.elamur.ai/free/en/authors/kholood-al-tabash/), [Shouq Sadah](https://md.research.elamur.ai/free/en/authors/shouq-sadah/) and [Lahouari Ghouti](https://md.research.elamur.ai/free/en/authors/lahouari-ghouti/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.104/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.104.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.104](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.104) · [OpenAlex](https://openalex.org/W4416037793) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [6](https://md.research.elamur.ai/free/en/papers/ghannam-2025-abjad-ai-nadi-2025-catt/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper addresses restoring missing diacritics in Arabic text. It targets dialectal, Classical and code-switched Arabic, where spelling alone often leaves vowelling ambiguous. The authors, from Abjad AI, submitted CATT-Whisper to NADI 2025 Subtask 3. The system pairs the encoder of their earlier character-based CATT model with the encoder of Whisper-base, so the audio of an utterance can guide token-level diacritic classification. They compared two ways of joining the modalities: - **Early fusion:** pooled and projected audio frames are concatenated with the text tokens before the CATT encoder. - **Cross-attention fusion:** separately encoded text and speech are merged before the classifier. Both performed about equally, so the cheaper early-fusion variant was used for all reported results. During training, speech input is randomly dropped so the model still works on text alone. Fine-tuned only on the shared-task data, the system ranked first on both the development and test sets. This shows that acoustic cues can resolve dialectal ambiguities that text-only models miss. ## Key points - **Training data:** the NADI 2025 DR dataset (Hugging Face, MBZUAI/NADI-2025-Sub-task-3-all), built from six corpora (sentence counts): - MDASPC: 60,677, multi-dialectal, diacritized - TunSwitch: 5,212, dialectal/code-switched, diacritized - ArzEn: 3,344, dialectal/code-switched, not diacritized - Mixat: 3,721, dialectal/code-switched, not diacritized - ClArTTS: 9,500, Classical Arabic - ArVoice: 2,507, MSA - **Early fusion details:** Whisper's 1,500 audio frames are averaged in groups of 10, giving 150 speech tokens. A linear layer projects these to the CATT embedding size, and they act like soft-prompt placeholders. - **Training setup:** - Batch size 32, learning rate 1e-5, dropout 0.1, AdamW. - Time-frequency warping is used as spectrogram augmentation. - The Whisper encoder stays frozen for 5 epochs, then everything is trained jointly for 5 more. - **Development set:** WER 0.25 and CER 0.09, versus 0.46/0.22 for the runner-up and the baseline. The abstract prints the CER as 0.9, which looks like a typo for 0.09. - **Test set:** - WER 0.55 and CER 0.13, against 0.64/0.15 for the next team and 0.65/0.16 for the baseline. - An in-house recorded set, in which one word is pronounced differently within a single sentence, remained only partly solved. ## Interesting ideas - Randomly switching off the auxiliary input during training teaches one model to work whether or not that extra signal exists, instead of needing two separate systems. - Extra-modality information is pooled into a small number of pseudo-tokens and prepended to the encoder input. This reuses a pretrained encoder without changing its architecture. - A new adapter layer is trained first while the pretrained encoder stays frozen, and only then is everything unfrozen. This protects pretrained weights from noisy early gradients. - When two designs score about the same, the authors keep the computationally cheaper one. - A small, hand-recorded challenge set isolates one known hard phenomenon, so progress on it is visible separately from aggregate scores. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (tree): gparse could be trained with optional side inputs (e.g. hints from the perceptron parser) that are randomly dropped, so it parses well with or without them. - **I2** (tree): sentence-external context, such as pooled embeddings of the previous sentence or document entities, could be fed to gparse as a few prefix tokens to aid attachment decisions explainably. - **I3** (learn): when attaching new heads (e.g. semantic-role or frame classifiers) to gparse's encoder, a freeze-then-unfreeze schedule could stabilize training on small silver sets. - **I4** (learn): parser or tagger variants could be compared on accuracy per unit of compute, so cheaper designs win ties. - **I5** (tokens): build targeted suites where the same word form takes different POS or lemmas within one sentence, to measure the tagger's context use directly. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works 🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works ## Cite Cite as: Ahmad Ghannam, Naif Alharthi, Faris Alasmary, Kholood Al Tabash, Shouq Sadah and Lahouari Ghouti. 2025. Abjad AI at NADI 2025: CATT-Whisper: Multimodal Diacritic Restoration Using Text and Speech Representations. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{ghannam2025abjad, title = {{Abjad AI at NADI 2025: CATT-Whisper: Multimodal Diacritic Restoration Using Text and Speech Representations}}, author = {Ahmad Ghannam, Naif Alharthi, Faris Alasmary, Kholood {Al Tabash}, Shouq Sadah and Lahouari Ghouti}, booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks}, year = {2025}, doi = {10.18653/v1/2025.arabicnlp-sharedtasks.104}, url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.104} } ``` - [← previous A Systematic Survey of Semantic Role Labeling in the Era of Pretrained Language Models](https://md.research.elamur.ai/free/en/papers/chen-2025-systematic-survey-semantic-role-labeling/) - [next → ADOR: Dataset for Arabic Dialects in Hotel Reviews: A Human Benchmark for Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/alharbi-2025-ador-dataset-arabic-dialects-hotel/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/10/) ## References 3 references, 3 of them with a card of ours. ↗ leads to the work. 1. H. Qå • H. Qå • H. Qå • H. Q å • H. Q å • H. Q å • H  . Qå  • H. Q å • H. Qå • I.ë X I.ë X I.ë X I.ë X I  . ë X I  . ë X Ahmed Amine Ben Abdallah, Ata Kabboudi, Amir Kanoun, and Salah Zaiem. 2023. [Leveraging data collection and unsupervised learning for code-switched tunisian arabic automatic speech recognition](https://md.research.elamur.ai/free/en/papers/abdallah-2023-leveraging-data-collection-unsupervised-learning/). Preprint, arXiv:2309.11327. [↗](https://arxiv.org/abs/2309.11327) 2. Maryam Khalifa Al Ali and Hanan Aldarmaki. 2024. [Mixat: A data set of bilingual emirati-English speech](https://md.research.elamur.ai/free/en/papers/ali-2024-mixat-data-set-bilingual-emirati/). In Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024, pages 222–226, Torino, Italia. ELRA and ICCL. [↗](https://doi.org/10.48550/arxiv.2405.02578) 3. Faris Alasmary, Orjuwan Zaafarani, and Ahmad Ghannam. 2024. [CATT: Character-based Arabic tashkeel transformer](https://md.research.elamur.ai/free/en/papers/alasmary-2024-catt-character-based-arabic-tashkeel/). In Proceedings of the Second Arabic Natural Language Processing Conference, pages 250–257, Bangkok, Thailand. Association for Computational Linguistics. *Table 5: Model performance on challenging test cases with variable word pronunciations.* *Updated 1 November 2026*