--- title: "Sirius_Translators at OSACT6 2024 Shared Task: Fin-tuning" description: "The work offers a simple, reproducible multi-dialect baseline and points to data scarcity as the main bottleneck. 9.57 BLEU. 21 references." authors: ["Salwa Saad Alahmari"] year: 2024 venue: "Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024" source: "https://aclanthology.org/2024.osact-1.15/" licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)" licence_url: "https://creativecommons.org/licenses/by-nc/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/alahmari-2024-sirius-translators-osact6-2024-shared/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alahmari-2024-sirius-translators-osact6-2024-shared/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/) / Sirius\_Translators at OSACT6 2024 Shared Task: Fin-tuning Ara-T5 Models for Translating Arabic Dialectal Text to Modern Standard Arabic # Sirius\_Translators at OSACT6 2024 Shared Task: Fin-tuning Ara-T5 Models for Translating Arabic Dialectal Text to Modern Standard Arabic **Authors:** Salwa Saad Alahmari · **Year:** 2024 · **Venue:** [Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024](https://md.research.elamur.ai/free/en/venues/osact/) ([Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024](https://md.research.elamur.ai/free/en/venues/osact/2024/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2024.osact-1.15/) · [PDF](https://aclanthology.org/2024.osact-1.15.pdf) · [OpenAlex](https://openalex.org/W7131786946) **Citations:** 4 citations (OpenAlex, 2026-10-14) ## Summary The paper addresses translating sentences in Arabic dialects into Modern Standard Arabic (MSA). The task comes from Shared Task 2 of OSACT6 (LREC-COLING 2024) and covers five dialect groups: Gulf, Egyptian, Levantine, Iraqi and Maghrebi. Most earlier work built one system per dialect. The Sirius_Translators team (Alahmari, with Atwell and Saadany) instead trained a single model for all five dialects. They pooled about 180k dialect–MSA sentence pairs from five public corpora: MADAR, PADIC, Dial2MSA, Arabic STS and SADID. They also added extra pairs to help the model with vocabulary that dialects share with MSA. Four AraT5 variants were fine-tuned with identical hyper-parameters so they could be compared fairly. AraT5v2-base-1024 performed best, with BLEU 21.0 on the development set. Its score fell to 9.57 on the hidden test set. The authors attribute the weak results mainly to uneven data coverage, especially for Iraqi, and to having no time to tune hyper-parameters. The work offers a simple, reproducible multi-dialect baseline and points to data scarcity as the main bottleneck. ## Key points - Data splits: - Train: 180,211 sentences. - Dev: 1,001 sentences, roughly 200 per dialect. - Test: 1,888 sentences, roughly 377 per dialect. - Training pairs per dialect after pooling: - Levantine: 52,421 - Maghrebi: 37,112 - Gulf: 36,168 - Egyptian: 35,910 - Iraqi: 18,600 - MADAR was the only training corpus covering all five dialects. PADIC was used in full, including translations that had never been manually validated. - Dev BLEU by model: - AraT5v2-base-1024: 21.0 - AraT5 base: 19.26 - AraT5-MSA-Base: 16.88 - AraT5-MSA-Small: 15.97 - The best model reached only 9.57 BLEU on the test set. - Shared fine-tuning setup: - learning rate 5e-5 - source and target length 128 - batch size 16 - 2 epochs - Hugging Face / PyTorch Transformers ## Interesting ideas - Training one model jointly on all dialect varieties, instead of one model per dialect, lets scarce dialects borrow strength from richer ones through shared structure. - Extra pairs were added so the model learns which words stay unchanged between a dialect and the standard form. This guards against "translating" tokens that are already correct. - All candidate models were fine-tuned with exactly the same hyper-parameters, so differences in score reflect the architecture rather than tuning luck. - Counting training pairs per variety made it possible to trace weak results to a specific under-represented slice, in this case Iraqi. - The large drop from development to test score shows that a dev set close to the training data can hide poor generalization. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (tree): The same idea could guide gparse: train one parser on several English varieties or UD dialect conversions together, rather than keeping a separate parser per source. - **I2** (back): In translation through the meta-world, identity or near-identity examples could teach the generator to keep shared lexical items verbatim instead of paraphrasing them. - **I3** (learn): When comparing witness parsers or teacher tiers, holding the training recipe fixed keeps the comparison honest and the outcome explainable. - **I4** (learn): Per-category coverage tables for silver data (genre, construction, verb class) would show which gaps cause errors before retraining starts. - **I5** (learn): Evaluation should always include held-out, differently sourced control sets, so that dev-set gains are not mistaken for real progress. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 1,485 works ## Cite Cite as: Salwa Saad Alahmari. 2024. Sirius\_Translators at OSACT6 2024 Shared Task: Fin-tuning Ara-T5 Models for Translating Arabic Dialectal Text to Modern Standard Arabic. Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{alahmari2024sirius, title = {{Sirius\_Translators at OSACT6 2024 Shared Task: Fin-tuning Ara-T5 Models for Translating Arabic Dialectal Text to Modern Standard Arabic}}, author = {Salwa Saad Alahmari}, booktitle = {Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024}, year = {2024}, doi = {10.63317/59fi88he3h8x}, url = {https://doi.org/10.63317/59fi88he3h8x} } ``` - [← previous Pedagogical Insights from Al-Mawahib Al-Rabbaniyah: Enhancing Arabic Grammar Learning through the Nazm of Al-Ajrumiyyah](https://md.research.elamur.ai/free/en/papers/solehudin-2024-pedagogical-insights-al-mawahib-al/) - [next → Ta'keed: The First Generative Fact-Checking System for Arabic Claims](https://md.research.elamur.ai/free/en/papers/althabiti-2024-ta-keed-first-generative-fact/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/) ## References 21 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2022. [NADI 2022: The third nuanced Arabic dialect identification shared task](https://md.research.elamur.ai/free/en/papers/mageed-2022-nadi-2022-third-nuanced-arabic/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 85–97, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. 2. Wael Abid. 2020. [The sadid evaluation datasets for low-resource spoken language machine translation of arabic dialects](https://md.research.elamur.ai/free/en/papers/abid-2020-sadid-evaluation-datasets-low-resource/). In International Conference on Computational Linguistics. 3. Roqayah M. Al-Ibrahim and Rehab Duwairi. 2020. Neural machine translation from jordanian dialect to modern standard arabic. 2020 11th International Conference on Information and Communication Systems (ICICS), pages 173–178. [find](https://scholar.google.com/scholar?q=Roqayah+M%2E+Al%2DIbrahim+and+Rehab+Duwairi%2E+2020%2E+Neural+machine+translation+from+jordanian+dialect+to+modern+standard+arabic%2E+2020+11th+International+Conference+on+Information+and+Communication+Systems+) 4. Mahmoud A Al-Sobh, Abdel-Rahman H Abu-Melhim, and Nedal A Bani-Hani. 2015. Diglossia as a result of language variation in arabic: Possible solutions in light of language planning. Journal of Language Teaching and Research, 6(2):274. [↗](https://doi.org/10.17507/jltr.0602.05) 5. Mansour Al Sulaiman, Abdullah M. Moussa, Sherif Abdou, Hebah Elgibreen, Mohammed Faisal, and Mohsen Rashwan. 2022. Semantic textual similarity for modern standard and dialectal arabic using transfer learning. PLOS ONE, 17(8):1–14. [find](https://scholar.google.com/scholar?q=Mansour+Al+Sulaiman%2C+Abdullah+M%2E+Moussa%2C+Sherif+Abdou%2C+Hebah+Elgibreen%2C+Mohammed+Faisal%2C+and+Mohsen+Rashwan%2E+2022%2E+Semantic+textual+similarity+for+modern+standard+and+dialectal+arabic+using+transfer+l) 6. Laith H. Baniata, Seyoung Park, and Seong-Bae Park. 2018. A multitask-based neural machine translation model with part-of-speech tags integration for arabic dialects. Applied Sciences, 8(12). [↗](https://doi.org/10.3390/app8122502) 7. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. [A multidialectal parallel corpus of arabic](https://md.research.elamur.ai/free/en/papers/bouamor-2014-multidialectal-parallel-corpus-arabic/). In LREC, pages 1240–1245. 8. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. [The MADAR shared task on Arabic fine-grained dialect identification](https://md.research.elamur.ai/free/en/papers/bouamor-2019-madar-shared-task-arabic-fine/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207, Florence, Italy. Association for Computational Linguistics. 9. Injy Hamed, Nizar Habash, Slim Abdennadher, and Ngoc Thang Vu. 2023. Investigating lexical replacements for Arabic-English code-switched data augmentation. In Proceedings of the The Sixth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2023), pages 86–100, Dubrovnik, Croatia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Injy+Hamed%2C+Nizar+Habash%2C+Slim+Abdennadher%2C+and+Ngoc+Thang+Vu%2E+2023%2E+Investigating+lexical+replacements+for+Arabic%2DEnglish+code%2Dswitched+data+augmentation%2E+In+Proceedings+of+the+The+Sixth+Workshop+on+) 10. Saméh Kchaou, Rahma Boujelbane, Emna Fsih, and Lamia Hadrich-Belguith. 2022. Standardisation of dialect comments in social networks in view of sentiment analysis : Case of Tunisian dialect. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5436–5443, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Sam%C3%A9h+Kchaou%2C+Rahma+Boujelbane%2C+Emna+Fsih%2C+and+Lamia+Hadrich%2DBelguith%2E+2022%2E+Standardisation+of+dialect+comments+in+social+networks+in+view+of+sentiment+analysis+%3A+Case+of+Tunisian+dialect%2E+In+Proceed) 11. Abdullah Khered, Ingy Abdelhalim, Nadine Abdelhalim, Ahmed Soliman, and Riza Batista-Navarro. 2023. UniManc at NADI 2023 shared task: A comparison of various t5-based models for translating Arabic dialectical text to Modern Standard Arabic. In Proceedings of ArabicNLP 2023, pages 658–664, Singapore (Hybrid). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Abdullah+Khered%2C+Ingy+Abdelhalim%2C+Nadine+Abdelhalim%2C+Ahmed+Soliman%2C+and+Riza+Batista%2DNavarro%2E+2023%2E+UniManc+at+NADI+2023+shared+task%3A+A+comparison+of+various+t5%2Dbased+models+for+translating+Arabic+dia) 12. K. Meftouh, S Harrat, and Kamel Smaïli. 2018. PADIC: extension and new experiments. In 7th International Conference on Advanced Technologies ICAT, Antalya, Turkey. [find](https://scholar.google.com/scholar?q=K%2E+Meftouh%2C+S+Harrat%2C+and+Kamel+Sma%C3%AFli%2E+2018%2E+PADIC%3A+extension+and+new+experiments%2E+In+7th+International+Conference+on+Advanced+Technologies+ICAT%2C+Antalya%2C+Turkey%2E) 13. Hamdy Mubarak. 2018. Dial2msa: A tweets corpus for converting dialectal arabic to modern standard arabic. OSACT, 3:49. [find](https://scholar.google.com/scholar?q=Hamdy+Mubarak%2E+2018%2E+Dial2msa%3A+A+tweets+corpus+for+converting+dialectal+arabic+to+modern+standard+arabic%2E+OSACT%2C+3%3A49%2E) 14. El Moatez Billah Nagoudi, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2021. Investigating code-mixed Modern Standard Arabic-Egyptian to English machine translation. In Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching, pages 56–64, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=El+Moatez+Billah+Nagoudi%2C+AbdelRahim+Elmadany%2C+and+Muhammad+Abdul%2DMageed%2E+2021%2E+Investigating+code%2Dmixed+Modern+Standard+Arabic%2DEgyptian+to+English+machine+translation%2E+In+Proceedings+of+the+Fifth+Wor) 15. El Moatez Billah Nagoudi, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2022a. [AraT5: Text-to-text transformers for Arabic language generation](https://md.research.elamur.ai/free/en/papers/nagoudi-2022-arat5-text-text-transformers-arabic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 628–647, Dublin, Ireland. Association for Computational Linguistics. 16. El Moatez Billah Nagoudi, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2022b. TURJUMAN: A public toolkit for neural Arabic machine translation. In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur’an QA and Fine-Grained Hate Speech Detection, pages 1–11, Marseille, France. European Language Resources Association. [↗](https://doi.org/10.48550/arxiv.2206.03933) 17. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. [↗](https://doi.org/10.3115/1073083.1073135) 18. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21(140):1–67. [↗](https://doi.org/10.48550/arxiv.1910.10683) 19. Houcemeddine Turki, Emad Adel, Tariq Daouda, and Nassim Regragui. 2016. A conventional orthography for maghrebi arabic. In International Conference on Language Resources and Evaluation. [find](https://scholar.google.com/scholar?q=Houcemeddine+Turki%2C+Emad+Adel%2C+Tariq+Daouda%2C+and+Nassim+Regragui%2E+2016%2E+A+conventional+orthography+for+maghrebi+arabic%2E+In+International+Conference+on+Language+Resources+and+Evaluation%2E) 20. Omar F. Zaidan and Chris Callison-Burch. 2014. [Arabic dialect identification](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/). Computational Linguistics, 40(1):171–202. [↗](https://doi.org/10.1162/coli_a_00169) 21. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1568–1575, Austin, Texas. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Barret+Zoph%2C+Deniz+Yuret%2C+Jonathan+May%2C+and+Kevin+Knight%2E+2016%2E+Transfer+learning+for+low%2Dresource+neural+machine+translation%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Lan) *Updated 24 October 2026*