---
title: "Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts"
description: "The authors build Nile-Chat, a family of models that read and write Egyptian Arabic in both scripts as… 47 references, cited by 9 papers here."
authors: ["Guokan Shang", "Hadi Abdine", "Ahmad Chamma", "Amr Mohamed", "Mohamed Anwar", "Abdelaziz Bounhar", "Omar El Herraoui", "Preslav Nakov", "Michalis Vazirgiannis", "Eric P. Xing"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2025.arabicnlp-main.25/"
doi: "10.18653/v1/2025.arabicnlp-main.25"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shang-2025-nile-chat-egyptian-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shang-2025-nile-chat-egyptian-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/) / Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts

# Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts

**Authors:** [Guokan Shang](https://md.research.elamur.ai/free/en/authors/guokan-shang/), [Hadi Abdine](https://md.research.elamur.ai/free/en/authors/hadi-abdine/), Ahmad Chamma, [Amr Mohamed](https://md.research.elamur.ai/free/en/authors/amr-mohamed/), [Mohamed Anwar](https://md.research.elamur.ai/free/en/authors/mohamed-anwar/), Abdelaziz Bounhar, [Omar El Herraoui](https://md.research.elamur.ai/free/en/authors/omar-el-herraoui/), [Preslav Nakov](https://md.research.elamur.ai/free/en/authors/preslav-nakov/), [Michalis Vazirgiannis](https://md.research.elamur.ai/free/en/authors/michalis-vazirgiannis/) and [Eric P. Xing](https://md.research.elamur.ai/free/en/authors/eric-p-xing/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-main.25/) · [PDF](https://aclanthology.org/2025.arabicnlp-main.25.pdf) · [DOI 10.18653/v1/2025.arabicnlp-main.25](https://doi.org/10.18653/v1/2025.arabicnlp-main.25) · [OpenAlex](https://openalex.org/W4416037685)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/shang-2025-nile-chat-egyptian-language-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Egyptian Arabic is often written in two ways: in Arabic script and in a Latin-based romanization called Arabizi or Franco-Arabic. Most Arabic LLMs either target MSA or treat romanized text as noise. The authors build Nile-Chat, a family of models that read and write Egyptian Arabic in both scripts as first-class forms. They release dense 4B and 12B models and Nile-Chat-3x4B-A6B, a Mixture-of-Experts model built with Branch-Train-MiX (BTX). For the MoE, separate experts are continually pre-trained on Arabic-script and Latin-script data and then merged with the base model behind a router.

The training pipeline has three stages: continual pre-training on dialect corpora, instruction tuning, and DPO alignment. About a quarter of all data is in Latin script. The authors also release new Egyptian benchmarks for understanding and generation in both scripts. Nile-Chat beats LLaMa, Jais, ALLaM and Qwen2.5 baselines, with the largest margins on Latin-script tasks. This suggests existing models largely neglect romanized dialect, and it offers a reusable recipe for other languages written in two scripts.

## Key points
- Pre-training corpus: about 1.15B words.
  - It includes 854K audio/video transcripts (829M words), EFC-mini, EDC, Egyptian Wikipedia, ADD, FineWeb-2 (Egyptian part), Habibi lyrics and Fatakat forum posts.
  - Claude transliterated informal subsets into about 255M Latin-script words.
- Egyptian-SFT-Mixture: 1.85M instructions.
  - Sources include an Aya Collection subset filtered with Glotlid, and TÜLU-v2&3-mix (deduplicated, 9,660 duplicates removed) translated by Claude, which was chosen over GPT-4o.
  - It also contains long UltraChat dialogues (7–8 turns) plus translation and transliteration tasks.
- Nile-Chat-12B scores 14.4% higher than Qwen2.5-14B-Instruct on Latin-script benchmarks and is best on every Arabic-script task.
- Nile-Chat-4B beats same-size models on Latin-script tasks by large relative margins, for example +18.38% on EgyptianHellaSwag.
- The MoE variants sit between 4B and 12B on Arabic-script discriminative tasks: EgyptianHellaSwag 59.69% vs 55.92% (4B) and 64.04% (12B). 3x4B-A6B leads all translation and transliteration metrics.

## Interesting ideas
- Treat the second writing system as a legitimate input and output form, not as corrupted text to be normalized first; this keeps the information users actually produce.
- Train separate specialists on each variety from a shared base, then merge them behind a learned router so each token goes to the fitting expert.
- Create data for the underrepresented form by having an LLM convert existing text, but only in the domains where that form naturally occurs, such as conversations and comments.
- Filter borrowed datasets with an automatic language identifier before mixing, so off-target samples do not dilute the training signal.
- Report results separately per script; strong baselines that look fine on one script collapse on the other, exposing a hidden blind spot.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Nonstandard English spellings (chat-style, phonetic, numeric substitutions) could get their own lemma and morphology paths rather than being forced through a lossy normalizer.
- **I2** (tree): gparse could use register-specific heads or adapters (news, fiction, chat treebanks) combined with a router, instead of one averaged model.
- **I3** (learn): Silver treebanks for informal English could come from LLM-rewritten conversational sentences, with labels carried over and then checked by the witness parsers.
- **I4** (learn): A language and register filter could screen incoming corpora before silver parsing, acting as a cheap negative control on data quality.
- **I5** (answer): School-test and fable QA scores could be sliced by orthography and register, so failures on informal or noisy text are not masked by averages.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [École Polytechnique](https://md.research.elamur.ai/free/en/universities/ecole-polytechnique/) — University, Palaiseau, France · 14 works · 898 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Guokan Shang, Hadi Abdine, Ahmad Chamma, Amr Mohamed, Mohamed Anwar, Abdelaziz Bounhar, Omar El Herraoui, Preslav Nakov, Michalis Vazirgiannis and Eric P. Xing. 2025. Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts. Proceedings of The Third Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shang2025nile,
  title = {{Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts}},
  author = {Guokan Shang, Hadi Abdine, Ahmad Chamma, Amr Mohamed, Mohamed Anwar, Abdelaziz Bounhar, Omar {El Herraoui}, Preslav Nakov, Michalis Vazirgiannis and Eric P. Xing},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-main.25},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-main.25}
}
```

- [← previous Liger: Linearizing Large Language Models to Gated Recurrent Structures](https://md.research.elamur.ai/free/en/papers/lan-2025-liger-linearizing-large-language-models/)
- [next → Parenting: Optimizing Knowledge Selection of Retrieval-Augmented Language Models with Parameter Decoupling and Tailored Tuning](https://md.research.elamur.ai/free/en/papers/xu-2025-parenting-optimizing-knowledge-selection-retrieval/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/)

## References

47 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yuntian Deng, Wenting Zhao, Jack Hessel, Xiang Ren, Claire Cardie, and Yejin Choi. 2024. Wildvis: Open source visualizer for million-scale chat logs in the wild. arXiv preprint arXiv:2409.03753. [↗](https://arxiv.org/abs/2409.03753)
2. Rania Al-Sabbagh. 2024. [Arzen-multigenre: An aligned parallel dataset of egyptian arabic song lyrics, novels, and subtitles, with english translations](https://md.research.elamur.ai/free/en/papers/al-sabbagh-2025-arzen-multigenre-aligned-parallel-dataset/). Data in Brief, 54:110271. [↗](https://doi.org/10.2139/ssrn.4674389)
3. Hamdah Abdullah Alghamdi. 2018. Arabizi: An exploration of the use of the contemporary youth netspeak on Social Networking Sites in Saudi Arabia. Ph.D. thesis, University of Canberra. [find](https://scholar.google.com/scholar?q=Hamdah+Abdullah+Alghamdi%2E+2018%2E+Arabizi%3A+An+exploration+of+the+use+of+the+contemporary+youth+netspeak+on+Social+Networking+Sites+in+Saudi+Arabia%2E+Ph%2ED%2E+thesis%2C+University+of+Canberra%2E)
4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104)
5. Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. 2023. [The belebele benchmark: a parallel reading comprehension dataset in 122 language variants](https://md.research.elamur.ai/free/en/papers/bandarkar-2023-belebele-benchmark-parallel-reading-comprehension/). arXiv preprint arXiv:2308.16884. [↗](https://arxiv.org/abs/2308.16884)
6. M Saiful Bari, Yazeed Alnumay, Norah A Alzahrani, Nouf M Alotaibi, Hisham A Alyahya, Sultan Al-Rashed, Faisal A Mirza, Shaykhah Z Alsubaie, Hassan A Alahmed, Ghadah Alabduljabbar, and 1 others. 2024. [Allam: Large language models for arabic and english](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/). arXiv preprint arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390)
7. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439.
8. Faeze Brahman, Sachin Kumar, Vidhisha Balachandran, Pradeep Dasigi, Valentina Pyatkin, Abhilasha Ravichander, Sarah Wiegreffe, Nouha Dziri, Khyathi Chandu, Jack Hessel, and 1 others. 2024. [The art of saying no: Contextual noncompliance in language models](https://md.research.elamur.ai/free/en/papers/brahman-2024-art-saying-no-contextual-noncompliance/). Advances in Neural Information Processing Systems, 37:49706–49748.
9. Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. 2024. Sharegpt4v: Improving large multi-modal Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
10. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. [Enhancing chat language models by scaling high-quality instructional conversations](https://md.research.elamur.ai/free/en/papers/ding-2023-enhancing-chat-language-models-scaling/). arXiv preprint arXiv:2305.14233. [↗](https://arxiv.org/abs/2305.14233)
11. Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. 2024. Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475. [↗](https://arxiv.org/abs/2404.04475)
12. Mahmoud El-Haj. 2020. [Habibi - a multi dialect multi national Arabic song lyrics corpus](https://md.research.elamur.ai/free/en/papers/haj-2020-habibi-multi-dialect-multi-national/). In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1318–1326, Marseille, France. European Language Resources Association.
13. Mohamed Atta Faheem, Khaled Tawfik Wassif, Hanaa Bayomi, and Sherif Mahdy Abdou. 2024. Improving neural machine translation for low resource languages through non-parallel corpora: a case study of egyptian dialect to modern standard arabic translation. Scientific Reports, 14(1):2265. [↗](https://doi.org/10.1038/s41598-023-51090-4)
14. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, and 5 others. 2024. A framework for few-shot language model evaluation. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci)
15. Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. 2024. [Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms](https://md.research.elamur.ai/free/en/papers/han-2024-wildguard-open-one-stop-moderation/). arXiv preprint arXiv:2406.18495. [↗](https://arxiv.org/abs/2406.18495)
16. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300. [↗](https://arxiv.org/abs/2009.03300)
17. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Song Dingjie, Zhihong Chen, Mosen Alharthi, Bang An, Juncai He, Ziche Liu, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, and Jinchao Xu. 2024. AceGPT, localizing large language models in Arabic. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8139–8163, Mexico City, Mexico. Association for Computational Linguistics. Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A Smith, Iz Beltagy, and 1 others. 2023. [Camels in a changing climate: Enhancing lm adaptation with tulu 2](https://md.research.elamur.ai/free/en/papers/ivison-2023-camels-changing-climate-enhancing-lm/). arXiv preprint arXiv:2311.10702. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, and 1 others. 2024a. Mixtral of experts. arXiv preprint arXiv:2401.04088. Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, and 1 others. 2024b. Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. Advances in Neural Information Processing Systems, 37:47094–47165. [↗](https://arxiv.org/abs/2311.10702)
18. Amir Hossein Kargaran, Ayyoob Imani, François Yvon, and Hinrich Schuetze. 2023. [GlotLID: Language identification for low-resource languages](https://md.research.elamur.ai/free/en/papers/kargaran-2023-glotlid-language-identification-low-resource/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6155–6218, Singapore. Association for Computational Linguistics. Andreas Köpf, Yannic Kilcher, Dimitri Von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Richárd Nagyfi, and 1 others. 2023. Openassistant conversations-democratizing large language model alignment. Advances in Neural Information Processing Systems, 36:47669–47681. Fajri Koto, Rituraj Joshi, Nurdaulet Mukhituly, Yuxia Wang, Zhuohan Xie, Rahul Pal, Daniil Orel, Parvez Mullah, Diana Turmakhan, Maiya Goloburda, and 1 others. 2025. Llama-3.1-sherkala-8b-chat: An open large language model for kazakh. arXiv preprint arXiv:2503.01493. [↗](https://arxiv.org/abs/2503.01493)
19. Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, and Timothy Baldwin. 2024. [ArabicMMLU: Assessing massive multitask language understanding in Arabic](https://md.research.elamur.ai/free/en/papers/koto-2024-arabicmmlu-assessing-massive-multitask-language/). In Findings of the Association for Computational Linguistics: ACL 2024, pages 5622–5640, Bangkok, Thailand. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2402.12840)
20. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. [Race: Large-scale reading comprehension dataset from examinations](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/). arXiv preprint arXiv:1704.04683. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, and 1 others. 2024. T\" ulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124. [↗](https://arxiv.org/abs/1704.04683)
21. Hongbo Li, Sen Lin, Lingjie Duan, Yingbin Liang, and Ness B Shroff. 2024a. Theory on mixture-of-experts in continual learning. arXiv preprint arXiv:2406.16437. Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, and 1 others. 2024b. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13:9. [↗](https://arxiv.org/abs/2406.16437)
22. Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, and Surajit Chaudhuri. 2023. Table-gpt: Table-tuned gpt for diverse table tasks. arXiv preprint arXiv:2310.09263. [↗](https://arxiv.org/abs/2310.09263)
23. Ka Man Lo, Zeyu Huang, Zihan Qiu, Zili Wang, and Jie Fu. 2024. A closer look into mixture-of-experts in large language models. arXiv preprint arXiv:2406.18219. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, and 1 others. 2023. [The flan collection: Designing data and methods for effective instruction tuning](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/). In International Conference on Machine Learning, pages 22631–22648. PMLR. [↗](https://arxiv.org/abs/2406.18219)
24. Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568. [↗](https://arxiv.org/abs/2306.08568)
25. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789. [↗](https://arxiv.org/abs/1809.02789)
26. Amr Mohamed, Yang Zhang, Michalis Vazirgiannis, and Guokan Shang. 2025. [Lost in the mix: Evaluating llm understanding of code-switched text](https://md.research.elamur.ai/free/en/papers/mohamed-2026-lost-mix-evaluating-llm-understanding/). arXiv preprint arXiv:2506.14012. Basel Mousi, Nadir Durrani, Fatema Ahmad, Md. [↗](https://arxiv.org/abs/2506.14012)
27. Arid Hasan, Maram Hasanain, Tameem Kabbani, Fahim Dalvi, Shammur Absar Chowdhury, and Firoj Alam. 2025. [AraDiCE: Benchmarks for dialectal and cultural capabilities in LLMs](https://md.research.elamur.ai/free/en/papers/mousi-2025-aradice-benchmarks-dialectal-cultural-capabilities/). In Proceedings of the 31st International Conference on Computational Linguistics, pages 4186–4218, Abu Dhabi, UAE. Association for Computational Linguistics.
28. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. [Orca: Progressive learning from complex explanation traces of gpt-4](https://md.research.elamur.ai/free/en/papers/mukherjee-2023-orca-progressive-learning-complex-explanation/). arXiv preprint arXiv:2306.02707. [↗](https://arxiv.org/abs/2306.02707)
29. Guilherme Penedo, Hynek Kydlíček, Vinko Sabolčec, Bettina Messmer, Negar Foroutan, Amir Hossein Kargaran, Colin Raffel, Martin Jaggi, Leandro Von Werra, and Thomas Wolf. 2025. [Fineweb2: One pipeline to scale them all–adapting pre-training data processing to every language](https://md.research.elamur.ai/free/en/papers/penedo-2025-fineweb2-one-pipeline-scale-them/). arXiv preprint arXiv:2506.20920. [↗](https://arxiv.org/abs/2506.20920)
30. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. [Instruction tuning with gpt-4](https://md.research.elamur.ai/free/en/papers/peng-2023-instruction-tuning-gpt-4/). arXiv preprint arXiv:2304.03277. [↗](https://arxiv.org/abs/2304.03277)
31. Faisal Qarah. 2024. [Egybert: A large language model pretrained on egyptian dialect corpora](https://md.research.elamur.ai/free/en/papers/qarah-2024-egybert-large-language-model-pretrained/). arXiv preprint arXiv:2408.03524. [↗](https://arxiv.org/abs/2408.03524)
32. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Advances in Neural Information Processing Systems, 36:53728– 53741. [↗](https://doi.org/10.48550/arxiv.2305.18290)
33. Nathaniel R Robinson, Shahd Abdelmoneim, Kelly Marchisio, and Sebastian Ruder. 2024. [Al-qasida: Analyzing llm quality and accuracy systematically in dialectal arabic](https://md.research.elamur.ai/free/en/papers/robinson-2025-al-qasida-analyzing-llm-quality/). arXiv preprint arXiv:2412.04193. 11567, Bangkok, Thailand. Association for Computational Linguistics. [↗](https://arxiv.org/abs/2412.04193)
34. Sainbayar Sukhbaatar, Olga Golovneva, Vasu Sharma, Hu Xu, Xi Victoria Lin, Baptiste Rozière, Jacob Kahn, Daniel Li, Wen-tau Yih, Jason Weston, and 1 others. 2024. Branch-train-mix: Mixing expert llms into a mixture-of-experts llm. arXiv preprint arXiv:2403.07816. [↗](https://arxiv.org/abs/2403.07816)
35. Toshiyuki Takezawa, Genichiro Kikui, Masahide Mizushima, and Eiichiro Sumita. 2007. Multilingual spoken language corpus development for communication research. In International Journal of Computational Linguistics & Chinese Language Processing, Volume 12, Number 3, September 2007: Special Issue on Invited Papers from ISCSLP 2006, pages 303–324. [find](https://scholar.google.com/scholar?q=Toshiyuki+Takezawa%2C+Genichiro+Kikui%2C+Masahide+Mizushima%2C+and+Eiichiro+Sumita%2E+2007%2E+Multilingual+spoken+language+corpus+development+for+communication+research%2E+In+International+Journal+of+Computationa)
36. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. [↗](https://arxiv.org/abs/2503.19786)
37. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). arXiv preprint arXiv:2304.12244. [↗](https://arxiv.org/abs/2304.12244)
38. Mohammad Ali Yaghan. 2008. " arabizi": A contemporary style of arabic slang. Design issues, 24(2):39– 52. [↗](https://doi.org/10.1162/desi.2008.24.2.39)
39. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
40. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, and 1 others. 2023. [Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models](https://md.research.elamur.ai/free/en/papers/sengupta-2023-jais-jais-chat-arabic-centric/). arXiv preprint arXiv:2308.16149. [↗](https://arxiv.org/abs/2308.16149)
41. Guokan Shang, Hadi Abdine, Yousef Khoubrane, Amr Mohamed, Yassine Abbahaddou, Sofiane Ennadir, Imane Momayiz, Xuguang Ren, Eric Moulines, Preslav Nakov, Michalis Vazirgiannis, and Eric Xing. 2025. [Atlas-chat: Adapting large language models for low-resource Moroccan Arabic dialect](https://md.research.elamur.ai/free/en/papers/shang-2025-atlas-chat-adapting-large-language/). In Proceedings of the First Workshop on Language Models for Low-Resource Languages, pages 9–30, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2409.17912)
42. Shivalika Singh, Freddie Vargus, Daniel D’souza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura O’Mahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergun, Ifeoma Okoh, and 14 others. 2024. [Aya dataset: An open-access collection for multilingual instruction tuning](https://md.research.elamur.ai/free/en/papers/singh-2024-aya-dataset-open-access-collection/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11521– Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830. [↗](https://arxiv.org/abs/1905.07830)
43. Hao Zhao, Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. 2024a. Long is more for alignment: A simple but tough-to-beat baseline for instruction fine-tuning. arXiv preprint arXiv:2402.04833. [↗](https://arxiv.org/abs/2402.04833)
44. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024b. [Wildchat: 1m chatgpt interaction logs in the wild](https://md.research.elamur.ai/free/en/papers/zhao-2024-wildchat-1m-chatgpt-interaction-logs/). arXiv preprint arXiv:2405.01470. [↗](https://arxiv.org/abs/2405.01470)
45. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in Neural Information Processing Systems, 36:46595–46623.
46. Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, and 1 others. 2023. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36:55006–55021. [find](https://scholar.google.com/scholar?q=Chunting+Zhou%2C+Pengfei+Liu%2C+Puxin+Xu%2C+Srinivasan+Iyer%2C+Jiao+Sun%2C+Yuning+Mao%2C+Xuezhe+Ma%2C+Avia+Efrat%2C+Ping+Yu%2C+Lili+Yu%2C+and+1+others%2E+2023%2E+Lima%3A+Less+is+more+for+alignment%2E+Advances+in+Neural+Informati)
47. Jianqing Zhu, Huang Huang, Zhihang Lin, Juhao Liang, Zhengyang Tang, Khalid Almubarak, Abdulmohsen Alharthik, Bang An, Juncai He, Xiangbo Wu, and 1 others. 2024. [Second language (arabic) acquisition of llms via progressive vocabulary expansion](https://md.research.elamur.ai/free/en/papers/zhu-2025-second-language-arabic-acquisition-llms/). arXiv preprint arXiv:2412.12310. [↗](https://arxiv.org/abs/2412.12310)

*Updated 1 November 2026*