---
title: "Palm — paper by Alwajih et al. (2025) — summary and ideas"
description: "Arabic LLMs often reflect Western cultural assumptions, largely because their instruction data is… 40 references, cited by 35 papers here."
authors: ["Fakhraddin Alwajih", "Abdellah El Mekki", "Samar Mohamed Magdy", "AbdelRahim A. Elmadany", "Omer Nacar", "El Moatez Billah Nagoudi", "Reem Abdel-Salam", "Hanin Atwany", "Youssef Nafea", "Abdulfattah Mohammed Yahya", "Rahaf Alhamouri", "Hamzah A. Alsayadi", "Hiba Zayed", "Sara Shatnawi", "Serry Sibaee", "Yasir Ech-chammakhy", "Walid Al-Dhabyani", "Marwa Mohamed Ali", "Imen Jarraya", "Ahmed Oumar El-Shangiti", "Aisha Alraeesi", "Mohammed Anwar AL-Ghrawi", "Abdulrahman S. Al-Batati", "Elgizouli Mohamed", "Noha Taha Elgindi", "Muhammed Saeed", "Houdaifa Atou", "Issam Ait Yahia", "Abdelhak Bouayad", "Mohammed Machrouh", "Amal Makouar", "Dania Alkawi", "Mukhtar Mohamed", "Safaa Taher Abdelfadil", "Amine Ziad Ounnoughene", "Anfel Rouabhia", "Rwaa Assi", "Ahmed Sorkatti", "Mohamedou Cheikh Tourad", "Anis Koubaa", "Ismail Berrada", "Mustafa Jarrar", "Shady Shehata", "Muhammad Abdul-Mageed"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.1579/"
doi: "10.18653/v1/2025.acl-long.1579"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/alwajih-2025-palm-culturally-inclusive-linguistically-diverse/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alwajih-2025-palm-culturally-inclusive-linguistically-diverse/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs

# Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs

**Authors:** [Fakhraddin Alwajih](https://md.research.elamur.ai/free/en/authors/fakhraddin-alwajih/), [Abdellah El Mekki](https://md.research.elamur.ai/free/en/authors/abdellah-el-mekki/), [Samar Mohamed Magdy](https://md.research.elamur.ai/free/en/authors/samar-m-magdy/), [AbdelRahim A. Elmadany](https://md.research.elamur.ai/free/en/authors/abdelrahim-elmadany/), [Omer Nacar](https://md.research.elamur.ai/free/en/authors/omer-nacar/), [El Moatez Billah Nagoudi](https://md.research.elamur.ai/free/en/authors/el-moatez-billah-nagoudi/), [Reem Abdel-Salam](https://md.research.elamur.ai/free/en/authors/reem-abdel-salam/), [Hanin Atwany](https://md.research.elamur.ai/free/en/authors/hanin-atwany/), [Youssef Nafea](https://md.research.elamur.ai/free/en/authors/youssef-nafea/), [Abdulfattah Mohammed Yahya](https://md.research.elamur.ai/free/en/authors/abdulfattah-mohammed-yahya/), [Rahaf Alhamouri](https://md.research.elamur.ai/free/en/authors/rahaf-alhamouri/), [Hamzah A. Alsayadi](https://md.research.elamur.ai/free/en/authors/hamzah-a-alsayadi/), Hiba Zayed, [Sara Shatnawi](https://md.research.elamur.ai/free/en/authors/sara-shatnawi/), [Serry Sibaee](https://md.research.elamur.ai/free/en/authors/serry-sibaee/), Yasir Ech-chammakhy, Walid Al-Dhabyani, Marwa Mohamed Ali, Imen Jarraya, [Ahmed Oumar El-Shangiti](https://md.research.elamur.ai/free/en/authors/ahmed-oumar-el-shangiti/), [Aisha Alraeesi](https://md.research.elamur.ai/free/en/authors/aisha-alraeesi/), Mohammed Anwar AL-Ghrawi, Abdulrahman S. Al-Batati, Elgizouli Mohamed, Noha Taha Elgindi, [Muhammed Saeed](https://md.research.elamur.ai/free/en/authors/muhammed-saeed/), [Houdaifa Atou](https://md.research.elamur.ai/free/en/authors/houdaifa-atou/), [Issam Ait Yahia](https://md.research.elamur.ai/free/en/authors/issam-ait-yahia/), Abdelhak Bouayad, Mohammed Machrouh, Amal Makouar, Dania Alkawi, Mukhtar Mohamed, [Safaa Taher Abdelfadil](https://md.research.elamur.ai/free/en/authors/safaa-taher-abdelfadil/), Amine Ziad Ounnoughene, Anfel Rouabhia, Rwaa Assi, Ahmed Sorkatti, Mohamedou Cheikh Tourad, [Anis Koubaa](https://md.research.elamur.ai/free/en/authors/anis-koubaa/), [Ismail Berrada](https://md.research.elamur.ai/free/en/authors/ismail-berrada/), [Mustafa Jarrar](https://md.research.elamur.ai/free/en/authors/mustafa-jarrar/), [Shady Shehata](https://md.research.elamur.ai/free/en/authors/shady-shehata/) and [Muhammad Abdul-Mageed](https://md.research.elamur.ai/free/en/authors/muhammad-abdul-mageed/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.1579/) · [PDF](https://aclanthology.org/2025.acl-long.1579.pdf) · [DOI 10.18653/v1/2025.acl-long.1579](https://doi.org/10.18653/v1/2025.acl-long.1579) · [OpenAlex](https://openalex.org/W4412889657)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [35](https://md.research.elamur.ai/free/en/papers/alwajih-2025-palm-culturally-inclusive-linguistically-diverse/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic LLMs often reflect Western cultural assumptions, largely because their instruction data is machine-generated or translated from English. Their knowledge of individual Arab countries and dialects is also rarely tested. To address this, the authors built PALM, a fully human-written instruction dataset of input–response pairs covering all 22 Arab countries and 20 topics, including celebrations, history, geography, proverbs, food and sports. The pairs are written in Modern Standard Arabic (MSA) and in ten dialects. A team of 44 native-speaker co-authors created it over about a year, using reliable country-specific sources. The workflow combined shared guidelines, annotation in Label Studio, quality checks and revisions. The authors then benchmarked open and closed LLMs on PALM, scoring them with three LLM judges and checking those judgments with human evaluation. Closed models such as GPT-4o and Claude-3.5-Sonnet scored well but still made errors, while small open models struggled noticeably. Coverage also differed by country: Egypt and the UAE fared better than Iraq, Mauritania or Yemen. PALM can be used both to instruction-tune models and to audit their cultural competence.

## Key points
- **Scope:** PALM covers 22 countries, 20 topical areas, MSA plus ten Arabic dialects, and all instructions are human-produced. Data, guidelines and code are released at github.com/UBC-NLP/palm, and a private test set is planned for a leaderboard.
- **Team:** Annotators were local to 15 of the 22 countries, with at least two per country. The seven countries without local annotators (Qatar, Iraq, Bahrain, Libya, Somalia, Djibouti and Comoros) were handled by annotators from nearby countries. The guidelines were refined over about three months.
- **Dialect results (LLM judges):** GPT-4o reached 8.1 for Egypt and 8.4 for Tunisia, while Llama-3.1-8B and Jais-13b mostly scored below 4.0. Moroccan and Palestinian dialects proved harder for most models.
- **Human evaluation (Egypt, Morocco, Syria, Yemen):** AceGPT-v2-32B scored highest for Egypt (6.47), and Claude-3-5-Sonnet scored highest for Morocco (6.23). Jais-13b was generally the weakest.
- **Topics:** Models did better on celebrations, history and travel, and worse on sports and food. The authors warn that LLM judges can misjudge dialectal and cultural content.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of British Columbia](https://md.research.elamur.ai/free/en/universities/university-of-british-columbia/) — University, Vancouver, Canada · 104 works · 37,403 citations
- [Hamad bin Khalifa University](https://md.research.elamur.ai/free/en/universities/hamad-bin-khalifa-university/) — University, Doha, Qatar · 144 works · 14,175 citations
- [Mohamed bin Zayed University of Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/mohamed-bin-zayed-university-of-artificial-intelligence/) — University, Abu Dhabi, United Arab Emirates · 138 works · 6,590 citations
- [Cairo University](https://md.research.elamur.ai/free/en/universities/cairo-university/) — University, Giza, Egypt · 43 works · 1,352 citations
- [Birzeit University](https://md.research.elamur.ai/free/en/universities/birzeit-university/) — University, Birzeit, Palestine · 38 works · 798 citations
- [Prince Sultan University](https://md.research.elamur.ai/free/en/universities/prince-sultan-university/) — University, Riyadh, Saudi Arabia · 20 works · 120 citations
- [Ain Shams University](https://md.research.elamur.ai/free/en/universities/ain-shams-university/) — University, Cairo, Egypt · 13 works · 109 citations
- [University of Khartoum](https://md.research.elamur.ai/free/en/universities/university-of-khartoum/) — University, Khartoum, Sudan · 4 works · 71 citations
- [Alfaisal University](https://md.research.elamur.ai/free/en/universities/alfaisal-university/) — University, Riyadh, Saudi Arabia · 6 works · 60 citations
- [Damascus University](https://md.research.elamur.ai/free/en/universities/damascus-university/) — University, Damascus, Syria · 4 works · 43 citations
- [Université de Nouakchott](https://md.research.elamur.ai/free/en/universities/universite-de-nouakchott/) — University, Nouakchott, Mauritania · 2 works · 82 citations

Also on the byline: [Polytechnic School of Algiers](https://ror.org/05t0zwy08).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🎓 Education](https://md.research.elamur.ai/free/en/shelves/social-sciences/education/) · 212 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works

## Cite

Cite as: Fakhraddin Alwajih, Abdellah El Mekki, Samar Mohamed Magdy, AbdelRahim A. Elmadany, Omer Nacar, El Moatez Billah Nagoudi, Reem Abdel-Salam, Hanin Atwany, Youssef Nafea, Abdulfattah Mohammed Yahya, Rahaf Alhamouri, Hamzah A. Alsayadi, Hiba Zayed, Sara Shatnawi, Serry Sibaee, Yasir Ech-chammakhy, Walid Al-Dhabyani, Marwa Mohamed Ali, Imen Jarraya, Ahmed Oumar El-Shangiti, Aisha Alraeesi, Mohammed Anwar AL-Ghrawi, Abdulrahman S. Al-Batati, Elgizouli Mohamed, Noha Taha Elgindi, Muhammed Saeed, Houdaifa Atou, Issam Ait Yahia, Abdelhak Bouayad, Mohammed Machrouh, Amal Makouar, Dania Alkawi, Mukhtar Mohamed, Safaa Taher Abdelfadil, Amine Ziad Ounnoughene, Anfel Rouabhia, Rwaa Assi, Ahmed Sorkatti, Mohamedou Cheikh Tourad, Anis Koubaa, Ismail Berrada, Mustafa Jarrar, Shady Shehata and Muhammad Abdul-Mageed. 2025. Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{alwajih2025palm,
  title = {{Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs}},
  author = {Fakhraddin Alwajih, Abdellah {El Mekki}, Samar Mohamed Magdy, AbdelRahim A. Elmadany, Omer Nacar, El Moatez Billah Nagoudi, Reem Abdel-Salam, Hanin Atwany, Youssef Nafea, Abdulfattah Mohammed Yahya, Rahaf Alhamouri, Hamzah A. Alsayadi, Hiba Zayed, Sara Shatnawi, Serry Sibaee, Yasir Ech-chammakhy, Walid Al-Dhabyani, Marwa Mohamed Ali, Imen Jarraya, Ahmed Oumar El-Shangiti, Aisha Alraeesi, Mohammed Anwar AL-Ghrawi, Abdulrahman S. Al-Batati, Elgizouli Mohamed, Noha Taha Elgindi, Muhammed Saeed, Houdaifa Atou, Issam Ait Yahia, Abdelhak Bouayad, Mohammed Machrouh, Amal Makouar, Dania Alkawi, Mukhtar Mohamed, Safaa Taher Abdelfadil, Amine Ziad Ounnoughene, Anfel Rouabhia, Rwaa Assi, Ahmed Sorkatti, Mohamedou Cheikh Tourad, Anis Koubaa, Ismail Berrada, Mustafa Jarrar, Shady Shehata and Muhammad Abdul-Mageed},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1579},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1579}
}
```

- [← previous OG-RAG: Ontology-grounded retrieval-augmented generation for large language models](https://md.research.elamur.ai/free/en/papers/sharma-2025-og-rag-ontology-grounded-retrieval/)
- [next → SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models](https://md.research.elamur.ai/free/en/papers/chen-2025-sft-rl-early-investigation-training/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/)

## References

40 references, 34 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.acl-long.551)
2. Muhammad Abdul-Mageed, Amr Keleg, AbdelRahim Elmadany, Chiyu Zhang, Injy Hamed, Walid Magdy, Houda Bouamor, and Nizar Habash. 2024. [NADI 2024: The fifth nuanced Arabic dialect identification shared task](https://md.research.elamur.ai/free/en/papers/mageed-2024-nadi-2024-fifth-nuanced-arabic/). In Proceedings of The Second Arabic Natural Language Processing Conference, pages 709–728, Bangkok, Thailand. Association for Computational Linguistics.
3. Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Alham Fikri Aji, Jacki O’Neill, Ashutosh Modi, and Monojit Choudhury. 2024. [Towards measuring and modeling "culture" in llms: A survey](https://md.research.elamur.ai/free/en/papers/adilazuarda-2024-measuring-modeling-culture-llms-survey/). Preprint, arXiv:2403.15412. [↗](https://arxiv.org/abs/2403.15412)
4. Badr AlKhamissi, Muhammad ElNokrashy, Mai Alkhamissi, and Mona Diab. 2024. [Investigating cultural alignment of large language models](https://md.research.elamur.ai/free/en/papers/alkhamissi-2024-investigating-cultural-alignment-large-language/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12404–12422, Bangkok, Thailand. Association for Computational Linguistics.
5. Fakhraddin Alwajih, Samar Mohamed Magdy, Abdellah El Mekki, Omer Nacar, Youssef Nafea, Safaa Taher Abdelfadil, Abdulfattah Mohammed Yahya, Hamzah Luqman, Nada Almarwani, Samah Aloufi, Baraah Qawasmeh, Houdaifa Atou, Serry Sibaee, Hamzah A. Alsayadi, Walid Al-Dhabyani, Maged S. Al-shaibani, Aya El aatar, Nour Qandos, Rahaf Alhamouri, Samar Ahmad, Razan Khassib, Lina Hamad, Mohammed Anwar AL-Ghrawi, Fatimah Alshamari, Cheikh Malainine, Doaa Qawasmeh, Aminetou Yacoub, Tfeil moilid, Ruwa AbuH-weidi, Ahmed Aboeitta, Vatimetou Mohamed Lemin, Reem Abdel-Salam, Ahlam Bashiti, Adel Ammar, Aisha Alansari, Ahmed Ashraf, Nora Alturayeif, Sara Shatnawi, Alcides Alcoba Inciarte, Abdel-Rahim A. Elmadany, Mohamedou cheikh tourad, Ismail Berrada, Mustafa Jarrar, Shady Shehata, and Muhammad Abdul-Mageed. 2025. [Pearl: A multimodal culturally-aware arabic instruction dataset](https://md.research.elamur.ai/free/en/papers/alwajih-2025-pearl-multimodal-culturally-aware-arabic/). Preprint, arXiv:2505.21979. [↗](https://arxiv.org/abs/2505.21979)
6. Fakhraddin Alwajih, El Moatez Billah Nagoudi, Gagan Bhatia, Abdelrahman Mohamed, and Muhammad Abdul-Mageed. 2024. [Peacock: A family of Arabic multimodal large language models and benchmarks](https://md.research.elamur.ai/free/en/papers/alwajih-2024-peacock-family-arabic-multimodal-large/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12753–12776, Bangkok, Thailand. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2403.01031)
7. Zaid Alyafeai, Khalid Almubarak, Ahmed Ashraf, Deema Alnuhait, Saied Alshahrani, Gubran Abdulrahman, Gamil Ahmed, Qais Gawah, Zead Saleh, Mustafa Ghaleb, Yousef Ali, and Maged Al-shaibani. 2024a. [CIDAR: Culturally relevant instruction dataset for Arabic](https://md.research.elamur.ai/free/en/papers/alyafeai-2024-cidar-culturally-relevant-instruction-dataset/). In Findings of the Association for Computational Linguistics ACL 2024, pages 12878– 12901, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2402.03177)
8. Zaid Alyafeai, Khalid Almubarak, Ahmed Ashraf, Deema Alnuhait, Saied Alshahrani, Gubran AQ Abdulrahman, Gamil Ahmed, Qais Gawah, Zead Saleh, Mustafa Ghaleb, et al. 2024b. [Cidar: Culturally relevant instruction dataset for arabic](https://md.research.elamur.ai/free/en/papers/alyafeai-2024-cidar-culturally-relevant-instruction-dataset/). arXiv preprint arXiv:2402.03177. [↗](https://arxiv.org/abs/2402.03177)
9. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104)
10. Shane Arora, Marzena Karpinska, Hung-Ting Chen, Ipsita Bhattacharjee, Mohit Iyyer, and Eunsol Choi. 2024. [Calmqa: Exploring culturally specific long-form question answering across 23 languages](https://md.research.elamur.ai/free/en/papers/arora-2025-calmqa-exploring-culturally-specific-long/). Preprint, arXiv:2406.17761. [↗](https://arxiv.org/abs/2406.17761)
11. M Saiful Bari, Yazeed Alnumay, Norah A. Alzahrani, Nouf M. Alotaibi, Hisham A. Alyahya, Sultan Al-Rashed, Faisal A. Mirza, Shaykhah Z. Alsubaie, Hassan A. Alahmed, Ghadah Alabduljabbar, Raghad Alkhathran, Yousef Almushayqih, Raneem Alnajim, Salman Alsubaihi, Maryam Al Mansour, Majed Alrubaian, Ali Alammari, Zaki Alawami, Abdulmohsen Al-Thubaity, Ahmed Abdelali, Jeril Kuriakose, Abdalghani Abujabal, Nora Al-Twairesh, Areeb Alowisheq, and Haidar Khan. 2024. [Allam: Large language models for arabic and english](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/). Preprint, arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390)
12. El Moatez Billah Nagoudi, Muhammad Abdul-Mageed, AbdelRahim Elmadany, Alcides Inciarte, and Md Tawkat Islam Khondaker. 2023. [JASMINE: Arabic GPT models for few-shot learning](https://md.research.elamur.ai/free/en/papers/nagoudi-2023-jasmine-arabic-gpt-models-few/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16721–16744, Singapore. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2212.10755)
13. Anastasiia Demidova, Hanin Atwany, Nour Rabih, Sanad Sha’ban, and Muhammad Abdul-Mageed. 2024. [John vs. ahmed: Debate-induced bias in multilingual LLMs](https://md.research.elamur.ai/free/en/papers/demidova-2024-john-vs-ahmed-debate-induced/). In Proceedings of The Second Arabic Natural Language Processing Conference, pages 193–209, Bangkok, Thailand. Association for Computational Linguistics.
14. Abdellah El Mekki, Houdaifa Atou, Omer Nacar, Shady Shehata, and Muhammad Abdul-Mageed. 2025. Nilechat: Towards linguistically diverse and culturally aware llms for local communities. Preprint, arXiv:2505.18383. [↗](https://arxiv.org/abs/2505.18383)
15. AbdelRahim Elmadany, ElMoatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2023. [ORCA: A challenging benchmark for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/elmadany-2023-orca-challenging-benchmark-arabic-language/). In Findings of the Association for Computational Linguistics: ACL 2023, pages 9559–9586, Toronto, Canada. Association for Computational Linguistics.
16. Yi Fung, Ruining Zhao, Jae Doo, Chenkai Sun, and Heng Ji. 2024. [Massively multi-cultural knowledge acquisition & lm benchmarking](https://md.research.elamur.ai/free/en/papers/fung-2024-massively-multi-cultural-knowledge-acquisition/). Preprint, arXiv:2402.09369. [↗](https://arxiv.org/abs/2402.09369)
17. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Abdulmohsen Alharthi, Bang An, Juncai He, Ziche Liu, Zhiyi Zhang, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, and Jinchao Xu. 2024. [Acegpt, localizing large language models in arabic](https://md.research.elamur.ai/free/en/papers/huang-2024-acegpt-localizing-large-language-models/). Preprint, arXiv:2309.12053. [↗](https://arxiv.org/abs/2309.12053)
18. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. [The interplay of variant, size, and task type in Arabic pre-trained language models](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 92– 104, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
19. Amr Keleg and Walid Magdy. 2023. DLAMA: A framework for curating culturally diverse facts for probing the knowledge of pretrained language models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6245–6266, Toronto, Canada. Association for Computational Linguistics. Basel Mousi, Nadir Durrani, Fatema Ahmad, Md. [↗](https://doi.org/10.48448/8f4w-9n61)
20. Arid Hasan, Maram Hasanain, Tameem Kabbani, Fahim Dalvi, Shammur Absar Chowdhury, and Firoj Alam. 2024a. [AraDiCE: Benchmarks for dialectal and cultural capabilities in llms](https://md.research.elamur.ai/free/en/papers/mousi-2025-aradice-benchmarks-dialectal-cultural-capabilities/).
21. Basel Mousi, Nadir Durrani, Fatema Ahmad, Md Arid Hasan, Maram Hasanain, Tameem Kabbani, Fahim Dalvi, Shammur Absar Chowdhury, and Firoj Alam. 2024b. [Aradice: Benchmarks for dialectal and cultural capabilities in llms](https://md.research.elamur.ai/free/en/papers/mousi-2025-aradice-benchmarks-dialectal-cultural-capabilities/). arXiv preprint arXiv:2409.11404. [↗](https://arxiv.org/abs/2409.11404)
22. Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, Víctor Gutiérrez-Basulto, Yazmín Ibáñez-García, Hwaran Lee, Shamsuddeen Hassan Muhammad, Kiwoong Park, Anar Sabuhi Rzayev, Nina White, Seid Muhie Yimam, Mohammad Taher Pilehvar, Nedjma Ousidhoum, Jose Camacho-Collados, and Alice Oh. 2024a. [Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages](https://md.research.elamur.ai/free/en/papers/myung-2024-blend-benchmark-llms-everyday-knowledge/). Preprint, arXiv:2406.09948. [↗](https://arxiv.org/abs/2406.09948)
23. Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, et al. 2024b. [Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages](https://md.research.elamur.ai/free/en/papers/myung-2024-blend-benchmark-llms-everyday-knowledge/). arXiv preprint arXiv:2406.09948. [↗](https://arxiv.org/abs/2406.09948)
24. El Moatez Billah Nagoudi, AbdelRahim Elmadany, Ahmed El-Shangiti, and Muhammad Abdul-Mageed. 2023. [Dolphin: A challenging and diverse benchmark for Arabic NLG](https://md.research.elamur.ai/free/en/papers/nagoudi-2023-dolphin-challenging-diverse-benchmark-arabic/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1404–1422, Singapore. Association for Computational Linguistics.
25. Tarek Naous, Michael Ryan, Alan Ritter, and Wei Xu. 2024. [Having beer after prayer? measuring cultural bias in large language models](https://md.research.elamur.ai/free/en/papers/naous-2023-having-beer-after-prayer-measuring/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16366–16393, Bangkok, Thailand. Association for Computational Linguistics.
26. Tuan-Phong Nguyen, Simon Razniewski, Aparna Varde, and Gerhard Weikum. 2023. [Extracting cultural commonsense knowledge at scale](https://md.research.elamur.ai/free/en/papers/nguyen-2022-extracting-cultural-commonsense-knowledge-scale/). In Proceedings of the ACM Web Conference 2023, volume 21 of WWW ’23, page 1907–1917. ACM.
27. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Preprint, arXiv:2203.02155. [↗](https://arxiv.org/abs/2203.02155)
28. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. [Language models as knowledge bases?](https://md.research.elamur.ai/free/en/papers/petroni-2019-language-models-as-knowledge-bases/) In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463–2473, Hong Kong, China. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d19-1250)
29. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+and+Ilya+Sutskever%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
30. Aida Ramezani and Yang Xu. 2023. [Knowledge of cultural moral norms in large language models](https://md.research.elamur.ai/free/en/papers/ramezani-2023-knowledge-cultural-moral-norms-large/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 428–446, Toronto, Canada. Association for Computational Linguistics. [↗](https://doi.org/10.48448/gacr-7y86)
31. Haitham Seelawi, Ibraheem Tuffaha, Mahmoud Gzawi, Wael Farhan, Bashar Talafha, Riham Badawi, Zyad Sober, Oday Al-Dweik, Abed Alhakim Freihat, and Hussein Al-Natsheh. 2021. [ALUE: Arabic language understanding evaluation](https://md.research.elamur.ai/free/en/papers/seelawi-2021-alue-arabic-language-understanding-evaluation/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 173–184, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
32. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondos Mahmoud Bsharat, Alham Fikri Aji, Zhiqiang Shen, Zhengzhong Liu, Natalia Vassilieva, Joel Hestness, Andy Hock, Andrew Feldman, Jonathan Lee, Andrew Jackson, Hector Xuguang Ren, Preslav Nakov, Timothy Baldwin, and Eric Xing. 2023. [Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models](https://md.research.elamur.ai/free/en/papers/sengupta-2023-jais-jais-chat-arabic-centric/). Preprint, arXiv:2308.16149. [↗](https://arxiv.org/abs/2308.16149)
33. Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, Majd Hawasly, Mus’ab Husaini, Soon-Gyo Jung, Ji Kim Lucas, Walid Magdy, Safa Messaoud, Abubakr Mohamed, Tasnim Mohiuddin, Basel Mousi, Hamdy Mubarak, Ahmad Musleh, Zan Naeem, Mourad Ouzzani, Dorde Popovic, Amin Sadeghi, Husrev Taha Sencar, Mohammed Shinoy, Omar Sinan, Yifan Zhang, Ahmed Ali, Yassine El Kheir, Xiaosong Ma, and Chaoyi Ruan. 2025. [Fanar: An arabic-centric multimodal generative ai platform](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/). Preprint, arXiv:2501.13944. [↗](https://arxiv.org/abs/2501.13944)
34. Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D. Sculley. 2017. No classification without representation: Assessing geodiversity issues in open data sets for the developing world. Preprint, arXiv:1711.08536. [↗](https://arxiv.org/abs/1711.08536)
35. Maxim Tkachenko, Mikhail Malyuk, Andrey Holmanyuk, and Nikolai Liubimov. 2020. Label studio: Data labeling software. Open source software available from [https://github.com/heartexlabs/labelstudio](https://github.com/heartexlabs/labelstudio), 2022. [↗](https://github.com/heartexlabs/labelstudio)
36. Weiyan Shi, Ryan Li, Yutong Zhang, Caleb Ziems, Chunhua yu, Raya Horesh, Rogério Abreu de Paula, and Diyi Yang. 2024. [Culturebank: An online community-driven knowledge base towards culturally aware language technologies](https://md.research.elamur.ai/free/en/papers/shi-2024-culturebank-online-community-driven-knowledge/). Preprint, arXiv:2404.15238. [↗](https://arxiv.org/abs/2404.15238)
37. Yuemei Xu, Ling Hu, Jiayi Zhao, Zihan Qiu, Yuqi Ye, and Hanwen Gu. 2024. A survey on multilingual large language models: Corpora, alignment, and bias. Preprint, arXiv:2404.00929. [↗](https://arxiv.org/abs/2404.00929)
38. Shivalika Singh, Freddie Vargus, Daniel D’souza, Börje Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura O’Mahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergun, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Chien, Sebastian Ruder, Surya Guthikonda, Emad Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, and Sara Hooker. 2024a. [Aya dataset: An open-access collection for multilingual instruction tuning](https://md.research.elamur.ai/free/en/papers/singh-2024-aya-dataset-open-access-collection/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11521–11567, Bangkok, Thailand. Association for Computational Linguistics.
39. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Preprint, arXiv:2306.05685. [↗](https://arxiv.org/abs/2306.05685)
40. Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergün, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Minh Chien, Sebastian Ruder, Surya Guthikonda, Emad A. Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, and Sara Hooker. 2024b. [Aya dataset: An open-access collection for multilingual instruction tuning](https://md.research.elamur.ai/free/en/papers/singh-2024-aya-dataset-open-access-collection/). Preprint, arXiv:2402.06619. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, [↗](https://arxiv.org/abs/2402.06619)

*Updated 2 November 2026*