---
title: "Phoenix at Palmx — paper by Atou et al. (2025)"
description: "Open small LLMs often know little about Arab culture, largely because their training data is dominated… 25 references, cited by 1 paper here."
authors: ["Houdaifa Atou", "Issam Ait Yahia", "Ismail Berrada"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks"
source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.116/"
doi: "10.18653/v1/2025.arabicnlp-sharedtasks.116"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/atou-2025-phoenix-palmx-exploring-data-augmentation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/atou-2025-phoenix-palmx-exploring-data-augmentation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) / Phoenix at Palmx: Exploring Data Augmentation for Arabic Cultural Question Answering

# Phoenix at Palmx: Exploring Data Augmentation for Arabic Cultural Question Answering

**Authors:** [Houdaifa Atou](https://md.research.elamur.ai/free/en/authors/houdaifa-atou/), [Issam Ait Yahia](https://md.research.elamur.ai/free/en/authors/issam-ait-yahia/) and [Ismail Berrada](https://md.research.elamur.ai/free/en/authors/ismail-berrada/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.116/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.116.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.116](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.116) · [OpenAlex](https://openalex.org/W4416037750)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/atou-2025-phoenix-palmx-exploring-data-augmentation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Open small LLMs often know little about Arab culture, largely because their training data is dominated by English. For the Palmx 2025 shared task, which uses MSA multiple-choice questions on general Arab culture and on Islamic culture, the authors built two systems: Phoenix (general culture) and PhoenixIs (Islamic culture). The official training sets are small, so they used Gemini models to expand them in three ways. First, they rewrote each existing question in other words. Second, they generated new questions modelled on a single question–answer pair. Third, they gave the model the entire seed dataset and asked for new questions on related themes. A random portion of the generated items was checked by people. They selected a base model for each subtask by comparing zero-shot results, then fine-tuned it with LoRA. Phoenix reached 71.35% on the blind test and placed fourth, less than a point behind the winner. PhoenixIs reached 83.82% and placed second. The work shows that cheap LLM-driven augmentation can make a small open model competitive on culturally specific QA.

## Key points
- Seed data: Palmx-GC has 2,000 train, 500 dev and 1,000 blind-test items. Palmx-IC has 600 train, 300 dev and 1,000 blind-test items.
- Phoenix was fine-tuned on 18,742 questions: 2,000 seed, 6,000 paraphrased, 6,411 sample-based and 4,331 dataset-based. Paraphrasing and dataset-based generation used Gemini 2.5 Pro; sample-based generation used Gemini 2.5 Flash.
- PhoenixIs was fine-tuned on 4,400 questions: 600 Palmx-IC, 1,800 paraphrases and all 2,000 Palmx-GC items. For religious content, augmentation was limited to paraphrasing to protect theological accuracy.
- Base models were chosen by zero-shot dev accuracy: Fanar-1-9B-Instruct for GC (72.40%) and ALLaM-7B-Instruct for IC (73.00%). Training used LoRA (R=64, α=16, dropout 0.1, lr 2e-4, effective batch 128) on a single A100.
- Ablations report the mean of three seeds:
  - GC rose from 77.73 (seed only) to 80.07 with paraphrases, 80.60 with sample-based items and 80.93 with dataset-based items.
  - IC rose from 73.73 to 75.11 with paraphrases and to 78.56 after adding Palmx-GC.

## Interesting ideas
- The authors generate synthetic data at three levels (rewording one item, building new items from one example, building new items from the whole set) and add each level in turn to measure its separate effect, which gives a clean picture of where the gains come from.
- How freely data is generated depends on how risky the domain is: sensitive religious material got only meaning-preserving rewrites, while general culture got fully new questions.
- Several candidate backbones are compared zero-shot on the dev set before any fine-tuning, so the expensive training budget goes only to the most promising one.
- Adding the related-domain general-culture set to the small Islamic-culture training set gave the largest single IC gain, more than paraphrasing did.
- Synthetic items are quality-checked by human review of a random sample, and the authors openly state that most generated data went unreviewed.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): School-test and fable QA sets could be expanded the same way, and paraphrased versions of a question make a natural consistency check, since an explainable reader should return the same answer for each rewording.
- **I2** (learn): LLM teacher tiers could follow a risk policy, allowing only paraphrase for fragile content such as maths word problems with exact quantities and free generation elsewhere.
- **I3** (tree): A quick zero-shot or probe comparison of encoder backbones could pick the one for gparse before committing GPU time to full training.
- **I4** (tree): Treebanks from neighbouring genres or other UD dialects, after conversion, could be mixed in when in-domain training data is thin, with an ablation to confirm the benefit.
- **I5** (learn): Silver data agreed by witness parsers could get regular random audits, and the share actually reviewed could be reported next to results as an honest quality signal.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Université Mohammed VI Polytechnique](https://md.research.elamur.ai/free/en/universities/universite-mohammed-vi-polytechnique/) — University, Marrakesh, Morocco · 18 works · 171 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works

## Cite

Cite as: Houdaifa Atou, Issam Ait Yahia and Ismail Berrada. 2025. Phoenix at Palmx: Exploring Data Augmentation for Arabic Cultural Question Answering. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{atou2025phoenix,
  title = {{Phoenix at Palmx: Exploring Data Augmentation for Arabic Cultural Question Answering}},
  author = {Houdaifa Atou, Issam Ait Yahia and Ismail Berrada},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.116},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.116}
}
```

- [← previous Phantoms at BAREC Shared Task 2025: Enhancing Arabic Readability Prediction with Hybrid BERT and Linguistic Features](https://md.research.elamur.ai/free/en/papers/alhassan-2025-phantoms-barec-shared-task-2025/)
- [next → Pixels at BAREC Shared Task 2025: Visual Arabic Readability Assessment](https://md.research.elamur.ai/free/en/papers/sapirstein-2025-pixels-barec-shared-task-2025/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/)

## References

25 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Oussama Akallouch and Khalid Fardousse. 2025. In-context learning for low-resource machine translation: A study on tarifit with large language models. Algorithms, 18(8):489. [↗](https://doi.org/10.3390/a18080489)
2. Badr AlKhamissi, Muhammad ElNokrashy, Mai Alkhamissi, and Mona Diab. 2024. [Investigating cultural alignment of large language models](https://md.research.elamur.ai/free/en/papers/alkhamissi-2024-investigating-cultural-alignment-large-language/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12404–12422, Bangkok, Thailand. Association for Computational Linguistics.
3. Fakhraddin Alwajih, Abdellah El Mekki, Samar Mohamed Magdy, AbdelRahim A. Elmadany, Omer Nacar, El Moatez Billah Nagoudi, Reem Abdel-Salam, Hanin Atwany, Youssef Nafea, Abdulfattah Mohammed Yahya, Rahaf Alhamouri, Hamzah A. Alsayadi, Hiba Zayed, Sara Shatnawi, Serry Sibaee, Yasir Ech-chammakhy, Walid Al-Dhabyani, Marwa Mohamed Ali, Imen Jarraya, and 25 others. 2025a. [Palm: A culturally inclusive and linguistically diverse dataset for Arabic LLMs](https://md.research.elamur.ai/free/en/papers/alwajih-2025-palm-culturally-inclusive-linguistically-diverse/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 32871–32894, Vienna, Austria. Association for Computational Linguistics. of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2016–2039, Mexico City, Mexico. Association for Computational Linguistics.
4. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). ICLR, 1(2):3.
5. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Song Dingjie, Zhihong Chen, Mosen Alharthi, Bang An, Juncai He, Ziche Liu, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, and Jinchao Xu. 2024. [AceGPT, localizing large language models in Arabic](https://md.research.elamur.ai/free/en/papers/huang-2024-acegpt-localizing-large-language-models/). In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8139–8163, Mexico City, Mexico. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2309.12053)
6. Mustafa Jarrar, Nagham Hamad, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2024. [Wojoodner 2024: The second arabic named entity recognition shared task](https://md.research.elamur.ai/free/en/papers/jarrar-2024-wojoodner-2024-second-arabic-named/). arXiv preprint arXiv:2407.09936. [↗](https://arxiv.org/abs/2407.09936)
7. Raviraj Joshi, Kanishk Singla, Anusha Kamath, Raunak Kalani, Rakesh Paul, Utkarsh Vaidya, Sanjay Singh Chauhan, Niranjan Wartikar, and Eileen Long. 2024. Adapting multilingual llms to low-resource languages using continued pre-training and synthetic corpus. arXiv preprint arXiv:2410.14815. [↗](https://arxiv.org/abs/2410.14815)
8. Fakhraddin Alwajih, Abdellah El Mekki, Hamdy Mubarak, Majd Hawasly, Abubakr Mohamed, and Muhammad Abdul-Mageed. 2025b. [PalmX 2025: The First Shared Task on Benchmarking LLMs on Arabic and Islamic Culture](https://md.research.elamur.ai/free/en/papers/alwajih-2025-palmx-2025-first-shared-task/). In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics. Co-located with EMNLP 2025, November 5–9.
9. Raviraj Joshi, Kanishk Singla, Anusha Kamath, Raunak Kalani, Rakesh Paul, Utkarsh Vaidya, Sanjay Singh Chauhan, Niranjan Wartikar, and Eileen Long. 2025. Adapting multilingual LLMs to low-resource languages using continued pre-training and synthetic corpus: A case study for Hindi LLMs. In Proceedings of the First Workshop on Natural Language Processing for Indo-Aryan and Dravidian Languages, pages 50–57, Abu Dhabi. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2410.14815)
10. M Saiful Bari, Yazeed Alnumay, Norah A Alzahrani, Nouf M Alotaibi, Hisham A Alyahya, Sultan Al-Rashed, Faisal A Mirza, Shaykhah Z Alsubaie, Hassan A Alahmed, Ghadah Alabduljabbar, and 1 others. 2024. [Allam: Large language models for arabic and english](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/). arXiv preprint arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390)
11. Cheng Li, Mengzhuo Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. 2024. [Culturellm: Incorporating cultural differences into large language models](https://md.research.elamur.ai/free/en/papers/li-2024-culturellm-incorporating-cultural-differences-large/). In Advances in Neural Information Processing Systems, volume 37, pages 84799–84838. Curran Associates, Inc.
12. Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. [Assessing cross-cultural alignment between chatgpt and human societies: An empirical study](https://md.research.elamur.ai/free/en/papers/cao-2023-assessing-cross-cultural-alignment-between/). arXiv preprint arXiv:2303.17466. [↗](https://arxiv.org/abs/2303.17466)
13. Chen Cecilia Liu, Anna Korhonen, and Iryna Gurevych. 2025. Cultural learning-based culture adaptation of language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3114–3134, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Chen+Cecilia+Liu%2C+Anna+Korhonen%2C+and+Iryna+Gurevych%2E+2025%2E+Cultural+learning%2Dbased+culture+adaptation+of+language+models%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Association+for+Computational)
14. Chen Cecilia Liu, Fajri Koto, Timothy Baldwin, and Iryna Gurevych. 2024. Are multilingual LLMs culturally-diverse reasoners? an investigation into multicultural proverbs and sayings. In Proceedings Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, and 1 others. 2024. [Best practices and lessons learned on synthetic data](https://md.research.elamur.ai/free/en/papers/liu-2024-best-practices-lessons-learned-synthetic/). arXiv preprint arXiv:2404.07503. [↗](https://arxiv.org/abs/2404.07503)
15. Reem I Masoud, Martin Ferianc, Philip Colin Treleaven, and Miguel RD Rodrigues. 2024. Llm alignment using soft prompt tuning: The case of cultural alignment. In Workshop on Socially Responsible Language Modelling Research. [find](https://scholar.google.com/scholar?q=Reem+I+Masoud%2C+Martin+Ferianc%2C+Philip+Colin+Treleaven%2C+and+Miguel+RD+Rodrigues%2E+2024%2E+Llm+alignment+using+soft+prompt+tuning%3A+The+case+of+cultural+alignment%2E+In+Workshop+on+Socially+Responsible+Langua)
16. Abdellah El Mekki, Houdaifa Atou, Omer Nacar, Shady Shehata, and Muhammad Abdul-Mageed. 2025. Nilechat: Towards linguistically diverse and culturally aware llms for local communities. arXiv preprint arXiv:2505.18383. [↗](https://arxiv.org/abs/2505.18383)
17. Hussein Mozannar, Elie Maamary, Karl El Hajal, and Hazem Hajj. 2019. [Neural Arabic question answering](https://md.research.elamur.ai/free/en/papers/mozannar-2019-neural-arabic-question-answering/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 108–118, Florence, Italy. Association for Computational Linguistics. City, Mexico. Association for Computational Linguistics.
18. Yan Tao, Olga Viberg, Ryan S Baker, and René F Kizilcec. 2024. Cultural bias and cultural alignment of large language models. PNAS nexus, 3(9):pgae346. [↗](https://doi.org/10.1093/pnasnexus/pgae346)
19. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, and 1 others. 2025. [Fanar: An arabic-centric multimodal generative ai platform](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/). arXiv preprint arXiv:2501.13944. [↗](https://arxiv.org/abs/2501.13944)
20. Issam Yahia, Houdaifa Atou, and Ismail Berrada. 2024. [Addax at WojoodNER 2024: Attention-based dual-channel neural network for Arabic named entity recognition](https://md.research.elamur.ai/free/en/papers/yahia-2024-addax-wojoodner-2024-attention-based/). In Proceedings of the Second Arabic Natural Language Processing Conference, pages 867– 873, Bangkok, Thailand. Association for Computational Linguistics.
21. Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, Víctor Gutiérrez-Basulto, Yazmín Ibáñez García, Hwaran Lee, Shamsuddeen Hassan Muhammad, Kiwoong Park, Anar Sabuhi Rzayev, Nina White, Seid Muhie Yimam, Mohammad Taher Pilehvar, and 3 others. 2024. [Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages](https://md.research.elamur.ai/free/en/papers/myung-2024-blend-benchmark-llms-everyday-knowledge/). In Advances in Neural Information Processing Systems, volume 37, pages 78104–78146. Curran Associates, Inc.
22. Tarek Naous, Michael J Ryan, Alan Ritter, and Wei Xu. 2024. [Having beer after prayer? measuring cultural bias in large language models](https://md.research.elamur.ai/free/en/papers/naous-2023-having-beer-after-prayer-measuring/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16366–16393, Bangkok, Thailand. Association for Computational Linguistics.
23. Roberto Navigli, Simone Conia, and Björn Ross. 2023. Biases in large language models: origins, inventory, and discussion. ACM Journal of Data and Information Quality, 15(2):1–21. [find](https://scholar.google.com/scholar?q=Roberto+Navigli%2C+Simone+Conia%2C+and+Bj%C3%B6rn+Ross%2E+2023%2E+Biases+in+large+language+models%3A+origins%2C+inventory%2C+and+discussion%2E+ACM+Journal+of+Data+and+Information+Quality%2C+15%282%29%3A1%E2%80%9321%2E)
24. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, and 1 others. 2023. [Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models](https://md.research.elamur.ai/free/en/papers/sengupta-2023-jais-jais-chat-arabic-centric/). arXiv preprint arXiv:2308.16149. [↗](https://arxiv.org/abs/2308.16149)
25. Siqi Shen, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Soujanya Poria, and Rada Mihalcea. 2024. Understanding the capabilities and limitations of large language models for cultural commonsense. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 5668–5680, Mexico Effectiveness Analysis [↗](https://doi.org/10.48550/arxiv.2405.04655)

*Updated 1 November 2026*