---
title: "Fann or Flop — paper by Ghallabi et al. (2025)"
description: "The authors introduce Fann or Flop, an open benchmark of 6,984 poems paired with explanations. 44 references, cited by 13 papers here."
authors: ["Wafa Al Ghallabi", "Ritesh Thawkar", "Sara Ghaboura", "Ketan Pravin More", "Omkar Thawakar", "Hisham Cholakkal", "Salman Khan", "Rao Muhammad Anwer"]
year: 2025
venue: "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2025.emnlp-main.1023/"
doi: "10.18653/v1/2025.emnlp-main.1023"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ghallabi-2025-fann-or-flop-multigenre-multiera/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ghallabi-2025-fann-or-flop-multigenre-multiera/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/5/) / Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs

# Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs

**Authors:** Wafa Al Ghallabi, Ritesh Thawkar, [Sara Ghaboura](https://md.research.elamur.ai/free/en/authors/sara-ghaboura/), [Ketan Pravin More](https://md.research.elamur.ai/free/en/authors/ketan-pravin-more/), [Omkar Thawakar](https://md.research.elamur.ai/free/en/authors/omkar-thawakar/), [Hisham Cholakkal](https://md.research.elamur.ai/free/en/authors/hisham-cholakkal/), [Salman Khan](https://md.research.elamur.ai/free/en/authors/salman-khan/) and [Rao Muhammad Anwer](https://md.research.elamur.ai/free/en/authors/rao-muhammad-anwer/) · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2025](https://md.research.elamur.ai/free/en/venues/emnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.emnlp-main.1023/) · [PDF](https://aclanthology.org/2025.emnlp-main.1023.pdf) · [DOI 10.18653/v1/2025.emnlp-main.1023](https://doi.org/10.18653/v1/2025.emnlp-main.1023) · [OpenAlex](https://openalex.org/W4416036456)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [13](https://md.research.elamur.ai/free/en/papers/ghallabi-2025-fann-or-flop-multigenre-multiera/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large language models handle Arabic well on mainstream benchmarks, but it is unclear whether they grasp Classical Arabic as found in poetry. Poetry depends on metaphor, meter, rhetoric and historical context, and existing Arabic resources cover these only partly. The authors introduce Fann or Flop, an open benchmark of 6,984 poems paired with explanations. The poems span 12 historical eras, from pre-Islamic times to the modern period, and include 14 genres and many metrical forms. Native Arabic speakers with domain expertise checked each sample, and literature scholars validated the era and genre taxonomy. Models generate explanations of poems, which are scored against human-written references using BERTScore, textual entailment, and human ratings of fluency and faithfulness. Closed models (GPT-4o, Gemini-2.5-Flash) score highest. Several open models are competitive, but most systems still miss cultural and figurative meaning, especially in older eras. The benchmark therefore shows a gap that standard Arabic tasks hide, and it gives a diagnostic for how deeply a model knows Classical Arabic.

## Key points
- The dataset contains 6,984 expert-verified poem–explanation pairs across 12 eras (e.g., Pre-Islamic, Umayyad, Abbasid, Andalusian, Mamluk, Ottoman, Modern) and 14 genres, with meters such as At-Tawil, Al-Kamel and Ar-Rojz alongside modern free verse.
- Gemini-2.5-Flash achieved the best textual entailment score (0.7475), and GPT-4o had the highest BERTScore.
- Among open models, Deepseek-V3, Aya-Expanse and Qwen-3 were competitive, while ALLaM-7B and AceGPT-v2 trailed clearly.
- In the era-wise BERTScore analysis, models performed worst on older poetry such as the Pre-Islamic and Umayyad eras, which suggests their strength lies in MSA rather than classical forms.
- The data is released on Hugging Face (omkarthawakar/FannOrFlop) and the evaluation suite on GitHub (mbzuai-oryx/FannOrFlop). The authors note two limitations: the benchmark covers only part of the poetic tradition, and current metrics cannot fully account for multiple valid interpretations of a poem.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Mohamed bin Zayed University of Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/mohamed-bin-zayed-university-of-artificial-intelligence/) — University, Abu Dhabi, United Arab Emirates · 124 works · 5,676 citations
- [Australian National University](https://md.research.elamur.ai/free/en/universities/australian-national-university/) — University, Canberra, Australia · 23 works · 6,522 citations
- [Aalto University](https://md.research.elamur.ai/free/en/universities/aalto-university/) — University, Espoo, Finland · 13 works · 648 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works

## Cite

Cite as: Wafa Al Ghallabi, Ritesh Thawkar, Sara Ghaboura, Ketan Pravin More, Omkar Thawakar, Hisham Cholakkal, Salman Khan and Rao Muhammad Anwer. 2025. Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ghallabi2025fann,
  title = {{Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs}},
  author = {Wafa {Al Ghallabi}, Ritesh Thawkar, Sara Ghaboura, Ketan Pravin More, Omkar Thawakar, Hisham Cholakkal, Salman Khan and Rao Muhammad Anwer},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.1023},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.1023}
}
```

- [← previous FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation](https://md.research.elamur.ai/free/en/papers/bayat-2025-factbench-dynamic-benchmark-wild-language/)
- [next → Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2025-harnessing-reasoning-economy-survey-efficient/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/5/)

## References

44 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, Shady Elbassuoni, Jad Doughman, AbdelRahim Elmadany, El Moatez Billah Nagoudi, Yorgo Zoughby, Ahmad Shaher, Iskander Gaba, Ahmed Helal, and Mohammed El-Razzaz. 2021. DiaLex: A benchmark for evaluating multi-dialectal Arabic word embeddings. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 11–20, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Muhammad+Abdul%2DMageed%2C+Shady+Elbassuoni%2C+Jad+Doughman%2C+AbdelRahim+Elmadany%2C+El+Moatez+Billah+Nagoudi%2C+Yorgo+Zoughby%2C+Ahmad+Shaher%2C+Iskander+Gaba%2C+Ahmed+Helal%2C+and+Mohammed+El%2DRazzaz%2E+2021%2E+DiaLex%3A+A+b)
2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2020. [Arbert & marbert: Deep bidirectional transformers for arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). arXiv preprint arXiv:2101.01785. [↗](https://arxiv.org/abs/2101.01785)
3. Sajawel Ahmed, Rob van der Goot, Misbahur Rehman, Carl Kruse, Ömer Özsoy, Alexander Mehler, and Gemma Roig. 2022. [Tafsir dataset: A novel multi-task benchmark for named entity recognition and topic modeling in classical Arabic literature](https://md.research.elamur.ai/free/en/papers/ahmed-2022-tafsir-dataset-novel-multi-task/). In Proceedings of the 29th International Conference on Computational Linguistics, pages 3753–3768, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
4. Google AI. 2025a. Gemini 2.0 flash. Large language model, accessed May 20, 2025. [find](https://scholar.google.com/scholar?q=Google+AI%2E+2025a%2E+Gemini+2%2E0+flash%2E+Large+language+model%2C+accessed+May+20%2C+2025%2E)
5. Google AI. 2025b. Gemini 2.5 flash. Large language model (Preview), accessed May 20, 2025. [find](https://scholar.google.com/scholar?q=Google+AI%2E+2025b%2E+Gemini+2%2E5+flash%2E+Large+language+model+%28Preview%29%2C+accessed+May+20%2C+2025%2E)
6. Abu Nasr al Jawhari. 10th Century. Taj al-lugha wa sihah al-arabiya - Wikipedia — en.wikipedia.org. [https://en.wikipedia.org/wiki/Abu_Nasr_](https://en.wikipedia.org/wiki/Abu_Nasr_) al-Jawhari. [Accessed 06-05-2025]. alsharekh. 2019. Al-mujam al muaser– lexicon.alsharekh.org. https://lexicon.alsharekh. org/. [Accessed 06-05-2025]. [↗](https://en.wikipedia.org/wiki/Abu_Nasr_)
7. AlSuyuti. 15th Century. Al-mizhar fi eulum allughat wa’anwaeiha - Wikipedia — en.wikipedia.org. [https://en.wikipedia.org/wiki/Al-Suyuti](https://en.wikipedia.org/wiki/Al-Suyuti). [Accessed 06-05-2025]. [↗](https://en.wikipedia.org/wiki/Al-Suyuti)
8. Zaid Alyafeai, Maged S Al-Shaibani, and Moataz Ahmed. 2023. [Ashaar: Automatic analysis and generation of arabic poetry using deep learning approaches](https://md.research.elamur.ai/free/en/papers/alyafeai-2023-ashaar-automatic-analysis-generation-arabic/). arXiv preprint arXiv:2307.06218. [↗](https://arxiv.org/abs/2307.06218)
9. Toni Andrews. 2024. Is Arabic The Richest Language In Words? - Interpreters & Translators, Inc. — ititranslates.com. [https://ititranslates.com/](https://ititranslates.com/) is-arabic-the-richest-language-in-words/. [Accessed 06-05-2025]. [↗](https://ititranslates.com/)
10. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). arXiv preprint arXiv:2003.00104. [↗](https://arxiv.org/abs/2003.00104)
11. Mujam Ar-Riyadh. 2025. Mujam ar-riyadh— dictionary.ksaa.gov.sa. [https://dictionary.ksaa.gov.sa/](https://dictionary.ksaa.gov.sa/). [Accessed 06-05-2025]. [↗](https://dictionary.ksaa.gov.sa/)
12. M Saiful Bari, Yazeed Alnumay, Norah A. Alzahrani, Nouf M. Alotaibi, Hisham A. Alyahya, Sultan Al-Rashed, Faisal A. Mirza, Shaykhah Z. Alsubaie, Hassan A. Alahmed, Ghadah Alabduljabbar, Raghad Alkhathran, Yousef Almushayqih, Raneem Alnajim, Salman Alsubaihi, Maryam Al Mansour, Majed Alrubaian, Ali Alammari, Zaki Alawami, Abdulmohsen Al-Thubaity, Ahmed Abdelali, Jeril Kuriakose, Abdalghani Abujabal, Nora Al-Twairesh, Areeb Alowisheq, and Haidar Khan. 2024. [Allam: Large language models for arabic and english](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/). Preprint, arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390)
13. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439. [↗](https://doi.org/10.1609/aaai.v34i05.6239)
14. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, et al. 2018. The madar arabic dialect corpus and lexicon. In Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018). [find](https://scholar.google.com/scholar?q=Houda+Bouamor%2C+Nizar+Habash%2C+Mohammad+Salameh%2C+Wajdi+Zaghouani%2C+Owen+Rambow%2C+Dana+Abdulrahim%2C+Ossama+Obeid%2C+Salam+Khalifa%2C+Fadhl+Eryani%2C+Alexander+Erdmann%2C+et+al%2E+2018%2E+The+madar+arabic+dialect+corpus)
15. Tuhin Chakrabarty, Arkadiy Saakyan, Debanjan Ghosh, and Smaranda Muresan. 2022. [Flute: Figurative language understanding through textual explanations](https://md.research.elamur.ai/free/en/papers/chakrabarty-2022-flute-figurative-language-understanding-through/). arXiv preprint arXiv:2205.12404. [↗](https://arxiv.org/abs/2205.12404)
16. Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. 2025. [Sharegpt4v: Improving large multi-modal models with better captions](https://md.research.elamur.ai/free/en/papers/chen-2023-sharegpt4v-improving-large-multi-modal/). In European Conference on Computer Vision, pages 370–387. Springer.
17. John Dang, Shivalika Singh, Daniel D’souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, et al. 2024. [Aya expanse: Combining research breakthroughs for a new multilingual frontier](https://md.research.elamur.ai/free/en/papers/dang-2024-aya-expanse-combining-research-breakthroughs/). arXiv preprint arXiv:2412.04261. [↗](https://arxiv.org/abs/2412.04261)
18. FIGLANG202. 2024. FigLang2024 — sites.google.com. [https://sites.google.com/view/figlang2024](https://sites.google.com/view/figlang2024). [Accessed 07-05-2025]. [↗](https://sites.google.com/view/figlang2024)
19. Giuseppe Gallipoli and Luca Cagliero. 2025. [It is not a piece of cake for gpt: Explaining textual entailment recognition in the presence of figurative language](https://md.research.elamur.ai/free/en/papers/gallipoli-2025-it-is-not-piece-cake/). In Proceedings of the 31st International Conference on Computational Linguistics, pages 9656–9674.
20. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948. [↗](https://arxiv.org/abs/2501.12948)
21. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. [Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/). arXiv preprint arXiv:2111.09543. [↗](https://arxiv.org/abs/2111.09543)
22. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Abdulmohsen Alharthi, Bang An, Juncai He, et al. 2023. [Acegpt, localizing large language models in arabic](https://md.research.elamur.ai/free/en/papers/huang-2024-acegpt-localizing-large-language-models/). arXiv preprint arXiv:2309.12053. [↗](https://arxiv.org/abs/2309.12053)
23. Salam Khalifa, Nizar Habash, Fadhl Eryani, Ossama Obeid, Dana Abdulrahim, and Meera Al Kaabi. 2018. [A morphologically annotated corpus of emirati Arabic](https://md.research.elamur.ai/free/en/papers/khalifa-2018-morphologically-annotated-corpus-emirati-arabic/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
24. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. [↗](https://arxiv.org/abs/2412.19437)
25. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. [↗](https://doi.org/10.3115/1073083.1073135)
26. Emmy Liu, Chen Cui, Kenneth Zheng, and Graham Neubig. 2022. [Testing the ability of language models to interpret figurative language](https://md.research.elamur.ai/free/en/papers/liu-2022-testing-ability-language-models-interpret/). arXiv preprint arXiv:2204.12632. [↗](https://arxiv.org/abs/2204.12632)
27. Maja Popović. 2017. [chrF++: words helping character n-grams](https://md.research.elamur.ai/free/en/papers/popovic-2017-chrf-words-helping-character-n/). In Proceedings of the Second Conference on Machine Translation, pages 612–618, Copenhagen, Denmark. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/w17-4770)
28. Quentin Malartic, Nilabhra Roy Chowdhury, Ruxandra Cojocaru, Mugariya Farooq, Giulia Campesan, Yasser Abdelaziz Dahou Djilali, Sanath Narayan, Ankit Singh, Maksim Velikanov, Basma El Amel Boussaha, et al. 2024. Falcon2-11b technical report. arXiv preprint arXiv:2407.14885. [↗](https://arxiv.org/abs/2407.14885)
29. Ibn Manzur. 14th Century. Lisan al-arab — en.wikipedia.org. [https://en.wikipedia.org/](https://en.wikipedia.org/) wiki/Ibn_Manzur. [Accessed 06-05-2025]. [↗](https://en.wikipedia.org/)
30. Karima Meftouh, Salima Harrat, Salma Jamoussi, Mourad Abbas, and Kamel Smaili. 2015. [Machine translation experiments on padic: A parallel arabic dialect corpus](https://md.research.elamur.ai/free/en/papers/meftouh-2015-machine-translation-experiments-padic-parallel/). In Proceedings of the 29th Pacific Asia conference on language, information and computation, pages 26–34.
31. Meta AI. 2024. Llama 3.2: Revolutionizing edge ai and vision with open, customizable models. [find](https://scholar.google.com/scholar?q=Meta+AI%2E+2024%2E+Llama+3%2E2%3A+Revolutionizing+edge+ai+and+vision+with+open%2C+customizable+models%2E)
32. Behrang Mohit, Nathan Schneider, Rishav Bhowmick, Kemal Oflazer, and Noah A Smith. 2012. [Recall-oriented learning of named entities in arabic wikipedia](https://md.research.elamur.ai/free/en/papers/mohit-2012-recall-oriented-learning-named-entities/). In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 162–173.
33. Hussein Mozannar, Karl El Hajal, Elie Maamary, and Hazem Hajj. 2019. [Neural arabic question answering](https://md.research.elamur.ai/free/en/papers/mozannar-2019-neural-arabic-question-answering/). arXiv preprint arXiv:1906.05394. [↗](https://arxiv.org/abs/1906.05394)
34. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. [Camel tools: An open source python toolkit for arabic natural language processing](https://md.research.elamur.ai/free/en/papers/obeid-2020-camel-tools-open-source-python/). In Proceedings of the twelfth language resources and evaluation conference, pages 7022–7032.
35. Susanna Olivero. 2024. Figurative Language Understanding based on Large Language Models. Ph.D. thesis, Politecnico di Torino. [find](https://scholar.google.com/scholar?q=Susanna+Olivero%2E+2024%2E+Figurative+Language+Understanding+based+on+Large+Language+Models%2E+Ph%2ED%2E+thesis%2C+Politecnico+di+Torino%2E)
36. OpenAI. 2024. Gpt-4o mini: advancing cost-efficient intelligence. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024%2E+Gpt%2D4o+mini%3A+advancing+cost%2Defficient+intelligence%2E)
37. OpenAI. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). arXiv:2410.21276. Preprint, Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, and et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. Preprint, arXiv:2403.05530. [↗](https://arxiv.org/abs/2410.21276)
38. Hassan Sajjad, Ahmed Abdelali, Nadir Durrani, and Fahim Dalvi. 2020. Arabench: Benchmarking dialectal arabic-english machine translation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5094–5107. [find](https://scholar.google.com/scholar?q=Hassan+Sajjad%2C+Ahmed+Abdelali%2C+Nadir+Durrani%2C+and+Fahim+Dalvi%2E+2020%2E+Arabench%3A+Benchmarking+dialectal+arabic%2Denglish+machine+translation%2E+In+Proceedings+of+the+28th+International+Conference+on+Computa)
39. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, et al. 2023. [Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models](https://md.research.elamur.ai/free/en/papers/sengupta-2023-jais-jais-chat-arabic-centric/). arXiv preprint arXiv:2308.16149. [↗](https://arxiv.org/abs/2308.16149)
40. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, Majd Hawasly, Mus’ab Husaini, Soon-Gyo Jung, Ji Kim Lucas, Walid Magdy, Safa Messaoud, Abubakr Mohamed, Tasnim Mohiuddin, Basel Mousi, Hamdy Mubarak, Ahmad Musleh, Zan Naeem, Mourad Ouzzani, Dorde Popovic, Amin Sadeghi, Husrev Taha Sencar, Mohammed Shinoy, Omar Sinan, Yifan Zhang, Ahmed Ali, Yassine El Kheir, Xiaosong Ma, and Chaoyi Ruan. 2025. [Fanar: An arabic-centric multimodal generative ai platform](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/). Preprint, arXiv:2501.13944. [↗](https://arxiv.org/abs/2501.13944)
41. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971. oussama. 2024. Modern Standard Arabic – The Missing Glossary - — blog.jarrousse.org. wikipedia. 2025. ar.wikipedia.org. https: https://blog.jarrousse.org/2024/03/27//ar.wikipedia.org/wiki/%D8%A7%D9%84% modern-standard-arabic-the-missing-glossary/. D9%84%D8%BA%D8%A9_%D8%A7%D9%84%D8%B9%D8% [Accessed 06-05-2025]. B1%D8%A8%D9%8A%D8%A9. [Accessed 06-05-2025]. wikipediaArabic. 2025. Varieties of Arabic - Wikipedia — en.wikipedia.org. https://en.wikipedia.org/ wiki/Varieties_of_Arabic. [Accessed 06-05-2025]. [↗](https://arxiv.org/abs/2302.13971)
42. Taha Zerrouki and Amar Balla. 2017. Tashkeela: Novel corpus of arabic vocalized texts, data for auto-diacritization systems. Data in brief, 11:147. [find](https://scholar.google.com/scholar?q=Taha+Zerrouki+and+Amar+Balla%2E+2017%2E+Tashkeela%3A+Novel+corpus+of+arabic+vocalized+texts%2C+data+for+auto%2Ddiacritization+systems%2E+Data+in+brief%2C+11%3A147%2E)
43. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. [↗](https://arxiv.org/abs/1904.09675)
44. Cheng Zhao, Bin Wang, and Zhen Wang. 2024. Understanding literary texts by llms: A case study of ancient chinese poetry. arXiv preprint arXiv:2409.00060. [↗](https://arxiv.org/abs/2409.00060)

*Updated 1 November 2026*