---
title: "AbjadGenEval — paper by Ezzini et al. (2026)"
description: "The task builds on the earlier AraGenEval/ARATECT benchmark and widens it from… 0.93 F1 on Arabic. 22 references, cited by 9 papers here."
authors: ["Saad Ezzini", "Irfan Ahmad", "Salmane Chafik", "Shadi Abudalfa", "Mo El-Haj", "Ahmed Abdelali", "Mustafa Jarrar", "Nadir Durrani", "Hassan Sajjad", "Farah Adeeba"]
year: 2026
venue: "Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script"
source: "https://aclanthology.org/2026.abjadnlp-1.68/"
doi: "10.18653/v1/2026.abjadnlp-1.68"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ezzini-2026-abjadgeneval-abjad-ai-generated-text/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ezzini-2026-abjadgeneval-abjad-ai-generated-text/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/) / AbjadGenEval: Abjad AI Generated Text Detection Shared Task for Languages Using Arabic Script at AbjadNLP 2026

# AbjadGenEval: Abjad AI Generated Text Detection Shared Task for Languages Using Arabic Script at AbjadNLP 2026

**Authors:** [Saad Ezzini](https://md.research.elamur.ai/free/en/authors/saad-ezzini/), [Irfan Ahmad](https://md.research.elamur.ai/free/en/authors/irfan-ahmad/), [Salmane Chafik](https://md.research.elamur.ai/free/en/authors/salmane-chafik/), [Shadi Abudalfa](https://md.research.elamur.ai/free/en/authors/shadi-abudalfa/), [Mo El-Haj](https://md.research.elamur.ai/free/en/authors/mo-el-haj/), [Ahmed Abdelali](https://md.research.elamur.ai/free/en/authors/ahmed-abdelali/), [Mustafa Jarrar](https://md.research.elamur.ai/free/en/authors/mustafa-jarrar/), [Nadir Durrani](https://md.research.elamur.ai/free/en/authors/nadir-durrani/), [Hassan Sajjad](https://md.research.elamur.ai/free/en/authors/hassan-sajjad/) and [Farah Adeeba](https://md.research.elamur.ai/free/en/authors/farah-adeeba/) · **Year:** 2026 · **Venue:** [Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script](https://md.research.elamur.ai/free/en/venues/abjadnlp/) ([Workshop on NLP for Languages Using Arabic Script 2026](https://md.research.elamur.ai/free/en/venues/abjadnlp/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.abjadnlp-1.68/) · [PDF](https://aclanthology.org/2026.abjadnlp-1.68.pdf) · [DOI 10.18653/v1/2026.abjadnlp-1.68](https://doi.org/10.18653/v1/2026.abjadnlp-1.68) · [OpenAlex](https://openalex.org/W7140104575)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/ezzini-2026-abjadgeneval-abjad-ai-generated-text/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
AbjadGenEval is a shared task held at the AbjadNLP workshop (EACL 2026). It tests whether systems can tell human-written news from LLM-generated news in languages written in Arabic script. The organizers argue that existing detectors were built mostly for English and break down on Arabic-script text, where rich morphology, orthographic ambiguity, diacritics and scarce labelled data cause problems. The task builds on the earlier AraGenEval/ARATECT benchmark and widens it from Arabic alone to Arabic plus Urdu. It has two separate binary-classification subtasks, scored on Codabench. Each subtask uses balanced data: articles from verified news outlets alongside articles generated by models such as GPT and Gemini under varied prompts. Twenty teams took part. The best systems reached about 0.93 F1 on Arabic and 0.89 F1 on Urdu, and most strong entries were fine-tuned multilingual transformers. The overview reports two notable findings. Out-of-domain augmentation often hurt results, while a cheap stylometric SVM held its own on Urdu. Together these point to data domain and robustness to distribution shift, more than model size, as the open problems.

## Key points
- The task has two subtasks, Arabic and Urdu news, each framed as human vs. AI binary classification and evaluated with F1, accuracy, precision, recall and balanced accuracy.
- The Arabic leaderboard was led by HCMUS_TheFangs, with F1 0.9271, using DeBERTa-v3-base. Its representation joined weighted layer pooling over layers 4–12 with attention pooling over tokens.
- The Urdu leaderboard was led by alizain157 (LoRAD), with F1 0.8878, using plain XLM-RoBERTa. The top Urdu scores were tightly packed, with six teams between about 0.84 and 0.89.
- Adding external data such as Arabic Generated Abstracts led models to learn academic-style artifacts rather than features of news, which lowered performance.
- REGLAT reached a recall of 0.98 but a precision of only 0.62. Listed limitations include:
  - coverage of news only,
  - a fixed set of generator models,
  - generalization gaps that appeared on the blind test set.
- Planned extensions include dialectal Arabic, more genres, explanations of detector decisions, and Farsi, Pashto and Sindhi.

## Interesting ideas
- Team mohannad_hendi split long documents into sliding-window segments, classified each segment, averaged the segment probabilities, and tuned the decision threshold on validation data for F1. This handles length limits without truncating content.
- Training material from a different genre (academic abstracts) taught detectors that genre's surface habits instead of the target task. This is a clear case of unmatched extra data doing harm.
- The winning Arabic system concatenated a weighted mix of middle-to-top layer [CLS] vectors with an attention-pooled summary over tokens. The aim was to capture signals at both depth and position levels.
- A linear SVM over character 2–4-grams plus a few readable stylometric counts nearly matched the transformers on Urdu at a fraction of the cost. The counts were repeated-word ratio, punctuation ratio and formal/function markers.
- The organizers report precision and recall separately and stress tunable thresholds, because the cost of an error depends on the application (missing misinformation vs. flagging legitimate content).

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): Long tales could be judged chunk by chunk, with the per-chunk evidence combined into a document-level answer under a calibrated cutoff, and each chunk's score kept visible to support the explanation.
- **I2** (learn): Silver data agreed by witness parsers should be filtered or weighted by genre match to the evaluation texts, and an out-of-genre slice can act as a negative control to catch artifact learning.
- **I3** (tree): gparse could test learned layer mixing combined with token attention for arc and label scoring, since different layers may hold morphological versus syntactic cues.
- **I4** (back): Such a cheap, explainable detector could check whether text regenerated by the text → graph → text compressor reads as machine-like, and point to the specific markers responsible.
- **I5** (learn): Evaluation reports could always separate precision and recall per decision type, such as frame or repair-rule firing, and set thresholds per use rather than relying on a single score.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hamad bin Khalifa University](https://md.research.elamur.ai/free/en/universities/hamad-bin-khalifa-university/) — University, Doha, Qatar · 141 works · 14,175 citations
- [King Fahd University of Petroleum and Minerals](https://md.research.elamur.ai/free/en/universities/king-fahd-university-of-petroleum-and-minerals/) — University, Dhahran, Saudi Arabia · 52 works · 4,666 citations
- [Birzeit University](https://md.research.elamur.ai/free/en/universities/birzeit-university/) — University, Birzeit, Palestine · 36 works · 798 citations
- [Université Mohammed VI Polytechnique](https://md.research.elamur.ai/free/en/universities/universite-mohammed-vi-polytechnique/) — University, Marrakesh, Morocco · 18 works · 171 citations
- [Dalhousie University](https://md.research.elamur.ai/free/en/universities/dalhousie-university/) — University, Halifax, Canada · 9 works · 480 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Saad Ezzini, Irfan Ahmad, Salmane Chafik, Shadi Abudalfa, Mo El-Haj, Ahmed Abdelali, Mustafa Jarrar, Nadir Durrani, Hassan Sajjad and Farah Adeeba. 2026. AbjadGenEval: Abjad AI Generated Text Detection Shared Task for Languages Using Arabic Script at AbjadNLP 2026. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ezzini2026abjadgeneval,
  title = {{AbjadGenEval: Abjad AI Generated Text Detection Shared Task for Languages Using Arabic Script at AbjadNLP 2026}},
  author = {Saad Ezzini, Irfan Ahmad, Salmane Chafik, Shadi Abudalfa, Mo El-Haj, Ahmed Abdelali, Mustafa Jarrar, Nadir Durrani, Hassan Sajjad and Farah Adeeba},
  booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script},
  year = {2026},
  doi = {10.18653/v1/2026.abjadnlp-1.68},
  url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.68}
}
```

- [← previous A Survey of Inductive Reasoning for Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2026-survey-inductive-reasoning-large-language/)
- [next → An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliability](https://md.research.elamur.ai/free/en/papers/yamauchi-2026-empirical-study-llm-as-judge/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/)

## References

22 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. 2. Adversarial Evaluation: Testing against "jailbroken" or adversarially prompted LLMs explicitly trying to evade detection. — [GenAI Content Detection Task 2: AI vs. Human – Academic Essay Authenticity Challenge](https://md.research.elamur.ai/free/en/papers/chowdhury-2025-genai-content-detection-task-2/) [↗](https://arxiv.org/abs/2412.18274)
2. Liam Dugan, Andrew Zhu, Firoj Alam, Preslav Nakov, Marianna Apidianaki, and Chris Callison-Burch. 2025. Genai content detection task 3: Cross-domain machine generated text detection challenge. In Proceedings of the 1stWorkshop on GenAI Content Detection (GenAIDetect), pages 377–388. [↗](https://doi.org/10.48550/arxiv.2501.08913)
3. Mahmoud El-Haj. 2020. [Habibi-a multi dialect multi national arabic song lyrics corpus](https://md.research.elamur.ai/free/en/papers/haj-2020-habibi-multi-dialect-multi-national/). In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1318–1326.
4. Mahmoud El-Haj, Paul Rayson, and Mariam Aboelezz. 2018. [Arabic dialect identification in the context of bivalency and code-switching](https://md.research.elamur.ai/free/en/papers/haj-2018-arabic-dialect-identification-context-bivalency/). In Proceedings of the 11th International Conference on Language Resources and Evaluation, Miyazaki, Japan., pages 3622–3627. European Language Resources Association.
5. Mo El-Haj. 2025a. [Arabjobs: A multinational corpus of arabic job ads](https://md.research.elamur.ai/free/en/papers/haj-2025-arabjobs-multinational-corpus-arabic-job/). In Proceedings of The Third Arabic Natural Language Processing Conference, pages 16– 25. [↗](https://doi.org/10.48550/arxiv.2509.22589)
6. Mo El-Haj. 2025b. Proceedings of the 1st workshop on nlp for languages using arabic script (abjadnlp 2025). In Proceedings of the 1st Workshop on NLP for Languages Using Arabic Script, held at COLING 2025, Abu Dhabi, United Arab Emirates. [find](https://scholar.google.com/scholar?q=Mo+El%2DHaj%2E+2025b%2E+Proceedings+of+the+1st+workshop+on+nlp+for+languages+using+arabic+script+%28abjadnlp+2025%29%2E+In+Proceedings+of+the+1st+Workshop+on+NLP+for+Languages+Using+Arabic+Script%2C+held+at+COLING+)
7. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. [Detectgpt: Zero-shot machine-generated text detection using probability curvature](https://md.research.elamur.ai/free/en/papers/mitchell-2023-detectgpt-zero-shot-machine-generated/). In International conference on machine learning, pages 24950–24962. PMLR.
8. Mo El-Haj. 2026. Proceedings of the 2nd workshop on nlp for languages using arabic script (abjadnlp 2026). In Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script, held at EACL 2026, Rabat, Morocco. [find](https://scholar.google.com/scholar?q=Mo+El%2DHaj%2E+2026%2E+Proceedings+of+the+2nd+workshop+on+nlp+for+languages+using+arabic+script+%28abjadnlp+2026%29%2E+In+Proceedings+of+the+2nd+Workshop+on+NLP+for+Languages+Using+Arabic+Script%2C+held+at+EACL+202)
9. Mo El-Haj and Saad Ezzini. 2024. The multilingual corpus of world’s constitutions (mcwc). In Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation@ LREC-COLING 2024, pages 57–66. [find](https://scholar.google.com/scholar?q=Mo+El%2DHaj+and+Saad+Ezzini%2E+2024%2E+The+multilingual+corpus+of+world%E2%80%99s+constitutions+%28mcwc%29%2E+In+Proceedings+of+the+6th+Workshop+on+Open%2DSource+Arabic+Corpora+and+Processing+Tools+%28OSACT%29+with+Shared+Task)
10. Mo El-Haj and Paul Rayson. 2025. [Arafinnews: Arabic financial summarisation with domain-adapted llms](https://md.research.elamur.ai/free/en/papers/el-haj-2025-arafinnews-arabic-financial-summarisation-domain/). arXiv preprint arXiv:2511.01265. [↗](https://arxiv.org/abs/2511.01265)
11. Wooyoung Go, Hyoungshick Kim, Alice Oh, and Yongdae Kim. 2025. XDAC: XAI-driven detection and attribution of LLM-generated news comments in Korean. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 22728–22750, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wooyoung+Go%2C+Hyoungshick+Kim%2C+Alice+Oh%2C+and+Yongdae+Kim%2E+2025%2E+XDAC%3A+XAI%2Ddriven+detection+and+attribution+of+LLM%2Dgenerated+news+comments+in+Korean%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Ass)
12. Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2020. [Automatic detection of generated text is easiest when humans are fooled](https://md.research.elamur.ai/free/en/papers/ippolito-2019-automatic-detection-generated-text-easiest/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1808–1822, Online. Association for Computational Linguistics.
13. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. [A watermark for large language models](https://md.research.elamur.ai/free/en/papers/kirchenbauer-2023-watermark-large-language-models/). In International Conference on Machine Learning, pages 17061–17084. PMLR. [↗](https://doi.org/10.48550/arxiv.2301.10226)
14. Salima Lamsiyah, Saad Ezzini, Abdelkader Elmah-daouy, Hamza Alami, Abdessamad Benlahbib, Samir El Amrany, Salmane Chafik, and Hicham Hammouchi. 2025. Shared task on multi-domain detection of ai-generated text (m-daigt). In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing (RANLP). [find](https://scholar.google.com/scholar?q=Salima+Lamsiyah%2C+Saad+Ezzini%2C+Abdelkader+Elmah%2Ddaouy%2C+Hamza+Alami%2C+Abdessamad+Benlahbib%2C+Samir+El+Amrany%2C+Salmane+Chafik%2C+and+Hicham+Hammouchi%2E+2025%2E+Shared+task+on+multi%2Ddomain+detection+of+ai%2Dgenera)
15. Yuanfan Li, Zhaohan Zhang, Chengzhengxu Li, Chao Shen, and Xiaoming Liu. 2025. Iron sharpens iron: Defending against attacks in machine-generated text detection with adversarial training. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3091–3113, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yuanfan+Li%2C+Zhaohan+Zhang%2C+Chengzhengxu+Li%2C+Chao+Shen%2C+and+Xiaoming+Liu%2E+2025%2E+Iron+sharpens+iron%3A+Defending+against+attacks+in+machine%2Dgenerated+text+detection+with+adversarial+training%2E+In+Proceedin)
16. Dominik Macko, Jakub Kopál, Robert Moro, and Ivan Srba. 2025. [MultiSocial: Multilingual benchmark of machine-generated text detection of social-media texts](https://md.research.elamur.ai/free/en/papers/macko-2025-multisocial-multilingual-benchmark-machine-generated/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 727–752, Vienna, Austria. Association for Computational Linguistics.
17. Lujia Shen, Xuhong Zhang, Shouling Ji, Yuwen Pu, Chunpeng Ge, Xing Yang, and Yanghe Feng. 2023. Textdefense: Adversarial text detection based on word importance entropy. arXiv preprint arXiv:2302.05892. [↗](https://arxiv.org/abs/2302.05892)
18. Zhixiong Su, Yichen Wang, Herun Wan, Zhaohan Zhang, and Minnan Luo. 2025. HACo-det: A study towards fine-grained machine-generated text detection under human-AI coauthoring. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 22015–22036, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhixiong+Su%2C+Yichen+Wang%2C+Herun+Wan%2C+Zhaohan+Zhang%2C+and+Minnan+Luo%2E+2025%2E+HACo%2Ddet%3A+A+study+towards+fine%2Dgrained+machine%2Dgenerated+text+detection+under+human%2DAI+coauthoring%2E+In+Proceedings+of+the+63rd)
19. Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, and Thomas Arnold. 2024a. SemEval-2024 task 8: Multidomain, multimodel and multilingual machine-generated text detection. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 2057–2079, Mexico City, Mexico. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2404.14183)
20. Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. 2024b. M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generated text detection. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1369–1407, St. Julian’s, Malta. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yuxia+Wang%2C+Jonibek+Mansurov%2C+Petar+Ivanov%2C+Jinyan+Su%2C+Artem+Shelmanov%2C+Akim+Tsvigun%2C+Chenxi+Whitehouse%2C+Osama+Mohammed+Afzal%2C+Tarek+Mahmoud%2C+Toru+Sasaki%2C+Thomas+Arnold%2C+Alham+Fikri+Aji%2C+Nizar+Habash%2C)
21. Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. 2025. [A survey on llm-generated text detection: Necessity, methods, and future directions](https://md.research.elamur.ai/free/en/papers/wu-2025-survey-llm-generated-text-detection/). Computational Linguistics, 51(1):275–338. [↗](https://doi.org/10.1162/coli_a_00549)
22. Xuandong Zhao, Yu-Xiang Wang, and Lei Li. 2023. Protecting language generation models via invisible watermarking. In International Conference on Machine Learning, pages 42187–42199. PMLR. [↗](https://doi.org/10.48550/arxiv.2302.03162)

*Updated 1 November 2026*