---
title: "LMSA at AraGenEval Shared Task — paper by Zita et al. (2025)"
description: "The paper describes the winning system for ARATECT, the subtask of the AraGenEval 2025… 0.864 F1. 24 references, cited by 1 paper here."
authors: ["Kaoutar Zita", "Attia Nehar", "Abdelkader Khelil", "Slimane Bellaouar", "Hadda Cherroun"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks"
source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.4/"
doi: "10.18653/v1/2025.arabicnlp-sharedtasks.4"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zita-2025-lmsa-arageneval-shared-task-ensemble/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zita-2025-lmsa-arageneval-shared-task-ensemble/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/12/) / LMSA at AraGenEval Shared Task: Ensemble-Based Detection of AI-Generated Arabic Text Using Multilingual and Arabic-Specific Models

# LMSA at AraGenEval Shared Task: Ensemble-Based Detection of AI-Generated Arabic Text Using Multilingual and Arabic-Specific Models

**Authors:** Kaoutar Zita, [Attia Nehar](https://md.research.elamur.ai/free/en/authors/attia-nehar/), Abdelkader Khelil, [Slimane Bellaouar](https://md.research.elamur.ai/free/en/authors/slimane-bellaouar/) and [Hadda Cherroun](https://md.research.elamur.ai/free/en/authors/hadda-cherroun/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.4/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.4.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.4](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.4) · [OpenAlex](https://openalex.org/W4416037688)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/zita-2025-lmsa-arageneval-shared-task-ensemble/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes the winning system for ARATECT, the subtask of the AraGenEval 2025 shared task that asks whether an Arabic text was written by a person or produced by a language model. Arabic has received far less attention than English in this area. The authors worked through several families of models on the organizers' balanced dataset:

- classic classifiers on stylometric, TF-IDF and AraBERT-embedding features;
- a two-branch deep network called FusionNet;
- fine-tuned pre-trained transformers.

Three fine-tuned models turned out to be the strongest and to have complementary profiles: AraBERT had the best overall balance, XLM-R the best precision, and Fanar the best recall. Fanar is a 9B Arabic-centric LLM adapted with LoRA in 4-bit precision using instruction-style prompts. Combining the three by simple majority vote gave an F1 of 0.864, which placed first among 16 submissions. The vote only slightly beats AraBERT alone, but the authors argue it is steadier because it does not depend on any one model's behaviour. The code is released on GitHub.

## Key points
- **Data.** The ARATECT dataset has 5,798 Arabic texts: 4,798 for training, 500 for dev and 500 for test, each split 50/50 human/machine.
  - Human texts come from news and literary sources.
  - Machine texts were produced by Mistral, GPT-4 and LLaMA.
- **Ensemble members.**
  - FacebookAI/xlm-roberta-base, trained for 3 epochs.
  - aubmindlab/bert-base-arabertv2, trained for 1 epoch.
  - Both use batch size 4 and cross-entropy loss.
  - QCRI/Fanar-1-9B, tuned with LoRA in 4-bit, batch size 2, one epoch.
  - All training ran on a Kaggle GPU.
- **Single-model results.**
  - AraBERT: accuracy 0.864, F1 0.861.
  - XLM-R: precision 0.911.
  - Fanar: recall 0.920.
- **Ensemble result.** Majority voting reached accuracy 0.866, precision 0.877, recall 0.852 and F1 0.864, ranking 1st of 16 teams.
- **Weaker baselines.**
  - LR, MLP and FusionNet stayed at F1 ≤ 0.667. The MLP's 0.988 recall alongside 0.503 precision suggests it labels almost everything as machine-written.
  - Other transformers also lagged: AraElectra, MARBERT, DeBERTa (0.758), Qwen2.5 and CAMeL.

## Interesting ideas
- Choose ensemble members for opposite error profiles (one high-precision, one high-recall, one balanced) rather than just the top scorers, so a majority vote cancels their biases.
- Even when the vote barely beats the best single model, it is worth keeping because it lowers dependence on one model's quirks under distribution shift.
- FusionNet sends handcrafted stylometric/TF-IDF features and contextual embeddings through separate branches before attention-based fusion, so the dense embedding does not drown out the cheap surface cues.
- Reporting precision and recall next to F1 exposes degenerate classifiers, such as an MLP with near-perfect recall but chance precision, which a single score would hide.
- A large generative LLM can serve as one cheap voter by tuning it with LoRA in 4-bit on instruction-formatted examples, alongside small encoder models.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): when picking witness parsers for silver data, prefer witnesses whose disagreement patterns differ, and record each witness's precision/recall per relation so agreement means more than redundancy.
- **I2** (learn): justify multi-witness silver agreement by its stability across genres, measured per domain, not only by average gain over the best parser.
- **I3** (tokens): when the tagger combines explicit morphological features with learned embeddings, give the explicit features their own branch so they stay visible and inspectable in explanations.
- **I4** (learn): add a negative-control check that flags any repair rule or classifier whose gains come from predicting one label almost always.
- **I5** (learn): an LLM teacher tier could be lightly adapted the same way and act as one voting witness, rather than an oracle, when labeling speech acts or frames.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Amar Telidji University of Laghouat](https://md.research.elamur.ai/free/en/universities/amar-telidji-university-of-laghouat/) — University, Laghouat, Algeria · 8 works · 121 citations
- [University of Ghardaia](https://md.research.elamur.ai/free/en/universities/university-of-ghardaia/) — University, Ghardaïa, Algeria · 4 works · 19 citations
- [Ziane Achour University of Djelfa](https://md.research.elamur.ai/free/en/universities/ziane-achour-university-of-djelfa/) — University, Djelfa, Algeria · 4 works · 19 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works

## Cite

Cite as: Kaoutar Zita, Attia Nehar, Abdelkader Khelil, Slimane Bellaouar and Hadda Cherroun. 2025. LMSA at AraGenEval Shared Task: Ensemble-Based Detection of AI-Generated Arabic Text Using Multilingual and Arabic-Specific Models. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zita2025lmsa,
  title = {{LMSA at AraGenEval Shared Task: Ensemble-Based Detection of AI-Generated Arabic Text Using Multilingual and Arabic-Specific Models}},
  author = {Kaoutar Zita, Attia Nehar, Abdelkader Khelil, Slimane Bellaouar and Hadda Cherroun},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.4},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.4}
}
```

- [← previous Large Means Left: Political Bias in Large Language Models Increases with Their Number of Parameters](https://md.research.elamur.ai/free/en/papers/exler-2025-large-means-left-political-bias/)
- [next → LuxInstruct: A Cross-Lingual Instruction Tuning Dataset For Luxembourgish](https://md.research.elamur.ai/free/en/papers/philippy-2025-luxinstruct-cross-lingual-instruction-tuning/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/12/)

## References

24 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shammur Absar Chowdhury, Hind Almerekhi, Mucahid Kutlu, Kaan Efe Keleş, Fatema Ahmad, Tasnim Mohiuddin, George Mikros, and Firoj Alam. 2025. [GenAI content detection task 2: AI vs. human – academic essay authenticity challenge](https://md.research.elamur.ai/free/en/papers/chowdhury-2025-genai-content-detection-task-2/). In Proceedings of the 1stWorkshop on GenAI Content Detection (GenAIDetect), pages 323–333, Abu Dhabi, UAE. International Conference on Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2412.18274)
2. Shadi Abudalfa, Saad Ezzini, Ahmed Abdelali, Hamza Alami, Abdessamad Benlahbib, Salmane Chafik, Mo El-Haj, Abdelkader El Mahdaouy, Mustafa Jarrar, Salima Lamsiyah, and Hamzah Luqman. 2025. [The AraGenEval Shared Task on Arabic Authorship Style Transfer and AI-Generated Text Detection](https://md.research.elamur.ai/free/en/papers/abudalfa-2025-arageneval-shared-task-arabic-authorship/). In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Association for Computational Linguistics.
3. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.747)
4. Noura Saad Alghamdi and Jalal Suliman Alowibdi. 2024. Distinguishing Arabic GenAI-generated Tweets and Human Tweets utilizing Machine Learning. Engineering, Technology & Applied Science Research, 14(5):16720–16726. [↗](https://doi.org/10.48084/etasr.8249)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
6. Hamed Alshammari and Ahmed EI-Sayed. 2023. AIRABIC: Arabic Dataset for Performance Evaluation of AI Detectors. In 2023 International Conference on Machine Learning and Applications (ICMLA), pages 864–870. [find](https://scholar.google.com/scholar?q=Hamed+Alshammari+and+Ahmed+EI%2DSayed%2E+2023%2E+AIRABIC%3A+Arabic+Dataset+for+Performance+Evaluation+of+AI+Detectors%2E+In+2023+International+Conference+on+Machine+Learning+and+Applications+%28ICMLA%29%2C+pages+864%E2%80%93)
7. Xibin Dong, Zhiwen Yu, Wenming Cao, Yifan Shi, and Qianli Ma. 2020. A survey on ensemble learning. Front. Comput. Sci., 14(2):241–258. [↗](https://doi.org/10.1007/s11704-019-8208-z)
8. Hamed Alshammari, Ahmed El-Sayed, and Khaled Elleithy. 2024. AI-Generated Text Detector for Arabic Language Using Encoder-Based Transformer Architecture. Big Data and Cognitive Computing, 8(3). [↗](https://doi.org/10.3390/bdcc8030032)
9. Tiziano Fagni, Fabrizio Falchi, Margherita Gambini, Antonio Martella, and Maurizio Tesconi. 2021. Tweepfake: About detecting deepfake tweets. PLOS ONE, 16(5):1–16. [↗](https://doi.org/10.1371/journal.pone.0251415)
10. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104)
11. Kathleen Fraser, Hillary Dawkins, and Svetlana Kiritchenko. 2025. Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods. Journal of Artificial Intelligence Research, 82:2233– 2278. [↗](https://doi.org/10.1613/jair.1.16665)
12. Imane Guellil, Houda Saâdane, Faical Azouaou, Billel Gueni, and Damien Nouvel. 2021. [Arabic natural language processing: An overview](https://md.research.elamur.ai/free/en/papers/guellil-2019-arabic-natural-language-processing-overview/). Journal of King Saud University - Computer and Information Sciences, 33(5):497–507.
13. Wissam Antoun, Virginie Mouilleron, Benoît Sagot, and Djamé Seddah. 2023. Towards a robust detection of language model-generated text: Is ChatGPT that easy to detect? In Actes de CORIA-TALN 2023. Actes de la 30e Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 1 : travaux de recherche originaux – articles longs, pages 14–27, Paris, France. ATALA. [↗](https://doi.org/10.48550/arxiv.2306.05871)
14. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://md.research.elamur.ai/free/en/papers/guo-2023-close-chatgpt-human-experts-comparison/). ArXiv, abs/2301.07597. [↗](https://arxiv.org/abs/2301.07597)
15. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Md Ishraquzzaman, Ashraful Islam, Shahreen Rahman, and Riasat Khan. 2025. Ensemble Transformer-Based Detection of Fake and AI-Generated News. [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretchen)
16. Applied Computational Intelligence and Soft Computing, 2025. [find](https://scholar.google.com/scholar?q=Applied+Computational+Intelligence+and+Soft+Computing%2C+2025%2E)
17. Iyad Katib, Fatmah Y. Assiri, Hesham A. Abdushkour, Diaa Hamed, and Mahmoud Ragab. 2023. Differentiating Chat Generative Pretrained Transformer from Humans: Detecting ChatGPT-Generated Text and Human Text Using Machine Learning. Mathematics, 11(15). [↗](https://doi.org/10.3390/math11153400)
18. Satnam Kaur, Lalit K. Awasthi, A.L. Sangal, and Gaurav Dhiman. 2020. Tunicate Swarm Algorithm: A new bio-inspired based metaheuristic paradigm for global optimization. Engineering Applications of Artificial Intelligence, 90:103541. [↗](https://doi.org/10.1016/j.engappai.2020.103541)
19. Xin Liu, Yang Li, and Kan Li. 2025. Enhancing the Robustness of AI-Generated Text Detectors: A Survey. Mathematics, 13(13). [↗](https://doi.org/10.3390/math13132145)
20. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and Barret Zoph. 2023. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). [↗](https://doi.org/10.4230/lipics.cosit.2024.11)
21. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, and 23 others. 2025. [Fanar: An Arabic-Centric Multimodal Generative AI Platform](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/). Preprint, arXiv:2501.13944. [↗](https://arxiv.org/abs/2501.13944)
22. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. [LLaMA: Open and Efficient Foundation Language Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). [↗](https://doi.org/10.48550/arxiv.2302.13971)
23. Ahlam Wahdan, Sendeyah Hantoobi, Said Salloum, and Khaled Shaalan. 2020. A systematic review of text classification research based on deep learning models in Arabic language. International Journal of Electrical and Computer Engineering (IJECE), pages 6629–6643. [↗](https://doi.org/10.11591/ijece.v10i6.pp6629-6643)
24. Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. 2025. [A survey on LLM-generated text detection: Necessity, methods, and future directions](https://md.research.elamur.ai/free/en/papers/wu-2025-survey-llm-generated-text-detection/). Computational Linguistics, 51(1):275–338. [↗](https://doi.org/10.1162/coli_a_00549)

*Updated 1 November 2026*