---
title: "PTUK-HULAT at AraGenEval Shared Task: Fine-tuning"
description: "The paper describes the PTUK-HULAT team's system for ARATECT 4.3 Subtask 3 of… 0.9272 accuracy. 26 references, cited by 1 paper here."
authors: ["Tasneem Duridi", "Areej Jaber", "Paloma Martínez"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks"
source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.7/"
doi: "10.18653/v1/2025.arabicnlp-sharedtasks.7"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/duridi-2025-ptuk-hulat-arageneval-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/duridi-2025-ptuk-hulat-arageneval-shared-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) / PTUK-HULAT at AraGenEval Shared Task: Fine-tuning XLM-RoBERTa for AI-Generated Arabic News Detection

# PTUK-HULAT at AraGenEval Shared Task: Fine-tuning XLM-RoBERTa for AI-Generated Arabic News Detection

**Authors:** Tasneem Duridi, [Areej Jaber](https://md.research.elamur.ai/free/en/authors/areej-jaber/) and [Paloma Martínez](https://md.research.elamur.ai/free/en/authors/paloma-martinez/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.7/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.7.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.7](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.7) · [OpenAlex](https://openalex.org/W4416037833)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/duridi-2025-ptuk-hulat-arageneval-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes the PTUK-HULAT team's system for ARATECT 4.3 Subtask 3 of the AraGenEval shared task at ArabicNLP 2025. The task is binary: decide whether an Arabic news text was written by a person or produced by a machine. The organizers provided 4,798 labelled training articles and unlabelled development and test sets of 500 items each. The authors fine-tuned several transformers: mBERT, DistilBERT, QARiBERT, AraELECTRA, XLM-RoBERTa base and large, and an XLM-RoBERTa-base variant with a BiLSTM head.

On their internal validation, XLM-RoBERTa-large scored best, with a macro F1 of about 0.93. It was too costly to run on the test phase, so the submitted system was XLM-RoBERTa-base. That model reached F1 0.78 on the hidden test set and placed fourth.

The authors attribute the drop from validation to test to domain shift. The splits differ in text length and in how much English they contain. The work shows that multilingual encoders are competitive for Arabic AIGT detection, and that such detectors are fragile under distribution change.

## Key points
- **Data:** 4,798 training articles (id, content, label) and 500 development plus 500 test articles (id, title, content).
  - About 1.3% of training rows had empty content and were dropped.
  - Title and content were concatenated, and a stratified 90/10 split was used for validation.
- **Split differences:** average length falls from 485.77 words (train) to 288.74 (dev) and 238.96 (test). The English share is 15.86% in train, 56.60% in dev and 37.60% in test.
- **Validation results:**
  - XLM-RoBERTa-large: F1 0.9272, accuracy 92.4%, recall 0.968.
  - XLM-RoBERTa-base: F1 0.8532.
  - AraELECTRA: recall 0.912 but precision only 0.5078, so it was not carried forward.
- **Test results:**
  - XLM-RoBERTa-base: F1 0.7823, fourth on the leaderboard.
  - BiLSTM variant: precision 0.8029, recall 0.668.
- **Training setup:** AdamW, early stopping with patience 3, class-weighted loss with weighted sampling, run on Colab T4 GPUs.
  - The BiLSTM (hidden size 256) reads the sum of the last four encoder layers.
  - Only those four layers were unfrozen during fine-tuning.

## Interesting ideas
- Heavy Arabic cleaning hurt accuracy, because stylistic cues of machine text live in raw surface details; this covered removing diacritics, unifying letter variants and stripping punctuation.
- Profiling every split by average length and foreign-language share was enough to explain much of the validation-to-test drop without new experiments.
- Two models with similar F1 can have opposite error profiles: the BiLSTM head made the classifier cautious, with high precision and low recall. The authors argue the choice should follow which error the application tolerates.
- The BiLSTM head pools the last four encoder layers instead of only the top one, and unfreezes only those layers to protect pretrained knowledge on a small dataset.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Keep the untouched surface form (punctuation, casing, odd spellings) next to each normalized token and lemma, so later stages can still use style evidence.
- **I2** (learn): Print cheap per-split statistics (sentence length, non-English token rate, genre) alongside every evaluation, so a score drop can be traced to data shift rather than parser regressions.
- **I3** (answer): Report precision and recall separately for answer-versus-abstain decisions, and pick configurations by whether a wrong answer or a missed answer is costlier for school tests.
- **I4** (tree): gparse could feed arc and label scorers a sum of upper layers and freeze lower ones when training on small treebanks or converted UD dialects.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Palestine Technical University - Kadoorie](https://md.research.elamur.ai/free/en/universities/palestine-technical-university-kadoorie/) — University, Tulkarm, Palestine · 4 works · 20 citations
- [Universidad Carlos III de Madrid](https://md.research.elamur.ai/free/en/universities/universidad-carlos-iii-de-madrid/) — University, Madrid, Spain · 4 works · 11 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Tasneem Duridi, Areej Jaber and Paloma Martínez. 2025. PTUK-HULAT at AraGenEval Shared Task: Fine-tuning XLM-RoBERTa for AI-Generated Arabic News Detection. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{duridi2025ptuk,
  title = {{PTUK-HULAT at AraGenEval Shared Task: Fine-tuning XLM-RoBERTa for AI-Generated Arabic News Detection}},
  author = {Tasneem Duridi, Areej Jaber and Paloma Martínez},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.7},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.7}
}
```

- [← previous Proper Noun Diacritization for Arabic Wikipedia: A Benchmark Dataset](https://md.research.elamur.ai/free/en/papers/bondok-2025-proper-noun-diacritization-arabic-wikipedia/)
- [next → Qais at BAREC Shared Task 2025: A Fine-Grained Approach for Arabic Readability Classification Using a pre-trained model](https://md.research.elamur.ai/free/en/papers/ahmad-2025-qais-barec-shared-task-2025/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/)

## References

26 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shadi Abudalfa, Saad Ezzini, Ahmed Abdelali, Hamza Alami, Abdessamad Benlahbib, Salmane Chafik, Mo El-Haj, Abdelkader El Mahdaouy, Mustafa Jarrar, Salima Lamsiyah, and Hamzah Luqman. 2025. [The AraGenEval Shared Task on Arabic Authorship Style Transfer and AI-Generated Text Detection](https://md.research.elamur.ai/free/en/papers/abudalfa-2025-arageneval-shared-task-arabic-authorship/). In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Association for Computational Linguistics. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, *Table 2: Development Set Performance Comparison* Model XLM-RoBERTa-large XLM-RoBERTa-base AraELECTRA F1-score 0.9272 0.8532 0.6524 Precision 0.8897 0.8352 0.5078 Recall 0.9680 0.8720 0.9120 Accuracy 0.9240 0.8500 0.5140 *Table 3: Test Set Performance Comparison* Model XLM-RoBERTa-base XLM-RoBERTa + BiLSTM F1-score 0.7823 0.7293
2. Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774. Precision 0.7260 0.8029 Recall 0.8480 0.6680 Accuracy 0.7640 0.7520 [↗](https://arxiv.org/abs/2303.08774)
3. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116. [↗](https://arxiv.org/abs/1911.02116)
4. Noura Saad Alghamdi and Jalal Suliman Alowibdi. 2024. Distinguishing arabic genai-generated tweets and human tweets utilizing machine learning. Engineering, Technology & Applied Science Research, 14(5):16720–16726. [↗](https://doi.org/10.48084/etasr.8249)
5. Debby RE Cotton, Peter A Cotton, and J Reuben Shipway. 2024. Chatting and cheating: Ensuring academic integrity in the era of chatgpt. Innovations in education and teaching international, 61(2):228– 239. [↗](https://doi.org/10.1080/14703297.2023.2190148)
6. Haifa Alharthi. 2025. Investigation into the identification of ai-generated short dialectal arabic texts. IEEE Access. [↗](https://doi.org/10.1109/access.2025.3568696)
7. Lina Duaibes, Areej Jaber, Mustafa Jarrar, Ahmad Qadi, and Mais Qandeel. 2024. [Sina at fignews 2024: Multilingual datasets annotated with bias and propaganda](https://md.research.elamur.ai/free/en/papers/duaibes-2024-sina-fignews-2024-multilingual-datasets/). arXiv preprint arXiv:2407.09327. [↗](https://arxiv.org/abs/2407.09327)
8. Hind Almerekhi and Tamer Elsayed. 2015. Detecting automatically-generated arabic tweets. In AIRS, pages 123–134. Springer. [find](https://scholar.google.com/scholar?q=Hind+Almerekhi+and+Tamer+Elsayed%2E+2015%2E+Detecting+automatically%2Dgenerated+arabic+tweets%2E+In+AIRS%2C+pages+123%E2%80%93134%2E+Springer%2E)
9. Hamed Alshammari and El-Sayed Ahmed. 2023. Airabic: Arabic dataset for performance evaluation of ai detectors. In 2023 International Conference on Machine Learning and Applications (ICMLA), pages 864–870. IEEE. [find](https://scholar.google.com/scholar?q=Hamed+Alshammari+and+El%2DSayed+Ahmed%2E+2023%2E+Airabic%3A+Arabic+dataset+for+performance+evaluation+of+ai+detectors%2E+In+2023+International+Conference+on+Machine+Learning+and+Applications+%28ICMLA%29%2C+pages+864%E2%80%93)
10. Tasneem Duridi, Lour Atwe, Areej Jaber, Eman Daraghmi, and Paloma Martínez. 2025. Detection of propaganda and bias in social media: A case study of the israel-gaza war (2023). In 2025 International Conference on New Trends in Computing Sciences (ICTCS), pages 204–210. IEEE. [find](https://scholar.google.com/scholar?q=Tasneem+Duridi%2C+Lour+Atwe%2C+Areej+Jaber%2C+Eman+Daraghmi%2C+and+Paloma+Mart%C3%ADnez%2E+2025%2E+Detection+of+propaganda+and+bias+in+social+media%3A+A+case+study+of+the+israel%2Dgaza+war+%282023%29%2E+In+2025+International+Co)
11. Hamed Alshammari, Ahmed El-Sayed, and Khaled Elleithy. 2024. Ai-generated text detector for arabic language using encoder-based transformer architecture. Big Data and Cognitive Computing, 8(3):32. [↗](https://doi.org/10.3390/bdcc8030032)
12. Tasneem Duridi, Derar Eleyan, Amna Eleyan, and Tarek Bejaoui. 2024. Arabic fake news detection using machine learning approach. In 2024 International Symposium on Networks, Computers and Communications (ISNCC), pages 1–7. IEEE. [find](https://scholar.google.com/scholar?q=Tasneem+Duridi%2C+Derar+Eleyan%2C+Amna+Eleyan%2C+and+Tarek+Bejaoui%2E+2024%2E+Arabic+fake+news+detection+using+machine+learning+approach%2E+In+2024+International+Symposium+on+Networks%2C+Computers+and+Communication)
13. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020a. [Araelectra: Pre-training text discriminators for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2021-araelectra-pre-training-text-discriminators/). arXiv preprint arXiv:2012.15516. [↗](https://arxiv.org/abs/2012.15516)
14. Catherine A Gao, Frederick M Howard, Nikolay S Markov, Emma C Dyer, Siddhi Ramesh, Yuan Luo, and Alexander T Pearson. 2022. Comparing scientific abstracts generated by chatgpt to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. BioRxiv, pages 2022–12. [find](https://scholar.google.com/scholar?q=Catherine+A+Gao%2C+Frederick+M+Howard%2C+Nikolay+S+Markov%2C+Emma+C+Dyer%2C+Siddhi+Ramesh%2C+Yuan+Luo%2C+and+Alexander+T+Pearson%2E+2022%2E+Comparing+scientific+abstracts+generated+by+chatgpt+to+original+abstracts+us)
15. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020b. [Aragpt2: Pre-trained transformer for arabic language generation](https://md.research.elamur.ai/free/en/papers/antoun-2021-aragpt2-pre-trained-transformer-arabic/). arXiv preprint arXiv:2012.15520. [↗](https://arxiv.org/abs/2012.15520)
16. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, and 1 others. 2023. [Palm: Scaling language modeling with pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/). Journal of Machine Learning Research, 24(240):1–113.
17. Fouzi Harrag, Maria Debbah, Kareem Darwish, and Ahmed Abdelali. 2021. [Bert transformer model for detecting arabic gpt2 auto-generated tweets](https://md.research.elamur.ai/free/en/papers/harrag-2020-bert-transformer-model-detecting-arabic/). arXiv preprint arXiv:2101.09345. [↗](https://arxiv.org/abs/2101.09345)
18. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555. [↗](https://arxiv.org/abs/2003.10555)
19. Muhammad Imran and Norah Almusharraf. 2024. Google gemini as a next generation ai educational tool: a review of emerging educational technology. Smart Learning Environments, 11(1):22. [↗](https://doi.org/10.1186/s40561-024-00310-z)
20. Areej Jaber and Paloma Martínez. 2023. [Ptuk-hulat at araieval shared task fine-tuned distilbert to predict disinformative tweets](https://md.research.elamur.ai/free/en/papers/jaber-2023-ptuk-hulat-araieval-shared-task/). In Proceedings of ArabicNLP 2023, pages 525–529. [↗](https://doi.org/10.48448/bsjs-w641)
21. Mahmoud Jazzar and Tasneem Duridi. 2024. A comprehensive review of machine learning and deep learning techniques for cyberbullying detection. In International Conference on Smart Cyber Physical Systems, pages 1–12. Springer. [find](https://scholar.google.com/scholar?q=Mahmoud+Jazzar+and+Tasneem+Duridi%2E+2024%2E+A+comprehensive+review+of+machine+learning+and+deep+learning+techniques+for+cyberbullying+detection%2E+In+International+Conference+on+Smart+Cyber+Physical+System)
22. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. [↗](https://arxiv.org/abs/1711.05101)
23. Antonino Maniaci, Carlos M Chiesa-Estomba, and Jérôme R Lechien. 2024. Chatgpt-4 consistency in interpreting laryngeal clinical images of common lesions and disorders. Otolaryngology–Head and Neck Surgery, 171(4):1106–1113. [↗](https://doi.org/10.1002/ohn.897)
24. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2021. [Societal biases in language generation: Progress and challenges](https://md.research.elamur.ai/free/en/papers/sheng-2021-societal-biases-language-generation-progress/). arXiv preprint arXiv:2105.04054. [↗](https://arxiv.org/abs/2105.04054)
25. Bernd Carsten Stahl and Damian Eke. 2024. The ethics of chatgpt–exploring the ethical issues of an emerging technology. International Journal of Information Management, 74:102700. [↗](https://doi.org/10.1016/j.ijinfomgt.2023.102700)
26. Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, and 1 others. 2022. Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM conference on fairness, accountability, and transparency, pages 214–229. [find](https://scholar.google.com/scholar?q=Laura+Weidinger%2C+Jonathan+Uesato%2C+Maribeth+Rauh%2C+Conor+Griffin%2C+Po%2DSen+Huang%2C+John+Mellor%2C+Amelia+Glaese%2C+Myra+Cheng%2C+Borja+Balle%2C+Atoosa+Kasirzadeh%2C+and+1+others%2E+2022%2E+Taxonomy+of+risks+posed+by+lan)

*Updated 1 November 2026*