--- title: "CIOL at AraGenEval shared task: Authorship Identification and AI" description: "The authors, from CIOL, entered two subtasks of the AraGenEval shared task. 0.98 F1. 2 references." authors: ["Sadia Tasnim Meem", "Azmine Toushik Wasi"] year: 2025 venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks" source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.13/" doi: "10.18653/v1/2025.arabicnlp-sharedtasks.13" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/meem-2025-ciol-arageneval-shared-task-authorship/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/meem-2025-ciol-arageneval-shared-task-authorship/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/17/) / CIOL at AraGenEval shared task: Authorship Identification and AI Generated Text Detection in Arabic using Pretrained Models # CIOL at AraGenEval shared task: Authorship Identification and AI Generated Text Detection in Arabic using Pretrained Models **Authors:** Sadia Tasnim Meem and [Azmine Toushik Wasi](https://md.research.elamur.ai/free/en/authors/azmine-toushik-wasi/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.13/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.13.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.13](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.13) · [OpenAlex](https://openalex.org/W4416037826) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary The authors, from CIOL, entered two subtasks of the AraGenEval shared task. Task 2 asks which of 21 Arabic authors wrote a paragraph. Task 3 asks whether an Arabic news text was written by a human or by an LLM. Both are treated as plain sequence classification with fine-tuned Arabic BERT encoders. For authorship, CAMeL-Lab/bert-base-arabic-camelbert-mix gets a 21-way softmax head. Checkpoints are chosen by macro-F1 on validation. The data comes from ten books per author, split into paragraphs and paired with GPT-4o mini rewrites in a uniform formal register. For detection, two AraBERT variants were compared, and the longer-context one with a custom head won. Authorship transferred well from validation to test, with test macro-F1 around 0.83. Detection collapsed, falling from about 0.98 F1 on validation to 0.66 on the official test. The paper's main lesson is that a near-perfect score on in-distribution validation can be misleading. The authors blame domain shift and unseen generation styles, which makes cross-source evaluation essential for AI-text detectors. ## Key points - **Task 2 setup:** camelbert-mix, 4 epochs, AdamW (lr 2e-5), batch 16, weight decay 0.01. The paper is inconsistent about the input cap, giving 512 tokens in the system section and 256 in the setup section. - **Task 2 results:** - Validation: accuracy 0.872, macro-F1 0.809. - Test: F1 0.827, accuracy 0.864. - **Task 3 data:** 4,800 balanced training samples and 2,000 test samples. AI texts came from GPT-3.5, GPT-4 and Claude under varied prompts and generation parameters. - **Task 3 configurations:** - Configuration 1: aubmindlab/bert-base-arabertv2, 512 tokens, dropout 0.3, gradient clipping, early stopping. - Configuration 2: bert-base-arabert with the built-in head, 256 tokens. It performed worse. - **Task 3 results:** - Validation: F1 0.984, precision 1.0. - Test: F1 0.657, recall 0.568, specificity 0.840. The detector is cautious and misses many machine-written texts. ## Interesting ideas - When validation scores look almost perfect, treat them as a warning, because a held-out set from the same sources can hide a large generalization gap. - Report specificity and balanced accuracy next to F1, since this combination exposes a classifier that rarely predicts the positive class. - Select checkpoints by macro-F1 instead of accuracy, so every class counts equally even when class sizes differ only mildly. - Compare a long input window against a short one as a controlled variant, and measure what context length is actually worth. - Build parallel pairs in which an LLM rewrites original paragraphs into one neutral formal register, separating what is said from how it is said. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Score each model on silver data drawn from text sources it never saw, next to the usual in-source split, so source overfitting shows up early. - **I2** (learn): Per-label precision versus recall for rare dependency relations or speech acts would reveal parsers or taggers that systematically under-predict them. - **I3** (tree): Selecting gparse checkpoints by macro-averaged per-deprel LAS would stop frequent relations from hiding regressions on rare ones. - **I4** (tree): A similar ablation of the gparse context window, sentence-only versus neighbouring sentences, would show whether extra context earns its memory cost. - **I5** (back): Such pairs could test whether the text → graph → text compressor preserves meaning while normalizing style. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Shahjalal University of Science and Technology](https://md.research.elamur.ai/free/en/universities/shahjalal-university-of-science-and-technology/) — University, Sylhet, Bangladesh · 4 works · 1 citation 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works ## Cite Cite as: Sadia Tasnim Meem and Azmine Toushik Wasi. 2025. CIOL at AraGenEval shared task: Authorship Identification and AI Generated Text Detection in Arabic using Pretrained Models. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{meem2025ciol, title = {{CIOL at AraGenEval shared task: Authorship Identification and AI Generated Text Detection in Arabic using Pretrained Models}}, author = {Sadia Tasnim Meem and Azmine Toushik Wasi}, booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks}, year = {2025}, doi = {10.18653/v1/2025.arabicnlp-sharedtasks.13}, url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.13} } ``` - [← previous Chinese Parallelism Generation Based on Part-of-Speech Alignment and Dependency Relation](https://md.research.elamur.ai/free/en/papers/zhong-2025-chinese-parallelism-generation/) - [next → Classifying German Language Proficiency Levels Using Large Language Models](https://md.research.elamur.ai/free/en/papers/ahlers-2025-classifying-german-language-proficiency-levels/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/17/) ## References 2 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Ahmed Abbasi, Asad R. Javed, Fahad Iqbal, and 1 others. 2022. Authorship identification using ensemble learning. Scientific Reports, 12:9537. [find](https://scholar.google.com/scholar?q=Ahmed+Abbasi%2C+Asad+R%2E+Javed%2C+Fahad+Iqbal%2C+and+1+others%2E+2022%2E+Authorship+identification+using+ensemble+learning%2E+Scientific+Reports%2C+12%3A9537%2E) 2. Shadi Abudalfa, Saad Ezzini, Ahmed Abdelali, Hamza Alami, Abdessamad Benlahbib, Salmane Chafik, Mo El-Haj, Abdelkader El Mahdaouy, Mustafa Jarrar, Salima Lamsiyah, and Hamzah Luqman. 2025. [The arageneval shared task on arabic authorship style transfer and ai-generated text detection](https://md.research.elamur.ai/free/en/papers/abudalfa-2025-arageneval-shared-task-arabic-authorship/). In Proceedings of the Third Arabic Natural Language Process- *Updated 2 November 2026*