--- title: "MISSION at AraGenEval Shared Task — paper by Thamer (2025)" description: "The paper reports one participant's system for two subtasks of the AraGenEval shared task. 0.84 accuracy. 4 references, cited by 1 paper here." authors: ["Maseer Alharbi, Thamer"] year: 2025 venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks" source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.2/" doi: "10.18653/v1/2025.arabicnlp-sharedtasks.2" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/alharbi-2025-mission-arageneval-shared-task-enhanced/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alharbi-2025-mission-arageneval-shared-task-enhanced/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) / MISSION at AraGenEval Shared Task: Enhanced Arabic Authority Classification # MISSION at AraGenEval Shared Task: Enhanced Arabic Authority Classification **Authors:** Maseer Alharbi, Thamer · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.2/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.2.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.2](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.2) · [OpenAlex](https://openalex.org/W4416037842) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/alharbi-2025-mission-arageneval-shared-task-enhanced/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper reports one participant's system for two subtasks of the AraGenEval shared task. Subtask 2 is Arabic authorship identification, a multi-class problem with a fixed set of candidate authors. Subtask 3 (ARATECT) is binary detection of human versus machine-written Arabic text. For authorship, the author reused a pipeline that won an unrelated Kaggle math-problem classification contest. The backbone was replaced with ALLaM-7B-Instruct-preview, and a new prompt lists the candidate authors with numeric IDs. Training uses LoRA adapters, 4-bit quantization, mixed precision and a linear classification head. Because of compute limits, only one fold of stratified K-fold was trained. For AI-text detection, AraModernBERT was fine-tuned with a fresh classification head. Its training data combined the official set with an external Arabic AI-generated corpus. The systems placed 4th in Subtask 2 (Macro-F1 ≈ 0.84, accuracy ≈ 0.89) and 3rd in Subtask 3 (Macro-F1 0.80, accuracy 0.79). The work shows that recycling an existing pipeline and adding external data can be competitive on modest Kaggle hardware. All code is public. ## Key points - Data sizes: - Authorship (AID): 35,122 train, 4,157 dev and 8,413 test entries. - ARATECT: 17,604 train, 500 dev and 500 test entries, merged with the Al-Shaibani and Ahmed (2025) dataset. - Subtask 2 setup: - Model: ALLaM-7B-Instruct-preview with LoRA, 4-bit quantization, distributed data parallelism and mixed precision. - Loss and validation: weighted cross-entropy for class imbalance, with micro F1 for validation. - Hardware: four NVIDIA L4 GPUs. - Subtask 3 setup: - Model: AraModernBERT-Base-V1.0. - Training: max length 256 tokens, AdamW at learning rate 2e-5, 4 epochs, batch size 32 with dynamic padding. - Hardware: one Tesla P100. - The official metric was Macro-F1, with accuracy as the secondary metric. - Results: - Subtask 2: rank 4, F1 0.83753, accuracy 0.89053. - Subtask 3: rank 3, F1 0.80, accuracy 0.79. ## Interesting ideas - The prompt lists every candidate label with a numeric ID, so the model only has to name a number, which turns open generation into a tightly constrained choice. - A whole training framework from a different winning competition was kept intact, and only the backbone and prompt were swapped, which saved engineering effort under tight deadlines. - External data of the same kind was merged with the official set to make the detector more robust than the in-domain data alone would allow. - Weighted cross-entropy compensates for uneven class frequencies, and Macro-F1 makes rare classes count equally in evaluation. - With limited compute, the author trained one stratified fold instead of all folds and still kept a class-balanced validation split. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): LLM teacher tiers could receive an enumerated inventory (e.g. VerbNet classes or UD relations) and answer with an index, making their silver labels trivially checkable and parseable. - **I2** (learn): when a new classifier is needed (speech acts, irony), first reuse an existing proven training harness and change only inputs and labels, so differences in results are attributable. - **I3** (tree): extra English treebanks, once converted to Mova's UD dialect, could be pooled with the core treebank to harden the perceptron parser and gparse. - **I4** (frames): rare semantic roles and speech-act types could be weighted in training and reported by macro-averaged F1, so common agent/patient roles do not hide failures. - **I5** (learn): quick experiments could use a single stratified fold over sentence types as a cheap but honest check before full cross-validation. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works ## Cite Cite as: Maseer Alharbi, Thamer. 2025. MISSION at AraGenEval Shared Task: Enhanced Arabic Authority Classification. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{thamer2025mission, title = {{MISSION at AraGenEval Shared Task: Enhanced Arabic Authority Classification}}, author = {Maseer Alharbi, Thamer}, booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks}, year = {2025}, doi = {10.18653/v1/2025.arabicnlp-sharedtasks.2}, url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.2} } ``` - [← previous Mispronunciation Detection Without L2 Pronunciation Dataset in Low-Resource Setting: A Case Study in Finland Swedish](https://md.research.elamur.ai/free/en/papers/phan-2025-mispronunciation-detection-l2-pronunciation-dataset/) - [next → Mix, MinHash, and Match: Cross-Source Agreement for Multilingual Pretraining Datasets](https://md.research.elamur.ai/free/en/papers/alrashed-2025-mix-minhash-match-cross-source/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) ## References 4 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Shadi Abudalfa, Saad Ezzini, Ahmed Abdelali, Hamza Alami, Abdessamad Benlahbib, Salmane Chafik, Mo El-Haj, Abdelkader El Mahdaouy, Mustafa Jarrar, Salima Lamsiyah, and Hamzah Luqman. 2025. [The AraGenEval Shared Task on Arabic Authorship Style Transfer and AI-Generated Text Detection](https://md.research.elamur.ai/free/en/papers/abudalfa-2025-arageneval-shared-task-arabic-authorship/). In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Association for Computational Linguistics. 2. Maged S Al-Shaibani and Moataz Ahmed. 2025. [The arabic ai fingerprint: Stylometric analysis and detection of large language models text](https://md.research.elamur.ai/free/en/papers/al-shaibani-2025-arabic-ai-fingerprint-stylometric-analysis/). arXiv preprint arXiv:2505.23276. [↗](https://arxiv.org/abs/2505.23276) 3. M Saiful Bari, Yazeed Alnumay, Norah A. Alzahrani, Nouf M. Alotaibi, Hisham Abdullah Alyahya, Sultan AlRashed, Faisal Abdulrahman Mirza, Shaykhah Z. Alsubaie, Hassan A. Alahmed, Ghadah Alabduljabbar, Raghad Alkhathran, Yousef Almushayqih, Raneem Alnajim, Salman Alsubaihi, Maryam Al Mansour, Saad Amin Hassan, Dr. Majed Alrubaian, Ali Alammari, Zaki Alawami, and 7 others. 2025. AL-Lam: Large language models for arabic and english. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=M+Saiful+Bari%2C+Yazeed+Alnumay%2C+Norah+A%2E+Alzahrani%2C+Nouf+M%2E+Alotaibi%2C+Hisham+Abdullah+Alyahya%2C+Sultan+AlRashed%2C+Faisal+Abdulrahman+Mirza%2C+Shaykhah+Z%2E+Alsubaie%2C+Hassan+A%2E+Alahmed%2C+Ghadah+Alabduljabbar%2C+) 4. NAMAA. 2025. Aramodernbert: Advanced arabic language model through trans-tokenization and modernbert architecture. [https://huggingface.co/](https://huggingface.co/) NAMAA-Space/AraModernBert-Base-V1.0. Accessed: 2025-03-02. [↗](https://huggingface.co/) *Updated 31 October 2026*