--- title: "Jenin at AraGenEval Shared Task — Malhis 2025" description: "This paper reports one team's entry to the AraGenEval shared task, which covers three problems in Modern Standard Arabic." authors: ["Huthayfa Malhis", "Mohammad Tami", "Huthaifa I. Ashqar"] year: 2025 venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks" source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.17/" doi: "10.18653/v1/2025.arabicnlp-sharedtasks.17" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/malhis-2025-jenin-arageneval-shared-task-parameter/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/malhis-2025-jenin-arageneval-shared-task-parameter/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / Jenin at AraGenEval Shared Task: Parameter-Efficient Fine-Tuning and Layer-Wise Analysis of Arabic LLMs for Authorship Style Transfer and Classification # Jenin at AraGenEval Shared Task: Parameter-Efficient Fine-Tuning and Layer-Wise Analysis of Arabic LLMs for Authorship Style Transfer and Classification **Authors:** Huthayfa Malhis, Mohammad Tami, Huthaifa I. Ashqar · **Year:** 2025 · **Venue:** Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.17/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.17.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.17](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.17) · [OpenAlex](https://openalex.org/W4416037822) **Citations:** 0 citations (openalex-snapshot · 2026-10-14) · [1](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary This paper reports one team's entry to the AraGenEval shared task, which covers three problems in Modern Standard Arabic. The first is rewriting a passage in the voice of a chosen author (Author Style Transfer). The second is identifying which of 17 authors wrote a paragraph. The third is separating human-written text from LLM output. For style transfer, the team adapted small decoder models (Gemma3-1B, Qwen2.5-1.5B) with LoRA. The model receives the target author's name as a control signal, and a Sentence-BERT cosine penalty discourages drift in meaning. For the two classification tasks, AraBERTv2 and AraModernBert were fully fine-tuned. The team also trained a simple probe on each layer's hidden states to locate stylistic information. Gemma clearly outperformed Qwen at style transfer. Detection on the new ARATECT data was near ceiling (about 99% accuracy). Author attribution reached about 0.89 macro-F1, although the abstract's claim of >0.99 for classification conflicts with the results table. Middle layers proved more informative than the top layer. The work matters as an early multi-task benchmark for stylistic Arabic NLP. ## Key points - **AST corpus:** 39,279 paired samples from 17 authors, averaging about 335 words each (about 13.1M words in total). A disproportionate share of the samples comes from three writers: Mohammad Hussein Heikal, Ahmad Amin, and Hassan Hanafi. The author-classification set is the same corpus relabelled for multi-class prediction. - **Style transfer results (100 unseen articles):** - Gemma r=32 scored best, with BLEU 19.04 and chrF 55.14. - Gemma r=64 scored 18.85/55.00 and Gemma r=16 scored 18.13/54.75. - Qwen r=16 lagged far behind at 10.18/44.42. - **Author classification:** - AraBERTv2 reached 89.71% accuracy and 0.89 macro-F1. - AraModernBert reached 87.1% accuracy and 0.87 macro-F1. - The reported best layers are 7 and 20 respectively, even though probing is described only over layers 1–12. - **ARATECT detection data:** Human texts come from news and literary sources. Machine texts come from GPT-4, Mistral, LLaMA and similar models. AraModernBert achieved 99.4% accuracy (F1 0.9923) and AraBERTv2 achieved 99.3% (F1 0.9932). - **Setup:** - Splits were stratified: 70/15/15 for style transfer and classification, 80/10/10 for detection. - Preprocessing removed diacritics and normalized characters, with a 512-token maximum length. - Encoders were trained with AdamW (lr 2e-5, batch 4, 3 epochs, 5% warmup). ## Interesting ideas - Train a cheap logistic-regression probe on each layer's hidden states instead of trusting only the final representation; the most useful signal may sit mid-network. — For Mova (tree): probe each layer of gparse to choose which layer, or mix of layers, should feed the arc and label scorers, rather than defaulting to the top layer. - While training a rewriter, penalize the distance between source and output meaning so that changing style does not silently change content. — For Mova (back): in the text → graph → text compressor, re-parse the generated text and penalize divergence from the source graph, an explainable analogue of the embedding check. - Condition a single generator on a target identity (here an author name) passed as an input token, rather than training separate models per style. — For Mova (back): give the generator a register or style tag (plain, formal, child-level) so one realizer can produce answers for different readers. - Error analysis showed fluent machine text passing as human and messy human social-media text flagged as machine. Surface cues mislead, so evaluation needs adversarial slices. — For Mova (learn): include noisy human text and polished LLM text as negative controls when evaluating silver-data filters, so the filters are not merely rewarding fluency. - Paraphrase human samples to widen stylistic variance, so a classifier cannot rely on lexical overlap alone. — For Mova (answer): paraphrase school-test questions and fable QA items to verify that readers answer from the meaning graph, not from word matches with the text. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/) · [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 1 paper of our library cites it (of the 140884 whose bibliographies we read); each has a page here. - [The AraGenEval Shared Task on Arabic Authorship Style Transfer and AI Generated Text Detection](https://md.research.elamur.ai/free/en/papers/abudalfa-2025-arageneval-shared-task-arabic-authorship/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{malhis2025jenin, title = {{Jenin at AraGenEval Shared Task: Parameter-Efficient Fine-Tuning and Layer-Wise Analysis of Arabic LLMs for Authorship Style Transfer and Classification}}, author = {Malhis, Huthayfa and Tami, Mohammad and Ashqar, Huthaifa I.}, booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks}, year = {2025}, doi = {10.18653/v1/2025.arabicnlp-sharedtasks.17}, url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.17} } ``` - [← previous Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://md.research.elamur.ai/free/en/papers/chen-2025-janus-pro-unified-multimodal/) - [next → KernelBench: Can LLMs Write Efficient GPU Kernels?](https://md.research.elamur.ai/free/en/papers/ouyang-2025-kernelbench-llms-write-efficient-gpu/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) ## References 20 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Abudalfa, S., Ezzini, S., Abdelali, A., Alami, H., Benlahbib, A., Chafik, S., El-Haj, M., Mahdaouy, A. El, Jarrar, M., Lamsiyah, S., & Luqman, H. (2025). The AraGenEval Shared Task on Arabic Authorship Style Transfer and AI-Generated Text Detection. Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025). [find](https://scholar.google.com/scholar?q=Abudalfa%2C+S%2E%2C+Ezzini%2C+S%2E%2C+Abdelali%2C+A%2E%2C+Alami%2C+H%2E%2C+Benlahbib%2C+A%2E%2C+Chafik%2C+S%2E%2C+El%2DHaj%2C+M%2E%2C+Mahdaouy%2C+A%2E+El%2C+Jarrar%2C+M%2E%2C+Lamsiyah%2C+S%2E%2C+%26+Luqman%2C+H%2E+%282025%29%2E+The+AraGenEval+Shared+Task+on+Arabic+Authorshi) [our card](https://md.research.elamur.ai/free/en/papers/abudalfa-2025-arageneval-shared-task-arabic-authorship/) 2. Almanea, M. M. (2021). Automatic methods and neural networks in Arabic texts diacritization: a comprehensive survey. IEEE Access, 9, 145012–145032. [find](https://scholar.google.com/scholar?q=Almanea%2C+M%2E+M%2E+%282021%29%2E+Automatic+methods+and+neural+networks+in+Arabic+texts+diacritization%3A+a+comprehensive+survey%2E+IEEE+Access%2C+9%2C+145012%E2%80%93145032%2E) 3. Almarwani, N., & Aloufi, S. (2023). SANA at NADI 2023 shared task: Ensemble of Layer-Wise BERT-based models for Dialectal Arabic Identification. Proceedings of ArabicNLP 2023, 625– 630. [find](https://scholar.google.com/scholar?q=Almarwani%2C+N%2E%2C+%26+Aloufi%2C+S%2E+%282023%29%2E+SANA+at+NADI+2023+shared+task%3A+Ensemble+of+Layer%2DWise+BERT%2Dbased+models+for+Dialectal+Arabic+Identification%2E+Proceedings+of+ArabicNLP+2023%2C+625%E2%80%93+630%2E) 4. Alqahtani, F., & Dohler, M. (2023). Survey of authorship identification tasks on Arabic texts. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(4), 1–24. [find](https://scholar.google.com/scholar?q=Alqahtani%2C+F%2E%2C+%26+Dohler%2C+M%2E+%282023%29%2E+Survey+of+authorship+identification+tasks+on+Arabic+texts%2E+ACM+Transactions+on+Asian+and+Low%2DResource+Language+Information+Processing%2C+22%284%29%2C+1%E2%80%9324%2E) 5. Al-Sarem, M., Saeed, F., Alsaeedi, A., Boulila, W., & Al-Hadhrami, T. (2020). Ensemble methods for instance-based Arabic language authorship attribution. IEEE Access, 8, 17331–17345. [find](https://scholar.google.com/scholar?q=Al%2DSarem%2C+M%2E%2C+Saeed%2C+F%2E%2C+Alsaeedi%2C+A%2E%2C+Boulila%2C+W%2E%2C+%26+Al%2DHadhrami%2C+T%2E+%282020%29%2E+Ensemble+methods+for+instance%2Dbased+Arabic+language+authorship+attribution%2E+IEEE+Access%2C+8%2C+17331%E2%80%9317345%2E) 6. Ashqar, H. I., & Tami, M. (2025). Translation with LLMs through Prompting with Long-Form Context. Authorea Preprints. [find](https://scholar.google.com/scholar?q=Ashqar%2C+H%2E+I%2E%2C+%26+Tami%2C+M%2E+%282025%29%2E+Translation+with+LLMs+through+Prompting+with+Long%2DForm+Context%2E+Authorea+Preprints%2E) 7. Han, Z., Gao, C., Liu, J., Zhang, J., & Zhang, S. Q. (2024). Parameter-efficient fine-tuning for large models: A comprehensive survey. ArXiv Preprint ArXiv:2403.14608. [↗](https://arxiv.org/abs/2403.14608) [our card](https://md.research.elamur.ai/free/en/papers/han-2024-parameter-efficient-fine-tuning-large/) 8. Joshi, S., Khan, M. S., Dafe, A., Singh, K., Zope, V., & Jhamtani, T. (2024). Fine tuning LLMs for low resource languages. 2024 5th International Conference on Image Processing and Capsule Networks (ICIPCN), 511–519. [find](https://scholar.google.com/scholar?q=Joshi%2C+S%2E%2C+Khan%2C+M%2E+S%2E%2C+Dafe%2C+A%2E%2C+Singh%2C+K%2E%2C+Zope%2C+V%2E%2C+%26+Jhamtani%2C+T%2E+%282024%29%2E+Fine+tuning+LLMs+for+low+resource+languages%2E+2024+5th+International+Conference+on+Image+Processing+and+Capsule+Networks+%28I) 9. Khoboko, P. W., Marivate, V., & Sefara, J. (2025). Optimizing translation for low-resource languages: Efficient fine-tuning with custom prompt engineering in large language models. Machine Learning with Applications, 20, 100649. [find](https://scholar.google.com/scholar?q=Khoboko%2C+P%2E+W%2E%2C+Marivate%2C+V%2E%2C+%26+Sefara%2C+J%2E+%282025%29%2E+Optimizing+translation+for+low%2Dresource+languages%3A+Efficient+fine%2Dtuning+with+custom+prompt+engineering+in+large+language+models%2E+Machine+Learning+wi) 10. Liu, S., Agarwal, S., & May, J. (2024). Authorship style transfer with policy optimization. ArXiv Preprint ArXiv:2403.08043. [↗](https://arxiv.org/abs/2403.08043) 11. Lv, K., Yang, Y., Liu, T., Gao, Q., Guo, Q., & Qiu, X. (2023). Full parameter fine-tuning for large language models with limited resources. ArXiv Preprint ArXiv:2306.09782. [↗](https://arxiv.org/abs/2306.09782) 12. Masri, S., Raddad, Y., Khandaqji, F., Ashqar, H. I., & Elhenawy, M. (2024). Transformer Models in Education: Summarizing Science Textbooks with AraBART, MT5, AraT5, and mBART. ArXiv Preprint ArXiv:2406.07692. [↗](https://arxiv.org/abs/2406.07692) 13. Najjar, A. A., Ashqar, H. I., Darwish, O. A., & Hammad, E. (2025). Detecting AI-Generated Text in Educational Content: Leveraging Machine Learning and Explainable AI for Academic Integrity. ArXiv Preprint ArXiv:2501.03203. [↗](https://arxiv.org/abs/2501.03203) 14. Najjar, A., Ashqar, H. I., Darwish, O., & Hammad, E. (2025). Leveraging Explainable AI for LLM Text Attribution: Differentiating Human-Written and Multiple LLMs-Generated Text. ArXiv Preprint ArXiv:2501.03212. [↗](https://arxiv.org/abs/2501.03212) 15. Pasad, A., Chou, J.-C., & Livescu, K. (2021). Layer-wise analysis of a self-supervised speech representation model. 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 914–921. [find](https://scholar.google.com/scholar?q=Pasad%2C+A%2E%2C+Chou%2C+J%2E%2DC%2E%2C+%26+Livescu%2C+K%2E+%282021%29%2E+Layer%2Dwise+analysis+of+a+self%2Dsupervised+speech+representation+model%2E+2021+IEEE+Automatic+Speech+Recognition+and+Understanding+Workshop+%28ASRU%29%2C+914%E2%80%93921%2E) 16. Patel, A., Andrews, N., & Callison-Burch, C. (2022). Low-resource authorship style transfer: Can non-famous authors be imitated? ArXiv Preprint ArXiv:2212.08986. [↗](https://arxiv.org/abs/2212.08986) [our card](https://md.research.elamur.ai/free/en/papers/patel-2022-low-resource-authorship-style-transfer/) 17. Radhakrishnan, S., Yang, C.-H. H., Khan, S. A., Kiani, N. A., Gomez-Cabrero, D., & Tegner, J. N. (2023). A parameter-efficient learning approach to arabic dialect identification with pre-trained general-purpose speech model. ArXiv Preprint ArXiv:2305.11244. [↗](https://arxiv.org/abs/2305.11244) [our card](https://md.research.elamur.ai/free/en/papers/radhakrishnan-2023-parameter-efficient-learning-approach-arabic/) 18. Sammoudi, M., Habaybeh, A., Ashqar, H. I., & Elhenawy, M. (2024). Question-Answering (QA) Model for a Personalized Learning Assistant for Arabic Language. ArXiv Preprint ArXiv:2406.08519. [↗](https://arxiv.org/abs/2406.08519) [our card](https://md.research.elamur.ai/free/en/papers/sammoudi-2024-question-answering-qa-model-personalized/) 19. Tami, M., Ashqar, H. I., & Elhenawy, M. (2024). Automated Question Generation for Science Tests in Arabic Language Using NLP Techniques. ArXiv Preprint ArXiv:2406.08520. [↗](https://arxiv.org/abs/2406.08520) [our card](https://md.research.elamur.ai/free/en/papers/tami-2024-automated-question-generation-science-tests/) 20. Van Aken, B., Winter, B., Löser, A., & Gers, F. A. (2019). How does bert answer questions? a layer-wise analysis of transformer representations. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 1823– 1832. A Appendices [find](https://scholar.google.com/scholar?q=Van+Aken%2C+B%2E%2C+Winter%2C+B%2E%2C+L%C3%B6ser%2C+A%2E%2C+%26+Gers%2C+F%2E+A%2E+%282019%29%2E+How+does+bert+answer+questions%3F+a+layer%2Dwise+analysis+of+transformer+representations%2E+Proceedings+of+the+28th+ACM+International+Conference+on)