--- title: "NLP_wizard at AraGenEval shared task — paper by Hany (2025)" description: "The design deliberately avoids fine-tuning. 8 references, cited by 1 paper here. Among them: 0." authors: ["Mena Hany"] year: 2025 venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks" source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.6/" doi: "10.18653/v1/2025.arabicnlp-sharedtasks.6" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/hany-2025-nlp-wizard-arageneval-shared-task/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hany-2025-nlp-wizard-arageneval-shared-task/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) / NLP\_wizard at AraGenEval shared task: Embedding-Based Classification for AI Detection and Authorship Attribution # NLP\_wizard at AraGenEval shared task: Embedding-Based Classification for AI Detection and Authorship Attribution **Authors:** Mena Hany · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.6/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.6.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.6](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.6) · [OpenAlex](https://openalex.org/W4416037670) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/hany-2025-nlp-wizard-arageneval-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary This system paper reports one team's entry to the AraGenEval shared task at ArabicNLP 2025. The team took part in two Modern Standard Arabic subtasks: identifying which of 21 authors wrote a passage, and deciding whether a passage was written by a human or generated by an LLM (the ARATECT dataset). The design deliberately avoids fine-tuning. The authors run a frozen xlm-roberta-large encoder, build one document vector per text, and train default scikit-learn linear classifiers on those vectors: RidgeClassifierCV for AI detection and LinearSVC for authorship. They compare two ways of forming the vector. One averages every token embedding, the other takes the final-layer [CLS] vector. Averaging won by roughly three F1 points on both tasks, reaching macro-F1 0.7400 for detection and 0.8130 for attribution. Each result placed 10th on its leaderboard. The work matters as a cheap and reproducible baseline that runs on a single consumer GPU. It also gives evidence that style signals are spread across tokens and are not concentrated in a summary position. ## Key points - Authorship data covers 21 authors with 10 public books each, split into paragraphs. ARATECT mixes curated human news and literary texts with outputs from Mistral, GPT-4 and LLaMA. - Inputs were truncated to 512 tokens. Embeddings were computed once and cached, and every model ran with default hyperparameters on one RTX 4060 (8GB). - Blind-test macro-F1 for AI detection was 0.7400 with mean pooling and 0.7100 with [CLS]. - Blind-test macro-F1 for authorship was 0.8130 with mean pooling and 0.7830 with [CLS]. Both mean-pooling runs ranked 10th. - No extra annotated data, lexicons, handcrafted features or rule-based preprocessing were used, only the official train/dev/test splits. ## Interesting ideas - Encode the corpus once with a frozen encoder, cache the vectors, and then sweep many cheap classifiers over the cache, which makes iteration nearly free. - Run a controlled comparison of pooling choices where only the pooling changes, which isolates where the signal actually lives in the representation. - Treat stylistic properties as diffuse, spread over function words, rhythm and punctuation, and aggregate over all tokens instead of trusting one summary slot. - A frozen embedding fed into a linear model makes a strong, cheap reference point that any more elaborate system should beat. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Cache gparse encoder states for a treebank once, then train and compare light probes (e.g. for tag or relation hints) without re-running the transformer each time. - **I2** (tree): When gparse builds span or sentence vectors, benchmark mean versus single-position pooling under identical settings before committing to an architecture. - **I3** (frames): Register and irony cues for prag could be scored by aggregating per-token evidence across the whole utterance rather than reading a single head word. - **I4** (learn): Add such a bag-free embedding probe as a baseline control in evaluations, so we can show how much the explainable tree-to-frames pipeline adds on tasks like school tests. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [King Fahd University of Petroleum and Minerals](https://md.research.elamur.ai/free/en/universities/king-fahd-university-of-petroleum-and-minerals/) — University, Dhahran, Saudi Arabia · 52 works · 4,666 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works ## Cite Cite as: Mena Hany. 2025. NLP\_wizard at AraGenEval shared task: Embedding-Based Classification for AI Detection and Authorship Attribution. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{hany2025nlp, title = {{NLP\_wizard at AraGenEval shared task: Embedding-Based Classification for AI Detection and Authorship Attribution}}, author = {Mena Hany}, booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks}, year = {2025}, doi = {10.18653/v1/2025.arabicnlp-sharedtasks.6}, url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.6} } ``` - [← previous NguyenTriet at MAHED Shared Task: Ensemble of Arabic BERT Models with Hierarchical Prediction and Soft Voting for Text-Based Hope and Hate Detection](https://md.research.elamur.ai/free/en/papers/triet-2025-nguyentriet-mahed-shared-task-ensemble/) - [next → NUR at IslamicEval 2025 Shared Task: Retrieval-Augmented LLMs for Qur'an and Hadith QA](https://md.research.elamur.ai/free/en/papers/amin-2025-nur-islamiceval-2025-shared-task/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) ## References 8 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Shadi Abudalfa, Saad Ezzini, Ahmed Abdelali, Hamza Alami, Abdessamad Benlahbib, Salmane Chafik, Mo El-Haj, Abdelkader El Mahdaouy, Mustafa Jarrar, Salima Lamsiyah, and Hamzah Luqman. 2025. [The AraGenEval Shared Task on Arabic Authorship Style Transfer and AI-Generated Text Detection](https://md.research.elamur.ai/free/en/papers/abudalfa-2025-arageneval-shared-task-arabic-authorship/). In Proceedings of the Third Arabic Natural Language 2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901. 3. Nizar Y Habash. 2010. Introduction to Arabic natural language processing. Morgan & Claypool Publishers. [↗](https://doi.org/10.1007/978-3-031-02139-8) 4. Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks VS Lakshmanan. 2020. [Automatic detection of machine generated text: A critical survey](https://md.research.elamur.ai/free/en/papers/jawahar-2020-automatic-detection-machine-generated-text/). arXiv preprint arXiv:2011.01314. [↗](https://arxiv.org/abs/2011.01314) 5. Mike Kestemont. 2014. Function words in authorship attribution. from black magic to theory? In Proceedings of the 3rd Workshop on Computational Linguistics for Literature (CLFL), pages 59–66. [find](https://scholar.google.com/scholar?q=Mike+Kestemont%2E+2014%2E+Function+words+in+authorship+attribution%2E+from+black+magic+to+theory%3F+In+Proceedings+of+the+3rd+Workshop+on+Computational+Linguistics+for+Literature+%28CLFL%29%2C+pages+59%E2%80%9366%2E) 6. Efstathios Stamatatos. 2009. A survey of modern authorship attribution methods. Journal of the American Society for information Science and Technology, 60(3):538–556. [↗](https://doi.org/10.1002/asi.21001) 7. Adaku Uchendu, Thai Le, Kai Shu, and Dongwon Lee. 2020. [Authorship attribution for neural text generation](https://md.research.elamur.ai/free/en/papers/uchendu-2020-authorship-attribution-neural-text-generation/). In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 8384–8395. 8. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30. *Updated 1 November 2026*