--- title: "Amr&MohamedSabaa at AraGenEval shared task: Arabic" description: "The paper tackles Task 2 of the AraGenEval 2025 shared task: deciding which of 21… 88.90% accuracy. 7 references, cited by 1 paper here." authors: ["Amr Sabaa", "Mohamed Sabaa"] year: 2025 venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks" source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.5/" doi: "10.18653/v1/2025.arabicnlp-sharedtasks.5" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/sabaa-2025-amr-mohamedsabaa-arageneval-shared-task/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sabaa-2025-amr-mohamedsabaa-arageneval-shared-task/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/14/) / Amr&MohamedSabaa at AraGenEval shared task: Arabic Authorship Identification using Term Frequency – Inverse Document Frequency Features with Supervised Machine Learning # Amr&MohamedSabaa at AraGenEval shared task: Arabic Authorship Identification using Term Frequency – Inverse Document Frequency Features with Supervised Machine Learning **Authors:** Amr Sabaa and Mohamed Sabaa · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.5/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.5.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.5](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.5) · [OpenAlex](https://openalex.org/W4416037832) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/sabaa-2025-amr-mohamedsabaa-arageneval-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper tackles Task 2 of the AraGenEval 2025 shared task: deciding which of 21 Arabic writers produced a given literary passage. The authors deliberately avoid neural networks. Their pipeline has three parts: - **Cleaning:** keep only characters from the Arabic Unicode blocks, normalise whitespace, and drop very short snippets. - **Features:** each text is represented by two concatenated TF-IDF views, one built on word unigrams and bigrams and one on character 2–4-grams. - **Classifiers:** five standard models are compared on these features. Linear models clearly outperform tree-based ones. A linear SVM scored best on validation, but the team submitted Logistic Regression because it trained more stably and gives usable probability estimates. That submission reached 88.90% accuracy and 82.74% macro F1 on the hidden test set. An ablation shows that word and character signals add up rather than overlap. Per-author analysis exposes weak spots: rare authors, writers from the same era, and translated texts. The work matters as a cheap, reproducible, interpretable reference point for Arabic stylometry, with code released on GitHub. ## Key points - **Data:** 35,122 training and 4,157 validation passages from 21 authors, with typical lengths around 1,750–1,850 characters. The 8,413-sample test set is heavily imbalanced, ranging from Hassan Hanafi with 3,735 training samples down to 399 for the smallest author. - **Features:** - Word TF-IDF: 15,000 dimensions, n-grams 1–2, sublinear TF, max_df 0.9. - Character TF-IDF: 5,000 dimensions, n-grams 2–4, min_df 2, max_df 0.8. - Combined vector: 20,000 dimensions. - **Validation results:** | Model | Accuracy | Macro F1 | |---|---|---| | Linear SVM (SGD, hinge) | 93.22 | 87.52 | | Logistic Regression | 90.54 | 82.63 | | Naive Bayes | 79.22 | — | | Random Forest | 59.32 | — | | Decision Tree | 32.23 | — | The SVM's stratified 5-fold CV macro F1 was 97.67%. - **Official test (Logistic Regression):** 88.90% accuracy, 82.74% macro F1, 84.53% precision, 83.75% recall. - **Ablation (macro F1):** characters only 0.8199, words only 0.8508, both combined 0.8752. - **Per-author validation accuracy:** Salama Moussa and Gibran Khalil Gibran reach 100%, while Tharwat Abaza drops to 44.44% and Kamel Kilani to 16% (only 25 validation samples). ## Interesting ideas - A cheap lexical baseline (TF-IDF plus a linear classifier) gives any fancier system an honest floor to beat, and its per-class errors show where surface statistics alone fail. - Combining word-level and character n-gram views helped because sub-word chunks pick up morphology and spelling habits that whole-word features miss. - The team picked the submitted model for stability and calibrated probabilities rather than top validation score, and test results stayed close to validation. - Breaking scores down per class and setting them beside class support showed that failures concentrate on rare classes, translated texts and stylistically mixed authors. - A simple script filter (keep only the target language's Unicode blocks, drop tiny fragments) cleaned the noisy corpus before feature extraction. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Run such a bag-of-words baseline as a negative control on Mova's tests; if the structured text→tree→frames route doesn't beat it, the structure isn't earning its keep. - **I2** (tokens): Character 2–4-gram features could back up the English tagger and lemmatiser on unseen or misspelled words, where whole-word lookup has nothing to go on. - **I3** (learn): When choosing parser or tagger checkpoints, prefer calibrated confidence, so low-confidence sentences can be routed reliably to the LLM teacher tier or witness-parser voting. - **I4** (learn): Report evaluation per relation label, frame type or text genre with support counts, so weak spots surface instead of hiding inside an overall average. - **I5** (tokens): A comparable pre-tokenisation filter could strip stray foreign-script and junk fragments from silver-data sources before they reach the tokenizer. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Cairo University](https://md.research.elamur.ai/free/en/universities/cairo-university/) — University, Giza, Egypt · 43 works · 1,352 citations - [Najran University](https://md.research.elamur.ai/free/en/universities/najran-university/) — University, Najrān, Saudi Arabia · 2 works · 1 citation 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [🧾 Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/) · 193 works 💡 Concepts (1): [💡 TF-IDF](https://md.research.elamur.ai/free/en/concepts/tf-idf/) · 26 works ## Cite Cite as: Amr Sabaa and Mohamed Sabaa. 2025. Amr&MohamedSabaa at AraGenEval shared task: Arabic Authorship Identification using Term Frequency – Inverse Document Frequency Features with Supervised Machine Learning. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{sabaa2025amr, title = {{Amr\&MohamedSabaa at AraGenEval shared task: Arabic Authorship Identification using Term Frequency – Inverse Document Frequency Features with Supervised Machine Learning}}, author = {Amr Sabaa and Mohamed Sabaa}, booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks}, year = {2025}, doi = {10.18653/v1/2025.arabicnlp-sharedtasks.5}, url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.5} } ``` - [← previous AMAR at BAREC Shared Task 2025: Arabic Meta-learner for Assessing Readability](https://md.research.elamur.ai/free/en/papers/saeed-2025-amar-barec-shared-task-2025/) - [next → An Analysis of Majāzi and Haqīqi Meanings in Ilmu Dilālah: A Literature Study on Arabic Scholarly Texts](https://md.research.elamur.ai/free/en/papers/saepudin-2025-analysis-majazi-haqiqi-meanings-ilmu/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/14/) ## References 7 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Shadi Abudalfa, Saad Ezzini, Ahmed Abdelali, Hamza Alami, Abdessamad Benlahbib, Salmane Chafik, Mo El-Haj, Abdelkader El Mahdaouy, Mustafa Jarrar, Salima Lamsiyah, and Hamzah Luqman. 2025. [The AraGenEval Shared Task on Arabic Authorship Style Transfer and AI-Generated Text Detection](https://md.research.elamur.ai/free/en/papers/abudalfa-2025-arageneval-shared-task-arabic-authorship/). In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Association for Computational Linguistics, Suzhou, China. 2. Al-Falahi Ahmed, Ramdani Mohamed, and Bellafkih Mostafa. 2019. Arabic poetry authorship attribution using machine learning techniques. 15(7):1012–1021. [↗](https://doi.org/10.3844/jcssp.2019.1012.1021) 3. Ameerah Alothman and AbdulMalik Alsalman. 2020. Arabic morphological analysis techniques. International Journal of Advanced Computer Science and Applications, 11. [↗](https://doi.org/10.14569/ijacsa.2020.0110229) 4. Alaa Saleh Altheneyan and Mohamed El Bachir Menai. 2014. Naïve bayes classifiers for authorship attribution of arabic texts. Journal of King Saud University -Computer and Information Sciences, 26(4):473–484. Special Issue on Arabic NLP. [↗](https://doi.org/10.1016/j.jksuci.2014.06.006) 5. Moshe Koppel, Jonathan Schler, and Shlomo Argamon. 2009. Computational methods in authorship attribution. JASIST, 60:9–26. [↗](https://doi.org/10.1002/asi.20961) 6. Jacques Savoy. 2020. Machine Learning Methods for Stylometry: Authorship Attribution and Author Profiling. [find](https://scholar.google.com/scholar?q=Jacques+Savoy%2E+2020%2E+Machine+Learning+Methods+for+Stylometry%3A+Authorship+Attribution+and+Author+Profiling%2E) 7. Efstathios Stamatatos. 2009. A survey of modern authorship attribution methods. JASIST, 60:538–556. [↗](https://doi.org/10.1002/asi.21001) *Updated 1 November 2026*