--- title: "Osint at AraGenEval shared task — paper by Agrahari et al. (2025)" description: "The authors describe their entry to AraGenEval 2025, a three-part Arabic shared task. 54.97 F1. 1 reference, cited by 1 paper here." authors: ["Shifali Agrahari", "Hemanth Prakash Simhadri", "Ashutosh Kumar Verma", "Ranbir Singh Sanasam"] year: 2025 venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks" source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.14/" doi: "10.18653/v1/2025.arabicnlp-sharedtasks.14" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/agrahari-2025-osint-arageneval-shared-task-fine/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/agrahari-2025-osint-arageneval-shared-task-fine/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/16/) / Osint at AraGenEval shared task: Fine-Tuned Modeling for Tracking Style Signatures and AI Generation in Arabic Texts # Osint at AraGenEval shared task: Fine-Tuned Modeling for Tracking Style Signatures and AI Generation in Arabic Texts **Authors:** Shifali Agrahari, Hemanth Prakash Simhadri, Ashutosh Kumar Verma and [Ranbir Singh Sanasam](https://md.research.elamur.ai/free/en/authors/ranbir-singh-sanasam/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.14/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.14.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.14](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.14) · [OpenAlex](https://openalex.org/W4416037675) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/agrahari-2025-osint-arageneval-shared-task-fine/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The authors describe their entry to AraGenEval 2025, a three-part Arabic shared task. The subtasks were rewriting formal MSA paragraphs in the voice of one of 21 writers, attributing a paragraph to one of those writers, and separating human-written Arabic from LLM output. Arabic makes all three harder because of diglossia, ambiguous rich morphology, and noisy orthography such as alef variants and inconsistent diacritics. - **Style transfer:** they fine-tuned AraT5-base with an author tag and separator token prepended to the input. Preprocessing was kept minimal so stylistic quirks survive. - **Attribution:** they put a small MLP head on AraBERTv2-base and added English round-trip backtranslation plus class-balanced batches. - **Detection:** they joined mBERT's [CLS] vector with eight standardized surface statistics. Reported scores are BLEU 19.87 / chrF 54.97, F1 0.79673 for attribution (on the validation split), and F1 0.75 for detection (8th place). The work offers a compact set of cheap baselines. However, its ablation table conflicts with the main results and cites a contrastive loss never described in the method, so those numbers deserve caution. ## Key points - Subtasks 1–2 use books by 21 writers (10 each), split into paragraphs and normalized to formal register with GPT-4o mini. ARATECT has 50,000/5,000/10,000 train/valid/test paragraphs, with machine text from GPT-4, Claude, Jais and others. - **Style transfer setup:** lr 3e-5, batch 2, 3 epochs, 512-token limit. Plain 4-beam search (BLEU 19.87, chrF 54.97) narrowly outperformed a diversity-oriented decoding variant that divided 8 beams into 4 groups, used a 0.7 penalty and kept the shortest candidate (19.49 / 54.57). - **Attribution setup:** head of 768→256, ReLU, dropout 0.3, then 21 classes, with 256-token inputs. Character n-gram subword embeddings gave no gain and were dropped. Final scores were F1 0.79673 and accuracy 0.83335. - **Detection setup:** a 776-dim vector (768 + 8 features) feeds a 64-unit head. mBERT reached F1 0.75, precision 0.67, recall 0.86 and specificity 0.58. The AraBERTv2 variant collapsed (accuracy 0.498, specificity 0.156), labelling almost everything as machine-made. - **Error analysis:** - Over-normalized, flattened style outputs. - Confusion between thematically similar authors (e.g., rural Egypt topics). - Formulaic human news flagged as AI. - Casual-toned LLM text accepted as human. ## Interesting ideas - Controlling generation by prepending an explicit style/author token plus a separator lets one seq2seq model serve many target voices without separate models. - Concatenating a contextual embedding with a handful of standardized, human-readable surface features gives the classifier signals a person can inspect. - Reporting specificity and balanced accuracy, not just F1, exposed a detector that was effectively predicting one class. - Generating diverse candidates and picking by a naive criterion (shortest) did not help; diversity pays off only with a real scorer or reranker. - Class-balanced batch sampling counters skewed label distributions without new data. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (back): the graph→text generator could take a register or genre tag token, so one realizer produces fable-style, school-test-style or plain output from the same meaning graph. - **I2** (learn): a silver-parse quality filter could fuse encoder states with interpretable features (sentence length, punctuation, witness disagreement), keeping its accept/reject decisions explainable. - **I3** (learn): evaluations with negative controls should always report specificity on the controls, so a reader or repair rule that says "yes" to everything is caught. - **I4** (back): candidate realizations should be reranked by round-trip parsing back to the graph and checking meaning preservation, not by length. - **I5** (tree): gparse training could balance batches across treebank sources or rare dependency relations, so small UD dialect treebanks and rare relations are not drowned out. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Indian Institute of Technology Guwahati](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-guwahati/) — University, Guwahati, India · 3 works · 46 citations - [Manipal Academy of Higher Education](https://md.research.elamur.ai/free/en/universities/manipal-academy-of-higher-education/) — University, Manipal, India · 5 works · 57 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works ## Cite Cite as: Shifali Agrahari, Hemanth Prakash Simhadri, Ashutosh Kumar Verma and Ranbir Singh Sanasam. 2025. Osint at AraGenEval shared task: Fine-Tuned Modeling for Tracking Style Signatures and AI Generation in Arabic Texts. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{agrahari2025osint, title = {{Osint at AraGenEval shared task: Fine-Tuned Modeling for Tracking Style Signatures and AI Generation in Arabic Texts}}, author = {Shifali Agrahari, Hemanth Prakash Simhadri, Ashutosh Kumar Verma and Ranbir Singh Sanasam}, booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks}, year = {2025}, doi = {10.18653/v1/2025.arabicnlp-sharedtasks.14}, url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.14} } ``` - [← previous Open-domain Arabic Conversational Question Answering with Question Rewriting](https://md.research.elamur.ai/free/en/papers/hassib-2025-open-domain-arabic-conversational-question/) - [next → PalNLP at BAREC Shared Task 2025: Predicting Arabic Readability Using Ordinal Regression and K-Fold Ensemble Learning](https://md.research.elamur.ai/free/en/papers/ayesh-2025-palnlp-barec-shared-task-2025/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/16/) ## References 1 references, 1 of them with a card of ours. ↗ leads to the work. 1. Shadi Abudalfa, Saad Ezzini, Ahmed Abdelali, Hamza Alami, Abdessamad Benlahbib, Salmane Chafik, Mo El-Haj, Abdelkader El Mahdaouy, Mustafa Jarrar, Salima Lamsiyah, and Hamzah Luqman. 2025. [The AraGenEval Shared Task on Arabic Authorship Style Transfer and AI-Generated Text Detection](https://md.research.elamur.ai/free/en/papers/abudalfa-2025-arageneval-shared-task-arabic-authorship/). In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Association for Computational Linguistics. *Updated 31 October 2026*