--- title: "iWAN-NLP at AHaSIS 2025 — paper by Al-Khalifa (2025)" description: "The paper addresses three-way sentiment classification (positive, negative, neutral) of Arabic… 4.6 points. 2 references, cited by 1 paper here." authors: ["Hend Al-Khalifa"] year: 2025 venue: "Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects" source: "https://aclanthology.org/2025.ranlp-ahasis.2/" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/khalifa-2025-iwan-nlp-ahasis-2025-stacked/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/khalifa-2025-iwan-nlp-ahasis-2025-stacked/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) / iWAN-NLP at AHaSIS 2025: A Stacked Ensemble of Arabic Transformers for Sentiment Analysis on Arabic Dialects in the Hospitality Domain # iWAN-NLP at AHaSIS 2025: A Stacked Ensemble of Arabic Transformers for Sentiment Analysis on Arabic Dialects in the Hospitality Domain **Authors:** [Hend Al-Khalifa](https://md.research.elamur.ai/free/en/authors/hend-al-khalifa/) · **Year:** 2025 · **Venue:** [Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects](https://md.research.elamur.ai/free/en/venues/ranlp/) ([RANLP 2025](https://md.research.elamur.ai/free/en/venues/ranlp/2025/)) · **Publisher:** [INCOMA Ltd.](https://md.research.elamur.ai/free/en/publishers/incoma-ltd/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.ranlp-ahasis.2/) · [PDF](https://aclanthology.org/2025.ranlp-ahasis.2.pdf) **Citations:** [1](https://md.research.elamur.ai/free/en/papers/khalifa-2025-iwan-nlp-ahasis-2025-stacked/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper addresses three-way sentiment classification (positive, negative, neutral) of Arabic hotel reviews that mix Modern Standard Arabic with regional dialects, the problem posed by the AHaSIS 2025 shared task. The author used only the organizers' training split, with no external data and almost no text cleaning. Three Hugging Face encoders with different regional strengths were fine-tuned: MARBERTv2 (broad social-media Arabic), SaudiBERT (Saudi dialect) and DarijaBERT (Moroccan Darija). Each was trained under 5-fold stratified cross-validation with label smoothing, early stopping on validation F1, learning-rate warmup and FP16. At test time, the logits of the five fold models of each architecture are averaged, the three per-architecture averages are averaged again, and argmax gives the label. The title says "stacked", but the combination is plain soft averaging, not a learned meta-classifier. The system reached macro F1 0.8055 on the official test set, which ranked first, and 0.8513 in cross-validation. The result suggests that pooling dialect-specialized encoders is a cheap and strong recipe for dialect-heavy classification. ## Key points - Data: AHaSIS 2025 hotel reviews with three labels and official train/test splits. No augmentation or outside corpora were used. Preprocessing was limited to mapping labels to IDs and running each model's own tokenizer. - Base models: UBC-NLP/MARBERTv2, faisalq/SaudiBERT and SI2M-Lab/DarijaBERT, each with an AutoModelForSequenceClassification head and a custom Hugging Face Trainer. - Training setup: - AdamW, lr 2e-5 with warmup, batch size 8; - up to 6 epochs per fold, early stopping on F1; - nn.CrossEntropyLoss with label smoothing 0.1, FP16; - a single Google Colab GPU. - Scores: test macro F1 0.8055 (first place). Cross-validated macro F1 was 0.8513 with accuracy 0.8628, so the test score was roughly 4.6 points lower. - Error analysis: the main confusion was between neutral and negative. Likely causes listed are sarcasm, implicit sentiment, code-switching, annotation artifacts, and texts tagged as Darija that are actually mixed with MSA or other dialects. ## Interesting ideas - Two-level score averaging reuses all cross-validation checkpoints at inference: fold models are averaged first, then architectures, which smooths out run-to-run variance without retraining on the full data. - The ensemble's diversity comes from encoders pretrained on different language varieties, each fed through its own tokenizer, rather than from random seeds of one model. - Out-of-fold predictions were used to measure the ensemble during development, giving an estimate on data no model in that fold had seen. - The error analysis notes that a plain factual report of a service failure carries negative sentiment without any negative words. Such cases are where neutral and negative get confused. - The analysis also flags that metadata labels can be wrong: some texts tagged as Darija were really mixed dialect or MSA, which hurt the Darija specialist model on those items. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (tree): gparse could average arc and label score matrices from fold-trained checkpoints before decoding, which keeps the output a valid tree, unlike voting on finished trees. - **I2** (learn): witness parsers trained on different genre treebanks (web, spoken, academic) would make agreement-based silver data more trustworthy than witnesses that share one training domain. - **I3** (learn): self-induced repair rules could be induced on some folds and scored only on the held-out fold, which works as a built-in negative control against overfitting. - **I4** (frames): an implicature rule could turn a reported failed expected state ("the AC did not work") into an explicit, traceable negative evaluation of the related entity. - **I5** (learn): before routing text to genre- or dialect-specific components, the declared metadata should be checked against the text itself, and mismatches logged as a separate error class. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works 🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works ## Cite Cite as: Hend Al-Khalifa. 2025. iWAN-NLP at AHaSIS 2025: A Stacked Ensemble of Arabic Transformers for Sentiment Analysis on Arabic Dialects in the Hospitality Domain. Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{alkhalifa2025iwan, title = {{iWAN-NLP at AHaSIS 2025: A Stacked Ensemble of Arabic Transformers for Sentiment Analysis on Arabic Dialects in the Hospitality Domain}}, author = {Hend Al-Khalifa}, booktitle = {Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects}, year = {2025}, url = {https://aclanthology.org/2025.ranlp-ahasis.2/} } ``` - [← previous ISSR: Iterative Selection with Self-Review for Vocabulary Test Distractor Generation](https://md.research.elamur.ai/free/en/papers/liu-2025-issr-iterative-selection-self-review/) - [next → Jenin at AraGenEval Shared Task: Parameter-Efficient Fine-Tuning and Layer-Wise Analysis of Arabic LLMs for Authorship Style Transfer and Classification](https://md.research.elamur.ai/free/en/papers/malhis-2025-jenin-arageneval-shared-task-parameter/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) ## References 2 references, 2 of them with a card of ours. ↗ leads to the work. 1. ## Conclusion Abdul-Mageed, M., Elmadany, A., & Nagoudi, E. M. B. (2021). [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In C. Zong, F. Xia, W. Li, & R. Navigli (Eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 7088–7105). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.acl-long.551 [↗](https://doi.org/10.18653/v1/2021.acl-long.551) 2. Abdul-Mageed, M., Kuebler, S., & Diab, M. (2012). [SAMAR: A System for Subjectivity and Sentiment Analysis of Arabic Social Media](https://md.research.elamur.ai/free/en/papers/mageed-2012-samar-system-subjectivity-sentiment-analysis/). In A. Balahur, A. Montoyo, P. M. Barco, & E. Boldrini (Eds.), Proceedings of the 3rd Workshop in Computational Approaches to Subjectivity and Sentiment Analysis (pp. 19– 28). Association for Computational Linguistics. https://aclanthology.org/W12-3705/ [↗](https://aclanthology.org/W12-3705/) *Updated 1 November 2026*