---
title: "BDSI at AraSentEval Shared Task : A Multi-Transformer"
description: "The paper describes a system for Subtask 1 of the AraSentEval 2026 shared task: three-way sentiment… 0.8301 accuracy. 9 references."
authors: ["Mohamed M'haouach", "Kaouthar Elyoussoufi", "Abdessamad Benlahbib", "Hamza Alami"]
year: 2026
venue: "The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks"
source: "https://aclanthology.org/2026.osact-1.39/"
doi: "10.63317/4prectoefpgj"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/haouach-2026-bdsi-arasenteval-shared-task-multi/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/haouach-2026-bdsi-arasenteval-shared-task-multi/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/23/) / BDSI at AraSentEval Shared Task : A Multi-Transformer Contrastive Learning for Arabic Dialect Sentiment Analysis

# BDSI at AraSentEval Shared Task : A Multi-Transformer Contrastive Learning for Arabic Dialect Sentiment Analysis

**Authors:** [Mohamed M'haouach](https://md.research.elamur.ai/free/en/authors/mohamed-m-haouach/), [Kaouthar Elyoussoufi](https://md.research.elamur.ai/free/en/authors/kaouthar-elyoussoufi/), [Abdessamad Benlahbib](https://md.research.elamur.ai/free/en/authors/abdessamad-benlahbib/) and [Hamza Alami](https://md.research.elamur.ai/free/en/authors/hamza-alami/) · **Year:** 2026 · **Venue:** [The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks](https://md.research.elamur.ai/free/en/venues/osact/) ([Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks 2026](https://md.research.elamur.ai/free/en/venues/osact/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.osact-1.39/) · [PDF](https://aclanthology.org/2026.osact-1.39.pdf) · [DOI 10.63317/4prectoefpgj](https://doi.org/10.63317/4prectoefpgj) · [OpenAlex](https://openalex.org/W7166594916)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper describes a system for Subtask 1 of the AraSentEval 2026 shared task: three-way sentiment classification (positive, negative, neutral) of hotel reviews written in several Arabic dialects. Dialectal Arabic has no standard spelling, mixes with MSA, and has few labelled resources. The authors fine-tune two pretrained Arabic encoders, AraBERTv2 and CAMeLBERT-Mix, each with its own preprocessing. Training combines class-weighted, label-smoothed cross-entropy with a supervised contrastive term computed on a projected [CLS] embedding. At inference, the two models' softmax outputs are averaged. A small set of regex rules then corrects dialect-specific cases the models tend to miss. On a stratified validation split, each model scores about 0.93 macro F1. The ensemble reaches 0.83 macro F1 on the official test set. The authors attribute the gap to dialect variety in the unseen data. The work shows a compact, reproducible recipe for low-resource dialect classification, trained only on the provided data and run on Google Colab.

## Key points
- Data: 1,731 hotel reviews in Darija, Saudi, Jordanian and Egyptian dialects. Labels are positive 609, negative 657 and neutral 465. The data was split 85/15 with stratification, with no external data or augmentation.
- Models: aubmindlab/bert-base-arabertv2 and CAMeL-Lab/bert-base-arabic-camelbert-mix. Each has a two-layer GELU MLP classifier (dropout 0.3) and a 128-dimensional projection head for the contrastive loss.
- Loss: cross-entropy (label smoothing 0.1, class weights) plus 0.2 × supervised contrastive loss with temperature 0.07.
- Training: AdamW with learning rate 2e-5 for the backbone and 1e-5 for the heads, batch size 16, maximum length 128, and early stopping on validation macro F1 (patience 5).
- Results: validation macro F1 was 0.9355 for AraBERTv2 and 0.9319 for CAMeLBERT. The ensemble scored 0.8301 macro F1 and 0.8301 accuracy on the test set.

## Interesting ideas
- Each encoder gets the preprocessing that fits its pretraining: the official ArabertPreprocessor for AraBERTv2, and nearly raw text for CAMeLBERT so its dialect cues survive. This avoids one shared normalization that suits neither model.
- An auxiliary supervised contrastive loss, applied through a separate projection head, pulls same-label examples together in embedding space without changing the main classifier.
- The ensemble averages full probability distributions from two complementary models instead of taking hard votes, so it keeps their uncertainty.
- Post-hoc rules fire only on specific discourse patterns and only when the model's initial prediction matches the rule's condition. The patterns include a decline over time ("was excellent… became acceptable"), stated intent to return, and doubts about a review's authenticity.
- The roughly 0.10 drop from a random in-distribution validation split to the official test set warns that such splits overstate robustness on varied input.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Keep raw and normalized forms side by side, and give each downstream component (tagger, lemmatizer, gparse) the variant it was trained on, rather than forcing one normalization on all of them.
- **I2** (learn): When training gparse, a contrastive term over arc or relation-label embeddings could sharpen distinctions between often-confused UD relations without altering the scoring head.
- **I3** (learn): Silver-data agreement among witness parsers could average per-arc probability distributions, flagging low-margin arcs for the LLM teacher instead of discarding them.
- **I4** (frames): These patterns map naturally to explainable pragmatic cues: a temporal-contrast frame marking a state change, a commitment speech act, and a meta-comment that undercuts the text's sincerity.
- **I5** (learn): Evaluate on held-out genres or sources as a standard control, so scores from same-distribution dev sets are not mistaken for generalization.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Sidi Mohamed Ben Abdellah University](https://md.research.elamur.ai/free/en/universities/sidi-mohamed-ben-abdellah-university/) — University, Fes, Morocco · 8 works · 156 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Mohamed M'haouach, Kaouthar Elyoussoufi, Abdessamad Benlahbib and Hamza Alami. 2026. BDSI at AraSentEval Shared Task : A Multi-Transformer Contrastive Learning for Arabic Dialect Sentiment Analysis. The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mhaouach2026bdsi,
  title = {{BDSI at AraSentEval Shared Task : A Multi-Transformer Contrastive Learning for Arabic Dialect Sentiment Analysis}},
  author = {Mohamed M'haouach, Kaouthar Elyoussoufi, Abdessamad Benlahbib and Hamza Alami},
  booktitle = {The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks},
  year = {2026},
  doi = {10.63317/4prectoefpgj},
  url = {https://doi.org/10.63317/4prectoefpgj}
}
```

- [← previous Bayesian Social Deduction with Graph-Informed Language Models](https://md.research.elamur.ai/free/en/papers/rahimirad-2026-bayesian-social-deduction-graph-informed/)
- [next → Bears, all bears, and some bears. Language Constraints on Language Models' Inductive Inferences](https://md.research.elamur.ai/free/en/papers/padmanabhan-2026-bears-all-bears-some-bears/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/23/)

## References

9 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Maram Alharbi, Saad Ezzini, Hansi Hettiarachchi, Tharindu Ranasinghe, and Ruslan Mitkov. 2025a. Evaluating large language models on arabic dialect sentiment analysis. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing (RANLP). [find](https://scholar.google.com/scholar?q=Maram+Alharbi%2C+Saad+Ezzini%2C+Hansi+Hettiarachchi%2C+Tharindu+Ranasinghe%2C+and+Ruslan+Mitkov%2E+2025a%2E+Evaluating+large+language+models+on+arabic+dialect+sentiment+analysis%2E+In+Proceedings+of+the+15th+Intern)
2. Maram I Alharbi, Salmane Chafik, Saad Ezzini, Ruslan Mitkov, Tharindu Ranasinghe, and Hansi Hettiarachchi. 2025b. [Ahasis: Shared task on sentiment analysis for arabic dialects](https://md.research.elamur.ai/free/en/papers/alharbi-2025-ahasis-shared-task-sentiment-analysis/). In Proceedings of the Shared Task on Sentiment Analysis for Arabic Dialects, pages 1–6.
3. Maram I Alharbi, Saad Ezzini, Hansi Hettiarachchi, Tharindu Ranasinghe, and Ruslan Mitkov. 2025c. [Ador: Dataset for arabic dialects in hotel reviews: A human benchmark for sentiment analysis](https://md.research.elamur.ai/free/en/papers/alharbi-2025-ador-dataset-arabic-dialects-hotel/). In Proceedings of the First Workshop on Advancing NLP for Low-Resource Languages, pages 187– 191.
4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT4), pages 9–15. European Language Resources Association. Model available at: https://huggingface.co/aubmindlab/bert-base-arabertv2. [↗](https://doi.org/10.48550/arxiv.2003.00104)
5. Saad Ezzini, Shadi Abudalfa, Maram Alharbi, Salmane Chafik, Samaher Alghamdi, Reem Alotaibi, Hamzah Luqman, Mo El-Haj, and Paul Rayson. 2026. AraSentEval: A shared task on sentiment analysis and swapping in arabic. In Proceedings of the 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7), co-located with the Language Resources and Evaluation Conference (LREC 2026), Palma, Mallorca, Spain. [find](https://scholar.google.com/scholar?q=Saad+Ezzini%2C+Shadi+Abudalfa%2C+Maram+Alharbi%2C+Salmane+Chafik%2C+Samaher+Alghamdi%2C+Reem+Alotaibi%2C+Hamzah+Luqman%2C+Mo+El%2DHaj%2C+and+Paul+Rayson%2E+2026%2E+AraSentEval%3A+A+shared+task+on+sentiment+analysis+and+swapp)
6. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. [The interplay of variant, size, and task type in Arabic pre-trained language models](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop (WANLP 2021), pages 92–104. Association for Computational Linguistics. Model available at: [https://huggingface.co/CAMeL-Lab/bert-base-arabic-camelbert-mix](https://huggingface.co/CAMeL-Lab/bert-base-arabic-camelbert-mix). [↗](https://huggingface.co/CAMeL-Lab/bert-base-arabic-camelbert-mix)
7. Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. In Advances in Neural Information Processing Systems, volume 33, pages 18661–18673. Curran Associates, Inc. Also available as arXiv:2004.11362. [↗](https://arxiv.org/abs/2004.11362)
8. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. [↗](https://doi.org/10.48550/arxiv.1612.01474)
9. Priyanka Ray and Amlan Chakrabarti. 2022. A mixed approach of deep learning method and rule-based method to improve aspect level sentiment analysis. Applied Computing and Informatics, 18(1/2):163–178. [find](https://scholar.google.com/scholar?q=Priyanka+Ray+and+Amlan+Chakrabarti%2E+2022%2E+A+mixed+approach+of+deep+learning+method+and+rule%2Dbased+method+to+improve+aspect+level+sentiment+analysis%2E+Applied+Computing+and+Informatics%2C+18%281%2F2%29%3A163%E2%80%93178%2E)

*Updated 1 November 2026*