---
title: "AlexUNLP-BH at StanceEval2024 — paper by Badran et al. (2024)"
description: "The paper describes the winning system of the StanceEval 2024 shared task at ArabicNLP 2024. 19 references, cited by 4 papers here."
authors: ["Mohamed Badran", "Mo'men Hamdy", "Marwan Torki", "Nagwa El-Makky"]
year: 2024
venue: "Proceedings of the Second Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2024.arabicnlp-1.96/"
doi: "10.18653/v1/2024.arabicnlp-1.96"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/badran-2024-alexunlp-bh-stanceeval2024-multiple-contrastive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/badran-2024-alexunlp-bh-stanceeval2024-multiple-contrastive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/10/) / AlexUNLP-BH at StanceEval2024: Multiple Contrastive Losses Ensemble Strategy with Multi-Task Learning For Stance Detection in Arabic

# AlexUNLP-BH at StanceEval2024: Multiple Contrastive Losses Ensemble Strategy with Multi-Task Learning For Stance Detection in Arabic

**Authors:** Mohamed Badran, Mo'men Hamdy, [Marwan Torki](https://md.research.elamur.ai/free/en/authors/marwan-torki/) and [Nagwa El-Makky](https://md.research.elamur.ai/free/en/authors/nagwa-el-makky/) · **Year:** 2024 · **Venue:** [Proceedings of the Second Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2024](https://md.research.elamur.ai/free/en/venues/arabicnlp/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.arabicnlp-1.96/) · [PDF](https://aclanthology.org/2024.arabicnlp-1.96.pdf) · [DOI 10.18653/v1/2024.arabicnlp-1.96](https://doi.org/10.18653/v1/2024.arabicnlp-1.96) · [OpenAlex](https://openalex.org/W4402671313)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/badran-2024-alexunlp-bh-stanceeval2024-multiple-contrastive/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes the winning system of the StanceEval 2024 shared task at ArabicNLP 2024. Systems must assign each Arabic tweet a stance of Favor, Against or None with respect to one of three targets: women empowerment, the COVID-19 vaccine, or digital transformation. The authors use only the Mawqif dataset. Mawqif annotates each tweet for stance, sentiment and sarcasm.

They train a separate AraBERT-based classifier for each target and improve it in stages:
- on-the-fly text augmentation plus dropout;
- multi-task heads for sentiment and sarcasm whose outputs feed the stance classifier;
- class-weighted cross-entropy;
- a contrastive term computed on stance labels.

Five copies of the model are trained, each with a different contrastive objective, and their predictions are combined by majority vote. The ensemble reaches a macro F1 of 0.8438 on the official test set, which ranked first. The main takeaway is practical: when labelled data is small and imbalanced, varying the representation-shaping loss is a cheap way to obtain ensemble members that make different errors.

## Key points
- Mawqif contains 4,121 tweets, split roughly evenly across the three targets. The authors divided it 70/15/15 into train, validation and test. Label balance differs strongly by target: digital transformation is about 77% Favor, while the COVID vaccine target is nearly even between Favor and Against.
- Backbones are chosen per target:
  - AraBERTv0.2-Twitter-base for women empowerment and digital transformation;
  - AraBERT COVID-19 (AraBERTv2 further tuned on COVID tweets) for the vaccine target.
- The combined loss weights its terms as follows: stance 0.7, sentiment 0.2, sarcasm 0.1, plus the contrastive term with weight 1.
- The five contrastive variants in the ensemble are contrastive loss with L2 distance, triplet margin, FastAP, tuplet margin and signal-to-noise-ratio contrastive loss.
- On the authors' own test split, overall macro F1 improves as each step is added:

  | Step | Macro F1 |
  |---|---|
  | Baseline | 0.6882 |
  | + augmentation and dropout | 0.7589 |
  | + multi-task and weighted loss | 0.8008 |
  | Single contrastive models | 0.7914–0.8172 |
  | Majority-vote ensemble | 0.8432 |

  Official per-target scores were 0.8855 (women empowerment), 0.8331 (COVID vaccine) and 0.8127 (digital transformation).

## Interesting ideas
- The ensemble gets its diversity from training each member with a different metric-learning loss rather than different seeds or architectures, so the members carve up the embedding space differently and their errors decorrelate.
- Related labels (sentiment, sarcasm) get their own heads, and those heads feed the main classifier, with fixed low weights that keep the main task dominant.
- Augmentation is regenerated at every epoch rather than fixed in advance, using light word dropping and synonym substitution of about 10% of words.
- Results are reported as a cumulative ablation ladder broken down by target, which exposed that one target (digital transformation) started very weak (0.4345) and gained the most from the added steps.
- The task metric averages F1 over only the two polar classes and ignores the neutral class, so the score focuses on getting the opinionated cases right.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): witness parsers for silver data could share one architecture yet be trained with different auxiliary objectives, which would make their agreement a more meaningful signal than agreement between near-clones.
- **I2** (frames): irony and attitude detection in prag could be trained jointly with speech-act classification, with irony cues passed as explicit inputs to the act decision so the dependency remains inspectable.
- **I3** (tokens): resampling same-POS synonym swaps each epoch could make the tagger and lemmatizer less tied to specific lexical items without needing new annotation.
- **I4** (learn): evaluations should report cumulative gains per component and per subdomain, so a weak slice is not hidden by an averaged score.
- **I5** (answer): when scoring readers on attitude or yes/no questions, a polar-only F1 alongside overall accuracy would show whether the system handles the committed cases or just defaults to the neutral answer.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Alexandria University](https://md.research.elamur.ai/free/en/universities/alexandria-university/) — University, Alexandria, Egypt · 39 works · 287 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Mohamed Badran, Mo'men Hamdy, Marwan Torki and Nagwa El-Makky. 2024. AlexUNLP-BH at StanceEval2024: Multiple Contrastive Losses Ensemble Strategy with Multi-Task Learning For Stance Detection in Arabic. Proceedings of the Second Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{badran2024alexunlp,
  title = {{AlexUNLP-BH at StanceEval2024: Multiple Contrastive Losses Ensemble Strategy with Multi-Task Learning For Stance Detection in Arabic}},
  author = {Mohamed Badran, Mo'men Hamdy, Marwan Torki and Nagwa El-Makky},
  booktitle = {Proceedings of the Second Arabic Natural Language Processing Conference},
  year = {2024},
  doi = {10.18653/v1/2024.arabicnlp-1.96},
  url = {https://doi.org/10.18653/v1/2024.arabicnlp-1.96}
}
```

- [← previous A Comprehensive View of the Biases of Toxicity and Sentiment Analysis Methods Towards Utterances with African American English Expressions](https://md.research.elamur.ai/free/en/papers/resende-2024-comprehensive-view-biases-toxicity-sentiment/)
- [next → An Empirical Study of Synthetic Data Generation for Implicit Discourse Relation Recognition](https://md.research.elamur.ai/free/en/papers/omura-2024-empirical-study-synthetic-data-generation/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/10/)

## References

19 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mai Ibrahim, Marwan Torki, and Nagwa El-Makky. 2018. Imbalanced toxic comments classification using data augmentation and deep learning. In 2018 17th IEEE international conference on machine learning and applications (ICMLA), pages 875–878. IEEE. [find](https://scholar.google.com/scholar?q=Mai+Ibrahim%2C+Marwan+Torki%2C+and+Nagwa+El%2DMakky%2E+2018%2E+Imbalanced+toxic+comments+classification+using+data+augmentation+and+deep+learning%2E+In+2018+17th+IEEE+international+conference+on+machine+learning+)
2. Muhammad Abdul-Mageed, Haitham Alhuzali, and Mostafa Elaraby. 2018. [You tweet what you speak: A city-level dataset of arabic dialects](https://md.research.elamur.ai/free/en/papers/mageed-2018-you-tweet-what-you-speak/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
3. Abeer ALDayel and Walid Magdy. 2021. Stance detection on social media: State of the art and trends. Information Processing & Management, 58(4):102597. [↗](https://doi.org/10.1016/j.ipm.2021.102597)
4. Sarah Alqurashi, Ahmad Alhindi, and Eisa Alanazi. 2020. [Large arabic twitter dataset on covid-19](https://md.research.elamur.ai/free/en/papers/alqurashi-2020-large-arabic-twitter-dataset-covid/). arXiv preprint arXiv:2004.04315. [↗](https://arxiv.org/abs/2004.04315)
5. Nora Alturayeif, Hamzah Luqman, and Moataz Ahmed. 2023. Enhancing stance detection through sequential weighted multi-task learning. Social Network Analysis and Mining, 14(1):7. [↗](https://doi.org/10.1007/s13278-023-01169-7)
6. Nora Alturayeif, Hamzah Luqman, Zaid Alyafeai, and Asma Yamani. 2024. [Stanceeval 2024:the first arabic stance detection shared task](https://md.research.elamur.ai/free/en/papers/alturayeif-2024-stanceeval-2024-first-arabic-stance/). In Proceedings of The Second Arabic Natural Language Processing Conference (ArabicNLP 2024).
7. Nora Saleh Alturayeif, Hamzah Abdullah Luqman, and Moataz Aly Kamaleldin Ahmed. 2022. [Mawqif: A multi-label arabic dataset for target-specific stance detection](https://md.research.elamur.ai/free/en/papers/alturayeif-2022-mawqif-multi-label-arabic-dataset/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 174–184.
8. Mohamed Seghir Hadj Ameur and Hassina Aliane. 2021. [Aracovid19-mfh: Arabic covid-19 multi-label fake news and hate speech detection dataset](https://md.research.elamur.ai/free/en/papers/ameur-2021-aracovid19-mfh-arabic-covid-19/). Preprint, arXiv:2105.03143. [↗](https://arxiv.org/abs/2105.03143)
9. Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. Advances in neural information processing systems, 33:18661–18673. [↗](https://doi.org/10.48550/arxiv.2004.11362)
10. Dilek Küçük and Cemal Aykut Nufus Fazli. 2020. Stance detection: A survey. ACM Computing Surveys, 53. [find](https://scholar.google.com/scholar?q=Dilek+K%C3%BC%C3%A7%C3%BCk+and+Cemal+Aykut+Nufus+Fazli%2E+2020%2E+Stance+detection%3A+A+survey%2E+ACM+Computing+Surveys%2C+53%2E)
11. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. [Semeval-2016 task 6: Detecting stance in tweets](https://md.research.elamur.ai/free/en/papers/mohammad-2016-semeval-2016-task-6-detecting/). In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31–41.
12. Ahmad Shapiro, Ayman Khalafallah, and Marwan Torki. 2022. [Alexu-aic at arabic hate speech 2022: Contrast to classify](https://md.research.elamur.ai/free/en/papers/shapiro-2022-alexu-aic-arabic-hate-speech/). arXiv preprint arXiv:2207.08557. [↗](https://arxiv.org/abs/2207.08557)
13. Baosheng Yu and Dacheng Tao. 2019. Deep metric learning with tuplet margin loss. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6490–6499. [find](https://scholar.google.com/scholar?q=Baosheng+Yu+and+Dacheng+Tao%2E+2019%2E+Deep+metric+learning+with+tuplet+margin+loss%2E+In+Proceedings+of+the+IEEE%2FCVF+international+conference+on+computer+vision%2C+pages+6490%E2%80%936499%2E)
14. Tongtong Yuan, Weihong Deng, Jian Tang, Yinan Tang, and Binghui Chen. 2019. Signal-to-noise ratio: A robust distance metric for deep metric learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4815–4824. [find](https://scholar.google.com/scholar?q=Tongtong+Yuan%2C+Weihong+Deng%2C+Jian+Tang%2C+Yinan+Tang%2C+and+Binghui+Chen%2E+2019%2E+Signal%2Dto%2Dnoise+ratio%3A+A+robust+distance+metric+for+deep+metric+learning%2E+In+Proceedings+of+the+IEEE%2FCVF+conference+on+compu)
15. Wissam Antoun, Fady Baly, and Hazem Hajj. [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. [↗](https://doi.org/10.48550/arxiv.2003.00104)
16. Fatih Cakir, Kun He, Xide Xia, Brian Kulis, and Stan Sclaroff. 2019. Deep metric learning to rank. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1861–1870. [find](https://scholar.google.com/scholar?q=Fatih+Cakir%2C+Kun+He%2C+Xide+Xia%2C+Brian+Kulis%2C+and+Stan+Sclaroff%2E+2019%2E+Deep+metric+learning+to+rank%2E+In+Proceedings+of+the+IEEE%2FCVF+conference+on+computer+vision+and+pattern+recognition%2C+pages+1861%E2%80%931870)
17. Rich Caruana. 1997. Multitask learning. Machine learning, 28:41–75. [find](https://scholar.google.com/scholar?q=Rich+Caruana%2E+1997%2E+Multitask+learning%2E+Machine+learning%2C+28%3A41%E2%80%9375%2E)
18. Thomas G Dietterich. 2000. Ensemble methods in machine learning. In International workshop on multiple classifier systems, pages 1–15. Springer. [find](https://scholar.google.com/scholar?q=Thomas+G+Dietterich%2E+2000%2E+Ensemble+methods+in+machine+learning%2E+In+International+workshop+on+multiple+classifier+systems%2C+pages+1%E2%80%9315%2E+Springer%2E)
19. Mariam Hussein, Sarah Khaled, Marwan Torki, and Nagwa M El-Makky. 2023. Alex-u 2023 nlp at wo-joodner shared task: Arabinder (bi-encoder for arabic named entity recognition). In Proceedings of Arabic-NLP 2023, pages 797–802. [find](https://scholar.google.com/scholar?q=Mariam+Hussein%2C+Sarah+Khaled%2C+Marwan+Torki%2C+and+Nagwa+M+El%2DMakky%2E+2023%2E+Alex%2Du+2023+nlp+at+wo%2Djoodner+shared+task%3A+Arabinder+%28bi%2Dencoder+for+arabic+named+entity+recognition%29%2E+In+Proceedings+of+Arabic%2D)

*Updated 1 November 2026*