---
title: "StanceEval 2024 — Alturayeif 2024"
description: "StanceEval 2024 was the first shared task on stance detection for Arabic, held as part of the ArabicNLP 2024 conference."
authors: ["Nora Alturayeif", "Hamzah Luqman", "Zaid Alyafeai", "Asma Yamani"]
year: 2024
venue: "Proceedings of the Second Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2024.arabicnlp-1.88/"
doi: "10.18653/v1/2024.arabicnlp-1.88"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/alturayeif-2024-stanceeval-2024-first-arabic-stance/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/alturayeif-2024-stanceeval-2024-first-arabic-stance/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2024](https://md.research.elamur.ai/free/papers/2024/page/4/) / StanceEval 2024: The First Arabic Stance Detection Shared Task

# StanceEval 2024: The First Arabic Stance Detection Shared Task

**Authors:** Nora Alturayeif, Hamzah Luqman, [Zaid Alyafeai](https://md.research.elamur.ai/free/authors/zaid-alyafeai/), Asma Yamani · **Year:** 2024 · **Venue:** Proceedings of the Second Arabic Natural Language Processing Conference · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.arabicnlp-1.88/) · [PDF](https://aclanthology.org/2024.arabicnlp-1.88.pdf) · [DOI 10.18653/v1/2024.arabicnlp-1.88](https://doi.org/10.18653/v1/2024.arabicnlp-1.88) · [OpenAlex](https://openalex.org/W4402671378)

**Citations:** 5 citations (openalex-snapshot · 2026-10-14) · [16](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
StanceEval 2024 was the first shared task on stance detection for Arabic, held as part of the ArabicNLP 2024 conference. Stance detection asks whether a text's author supports, opposes, or takes no position on a target. It differs from sentiment analysis because the target may be abstract or never named, and a tweet's emotional tone can conflict with its stance. Participants built systems that assigned each tweet a Favor, Against, or None label with respect to one of three targets: women empowerment, the COVID-19 vaccine, or digital transformation. Data came from the Mawqif corpus, provided by the organizers, whose tweets also carry sentiment and sarcasm labels. Teams could therefore train on stance alone or use multi-task learning. Systems were ranked by macro F1 over the Favor and Against classes only. Most teams fine-tuned Arabic BERT variants such as MARBERT and AraBERT, while others used ensembles, multi-task setups, or prompted and fine-tuned LLMs. The best system reached an F-score of 84.38. The task gives Arabic NLP, where stance work has been scarce, a common benchmark and an overview of what currently works.

## Key points
- **Participation:** 28 teams registered, 16 submitted valid test runs, and 13 wrote system description papers.
- **Data:** The Mawqif dataset contains 4,121 Arabic tweets, about 1.35–1.4k per topic. It was split 85/15 into 3,502 training and 619 blind test tweets, and the "Favor" label dominates, especially for digital transformation (about 77%).
- **Annotation quality:** Each tweet was labeled by 3–7 annotators, accepted once confidence exceeded 0.7, and annotators who scored under 80% on test questions were excluded.
- **Evaluation:** The metric averages the F1 scores of Favor and Against per target. The None class was used in training but not scored.
- **Systems:** A fine-tuned GPT-3.5-Turbo-0125 placed second with 82.06 F1. MARBERT was the strongest backbone for several teams, including PICT and Rasid, and SMASH found that zero-shot LLMs did better with English prompts and labels.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

16 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 15 not yet.

- [MGKM at StanceEval2024 Fine-Tuning Large Language Models for Arabic Stance Detection](https://md.research.elamur.ai/free/papers/alghaslan-2024-mgkm-stanceeval2024-fine-tuning-large/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{alturayeif2024stanceeval,
  title = {{StanceEval 2024: The First Arabic Stance Detection Shared Task}},
  author = {Alturayeif, Nora and Luqman, Hamzah and Alyafeai, Zaid and Yamani, Asma},
  booktitle = {Proceedings of the Second Arabic Natural Language Processing Conference},
  year = {2024},
  doi = {10.18653/v1/2024.arabicnlp-1.88},
  url = {https://doi.org/10.18653/v1/2024.arabicnlp-1.88}
}
```

- [← previous Some Notes on Lexicographic Criticism and Biblical Hebrew Studies. Revisiting David J. A. Clines's Method](https://md.research.elamur.ai/free/papers/hbo-2024-ortiz-some-notes-lexicographic-criticism-biblical/)
- [next → Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs](https://md.research.elamur.ai/free/papers/lai-2024-step-dpo-step-wise-preference/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/papers/2024/page/4/)

## References

38 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2020. Arbert & marbert: Deep bidirectional transformers for arabic. arXiv preprint arXiv:2101.01785. [↗](https://arxiv.org/abs/2101.01785) [our card](https://md.research.elamur.ai/free/papers/abdul-2021-mageed-arbert-marbert-deep-bidirectional-transformers/)
2. Mamoun Alghaslan and Khaled Almutairy. 2024. Mgkm at stanceeval2024: Fine-tuning large language models for arabic stance detection. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [find](https://scholar.google.com/scholar?q=Mamoun+Alghaslan+and+Khaled+Almutairy%2E+2024%2E+Mgkm+at+stanceeval2024%3A+Fine%2Dtuning+large+language+models+for+arabic+stance+detection%2E+In+Proceedings+of+the+ArabicNLP+2024%2C+Bangkok%2C+Thailand+%28Hybrid%29%2E) [our card](https://md.research.elamur.ai/free/papers/alghaslan-2024-mgkm-stanceeval2024-fine-tuning-large/)
3. Rabab Alkhalifa, Elena Kochkina, and Arkaitz Zubiaga. 2021. Opinions are made to be changed: Temporally adaptive stance classification. In Proceedings of the 2021 Workshop on Open Challenges in Online Social Networks, HT ’21. ACM. [↗](https://doi.org/10.48550/arxiv.2108.12476)
4. Eman Sweiaad S. Alofi and Sami Mnasri. 2024. Stancealert: Effective logistic regression for stance detection. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [find](https://scholar.google.com/scholar?q=Eman+Sweiaad+S%2E+Alofi+and+Sami+Mnasri%2E+2024%2E+Stancealert%3A+Effective+logistic+regression+for+stance+detection%2E+In+Proceedings+of+the+ArabicNLP+2024%2C+Bangkok%2C+Thailand+%28Hybrid%29%2E)
5. Nouf AlShenaifi, Nourah Alangari, and Hadeel Al-Negheimish. 2024. Rasid at stanceeval: Fine-tuning marbert for arabic stance detection. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [find](https://scholar.google.com/scholar?q=Nouf+AlShenaifi%2C+Nourah+Alangari%2C+and+Hadeel+Al%2DNegheimish%2E+2024%2E+Rasid+at+stanceeval%3A+Fine%2Dtuning+marbert+for+arabic+stance+detection%2E+In+Proceedings+of+the+ArabicNLP+2024%2C+Bangkok%2C+Thailand+%28Hybrid%29)
6. Nora Alturayeif, Hamzah Luqman, and Moataz Ahmed. 2023a. Enhancing stance detection through sequential weighted multi-task learning. [↗](https://doi.org/10.1007/s13278-023-01169-7)
7. Nora Alturayeif, Hamzah Luqman, and Moataz Ahmed. 2023b. A systematic review of machine learning techniques for stance detection and its applications. Neural Computing and Applications, 35(7):5113–5144. [↗](https://doi.org/10.1007/s00521-023-08285-7)
8. Nora Saleh Alturayeif, Hamzah Abdullah Luqman, and Moataz Aly Kamaleldin Ahmed. 2022. Mawqif: A multi-label arabic dataset for target-specific stance detection. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 174–184. [find](https://scholar.google.com/scholar?q=Nora+Saleh+Alturayeif%2C+Hamzah+Abdullah+Luqman%2C+and+Moataz+Aly+Kamaleldin+Ahmed%2E+2022%2E+Mawqif%3A+A+multi%2Dlabel+arabic+dataset+for+target%2Dspecific+stance+detection%2E+In+Proceedings+of+the+The+Seventh+Arabi)
9. Mezghani Amal, Rahma Boujelbane, and Mariem Ellouze. 2024. Anlp rg at stanceeval2024: Comparative evaluation of stance, sentiment and sarcasm detection. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [find](https://scholar.google.com/scholar?q=Mezghani+Amal%2C+Rahma+Boujelbane%2C+and+Mariem+Ellouze%2E+2024%2E+Anlp+rg+at+stanceeval2024%3A+Comparative+evaluation+of+stance%2C+sentiment+and+sarcasm+detection%2E+In+Proceedings+of+the+ArabicNLP+2024%2C+Bangkok%2C+)
10. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. arXiv. [↗](https://doi.org/10.48550/arxiv.2003.00104) [our card](https://md.research.elamur.ai/free/papers/antoun-2020-arabert-transformer-based-model-arabic/)
11. Mohamed Badran, Mo’men Hamdy, Marwan Torki, and Nagwa El-Makky. 2024. Alexunlp-bh at stanceeval2024: Multiple contrastive losses ensemble strategy with multi-task learning for stance detection in arabic. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [find](https://scholar.google.com/scholar?q=Mohamed+Badran%2C+Mo%E2%80%99men+Hamdy%2C+Marwan+Torki%2C+and+Nagwa+El%2DMakky%2E+2024%2E+Alexunlp%2Dbh+at+stanceeval2024%3A+Multiple+contrastive+losses+ensemble+strategy+with+multi%2Dtask+learning+for+stance+detection+in+arab)
12. Iain J. Cruickshank and Lynnette Hui Xian Ng. 2024. Prompting and fine-tuning open-sourced large language models for stance classification. Preprint, arXiv:2309.13734. [↗](https://arxiv.org/abs/2309.13734)
13. Omar Galal and Abdelrahman Kaseb. 2024. Team_zero at stanceeval2024: Frozen plms for arabic stance detection. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [↗](https://doi.org/10.48448/tpk8-kp26)
14. Manuela Gómez-Suta, Julián Echeverry-Correa, and José A. Soto-Mejía. 2023. Stance detection in tweets: A topic modeling approach supporting explainability. Expert Systems with Applications, 214:119046. [↗](https://doi.org/10.1016/j.eswa.2022.119046)
15. Anant Khandelwal. 2021. Fine-tune longformer for jointly predicting rumor stance and veracity. In Proceedings of the 3rd ACM India Joint International Conference on Data Science & Management of Data (8th ACM IKDD CODS & 26th COMAD), CODS COMAD 2021. ACM. [find](https://scholar.google.com/scholar?q=Anant+Khandelwal%2E+2021%2E+Fine%2Dtune+longformer+for+jointly+predicting+rumor+stance+and+veracity%2E+In+Proceedings+of+the+3rd+ACM+India+Joint+International+Conference+on+Data+Science+%26+Management+of+Data+%28)
16. Elena Kochkina, Maria Liakata, and Arkaitz Zubiaga. 2018. All-in-one: Multi-task learning for rumour verification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3402–3413, Santa Fe, New Mexico, USA. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1806.03713)
17. Mirko Lai, Marcella Tambuscio, Viviana Patti, Giancarlo Ruffo, and Paolo Rosso. 2019. Stance polarity in political debates: A diachronic perspective of network homophily and conversations on twitter. Data & Knowledge Engineering, 124:101738. [↗](https://doi.org/10.1016/j.datak.2019.101738)
18. Xiaochong Lan, Chen Gao, Depeng Jin, and Yong Li. 2024. Stance detection with collaborative role-infused llm-based agents. Preprint, arXiv:2310.10467. [↗](https://arxiv.org/abs/2310.10467)
19. Mohamed Lichouri, Khaled Lounnas, OUARAS Khelil Rafik, Mohamed ABi, and Anis Guechtouli. 2024. dzstance@stanceeval2024: Arabic stance detection based on sentence transformers. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [find](https://scholar.google.com/scholar?q=Mohamed+Lichouri%2C+Khaled+Lounnas%2C+OUARAS+Khelil+Rafik%2C+Mohamed+ABi%2C+and+Anis+Guechtouli%2E+2024%2E+dzstance%40stanceeval2024%3A+Arabic+stance+detection+based+on+sentence+transformers%2E+In+Proceedings+of+the+Ar)
20. Youssef Al Hariri and Ibrahim Abu Farha. 2024. Smash at stanceeval 2024: Prompt engineering llms for arabic stance detection. In Proceedings of the Arabic-NLP 2024, Bangkok, Thailand (Hybrid). [find](https://scholar.google.com/scholar?q=Youssef+Al+Hariri+and+Ibrahim+Abu+Farha%2E+2024%2E+Smash+at+stanceeval+2024%3A+Prompt+engineering+llms+for+arabic+stance+detection%2E+In+Proceedings+of+the+Arabic%2DNLP+2024%2C+Bangkok%2C+Thailand+%28Hybrid%29%2E)
21. Yujian Liu, Xinliang Frederick Zhang, David Wegs-man, Nick Beauchamp, and Lu Wang. 2022. Politics: Pretraining with same-story article comparison for ideology prediction and stance detection. Preprint, arXiv:2205.00619. [↗](https://arxiv.org/abs/2205.00619) [our card](https://md.research.elamur.ai/free/papers/liu-2022-politics-pretraining-same-story-article/)
22. Ahmed Abul Hasanaath and Aisha Alansari. 2024. Stancecrafters at stanceeval2024: Multi-task stance detection using bert ensemble with attention based aggregation. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [find](https://scholar.google.com/scholar?q=Ahmed+Abul+Hasanaath+and+Aisha+Alansari%2E+2024%2E+Stancecrafters+at+stanceeval2024%3A+Multi%2Dtask+stance+detection+using+bert+ensemble+with+attention+based+aggregation%2E+In+Proceedings+of+the+ArabicNLP+2024%2C)
23. Anas Melhem, Osama Hamed, and Thaer Sammar. 2024. Tao at stanceeval2024 shared task: Arabic stance detection using arabert. In Proceedings of the Arabic-NLP 2024, Bangkok, Thailand (Hybrid). [find](https://scholar.google.com/scholar?q=Anas+Melhem%2C+Osama+Hamed%2C+and+Thaer+Sammar%2E+2024%2E+Tao+at+stanceeval2024+shared+task%3A+Arabic+stance+detection+using+arabert%2E+In+Proceedings+of+the+Arabic%2DNLP+2024%2C+Bangkok%2C+Thailand+%28Hybrid%29%2E)
24. Tamanna Hossain, Robert L. Logan IV, Arjuna Ugarte, Yoshitomo Matsubara, Sean Young, and Sameer Singh. 2020. COVIDLies: Detecting COVID-19 misinformation on social media. In Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tamanna+Hossain%2C+Robert+L%2E+Logan+IV%2C+Arjuna+Ugarte%2C+Yoshitomo+Matsubara%2C+Sean+Young%2C+and+Sameer+Singh%2E+2020%2E+COVIDLies%3A+Detecting+COVID%2D19+misinformation+on+social+media%2E+In+Proceedings+of+the+1st+Wor)
25. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016a. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31– 41, San Diego, California. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Saif+Mohammad%2C+Svetlana+Kiritchenko%2C+Parinaz+Sobhani%2C+Xiaodan+Zhu%2C+and+Colin+Cherry%2E+2016a%2E+SemEval%2D2016+task+6%3A+Detecting+stance+in+tweets%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semant)
26. Michael Ibrahim. 2024. Cufe at stanceeval2024: Arabic stance detection with fine-tuned llama-3 model. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [↗](https://doi.org/10.48448/a9mv-j406)
27. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016b. Semeval-2016 task 6: Detecting stance in tweets. 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31–41. [find](https://scholar.google.com/scholar?q=Saif+M%2E+Mohammad%2C+Svetlana+Kiritchenko%2C+Parinaz+Sobhani%2C+Xiaodan+Zhu%2C+and+Colin+Cherry%2E+2016b%2E+Semeval%2D2016+task+6%3A+Detecting+stance+in+tweets%2E+10th+International+Workshop+on+Semantic+Evaluation+%28SemE)
28. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in arabic pre-trained language models. arXiv preprint arXiv:2103.06678. [↗](https://arxiv.org/abs/2103.06678) [our card](https://md.research.elamur.ai/free/papers/inoue-2021-interplay-variant-size-task-type/)
29. Mustapha Jaballah. 2024. Ishfmg_tun at stanceeval: Ensemble method for arabic stance evaluation system. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [↗](https://doi.org/10.48448/t1s4-tt13)
30. Mitra Mohtarami, James Glass, and Preslav Nakov. 2019. Contrastive language adaptation for cross-lingual stance detection. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4442–4452, Hong Kong, China. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1910.02076)
31. Ishaan Shukla, Ankit Vaidya, and Geetanjali Vinayak Kale. 2024. Pict at stanceeval2024: Stance detection in arabic using ensemble of large language models. In Proceedings of the ArabicNLP 2024, Bangkok, Thailand (Hybrid). [find](https://scholar.google.com/scholar?q=Ishaan+Shukla%2C+Ankit+Vaidya%2C+and+Geetanjali+Vinayak+Kale%2E+2024%2E+Pict+at+stanceeval2024%3A+Stance+detection+in+arabic+using+ensemble+of+large+language+models%2E+In+Proceedings+of+the+ArabicNLP+2024%2C+Bangko)
32. Qingying Sun, Xuefeng Xi, Jiajun Sun, Zhongqing Wang, and Huiyan Xu. 2022. Stance detection with a multi-target adversarial attention network. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(2):1–21. [↗](https://doi.org/10.1145/3544490)
33. Ingmar Weber, Venkata R. Kiran Garimella, and Alaa Batayneh. 2013. Secular vs. islamist polarization in egypt on twitter. In Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM ’13. ACM. [find](https://scholar.google.com/scholar?q=Ingmar+Weber%2C+Venkata+R%2E+Kiran+Garimella%2C+and+Alaa+Batayneh%2E+2013%2E+Secular+vs%2E+islamist+polarization+in+egypt+on+twitter%2E+In+Proceedings+of+the+2013+IEEE%2FACM+International+Conference+on+Advances+in+So)
34. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [↗](https://arxiv.org/abs/1609.08144)
35. Yuanyu Yang, Bin Wu, Kai Zhao, and Wenying Guo. 2020. Tweet stance detection: A two-stage dc-bilstm model based on semantic attention. In 2020 IEEE Fifth International Conference on Data Science in Cyberspace (DSC). IEEE. [find](https://scholar.google.com/scholar?q=Yuanyu+Yang%2C+Bin+Wu%2C+Kai+Zhao%2C+and+Wenying+Guo%2E+2020%2E+Tweet+stance+detection%3A+A+two%2Dstage+dc%2Dbilstm+model+based+on+semantic+attention%2E+In+2020+IEEE+Fifth+International+Conference+on+Data+Science+in+Cy)
36. Kai Ye, Yangheran Piao, Kun Zhao, and Xiaohui Cui. 2021. Graph Enhanced BERT for Stance-Aware Rumor Verification on Social Media, page 422–435. Springer International Publishing. [find](https://scholar.google.com/scholar?q=Kai+Ye%2C+Yangheran+Piao%2C+Kun+Zhao%2C+and+Xiaohui+Cui%2E+2021%2E+Graph+Enhanced+BERT+for+Stance%2DAware+Rumor+Verification+on+Social+Media%2C+page+422%E2%80%93435%2E+Springer+International+Publishing%2E)
37. Chong Zhang, Zhenkun Zhou, Xingyu Peng, and Ke Xu. 2023. Doubleh: Twitter user stance detection via bipartite graph neural networks. Preprint, arXiv:2301.08774. [↗](https://arxiv.org/abs/2301.08774)
38. Shengping Zhou, Junjie Lin, Lianzhi Tan, and Xin Liu. 2019. Condensed convolution neural network by attention over self-attention for stance detection in twitter. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1–8. [find](https://scholar.google.com/scholar?q=Shengping+Zhou%2C+Junjie+Lin%2C+Lianzhi+Tan%2C+and+Xin+Liu%2E+2019%2E+Condensed+convolution+neural+network+by+attention+over+self%2Dattention+for+stance+detection+in+twitter%2E+In+2019+International+Joint+Conferenc)