--- title: "dzStance at StanceEval2024 — Lichouri 2024" description: "The paper addresses Arabic stance detection for the StanceEval 2024 shared task." authors: ["Mohamed Lichouri", "Khaled Lounnas", "Ouaras Rafik", "Mohamed ABi", "Anis Guechtouli"] year: 2024 venue: "Proceedings of the Second Arabic Natural Language Processing Conference" source: "https://aclanthology.org/2024.arabicnlp-1.91/" doi: "10.18653/v1/2024.arabicnlp-1.91" arxiv: "2407.13603" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/lichouri-2024-dzstance-stanceeval2024-arabic-stance-detection/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lichouri-2024-dzstance-stanceeval2024-arabic-stance-detection/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/) / dzStance at StanceEval2024: Arabic Stance Detection based on Sentence Transformers # dzStance at StanceEval2024: Arabic Stance Detection based on Sentence Transformers **Authors:** [Mohamed Lichouri](https://md.research.elamur.ai/free/en/authors/mohamed-lichouri/), [Khaled Lounnas](https://md.research.elamur.ai/free/en/authors/khaled-lounnas/), Ouaras Rafik, Mohamed ABi, Anis Guechtouli · **Year:** 2024 · **Venue:** Proceedings of the Second Arabic Natural Language Processing Conference · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2024.arabicnlp-1.91/) · [PDF](https://aclanthology.org/2024.arabicnlp-1.91.pdf) · [DOI 10.18653/v1/2024.arabicnlp-1.91](https://doi.org/10.18653/v1/2024.arabicnlp-1.91) · [arXiv 2407.13603](https://arxiv.org/abs/2407.13603) · [OpenAlex](https://openalex.org/W4402683885) **Citations:** 1 citations (openalex-snapshot · 2026-10-14) · 1 in our library, of the 140884 papers whose bibliographies we read ## Summary The paper addresses Arabic stance detection for the StanceEval 2024 shared task. Given a tweet and one of three targets (COVID-19 vaccine, digital transformation, women empowerment), the system must decide whether the writer is in favour, against, or neutral. Team dzStance used the Mawqif dataset of 3,502 tweets, whose labels lean heavily toward the favourable class, especially for digital transformation. They compared two lightweight pipelines: - **TF-IDF pipeline:** several TF-IDF n-gram vectorizers merged with scikit-learn's FeatureUnion, with grid-tuned block weights, feeding an LSVC. - **Embedding pipeline:** each tweet, after Arabic normalization and emoji replacement, is encoded with the frozen multilingual Sentence Transformers model xlm-r-bert-base-nli-stsb-mean-tokens, followed by a multinomial Logistic Regression. On the development set, the embedding pipeline won (68.48 vs 66.20 F1). On the official test it reached 71.77 F1 overall, placing 13th of 15 teams. The work matters less as a state-of-the-art result than as a cheap, reproducible baseline with public code. It shows that frozen sentence embeddings already beat carefully tuned sparse n-gram features. ## Key points - **Mawqif data:** 1167 / 1145 / 1190 tweets per target, with 2154 Favor, 1020 Against and 332 None in total. Digital transformation has only 22 None examples against 879 Favor. The prose also mentions "over 4,000" samples, which conflicts with the table. - **TF-IDF setup:** n-gram ranges from (1,1) to (1,10) were tested, with block weights swept from 0.1 to 1.0. The best weights, tw1 = (0.85, 0.85, 0.65), were used with LSVC at C=4. The peak was 66.20 F1 at ngram_range=(1,6), with a slight drop for longer ranges. - **Embedding setup:** the sentence embeddings were classified by LR with max_iter=1000, multi_class='multinomial' and solver='lbfgs', plus normalize_arabic and replace_emojis. This gave 68.48 development F1. - **Official test results:** | Target | F1 | Rank | |---|---|---| | Women Empowerment | 74.91 | 13th | | COVID-19 Vaccine | 73.43 | 10th | | Digital Transformation | 66.97 | 12th | | Overall | 71.77 | 13th of 15 | - **Caveats:** - The text cites a unigram baseline of 64.34, while the table lists 61.15 for (1,1). - The table scores do not rise steadily with n-gram length, despite what the prose says. - No ablation separates the effect of preprocessing from that of the embeddings. - Code is released at github.com/licvol/dzStanceEval_2024. ## Interesting ideas - Several parallel feature views are combined with separately tuned weights rather than being simply concatenated, and the weights are found by a small grid search on held-out data. — For Mova (learn): when merging silver trees from several witness parsers, per-witness vote weights could be tuned on a dev set instead of using equal-weight agreement. - A frozen pretrained sentence encoder plus a plain linear classifier, with no fine-tuning, is a cheap but strong reference point that is easy to reproduce. — For Mova (learn): such an opaque embedding+LR probe could serve as a control baseline, so that any explainable frame-based reader is required to match or beat it on the same task before claiming progress. - Reporting the label distribution separately for each target exposes extreme skews, such as 22 neutral items out of 1145, which can hide behind an overall score. — For Mova (answer): school-test and fable-QA evaluations could be broken down by answer type, so a reader that defaults to the majority answer is caught early. - Light, script-aware normalization and mapping emojis to text before encoding were part of the best configuration, so cheap input cleanup is treated as part of the model. — For Mova (tokens): the tokenizer could rewrite emojis and similar symbols into word-like tokens with a lemma, so the tagger and later stages can attach them to the tree instead of dropping them. - Stance is defined relative to an explicitly given target, not as free-floating sentiment, so the same text can be scored differently for different targets. — For Mova (frames): prag could record stance as a frame linking the author, a target entity and a polarity, which makes "who is for or against what" an explainable slot rather than a document label. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{lichouri2024dzstance, title = {{dzStance at StanceEval2024: Arabic Stance Detection based on Sentence Transformers}}, author = {Lichouri, Mohamed and Lounnas, Khaled and Rafik, Ouaras and ABi, Mohamed and Guechtouli, Anis}, booktitle = {Proceedings of the Second Arabic Natural Language Processing Conference}, year = {2024}, doi = {10.18653/v1/2024.arabicnlp-1.91}, eprint = {2407.13603}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/2024.arabicnlp-1.91} } ``` - [← previous DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads](https://md.research.elamur.ai/free/en/papers/xiao-2024-duoattention-efficient-long-context-llm/) - [next → EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://md.research.elamur.ai/free/en/papers/li-2024-eagle-2-faster-inference-language/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/) ## References 10 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Mourad Abbas, Mohamed Lichouri, and Abed Alhakim Freihat. 2019. St madar 2019 shared task: Arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 269–273. [find](https://scholar.google.com/scholar?q=Mourad+Abbas%2C+Mohamed+Lichouri%2C+and+Abed+Alhakim+Freihat%2E+2019%2E+St+madar+2019+shared+task%3A+Arabic+fine%2Dgrained+dialect+identification%2E+In+Proceedings+of+the+Fourth+Arabic+Natural+Language+Processing+W) 2. Abeer AlDayel and Walid Magdy. 2021. Stance detection on social media: State of the art and trends. Information Processing & Management, 58(4):102597. [find](https://scholar.google.com/scholar?q=Abeer+AlDayel+and+Walid+Magdy%2E+2021%2E+Stance+detection+on+social+media%3A+State+of+the+art+and+trends%2E+Information+Processing+%26+Management%2C+58%284%29%3A102597%2E) 3. Nora Alturayeif, Hamzah Luqman, and Moataz Ahmed. 2023. A systematic review of machine learning techniques for stance detection and its applications. Neural Computing and Applications, 35(7):5113–5144. [find](https://scholar.google.com/scholar?q=Nora+Alturayeif%2C+Hamzah+Luqman%2C+and+Moataz+Ahmed%2E+2023%2E+A+systematic+review+of+machine+learning+techniques+for+stance+detection+and+its+applications%2E+Neural+Computing+and+Applications%2C+35%287%29%3A5113%E2%80%935144) 4. Nora Alturayeif, Hamzah Luqman, Zaid Alyafeai, and Asma Yamani. 2024. Stanceeval 2024: The first arabic stance detection shared task. In Proceedings of The Second Arabic Natural Language Processing Conference (ArabicNLP 2024). [find](https://scholar.google.com/scholar?q=Nora+Alturayeif%2C+Hamzah+Luqman%2C+Zaid+Alyafeai%2C+and+Asma+Yamani%2E+2024%2E+Stanceeval+2024%3A+The+first+arabic+stance+detection+shared+task%2E+In+Proceedings+of+The+Second+Arabic+Natural+Language+Processing+Co) [our card](https://md.research.elamur.ai/free/en/papers/alturayeif-2024-stanceeval-2024-first-arabic-stance/) 5. Nora Saleh Alturayeif, Hamzah Abdullah Luqman, and Moataz Aly Kamaleldin Ahmed. 2022. Mawqif: A multi-label arabic dataset for target-specific stance detection. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 174–184. [find](https://scholar.google.com/scholar?q=Nora+Saleh+Alturayeif%2C+Hamzah+Abdullah+Luqman%2C+and+Moataz+Aly+Kamaleldin+Ahmed%2E+2022%2E+Mawqif%3A+A+multi%2Dlabel+arabic+dataset+for+target%2Dspecific+stance+detection%2E+In+Proceedings+of+the+The+Seventh+Arabi) 6. Mirko Lai, Alessandra Teresa Cignarella, Delia Irazú Hernández Farías, Cristina Bosco, Viviana Patti, and Paolo Rosso. 2020. Multilingual stance detection in social media political debates. Computer Speech & Language, 63:101075. [find](https://scholar.google.com/scholar?q=Mirko+Lai%2C+Alessandra+Teresa+Cignarella%2C+Delia+Iraz%C3%BA+Hern%C3%A1ndez+Far%C3%ADas%2C+Cristina+Bosco%2C+Viviana+Patti%2C+and+Paolo+Rosso%2E+2020%2E+Multilingual+stance+detection+in+social+media+political+debates%2E+Computer+S) 7. Mohamed Lichouri and Mourad Abbas. 2020. Speech-trans@ smm4h’20: Impact of preprocessing and ngrams on automatic classification of tweets that mention medications. In Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task, pages 118–120. [find](https://scholar.google.com/scholar?q=Mohamed+Lichouri+and+Mourad+Abbas%2E+2020%2E+Speech%2Dtrans%40+smm4h%E2%80%9920%3A+Impact+of+preprocessing+and+ngrams+on+automatic+classification+of+tweets+that+mention+medications%2E+In+Proceedings+of+the+Fifth+Social+M) 8. Mohamed Lichouri, Mourad Abbas, Khaled Lounnas, Besma Benaziz, and Aicha Zitouni. 2021. Arabic dialect identification based on a weighted concatenation of TF-IDF features. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 282–286, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mohamed+Lichouri%2C+Mourad+Abbas%2C+Khaled+Lounnas%2C+Besma+Benaziz%2C+and+Aicha+Zitouni%2E+2021%2E+Arabic+dialect+identification+based+on+a+weighted+concatenation+of+TF%2DIDF+features%2E+In+Proceedings+of+the+Sixth+) 9. Mohamed Lichouri, Khaled Lounnas, Aicha Zitouni, Houda Latrache, and Rachida Djeradi. 2023. Usthb at nadi 2023 shared task: Exploring preprocessing and feature engineering strategies for arabic dialect identification. arXiv preprint arXiv:2312.10536. [↗](https://arxiv.org/abs/2312.10536) 10. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2019%2E+Sentence%2Dbert%3A+Sentence+embeddings+using+siamese+bert%2Dnetworks%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E+Associ)