--- title: "MultiProSE: A Multi-label Arabic Dataset for Propaganda, Sentiment, and Emotion — Al-Henaki 2025" description: "Arabic has few resources for detecting propaganda in text. 1 reference, cited by 2 papers here." authors: ["Lubna Al-Henaki", "Hend Al-Khalifa", "Abdulmalik Al-Salman", "Hajar Alqubayshi", "Hind Al-Twailay", "Gheeda Alghamdi", "Hawra Aljasim"] year: 2025 venue: "arXiv:2502.08319" source: "https://arxiv.org/abs/2502.08319" doi: "10.48550/arxiv.2502.08319" arxiv: "2502.08319" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/al-henaki-2025-multiprose-multi-label-arabic-dataset/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/al-henaki-2025-multiprose-multi-label-arabic-dataset/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / MultiProSE: A Multi-label Arabic Dataset for Propaganda, Sentiment, and Emotion Detection # MultiProSE: A Multi-label Arabic Dataset for Propaganda, Sentiment, and Emotion Detection **Authors:** Lubna Al-Henaki, [Hend Al-Khalifa](https://md.research.elamur.ai/free/en/authors/hend-al-khalifa/), [Abdulmalik Al-Salman](https://md.research.elamur.ai/free/en/authors/abdulmalik-al-salman/), Hajar Alqubayshi, Hind Al-Twailay, Gheeda Alghamdi and Hawra Aljasim · **Year:** 2025 · **Venue:** arXiv:2502.08319 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2502.08319) · [PDF](https://arxiv.org/pdf/2502.08319) · [DOI 10.48550/arxiv.2502.08319](https://doi.org/10.48550/arxiv.2502.08319) · [arXiv 2502.08319](https://arxiv.org/abs/2502.08319) · [OpenAlex](https://openalex.org/W4411811287) **Citations:** 3 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 140,884 papers whose bibliographies we read ## Summary Arabic has few resources for detecting propaganda in text. Existing datasets also rarely record the emotional and evaluative tone that persuasive writing tends to carry. MultiProSE addresses both gaps. It takes ArPro, an existing benchmark of 8,000 Modern Standard Arabic news paragraphs with binary propaganda labels, and adds human labels for sentiment and emotion to every item. The result is a multi-label resource covering three tasks at once. The authors wrote bilingual annotation guidelines, had them checked by a rhetoric expert and a linguist, and trained the annotators before labelling began. They then built baselines with GPT-4o-mini and BERT-family encoders (AraBERT, ArabicBERT, XLM-RoBERTa). GPT-4o-mini performed best overall, especially on sentiment. Arabic-specific encoders came close, and the multilingual XLM-RoBERTa was consistently last. Emotion recognition proved the hardest of the three tasks. The dataset, guidelines and code are released publicly. This lets researchers study how propaganda relates to affect, not just train separate classifiers for each label. ## Key points - **Data and splits:** 8,000 paragraphs inherited from ArPro, split 6002 / 672 / 1326 for train / dev / test. About 63% are marked propagandistic, and the texts span 14 topics, with news and politics making up over half. - **Label scheme:** - Sentiment is three-way: positive, negative or neutral. - Emotion starts from Ekman's model. It drops disgust because annotators confused it with anger, and adds an explicit "none" label. The final set is happiness, sadness, anger, fear and none. - **Baseline results:** - On sentiment, GPT-4o-mini reached 0.842 Micro-F1, AraBERT was about 0.002 behind, and XLM-RoBERTa scored 0.698. - On emotion, GPT-4o-mini reached 0.750 Micro-F1 and AraBERT 0.675. - **Fine-tuning setup:** - Preprocessing: character normalization plus removal of diacritics, stopwords and tatweel. - Hyperparameters: maximum length 256 tokens, batch size 8, 5 epochs, AdamW with learning rate 2e-5. - **Annotation logistics:** - Annotation took four months, with a minimum quota of 100 texts per annotator per day. - Reported difficulties: missing diacritics, dialect inside quotations, and typos. ## Interesting ideas - Instead of collecting new text, the authors added fresh annotation layers to a corpus that was already labelled, so every dimension lines up on the same items and interactions can be studied directly. - The label inventory was pruned where humans kept confusing two classes (disgust vs anger), and an explicit "no emotion" class was added so annotators are never forced to pick one. - The paper observes that sentiment and emotion labels are correlated, which means one annotation dimension can be checked against another. - Annotators resolved words made ambiguous by missing diacritics by reading the wider context or the original news source, rather than guessing from the word alone. - The authors benchmarked a prompted LLM against fine-tuned encoders task by task and found the gap differs: small on sentiment, larger on emotion. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (frames): speech-act and irony labels could be added onto sentences that already have UD gold trees, giving aligned tree-plus-pragmatics data without new collection. - **I2** (frames): the speech-act and implicature inventories could merge classes that witnesses systematically confuse and include an explicit null option, making outputs more honest and explainable. - **I3** (learn): cross-layer agreement (e.g., frame polarity vs lexical sentiment) could serve as a filter for silver data or as a sanity check during evaluation. - **I4** (tokens): when morphology yields several equally plausible analyses, the tagger could keep them open and defer the choice to later context instead of committing early. - **I5** (learn): when choosing LLM teacher tiers, measure the teacher's advantage separately per subtask and trust it only where it clearly beats cheaper trained models. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) ## Institutions - [King Saud University](https://md.research.elamur.ai/free/en/universities/king-saud-university/) — University, Riyadh, Saudi Arabia · 13 works · 290 citations - [Alfaisal University](https://md.research.elamur.ai/free/en/universities/alfaisal-university/) — University, Riyadh, Saudi Arabia · 3 works · 21 citations - [Imam Mohammad ibn Saud Islamic University](https://md.research.elamur.ai/free/en/universities/imam-mohammad-ibn-saud-islamic-university/) — University, Riyadh, Saudi Arabia · 6 works · 12 citations Also on the byline: [Majmaah University](https://ror.org/01mcrnj60). ## Cite Cite as: Lubna Al-Henaki, Hend Al-Khalifa, Abdulmalik Al-Salman, Hajar Alqubayshi, Hind Al-Twailay, Gheeda Alghamdi and Hawra Aljasim. 2025. MultiProSE: A Multi-label Arabic Dataset for Propaganda, Sentiment, and Emotion Detection. arXiv:2502.08319. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{alhenaki2025multiprose, title = {{MultiProSE: A Multi-label Arabic Dataset for Propaganda, Sentiment, and Emotion Detection}}, author = {Lubna Al-Henaki, Hend Al-Khalifa, Abdulmalik Al-Salman, Hajar Alqubayshi, Hind Al-Twailay, Gheeda Alghamdi and Hawra Aljasim}, year = {2025}, doi = {10.48550/arxiv.2502.08319}, eprint = {2502.08319}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2502.08319} } ``` - [← previous MizanQA: Benchmarking Large Language Models on Moroccan Legal Question Answering](https://md.research.elamur.ai/free/en/papers/bahaj-2025-mizanqa-benchmarking-large-language-models/) - [next → Parsing the Switch: LLM-Based UD Annotation for Complex Code-Switched and Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/kellert-2025-parsing-switch-llm-based-ud/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) ## References 1 references, 1 of them with a card of ours. ↗ leads to the work. 1. “Digital Around the World,” DataReportal – Global Digital Insights. Accessed: Feb. 13, 2024. [Online]. Available: https://datareportal.com/global-digital-overview “PROPAGANDA | English meaning - Cambridge Dictionary.” Accessed: Feb. 13, 2024. [Online]. Available: https://dictionary.cambridge.org/dictionary/english/p ropaganda G. Jowett, V. O’Donnell, and G. Jowett, Propaganda & persuasion, 5th ed. Thousand Oaks, Calif: SAGE, 2012. G. Da San Martino, A. Barrón-Cedeño, and P. Nakov, “Findings of the NLP4IF-2019 Shared Task on Fine-Grained Propaganda Detection,” in Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, Hong Kong, China: Association for Computational Linguistics, 2019, pp. 162–170. doi: 10.18653/v1/D19-5024. G. Krishnamurthy, R. K. Gupta, and Y. Yang, “SocCogCom at SemEval-2020 Task 11: Characterizing and Detecting Propaganda Using Sentence-Level Emotional Salience Features,” in Proceedings of the Fourteenth Workshop on Semantic Evaluation, Barcelona (online): International Committee for Computational Linguistics, 2020, pp. 1793–1801. doi: 10.18653/v1/2020.semeval-1.235. L. Wang, X. Shen, G. de Melo, and G. Weikum, “Cross-Domain Learning for Classifying Propaganda in Online Contents,” Nov. 22, 2020, arXiv: arXiv:2011.06844. Accessed: Feb. 05, 2024. [Online]. Available: http://arxiv.org/abs/2011.06844 M. S. I. Malik, T. Imran, and J. Mona Mamdouh, “How to detect propaganda from social media? Exploitation of semantic and fine-tuned language models,” PeerJ Computer Science, vol. 9, Art.no. e1248, Feb. 2023, doi: 10.7717/peerj-cs.1248. Amazon Alexa AI, Seattle, WA, USA et al., “Interpretable Propaganda Detection in News Articles,” in Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing Methods and Applications, INCOMA Ltd. Shoumen, BULGARIA, 2021, pp. 1597–1605. doi: 10.26615/978-954-452-072-4_179. S. Lamsiyah, A. Mahdaouy, H. Alami, I. Berrada, and C. Schommer, “UL & UM6P at ArAIEval Shared Task: Transformer-based model for Persuasion Techniques and Disinformation detection in Arabic,” in Proceedings of ArabicNLP 2023, Singapore (Hybrid): Association for Computational Linguistics, 2023, pp. 558–564. doi: 10.18653/v1/2023.arabicnlp-1.55. [12] [13] [14] [15] [16] [17] [18] [19] [20] Qatar Computing Research Institute, HBKU, Qatar et al., “COVID-19 in Bulgarian Social Media: Factuality, Harmfulness, Propaganda, and Framing,” in Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing Methods and Applications, INCOMA Ltd. Shoumen, BULGARIA, 2021, pp. 997–1009. doi: 10.26615/978-954-452-072-4_113. “Top Ten Internet Languages in The World - Internet Statistics.” Accessed: Feb. 17, 2024. [Online]. Available: https://www.internetworldstats.com/stats7.htm#goog le_vignette H. Rashkin, E. Choi, J. Y. Jang, S. Volkova, and Y. Choi, “Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark: Association for Computational Linguistics, Sep. 2017, pp. 2931– 2937. doi: 10.18653/v1/D17-1317. A. Barrón-Cedeño, I. Jaradat, G. Da San Martino, and P. Nakov, “Proppy: Organizing the news based on their propagandistic content,” Information Processing & Management, vol. 56, no. 5, pp. 1849–1864, Sep. 2019, doi: 10.1016/j.ipm.2019.03.005. S. Kausar, B. Tahir, and M. A. Mehmood, “ProSOUL: A Framework to Identify Propaganda From Online Urdu Content,” IEEE Access, vol. 8, pp. 186039– 186054, 2020, doi: 10.1109/ACCESS.2020.3028131. D. Chaudhari, A. V. Pawar, and A. Barrón-Cedeño, “H-Prop and H-Prop-News: Computational Propaganda Datasets in Hindi,” Data, vol. 7, no. 3, Art.no. 29, Feb. 2022, doi: 10.3390/data7030029. G. Faye et al., “Exposing propaganda: an analysis of stylistic cues comparing human annotations and machine classification,” Feb. 26, 2024, arXiv: arXiv:2402.03780. Accessed: Sep. 10, 2024. [Online]. Available: http://arxiv.org/abs/2402.03780 G. Da San Martino, S. Yu, A. Barrón-Cedeño, R. Petrov, and P. Nakov, “Fine-Grained Analysis of Propaganda in News Article,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 5636– 5646. doi: 10.18653/v1/D19-1565. Natural Language Processing Centre Masaryk University, Faculty of Informatics, Brno, Czech Republic, V. Baisa, O. Herman, and A. Horák, “Benchmark Dataset for Propaganda Detection in Czech Newspaper Texts,” in Proceedings - Natural Language Processing in a Deep Learning World, Incoma Ltd., Shoumen, Bulgaria, Oct. 2019, pp. 77– 83. doi: 10.26615/978-954-452-056-4_010. G. Da San Martino, A. Barrón-Cedeño, H. Wachsmuth, R. Petrov, and P. Nakov, “SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles,” in Proceedings of the Fourteenth Workshop on Semantic Evaluation, Barcelona (online): International Committee for Computational Linguistics, Dec. 2020, pp. 1377–1414. doi: 10.18653/v1/2020.semeval-1.186. D. Dimitrov et al., “SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images,” in [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), Online: Association for Computational Linguistics, Aug. 2021, pp. 70–98. doi: 10.18653/v1/2021.semeval-1.7. F. Alam, H. Mubarak, W. Zaghouani, G. Da San Martino, and P. Nakov, “Overview of the WANLP 2022 Shared Task on Propaganda Detection in Arabic,” in Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), Abu Dhabi, United Arab Emirates (Hybrid): Association for Computational Linguistics, Dec. 2022, pp. 108–118. Accessed: Feb. 18, 2023. [Online]. Available: https://aclanthology.org/2022.wanlp-1.11 P. Vijayaraghavan and S. Vosoughi, “TWEETSPIN: Fine-grained Propaganda Detection in Social Media Using Multi-View Representations,” in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Seattle, United States: Association for Computational Linguistics, 2022, pp. 3433–3448. doi: 10.18653/v1/2022.naacl-main.251. P. N. Ahmad, Y. Liu, G. Ali, M. A. Wani, and M. ElAffendi, “Robust Benchmark for Propagandist Text Detection and Mining High-Quality Data,” Mathematics, vol. 11, no. 12, Art.no.. 2668, Jun. 2023, doi: 10.3390/math11122668. M. Hasanain et al., “ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text,” Nov. 06, 2023, arXiv: arXiv:2311.03179. Accessed: Feb. 05, 2024. [Online]. Available: http://arxiv.org/abs/2311.03179 J. Piskorski, N. Stefanovitch, G. Da San Martino, and P. Nakov, “SemEval-2023 Task 3: Detecting the Category, the Framing, and the Persuasion Techniques in Online News in a Multi-lingual Setup,” in Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), Toronto, Canada: Association for Computational Linguistics, 2023, pp. 2343–2361. doi: 10.18653/v1/2023.semeval-1.317. M. Hasanain, F. Ahmad, and F. Alam, “Can GPT-4 Identify Propaganda? Annotation and Detection of Propaganda Spans in News Articles”. arXiv preprint, arXiv:2402.17478, 2024. B. M. Almotairy, M. Abdullah, and D. H. Alahmadi, “Dataset for detecting and characterizing Arab computation propaganda on X,” Data in Brief, vol. 53, p. 110089, Apr. 2024, doi: 10.1016/j.dib.2024.110089. A. M. U. D. Khanday, M. A. Wani, S. T. Rabani, Q. R. Khan, and A. A. Abd El-Latif, “HAPI: An efficient Hybrid Feature Engineering-based Approach for Propaganda Identification in social media,” PLoS ONE, vol. 19, no. 7,Art.no. e0302583, Jul. 2024, doi: 10.1371/journal.pone.0302583. R.-C. Chang, C.-M. Lai, K.-L. Chang, and C.-H. Lin, “Dataset of Propaganda Techniques of the State-Sponsored Information Operation of the People’s Republic of China,” Jun. 14, 2021, arXiv: arXiv:2106.07544. Accessed: Sep. 10, 2024. [Online]. Available: http://arxiv.org/abs/2106.07544 Z. S. Ali, W. Mansour, T. Elsayed, and A. Al‑Ali, “AraFacts: The First Large Arabic Dataset of Naturally Occurring Claims”. in Proc. 6th Arabic Natural Language Processing Workshop, 2021. [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] A. M. Alayba, V. Palade, M. England, and R. Iqbal, “Arabic language sentiment analysis on health services,” in 2017 1st International Workshop on Arabic Script Analysis and Recognition (ASAR), Nancy, France: IEEE, Apr. 2017, pp. 114–118. doi: 10.1109/ASAR.2017.8067771. N. Al-Twairesh, “Sentiment Analysis of Twitter: A Study on the Saudi Community,” Ph.D. dissertation, King Saud Univ., Riyadh, Saudi Arabia, 2016. J. Pustejovsky and A. Stubbs, Natural language annotation for machine learning: a guide to corpus-building for applications, 1. ed. Beijing Köln: O’Reilly, 2013. S. Rosenthal, N. Farra, and P. Nakov, “SemEval-2017 Task 4: Sentiment Analysis in Twitter”. arXiv preprint, arXiv:1912.00741, 2019. S. Mohammad, F. Bravo-Marquez, M. Salameh, and S. Kiritchenko, “SemEval-2018 Task 1: Affect in Tweets,” in Proceedings of The 12th International Workshop on Semantic Evaluation, New Orleans, Louisiana: Association for Computational Linguistics, 2018, pp. 1–17. doi: 10.18653/v1/S18-1001. P. Ekman, “An argument for basic emotions,” Cognition & Emotion, vol. 6, no. 3–4, pp. 169–200, 1992, A. Al-Khatib and S. R. El-Beltagy, “Emotional Tone Detection in Arabic Tweets,” in Computational Linguistics and Intelligent Text Processing, A. Gelbukh, Ed., Lecture Notes in Computer Science, vol. 10762, Cham, Switzerland: Springer International Publishing, 2018, pp. 105–114, doi: 10.1007/978-3-319-77116-8_8. S. Poria, D. Hazarika, N. Majumder, G. Naik, E. Cambria, and R. Mihalcea, “MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy: Association for Computational Linguistics, 2019, pp. 527–536. doi: 10.18653/v1/P19-1050. C. Hutto and E. Gilbert, “VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text,” ICWSM, vol. 8, no. 1, pp. 216–225, May 2014, doi: 10.1609/icwsm.v8i1.14550. J. R. Landis and G. G. Koch, “The Measurement of Observer Agreement for Categorical Data,” Biometrics, vol. 33, no. 1, pp. 159–174, Mar. 1977, doi: 10.2307/2529310. R. Light, “Measures of response agreement for qualitative data: some generalizations and alternatives,” American Psychological Association, vol. 76, no. 5, pp. 365–377, 1971. J. Fleiss and M. Davies, “Measuring agreement for multinomial data,” Biometrics, vol. 38, no. 4, pp. 1047–1051, Dec. 1982 P. S. Bayerl and K. I. Paul, “What Determines Inter-Coder Agreement in Manual Annotations? A Meta-Analytic Investigation,” Computational Linguistics, vol. 37, no. 4, pp. 699–725, Dec. 2011, doi: 10.1162/COLI_a_00074. E. Ohman, “Emotion Annotation: Rethinking Emotion Categorization,” in DHN Post-Proceedings, 2020. “OpenAI Platform.” Accessed: Sep. 10, 2024. [Online]. Available: https://platform.openai.com [47] [48] [49] [ 50] S. González-Carvajal and E. C. Garrido-Merchán, “Comparing BERT against traditional machine learning text classification,” JCCE, vol. 2, no. 4, pp. 352–356, Apr. 2023, doi: 10.47852/bonviewJCCE3202838. W. Antoun, F. Baly, and H. Hajj, “AraBERT: Transformer-based Model for Arabic Language Understanding,” arXiv:2003.00104 [cs], Mar. 2021, Accessed: May 01, 2022. [Online]. Available: http://arxiv.org/abs/2003.00104 A. Conneau et al., “Unsupervised Cross-lingual Representation Learning at Scale,” Apr. 07, 2020, arXiv: arXiv:1911.02116. Accessed: Oct. 18, 2022. [Online]. Available: http://arxiv.org/abs/1911.02116 T. Wolf et al., “Transformers: State-of-the-Art Natural Language Processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online: Association for Computational Linguistics, 2020, pp. 38–45. doi: 10.18653/v1/2020.emnlp-demos.6. I. Loshchilov and F. Hutter, “Decoupled Weight Decay Regularization,” Jan. 04, 2019, arXiv: arXiv:1711.05101. Accessed: Sep. 10, 2024. [Online]. Available: http://arxiv.org/abs/1711.05101 [↗](https://doi.org/10.18653/v1/d19-5024) [our card](https://md.research.elamur.ai/free/en/papers/ekman-1992-argument-basic-emotions/)