--- title: "SemanticCuetSync at ArAIEval Shared Task: Detecting" description: "This paper describes a system for Task-1 of the ArAIEval 2024 shared task. 4 references." authors: ["Symom Shohan", "Md. Hossain", "Ashraful Paran", "Shawly Ahsan", "Jawad Hossain", "Mohammed Moshiul Hoque"] year: 2024 venue: "Proceedings of the Second Arabic Natural Language Processing Conference" source: "https://aclanthology.org/2024.arabicnlp-1.54/" doi: "10.18653/v1/2024.arabicnlp-1.54" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/shohan-2024-semanticcuetsync-araieval-shared-task-detecting/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shohan-2024-semanticcuetsync-araieval-shared-task-detecting/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/) / SemanticCuetSync at ArAIEval Shared Task: Detecting Propagandistic Spans with Persuasion Techniques Identification using Pre-trained Transformers # SemanticCuetSync at ArAIEval Shared Task: Detecting Propagandistic Spans with Persuasion Techniques Identification using Pre-trained Transformers **Authors:** [Symom Shohan](https://md.research.elamur.ai/free/en/authors/symom-shohan/), [Md. Hossain](https://md.research.elamur.ai/free/en/authors/md-hossain/), [Ashraful Paran](https://md.research.elamur.ai/free/en/authors/ashraful-paran/), [Shawly Ahsan](https://md.research.elamur.ai/free/en/authors/shawly-ahsan/), [Jawad Hossain](https://md.research.elamur.ai/free/en/authors/jawad-hossain/) and [Mohammed Moshiul Hoque](https://md.research.elamur.ai/free/en/authors/mohammed-moshiul-hoque/) · **Year:** 2024 · **Venue:** [Proceedings of the Second Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2024](https://md.research.elamur.ai/free/en/venues/arabicnlp/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2024.arabicnlp-1.54/) · [PDF](https://aclanthology.org/2024.arabicnlp-1.54.pdf) · [DOI 10.18653/v1/2024.arabicnlp-1.54](https://doi.org/10.18653/v1/2024.arabicnlp-1.54) · [OpenAlex](https://openalex.org/W4402671295) **Citations:** 1 citations (OpenAlex, 2026-10-14) ## Summary This paper describes a system for Task-1 of the ArAIEval 2024 shared task. The task asks a model to find the stretches of Arabic text that carry propaganda and to name the persuasion technique behind each one. The data are Arabic tweets and paragraphs from news articles, labelled with 23 technique classes. The authors frame the problem as token-level sequence labelling with BIO tags. They compare three families of models: classical classifiers over BoW/TF-IDF features, convolutional and recurrent networks, and fine-tuned Arabic BERT variants with a token-classification head. CamelBERT gives the best micro-F1 at 24.09%, slightly ahead of AraBERTv2, and the submission ranked 6th. Error analysis shows three problems. The dominant Loaded-Language class absorbs many predictions. Name-Calling-Labeling is often confused with it. Long texts containing several spans are especially hard, and the model sometimes picks the right technique but marks the wrong boundaries. The paper is useful as a baseline sweep for fine-grained, span-level persuasion analysis in Arabic. ## Key points - The dataset (Hasanain et al., 2024b) splits into 6997 train, 921 dev and 1046 test items, combining tweets and news paragraphs, with 23 technique labels. Fourteen of those labels never occur in the test set. - Models compared: - LR, RF and MNB over CountVectorizer/TF-IDF features - CNN, CNN+LSTM and CNN+BiLSTM - AraBERTv2, AraBERT-NER, CamelBERT and BERT-Base-Arabic - Micro-F1 on the test set: - CamelBERT: 24.09 - AraBERTv2: 23.84 - BERT-Base-Arabic: 20.31 - AraBERT-NER: about 7.7 - Best classical model (LR): 11.76 - CNN: 11.26 - The non-transformer models show recall near 60% but precision around 6%, and their macro-F1 is close to zero. AraBERTv2 is the most precise model (34.81 P), but its recall is lower than CamelBERT's. - Confusion counts: Loaded-Language is right 1,680 times and wrong 1,067 times, while Name-Calling-Labeling is right only 50 times against 254 errors. ## Interesting ideas - Span finding and span typing are merged into one tagging problem: each token gets a technique-specific B/I tag or O, and the added [CLS]/[SEP] tokens receive the label −100 so the loss skips them. This avoids a separate boundary detector. - Reporting precision and recall next to both micro- and macro-F1 exposes degenerate systems. The classical baselines reach about 60% recall only by over-predicting, and their near-zero macro-F1 confirms they learned almost nothing. - The error analysis separates "technique correct, span wrong" from "both wrong." This shows whether a failure lies in labelling or in boundary placement. - Checking which classes actually occur in the test split showed that 14 of 23 never appear, which makes macro scores hard to interpret. - The authors observe that two neighbouring categories (Name-Calling-Labeling vs. Loaded-Language) absorb each other's predictions. This suggests checking confusion between closely related labels rather than relying only on aggregate scores. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (frames): spans of irony, implicature or speech-act cues could be tagged token by token in the same way and then attached to the UD subtree they cover, keeping the evidence traceable. - **I2** (learn): every evaluation could include a "predict-everything" or majority-class control, so a model is flagged when its scores match that trivial baseline. - **I3** (answer): answer checking could score the chosen entity or frame type separately from the exact text extent quoted, so readers get partial-credit diagnostics. - **I4** (learn): before trusting a benchmark, list the label coverage of each split and report per-class results only for classes that are present. - **I5** (world): when compiling verb classes or ontology categories, a confusion check between sibling classes could reveal where definitions overlap and need sharper distinguishing features. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Chittagong University of Engineering & Technology](https://md.research.elamur.ai/free/en/universities/chittagong-university-of-engineering-technology/) — University, Chittagong, Bangladesh · 25 works · 29 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works ## Cite Cite as: Symom Shohan, Md. Hossain, Ashraful Paran, Shawly Ahsan, Jawad Hossain and Mohammed Moshiul Hoque. 2024. SemanticCuetSync at ArAIEval Shared Task: Detecting Propagandistic Spans with Persuasion Techniques Identification using Pre-trained Transformers. Proceedings of the Second Arabic Natural Language Processing Conference. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{shohan2024semanticcuetsync, title = {{SemanticCuetSync at ArAIEval Shared Task: Detecting Propagandistic Spans with Persuasion Techniques Identification using Pre-trained Transformers}}, author = {Symom Shohan, Md. Hossain, Ashraful Paran, Shawly Ahsan, Jawad Hossain and Mohammed Moshiul Hoque}, booktitle = {Proceedings of the Second Arabic Natural Language Processing Conference}, year = {2024}, doi = {10.18653/v1/2024.arabicnlp-1.54}, url = {https://doi.org/10.18653/v1/2024.arabicnlp-1.54} } ``` - [← previous Research Progress of Pre-trained Model in Software Engineering](https://md.research.elamur.ai/free/en/papers/gong-2024-pre-trained-model/) - [next → Social ties and negotiation of lexical norms in Old English: The vocabularies of vices](https://md.research.elamur.ai/free/en/papers/timofeeva-2024-social-ties-negotiation-lexical-norms/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/) ## References 4 references, 4 of them with a card of ours. ↗ leads to the work. 1. Firoj Alam, Hamdy Mubarak, Wajdi Zaghouani, Giovanni Da San Martino, and Preslav Nakov. 2022. [Overview of the WANLP 2022 shared task on propaganda detection in Arabic](https://md.research.elamur.ai/free/en/papers/alam-2022-overview-wanlp-2022-shared-task/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 108–118, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. 0 4 Questioning-the-Reputation ¼ð@ (Okay) 2. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). arXiv preprint arXiv:2003.00104. [↗](https://arxiv.org/abs/2003.00104) 3. Joseph Attieh and Fadi Hassan. 2022. [Pythoneers at wanlp 2022 shared task: Monolingual arabert for arabic propaganda detection and span extraction](https://md.research.elamur.ai/free/en/papers/attieh-2022-pythoneers-wanlp-2022-shared-task/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 534– 540. 229 232 Loaded-Language Ðñj.ë (Attack) 4. Ramy Baly, Giovanni Da San Martino, James Glass, and Preslav Nakov. 2020. [We can detect your bias: Predicting the political ideology of news articles](https://md.research.elamur.ai/free/en/papers/baly-2020-we-can-detect-your-bias/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4982–4991. 192 195 *Table 6: Few predictions by CamelBERT.* *Updated 1 November 2026*