--- title: "ChavanKane at WANLP 2022 Shared Task: Large Language" description: "This paper presents one team's system for the WANLP 2022 shared task on detecting propaganda… 14 references, cited by 1 paper here." authors: ["Tanmay Chavan", "Aditya Manish Kane"] year: 2022 venue: "Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)" source: "https://aclanthology.org/2022.wanlp-1.60/" doi: "10.18653/v1/2022.wanlp-1.60" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/chavan-2022-chavankane-wanlp-2022-shared-task/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chavan-2022-chavankane-wanlp-2022-shared-task/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/7/) / ChavanKane at WANLP 2022 Shared Task: Large Language Models for Multi-label Propaganda Detection # ChavanKane at WANLP 2022 Shared Task: Large Language Models for Multi-label Propaganda Detection **Authors:** Tanmay Chavan and Aditya Manish Kane · **Year:** 2022 · **Venue:** [Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2022](https://md.research.elamur.ai/free/en/venues/wanlp/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2022.wanlp-1.60/) · [PDF](https://aclanthology.org/2022.wanlp-1.60.pdf) · [DOI 10.18653/v1/2022.wanlp-1.60](https://doi.org/10.18653/v1/2022.wanlp-1.60) · [OpenAlex](https://openalex.org/W4385567020) **Citations:** 3 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/chavan-2022-chavankane-wanlp-2022-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary This paper presents one team's system for the WANLP 2022 shared task on detecting propaganda techniques in Arabic tweets. The task is multi-label: a single tweet may use several persuasion techniques at once. The abstract says there are 21 candidate techniques, while the data section says 20. The training data is very small, with only a few hundred tweets, and both the labels and the number of labels per tweet are heavily skewed. The authors fine-tuned several pretrained Arabic and multilingual transformers using a binary cross-entropy objective. The models were AraBERT variants, ARBERT, MARBERT, XLM-RoBERTa, AraELECTRA and an Arabic-tuned DeHateBERT. They then merged five of these models by majority voting, applied separately to each label. This ensemble scored best, with a micro-F1 of about 59.7. Two other attempts did not help: oversampling rare classes, and continuing masked-language-model pretraining on one million sentiment tweets. The authors suggest the ensemble works because its members were pretrained on different corpora. The work shows that combining diverse pretrained models is a simple, robust baseline for low-resource multi-label classification. ## Key points - **Data splits:** - 504 training, 52 validation and 52 development-test tweets. - The official evaluation used a separate set of 323 tweets. - Most tweets carry a single label; one tweet has 7. - **Preprocessing:** - Links, @-mentions and #-hashtags are removed. - Underscores become spaces. - Emojis are kept on purpose. - **Training:** - 30 epochs at a learning rate of 3e-6, with binary cross-entropy over multi-hot targets. - The best epoch is picked by validation micro-F1. - The DeHateBERT variant starts from multilingual BERT and was fine-tuned only on Arabic data. - **Micro-F1 results:** | System | Micro-F1 | |---|---| | Ensemble (DeHateBERT, ARBERT, AraBERTv02, AraBERTv01, MARBERT) | 59.725 | | ARBERT | 59.048 | | MARBERT | 56.556 | | DeHateBERT | 56.484 | | AraBERTv01 | 54.195 | | DeHateBERT with oversampling | 52.529 | | AraBERTv2 | 50.841 | - **Negative results:** - Continuing MLM pretraining of DeHateBERT on 1M tweets did not improve results. The tweets came from a large Arabic Twitter corpus built for sentiment analysis. - The authors blame the small size of this corpus compared with the original pretraining data. - Oversampling also lowered the score. ## Interesting ideas - In the multi-label ensemble, each label is decided by its own majority vote among the models, so a label survives only if most members propose it; this naturally limits false positives. - The authors credit ensemble gains to members pretrained on different corpora, so witness diversity matters more than the strength of any single model. - Rare-class oversampling here repeats each example by a factor based on the class average, caps that factor at 10, and skips examples that also carry frequent labels. - Hashtag and handle cleanup turns underscores into spaces so that joined words count as ordinary words, while emojis are kept as meaningful signals. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Silver data from witness parsers could be accepted per arc and per label by a majority of witnesses rather than per whole tree, keeping the agreed parts of partially disputed sentences. - **I2** (learn): When choosing witness parsers or LLM teacher tiers, prefer ones trained on different treebanks or domains, so that their agreement is real evidence rather than a shared bias. - **I3** (frames): Rare frame labels such as irony or uncommon speech acts could be balanced this way, with the frequent-label exclusion guarding against simply inflating common classes. - **I4** (tokens): The tokenizer could split underscore- or hashtag-compounds into separate tokens and keep emojis as their own tokens instead of discarding them as noise. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works ## Cite Cite as: Tanmay Chavan and Aditya Manish Kane. 2022. ChavanKane at WANLP 2022 Shared Task: Large Language Models for Multi-label Propaganda Detection. Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{chavan2022chavankane, title = {{ChavanKane at WANLP 2022 Shared Task: Large Language Models for Multi-label Propaganda Detection}}, author = {Tanmay Chavan and Aditya Manish Kane}, booktitle = {Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)}, year = {2022}, doi = {10.18653/v1/2022.wanlp-1.60}, url = {https://doi.org/10.18653/v1/2022.wanlp-1.60} } ``` - [← previous Automatically Detecting Reduced-formed English Pronunciations by Using Deep Learning](https://md.research.elamur.ai/free/en/papers/chen-2022-automatically-detecting-reduced-formed-english/) - [next → Comparing Open Arabic Named Entity Recognition Tools](https://md.research.elamur.ai/free/en/papers/aldumaykhi-2022-comparing-open-arabic-named-entity/) - [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/7/) ## References 14 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.acl-long.551) 2. Firoj Alam, Hamdy Mubarak, Wajdi Zaghouani, Preslav Nakov, and Giovanni Da San Martino. 2022. [Overview of the WANLP 2022 shared task on propaganda detection in Arabic](https://md.research.elamur.ai/free/en/papers/alam-2022-overview-wanlp-2022-shared-task/). In Proceedings of the Seventh Arabic Natural Language Processing Workshop, Abu Dhabi, UAE. Association for Computational Linguistics. 3. Sai Saketh Aluru, Binny Mathew, Punyajoy Saha, and Animesh Mukherjee. 2020. Deep learning models for multilingual hate speech detection. CoRR, abs/2004.06465. [↗](https://arxiv.org/abs/2004.06465) 4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104) 5. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. [AraELECTRA: Pre-training text discriminators for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2021-araelectra-pre-training-text-discriminators/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 191–195, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2012.15516) 6. Oliver J. Brady. 2021. Aitbert : Domain specific pre-training on alternative social media to improve hate speech classification. [find](https://scholar.google.com/scholar?q=Oliver+J%2E+Brady%2E+2021%2E+Aitbert+%3A+Domain+specific+pre%2Dtraining+on+alternative+social+media+to+improve+hate+speech+classification%2E) 7. Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, and Preslav Nakov. 2019. Fine-grained analysis of propaganda in news article. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5636–5646, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Giovanni+Da+San+Martino%2C+Seunghak+Yu%2C+Alberto+Barr%C3%B3n%2DCede%C3%B1o%2C+Rostislav+Petrov%2C+and+Preslav+Nakov%2E+2019%2E+Fine%2Dgrained+analysis+of+propaganda+in+news+article%2E+In+Proceedings+of+the+2019+Conference+on+Em) 8. Kareem Darwish and Hamdy Mubarak. 2016. [Farasa: A new fast and accurate Arabic word segmenter](https://md.research.elamur.ai/free/en/papers/darwish-2016-farasa-new-fast-accurate-arabic/). In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1070–1074, Portorož, Slovenia. European Language Resources Association (ELRA). 9. Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, and Giovanni Da San Martino. 2021. [SemEval-2021 task 6: Detection of persuasion techniques in texts and images](https://md.research.elamur.ai/free/en/papers/dimitrov-2021-semeval-2021-task-6-detection/). In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 70–98, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.semeval-1.7) 10. Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heinz, and Dan Roth. 2021. [Recent advances in natural language processing via large pre-trained language models: A survey](https://md.research.elamur.ai/free/en/papers/min-2021-recent-advances-natural-language-processing/). 11. Seunghak Yu, Giovanni Da San Martino, Mitra Mohtarami, James Glass, and Preslav Nakov. 2021. Interpretable propaganda detection in news articles. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1597–1605, Held Online. IN-COMA Ltd. [↗](https://doi.org/10.48550/arxiv.2108.12802) 12. Matteo Cinelli, Walter Quattrociocchi, Alessandro Galeazzi, Carlo Michele Valensise, Emanuele Brugnoli, Ana Lucia Schmidt, Paola Zola, Fabiana Zollo, and Antonio Scala. 2020. The COVID-19 social media infodemic. Scientific Reports, 10(1). [↗](https://doi.org/10.1038/s41598-020-73510-5) 13. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.747) 14. Giovanni Da San Martino, Alberto Barrón-Cedeño, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov. 2020. [SemEval-2020 task 11: Detection of propaganda techniques in news articles](https://md.research.elamur.ai/free/en/papers/martino-2020-semeval-2020-task-11-detection/). In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1377–1414, Barcelona (online). International Committee for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2009.02696) *Updated 1 November 2026*