---
title: "UL & UM6P at ArAIEval Shared Task: Transformer-based model"
description: "The paper describes a system submitted to the ArAIEval shared task at ArabicNLP 2023, which targets… 22 references, cited by 2 papers here."
authors: ["Salima Lamsiyah", "Abdelkader El Mahdaouy", "Hamza Alami", "Ismail Berrada", "Christoph Schommer"]
year: 2023
venue: "Proceedings of ArabicNLP 2023"
source: "https://aclanthology.org/2023.arabicnlp-1.55/"
doi: "10.18653/v1/2023.arabicnlp-1.55"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lamsiyah-2023-ul-um6p-araieval-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lamsiyah-2023-ul-um6p-araieval-shared-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/) / UL & UM6P at ArAIEval Shared Task: Transformer-based model for Persuasion Techniques and Disinformation detection in Arabic

# UL & UM6P at ArAIEval Shared Task: Transformer-based model for Persuasion Techniques and Disinformation detection in Arabic

**Authors:** [Salima Lamsiyah](https://md.research.elamur.ai/free/en/authors/salima-lamsiyah/), [Abdelkader El Mahdaouy](https://md.research.elamur.ai/free/en/authors/abdelkader-el-mahdaouy/), [Hamza Alami](https://md.research.elamur.ai/free/en/authors/hamza-alami/), [Ismail Berrada](https://md.research.elamur.ai/free/en/authors/ismail-berrada/) and [Christoph Schommer](https://md.research.elamur.ai/free/en/authors/christoph-schommer/) · **Year:** 2023 · **Venue:** [Proceedings of ArabicNLP 2023](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2023](https://md.research.elamur.ai/free/en/venues/arabicnlp/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.arabicnlp-1.55/) · [PDF](https://aclanthology.org/2023.arabicnlp-1.55.pdf) · [DOI 10.18653/v1/2023.arabicnlp-1.55](https://doi.org/10.18653/v1/2023.arabicnlp-1.55) · [OpenAlex](https://openalex.org/W4389518238)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/lamsiyah-2023-ul-um6p-araieval-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes a system submitted to the ArAIEval shared task at ArabicNLP 2023, which targets two kinds of manipulative content in Arabic. The first is persuasion techniques: Sub-Task 1A asks whether a text uses any technique (binary), and Sub-Task 1B asks which of 24 techniques it uses (multi-label). The second is disinformation in tweets: Sub-Task 2A is binary, and Sub-Task 2B is a four-way classification. The architecture is deliberately simple. A BERT-style Arabic encoder (AraBERT-large, ARBERTv2 or MARBERTv2) produces the pooled [CLS] vector, which goes through dropout and then a linear layer with softmax or sigmoid. Most of the experimentation concerns the loss function: cross-entropy, binary cross-entropy, an asymmetric multi-label loss, Focal Tversky loss, and a Mixup regulariser applied to pooled embeddings. The official micro-F1 scores were 0.7515, 0.5666, 0.904 and 0.8333 on the four sub-tasks, which placed 4th, 1st, 3rd and 2nd. The study is useful because it shows that, for Arabic social-media text, the choice of encoder and loss matters a great deal. The encoder pre-trained on in-domain data (MARBERTv2) was usually the strongest.

## Key points
- **Data splits:**
  - Sub-Tasks 1A and 1B share the same tweets and news paragraphs: 2427 train, 259 dev, 503 test.
  - Sub-Task 2A uses 14,147 / 2115 / 3729 tweets.
  - Sub-Task 2B uses 2656 / 397 / 876 tweets, labelled hate-speech, offensive, rumor or spam.
- **Losses tried per sub-task:**
  - Sub-Tasks 1A and 2A: CE, and CE combined with RegMixup.
  - Sub-Task 1B: BCE, ASL, and ASL combined with RegMixup.
  - Sub-Task 2B: CE and Focal Tversky.
- **Training setup:**
  - Up to 10 epochs, batch size 16, learning rate 1e-5, early-stopping patience of 3.
  - Weight decay of 1e-3, not applied to biases or LayerNorm.
  - Maximum sequence length of 128.
  - RegMix settings: α=20, p=0.2. ASL settings: γ−=4, γ+=1. Focal Tversky α=0.5.
- **Effect of ASL on Sub-Task 1B:** dev macro-F1 rose from roughly 0.11 with BCE to about 0.33 with ASL. The best test micro-F1 was 0.5766 (MARBERTv2 with ASL). The submitted run used AraBERT with ASL and scored 0.5666.
- **Dev gains that did not hold on test:** RegMixup and Focal Tversky sometimes raised dev scores but gave little or no improvement on test, and occasionally hurt. In several sub-tasks, the submitted configuration was not the one with the best test score.

## Interesting ideas
- Mixup normally needs inputs that can be interpolated, which raw text cannot. The authors instead blend the pooled sentence vectors and their labels, and add the loss on these mixed examples as a weighted regulariser on top of the ordinary loss. This makes mixup usable for any text encoder.
- For multi-label output with many rare labels, an asymmetric loss down-weights the many easy negatives far more than the positives. This roughly tripled macro-F1 over plain BCE on the 24-technique task.
- Reporting micro-F1 and macro-F1 side by side exposed systems that look reasonable on micro-F1 (about 0.6) while almost completely failing on rare classes (macro-F1 about 0.1).
- Losses that improved the dev set but not the test set show that small dev splits can mislead the choice of model and training objective.
- The encoder pre-trained on data close to the target domain (MARBERTv2, built for social-media Arabic) generally beat the alternatives.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): The same interpolation could be applied to gparse's contextual token vectors when training on small or silver treebanks, to make the parser less brittle.
- **I2** (frames): Speech-act and irony tagging in prag is a sparse multi-label problem, and ASL-style weighting could keep rare acts from being swamped by the "absent" decision.
- **I3** (learn): Every Mova evaluation report should include per-class or macro scores, so that rare semantic roles or rare dependency relations cannot hide behind a strong overall accuracy.
- **I4** (learn): Model selection should be checked against several held-out sets, with negative controls, before a configuration is adopted.
- **I5** (tree): When choosing or fine-tuning the encoder behind gparse, the closeness of its pre-training text to the target genre (fables, school texts) should be weighed alongside model size.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Université Mohammed VI Polytechnique](https://md.research.elamur.ai/free/en/universities/universite-mohammed-vi-polytechnique/) — University, Marrakesh, Morocco · 18 works · 171 citations
- [University of Luxembourg](https://md.research.elamur.ai/free/en/universities/university-of-luxembourg/) — University, Luxembourg, Luxembourg · 27 works · 337 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Salima Lamsiyah, Abdelkader El Mahdaouy, Hamza Alami, Ismail Berrada and Christoph Schommer. 2023. UL & UM6P at ArAIEval Shared Task: Transformer-based model for Persuasion Techniques and Disinformation detection in Arabic. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lamsiyah2023ul,
  title = {{UL \& UM6P at ArAIEval Shared Task: Transformer-based model for Persuasion Techniques and Disinformation detection in Arabic}},
  author = {Salima Lamsiyah, Abdelkader {El Mahdaouy}, Hamza Alami, Ismail Berrada and Christoph Schommer},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.55},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.55}
}
```

- [← previous The User-Aware Arabic Gender Rewriter](https://md.research.elamur.ai/free/en/papers/alhafni-2023-user-aware-arabic-gender-rewriter/)
- [next → Who is Speaking? Speaker-Aware Multiparty Dialogue Act Classification](https://md.research.elamur.ai/free/en/papers/qamar-2023-who-is-speaking-speaker-aware/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/)

## References

22 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. shared task on fine-grained propaganda detection. In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 162–170, Hong Kong, China. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2101.01785)
2. Nabila Abraham and Naimul Mefraz Khan. 2018. A novel focal tversky loss function with improved attention u-net for lesion segmentation. [↗](https://doi.org/10.48550/arxiv.1810.07842)
3. Giovanni Da San Martino, Alberto Barrón-Cedeño, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov. 2020. [SemEval-2020 task 11: Detection of propaganda techniques in news articles](https://md.research.elamur.ai/free/en/papers/martino-2020-semeval-2020-task-11-detection/). In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1377–1414, Barcelona (online). International Committee for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2009.02696)
4. Firoj Alam, Hamdy Mubarak, Wajdi Zaghouani, Giovanni Da San Martino, and Preslav Nakov. 2022. [Overview of the WANLP 2022 shared task on propaganda detection in Arabic](https://md.research.elamur.ai/free/en/papers/alam-2022-overview-wanlp-2022-shared-task/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 108–118, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
5. Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, and Giovanni Da San Martino. 2021. [SemEval-2021 task 6: Detection of persuasion techniques in texts and images](https://md.research.elamur.ai/free/en/papers/dimitrov-2021-semeval-2021-task-6-detection/). In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 70–98, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.semeval-1.7)
6. Firoj Alam, Shaden Shaar, Fahim Dalvi, Hassan Sajjad, Alex Nikolov, Hamdy Mubarak, Giovanni Da San Martino, Ahmed Abdelali, Nadir Durrani, Kareem Darwish, Abdulaziz Al-Homaid, Wajdi Zaghouani, Tommaso Caselli, Gijs Danoe, Friso Stolk, Britt Bruntink, and Preslav Nakov. 2021. Fighting the COVID-19 infodemic: Modeling the perspective of journalists, fact-checkers, social media platforms, policy makers, and the society. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 611–649, Punta Cana, Dominican Republic. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2005.00033)
7. AbdelRahim Elmadany, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2022. [Orca: A challenging benchmark for arabic language understanding](https://md.research.elamur.ai/free/en/papers/elmadany-2023-orca-challenging-benchmark-arabic-language/). arXiv preprint arXiv:2212.10758. [↗](https://arxiv.org/abs/2212.10758)
8. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104)
9. Alberto Barrón-Cedeno, Tamer Elsayed, Preslav Nakov, Giovanni Da San Martino, Maram Hasanain, Reem Suwaileh, Fatima Haouari, Nikolay Babulkov, Bayan Hamdan, Alex Nikolov, et al. 2020. Overview of checkthat! 2020: Automatic identification and verification of claims in social media. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 11th International Conference of the CLEF Association, CLEF 2020, Thessaloniki, Greece, September 22–25, 2020, Proceedings 11, pages 215–236. Springer. [find](https://scholar.google.com/scholar?q=Alberto+Barr%C3%B3n%2DCedeno%2C+Tamer+Elsayed%2C+Preslav+Nakov%2C+Giovanni+Da+San+Martino%2C+Maram+Hasanain%2C+Reem+Suwaileh%2C+Fatima+Haouari%2C+Nikolay+Babulkov%2C+Bayan+Hamdan%2C+Alex+Nikolov%2C+et+al%2E+2020%2E+Overview+of+chec)
10. Emanuel Ben-Baruch, Tal Ridnik, Nadav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, and Lihi Zelnik-Manor. 2020. Asymmetric loss for multi-label classification. [↗](https://doi.org/10.48550/arxiv.2009.14119)
11. Giovanni Da San Martino, Alberto Barrón-Cedeño, and Preslav Nakov. 2019. Findings of the NLP4IF-2019 [find](https://scholar.google.com/scholar?q=Giovanni+Da+San+Martino%2C+Alberto+Barr%C3%B3n%2DCede%C3%B1o%2C+and+Preslav+Nakov%2E+2019%2E+Findings+of+the+NLP4IF%2D2019)
12. Kabil Essefar, Abdellah El Mekki, Abdelkader El Mahdaouy, Nabil El Mamoun, and Ismail Berrada. 2021. CS-UM6P at SemEval-2021 task 7: Deep multi-task learning model for detecting and rating humor and offense. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1135–1140, Online. Association for Computational Linguistics. [↗](https://doi.org/10.60692/7jy08-3cy50)
13. Maram Hasanain, Firoj Alam, Hamdy Mubarak, Samir Abdaljalil, Wajdi Zaghouani, Preslav Nakov, Giovanni Da San Martino, and Abed Alhakim Freihat. 2023. [ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text](https://md.research.elamur.ai/free/en/papers/hasanain-2023-araieval-shared-task-persuasion-techniques/). In Proceedings of the First Arabic Natural Language Processing Conference (ArabicNLP 2023), Singapore. Association for Computational Linguistics.
14. Maram Hasanain, Fatima Haouari, Reem Suwaileh, Zien Sheikh Ali, Bayan Hamdan, Tamer Elsayed, Alberto Barrón-Cedeño, Giovanni Da San Martino, and Preslav Nakov. 2020. Overview of checkthat! 2020i arabic: Automatic identification and verification of claims in social media. In Conference and Labs of the Evaluation Forum. [find](https://scholar.google.com/scholar?q=Maram+Hasanain%2C+Fatima+Haouari%2C+Reem+Suwaileh%2C+Zien+Sheikh+Ali%2C+Bayan+Hamdan%2C+Tamer+Elsayed%2C+Alberto+Barr%C3%B3n%2DCede%C3%B1o%2C+Giovanni+Da+San+Martino%2C+and+Preslav+Nakov%2E+2020%2E+Overview+of+checkthat%21+2020i+arabi)
15. Salima Lamsiyah, Abdelkader El Mahdaouy, Hamza Alami, Ismail Berrada, and Christoph Schommer. 2023. UL & UM6P at SemEval-2023 task 10: Semi-supervised multi-task learning for explainable detection of online sexism. In Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 644–650, Toronto, Canada. Association for Computational Linguistics. [↗](https://doi.org/10.60692/k23qy-55h22)
16. Hamdy Mubarak, Samir Abdaljalil, Azza Nassar, and Firoj Alam. 2023. Detecting and identifying the reasons for deleted tweets before they are posted. Frontiers in Artificial Intelligence, 6. [↗](https://doi.org/10.3389/frai.2023.1219767)
17. Preslav Nakov, Firoj Alam, Shaden Shaar, Giovanni Da San Martino, and Yifan Zhang. 2021a. Covid-19 in bulgarian social media: Factuality, harmfulness, propaganda, and framing. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 997–1009. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2C+Firoj+Alam%2C+Shaden+Shaar%2C+Giovanni+Da+San+Martino%2C+and+Yifan+Zhang%2E+2021a%2E+Covid%2D19+in+bulgarian+social+media%3A+Factuality%2C+harmfulness%2C+propaganda%2C+and+framing%2E+In+Proceedings+of+the+In)
18. Preslav Nakov, Giovanni Da San Martino, Tamer Elsayed, Alberto Barrón-Cedeno, Rubén Míguez, Shaden Shaar, Firoj Alam, Fatima Haouari, Maram Hasanain, Nikolay Babulkov, et al. 2021b. The clef-2021 checkthat! lab on detecting check-worthy claims, previously fact-checked claims, and fake news. In Advances in Information Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28–April 1, 2021, Proceedings, Part II 43, pages 639–649. Springer. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2C+Giovanni+Da+San+Martino%2C+Tamer+Elsayed%2C+Alberto+Barr%C3%B3n%2DCedeno%2C+Rub%C3%A9n+M%C3%ADguez%2C+Shaden+Shaar%2C+Firoj+Alam%2C+Fatima+Haouari%2C+Maram+Hasanain%2C+Nikolay+Babulkov%2C+et+al%2E+2021b%2E+The+clef%2D2021+chec)
19. Ray Oshikawa, Jing Qian, and William Yang Wang. 2020. A survey on natural language processing for fake news detection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6086–6093, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Ray+Oshikawa%2C+Jing+Qian%2C+and+William+Yang+Wang%2E+2020%2E+A+survey+on+natural+language+processing+for+fake+news+detection%2E+In+Proceedings+of+the+Twelfth+Language+Resources+and+Evaluation+Conference%2C+pages)
20. Francesco Pinto, Harry Yang, Ser-Nam Lim, Philip H. S. Torr, and Puneet K. Dokania. 2023. Regmixup: Mixup as a regularizer can surprisingly improve accuracy and out distribution robustness. [↗](https://doi.org/10.48550/arxiv.2206.14502)
21. Gautam Kishore Shahi, Julia Maria Struß, and Thomas Mandl. 2021. Overview of the clef-2021 checkthat! lab: Task 3 on fake news detection. In CLEF (Working Notes), pages 406–423. [find](https://scholar.google.com/scholar?q=Gautam+Kishore+Shahi%2C+Julia+Maria+Stru%C3%9F%2C+and+Thomas+Mandl%2E+2021%2E+Overview+of+the+clef%2D2021+checkthat%21+lab%3A+Task+3+on+fake+news+detection%2E+In+CLEF+%28Working+Notes%29%2C+pages+406%E2%80%93423%2E)
22. Mohamad Sharara, Wissam Mohamad, Ralph Tawil, Ralph Chobok, Wolf Assi, and Antonio Tannoury. 2022. [Arabert model for propaganda detection](https://md.research.elamur.ai/free/en/papers/sharara-2022-arabert-model-propaganda-detection/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 520– 523.

*Updated 1 November 2026*