---
title: "Deep Multi-Task Models for Misogyny — Mahdaouy 2022"
description: "This paper describes the UM6P-NLP systems for the first Arabic Misogyny Identification (ArMI) shared task at FIRE 2021."
authors: ["Abdelkader El Mahdaouy", "Abdellah El Mekki", "Ahmed Oumar", "Hajar Mousannif", "Ismail Berrada"]
year: 2022
venue: "arXiv:2206.08407"
source: "https://arxiv.org/abs/2206.08407"
doi: "10.48550/arxiv.2206.08407"
arxiv: "2206.08407"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mahdaouy-2022-deep-multi-task-models-misogyny/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mahdaouy-2022-deep-multi-task-models-misogyny/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/2/) / Deep Multi-Task Models for Misogyny Identification and Categorization on Arabic Social Media

# Deep Multi-Task Models for Misogyny Identification and Categorization on Arabic Social Media

**Authors:** [Abdelkader El Mahdaouy](https://md.research.elamur.ai/free/en/authors/abdelkader-el-mahdaouy/), [Abdellah El Mekki](https://md.research.elamur.ai/free/en/authors/abdellah-el-mekki/), Ahmed Oumar, [Hajar Mousannif](https://md.research.elamur.ai/free/en/authors/hajar-mousannif/), [Ismail Berrada](https://md.research.elamur.ai/free/en/authors/ismail-berrada/) · **Year:** 2022 · **Venue:** arXiv:2206.08407 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2206.08407) · [PDF](https://arxiv.org/pdf/2206.08407) · [DOI 10.48550/arxiv.2206.08407](https://doi.org/10.48550/arxiv.2206.08407) · [arXiv 2206.08407](https://arxiv.org/abs/2206.08407) · [OpenAlex](https://openalex.org/W4283211152)

**Citations:** 2 citations (openalex-snapshot · 2026-10-14) · [4](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
This paper describes the UM6P-NLP systems for the first Arabic Misogyny Identification (ArMI) shared task at FIRE 2021. The task has two parts: deciding whether a tweet is misogynistic, and assigning misogynistic tweets to one of seven behaviour types. The tweets are mostly MSA, with some Egyptian, Gulf and Levantine dialect. Arabic has little labelled data for this problem, and the category labels are strongly skewed. The authors fine-tune the MARBERT encoder, which was pre-trained on about a billion Arabic tweets. On top of it they compare three multi-task architectures with single-task versions of the same designs. The architectures differ in how they pool features: the [CLS] vector only, task-specific attention over tokens, or extra attention across upper encoder layers. For the imbalanced categorization task they train with Focal Loss. All three official runs took the top three places in both subtasks. Multi-task models usually beat their single-task twins, and Focal Loss mainly improved macro F1. The work shows that a careful pooling design plus a loss suited to imbalance can be enough for a low-resource toxic-content task.

## Key points
- Data: 7,866 labelled training tweets and 1,966 test tweets. The Levantine part comes from Let-Mi, and the rest was collected by hashtag and annotated by native speakers.
- Models:
  - MT_CLS / ST_CLS use [CLS] classification heads.
  - MT_ATT / ST_ATT add per-task attention pooling concatenated with [CLS].
  - MT_VHATT / ST_VHATT add six "vertical" attention layers over the top intermediate MARBERT layers, which are then aggregated by another attention layer.
- Training settings:
  - Adam optimizer with learning rate 1e-5, 5 epochs, batch size 16, and a 90/10 train/dev split.
  - Focal Loss with γ = 2, and class weights equal to the dominant-class count divided by the class count.
- Task 1 result: the ensemble (run3, averaged logits of the three MTL models) reached 91.4 F1 and 91.9 accuracy. The next team (UoT) scored 90.0 F1.
- Task 2 result: MT_ATT (run2) reached the best F1 of 66.5, while the ensemble had the best accuracy at 83.3. The best other team (SOA_NLP) scored 53.1 F1.

## Interesting ideas
- Emojis are pulled out of the tweet and passed as a second segment after [SEP], so the encoder sees the cleaned text and the emoji signal as separate inputs. — For Mova (tokens): Non-lexical symbols such as emoji or emoticons could be split off the token stream into a side channel, keeping UD tokenization clean while still preserving the signal for later stages.
- Each task gets its own attention pooling over the shared encoder's token outputs, so one backbone can provide different feature views for different labels at little cost. — For Mova (frames): A shared encoder with separate attention heads for speech act, irony and sentiment-like cues could expose per-task attention weights, giving an explainable trace of which words drove each pragmatic decision.
- "Vertical" attention learns how much to draw from several upper transformer layers instead of trusting only the final layer. — For Mova (tree): gparse could learn a weighted, inspectable mix of intermediate layers for arc and label scoring, which may help with syntax that is better encoded in middle layers.
- Focal Loss with inverse-frequency class weights raised F1 on rare categories without hurting overall accuracy much, and in the multi-task setup it even helped the other task. — For Mova (learn): Rare UD relations or rare frame roles in silver-data training could be trained with focal, frequency-weighted loss, with per-class reports checked to confirm that rare labels improve.
- Averaging logits from several differently pooled models gave the best accuracy, but a single model kept the best macro F1, so the choice of evaluation metric decides which system to ship. — For Mova (learn): When combining witness parsers or models, the evaluation should report per-class and macro scores next to accuracy, so that an ensemble that only helps frequent labels is not taken as a general improvement.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cited by

Cited by

4 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 3 not yet.

- [Transformers and Ensemble methods: A solution for Hate Speech Detection in Arabic languages](https://md.research.elamur.ai/free/en/papers/paula-2023-transformers-ensemble-methods-solution-hate/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{mahdaouy2022deep,
  title = {{Deep Multi-Task Models for Misogyny Identification and Categorization on Arabic Social Media}},
  author = {Mahdaouy, Abdelkader El and Mekki, Abdellah El and Oumar, Ahmed and Mousannif, Hajar and Berrada, Ismail},
  year = {2022},
  doi = {10.48550/arxiv.2206.08407},
  eprint = {2206.08407},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2206.08407}
}
```

- [← previous Deduplicating Training Data Mitigates Privacy Risks in Language Models](https://md.research.elamur.ai/free/en/papers/kandpal-2022-deduplicating-training-data-mitigates-privacy/)
- [next → DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale](https://md.research.elamur.ai/free/en/papers/rajbhandari-2022-deepspeed-moe-advancing-mixture-experts/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/2/)

## References

20 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. Moloney, T. P. Love, Assessing online misogyny: Perspectives from sociology and feminist media studies, Sociology Compass 12 (2018). [find](https://scholar.google.com/scholar?q=M%2E+Moloney%2C+T%2E+P%2E+Love%2C+Assessing+online+misogyny%3A+Perspectives+from+sociology+and+feminist+media+studies%2C+Sociology+Compass+12+%282018%29%2E)
2. B. Poland, Haters: Harassment, abuse, and violence online, 2016. [find](https://scholar.google.com/scholar?q=B%2E+Poland%2C+Haters%3A+Harassment%2C+abuse%2C+and+violence+online%2C+2016%2E)
3. J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 2019, pp. 4171–4186. URL: https://aclanthology.org/ N19-1423. doi: 10.18653/v1/N19-1423 . [↗](https://doi.org/10.18653/v1/n19-1423)
4. W. Antoun, F. Baly, H. Hajj, AraBERT: Transformer-based model for Arabic language understanding, in: Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, European Language Resource Association, Marseille, France, 2020, pp. 9–15. URL: https://aclanthology.org/ 2020.osact-1.2. [↗](https://aclanthology.org/2020.osact-1.2/) [our card](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/)
5. M. Abdul-Mageed, A. Elmadany, E. M. B. Nagoudi, ARBERT & MARBERT: Deep bidirectional transformers for Arabic, in: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Association for Computational Linguistics, Online, 2021, pp. 7088–7105. URL: https://aclanthology.org/2021.acl-long.551. doi: 10.18653/v1/2021.acl-long.551 . [↗](https://doi.org/10.18653/v1/2021.acl-long.551) [our card](https://md.research.elamur.ai/free/en/papers/abdul-2021-mageed-arbert-marbert-deep-bidirectional-transformers/)
6. A. El Mekki, A. El Mahdaouy, I. Berrada, A. Khoumsi, Domain adaptation for Arabic cross-domain and cross-dialect sentiment analysis from contextualized word embedding, in: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Online, 2021, pp. 2824–2837. URL: https://aclanthology.org/2021.naacl-main. 226. doi: 10.18653/v1/2021.naacl-main.226 . [↗](https://doi.org/10.18653/v1/2021.naacl-main.226)
7. A. El Mekki, A. El Mahdaouy, K. Essefar, N. El Mamoun, I. Berrada, A. Khoumsi, BERT-based multi-task model for country and province level MSA and dialectal Arabic identification, in: Proceedings of the Sixth Arabic Natural Language Processing Workshop, Association for Computational Linguistics, Kyiv, Ukraine (Virtual), 2021, pp. 271–275. URL: https://aclanthology.org/2021.wanlp-1.31. [↗](https://aclanthology.org/2021.wanlp-1.31/)
8. A. El Mahdaouy, A. El Mekki, K. Essefar, N. El Mamoun, I. Berrada, A. Khoumsi, Deep multi-task model for sarcasm detection and sentiment analysis in Arabic language, in: Proceedings of the Sixth Arabic Natural Language Processing Workshop, Association for Computational Linguistics, Kyiv, Ukraine (Virtual), 2021, pp. 334–339. URL: https://aclanthology.org/2021.wanlp-1.42. [↗](https://aclanthology.org/2021.wanlp-1.42/) [our card](https://md.research.elamur.ai/free/en/papers/mahdaouy-2021-deep-multi-task-model-sarcasm/)
9. N. Safi Samghabadi, P. Patwa, S. PYKL, P. Mukherjee, A. Das, T. Solorio, Aggression and misogyny detection using BERT: A multi-task approach, in: Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, European Language Resources Association (ELRA), Marseille, France, 2020, pp. 126–131. URL: https://aclanthology.org/ 2020.trac-1.20. [↗](https://aclanthology.org/2020.trac-1.20/)
10. E. Fersini, D. Nozza, P. Rosso, AMI @ EVALITA2020: automatic misogyny identification, in: V. Basile, D. Croce, M. D. Maro, L. C. Passaro (Eds.), Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2020), Online event, December 17th, 2020, volume 2765 of CEUR Workshop Proceedings, CEUR-WS.org, 2020. [find](https://scholar.google.com/scholar?q=E%2E+Fersini%2C+D%2E+Nozza%2C+P%2E+Rosso%2C+AMI+%40+EVALITA2020%3A+automatic+misogyny+identification%2C+in%3A+V%2E+Basile%2C+D%2E+Croce%2C+M%2E+D%2E+Maro%2C+L%2E+C%2E+Passaro+%28Eds%2E%29%2C+Proceedings+of+the+Seventh+Evaluation+Campaign+of+Natur)
11. F. Rodríguez-Sánchez, J. Carrillo-de Albornoz, L. Plaza, Automatic classification of sexism in social networks: An empirical study on twitter data, IEEE Access 8 (2020) 219563–219576. doi: 10.1109/ACCESS.2020.3042604 . [↗](https://doi.org/10.1109/access.2020.3042604)
12. H. Mulki, B. Ghanem, Let-mi: An Arabic Levantine Twitter dataset for misogynistic language, in: Proceedings of the Sixth Arabic Natural Language Processing Workshop, Association for Computational Linguistics, Kyiv, Ukraine (Virtual), 2021, pp. 154–163. URL: https://aclanthology.org/2021.wanlp-1.16. [↗](https://aclanthology.org/2021.wanlp-1.16/)
13. H. Mulki, B. Ghanem, ArMI at FIRE2021: Overview of the First Shared Task on Arabic Misogyny Identification, in: Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, CEUR, 2021. [find](https://scholar.google.com/scholar?q=H%2E+Mulki%2C+B%2E+Ghanem%2C+ArMI+at+FIRE2021%3A+Overview+of+the+First+Shared+Task+on+Arabic+Misogyny+Identification%2C+in%3A+Working+Notes+of+FIRE+2021+%2D+Forum+for+Information+Retrieval+Evaluation%2C+CEUR%2C+2021%2E)
14. T. Lin, P. Goyal, R. B. Girshick, K. He, P. Dollár, Focal loss for dense object detection, CoRR abs/1708.02002 (2017). URL: http://arxiv.org/abs/1708.02002. arXiv:1708.02002 . [↗](https://arxiv.org/abs/1708.02002) [our card](https://md.research.elamur.ai/free/en/papers/lin-2017-focal-loss-dense-object-detection/)
15. D. Bahdanau, K. Cho, Y. Bengio, Neural machine translation by jointly learning to align and translate, in: Y. Bengio, Y. LeCun (Eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL: http://arxiv.org/abs/1409.0473. [↗](https://arxiv.org/abs/1409.0473) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
16. Z. Yang, D. Yang, C. Dyer, X. He, A. Smola, E. Hovy, Hierarchical attention networks for document classification, in: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, San Diego, California, 2016, pp. 1480–1489. URL: https://www.aclweb.org/anthology/N16-1174. doi: 10.18653/v1/N16-1174 . [↗](https://doi.org/10.18653/v1/n16-1174) [our card](https://md.research.elamur.ai/free/en/papers/yang-2016-hierarchical-attention-networks-document-classification/)
17. K. Essefar, A. El Mekki, A. El Mahdaouy, N. El Mamoun, I. Berrada, CS-UM6P at SemEval-2021 task 7: Deep multi-task learning model for detecting and rating humor and offense, in: Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), Association for Computational Linguistics, Online, 2021, pp. 1135–1140. URL: https://aclanthology.org/2021.semeval-1.159. doi: 10.18653/v1/2021.semeval-1.159 . [↗](https://doi.org/10.18653/v1/2021.semeval-1.159)
18. N. El Mamoun, A. El Mahdaouy, A. El Mekki, K. Essefar, I. Berrada, CS-UM6P at SemEval-2021 task 1: A deep learning model-based pre-trained transformer encoder for lexical complexity, in: Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), Association for Computational Linguistics, Online, 2021, pp. 585–589. URL: https://aclanthology.org/2021.semeval-1.73. doi: 10.18653/v1/2021.semeval-1.73 . [↗](https://doi.org/10.18653/v1/2021.semeval-1.73)
19. R. Caruana, Learning many related tasks at the same time with backpropagation, in: Proceedings of the 7th International Conference on Neural Information Processing Systems, NIPS’94, MIT Press, Cambridge, MA, USA, 1994, p. 657–664. [find](https://scholar.google.com/scholar?q=R%2E+Caruana%2C+Learning+many+related+tasks+at+the+same+time+with+backpropagation%2C+in%3A+Proceedings+of+the+7th+International+Conference+on+Neural+Information+Processing+Systems%2C+NIPS%E2%80%9994%2C+MIT+Press%2C+Cambrid)
20. Y. Sun, S. Wang, Y. Li, S. Feng, H. Tian, H. Wu, H. Wang, Ernie 2.0: A continual pre-training framework for language understanding, arXiv preprint arXiv:1907.12412 (2019). [↗](https://arxiv.org/abs/1907.12412) [our card](https://md.research.elamur.ai/free/en/papers/sun-2019-ernie-2-0-continual/)