---
title: "joy_2004114 at MAHED Shared Task : Filtering Hate Speech"
description: "The authors address hateful meme detection in Arabic for Subtask 3 of the MAHED 2025 shared task at… 27 references, cited by 1 paper here."
authors: ["Joy Das", "Alamgir Hossain", "Mohammed Moshiul Hoque"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks"
source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.94/"
doi: "10.18653/v1/2025.arabicnlp-sharedtasks.94"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/das-2025-joy-2004114-mahed-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/das-2025-joy-2004114-mahed-shared-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) / joy\_2004114 at MAHED Shared Task : Filtering Hate Speech from Memes using A Multimodal Fusion-based Approach

# joy\_2004114 at MAHED Shared Task : Filtering Hate Speech from Memes using A Multimodal Fusion-based Approach

**Authors:** Joy Das, Alamgir Hossain and [Mohammed Moshiul Hoque](https://md.research.elamur.ai/free/en/authors/mohammed-moshiul-hoque/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.94/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.94.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.94](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.94) · [OpenAlex](https://openalex.org/W4416037632)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/das-2025-joy-2004114-mahed-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The authors address hateful meme detection in Arabic for Subtask 3 of the MAHED 2025 shared task at ArabicNLP 2025. In a meme, hostility toward a group may appear only when the overlaid text and the picture are read together. Arabic adds further difficulty: annotated multimodal data is scarce, and most strong models were built for English. The team used the QCRI/Prop2Hate-Meme corpus and built a deliberately lightweight dual-encoder system. Image features come from a pretrained InceptionResNetV2, and the meme text is encoded with mBERT's [CLS] vector. The two vectors are concatenated and passed to a small feed-forward classifier. They first compared several text-only and image-only encoders, then several fused pairs. InceptionNet + mBERT performed best on validation data, with a macro F1 of 0.69. On the official CodaBench test it scored 0.63, which ranked sixth. The study shows that a cheap fusion of off-the-shelf encoders clearly beats either modality alone. This makes it a practical option where large vision–language models are too expensive.

## Key points
- **Data and task:** QCRI/Prop2Hate-Meme pairs images with Arabic text and labels each meme hateful or non-hateful. It was distributed as Parquet files with ready-made train, validation and test splits.
- **Preprocessing:**
  - Images were resized to 128×128 and scaled to [0, 1].
  - Text was lowercased, whitespace-cleaned and padded or truncated to 128 tokens.
  - Non-hateful texts longer than 62 characters were removed. All hateful samples were kept.
- **Architecture:**
  - A 1536-D InceptionResNetV2 global-average-pooled vector (ImageNet weights, no top) is concatenated with a 768-D mBERT [CLS] vector, giving 2304-D.
  - An MLP follows: 1024 → 512 → 256 → 128 with ReLU, dropout 0.5 after the 512 and 128 layers, and a 2-way softmax.
- **Unimodal validation results (macro F1):**
  - AraBERT 0.63, mBERT 0.63, BERT-base-uncased 0.55.
  - InceptionNetV2 0.58, EfficientNetB7 0.48, EfficientNetB3 0.45.
- **Fused validation results (macro F1):**
  - InceptionNet + mBERT 0.69 (0.63 official, 6th place).
  - InceptionNet + BERT 0.67 and EfficientNetB7 + mBERT 0.67.
  - InceptionNet + AraBERT 0.59 and EfficientNetB3 + mBERT 0.56.

## Interesting ideas
- The paper builds on the "benign confounder" idea: test pairs where each part looks innocent alone but the combination carries the harmful meaning, so shortcuts through one channel fail.
- The encoders are run once and their vectors are stored as fixed arrays, so many small classifier heads can be compared cheaply without retraining the backbones.
- Every channel is scored on its own before fusion, which shows how much each source contributes and why the best pair wins.
- The authors report both their own validation score and the hidden official score, and the gap between them (0.69 vs 0.63) reveals optimism from model selection on dev.
- Noise filtering was applied asymmetrically: long samples were dropped only from the majority class, while every minority example was kept.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Build negative-control test sets where each sentence alone is neutral but only their combination licenses an inference. A step that answers correctly from a single sentence is then exposed as a shortcut.
- **I2** (tree): gparse could cache frozen transformer token vectors per treebank sentence. Parser-head variants and dialect-converted data could then be compared quickly on the same features.
- **I3** (frames): For irony or implicature decisions, score each cue alone (lexical, syntactic, contextual) before combining them. This keeps the final decision explainable as a sum of measured contributions.
- **I4** (learn): Always keep a held-out set that is never used for choosing rule repairs or teacher tiers. Report its score next to the dev score, so the size of any overfitting gap stays visible.
- **I5** (learn): When pruning silver data from witness parsers, apply stricter filters to over-represented easy constructions than to rare ones. That way scarce phenomena are not lost along with the noise.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Chittagong University of Engineering & Technology](https://md.research.elamur.ai/free/en/universities/chittagong-university-of-engineering-technology/) — University, Chittagong, Bangladesh · 25 works · 29 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works

🧭 Topics (4): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Joy Das, Alamgir Hossain and Mohammed Moshiul Hoque. 2025. joy\_2004114 at MAHED Shared Task : Filtering Hate Speech from Memes using A Multimodal Fusion-based Approach. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{das2025joy,
  title = {{joy\_2004114 at MAHED Shared Task : Filtering Hate Speech from Memes using A Multimodal Fusion-based Approach}},
  author = {Joy Das, Alamgir Hossain and Mohammed Moshiul Hoque},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.94},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.94}
}
```

- [← previous Jenin at AraGenEval Shared Task: Parameter-Efficient Fine-Tuning and Layer-Wise Analysis of Arabic LLMs for Authorship Style Transfer and Classification](https://md.research.elamur.ai/free/en/papers/malhis-2025-jenin-arageneval-shared-task-parameter/)
- [next → Lahjati at NADI 2025 A ECAPA-WavLM Fusion with Multi-Stage Optimization](https://md.research.elamur.ai/free/en/papers/albawwab-2025-lahjati-nadi-2025-ecapa-wavlm/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/)

## References

27 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shawly Ahsan, Eftekhar Hossain, Omar Sharif, Avishek Das, Mohammed Moshiul Hoque, and M Dewan. 2024. A multimodal framework to detect target aware aggression in memes. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2487–2500. [find](https://scholar.google.com/scholar?q=Shawly+Ahsan%2C+Eftekhar+Hossain%2C+Omar+Sharif%2C+Avishek+Das%2C+Mohammed+Moshiul+Hoque%2C+and+M+Dewan%2E+2024%2E+A+multimodal+framework+to+detect+target+aware+aggression+in+memes%2E+In+Proceedings+of+the+18th+Confe)
2. Firoj Alam, Md Rafiul Biswas, Uzair Shah, Wajdi Zaghouani, and Georgios Mikros. 2024. [Propaganda to hate: A multimodal analysis of arabic memes with multi-agent llms](https://md.research.elamur.ai/free/en/papers/alam-2024-propaganda-hate-multimodal-analysis-arabic/). In International Conference on Web Information Systems Engineering, pages 380– 390. Springer.
3. Chandni Kaundilya, Diksha Chawla, and Yatin Chopra. 2019. Automated text extraction from images using ocr system. In 2019 6th International Conference on Computing for Sustainable Global Development (INDIACom), pages 145–150. IEEE. [find](https://scholar.google.com/scholar?q=Chandni+Kaundilya%2C+Diksha+Chawla%2C+and+Yatin+Chopra%2E+2019%2E+Automated+text+extraction+from+images+using+ocr+system%2E+In+2019+6th+International+Conference+on+Computing+for+Sustainable+Global+Development+%28)
4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104)
5. Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. 2021. [The hateful memes challenge: Detecting hate speech in multimodal memes](https://md.research.elamur.ai/free/en/papers/kiela-2020-hateful-memes-challenge-detecting-hate/). [↗](https://doi.org/10.48550/arxiv.2005.04790)
6. Greeshma Arya, Mohammad Kamrul Hasan, Ashish Bagwari, Nurhizam Safie, Shayla Islam, Fatima Rayan Awad Ahmed, Aaishani De, Muhammad Attique Khan, and Taher M Ghazal. 2024. Multimodal hate speech detection in memes using contrastive language-image pre-training. IEEe Access, 12:22359–22375. [↗](https://doi.org/10.1109/access.2024.3361322)
7. Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. [Uniter: Universal image-text representation learning](https://md.research.elamur.ai/free/en/papers/chen-2019-uniter-universal-image-text-representation/).
8. Abhishek Das, Japsimar Singh Wahi, and Siyao Li. 2020. Detecting hate speech in multi-modal memes. [↗](https://doi.org/10.48550/arxiv.2012.14891)
9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). [↗](https://doi.org/10.18653/v1/n19-1423)
10. Raul Gomez, Jaume Gibert, Lluis Gomez, and Dimosthenis Karatzas. 2020. Exploring hate speech detection in multimodal publications. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1459–1467. Maram Hasanain, Md. Arid Hasan, Fatema Ahmed, Reem Suwaileh, Md. [find](https://scholar.google.com/scholar?q=Raul+Gomez%2C+Jaume+Gibert%2C+Lluis+Gomez%2C+and+Dimosthenis+Karatzas%2E+2020%2E+Exploring+hate+speech+detection+in+multimodal+publications%2E+In+2020+IEEE+Winter+Conference+on+Applications+of+Computer+Vision+%28WA)
11. Rafiul Biswas, Wajdi Zaghouani, and Firoj Alam. 2024. [Araieval shared task: Propagandistic techniques detection in unimodal and multimodal arabic content](https://md.research.elamur.ai/free/en/papers/hasanain-2024-araieval-shared-task-propagandistic-techniques/).
12. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). [↗](https://doi.org/10.48550/arxiv.1512.03385)
13. Eftekhar Hossain, Omar Sharif, and Mohammed Moshiul Hoque. 2022. Mute: A multimodal dataset for detecting hateful memes. In Proceedings of the 2nd conference of the asia-pacific chapter of the association for computational linguistics and the 12th international joint conference on natural language processing: student research workshop, pages 32–39. [find](https://scholar.google.com/scholar?q=Eftekhar+Hossain%2C+Omar+Sharif%2C+and+Mohammed+Moshiul+Hoque%2E+2022%2E+Mute%3A+A+multimodal+dataset+for+detecting+hateful+memes%2E+In+Proceedings+of+the+2nd+conference+of+the+asia%2Dpacific+chapter+of+the+associa)
14. Homa Hosseinmardi, Sabrina Arredondo Mattson, Rahat Ibn Rafiq, Richard Han, Qin Lv, and Shivakant Mishra. 2015. Analyzing labeled cyberbullying incidents on the instagram social network. In International conference on social informatics, pages 49–66. Springer. [find](https://scholar.google.com/scholar?q=Homa+Hosseinmardi%2C+Sabrina+Arredondo+Mattson%2C+Rahat+Ibn+Rafiq%2C+Richard+Han%2C+Qin+Lv%2C+and+Shivakant+Mishra%2E+2015%2E+Analyzing+labeled+cyberbullying+incidents+on+the+instagram+social+network%2E+In+Internatio)
15. Wonjae Kim, Bokyung Son, and Ildoo Kim. 2021. [Vilt: Vision-and-language transformer without convolution or region supervision](https://md.research.elamur.ai/free/en/papers/kim-2021-vilt-vision-language-transformer-convolution/). [↗](https://doi.org/10.48550/arxiv.2102.03334)
16. Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019. [Visualbert: A simple and performant baseline for vision and language](https://md.research.elamur.ai/free/en/papers/li-2019-visualbert-simple-performant-baseline-vision/). [↗](https://doi.org/10.48550/arxiv.1908.03557)
17. Phillip Lippe, Nithin Holla, Shantanu Chandra, Santhosh Rajamanickam, Georgios Antoniou, Ekaterina Shutova, and Helen Yannakoudakis. 2020. A multimodal framework for the detection of hateful memes. [↗](https://doi.org/10.48550/arxiv.2012.12871)
18. Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. [Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks](https://md.research.elamur.ai/free/en/papers/lu-2019-vilbert-pretraining-task-agnostic-visiolinguistic/).
19. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. [How multilingual is multilingual bert?](https://md.research.elamur.ai/free/en/papers/pires-2019-how-multilingual-is-multilingual-bert/) Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md.
20. Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021. [MOMENTA: A multimodal framework for detecting harmful memes and their targets](https://md.research.elamur.ai/free/en/papers/pramanick-2021-momenta-multimodal-framework-detecting-harmful/). In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4439–4455, Punta Cana, Dominican Republic. Association for Computational Linguistics.
21. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). [↗](https://doi.org/10.48550/arxiv.2103.00020)
22. Chhavi Sharma, Deepesh Bhageria, William Scott, Srinivas PYKL, Amitava Das, Tanmoy Chakraborty, Viswanath Pulabaigari, and Björn Gambäck. 2020a. [SemEval-2020 task 8: Memotion analysis- the visuo-lingual metaphor!](https://md.research.elamur.ai/free/en/papers/sharma-2020-semeval-2020-task-8-memotion/) In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 759– 773, Barcelona (online). International Committee for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2008.03781)
23. Chhavi Sharma, Deepesh Bhageria, William Scott, Srinivas PYKL, Amitava Das, Tanmoy Chakraborty, Viswanath Pulabaigari, and Björn Gambäck. 2020b. [SemEval-2020 task 8: Memotion analysis- the visuo-lingual metaphor!](https://md.research.elamur.ai/free/en/papers/sharma-2020-semeval-2020-task-8-memotion/) In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 759– 773, Barcelona (online). International Committee for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2008.03781)
24. Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alex Alemi. 2016. Inception-v4, inception-resnet and the impact of residual connections on learning. [↗](https://doi.org/10.1609/aaai.v31i1.11231)
25. Mingxing Tan and Quoc V. Le. 2020. [Efficientnet: Rethinking model scaling for convolutional neural networks](https://md.research.elamur.ai/free/en/papers/tan-2019-efficientnet-rethinking-model-scaling-convolutional/). [↗](https://doi.org/10.48550/arxiv.1905.11946)
26. Wajdi Zaghouani, Md Rafiul Biswas, Mabrouka Bessghaier, Shimaa Ibrahim, Georgio Mikros, Abul Hasnat, and Firoj Alam. 2025. [MAHED shared task: Multimodal detection of hope and hate emotions in arabic content](https://md.research.elamur.ai/free/en/papers/zaghouani-2025-mahed-shared-task-multimodal-detection/). In Proceedings of the Third Arabic Natural Language Processing Conference (Arabic-NLP 2025), Suzhou, China. Association for Computational Linguistics.
27. Mohamed Zaytoon, Nagwa El-Makky, and Marwan Torki. 2024. AlexUNLP-MZ at ArAIEval shared task: Contrastive learning, LLM features extraction and multi-objective optimization for Arabic multimodal meme propaganda detection. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 512–517, Bangkok, Thailand. Association for Computational Linguistics. Table A.1 shows that the dataset suffers from a notable class imbalance, with non-hateful samples dominating across all splits. This imbalance poses challenges for training reliable classifiers and highlights the need for robust evaluation strategies. [find](https://scholar.google.com/scholar?q=Mohamed+Zaytoon%2C+Nagwa+El%2DMakky%2C+and+Marwan+Torki%2E+2024%2E+AlexUNLP%2DMZ+at+ArAIEval+shared+task%3A+Contrastive+learning%2C+LLM+features+extraction+and+multi%2Dobjective+optimization+for+Arabic+multimodal+meme+)

*Updated 1 November 2026*