---
title: "Toxic language detection: a systematic review of Arabic datasets"
description: "Research on detecting harmful Arabic online language has grown quickly, but beyond a handful of… 117 references, cited by 3 papers here."
authors: ["Imene Bensalem", "Paolo Rosso", "Hanane Zitouni"]
year: 2023
venue: "arXiv:2312.07228"
source: "https://arxiv.org/abs/2312.07228"
doi: "10.48550/arxiv.2312.07228"
arxiv: "2312.07228"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bensalem-2023-toxic-language-detection-systematic-review/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bensalem-2023-toxic-language-detection-systematic-review/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/) / Toxic language detection: a systematic review of Arabic datasets

# Toxic language detection: a systematic review of Arabic datasets

**Authors:** [Imene Bensalem](https://md.research.elamur.ai/free/en/authors/imene-bensalem/), [Paolo Rosso](https://md.research.elamur.ai/free/en/authors/paolo-rosso/) and Hanane Zitouni · **Year:** 2023 · **Venue:** arXiv:2312.07228 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2312.07228) · [PDF](https://arxiv.org/pdf/2312.07228) · [DOI 10.48550/arxiv.2312.07228](https://doi.org/10.48550/arxiv.2312.07228) · [arXiv 2312.07228](https://arxiv.org/abs/2312.07228) · [OpenAlex](https://openalex.org/W4389712439)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/bensalem-2023-toxic-language-detection-systematic-review/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Research on detecting harmful Arabic online language has grown quickly, but beyond a handful of well-known corpora and shared-task releases, little is documented about which datasets exist, how they were labelled, and whether they can actually be reused. The authors run a systematic review adapted from the Kitchenham & Charters software-engineering guidelines. They searched publication databases and dataset repositories, with the last search in October 2023. "Toxic language" is used as an umbrella term covering profanity, hate speech, abuse, violence, and even non-explicit harm such as disinformation. The review gathers 54 accessible datasets, each tied to at least one paper. Each dataset is coded on 18 criteria grouped into four axes: availability, content, annotation process, and reusability. This is the largest such inventory to date, since no earlier survey covered more than 11 Arabic toxic-language datasets. The authors identify gaps, give recommendations, and maintain a public, updatable catalogue on GitHub. The work gives Arabic NLP researchers a map of usable training data and of its quality.

## Key points
- The review includes 54 datasets, analysed across 18 criteria on four axes (availability details, content, annotation process, reusability). The catalogue is at github.com/Imene1/Arabic-toxic-language.
- Dataset output over time:
  - The first accessible dataset dates from 2015.
  - Twelve datasets appeared in 2021.
  - Seventeen datasets appeared in 2022.
- Hosting is concentrated:
  - GitHub holds 23 datasets.
  - QCRI holds 5.
  - Two datasets were obtained only by writing to their authors.
- More than half of all datasets draw their text from Twitter, with Facebook and YouTube as the next most common sources. Reddit and Instagram see very little use, and no dataset yet draws on Wikipedia or TikTok. Web-derived data includes OSCAR with adult-content labels.
- Earlier catalogues missed many datasets. For example, Masader Plus lists only 11 toxic-language sets among 500+ Arabic resources, and the more general surveys cite about six Arabic datasets each.

## Interesting ideas
- The search went beyond paper databases to dataset repositories, using a wide keyword list that included adjacent tasks such as stance, propaganda, polarization and bot detection; this broad net recovered resources that narrower catalogues missed.
- For each of the 18 criteria, the review records whether the information came from inspecting the dataset files or from the paper's own description, which separates verified facts from claims.
- Derived corpora count as separate datasets whenever they have their own URL, their own paper, and any real difference, so re-releases across shared tasks are tracked rather than merged.
- Toxicity is treated as a hierarchy: a broad harm concept with fine-grained subtypes such as racism, sexism and threats, and it includes harm with no explicit offensive wording, such as disinformation.
- Annotation practice (agreement, guidelines, reuse readiness) is treated as a first-class review axis rather than reporting only dataset size and labels.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): When collecting treebanks or conversion targets, combine repository crawling with a broad synonym list so that small or oddly labelled UD-style resources are not missed.
- **I2** (learn): Evaluation and training-data registries could tag each property as "checked in data" versus "stated by source", which keeps provenance explainable.
- **I3** (tree): Treebank versions and dialect-converted copies could be registered as distinct, linked entries, so parser results always point to the exact variant used.
- **I4** (frames): A speech-act layer could model harmful utterances as a typed hierarchy and allow implicit harm detected through implicature, not only lexical cues.
- **I5** (learn): Silver data from witness parsers could carry agreement statistics and labelling-procedure metadata, so downstream training can weight or filter examples by annotation quality.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Universitat Politècnica de València](https://md.research.elamur.ai/free/en/universities/universitat-politecnica-de-valencia/) — University, Valencia, Spain · 34 works · 3,575 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Imene Bensalem, Paolo Rosso and Hanane Zitouni. 2023. Toxic language detection: a systematic review of Arabic datasets. arXiv:2312.07228.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{bensalem2023toxic,
  title = {{Toxic language detection: a systematic review of Arabic datasets}},
  author = {Imene Bensalem, Paolo Rosso and Hanane Zitouni},
  year = {2023},
  doi = {10.48550/arxiv.2312.07228},
  eprint = {2312.07228},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2312.07228}
}
```

- [← previous The Conversational Discourse Unit: Identification and Its Role in Conversational Turn-taking Management](https://md.research.elamur.ai/free/en/papers/hu-2023-conversational-discourse-unit-identification-its/)
- [next → USTHB at NADI 2023 shared task: Exploring Preprocessing and Feature Engineering Strategies for Arabic Dialect Identification](https://md.research.elamur.ai/free/en/papers/lichouri-2023-usthb-nadi-2023-shared-task/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/)

## References

117 references, 35 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abadji, J., Suarez, P. O., Romary, L., & Sagot, B. (2022). [Towards a Cleaner Document-Oriented Multilingual Crawled Corpus](https://md.research.elamur.ai/free/en/papers/abadji-2022-towards-cleaner-document-oriented-multilingual/). 2022 Language Resources and Evaluation Conference, LREC 2022, June, 4344–4355.
2. Abainia, K. (2020). DZDC12: a new multipurpose parallel Algerian Arabizi–French code-switched corpus. Language Resources and Evaluation, 54(2), 419–455. https://doi.org/10.1007/s10579-019-09454-8 [↗](https://doi.org/10.1007/s10579-019-09454-8)
3. Abdelali, A., Mubarak, H., Chowdhury, S. A., Hasanain, M., Mousi, B., Boughorbel, S., Kheir, Y. El, Izham, D., Dalvi, F., Hawasly, M., Nazar, N., Elshahawy, Y., Ali, A., Durrani, N., Milic-Frayling, N., & Alam, F. (2023). Benchmarking Arabic AI with Large Language Models. Preprint ArXiv.2305.14982. https://doi.org/10.48550/arXiv.2305.14982 [↗](https://doi.org/10.48550/arxiv.2305.14982)
4. Abozinadah, E. A., & Jones, J. H. (2017). A statistical learning approach to detect abusive Twitter accounts. ACM International Conference Proceeding Series, Part F1302, 6–13. https://doi.org/10.1145/3093241.3093281 [↗](https://doi.org/10.1145/3093241.3093281)
5. Ahmed, A., Ali, N., Alzubaidi, M., Zaghouani, W., Abd-alrazaq, A. A., & Househ, M. (2022). Freely Available Arabic Corpora: A Scoping Review. Computer Methods and Programs in Biomedicine Update, 2(October 2021), 100049. https://doi.org/10.1016/j.cmpbup.2022.100049 [↗](https://doi.org/10.1016/j.cmpbup.2022.100049)
6. Al-Hassan, A., & Al-Dossari, H. (2019). Detection of Hate Speech in Social Networks: A Survey on Multilingual Corpus. Computer Science & Information Technology(CS & IT), 9(2), 83–100. https://doi.org/10.5121/csit.2019.90208 [↗](https://doi.org/10.5121/csit.2019.90208)
7. Al Bayari, R., & Abdallah, S. (2022). Instagram-Based Benchmark Dataset for Cyberbullying Detection in Arabic Text. Data, 7(83), 1–11. https://doi.org/10.3390/data7070083 [↗](https://doi.org/10.3390/data7070083)
8. Alakrot, A., Murray, L., & Nikolov, N. S. (2018). Dataset Construction for the Detection of Anti-Social Behaviour in Online Communication in Arabic. Procedia Computer Science, 142, 174–181. https://doi.org/10.1016/j.procs.2018.10.473 [↗](https://doi.org/10.1016/j.procs.2018.10.473)
9. Alam, F., Dalvi, F., Shaar, S., Durrani, N., Mubarak, H., Nikolov, A., Martino, G. D. S., Abdelali, A., Sajjad, H., Darwish, K., & Nakov, P. (2021). Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Arms. Proceedings of the Fifteenth International AAAI Conference OnWeb and Social Media (ICWSM 2021) Fighting, 913–922. [↗](https://doi.org/10.48550/arxiv.2007.07996)
10. Alam, F., Mubarak, H., Zaghouani, W., Martino, G. D. S., & Nakov, P. (2022). [Overview of the WANLP 2022 Shared Task on Propaganda Detection in Arabic](https://md.research.elamur.ai/free/en/papers/alam-2022-overview-wanlp-2022-shared-task/). Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), 108–118.
11. Alam, F., Shaar, S., Dalvi, F., Sajjad, H., Nikolov, A., Mubarak, H., Da San Martino, G., Abdelali, A., Durrani, N., Darwish, K., Al-Homai, A., Zaghouani, W., Caselli, T., Danoe, G., Stolk, F., Bruntink, B., & Nakov, P. (2021). Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society. Findings of the Association for Computational Linguistics, Findings of ACL: EMNLP 2021, 611–649. https://doi.org/10.18653/v1/2021.findings-emnlp.56 [↗](https://doi.org/10.18653/v1/2021.findings-emnlp.56)
12. Albadi, N., Kurdi, M., & Mishra, S. (2018). Are They Our Brothers? Analysis and Detection of Religious Hate Speech in the Arabic Twittersphere. Proceedings of the 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 69– 76. [find](https://scholar.google.com/scholar?q=Albadi%2C+N%2E%2C+Kurdi%2C+M%2E%2C+%26+Mishra%2C+S%2E+%282018%29%2E+Are+They+Our+Brothers%3F+Analysis+and+Detection+of+Religious+Hate+Speech+in+the+Arabic+Twittersphere%2E+Proceedings+of+the+2018+IEEE%2FACM+International+Conferenc)
13. Albadi, N., Kurdi, M., & Mishra, S. (2019). [Hateful people or hateful bots? Detection and characterization of bots spreading religious hatred in Arabic social media](https://md.research.elamur.ai/free/en/papers/albadi-2019-hateful-people-hateful-bots-detection/). Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1–25. https://doi.org/10.1145/3359163 [↗](https://doi.org/10.1145/3359163)
14. Albadi, N., Kurdi, M., & Mishra, S. (2022). Deradicalizing YouTube: Characterization, Detection, and Personalization of Religiously Intolerant Arabic Videos. Proceedings of the ACM on Human-Computer Interaction, 6(CSCW2), 1–25. https://doi.org/10.1145/3555618 [↗](https://doi.org/10.1145/3555618)
15. ALBayari, R., Abdullah, S., & Salloum, S. A. (2021). Cyberbullying Classification Methods for Arabic: A Systematic Review. Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2021). AICV 2021. Advances in Intelligent Systems and Computing, Vol 1377., May, 375–385. https://doi.org/10.1007/978-3-030-76346-6_35 [↗](https://doi.org/10.1007/978-3-030-76346-6_35)
16. Aldera, S., Emam, A., Al-Qurishi, M., Alrubaian, M., & Alothaim, A. (2021). Exploratory Data Analysis and Classification of a New Arabic Online Extremism Dataset. IEEE Access, 9, 161613–161626. https://doi.org/10.1109/ACCESS.2021.3132651 [↗](https://doi.org/10.1109/access.2021.3132651)
17. Alduailaj, A. M., & Belghith, A. (2023). Detecting Arabic Cyberbullying Tweets Using Machine Learning. Machine Learning https://doi.org/10.3390/make5010003 [↗](https://doi.org/10.3390/make5010003)
18. Alghamdi, H., & Petraki, E. (2018). Arabizi in Saudi Arabia: A deviant form of language or simply a form of expression? Social Sciences, 7(9). https://doi.org/10.3390/SOCSCI7090155 [↗](https://doi.org/10.3390/socsci7090155)
19. Alhelbawy, A., Kruschwitz, U., & Poesio, M. (2016). [Towards a corpus of violence acts in Arabic social media](https://md.research.elamur.ai/free/en/papers/alhelbawy-2016-corpus-violence-acts-arabic-social/). Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016, 1627–1631.
20. Almanea, D., & Poesio, M. (2022). [ArMIS - The Arabic Misogyny and Sexism Corpus with Annotator Subjective Disagreements](https://md.research.elamur.ai/free/en/papers/almanea-2022-armis-arabic-misogyny-sexism-corpus/). Proceedings of the Language Resources and Evaluation Conference, June, 2282–2291.
21. Alqmase, M., Al-Muhtaseb, H., & Rabaan, H. (2021). Sports-fanaticism formalism for sentiment analysis in Arabic text. Social Network Analysis and Mining, 11(1), 1–24. https://doi.org/10.1007/s13278-021-00757-9 [↗](https://doi.org/10.1007/s13278-021-00757-9)
22. Alsafari, S., Sadaoui, S., & Mouhoub, M. (2020). Hate and offensive speech detection on Arabic social media. Online Social Networks and Media, 19(March), 100096. https://doi.org/10.1016/j.osnem.2020.100096 [↗](https://doi.org/10.1016/j.osnem.2020.100096)
23. Alshalan, R., & Al-Khalifa, H. (2020). A Deep Learning Approach for Automatic Hate Speech Detection in the Saudi Twittersphere. Applied Sciences, 10(23), 8614. https://doi.org/10.3390/app10238614 [↗](https://doi.org/10.3390/app10238614)
24. Alshehri, A., Nagoudi, E. M. B., & Abdul-Mageed, M. (2020). [Understanding and Detecting Dangerous Speech in Social Media](https://md.research.elamur.ai/free/en/papers/alshehri-2020-understanding-detecting-dangerous-speech-social/). Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, May, 40–47. [↗](https://doi.org/10.48550/arxiv.2005.06608)
25. Alsunaidi, N., Aljbali, S., Yasin, Y., & Aljamaan, H. (2023). Arabic Cyberbullying Detection Using Machine Learning: State of the Art Survey. ACM International Conference Proceeding Series, 499–504. https://doi.org/10.1145/3593434.3593968 [↗](https://doi.org/10.1145/3593434.3593968)
26. Altaher, Y., Fadel, A., Alotaibi, M., Alyazidi, M., Al-Mutairi, M., Aldhbuiub, M., Mosaibah, A., Rezk, A., Alhendi, A., Shal, M. A., Alghamdi, E. A., Alshaibani, M. S., Zakraoui, J., Mohammed, W., Gaanoun, K., Elmadani, K. N., Ghaleb, M., Tazi, N., Alharbi, R., … Alyafeai, Z. (2022). [Masader Plus: A New Interface for Exploring +500 Arabic NLP Datasets](https://md.research.elamur.ai/free/en/papers/altaher-2022-masader-plus-new-interface-exploring/). Preprint ArXiv:2208.00932. https://doi.org/https://doi.org/10.48550/arXiv.2208.00932 [↗](https://doi.org/10.48550/arxiv.2208.00932)
27. Alyafeai, Z., Masoud, M., Ghaleb, M., & Al-Shaibani, M. S. (2022). [Masader: Metadata Sourcing for Arabic Text and Speech Data Resources](https://md.research.elamur.ai/free/en/papers/alyafeai-2022-masader-metadata-sourcing-arabic-text/). 2022 Language Resources and Evaluation Conference, LREC 2022, 6340–6351.
28. Antoun, W., Baly, F., & Hajj, H. (2020). [AraBERT: Transformer-based Model for Arabic Language Understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, May, 9–15. [↗](https://doi.org/10.48550/arxiv.2003.00104)
29. Arora, A., Nakov, P., Hardalov, M., Sarwar, S. M., Nayak, V., Dinkov, Y., Zlatkova, D., Dent, K., Bhatawdekar, A., Bouchard, G., & Augenstein, I. (2023). Detecting Harmful Content on Online Platforms: What Platforms Need vs. Where Research Efforts Go. ACM Computing Surveys, 56(3), 1–17. https://doi.org/10.1145/3603399 [↗](https://doi.org/10.1145/3603399)
30. Artstein, R., & Poesio, M. (2008). [Inter-Coder Agreement for Computational Linguistics](https://md.research.elamur.ai/free/en/papers/artstein-2008-survey-article-inter-coder-agreement/). Computational Linguistics, 34(4), 555–596. [↗](https://doi.org/10.1162/coli.07-034-r2)
31. ASDA’A BCW. (2021). Hope for the future [A White Paper on the findings of the 13th annual ASDA’A BCW Arab Youth Survey]. ASDA’A BCW. [find](https://scholar.google.com/scholar?q=ASDA%E2%80%99A+BCW%2E+%282021%29%2E+Hope+for+the+future+%5BA+White+Paper+on+the+findings+of+the+13th+annual+ASDA%E2%80%99A+BCW+Arab+Youth+Survey%5D%2E+ASDA%E2%80%99A+BCW%2E)
32. Awane, W., Lahmar, E. H. Ben, & Falaki, A. El. (2021). Hate Speech in the Arab Electronic Press and and Knowledge Extraction, 5(1), 29–42. Social Networks. Revue https://doi.org/10.18280/ria.350603 [↗](https://doi.org/10.18280/ria.350603)
33. Badri, N., Kboubi, F., & Habacha Chaibi, A. (2022). Towards Automatic Detection of Inappropriate Content in Multi-dialectic Arabic Text. Advances in Computational Collective Intelligence. ICCCI 2022. Communications in Computer and Information Science, 1653, 84–100. https://doi.org/10.1007/978-3-031-16210-7_7 [↗](https://doi.org/10.1007/978-3-031-16210-7_7)
34. Balayn, A., Yang, J., Szlavik, Z., & Bozzon, A. (2021). Automatic Identification of Harmful, Aggressive, Abusive, and Offensive Language on the Web: A Survey of Technical Biases Informed by Psychology Literature. ACM Transactions on Social Computing, 4(3), 1–56. https://doi.org/10.1145/3479158 [↗](https://doi.org/10.1145/3479158)
35. Banko, M., MacKeen, B., & Ray, L. (2020). A Unified Taxonomy of Harmful Content. Proceedings of the Fourth Workshop on Online Abuse and Harms, 125–137. https://doi.org/10.18653/v1/2020.alw-1.16 [↗](https://doi.org/10.18653/v1/2020.alw-1.16)
36. Bensalem, I., Mout, M., & Rosso, P. (2023). [Offensive Language Detection in Arabizi](https://md.research.elamur.ai/free/en/papers/bensalem-2023-offensive-language-detection-arabizi/). Proceedings of ArabicNLP 2023, 423–434. https://doi.org/10.18653/v1/2023.arabicnlp-1.36 [↗](https://doi.org/10.18653/v1/2023.arabicnlp-1.36)
37. Bergman, A. S., & Diab, M. T. (2022). [Towards Responsible Natural Language Annotation for the Varieties of Arabic](https://md.research.elamur.ai/free/en/papers/bergman-2022-responsible-natural-language-annotation-varieties/). Findings of the Association for Computational Linguistics: ACL 2022, 364– 371. https://doi.org/10.18653/v1/2022.findings-acl.31 [↗](https://doi.org/10.18653/v1/2022.findings-acl.31)
38. Boucherit, O., & Abainia, K. (2022). [Offensive Language Detection in Under-resourced Algerian Dialectal Arabic Language](https://md.research.elamur.ai/free/en/papers/boucherit-2022-offensive-language-detection-under-resourced/) [ArXiv preprint arXiv:2203.10024]. Preprint ArXiv:2203.10024, 1–9. https://doi.org/https://doi.org/10.48550/arXiv.2203.10024 [↗](https://doi.org/10.48550/arxiv.2203.10024)
39. Chhabra, A., & Kumar, D. (2023). A literature survey on multimodal and multilingual automatic hate speech identification. Multimedia Systems, 0123456789. https://doi.org/10.1007/s00530-023-01051-8 [↗](https://doi.org/10.1007/s00530-023-01051-8)
40. Chowdhury, S. A., Mubarak, H., Abdelali, A., Jung, S.-G., Jansen, B. J., & Salminen, J. (2020). [A Multi-Platform Arabic News Comment Dataset for Offensive Language Detection](https://md.research.elamur.ai/free/en/papers/chowdhury-2020-multi-platform-arabic-news-comment/). Proceedings of the 12th Language Resources and Evaluation Conference, 6203–6212.
41. Darwish, K. (2014). [Arabizi Detection and Conversion to Arabic](https://md.research.elamur.ai/free/en/papers/darwish-2014-arabizi-detection-conversion-arabic/). ANLP 2014 - EMNLP 2014 Workshop on Arabic Natural Language Processing, Proceedings, 217–224. https://doi.org/10.3115/V1/W14-3629 [↗](https://doi.org/10.3115/v1/w14-3629)
42. de Paula, A. F. M., Bensalem, I., Rosso, P., & Zaghouani, W. (2023). [Transformers and Ensemble methods: A solution for Hate Speech Detection in Arabic languages](https://md.research.elamur.ai/free/en/papers/paula-2023-transformers-ensemble-methods-solution-hate/). CERIST Natural Language Processing Challenge, March, 29th, 2023, RIST Special Issue, 27(2), 7–12. [↗](https://doi.org/10.48550/arxiv.2303.09823)
43. De Smedt, T., De Pauw, G., & Van Ostaeyen, P. (2018). Automatic Detection of Online Jihadist Hate Speech (Issue February) [CLiPS Technical Report Series (CTRS)]. [find](https://scholar.google.com/scholar?q=De+Smedt%2C+T%2E%2C+De+Pauw%2C+G%2E%2C+%26+Van+Ostaeyen%2C+P%2E+%282018%29%2E+Automatic+Detection+of+Online+Jihadist+Hate+Speech+%28Issue+February%29+%5BCLiPS+Technical+Report+Series+%28CTRS%29%5D%2E)
44. De Smedt, T., Voué, P., Jaki, S., Duffy, E., & El-khouri, L. (2021). A feast for trolls - Engagement analysis of counternarratives against online toxicity [Technical report]. Preprint ArXiv:2111.07188, 1–15. [↗](https://arxiv.org/abs/2111.07188)
45. Elmadany, A., Nagoudi, E. B., & Abdul-Mageed, M. (2023). [ORCA: A Challenging Benchmark for Arabic Language Understanding](https://md.research.elamur.ai/free/en/papers/elmadany-2023-orca-challenging-benchmark-arabic-language/). Findings of the Association for Computational Linguistics: ACL 2023, 9559–9586. https://doi.org/10.18653/v1/2023.findings-acl.609 [↗](https://doi.org/10.18653/v1/2023.findings-acl.609)
46. Elzayady, H., Mohamed, M. S., Badran, K. M., & Salama, G. I. (2022). Detecting Arabic textual threats in social media using artificial intelligence: An overview. Indonesian Journal of Electrical Engineering and Computer Science, 25(3), 1712–1722. https://doi.org/10.11591/ijeecs.v25.i3.pp1712-1722 d’Intelligence Artificielle, 35(6), 457–465. [↗](https://doi.org/10.11591/ijeecs.v25.i3.pp1712-1722)
47. Essefar, K., Ait Baha, H., El Mahdaouy, A., El Mekki, A., & Berrada, I. (2023). OMCD: Offensive Moroccan Comments Dataset. Language Resources and Evaluation, June. https://doi.org/10.1007/s10579-023-09663-2 [↗](https://doi.org/10.1007/s10579-023-09663-2)
48. Fortuna, P., & Nunes, S. (2019). A Survey on Automatic Detection of Hate Speech in Text. ACM Computing Surveys, 51(4), 1–30. https://doi.org/10.1145/3232676 [↗](https://doi.org/10.1145/3232676)
49. Fraiwan, M. (2022). Identification of markers and artificial intelligence-based classification of radical Twitter data. Applied Computing and Informatics, 20170247. https://doi.org/10.1108/aci-12-2021-0326 [↗](https://doi.org/10.1108/aci-12-2021-0326)
50. Garland, J., Ghazi-Zahedi, K., Young, J.-G., Hébert-Dufresne, L., & Galesic, M. (2020). Countering hate on social media: Large scale classification of hate and counter speech. Proceedings of the Fourth Workshop on Online Abuse and Harms, 102–112. https://doi.org/10.18653/v1/2020.alw-1.13 [↗](https://doi.org/10.18653/v1/2020.alw-1.13)
51. Garland, J., Ghazi-Zahedi, K., Young, J. G., Hébert-Dufresne, L., & Galesic, M. (2022). Impact and dynamics of hate and counter speech online. EPJ Data Science, 11(1). https://doi.org/10.1140/epjds/s13688-021-00314-6 [↗](https://doi.org/10.1140/epjds/s13688-021-00314-6)
52. Geiger, R. S., Cope, D., Ip, J., Lotosh, M., Shah, A., Weng, J., & Tang, R. (2021). “Garbage in, garbage out” revisited: What do machine learning application papers report about human-labeled training data? Quantitative Science Studies, 2(3), 795–827. https://doi.org/10.1162/qss_a_00144 [↗](https://doi.org/10.1162/qss_a_00144)
53. Gelber, K., & McNamara, L. (2016). Evidencing the harms of hate speech. Journal for the Study of Race, Nation and Culture, 22(3), 324–341. https://doi.org/10.1080/13504630.2015.1128810 [↗](https://doi.org/10.1080/13504630.2015.1128810)
54. Gharbi, S., Arfaoui, H., Haddad, H., & Kchaou, M. (2021). TEET! Tunisian Dataset for Toxic Speech Detection. Proceedings of the Fifth Workshop on Widening Natural Language Processing, 5–17. [find](https://scholar.google.com/scholar?q=Gharbi%2C+S%2E%2C+Arfaoui%2C+H%2E%2C+Haddad%2C+H%2E%2C+%26+Kchaou%2C+M%2E+%282021%29%2E+TEET%21+Tunisian+Dataset+for+Toxic+Speech+Detection%2E+Proceedings+of+the+Fifth+Workshop+on+Widening+Natural+Language+Processing%2C+5%E2%80%9317%2E)
55. Guellil, I., Adeel, A., Azouaou, F., Boubred, M., Houichi, Y., & Moumna, A. A. (2022). [Ara-Women-Hate: An Annotated Corpus Dedicated to Hate Speech Detection against Women in the Arabic Community](https://md.research.elamur.ai/free/en/papers/guellil-2022-ara-women-hate-annotated-corpus/). Proceedings of the Workshop on Dataset Creation for Lower-Resourced Languages within the 13th Language Resources and Evaluation Conference (LREC), 24(June), 68–75.
56. Habash, M., & Daqour, Y. (2022). Hate Speech and Offense Detection in Arabic Memes [Graduation Project]. Jordan University of Science and Technology. [find](https://scholar.google.com/scholar?q=Habash%2C+M%2E%2C+%26+Daqour%2C+Y%2E+%282022%29%2E+Hate+Speech+and+Offense+Detection+in+Arabic+Memes+%5BGraduation+Project%5D%2E+Jordan+University+of+Science+and+Technology%2E)
57. Habash, N. (2010). Introduction to Arabic Natural Language Processing. Morgan & Claypool. [↗](https://doi.org/10.1007/978-3-031-02139-8)
58. Haddad, H., Mulki, H., & Oueslati, A. (2019). T-HSAB: A Tunisian Hate Speech and Abusive Dataset. International Conference on Arabic Language Processing, December, 251–263. https://doi.org/10.1007/978-3-030-32959-4_18 [↗](https://doi.org/10.1007/978-3-030-32959-4_18)
59. Hadj Ameur, M. S., & Aliane, H. (2021). AraCOVID19-MFH: Arabic COVID-19 Multi-label Fake News & Hate Speech Detection Dataset. Procedia Computer Science, Part of Special Issue: AI in Computational Linguistics, 189, 232–241. https://doi.org/10.1016/j.procs.2021.05.086 [↗](https://doi.org/10.1016/j.procs.2021.05.086)
60. Haghegh, M. (2021). Arabizi across Three Different Generations of Arab Users Living Abroad: A Case Study. Arab World English Journal For Translation and Literary Studies, 5(2), 156–173. https://doi.org/10.24093/awejtls/vol5no2.12 [↗](https://doi.org/10.24093/awejtls/vol5no2.12)
61. Hartvigsen, T., Gabriel, S., Palangi, H., Sap, M., Ray, D., & Kamar, E. (2022). [TOXIGEN: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection](https://md.research.elamur.ai/free/en/papers/hartvigsen-2022-toxigen-large-scale-machine-generated/). Proceedings of the Annual Meeting of the Association for Computational Linguistics, 1, 3309– 3326. https://doi.org/10.18653/v1/2022.acl-long.234 [↗](https://doi.org/10.18653/v1/2022.acl-long.234)
62. Hosseini, H., Kannan, S., Zhang, B., & Poovendran, R. (2017). Deceiving Google’s Perspective API Built for Detecting Toxic Comments. ArXiv Preprint ArXiv:1702.08138, 1–4. [↗](https://arxiv.org/abs/1702.08138)
63. Husain, F., & Uzuner, O. (2021). A Survey of Offensive Language Detection for the Arabic Language. ACM Transactions on Asian and Low-Resource Language Information Processing, 20(1), 1–44. https://doi.org/10.1145/3421504 [↗](https://doi.org/10.1145/3421504)
64. Jahan, M. S., & Oussalah, M. (2023). A systematic review of hate speech automatic detection using natural language processing. Neurocomputing, 546, 126232. https://doi.org/10.1016/j.neucom.2023.126232 [↗](https://doi.org/10.1016/j.neucom.2023.126232)
65. Jamal, A. A., Keohane, R. O., Romney, D., & Tingley, D. (2015). Anti-Americanism and Anti-Interventionism in Arabic Twitter Discourses. Perspectives on Politics, 13(1), 55–73. https://doi.org/10.1017/S1537592714003132 [↗](https://doi.org/10.1017/s1537592714003132)
66. Jansen, T., Tong, Y., Zevallos, V., & Suarez, P. O. (2022). Perplexed by Quality: A Perplexity-based Method for Adult and Harmful Content Detection in Multilingual Heterogeneous Web Data. [find](https://scholar.google.com/scholar?q=Jansen%2C+T%2E%2C+Tong%2C+Y%2E%2C+Zevallos%2C+V%2E%2C+%26+Suarez%2C+P%2E+O%2E+%282022%29%2E+Perplexed+by+Quality%3A+A+Perplexity%2Dbased+Method+for+Adult+and+Harmful+Content+Detection+in+Multilingual+Heterogeneous+Web+Data%2E)
67. Karan, M., & Šnajder, J. (2019). Preemptive Toxic Language Detection in Wikipedia Comments Using Thread-Level Context. Proceedings of the Third Workshop on Abusive Language Online, 129– 134. https://doi.org/10.18653/v1/W19-3514 [↗](https://doi.org/10.18653/v1/w19-3514)
68. Khairy, M., Mahmoud, T. M., & Abd-El-Hafeez, T. (2021). Automatic Detection of Cyberbullying and Abusive Language in Arabic Content on Social Networks: A Survey. Procedia CIRP, 189, 156– 166. https://doi.org/10.1016/j.procs.2021.05.080 [↗](https://doi.org/10.1016/j.procs.2021.05.080)
69. Khairy, M., Mahmoud, T. M., Omar, A., & Abd El-Hafeez, T. (2022). Comparative Performance of Ensemble Machine Learning for Arabic Cyberbullying and Offensive Language Detection. https://doi.org/10.21203/rs.3.rs-1730412/v1 [↗](https://doi.org/10.21203/rs.3.rs-1730412/v1)
70. Kirk, H. R., Birhane, A., Vidgen, B., & Derczynski, L. (2022). Handling and Presenting Harmful Text in NLP Research. Findings of the Association for Computational Linguistics: EMNLP 2022, 497– 510. https://doi.org/10.18653/v1/2022.findings-emnlp.35 [↗](https://doi.org/10.18653/v1/2022.findings-emnlp.35)
71. Kitchenham, B., & Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering. In EBSE Technical Report. [find](https://scholar.google.com/scholar?q=Kitchenham%2C+B%2E%2C+%26+Charters%2C+S%2E+%282007%29%2E+Guidelines+for+performing+Systematic+Literature+Reviews+in+Software+Engineering%2E+In+EBSE+Technical+Report%2E)
72. Kivlichan, I., Sorensen, J., Dixon, L., Vasserman, L., Graham, M., Acosta, T., & Reade, W. (2021). Jigsaw Rate Severity of Toxic Comments. Kaggle. [https://kaggle.com/competitions/jigsaw-toxic-severity-rating](https://kaggle.com/competitions/jigsaw-toxic-severity-rating) [↗](https://kaggle.com/competitions/jigsaw-toxic-severity-rating)
73. Klie, J.-C., de Castilho, R. E., & Gurevych, I. (2023). [Analyzing Dataset Annotation Quality Management in the Wild](https://md.research.elamur.ai/free/en/papers/klie-2024-analyzing-dataset-annotation-quality-management/). [↗](https://doi.org/10.1162/coli_a_00516)
74. Kreutzer, J., Caswell, I., Wang, L., Wahab, A., Van Esch, D., Ulzii-Orshikh, N., Tapo, A., Subramani, N., Sokolov, A., Sikasote, C., Setyawan, M., Sarin, S., Samb, S., Sagot, B., Rivera, C., Rios, A., Papadimitriou, I., Osei, S., Suarez, P. O., … Adeyemi, M. (2022). [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://md.research.elamur.ai/free/en/papers/kreutzer-2021-quality-glance-audit-web-crawled/). Transactions of the Association for Computational Linguistics, 10, 50–72. https://doi.org/10.1162/tacl_a_00447 [↗](https://doi.org/10.1162/tacl_a_00447)
75. Laub, Z. (2019). Hate Speech on Social Media: Global Comparisons. Backgrounder; Council on Foreign Relations (CFR). [https://www.cfr.org/backgrounder/hate-speech-social-media-global-comparisons](https://www.cfr.org/backgrounder/hate-speech-social-media-global-comparisons) [↗](https://www.cfr.org/backgrounder/hate-speech-social-media-global-comparisons)
76. Leonardelli, E., Abercrombie, G., Almanea, D., Basile, V., Fornaciari, T., Plank, B., Rieser, V., Uma, A., & Poesio, M. (2023). [SemEval-2023 Task 11: Learning with Disagreements (LeWiDi)](https://md.research.elamur.ai/free/en/papers/leonardelli-2023-semeval-2023-task-11-learning/). 2304– 2318. https://doi.org/10.18653/v1/2023.semeval-1.314 [↗](https://doi.org/10.18653/v1/2023.semeval-1.314)
77. Mahmud, T., Ptaszynski, M., Eronen, J., & Masui, F. (2023). Cyberbullying detection for low-resource languages and dialects: Review of the state of the art. Information Processing and Management, 60(5), 103454. https://doi.org/10.1016/j.ipm.2023.103454 [↗](https://doi.org/10.1016/j.ipm.2023.103454)
78. Mathew, B., Saha, P., Tharad, H., Rajgaria, S., Singhania, P., Maity, S. K., Goyal, P., & Mukherjee, A. (2019). Thou shalt not hate: Countering online hate speech. Proceedings of the 13th International Conference on Web and Social https://doi.org/10.1609/icwsm.v13i01.3237 [↗](https://doi.org/10.1609/icwsm.v13i01.3237)
79. Mathew, B., Saha, P., Yimam, S. M., Biemann, C., Goyal, P., & Mukherjee, A. (2021). HateXplain : A Benchmark Dataset for Explainable Hate Speech Detection HateXplain : A Benchmark Dataset for Explainable Hate Speech Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 25(17), 14867–14875. [find](https://scholar.google.com/scholar?q=Mathew%2C+B%2E%2C+Saha%2C+P%2E%2C+Yimam%2C+S%2E+M%2E%2C+Biemann%2C+C%2E%2C+Goyal%2C+P%2E%2C+%26+Mukherjee%2C+A%2E+%282021%29%2E+HateXplain+%3A+A+Benchmark+Dataset+for+Explainable+Hate+Speech+Detection+HateXplain+%3A+A+Benchmark+Dataset+for+Explaina)
80. Mazari, A. C., & Kheddar, H. (2023). Deep Learning-based Analysis of Algerian Dialect Dataset Targeted Hate Speech, Offensive Language and Cyberbullying. International Journal of Computing and Digital Systems, 13(1), 965–972. https://doi.org/10.12785/ijcds/130177 [↗](https://doi.org/10.12785/ijcds/130177)
81. Mohdeb, D., Laifa, M., Zerargui, F., & Benzaoui, O. (2022). Evaluating transfer learning approach for detecting Arabic anti-refugee/migrant speech on social media. Aslib Journal of Information Management, 74(6), 1070–1088. https://doi.org/10.1108/AJIM-10-2021-0293 [↗](https://doi.org/10.1108/ajim-10-2021-0293)
82. Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2010). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. International Journal of Surgery, 8(5), 336– 341. https://doi.org/10.1016/j.ijsu.2010.02.007 [↗](https://doi.org/10.1016/j.ijsu.2010.02.007)
83. Mubarak, H., Al-Khalifa, H., & Al-Thubaity, A. (2022). [Overview of OSACT5 Shared Task on Arabic Offensive Language and Hate Speech Detection](https://md.research.elamur.ai/free/en/papers/mubarak-2022-overview-osact5-shared-task-arabic/). Proceedings of the OSACT 2022 Workshop @LREC2022, 162–166.
84. Mubarak, H., Darwish, K., & Magdy, W. (2017). [Abusive Language Detection on Arabic Social Media](https://md.research.elamur.ai/free/en/papers/mubarak-2017-abusive-language-detection-arabic-social/). Proceedings of the First Workshop on Abusive Language Online, 52–56. https://doi.org/10.18653/v1/W17-3008 [↗](https://doi.org/10.18653/v1/w17-3008)
85. Mubarak, H., Darwish, K., Magdy, W., & Al-Khalifa, H. (2020). [Overview of OSACT4 Arabic Offensive Language Detection Shared Task](https://md.research.elamur.ai/free/en/papers/mubarak-2020-overview-osact4-arabic-offensive-language/). Proceedings of the OSACT 2020 Workshop @LREC 2020, 48–52.
86. Mubarak, H., Hassan, S., & Abdelali, A. (2021). Adult Content Detection on Arabic Twitter: Analysis and Experiments. 6th Arabic Natural Language Processing Workshop (WANLP 2021), 136–144. [find](https://scholar.google.com/scholar?q=Mubarak%2C+H%2E%2C+Hassan%2C+S%2E%2C+%26+Abdelali%2C+A%2E+%282021%29%2E+Adult+Content+Detection+on+Arabic+Twitter%3A+Analysis+and+Experiments%2E+6th+Arabic+Natural+Language+Processing+Workshop+%28WANLP+2021%29%2C+136%E2%80%93144%2E)
87. Mubarak, H., Hassan, S., & Chowdhury, S. A. (2022). [Emojis as Anchors to Detect Arabic Offensive Language and Hate Speech](https://md.research.elamur.ai/free/en/papers/mubarak-2022-emojis-anchors-detect-arabic-offensive/). Natural Language Engineering, 1–22. [↗](https://doi.org/10.1017/s1351324923000402)
88. Mubarak, H., Rashed, A., Darwish, K., Samih, Y., & Abdelali, A. (2020). [Arabic Offensive Language on Twitter: Analysis and Experiments](https://md.research.elamur.ai/free/en/papers/mubarak-2021-arabic-offensive-language-twitter-analysis/). WANLP 2021 - 6th Arabic Natural Language Processing Workshop, Proceedings of the Workshop. First Appeared in ArXiv:2004.02192, 126–135. https://doi.org/https://doi.org/10.48550/arXiv.2004.02192 [↗](https://doi.org/10.48550/arxiv.2004.02192)
89. Mulki, H., & Ghanem, B. (2021a). ArMI at FIRE 2021: Overview of the First Shared Task on Arabic Misogyny Identification. Working Notes of FIRE 2021, CEUR Workshop Proceedings Vol. 3159, 820–830. [find](https://scholar.google.com/scholar?q=Mulki%2C+H%2E%2C+%26+Ghanem%2C+B%2E+%282021a%29%2E+ArMI+at+FIRE+2021%3A+Overview+of+the+First+Shared+Task+on+Arabic+Misogyny+Identification%2E+Working+Notes+of+FIRE+2021%2C+CEUR+Workshop+Proceedings+Vol%2E+3159%2C+820%E2%80%93830%2E)
90. Mulki, H., & Ghanem, B. (2021b). [Let-Mi: An Arabic Levantine Twitter Dataset for Misogynistic Language](https://md.research.elamur.ai/free/en/papers/mulki-2021-let-mi-arabic-levantine-twitter/). Proceedings of the Sixth Arabic Natural Language Processing Workshop, 154–163. [↗](https://doi.org/10.48550/arxiv.2103.10195)
91. Mulki, H., & Ghanem, B. (2021c). Working Notes of the Workshop Arabic Misogyny Identification (ArMI-2021). FIRE ’21: Proceedings of the 13th Annual Meeting of the Forum for Information Retrieval Evaluation, 7–8. https://doi.org/10.1145/3503162.3503178 [↗](https://doi.org/10.1145/3503162.3503178)
92. Mulki, H., Haddad, H., Bechikh Ali, C., & Alshabani, H. (2019). [L-HSAB: A Levantine Twitter Dataset for Hate Speech and Abusive Language](https://md.research.elamur.ai/free/en/papers/mulki-2019-l-hsab-levantine-twitter-dataset/). Proceedings of the Third Workshop on Abusive Language Online, 111–118.
93. Nakov, P., Alam, F., Shaar, S., Da San Martino, G., & Zhang, Y. (2021). [A Second Pandemic? Analysis of Fake News About COVID-19 Vaccines in Qatar](https://md.research.elamur.ai/free/en/papers/nakov-2021-second-pandemic-analysis-fake-news/). Proceedings of the Conference Recent Media, ICWSM 2019, Icwsm, 369–380. Advances in Natural Language Processing - Deep Learning for Natural Language Processing Methods and Applications, 1010–1021. https://doi.org/10.26615/978-954-452-072-4_114 [↗](https://doi.org/10.26615/978-954-452-072-4_114)
94. Nakov, P., Barrón-Cedeño, A., Da, G., Martino, S., Alam, F., Míguez, R., Caselli, T., Kutlu, M., Zaghouani, W., Li, C., Shaar, S., Mubarak, H., Nikolov, A., & Kartal, Y. S. (2022). Overview of the CLEF-2022 CheckThat! Lab Task 1 on Identifying Relevant Claims in Tweets. Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum. CEUR Proceedings Vol. 3180. [find](https://scholar.google.com/scholar?q=Nakov%2C+P%2E%2C+Barr%C3%B3n%2DCede%C3%B1o%2C+A%2E%2C+Da%2C+G%2E%2C+Martino%2C+S%2E%2C+Alam%2C+F%2E%2C+M%C3%ADguez%2C+R%2E%2C+Caselli%2C+T%2E%2C+Kutlu%2C+M%2E%2C+Zaghouani%2C+W%2E%2C+Li%2C+C%2E%2C+Shaar%2C+S%2E%2C+Mubarak%2C+H%2E%2C+Nikolov%2C+A%2E%2C+%26+Kartal%2C+Y%2E+S%2E+%282022%29%2E+Overview+of+the+CLE)
95. Obeidat, R., Gharaibeh, M., Abdullah, M., & Alharahsheh, Y. (2022). Multi-label multi-class COVID-19 Arabic Twitter dataset with fine-grained misinformation and situational information annotations. PeerJ Computer Science, 8, e1151. https://doi.org/10.7717/peerj-cs.1151 [↗](https://doi.org/10.7717/peerj-cs.1151)
96. Ousidhoum, N. D., Lin, Z., Zhang, H., Song, Y., & Yeung, D.-Y. (2019). [Multilingual and Multi-Aspect Hate Speech Analysis](https://md.research.elamur.ai/free/en/papers/ousidhoum-2019-multilingual-multi-aspect-hate-speech/). Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 4675–4684. https://doi.org/10.18653/v1/D19-1474 [↗](https://doi.org/10.18653/v1/d19-1474)
97. Ousidhoum, N. D., Zhao, X., Fang, T., Song, Y., & Yeung, D. Y. (2021). Probing Toxic Content in Large Pre-trained Language Models. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 4262–4274. [find](https://scholar.google.com/scholar?q=Ousidhoum%2C+N%2E+D%2E%2C+Zhao%2C+X%2E%2C+Fang%2C+T%2E%2C+Song%2C+Y%2E%2C+%26+Yeung%2C+D%2E+Y%2E+%282021%29%2E+Probing+Toxic+Content+in+Large+Pre%2Dtrained+Language+Models%2E+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computa)
98. Pavlopoulos, J., Sorensen, J., Laugier, L., & Androutsopoulos, I. (2021). [SemEval-2021 Task 5: Toxic Spans Detection](https://md.research.elamur.ai/free/en/papers/pavlopoulos-2021-semeval-2021-task-5-toxic/). SemEval 2021 - 15th International Workshop on Semantic Evaluation, Proceedings of the Workshop, 59–69.
99. Poletto, F., Basile, V., & Sanguinetti, M. (2021). Resources and benchmark corpora for hate speech detection : a systematic review. Language Resources and Evaluation, 55(2), 477–523. https://doi.org/10.1007/s10579-020-09502-8 [↗](https://doi.org/10.1007/s10579-020-09502-8)
100. Raïdy, M., & Harmanani, H. (2023). A Deep Learning Approach for Sentiment and Emotional Analysis of Lebanese Arabizi Twitter Data. In ITNG 2023 20th International Conference on Information Technology-New Generations, Advances in Intelligent Systems and Computing 1445 (pp. 27–35). https://doi.org/10.1007/978-3-031-28332-1_4 [↗](https://doi.org/10.1007/978-3-031-28332-1_4)
101. Reimer, J., Häring, M., Loosen, W., Maalej, W., & Merten, L. (2021). Content Analyses of User Comments in Journalism: A Systematic Literature Review Spanning Communication Studies and Computer Science. Digital Journalism, 0(0), 1–25. https://doi.org/10.1080/21670811.2021.1882868 [↗](https://doi.org/10.1080/21670811.2021.1882868)
102. Riabi, A., Mahamdi, M., & Seddah, D. (2023). Enriching the NArabizi Treebank : A Multifaceted Approach to Supporting an Under-Resourced Language. Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII), 266–278. [find](https://scholar.google.com/scholar?q=Riabi%2C+A%2E%2C+Mahamdi%2C+M%2E%2C+%26+Seddah%2C+D%2E+%282023%29%2E+Enriching+the+NArabizi+Treebank+%3A+A+Multifaceted+Approach+to+Supporting+an+Under%2DResourced+Language%2E+Proceedings+of+the+17th+Linguistic+Annotation+Workshop)
103. Röttger, P., Seelawi, H., Nozza, D., Talat, Z., & Vidgen, B. (2022). Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models. Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH), 154–169. [find](https://scholar.google.com/scholar?q=R%C3%B6ttger%2C+P%2E%2C+Seelawi%2C+H%2E%2C+Nozza%2C+D%2E%2C+Talat%2C+Z%2E%2C+%26+Vidgen%2C+B%2E+%282022%29%2E+Multilingual+HateCheck%3A+Functional+Tests+for+Multilingual+Hate+Speech+Detection+Models%2E+Proceedings+of+the+Sixth+Workshop+on+Online)
104. Rowley, J., & Slack, F. (2004). Conducting a literature review. Management Research News, 27(6), 31– 39. https://doi.org/10.1108/01409170410784185 [↗](https://doi.org/10.1108/01409170410784185)
105. Saketh Aluru, S., Mathew, B., Saha, P., & Mukherjee, A. (2020). Deep Learning Models for Multilingual Hate Speech Detection. Preprint ArXiv:2004.06465, 1–8. [↗](https://arxiv.org/abs/2004.06465)
106. Sanguinetti, M., Poletto, F., Bosco, C., Patti, V., & Stranisci, M. (2019). [An Italian Twitter corpus of hate speech against immigrants](https://md.research.elamur.ai/free/en/papers/sanguinetti-2018-italian-twitter-corpus-hate-speech/). LREC 2018 - 11th International Conference on Language Resources and Evaluation, 2798–2805.
107. Seelawi, H., Tuffaha, I., Gzawi, M., Farhan, W., Talafha, B., Badawi, R., Sober, Z., Al-Dweik, O., Freihat, A. A., & Al-Natsheh, H. T. (2021). [ALUE: Arabic Language Understanding Evaluation](https://md.research.elamur.ai/free/en/papers/seelawi-2021-alue-arabic-language-understanding-evaluation/). WANLP 2021 - 6th Arabic Natural Language Processing Workshop, Proceedings of the Workshop, 173–184.
108. Shaar, S., Alam, F., Da, G., Martino, S., Nikolov, A., Zaghouani, W., Nakov, P., & Feldman, A. (2021). Findings of the NLP4IF-2021 Shared Tasks on Fighting the COVID-19 Infodemic and Censorship Detection. Proceedings Ofthe 4th NLP4IFWorkshop on NLP for Internet Freedom, 82–92. [↗](https://doi.org/10.48550/arxiv.2109.12986)
109. Shannag, F., Hammo, B. H., & Faris, H. (2022). The design, construction and evaluation of annotated Arabic cyberbullying corpus. Education and Information Technologies, 27(8), 10977–11023. https://doi.org/10.1007/s10639-022-11056-x [↗](https://doi.org/10.1007/s10639-022-11056-x)
110. Sharma, S., Agrawal, S., & Shrivastava, M. (2018). Degree based Classification of Harmful Speech using Twitter Data. COLING 2018 - 1st Workshop on Trolling, Aggression and Cyberbullying, TRAC 2018 - Proceedings of the Workshop, 106–112. [↗](https://doi.org/10.48550/arxiv.1806.04197)
111. Siegel, A. A., & Badaan, V. (2020). #No2Sectarianism: Experimental Approaches to Reducing Sectarian Hate Speech Online. American Political Science Review, 114(3), 837–855. https://doi.org/10.1017/S0003055420000283 [↗](https://doi.org/10.1017/s0003055420000283)
112. Stark, B., Stegmann, D., Magin, M., & Jürgens, P. (2020). Are Algorithms a Threat to Democracy? The Rise of Intermediaries: A Challenge for Public Discourse. In Governing Platforms (Issue May). [find](https://scholar.google.com/scholar?q=Stark%2C+B%2E%2C+Stegmann%2C+D%2E%2C+Magin%2C+M%2E%2C+%26+J%C3%BCrgens%2C+P%2E+%282020%29%2E+Are+Algorithms+a+Threat+to+Democracy%3F+The+Rise+of+Intermediaries%3A+A+Challenge+for+Public+Discourse%2E+In+Governing+Platforms+%28Issue+May%29%2E)
113. Uma, A. N., Fornaciari, T., Hovy, D., Paun, S., Plank, B., & Poesio, M. (2021). Learning from Disagreement: A Survey. Journal of Artificial Intelligence Research, 72, 1385–1470. https://doi.org/10.1613/JAIR.1.12752 [↗](https://doi.org/10.1613/jair.1.12752)
114. Vidgen, B., & Derczynski, L. (2020). Directions in abusive language training data, a systematic review: Garbage in, garbage out. PLoS ONE, 15(12 December), 1–32. https://doi.org/10.1371/journal.pone.0243300 [↗](https://doi.org/10.1371/journal.pone.0243300)
115. Vidgen, B., Harris, A., Nguyen, D., Tromble, R., Hale, S., & Margetts, H. (2019). [Challenges and frontiers in abusive content detection](https://md.research.elamur.ai/free/en/papers/vidgen-2019-challenges-frontiers-abusive-content-detection/). Proceedings of the Third Workshop on Abusive Language Online, 80–93. https://doi.org/10.18653/v1/W19-3509 [↗](https://doi.org/10.18653/v1/w19-3509)
116. Wich, M., Eder, T., Al Kuwatly, H., & Groh, G. (2022). Bias and comparison framework for abusive language datasets. AI and Ethics, 2(1), 79–101. https://doi.org/10.1007/s43681-021-00081-0 [↗](https://doi.org/10.1007/s43681-021-00081-0)
117. Zampieri, M., Nakov, P., Rosenthal, S., Atanasova, P., Karadzhov, G., Mubarak, H., Derczynski, L., Pitenis, Z., & Çöltekin, Ç. (2020). SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020). Proceedings of the Fourteenth Workshop on Semantic Evaluation, 1425–1447. [↗](https://doi.org/10.48550/arxiv.2006.07235)

*Updated 1 November 2026*