---
title: "A Multi-Platform Arabic News Comment — Chowdhury 2020"
description: "Most Arabic resources for detecting offensive language come from a single social network, usually Twitter or YouTube."
authors: ["Shammur Absar Chowdhury", "Hamdy Mubarak", "Ahmed Abdelali", "Soon-gyo Jung", "Bernard J. Jansen", "Joni Salminen"]
year: 2020
venue: "Proceedings of the Twelfth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2020.lrec-1.761/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chowdhury-2020-multi-platform-arabic-news-comment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chowdhury-2020-multi-platform-arabic-news-comment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/) / A Multi-Platform Arabic News Comment Dataset for Offensive Language Detection

# A Multi-Platform Arabic News Comment Dataset for Offensive Language Detection

**Authors:** [Shammur Absar Chowdhury](https://md.research.elamur.ai/free/en/authors/shammur-absar-chowdhury/), [Hamdy Mubarak](https://md.research.elamur.ai/free/en/authors/hamdy-mubarak/), [Ahmed Abdelali](https://md.research.elamur.ai/free/en/authors/ahmed-abdelali/), [Soon-gyo Jung](https://md.research.elamur.ai/free/en/authors/soon-gyo-jung/), Bernard J. Jansen, Joni Salminen · **Year:** 2020 · **Venue:** Proceedings of the Twelfth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.lrec-1.761/) · [PDF](https://aclanthology.org/2020.lrec-1.761.pdf) · [OpenAlex](https://openalex.org/W3030257662)

**Citations:** 34 citations (openalex-snapshot · 2026-10-14) · [30](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Most Arabic resources for detecting offensive language come from a single social network, usually Twitter or YouTube. Abusive commenting, however, happens on every platform. The authors build a dialectal Arabic dataset of news comments drawn from three platforms (Twitter, Facebook and YouTube), all posted under the accounts of one international news agency. They chose news because the posts themselves are not offensive, so any hostility in the replies reflects the commenters' own views.

From more than 100k collected comments they sampled 4000 and labelled them through crowdsourcing. Annotators had to pass a two-stage selection process, which matters for an under-represented language. Label quality was checked in two ways: agreement among the accepted workers, and comparison with expert annotations.

The paper also studies which words and emojis mark offensive comments. SVM classifiers trained on this data and on other public Arabic sets show that multi-platform training data helps models generalize across platforms, domains and dialects (Egyptian, Levantine). The dataset is publicly released.

## Key points
- The corpus contains 4000 annotated comments from Twitter, Facebook and YouTube. They were sampled from over 100k comments on posts a news agency published between 2011 and 2019. The data is released on GitHub (Arabic-Offensive-Multi-Platform-SocialMedia-Comment-Dataset).
- Twitter's Standard Search API only returns replies from the past 7 days. To get the full reply history, the authors re-queried every 6 hours.
- Filtering and cleaning steps:
  - comments needed at least 5 Arabic tokens, not counting emojis;
  - user mentions were replaced with `USER.IDX` and links with `URL`;
  - duplicate texts were removed.
- Annotation used a two-step crowd-worker screening process. Quality was measured by inter-annotator agreement and by accuracy against expert labels.
- SVM experiments covered in-domain and out-of-domain data, cross-platform tests and Egyptian/Levantine dialect sets. The external datasets were:
  - Mubarak et al. 2017: about 1.1k Egyptian tweets, plus about 32k deleted Aljazeera.net comments;
  - Mulki et al. 2019: L-HSAB, 5.8k Levantine tweets.

## Interesting ideas
- The source posts were chosen to be neutral, so offensiveness can be credited to the replying user rather than the content they react to; this kind of clean attribution makes analysis easier in any stance study. — For Mova (frames): A test set of neutral prompts with charged replies would let the speech-act and irony modules be checked on whether they assign the stance to the right speaker.
- Crowd workers had to pass a screening stage, and their labels were then measured against an expert sample, so cheap annotators earned trust before their labels were used. — For Mova (learn): LLM teacher tiers could go through the same qualification against a small gold set before their silver labels are allowed into training.
- Generalization was tested by training on one platform or dialect and evaluating on another, which reveals models that only fit one source's style. — For Mova (learn): Holding out an entire genre (tweets, fables, textbooks) when evaluating the parsers would serve as a negative control against genre overfitting.
- Emojis were kept and analysed as signals of offensiveness instead of being stripped as noise, since they carry meaning in informal text. — For Mova (tokens): The tokenizer and tagger could give emojis their own token class and lemma, so later pragmatic stages can use them as evaluative cues.
- Mentions were anonymized with indexed placeholders (`USER.IDX`), which hides identities but keeps track of which person is which. — For Mova (frames): Coreference could rely on indexed placeholders like these when anonymizing input, so frames still distinguish participants without exposing names.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cited by

Cited by

30 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 23 not yet.

- [Exploratory Arabic Offensive Language Dataset Analysis](https://md.research.elamur.ai/free/en/papers/husain-2021-exploratory-arabic-offensive-language-dataset/)
- [Estimating the Level of Dialectness Predicts Interannotator Agreement in Multi-dialect Arabic Datasets](https://md.research.elamur.ai/free/en/papers/keleg-2024-estimating-level-dialectness-predicts-interannotator/)
- [CultureLLM: Incorporating Cultural Differences into Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-culturellm-incorporating-cultural-differences-large/)
- [ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccination](https://md.research.elamur.ai/free/en/papers/mubarak-2022-arcovidvac-analyzing-arabic-tweets-about/)
- [Emojis as Anchors to Detect Arabic Offensive Language and Hate Speech](https://md.research.elamur.ai/free/en/papers/mubarak-2022-emojis-anchors-detect-arabic-offensive/)
- [Quranic Audio Dataset: Crowdsourced and Labeled Recitation from Non-Arabic Speakers](https://md.research.elamur.ai/free/en/papers/salameh-2024-quranic-audio-dataset-crowdsourced-labeled/)
- [ArabDiscrim: A Decade-Long Arabic Facebook Corpus on Racism and Discrimination](https://md.research.elamur.ai/free/en/papers/zaghouani-2026-arabdiscrim-decade-long-arabic-facebook/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{chowdhury2020multi,
  title = {{A Multi-Platform Arabic News Comment Dataset for Offensive Language Detection}},
  author = {Chowdhury, Shammur Absar and Mubarak, Hamdy and Abdelali, Ahmed and Jung, Soon-gyo and Jansen, Bernard J. and Salminen, Joni},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.761/}
}
```

- [← previous A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages](https://md.research.elamur.ai/free/en/papers/suarez-2020-monolingual-approach-contextualized-word-embeddings/)
- [next → A Multitask Learning Approach for Diacritic Restoration](https://md.research.elamur.ai/free/en/papers/acl-2020-alqahtani-multitask-learning-approach-diacritic-restoration/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/)

## References

35 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Agarwal, S. and Sureka, A. (2014). A focused crawler for mining hate and extremism promoting videos on youtube. In Proceedings of the 25th ACM conference on Hypertext and social media, pages 294–296. ACM. [find](https://scholar.google.com/scholar?q=Agarwal%2C+S%2E+and+Sureka%2C+A%2E+%282014%29%2E+A+focused+crawler+for+mining+hate+and+extremism+promoting+videos+on+youtube%2E+In+Proceedings+of+the+25th+ACM+conference+on+Hypertext+and+social+media%2C+pages+294%E2%80%93296%2E+)
2. Al-Ajlan, M. A. and Ykhlef, M. (2018). Optimized twitter cyberbullying detection based on deep learning. In 2018 21st Saudi Computer Society National Computer Conference (NCC), pages 1–5. IEEE. [find](https://scholar.google.com/scholar?q=Al%2DAjlan%2C+M%2E+A%2E+and+Ykhlef%2C+M%2E+%282018%29%2E+Optimized+twitter+cyberbullying+detection+based+on+deep+learning%2E+In+2018+21st+Saudi+Computer+Society+National+Computer+Conference+%28NCC%29%2C+pages+1%E2%80%935%2E+IEEE%2E)
3. Alakrot, A., Murray, L., and Nikolov, N. S. (2018). Dataset construction for the detection of anti-social behaviour in online communication in arabic. Procedia Computer Science, 142:174–181. [find](https://scholar.google.com/scholar?q=Alakrot%2C+A%2E%2C+Murray%2C+L%2E%2C+and+Nikolov%2C+N%2E+S%2E+%282018%29%2E+Dataset+construction+for+the+detection+of+anti%2Dsocial+behaviour+in+online+communication+in+arabic%2E+Procedia+Computer+Science%2C+142%3A174%E2%80%93181%2E)
4. Albadi, N., Kurdi, M., and Mishra, S. (2018). Are they our brothers? analysis and detection of religious hate speech in the arabic twittersphere. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 69–76. IEEE. [find](https://scholar.google.com/scholar?q=Albadi%2C+N%2E%2C+Kurdi%2C+M%2E%2C+and+Mishra%2C+S%2E+%282018%29%2E+Are+they+our+brothers%3F+analysis+and+detection+of+religious+hate+speech+in+the+arabic+twittersphere%2E+In+2018+IEEE%2FACM+International+Conference+on+Advances+)
5. Badjatiya, P., Gupta, S., Gupta, M., and Varma, V. (2017). Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion, pages 759–760. International World Wide Web Conferences Steering Committee. [find](https://scholar.google.com/scholar?q=Badjatiya%2C+P%2E%2C+Gupta%2C+S%2E%2C+Gupta%2C+M%2E%2C+and+Varma%2C+V%2E+%282017%29%2E+Deep+learning+for+hate+speech+detection+in+tweets%2E+In+Proceedings+of+the+26th+International+Conference+on+World+Wide+Web+Companion%2C+pages+759)
6. Chatzakou, D., Kourtellis, N., Blackburn, J., De Cristofaro, E., Stringhini, G., and Vakali, A. (2017a). Hate is not binary: Studying abusive behavior of# gamergate on twitter. In Proceedings of the 28th ACM conference on hypertext and social media, pages 65–74. ACM. [find](https://scholar.google.com/scholar?q=Chatzakou%2C+D%2E%2C+Kourtellis%2C+N%2E%2C+Blackburn%2C+J%2E%2C+De+Cristofaro%2C+E%2E%2C+Stringhini%2C+G%2E%2C+and+Vakali%2C+A%2E+%282017a%29%2E+Hate+is+not+binary%3A+Studying+abusive+behavior+of%23+gamergate+on+twitter%2E+In+Proceedings+of+the+2)
7. Chatzakou, D., Kourtellis, N., Blackburn, J., De Cristofaro, E., Stringhini, G., and Vakali, A. (2017b). Mean birds: Detecting aggression and bullying on twitter. In Proceedings of the 2017 ACM on web science conference, pages 13–22. ACM. [find](https://scholar.google.com/scholar?q=Chatzakou%2C+D%2E%2C+Kourtellis%2C+N%2E%2C+Blackburn%2C+J%2E%2C+De+Cristofaro%2C+E%2E%2C+Stringhini%2C+G%2E%2C+and+Vakali%2C+A%2E+%282017b%29%2E+Mean+birds%3A+Detecting+aggression+and+bullying+on+twitter%2E+In+Proceedings+of+the+2017+ACM+on+web)
8. Chatzakou, D., Kourtellis, N., Blackburn, J., De Cristofaro, E., Stringhini, G., and Vakali, A. (2017c). Measuring# gamergate: A tale of hate, sexism, and bullying. In Proceedings of the 26th international conference on world wide web companion, pages 1285–1290. International World Wide Web Conferences Steering Committee. [find](https://scholar.google.com/scholar?q=Chatzakou%2C+D%2E%2C+Kourtellis%2C+N%2E%2C+Blackburn%2C+J%2E%2C+De+Cristofaro%2C+E%2E%2C+Stringhini%2C+G%2E%2C+and+Vakali%2C+A%2E+%282017c%29%2E+Measuring%23+gamergate%3A+A+tale+of+hate%2C+sexism%2C+and+bullying%2E+In+Proceedings+of+the+26th+internat)
9. Conover, M. D., Ratkiewicz, J., Francisco, M., Gonçalves, B., Menczer, F., and Flammini, A. (2011). Political polarization on twitter. In Fifth international AAAI conference on weblogs and social media. [find](https://scholar.google.com/scholar?q=Conover%2C+M%2E+D%2E%2C+Ratkiewicz%2C+J%2E%2C+Francisco%2C+M%2E%2C+Gonc%CC%A7alves%2C+B%2E%2C+Menczer%2C+F%2E%2C+and+Flammini%2C+A%2E+%282011%29%2E+Political+polarization+on+twitter%2E+In+Fifth+international+AAAI+conference+on+weblogs+and+social+med)
10. Dadvar, M., Trieschnigg, D., Ordelman, R., and de Jong, F. (2013). Improving cyberbullying detection with user context. In European Conference on Information Retrieval, pages 693–696. Springer. [find](https://scholar.google.com/scholar?q=Dadvar%2C+M%2E%2C+Trieschnigg%2C+D%2E%2C+Ordelman%2C+R%2E%2C+and+de+Jong%2C+F%2E+%282013%29%2E+Improving+cyberbullying+detection+with+user+context%2E+In+European+Conference+on+Information+Retrieval%2C+pages+693%E2%80%93696%2E+Springer%2E)
11. Davidson, T., Warmsley, D., Macy, M., and Weber, I. (2017). Automated hate speech detection and the problem of offensive language. In Eleventh international aaai conference on web and social media. [find](https://scholar.google.com/scholar?q=Davidson%2C+T%2E%2C+Warmsley%2C+D%2E%2C+Macy%2C+M%2E%2C+and+Weber%2C+I%2E+%282017%29%2E+Automated+hate+speech+detection+and+the+problem+of+offensive+language%2E+In+Eleventh+international+aaai+conference+on+web+and+social+media%2E) [our card](https://md.research.elamur.ai/free/en/papers/davidson-2017-automated-hate-speech-detection-problem/)
12. Difallah, D., Filatova, E., and Ipeirotis, P. (2018). Demographics and dynamics of mechanical turk workers. In Proceedings of the eleventh acm international conference on web search and data mining, pages 135–143. ACM. [find](https://scholar.google.com/scholar?q=Difallah%2C+D%2E%2C+Filatova%2C+E%2E%2C+and+Ipeirotis%2C+P%2E+%282018%29%2E+Demographics+and+dynamics+of+mechanical+turk+workers%2E+In+Proceedings+of+the+eleventh+acm+international+conference+on+web+search+and+data+mining%2C+p)
13. ElSherief, M., Nilizadeh, S., Nguyen, D., Vigna, G., and Belding, E. (2018). Peer to peer hate: Hate speech instigators and their targets. In Twelfth International AAAI Conference on Web and Social Media. [find](https://scholar.google.com/scholar?q=ElSherief%2C+M%2E%2C+Nilizadeh%2C+S%2E%2C+Nguyen%2C+D%2E%2C+Vigna%2C+G%2E%2C+and+Belding%2C+E%2E+%282018%29%2E+Peer+to+peer+hate%3A+Hate+speech+instigators+and+their+targets%2E+In+Twelfth+International+AAAI+Conference+on+Web+and+Social+Me)
14. Falotico, R. and Quatto, P. (2015). Fleiss’ kappa statistic without paradoxes. Quality & Quantity, 49(2):463–470. [find](https://scholar.google.com/scholar?q=Falotico%2C+R%2E+and+Quatto%2C+P%2E+%282015%29%2E+Fleiss%E2%80%99+kappa+statistic+without+paradoxes%2E+Quality+%26+Quantity%2C+49%282%29%3A463%E2%80%93470%2E)
15. Gülaçtı, F. (2010). The effect of perceived social support on subjective well-being. Procedia-Social and Behavioral Sciences, 2(2):3844–3849. [find](https://scholar.google.com/scholar?q=Gu%CC%88lac%CC%A7t%C4%B1%2C+F%2E+%282010%29%2E+The+effect+of+perceived+social+support+on+subjective+well%2Dbeing%2E+Procedia%2DSocial+and+Behavioral+Sciences%2C+2%282%29%3A3844%E2%80%933849%2E)
16. Hosseinmardi, H., Mattson, S. A., Rafiq, R. I., Han, R., Lv, Q., and Mishra, S. (2015). Analyzing labeled cyberbullying incidents on the instagram social network. In International conference on social informatics, pages 49–66. Springer. [find](https://scholar.google.com/scholar?q=Hosseinmardi%2C+H%2E%2C+Mattson%2C+S%2E+A%2E%2C+Rafiq%2C+R%2E+I%2E%2C+Han%2C+R%2E%2C+Lv%2C+Q%2E%2C+and+Mishra%2C+S%2E+%282015%29%2E+Analyzing+labeled+cyberbullying+incidents+on+the+instagram+social+network%2E+In+International+conference+on+social)
17. Intapong, P., Charoenpit, S., Achalakul, T., and Ohkura, M. (2017). Assessing symptoms of excessive sns usage based on user behavior and emotion: analysis of data obtained by sns apis. In 9th International Conference on Social Computing and Social Media, SCSM 2017 held as part of the 19th International Conference on Human-Computer Interaction, HCI International 2017, pages 71–83. Springer Verlag. [find](https://scholar.google.com/scholar?q=Intapong%2C+P%2E%2C+Charoenpit%2C+S%2E%2C+Achalakul%2C+T%2E%2C+and+Ohkura%2C+M%2E+%282017%29%2E+Assessing+symptoms+of+excessive+sns+usage+based+on+user+behavior+and+emotion%3A+analysis+of+data+obtained+by+sns+apis%2E+In+9th+Internat)
18. Jay, T. and Janschewitz, K. (2008). The pragmatics of swearing. Journal of Politeness Research. Language, Behaviour, Culture, 4(2):267–288. [find](https://scholar.google.com/scholar?q=Jay%2C+T%2E+and+Janschewitz%2C+K%2E+%282008%29%2E+The+pragmatics+of+swearing%2E+Journal+of+Politeness+Research%2E+Language%2C+Behaviour%2C+Culture%2C+4%282%29%3A267%E2%80%93288%2E)
19. Kumar, S., Hamilton, W. L., Leskovec, J., and Jurafsky, D. (2018). Community interaction and conflict on the web. In Proceedings of the 2018 World Wide Web Conference, pages 933–943. International World Wide Web Conferences Steering Committee. [find](https://scholar.google.com/scholar?q=Kumar%2C+S%2E%2C+Hamilton%2C+W%2E+L%2E%2C+Leskovec%2C+J%2E%2C+and+Jurafsky%2C+D%2E+%282018%29%2E+Community+interaction+and+conflict+on+the+web%2E+In+Proceedings+of+the+2018+World+Wide+Web+Conference%2C+pages+933%E2%80%93943%2E+International+Wor)
20. Mondal, M., Silva, L. A., and Benevenuto, F. (2017). A measurement study of hate speech in social media. In Proceedings of the 28th ACM Conference on Hypertext and Social Media, pages 85–94. ACM. [find](https://scholar.google.com/scholar?q=Mondal%2C+M%2E%2C+Silva%2C+L%2E+A%2E%2C+and+Benevenuto%2C+F%2E+%282017%29%2E+A+measurement+study+of+hate+speech+in+social+media%2E+In+Proceedings+of+the+28th+ACM+Conference+on+Hypertext+and+Social+Media%2C+pages+85%E2%80%9394%2E+ACM%2E)
21. Mubarak, H. and Darwish, K. (2019). Arabic offensive language classification on twitter. In International Conference on Social Informatics, pages 269–276. Springer. [find](https://scholar.google.com/scholar?q=Mubarak%2C+H%2E+and+Darwish%2C+K%2E+%282019%29%2E+Arabic+offensive+language+classification+on+twitter%2E+In+International+Conference+on+Social+Informatics%2C+pages+269%E2%80%93276%2E+Springer%2E)
22. Mubarak, H., Darwish, K., and Magdy, W. (2017). Abusive language detection on arabic social media. In Proceedings of the First Workshop on Abusive Language Online, pages 52–56. [find](https://scholar.google.com/scholar?q=Mubarak%2C+H%2E%2C+Darwish%2C+K%2E%2C+and+Magdy%2C+W%2E+%282017%29%2E+Abusive+language+detection+on+arabic+social+media%2E+In+Proceedings+of+the+First+Workshop+on+Abusive+Language+Online%2C+pages+52%E2%80%9356%2E) [our card](https://md.research.elamur.ai/free/en/papers/mubarak-2017-abusive-language-detection-arabic-social/)
23. Mulki, H., Haddad, H., Ali, C. B., and Alshabani, H. (2019). L-hsab: A levantine twitter dataset for hate speech and abusive language. In Proceedings of the Third Workshop on Abusive Language Online, pages 111–118. [find](https://scholar.google.com/scholar?q=Mulki%2C+H%2E%2C+Haddad%2C+H%2E%2C+Ali%2C+C%2E+B%2E%2C+and+Alshabani%2C+H%2E+%282019%29%2E+L%2Dhsab%3A+A+levantine+twitter+dataset+for+hate+speech+and+abusive+language%2E+In+Proceedings+of+the+Third+Workshop+on+Abusive+Language+Online%2C+)
24. Pavlopoulos, J., Malakasiotis, P., and Androutsopoulos, I. (2017). Deeper attention to abusive user content moderation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1125–1135. [find](https://scholar.google.com/scholar?q=Pavlopoulos%2C+J%2E%2C+Malakasiotis%2C+P%2E%2C+and+Androutsopoulos%2C+I%2E+%282017%29%2E+Deeper+attention+to+abusive+user+content+moderation%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+P)
25. Pitsilis, G. K., Ramampiaro, H., and Langseth, H. (2018). Detecting offensive language in tweets using deep learning. arXiv preprint arXiv:1801.04433. [↗](https://arxiv.org/abs/1801.04433)
26. Platt, J. (1998). Sequential minimal optimization: A fast algorithm for training support vector machines. [find](https://scholar.google.com/scholar?q=Platt%2C+J%2E+%281998%29%2E+Sequential+minimal+optimization%3A+A+fast+algorithm+for+training+support+vector+machines%2E)
27. Rish, I. et al. (2001). An empirical study of the naive bayes classifier. In IJCAI 2001 workshop on empirical methods in artificial intelligence, volume 3, pages 41–46. [find](https://scholar.google.com/scholar?q=Rish%2C+I%2E+et+al%2E+%282001%29%2E+An+empirical+study+of+the+naive+bayes+classifier%2E+In+IJCAI+2001+workshop+on+empirical+methods+in+artificial+intelligence%2C+volume+3%2C+pages+41%E2%80%9346%2E)
28. Ross, J., Zaldivar, A., Irani, L., and Tomlinson, B. (2009). Who are the turkers? worker demographics in amazon mechanical turk. Department of Informatics, University of California, Irvine, USA, Tech. Rep. [find](https://scholar.google.com/scholar?q=Ross%2C+J%2E%2C+Zaldivar%2C+A%2E%2C+Irani%2C+L%2E%2C+and+Tomlinson%2C+B%2E+%282009%29%2E+Who+are+the+turkers%3F+worker+demographics+in+amazon+mechanical+turk%2E+Department+of+Informatics%2C+University+of+California%2C+Irvine%2C+USA%2C+Tech%2E)
29. Salminen, J., Almerekhi, H., Milenkovic, M., Jung, S.-g., An, J., Kwak, H., and Jansen, B. J. (2018). Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for Identifying and Classifying Hate in Online News Media. In Proceedings of The International AAAI Conference on Web and Social Media (ICWSM 2018), San Francisco, California, USA, June. [find](https://scholar.google.com/scholar?q=Salminen%2C+J%2E%2C+Almerekhi%2C+H%2E%2C+Milenkovic%2C+M%2E%2C+Jung%2C+S%2E%2Dg%2E%2C+An%2C+J%2E%2C+Kwak%2C+H%2E%2C+and+Jansen%2C+B%2E+J%2E+%282018%29%2E+Anatomy+of+Online+Hate%3A+Developing+a+Taxonomy+and+Machine+Learning+Models+for+Identifying+and+Clas)
30. Salminen, J., Hopf, M., Chowdhury, S. A., Jung, S.-g., Almerekhi, H., and Jansen, B. J. (2020). Developing an online hate classifier for multiple social media platforms. Human-centric Computing and Information Sciences, 10(1). [find](https://scholar.google.com/scholar?q=Salminen%2C+J%2E%2C+Hopf%2C+M%2E%2C+Chowdhury%2C+S%2E+A%2E%2C+Jung%2C+S%2E%2Dg%2E%2C+Almerekhi%2C+H%2E%2C+and+Jansen%2C+B%2E+J%2E+%282020%29%2E+Developing+an+online+hate+classifier+for+multiple+social+media+platforms%2E+Human%2Dcentric+Computing+and+In)
31. Silva, L., Mondal, M., Correa, D., Benevenuto, F., and Weber, I. (2016). Analyzing the targets of hate in online social media. In Tenth International AAAI Conference on Web and Social Media. [find](https://scholar.google.com/scholar?q=Silva%2C+L%2E%2C+Mondal%2C+M%2E%2C+Correa%2C+D%2E%2C+Benevenuto%2C+F%2E%2C+and+Weber%2C+I%2E+%282016%29%2E+Analyzing+the+targets+of+hate+in+online+social+media%2E+In+Tenth+International+AAAI+Conference+on+Web+and+Social+Media%2E)
32. Unsvåg, E. F. and Gambäck, B. (2018). The effects of user features on twitter hate speech detection. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pages 75–85. [find](https://scholar.google.com/scholar?q=Unsva%CC%8Ag%2C+E%2E+F%2E+and+Gamba%CC%88ck%2C+B%2E+%282018%29%2E+The+effects+of+user+features+on+twitter+hate+speech+detection%2E+In+Proceedings+of+the+2nd+Workshop+on+Abusive+Language+Online+%28ALW2%29%2C+pages+75%E2%80%9385%2E)
33. Waldron, J. (2012). The harm in hate speech. Harvard University Press. [find](https://scholar.google.com/scholar?q=Waldron%2C+J%2E+%282012%29%2E+The+harm+in+hate+speech%2E+Harvard+University+Press%2E)
34. Wiegand, M., Siegel, M., and Ruppenhofer, J. (2018). Overview of the germeval 2018 shared task on the identification of offensive language. [find](https://scholar.google.com/scholar?q=Wiegand%2C+M%2E%2C+Siegel%2C+M%2E%2C+and+Ruppenhofer%2C+J%2E+%282018%29%2E+Overview+of+the+germeval+2018+shared+task+on+the+identification+of+offensive+language%2E)
35. Zhang, Z., Robinson, D., and Tepper, J. (2018). Detecting hate speech on twitter using a convolution-gru based deep neural network. In European Semantic Web Conference, pages 745–760. Springer. [find](https://scholar.google.com/scholar?q=Zhang%2C+Z%2E%2C+Robinson%2C+D%2E%2C+and+Tepper%2C+J%2E+%282018%29%2E+Detecting+hate+speech+on+twitter+using+a+convolution%2Dgru+based+deep+neural+network%2E+In+European+Semantic+Web+Conference%2C+pages+745%E2%80%93760%2E+Springer%2E)