---
title: "Arabic Offensive Language on Twitter — Mubarak 2021"
description: "Only about 1–2% of Arabic tweets are offensive, so annotating random samples to build training data is very inefficient."
authors: ["Hamdy Mubarak", "Ammar Rashed", "Kareem Darwish", "Younes Samih", "Ahmed Abdelali"]
year: 2021
venue: "Proceedings of the Sixth Arabic Natural Language Processing Workshop"
source: "https://aclanthology.org/2021.wanlp-1.13/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mubarak-2021-arabic-offensive-language-twitter-analysis/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mubarak-2021-arabic-offensive-language-twitter-analysis/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/) / Arabic Offensive Language on Twitter: Analysis and Experiments

# Arabic Offensive Language on Twitter: Analysis and Experiments

**Authors:** [Hamdy Mubarak](https://md.research.elamur.ai/free/en/authors/hamdy-mubarak/), Ammar Rashed, [Kareem Darwish](https://md.research.elamur.ai/free/en/authors/kareem-darwish/), [Younes Samih](https://md.research.elamur.ai/free/en/authors/younes-samih/), [Ahmed Abdelali](https://md.research.elamur.ai/free/en/authors/ahmed-abdelali/) · **Year:** 2021 · **Venue:** Proceedings of the Sixth Arabic Natural Language Processing Workshop · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.wanlp-1.13/) · [PDF](https://aclanthology.org/2021.wanlp-1.13.pdf) · [OpenAlex](https://openalex.org/W3014459433)

**Citations:** 83 citations (openalex-snapshot · 2026-10-14) · [82](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Only about 1–2% of Arabic tweets are offensive, so annotating random samples to build training data is very inefficient. Filtering with a seed list of offensive words would raise the yield but skew the data toward particular dialects, topics or targets. The authors avoid this by selecting tweets that contain the Arabic vocative particle "yA" ("O") at least twice. People use this construction to address someone directly, and it appears in every dialect and on any subject. From 660k matching tweets, they kept one random tweet per frequent unique word sequence between the particles and annotated 10,000 tweets. Labels follow guidelines compatible with OffensEval2019: offensive, with optional vulgar and/or hate speech sub-tags, or clean. About 19% of the resulting set is offensive. The paper describes this as the largest Arabic offensive-language dataset to date, and the authors also analyse its topics, dialects and author gender. In classification experiments, fine-tuned AraBERT performed best (F1 = 83.2), with Mazajak embeddings plus an SVM close behind. The resource gives Arabic offensive-language detection a sizeable benchmark that is less biased by dialect or topic.

## Key points
- **Data collection**
  - 660k Arabic tweets with the "yA … yA" pattern were gathered via the Twitter API between April 15 and May 6, 2019.
  - The 10,000 most frequent unique in-between word sequences were kept, with one random tweet per sequence.
  - 1,915 of these tweets (~19%) were labeled offensive.
- **Yield of the vocative filter:** the offensive share rises from 1–2% in random tweets to about 5% with one vocative particle and about 19% with two or more.
- **Label scheme:**
  - The labels are OFF, VLG, HS and CLN, and vulgarity and hate speech can overlap.
  - Insults aimed at sports-club fans count as hate speech, alongside insults based on race, ethnicity, political group and religion.
- **Classification results** (5-fold cross-validation, identical folds for all runs):

  | Model | F1 |
  |---|---|
  | AraBERT | 83.2 |
  | Mazajak/SVM | 79.7 |
  | AraVec/SVM | 76.4 |
  | BERT base-multilingual | 76.0 |
  | fastText trained on the data | 74.4 |
  | Lexical-feature SVM | 46.6 |

  An RBF-kernel SVM beat seven other classifiers when using Mazajak embeddings.
- **Error analysis** (one fold, Mazajak/SVM: 36 false positives, 121 false negatives):
  - False positives came from gloating, quoting someone else's insult, idioms and sarcasm.
  - False negatives came from insults mixed with admiration and from implicit offensiveness.

## Interesting ideas
- **Use a neutral structural cue to oversample a rare class.** Instead of a lexicon of bad words, filter on a grammatical marker associated with addressing someone, which is independent of topic and dialect. Any rare-phenomenon annotation project could look for similar markers to raise yield without baking a keyword list's biases into the data.
- **Deduplicate by pattern content to increase diversity.** Keeping one tweet per unique word sequence between the particles stops a few popular phrases from dominating the sample. This is a cheap diversity trick for any pattern-mined corpus.
- **Use hierarchical, overlapping labels.** Treating vulgarity and hate speech as non-exclusive sub-tags of "offensive" keeps one simple binary task while still allowing finer analyses. This suits taxonomies where categories genuinely co-occur.
- **Adapt guideline categories to the culture.** Counting attacks on sports-club fans as hate speech reflects how these identities work in the target culture. It is a reminder to revise imported annotation guidelines to fit local social realities.
- **Weigh domain match against model size, and run a typed error analysis.** In-domain static embeddings with an SVM outperformed multilingual BERT, which suggests the data domain can matter more than model complexity. Grouping errors into types such as quoting, idioms and sarcasm also points to where context-aware modeling is needed.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

82 papers of our library cite it (of the 140884 whose bibliographies we read); 10 of them have a page here, 72 not yet.

- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://md.research.elamur.ai/free/en/papers/abdelali-2021-pre-training-bert-arabic-tweets/)
- [GHAD NLP at AdabEval2026: Transformer-Based Approach for Arabic Politeness and Pragmatic Category Classification](https://md.research.elamur.ai/free/en/papers/alfattni-2026-ghad-nlp-adabeval2026-transformer-based/)
- [ArMIS - The Arabic Misogyny and Sexism Corpus with Annotator Subjective Disagreements](https://md.research.elamur.ai/free/en/papers/almanea-2022-armis-arabic-misogyny-sexism-corpus/)
- [SalamNET at SemEval-2020 Task12: Deep Learning Approach for Arabic Offensive Language Detection](https://md.research.elamur.ai/free/en/papers/husain-2020-salamnet-semeval-2020-task12-deep/)
- [Exploratory Arabic Offensive Language Dataset Analysis](https://md.research.elamur.ai/free/en/papers/husain-2021-exploratory-arabic-offensive-language-dataset/)
- [Offensive Hebrew Corpus and Detection using BERT](https://md.research.elamur.ai/free/en/papers/langstream-2023-hamad-offensive-hebrew-corpus-detection-using/)
- [Automatic Expansion and Retargeting of Arabic Offensive Language Training](https://md.research.elamur.ai/free/en/papers/mubarak-2021-automatic-expansion-retargeting-arabic-offensive/)
- [Emojis as Anchors to Detect Arabic Offensive Language and Hate Speech](https://md.research.elamur.ai/free/en/papers/mubarak-2022-emojis-anchors-detect-arabic-offensive/)
- [KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media](https://md.research.elamur.ai/free/en/papers/safaya-2020-kuisail-semeval-2020-task-12/)
- [ArabDiscrim: A Decade-Long Arabic Facebook Corpus on Racism and Discrimination](https://md.research.elamur.ai/free/en/papers/zaghouani-2026-arabdiscrim-decade-long-arabic-facebook/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mubarak2021arabic,
  title = {{Arabic Offensive Language on Twitter: Analysis and Experiments}},
  author = {Mubarak, Hamdy and Rashed, Ammar and Darwish, Kareem and Samih, Younes and Abdelali, Ahmed},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.13/}
}
```

- [← previous Arabic Code-Switching Speech Recognition using Monolingual Data](https://md.research.elamur.ai/free/en/papers/ali-2021-arabic-code-switching-speech-recognition/)
- [next → Arabic Speech Recognition by End-to-End, Modular Systems and Human](https://md.research.elamur.ai/free/en/papers/hussein-2021-arabic-speech-recognition-end-end/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/)

## References

38 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Demonstrations, pages 11–16. [find](https://scholar.google.com/scholar?q=Ahmed+Abdelali%2C+Kareem+Darwish%2C+Nadir+Durrani%2C+and+Hamdy+Mubarak%2E+2016%2E+Farasa%3A+A+fast+and+furious+segmenter+for+arabic%2E+In+Proceedings+of+the+2016+conference+of+the+North+American+chapter+of+the+asso) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2016-abdelali-farasa-fast-furious-segmenter-arabic/)
2. Ehab Abozinadah. 2017. Detecting Abusive Arabic Language Twitter Accounts Using a Multidimensional Analysis Model. Ph.D. thesis, George Mason University. [find](https://scholar.google.com/scholar?q=Ehab+Abozinadah%2E+2017%2E+Detecting+Abusive+Arabic+Language+Twitter+Accounts+Using+a+Multidimensional+Analysis+Model%2E+Ph%2ED%2E+thesis%2C+George+Mason+University%2E)
3. Ibrahim Abu Farha and Walid Magdy. 2019. Mazajak: An online Arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ibrahim+Abu+Farha+and+Walid+Magdy%2E+2019%2E+Mazajak%3A+An+online+Arabic+sentiment+analyser%2E+In+Proceedings+of+the+Fourth+Arabic+Natural+Language+Processing+Workshop%2C+pages+192%E2%80%93198%2C+Florence%2C+Italy%2E+Associa)
4. Sweta Agrawal and Amit Awekar. 2018. Deep learning for detecting cyberbullying across multiple social media platforms. In European Conference on Information Retrieval, pages 141–153. Springer. [find](https://scholar.google.com/scholar?q=Sweta+Agrawal+and+Amit+Awekar%2E+2018%2E+Deep+learning+for+detecting+cyberbullying+across+multiple+social+media+platforms%2E+In+European+Conference+on+Information+Retrieval%2C+pages+141%E2%80%93153%2E+Springer%2E)
5. Azalden Alakrot, Liam Murray, and Nikola S Nikolov. 2018. Towards accurate detection of offensive language in online communication in arabic. Procedia computer science, 142:315–320. [find](https://scholar.google.com/scholar?q=Azalden+Alakrot%2C+Liam+Murray%2C+and+Nikola+S+Nikolov%2E+2018%2E+Towards+accurate+detection+of+offensive+language+in+online+communication+in+arabic%2E+Procedia+computer+science%2C+142%3A315%E2%80%93320%2E)
6. Nuha Albadi, Maram Kurdi, and Shivakant Mishra. 2018. Are they our brothers? analysis and detection of religious hate speech in the arabic twittersphere. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 69–76. IEEE. [find](https://scholar.google.com/scholar?q=Nuha+Albadi%2C+Maram+Kurdi%2C+and+Shivakant+Mishra%2E+2018%2E+Are+they+our+brothers%3F+analysis+and+detection+of+religious+hate+speech+in+the+arabic+twittersphere%2E+In+2018+IEEE%2FACM+International+Conference+on+A)
7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15. [find](https://scholar.google.com/scholar?q=Wissam+Antoun%2C+Fady+Baly%2C+and+Hazem+Hajj%2E+2020%2E+Arabert%3A+Transformer%2Dbased+model+for+arabic+language+understanding%2E+In+Proceedings+of+the+4th+Workshop+on+Open%2DSource+Arabic+Corpora+and+Processing+Tool) [our card](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/)
8. Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017. Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion, pages 759–760. International World Wide Web Conferences Steering Committee. [find](https://scholar.google.com/scholar?q=Pinkesh+Badjatiya%2C+Shashank+Gupta%2C+Manish+Gupta%2C+and+Vasudeva+Varma%2E+2017%2E+Deep+learning+for+hate+speech+detection+in+tweets%2E+In+Proceedings+of+the+26th+International+Conference+on+World+Wide+Web+Comp)
9. Pablo Barberá and Gaurav Sood. 2015. Follow your ideology: Measuring media ideology on social networks. In Annual Meeting of the European Political Science Association, Vienna, Austria. Retrieved from [http://www.gsood](http://www.gsood). com/research/papers/mediabias. pdf. [↗](http://www.gsood)
10. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. [↗](https://doi.org/10.1162/tacl_a_00051)
11. Kareem Darwish, Dimitar Alexandrov, Preslav Nakov, and Yelena Mejova. 2017. Seminar users in the arabic twitter sphere. In International Conference on Social Informatics, pages 91–108. Springer. [find](https://scholar.google.com/scholar?q=Kareem+Darwish%2C+Dimitar+Alexandrov%2C+Preslav+Nakov%2C+and+Yelena+Mejova%2E+2017%2E+Seminar+users+in+the+arabic+twitter+sphere%2E+In+International+Conference+on+Social+Informatics%2C+pages+91%E2%80%93108%2E+Springer%2E)
12. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Eleventh International Conference on Web and Social Media (ICWSM), pages 512–515. [find](https://scholar.google.com/scholar?q=Thomas+Davidson%2C+Dana+Warmsley%2C+Michael+Macy%2C+and+Ingmar+Weber%2E+2017%2E+Automated+hate+speech+detection+and+the+problem+of+offensive+language%2E+In+Eleventh+International+Conference+on+Web+and+Social+Medi) [our card](https://md.research.elamur.ai/free/en/papers/davidson-2017-automated-hate-speech-detection-problem/)
13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
14. Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015. Hate speech detection with comment embeddings. In Proceedings of the 24th international conference on world wide web, pages 29–30. ACM. [find](https://scholar.google.com/scholar?q=Nemanja+Djuric%2C+Jing+Zhou%2C+Robin+Morris%2C+Mihajlo+Grbovic%2C+Vladan+Radosavljevic%2C+and+Narayan+Bhamidipati%2E+2015%2E+Hate+speech+detection+with+comment+embeddings%2E+In+Proceedings+of+the+24th+international+c)
15. Samhaa R. El-Beltagy. 2016. NileULex: A phrase and word level sentiment lexicon for Egyptian and modern standard Arabic. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2900–2905, Portorož, Slovenia. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Samhaa+R%2E+El%2DBeltagy%2E+2016%2E+NileULex%3A+A+phrase+and+word+level+sentiment+lexicon+for+Egyptian+and+modern+standard+Arabic%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resources+and+)
16. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. [↗](https://doi.org/10.1037/h0031619)
17. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jeremy+Howard+and+Sebastian+Ruder%2E+2018%2E+Universal+language+model+fine%2Dtuning+for+text+classification%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volum)
18. Timothy Jay and Kristin Janschewitz. 2008. The pragmatics of swearing. Journal of Politeness Research. Language, Behaviour, Culture, 4(2):267–288. [find](https://scholar.google.com/scholar?q=Timothy+Jay+and+Kristin+Janschewitz%2E+2008%2E+The+pragmatics+of+swearing%2E+Journal+of+Politeness+Research%2E+Language%2C+Behaviour%2C+Culture%2C+4%282%29%3A267%E2%80%93288%2E)
19. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Armand+Joulin%2C+Edouard+Grave%2C+Piotr+Bojanowski%2C+and+Tomas+Mikolov%2E+2017%2E+Bag+of+tricks+for+efficient+text+classification%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Associati)
20. Irene Kwok and Yuzhou Wang. 2013. Locate the hate: Detecting tweets against blacks. In Twenty-seventh AAAI conference on artificial intelligence. [find](https://scholar.google.com/scholar?q=Irene+Kwok+and+Yuzhou+Wang%2E+2013%2E+Locate+the+hate%3A+Detecting+tweets+against+blacks%2E+In+Twenty%2Dseventh+AAAI+conference+on+artificial+intelligence%2E)
21. Thomas Chadefaux. 2014. Early warning signals for war in the news. Journal of Peace Research, 51(1):5–18. [find](https://scholar.google.com/scholar?q=Thomas+Chadefaux%2E+2014%2E+Early+warning+signals+for+war+in+the+news%2E+Journal+of+Peace+Research%2C+51%281%29%3A5%E2%80%9318%2E)
22. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. Race: Large-scale reading comprehension dataset from examinations. [find](https://scholar.google.com/scholar?q=Guokun+Lai%2C+Qizhe+Xie%2C+Hanxiao+Liu%2C+Yiming+Yang%2C+and+Eduard+Hovy%2E+2017%2E+Race%3A+Large%2Dscale+reading+comprehension+dataset+from+examinations%2E) [our card](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/)
23. Michael Conover, Jacob Ratkiewicz, Matthew R Francisco, Bruno Gonçalves, Filippo Menczer, and Alessandro Flammini. 2011. Political polarization on twitter. ICWSM, 133:89–96. [find](https://scholar.google.com/scholar?q=Michael+Conover%2C+Jacob+Ratkiewicz%2C+Matthew+R+Francisco%2C+Bruno+Gonc%CC%A7alves%2C+Filippo+Menczer%2C+and+Alessandro+Flammini%2E+2011%2E+Political+polarization+on+twitter%2E+ICWSM%2C+133%3A89%E2%80%9396%2E)
24. Alessandro Maisto, Serena Pelosi, Simonetta Vietri, Pierluigi Vitale, and Via Giovanni Paolo II. 2017. Mining offensive language on social media. CLiC-it 2017 11-12 December 2017, Rome, page 252. [find](https://scholar.google.com/scholar?q=Alessandro+Maisto%2C+Serena+Pelosi%2C+Simonetta+Vietri%2C+Pierluigi+Vitale%2C+and+Via+Giovanni+Paolo+II%2E+2017%2E+Mining+offensive+language+on+social+media%2E+CLiC%2Dit+2017+11%2D12+December+2017%2C+Rome%2C+page+252%2E)
25. Shervin Malmasi and Marcos Zampieri. 2017. Detecting hate speech in social media. arXiv preprint arXiv:1712.06427. [↗](https://arxiv.org/abs/1712.06427)
26. Elisa Mattiello. 2005. The pervasiveness of slang in standard and non-standard english. Mots Palabras Words, 5:7–41. [find](https://scholar.google.com/scholar?q=Elisa+Mattiello%2E+2005%2E+The+pervasiveness+of+slang+in+standard+and+non%2Dstandard+english%2E+Mots+Palabras+Words%2C+5%3A7%E2%80%9341%2E)
27. Abu Bakr Mohammad, Kareem Eissa, and Samhaa El-Beltagy. 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256–265. [find](https://scholar.google.com/scholar?q=Abu+Bakr+Mohammad%2C+Kareem+Eissa%2C+and+Samhaa+El%2DBeltagy%2E+2017%2E+Aravec%3A+A+set+of+arabic+word+embedding+models+for+use+in+arabic+nlp%2E+Procedia+Computer+Science%2C+117%3A256%E2%80%93265%2E)
28. Hamdy Mubarak and Kareem Darwish. 2019. Arabic offensive language classification on twitter. In International Conference on Social Informatics, pages 269–276. Springer. [find](https://scholar.google.com/scholar?q=Hamdy+Mubarak+and+Kareem+Darwish%2E+2019%2E+Arabic+offensive+language+classification+on+twitter%2E+In+International+Conference+on+Social+Informatics%2C+pages+269%E2%80%93276%2E+Springer%2E)
29. Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2017. Abusive language detection on arabic social media. In Proceedings of the First Workshop on Abusive Language Online, pages 52–56. [find](https://scholar.google.com/scholar?q=Hamdy+Mubarak%2C+Kareem+Darwish%2C+and+Walid+Magdy%2E+2017%2E+Abusive+language+detection+on+arabic+social+media%2E+In+Proceedings+of+the+First+Workshop+on+Abusive+Language+Online%2C+pages+52%E2%80%9356%2E) [our card](https://md.research.elamur.ai/free/en/papers/mubarak-2017-abusive-language-detection-arabic-social/)
30. Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive language detection in online user content. In Proceedings of the 25th international conference on world wide web, pages 145–153. International World Wide Web Conferences Steering Committee. [find](https://scholar.google.com/scholar?q=Chikashi+Nobata%2C+Joel+Tetreault%2C+Achint+Thomas%2C+Yashar+Mehdad%2C+and+Yi+Chang%2E+2016%2E+Abusive+language+detection+in+online+user+content%2E+In+Proceedings+of+the+25th+international+conference+on+world+wide+)
31. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL [https://s3-us-west-2.amazonaws](https://s3-us-west-2.amazonaws). com/openai-assets/researchcovers/languageunsupervised/language understanding paper. pdf. [↗](https://s3-us-west-2.amazonaws)
32. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Jian+Zhang%2C+Konstantin+Lopyrev%2C+and+Percy+Liang%2E+2016%2E+Squad%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
33. Younes Samih, Mohamed Eldesouki, Mohammed Attia, Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, and Laura Kallmeyer. 2017. Learning from relatives: unified dialectal arabic segmentation. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 432–441. [find](https://scholar.google.com/scholar?q=Younes+Samih%2C+Mohamed+Eldesouki%2C+Mohammed+Attia%2C+Kareem+Darwish%2C+Ahmed+Abdelali%2C+Hamdy+Mubarak%2C+and+Laura+Kallmeyer%2E+2017%2E+Learning+from+relatives%3A+unified+dialectal+arabic+segmentation%2E+In+Proceeding)
34. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. [find](https://scholar.google.com/scholar?q=Alex+Wang%2C+Amanpreet+Singh%2C+Julian+Michael%2C+Felix+Hill%2C+Omer+Levy%2C+and+Samuel+R%2E+Bowman%2E+2018%2E+Glue%3A+A+multi%2Dtask+benchmark+and+analysis+platform+for+natural+language+understanding%2E) [our card](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/)
35. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the NAACL student research workshop, pages 88–93. [find](https://scholar.google.com/scholar?q=Zeerak+Waseem+and+Dirk+Hovy%2E+2016%2E+Hateful+symbols+or+hateful+people%3F+predictive+features+for+hate+speech+detection+on+twitter%2E+In+Proceedings+of+the+NAACL+student+research+workshop%2C+pages+88%E2%80%9393%2E) [our card](https://md.research.elamur.ai/free/en/papers/waseem-2016-hateful-symbols-or-hateful-people/)
36. Dawei Yin, Zhenzhen Xue, Liangjie Hong, Brian D Davison, April Kontostathis, and Lynne Edwards. 2009. Detection of harassment on web 2.0. Proceedings of the Content Analysis in the WEB, 2:1–7. [find](https://scholar.google.com/scholar?q=Dawei+Yin%2C+Zhenzhen+Xue%2C+Liangjie+Hong%2C+Brian+D+Davison%2C+April+Kontostathis%2C+and+Lynne+Edwards%2E+2009%2E+Detection+of+harassment+on+web+2%2E0%2E+Proceedings+of+the+Content+Analysis+in+the+WEB%2C+2%3A1%E2%80%937%2E)
37. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval). arXiv preprint arXiv:1903.08983. [↗](https://arxiv.org/abs/1903.08983) [our card](https://md.research.elamur.ai/free/en/papers/zampieri-2019-semeval-2019-task-6-identifying/)
38. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Çağrı Çöltekin. 2020. Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020). arXiv preprint arXiv:2006.07235. [↗](https://arxiv.org/abs/2006.07235)