---
title: "Multilingual Offensive Language Identification with Cross-lingual"
description: "Ranasinghe and Zampieri test whether a cross-lingual transformer, XLM-R, can carry knowledge… 45 references, cited by 63 papers here."
authors: ["Tharindu Ranasinghe", "Marcos Zampieri"]
year: 2020
venue: "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)"
source: "https://aclanthology.org/2020.emnlp-main.470/"
doi: "10.18653/v1/2020.emnlp-main.470"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ranasinghe-2020-multilingual-offensive-language-identification-cross/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ranasinghe-2020-multilingual-offensive-language-identification-cross/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/3/) / Multilingual Offensive Language Identification with Cross-lingual Embeddings

# Multilingual Offensive Language Identification with Cross-lingual Embeddings

**Authors:** [Tharindu Ranasinghe](https://md.research.elamur.ai/free/en/authors/tharindu-ranasinghe/) and [Marcos Zampieri](https://md.research.elamur.ai/free/en/authors/marcos-zampieri/) · **Year:** 2020 · **Venue:** [Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2020](https://md.research.elamur.ai/free/en/venues/emnlp/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.emnlp-main.470/) · [PDF](https://aclanthology.org/2020.emnlp-main.470.pdf) · [DOI 10.18653/v1/2020.emnlp-main.470](https://doi.org/10.18653/v1/2020.emnlp-main.470) · [OpenAlex](https://openalex.org/W3095035006)

**Citations:** 148 citations (OpenAlex, 2026-10-14) · [63](https://md.research.elamur.ai/free/en/papers/ranasinghe-2020-multilingual-offensive-language-identification-cross/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Most offensive-language resources exist only for English, so systems for other languages often lack training data. Ranasinghe and Zampieri test whether a cross-lingual transformer, XLM-R, can carry knowledge learned from English annotations over to Bengali, Hindi, and Spanish. They first fine-tune XLM-R large with a softmax head on level A of OLID (offensive vs. non-offensive tweets). They then use those weights to start training on each target language. For Hindi (HASOC 2019) and Spanish (HatEval), they reuse the entire model, classifier included. For Bengali (TRAC-2), which is Facebook data with three aggression classes, only the encoder is transferred. The transferred XLM-R beats plain XLM-R, multilingual BERT with and without transfer, and the top shared-task entries for all three languages. This holds even when domain and label set differ. The study suggests that OLID's general annotation scheme covers related phenomena such as hate speech and aggression. It also suggests that English data can usefully seed classifiers for lower-resource languages. Code and the English model are released as DeepOffense.

## Key points
- **Datasets:**
  - OLID (English): 14,100 tweets.
  - HASOC 2019 (Hindi): 8,000 tweets, hate-offensive vs. not.
  - HatEval (Spanish): 6,600 tweets, hateful vs. not.
  - TRAC-2 (Bengali): 4,000 Facebook posts labelled overtly aggressive, covertly aggressive, or non-aggressive.
- **Two transfer modes:**
  - Inter-language: encoder plus softmax layer reused (Hindi, Spanish).
  - Inter-task/inter-language: encoder only, with a new head for a different number of classes (Bengali).
- **Results for XLM-R (TL):**
  - Bengali: macro F1 0.8415 / weighted F1 0.8423, against 0.8219 macro F1 for Risch and Krestel (2020).
  - Hindi: macro F1 0.8568 / weighted F1 0.8580, against 0.8149 macro F1 for Bashar and Nayak (2019).
  - Spanish: macro F1 0.7513 / weighted F1 0.7591, against 0.7300 macro F1 for the best HatEval systems.
  - The prose of the paper contains some copy errors in these numbers; the values above follow the table.
- **XLM-R vs. BERT-m:** XLM-R beat BERT-m in every language. Transfer from English improved both models.
- **Training setup:**
  - Learning rate 1e-5 and 3 epochs worked best for all languages.
  - Hyperparameters were tuned manually on a 0.8:0.2 train/validation split.
  - Training ran on a Tesla K80.
- **Bengali per-class behaviour:** the non-aggressive class was predicted much better than either aggressive class.

## Interesting ideas
- **English model as a warm start:** rather than zero-shot transfer, fine-tune a cross-lingual model on a large English task and use it to initialise training on the small target dataset. This is a cheap way to bootstrap Ukrainian classifiers where only a few thousand labelled examples exist.
- **Drop the head when labels differ:** when the target task has a different label set, keep only the transferred encoder and attach a fresh classifier. This lets a binary source task support a three-way target task, and it applies whenever annotation schemes don't align one-to-one.
- **Pick a broad source taxonomy:** choosing a source dataset with a hierarchical, general-purpose scheme (OLID level A) makes it easier to map onto related but differently defined targets such as hate speech or aggression. This is worth considering when choosing a pivot dataset for transfer.
- **Benchmark against shared-task leaderboards:** evaluating on official shared-task test sets with each organiser's own metric (macro vs. weighted F1) gives direct, credible comparison with prior systems. This is a good practice for any low-resource evaluation.
- **Transfer as a probe of label compatibility:** how well the source model transfers can serve as a rough test of how closely two datasets' labels correspond. This could help audit whether separately annotated corpora are compatible before merging them.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Wolverhampton](https://md.research.elamur.ai/free/en/universities/university-of-wolverhampton/) — University, Wolverhampton, United Kingdom · 30 works · 6,272 citations
- [Rochester Institute of Technology](https://md.research.elamur.ai/free/en/universities/rochester-institute-of-technology/) — University, Rochester, United States · 10 works · 4,906 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 608 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 467 works

## Cite

Cite as: Tharindu Ranasinghe and Marcos Zampieri. 2020. Multilingual Offensive Language Identification with Cross-lingual Embeddings. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ranasinghe2020multilingual,
  title = {{Multilingual Offensive Language Identification with Cross-lingual Embeddings}},
  author = {Tharindu Ranasinghe and Marcos Zampieri},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.470},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.470}
}
```

- [← previous HopeEDI: A Multilingual Hope Speech Detection Dataset for Equality, Diversity, and Inclusion](https://md.research.elamur.ai/free/en/papers/chakravarthi-2020-hopeedi-multilingual-hope-speech-detection/)
- [next → Logical Natural Language Generation from Open-Domain Tables](https://md.research.elamur.ai/free/en/papers/chen-2020-logical-natural-language-generation-open/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/3/)

## References

45 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Segun Taofeek Aroyehun and Alexander Gelbukh. 2018. Aggression detection in social media: Using deep neural networks, data augmentation, and pseudo labeling. In Proceedings of TRAC. [find](https://scholar.google.com/scholar?q=Segun+Taofeek+Aroyehun+and+Alexander+Gelbukh%2E+2018%2E+Aggression+detection+in+social+media%3A+Using+deep+neural+networks%2C+data+augmentation%2C+and+pseudo+labeling%2E+In+Proceedings+of+TRAC%2E)
2. Rienke Bannink, Suzanne Broeren, Petra M van de Looij-Jansen, Frouwkje G de Waart, and Hein Raat. 2014. Cyber and Traditional Bullying Victimization as a Risk Factor for Mental Health Problems and Suicidal Ideation in Adolescents. PloS one, 9(4). [find](https://scholar.google.com/scholar?q=Rienke+Bannink%2C+Suzanne+Broeren%2C+Petra+M+van+de+Looij%2DJansen%2C+Frouwkje+G+de+Waart%2C+and+Hein+Raat%2E+2014%2E+Cyber+and+Traditional+Bullying+Victimization+as+a+Risk+Factor+for+Mental+Health+Problems+and+Sui)
3. Md Abul Bashar and Richi Nayak. 2019. QutNoctur-nal@ HASOC’19: CNN for hate speech and offensive content identification in Hindi language. In Proceedings of FIRE. [find](https://scholar.google.com/scholar?q=Md+Abul+Bashar+and+Richi+Nayak%2E+2019%2E+QutNoctur%2Dnal%40+HASOC%E2%80%9919%3A+CNN+for+hate+speech+and+offensive+content+identification+in+Hindi+language%2E+In+Proceedings+of+FIRE%2E)
4. Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019. [SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter](https://md.research.elamur.ai/free/en/papers/basile-2019-semeval-2019-task-5-multilingual/). In Proceedings of SemEval.
5. Shiladitya Bhattacharya, Siddharth Singh, Ritesh Kumar, Akanksha Bansal, Akash Bhagat, Yogesh Dawer, Bornini Lahiri, and Atul Kr. Ojha. 2020. Developing a multilingual annotated corpus of misogyny and aggression. In Proceedings of TRAC. [find](https://scholar.google.com/scholar?q=Shiladitya+Bhattacharya%2C+Siddharth+Singh%2C+Ritesh+Kumar%2C+Akanksha+Bansal%2C+Akash+Bhagat%2C+Yogesh+Dawer%2C+Bornini+Lahiri%2C+and+Atul+Kr%2E+Ojha%2E+2020%2E+Developing+a+multilingual+annotated+corpus+of+misogyny+and)
6. Rina A Bonanno and Shelley Hymel. 2013. Cyber bullying and internalizing difficulties: Above and beyond the impact of traditional forms of bullying. Journal of youth and adolescence, 42(5):685–697. [find](https://scholar.google.com/scholar?q=Rina+A+Bonanno+and+Shelley+Hymel%2E+2013%2E+Cyber+bullying+and+internalizing+difficulties%3A+Above+and+beyond+the+impact+of+traditional+forms+of+bullying%2E+Journal+of+youth+and+adolescence%2C+42%285%29%3A685%E2%80%93697%2E)
7. Çağrı Çöltekin. 2020. [A Corpus of Turkish Offensive Language on Social Media](https://md.research.elamur.ai/free/en/papers/coltekin-2020-corpus-turkish-offensive-language-social/). In Proceedings of LREC.
8. Patricia Chiril, Farah Benamara Zitoune, Véronique Moriceau, Marlène Coulomb-Gully, and Abhishek Kumar. 2019. Multilingual and multitarget hate speech detection in tweets. In Proceedings of TALN. [find](https://scholar.google.com/scholar?q=Patricia+Chiril%2C+Farah+Benamara+Zitoune%2C+Ve%CC%81ronique+Moriceau%2C+Marle%CC%80ne+Coulomb%2DGully%2C+and+Abhishek+Kumar%2E+2019%2E+Multilingual+and+multitarget+hate+speech+detection+in+tweets%2E+In+Proceedings+of+TALN%2E)
9. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzma%CC%81n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Unsupervised+cross%2Dling)
10. Maral Dadvar, Dolf Trieschnigg, Roeland Ordelman, and Franciska de Jong. 2013. Improving Dyberbul-lying Detection with User Context. In Proceedings of ECIR. [find](https://scholar.google.com/scholar?q=Maral+Dadvar%2C+Dolf+Trieschnigg%2C+Roeland+Ordelman%2C+and+Franciska+de+Jong%2E+2013%2E+Improving+Dyberbul%2Dlying+Detection+with+User+Context%2E+In+Proceedings+of+ECIR%2E)
11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of NAACL.
12. Darja Fišer, Tomaž Erjavec, and Nikola Ljubešić. 2017. Legal Framework, Dataset and Annotation Schema for Socially Unacceptable On-line Discourse Practices in Slovene. In Proceedings ALW. [find](https://scholar.google.com/scholar?q=Darja+Fis%CC%8Cer%2C+Tomaz%CC%8C+Erjavec%2C+and+Nikola+Ljubes%CC%8Cic%CC%81%2E+2017%2E+Legal+Framework%2C+Dataset+and+Annotation+Schema+for+Socially+Unacceptable+On%2Dline+Discourse+Practices+in+Slovene%2E+In+Proceedings+ALW%2E)
13. Paula Fortuna, Joao Rocha da Silva, Leo Wanner, Sérgio Nunes, et al. 2019. [A Hierarchically-labeled Portuguese Hate Speech Dataset](https://md.research.elamur.ai/free/en/papers/fortuna-2019-hierarchically-labeled-portuguese-hate-speech/). In Proceedings of ALW.
14. Erfan Ghadery and Marie-Francine Moens. 2020. Liir at semeval-2020 task 12: A cross-lingual augmentation approach for multilingual offensive language identification. arXiv preprint arXiv:2005.03695. [find](https://scholar.google.com/scholar?q=Erfan+Ghadery+and+Marie%2DFrancine+Moens%2E+2020%2E+Liir+at+semeval%2D2020+task+12%3A+A+cross%2Dlingual+augmentation+approach+for+multilingual+offensive+language+identification%2E+arXiv+preprint+arXiv%3A2005%2E03695%2E)
15. K Karthikeyan, Zihan Wang, Stephen Mayhew, and Dan Roth. 2020. Cross-lingual ability of multilingual bert: An empirical study. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=K+Karthikeyan%2C+Zihan+Wang%2C+Stephen+Mayhew%2C+and+Dan+Roth%2E+2020%2E+Cross%2Dlingual+ability+of+multilingual+bert%3A+An+empirical+study%2E+In+Proceedings+of+ICLR%2E)
16. Ritesh Kumar, Atul Kr Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. [Benchmarking aggression identification in social media](https://md.research.elamur.ai/free/en/papers/kumar-2018-benchmarking-aggression-identification-social-media/). In Proceedings of TRAC.
17. Ritesh Kumar, Atul Kr. Ojha, Shervin Malmasi, and Marcos Zampieri. 2020. Evaluating Aggression Identification in Social Media. In Proceedings of TRAC. [find](https://scholar.google.com/scholar?q=Ritesh+Kumar%2C+Atul+Kr%2E+Ojha%2C+Shervin+Malmasi%2C+and+Marcos+Zampieri%2E+2020%2E+Evaluating+Aggression+Identification+in+Social+Media%2E+In+Proceedings+of+TRAC%2E)
18. Ping Liu, Wen Li, and Liang Zou. 2019. [NULI at SemEval-2019 task 6: Transfer learning for offensive language detection using bidirectional transformers](https://md.research.elamur.ai/free/en/papers/liu-2019-nuli-semeval-2019-task-6/). In Proceedings of SemEval.
19. Prasenjit Majumder, Thomas Mandl, et al. 2018. Filtering Aggression from the Multilingual Social Media Feed. In Proceedings TRAC. [find](https://scholar.google.com/scholar?q=Prasenjit+Majumder%2C+Thomas+Mandl%2C+et+al%2E+2018%2E+Filtering+Aggression+from+the+Multilingual+Social+Media+Feed%2E+In+Proceedings+TRAC%2E)
20. Shervin Malmasi and Marcos Zampieri. 2017. Detecting Hate Speech in Social Media. In Proceedings of RANLP. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi+and+Marcos+Zampieri%2E+2017%2E+Detecting+Hate+Speech+in+Social+Media%2E+In+Proceedings+of+RANLP%2E)
21. Shervin Malmasi and Marcos Zampieri. 2018. Challenges in Discriminating Profanity from Hate Speech. Journal of Experimental & Theoretical Artificial Intelligence, 30:1 – 16. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi+and+Marcos+Zampieri%2E+2018%2E+Challenges+in+Discriminating+Profanity+from+Hate+Speech%2E+Journal+of+Experimental+%26+Theoretical+Artificial+Intelligence%2C+30%3A1+%E2%80%93+16%2E)
22. Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019. Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages. In Proceedings of FIRE. [find](https://scholar.google.com/scholar?q=Thomas+Mandl%2C+Sandip+Modha%2C+Prasenjit+Majumder%2C+Daksh+Patel%2C+Mohana+Dave%2C+Chintak+Mandlia%2C+and+Aditya+Patel%2E+2019%2E+Overview+of+the+hasoc+track+at+fire+2019%3A+Hate+speech+and+offensive+content+identific)
23. Hamdy Mubarak, Darwish Kareem, and Magdy Walid. 2017. [Abusive language detection on Arabic social media](https://md.research.elamur.ai/free/en/papers/mubarak-2017-abusive-language-detection-arabic-social/). In Proceedings of ALW.
24. Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2020. [Arabic offensive language on twitter: Analysis and experiments](https://md.research.elamur.ai/free/en/papers/mubarak-2021-arabic-offensive-language-twitter-analysis/). arXiv preprint arXiv:2004.02192.
25. Endang Wahyu Pamungkas and Viviana Patti. 2019. Cross-domain and cross-lingual abusive language detection: A hybrid approach with deep learning and a multilingual lexicon. In Proceedings ACL:SRW. [find](https://scholar.google.com/scholar?q=Endang+Wahyu+Pamungkas+and+Viviana+Patti%2E+2019%2E+Cross%2Ddomain+and+cross%2Dlingual+abusive+language+detection%3A+A+hybrid+approach+with+deep+learning+and+a+multilingual+lexicon%2E+In+Proceedings+ACL%3ASRW%2E)
26. Juan Manuel Pérez and Franco M Luque. 2019. Atalaya at semeval 2019 task 5: Robust embeddings for tweet classification. In Proceedings of SemEval. [find](https://scholar.google.com/scholar?q=Juan+Manuel+Pe%CC%81rez+and+Franco+M+Luque%2E+2019%2E+Atalaya+at+semeval+2019+task+5%3A+Robust+embeddings+for+tweet+classification%2E+In+Proceedings+of+SemEval%2E)
27. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. [Deep Contextualized Word Representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In Proceedings of NAACL.
28. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. [Understanding Abuse: A Typology of Abusive Language Detection Subtasks](https://md.research.elamur.ai/free/en/papers/waseem-2017-understanding-abuse-typology-abusive-language/). In Proceedings of ALW.
29. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. [How multilingual is multilingual BERT?](https://md.research.elamur.ai/free/en/papers/pires-2019-how-multilingual-is-multilingual-bert/) In Proceedings of ACL.
30. Jun-Ming Xu, Kwang-Sung Jun, Xiaojin Zhu, and Amy Bellmore. 2012. [Learning from bullying traces in social media](https://md.research.elamur.ai/free/en/papers/xu-2012-learning-bullying-traces-social-media/). In Proceedings of NAACL.
31. Zeses Pitenis, Marcos Zampieri, and Tharindu Ranasinghe. 2020. [Offensive Language Identification in Greek](https://md.research.elamur.ai/free/en/papers/pitenis-2020-offensive-language-identification-greek/). In Proceedings of LREC.
32. Mengfan Yao, Charalampos Chelmis, and Daphney-Stavroula Zois. 2019. Cyberbullying Ends Here: Towards Robust Detection of Cyberbullying in Social Media. In Proceedings of WWW. [find](https://scholar.google.com/scholar?q=Mengfan+Yao%2C+Charalampos+Chelmis%2C+and+Daphney%2DStavroula+Zois%2E+2019%2E+Cyberbullying+Ends+Here%3A+Towards+Robust+Detection+of+Cyberbullying+in+Social+Media%2E+In+Proceedings+of+WWW%2E)
33. Fabio Poletto, Marco Stranisci, Manuela Sanguinetti, Viviana Patti, and Cristina Bosco. 2017. Hate Speech Annotation: Analysis of an Italian Twitter Corpus. In Proceedings of CLiC-it. [find](https://scholar.google.com/scholar?q=Fabio+Poletto%2C+Marco+Stranisci%2C+Manuela+Sanguinetti%2C+Viviana+Patti%2C+and+Cristina+Bosco%2E+2017%2E+Hate+Speech+Annotation%3A+Analysis+of+an+Italian+Twitter+Corpus%2E+In+Proceedings+of+CLiC%2Dit%2E)
34. Tharindu Ranasinghe, Marcos Zampieri, and Hansi Hettiarachchi. 2019. Brums at hasoc 2019: Deep learning models for multilingual hate speech and offensive language identification. In Proceedings of FIRE. [find](https://scholar.google.com/scholar?q=Tharindu+Ranasinghe%2C+Marcos+Zampieri%2C+and+Hansi+Hettiarachchi%2E+2019%2E+Brums+at+hasoc+2019%3A+Deep+learning+models+for+multilingual+hate+speech+and+offensive+language+identification%2E+In+Proceedings+of+FIR)
35. Michael Ridenhour, Arunkumar Bagavathi, Elaheh Raisi, and Siddharth Krishnan. 2020. Detecting Online Hate Speech: Approaches Using Weak Supervision and Network Embedding Models. arXiv preprint arXiv:2007.12724. — [arXiv:2007.12724](https://arxiv.org/abs/2007.12724) [find](https://scholar.google.com/scholar?q=Michael+Ridenhour%2C+Arunkumar+Bagavathi%2C+Elaheh+Raisi%2C+and+Siddharth+Krishnan%2E+2020%2E+Detecting+Online+Hate+Speech%3A+Approaches+Using+Weak+Supervision+and+Network+Embedding+Models%2E+arXiv+preprint+arXiv%3A2)
36. Julian Risch and Ralf Krestel. 2020. Bagging bert models for robust aggression identification. In Proceedings of TRAC. [find](https://scholar.google.com/scholar?q=Julian+Risch+and+Ralf+Krestel%2E+2020%2E+Bagging+bert+models+for+robust+aggression+identification%2E+In+Proceedings+of+TRAC%2E)
37. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019a. [Predicting the type and target of offensive posts in social media](https://md.research.elamur.ai/free/en/papers/zampieri-2019-predicting-type-target-offensive-posts/). In Proceedings of NAACL.
38. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019b. [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://md.research.elamur.ai/free/en/papers/zampieri-2019-semeval-2019-task-6-identifying/). In Proceedings of SemEval.
39. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Cagrı Cöltekin. 2020. Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020). Proceedings of SemEval. [find](https://scholar.google.com/scholar?q=Marcos+Zampieri%2C+Preslav+Nakov%2C+Sara+Rosenthal%2C+Pepa+Atanasova%2C+Georgi+Karadzhov%2C+Hamdy+Mubarak%2C+Leon+Derczynski%2C+Zeses+Pitenis%2C+and+Cagr%C4%B1+Co%CC%88ltekin%2E+2020%2E+Semeval%2D2020+task+12%3A+Multilingual+offensive)
40. Hugo Rosa, N Pereira, Ricardo Ribeiro, Paula Costa Ferreira, Joao Paulo Carvalho, S Oliveira, Luı́sa Coheur, Paula Paulino, AM Veiga Simão, and Isabel Trancoso. 2019. Automatic cyberbullying detection: A systematic review. Computers in Human Behavior, 93:333–345. [find](https://scholar.google.com/scholar?q=Hugo+Rosa%2C+N+Pereira%2C+Ricardo+Ribeiro%2C+Paula+Costa+Ferreira%2C+Joao+Paulo+Carvalho%2C+S+Oliveira%2C+Lu%C4%B1%CC%81sa+Coheur%2C+Paula+Paulino%2C+AM+Veiga+Sima%CC%83o%2C+and+Isabel+Trancoso%2E+2019%2E+Automatic+cyberbullying+detectio)
41. Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Marcos Zampieri, and Preslav Nakov. 2020. A Large-Scale Weakly Supervised Dataset for Offensive Language Identification. In arXiv preprint arXiv:2004.14454. — [arXiv:2004.14454](https://arxiv.org/abs/2004.14454) [find](https://scholar.google.com/scholar?q=Sara+Rosenthal%2C+Pepa+Atanasova%2C+Georgi+Karadzhov%2C+Marcos+Zampieri%2C+and+Preslav+Nakov%2E+2020%2E+A+Large%2DScale+Weakly+Supervised+Dataset+for+Offensive+Language+Identification%2E+In+arXiv+preprint+arXiv%3A2004%2E)
42. Gudbjartur Ingi Sigurbergsson and Leon Derczynski. 2020. [Offensive Language and Hate Speech Detection for Danish](https://md.research.elamur.ai/free/en/papers/sigurbergsson-2020-offensive-language-hate-speech-detection/). In Proceedings of LREC.
43. Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. [How to fine-tune bert for text classification?](https://md.research.elamur.ai/free/en/papers/sun-2019-fine-tune-bert-text-classification/) In Chinese Computational Linguistics, pages 194– 206.
44. Stéphan Tulkens, Lisa Hilte, Elise Lodewyckx, Ben Verhoeven, and Walter Daelemans. 2016. A Dictionary-based Approach to Racism Detection in Dutch Social Media. In Proceedings of TA-COS. [find](https://scholar.google.com/scholar?q=Ste%CC%81phan+Tulkens%2C+Lisa+Hilte%2C+Elise+Lodewyckx%2C+Ben+Verhoeven%2C+and+Walter+Daelemans%2E+2016%2E+A+Dictionary%2Dbased+Approach+to+Racism+Detection+in+Dutch+Social+Media%2E+In+Proceedings+of+TA%2DCOS%2E)
45. Luis Enrique Argota Vega, Jorge Carlos Reyes-Magaña, Helena Gómez-Adorno, and Gemma Bel-Enguix. 2019. Mineriaunam at semeval-2019 task 5: Detecting hate speech in twitter using multiple features in a combinatorial framework. In Proceedings of SemEval. [find](https://scholar.google.com/scholar?q=Luis+Enrique+Argota+Vega%2C+Jorge+Carlos+Reyes%2DMagan%CC%83a%2C+Helena+Go%CC%81mez%2DAdorno%2C+and+Gemma+Bel%2DEnguix%2E+2019%2E+Mineriaunam+at+semeval%2D2019+task+5%3A+Detecting+hate+speech+in+twitter+using+multiple+features+in+)

*Updated 24 October 2026*