---
title: "Predicting the Type and Target of Offensive — Zampieri 2019"
description: "Earlier work on harmful online language usually focused on a single phenomenon, such as hate speech, cyberbullying or aggression, and rarely asked who…"
authors: ["Marcos Zampieri", "Shervin Malmasi", "Preslav Nakov", "Sara Rosenthal", "Noura Farra", "Ritesh Kumar"]
year: 2019
venue: "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)"
source: "https://aclanthology.org/N19-1144/"
doi: "10.18653/v1/n19-1144"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zampieri-2019-predicting-type-target-offensive-posts/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zampieri-2019-predicting-type-target-offensive-posts/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / Predicting the Type and Target of Offensive Posts in Social Media

# Predicting the Type and Target of Offensive Posts in Social Media

**Authors:** [Marcos Zampieri](https://md.research.elamur.ai/free/en/authors/marcos-zampieri/), [Shervin Malmasi](https://md.research.elamur.ai/free/en/authors/shervin-malmasi/), [Preslav Nakov](https://md.research.elamur.ai/free/en/authors/preslav-nakov/), [Sara Rosenthal](https://md.research.elamur.ai/free/en/authors/sara-rosenthal/), [Noura Farra](https://md.research.elamur.ai/free/en/authors/noura-farra/), [Ritesh Kumar](https://md.research.elamur.ai/free/en/authors/ritesh-kumar/) · **Year:** 2019 · **Venue:** Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N19-1144/) · [PDF](https://aclanthology.org/N19-1144.pdf) · [DOI 10.18653/v1/n19-1144](https://doi.org/10.18653/v1/n19-1144) · [OpenAlex](https://openalex.org/W2916719435)

**Citations:** 720 citations (openalex-snapshot · 2026-10-14) · [393](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Earlier work on harmful online language usually focused on a single phenomenon, such as hate speech, cyberbullying or aggression, and rarely asked who the abuse was aimed at. This paper treats offensive content as one problem with a layered structure. The authors propose a three-level annotation scheme: whether a post is offensive (A), whether the offense is targeted or just profanity (B), and whether a targeted offense is directed at an individual, a group or something else (C). They used this scheme to build OLID, a publicly released corpus of English tweets. The tweets were collected through keyword searches, half of them with political terms. Crowdworkers on Figure Eight labelled them, and disagreements were settled by majority vote. The authors then trained an SVM, a BiLSTM and a CNN as baselines for each level. The CNN was strongest on the first two levels. Target identification proved hardest, and the OTH class was never predicted correctly. OLID became the official data for SemEval 2019 OffensEval and offers a single resource covering several related abuse-detection tasks.

## Key points
- OLID contains 14,100 tweets, split into 13,240 for training and 860 for testing. Level A labels are NOT/OFF, level B labels are TIN/UNT, and level C labels are IND/GRP/OTH.
- Keywords included "MAGA", "antifa", "liberals", "conservatives", "gun control" and replies to BreitBartNews. The authors aimed for roughly 30% offensive content. A 300-tweet expert trial reached a Fleiss' kappa of .83 on level A.
- The best macro-F1 scores were 0.80 for level A (CNN) and 0.69 for level B (CNN). For level C, the BiLSTM and CNN tied at 0.47, with the SVM close behind at 0.45.
- All models scored zero F1 on the OTH target class. The authors attribute this to the class being heterogeneous and having few examples (395 training tweets).
- The data was anonymised: URLs and user mentions were replaced with placeholders. OLID served as the official dataset of SemEval 2019 Task 6 (OffensEval).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

393 papers of our library cite it (of the 140673 whose bibliographies we read); 30 of them have a page here, 363 not yet.

- [From Linguistics to Practice: a Case Study of Offensive Language Taxonomy in Hebrew](https://md.research.elamur.ai/free/en/papers/aclbulk-2024-liebeskind-linguistics-practice-case-study-offensive/)
- [Towards Safer Hebrew Communication: A Dataset for Offensive Language Detoxification](https://md.research.elamur.ai/free/en/papers/aclbulk-2025-vanetik-safer-hebrew-communication-dataset-offensive/)
- [Offensive Language Detection in Under-resourced Algerian Dialectal Arabic Language](https://md.research.elamur.ai/free/en/papers/boucherit-2022-offensive-language-detection-under-resourced/)
- [I Feel Offended, Don't Be Abusive! Implicit/Explicit Messages in Offensive and Abusive Language](https://md.research.elamur.ai/free/en/papers/caselli-2020-i-feel-offended-don-t/)
- [HopeEDI: A Multilingual Hope Speech Detection Dataset for Equality, Diversity, and Inclusion](https://md.research.elamur.ai/free/en/papers/chakravarthi-2020-hopeedi-multilingual-hope-speech-detection/)
- [A Corpus of Turkish Offensive Language on Social Media](https://md.research.elamur.ai/free/en/papers/coltekin-2020-corpus-turkish-offensive-language-social/)
- [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://md.research.elamur.ai/free/en/papers/elsherief-2021-latent-hatred-benchmark-understanding-implicit/)
- [Toxic, Hateful, Offensive or Abusive? What Are We Really Classifying? An Empirical Analysis of Hate Speech Datasets](https://md.research.elamur.ai/free/en/papers/fortuna-2020-toxic-hateful-offensive-or-abusive/)
- [Evaluating Large Language Models: A Comprehensive Survey](https://md.research.elamur.ai/free/en/papers/guo-2023-evaluating-large-language-models-comprehensive/)
- [ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection](https://md.research.elamur.ai/free/en/papers/hartvigsen-2022-toxigen-large-scale-machine-generated/)
- [Offensive Hebrew Corpus and Detection using BERT](https://md.research.elamur.ai/free/en/papers/langstream-2023-hamad-offensive-hebrew-corpus-detection-using/)
- [Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreement](https://md.research.elamur.ai/free/en/papers/leonardelli-2021-agreeing-disagree-annotating-offensive-language/)
- [Offensive language detection in Hebrew: can other languages help?](https://md.research.elamur.ai/free/en/papers/lingbulk-2022-litvak-offensive-language-detection-hebrew-can/)
- [NULI at SemEval-2019 Task 6: Transfer Learning for Offensive Language Detection using Bidirectional Transformers](https://md.research.elamur.ai/free/en/papers/liu-2019-nuli-semeval-2019-task-6/)
- [A Holistic Approach to Undesired Content Detection in the Real World](https://md.research.elamur.ai/free/en/papers/markov-2022-holistic-approach-undesired-content-detection/)
- [Toxicity Detection: Does Context Really Matter?](https://md.research.elamur.ai/free/en/papers/pavlopoulos-2020-toxicity-detection-does-context-really/)
- [Offensive Language Identification in Greek](https://md.research.elamur.ai/free/en/papers/pitenis-2020-offensive-language-identification-greek/)
- [HatePrototypes: Interpretable and Transferable Representations for Implicit and Explicit Hate Speech Detection](https://md.research.elamur.ai/free/en/papers/proskurina-2026-hateprototypes-interpretable-transferable-representations-implicit/)
- [Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger](https://md.research.elamur.ai/free/en/papers/qi-2021-hidden-killer-invisible-textual-backdoor/)
- [ONION: A Simple and Effective Defense Against Textual Backdoor Attacks](https://md.research.elamur.ai/free/en/papers/qi-2021-onion-simple-effective-defense-against/)
- [HateCheck: Functional Tests for Hate Speech Detection Models](https://md.research.elamur.ai/free/en/papers/rottger-2021-hatecheck-functional-tests-hate-speech/)
- [Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks](https://md.research.elamur.ai/free/en/papers/rottger-2022-two-contrasting-data-annotation-paradigms/)
- [KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media](https://md.research.elamur.ai/free/en/papers/safaya-2020-kuisail-semeval-2020-task-12/)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://md.research.elamur.ai/free/en/papers/sap-2020-social-bias-frames-reasoning-about/)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://md.research.elamur.ai/free/en/papers/sap-2022-annotators-attitudes-how-annotator-beliefs/)
- [Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models](https://md.research.elamur.ai/free/en/papers/sengupta-2023-jais-jais-chat-arabic-centric/)
- [Large Language Model Alignment: A Survey](https://md.research.elamur.ai/free/en/papers/shen-2023-large-language-model-alignment-survey/)
- [Offensive Language and Hate Speech Detection for Danish](https://md.research.elamur.ai/free/en/papers/sigurbergsson-2020-offensive-language-hate-speech-detection/)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://md.research.elamur.ai/free/en/papers/vidgen-2021-learning-worst-dynamically-generated-datasets/)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://md.research.elamur.ai/free/en/papers/zampieri-2019-semeval-2019-task-6-identifying/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zampieri2019predicting,
  title = {{Predicting the Type and Target of Offensive Posts in Social Media}},
  author = {Zampieri, Marcos and Malmasi, Shervin and Nakov, Preslav and Rosenthal, Sara and Farra, Noura and Kumar, Ritesh},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1144},
  url = {https://doi.org/10.18653/v1/n19-1144}
}
```

- [← previous Predicate Catenae: A Dependency Grammar Analysis of It-Clefts](https://md.research.elamur.ai/free/en/papers/osborne-2019-predicate-catenae-dependency-grammar-analysis/)
- [next → Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview](https://md.research.elamur.ai/free/en/papers/shah-2019-predictive-biases-natural-language-processing/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

18 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5:135–146. [find](https://scholar.google.com/scholar?q=Piotr+Bojanowski%2C+Edouard+Grave%2C+Armand+Joulin%2C+and+Tomas+Mikolov%2E+2017%2E+Enriching+Word+Vectors+with+Subword+Information%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+5%3A135%E2%80%93146%2E)
2. Pete Burnap and Matthew L Williams. 2015. Cyber hate speech on twitter: An application of machine classification and statistical modeling for policy and decision making. Policy & Internet, 7(2):223–242. [find](https://scholar.google.com/scholar?q=Pete+Burnap+and+Matthew+L+Williams%2E+2015%2E+Cyber+hate+speech+on+twitter%3A+An+application+of+machine+classification+and+statistical+modeling+for+policy+and+decision+making%2E+Policy+%26+Internet%2C+7%282%29%3A223%E2%80%9324)
3. Maral Dadvar, Dolf Trieschnigg, Roeland Ordelman, and Franciska de Jong. 2013. Improving Cyberbullying Detection with User Context. In Advances in Information Retrieval, pages 693–696. Springer. [find](https://scholar.google.com/scholar?q=Maral+Dadvar%2C+Dolf+Trieschnigg%2C+Roeland+Ordelman%2C+and+Franciska+de+Jong%2E+2013%2E+Improving+Cyberbullying+Detection+with+User+Context%2E+In+Advances+in+Information+Retrieval%2C+pages+693%E2%80%93696%2E+Springer%2E)
4. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of ICWSM. [find](https://scholar.google.com/scholar?q=Thomas+Davidson%2C+Dana+Warmsley%2C+Michael+Macy%2C+and+Ingmar+Weber%2E+2017%2E+Automated+Hate+Speech+Detection+and+the+Problem+of+Offensive+Language%2E+In+Proceedings+of+ICWSM%2E) [our card](https://md.research.elamur.ai/free/en/papers/davidson-2017-automated-hate-speech-detection-problem/)
5. Michael Wiegand, Melanie Siegel, and Josef Ruppenhofer. 2018. Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language. In Proceedings of GermEval. [find](https://scholar.google.com/scholar?q=Michael+Wiegand%2C+Melanie+Siegel%2C+and+Josef+Ruppenhofer%2E+2018%2E+Overview+of+the+GermEval+2018+Shared+Task+on+the+Identification+of+Offensive+Language%2E+In+Proceedings+of+GermEval%2E)
6. Jun-Ming Xu, Kwang-Sung Jun, Xiaojin Zhu, and Amy Bellmore. 2012. Learning from Bullying Traces in Social Media. In Proceedings of NAACL. [find](https://scholar.google.com/scholar?q=Jun%2DMing+Xu%2C+Kwang%2DSung+Jun%2C+Xiaojin+Zhu%2C+and+Amy+Bellmore%2E+2012%2E+Learning+from+Bullying+Traces+in+Social+Media%2E+In+Proceedings+of+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/xu-2012-learning-bullying-traces-social-media/)
7. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Ahmed Ali, Suwon Shon, James Glass, Yves Scherrer, Tanja Samardžić, Nikola Ljubešić, Jörg Tiedemann, et al. 2018. Language Identification and Morphosyntactic Tagging: The Second VarDial Evaluation Campaign. In Proceedings of VarDial. [find](https://scholar.google.com/scholar?q=Marcos+Zampieri%2C+Shervin+Malmasi%2C+Preslav+Nakov%2C+Ahmed+Ali%2C+Suwon+Shon%2C+James+Glass%2C+Yves+Scherrer%2C+Tanja+Samardz%CC%8Cic%CC%81%2C+Nikola+Ljubes%CC%8Cic%CC%81%2C+Jo%CC%88rg+Tiedemann%2C+et+al%2E+2018%2E+Language+Identification+and+Morp) [our card](https://md.research.elamur.ai/free/en/papers/zampieri-2018-language-identification-morphosyntactic-tagging-second/)
8. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval). In Proceedings of SemEval. [find](https://scholar.google.com/scholar?q=Marcos+Zampieri%2C+Shervin+Malmasi%2C+Preslav+Nakov%2C+Sara+Rosenthal%2C+Noura+Farra%2C+and+Ritesh+Kumar%2E+2019%2E+SemEval%2D2019+Task+6%3A+Identifying+and+Categorizing+Offensive+Language+in+Social+Media+%28OffensEval%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/zampieri-2019-semeval-2019-task-6-identifying/)
9. Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015. Hate Speech Detection with Comment Embeddings. In Proceedings of WWW. [find](https://scholar.google.com/scholar?q=Nemanja+Djuric%2C+Jing+Zhou%2C+Robin+Morris%2C+Mihajlo+Grbovic%2C+Vladan+Radosavljevic%2C+and+Narayan+Bhamidipati%2E+2015%2E+Hate+Speech+Detection+with+Comment+Embeddings%2E+In+Proceedings+of+WWW%2E)
10. Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Yoon+Kim%2E+2014%2E+Convolutional+Neural+Networks+for+Sentence+Classification%2E+Proceedings+of+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/kim-2014-convolutional-neural-networks-sentence-classification/)
11. Ritesh Kumar, Atul Kr Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. Benchmarking Aggression Identification in Social Media. In Proceedings of TRAC. [find](https://scholar.google.com/scholar?q=Ritesh+Kumar%2C+Atul+Kr+Ojha%2C+Shervin+Malmasi%2C+and+Marcos+Zampieri%2E+2018%2E+Benchmarking+Aggression+Identification+in+Social+Media%2E+In+Proceedings+of+TRAC%2E) [our card](https://md.research.elamur.ai/free/en/papers/kumar-2018-benchmarking-aggression-identification-social-media/)
12. Irene Kwok and Yuzhou Wang. 2013. Locate the Hate: Detecting Tweets Against Blacks. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Irene+Kwok+and+Yuzhou+Wang%2E+2013%2E+Locate+the+Hate%3A+Detecting+Tweets+Against+Blacks%2E+In+Proceedings+of+AAAI%2E)
13. Shervin Malmasi and Marcos Zampieri. 2017. Detecting Hate Speech in Social Media. In Proceedings of RANLP. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi+and+Marcos+Zampieri%2E+2017%2E+Detecting+Hate+Speech+in+Social+Media%2E+In+Proceedings+of+RANLP%2E)
14. Shervin Malmasi and Marcos Zampieri. 2018. Challenges in Discriminating Profanity from Hate Speech. Journal of Experimental & Theoretical Artificial Intelligence, 30:1 – 16. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi+and+Marcos+Zampieri%2E+2018%2E+Challenges+in+Discriminating+Profanity+from+Hate+Speech%2E+Journal+of+Experimental+%26+Theoretical+Artificial+Intelligence%2C+30%3A1+%E2%80%93+16%2E)
15. Hamdy Mubarak, Darwish Kareem, and Magdy Walid. 2017. Abusive Language Detection on Arabic Social Media. In Proceedings of ALW. [find](https://scholar.google.com/scholar?q=Hamdy+Mubarak%2C+Darwish+Kareem%2C+and+Magdy+Walid%2E+2017%2E+Abusive+Language+Detection+on+Arabic+Social+Media%2E+In+Proceedings+of+ALW%2E) [our card](https://md.research.elamur.ai/free/en/papers/mubarak-2017-abusive-language-detection-arabic-social/)
16. Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive Language Detection in Online User Content. In Proceedings of WWW. [find](https://scholar.google.com/scholar?q=Chikashi+Nobata%2C+Joel+Tetreault%2C+Achint+Thomas%2C+Yashar+Mehdad%2C+and+Yi+Chang%2E+2016%2E+Abusive+Language+Detection+in+Online+User+Content%2E+In+Proceedings+of+WWW%2E)
17. Mohammad Sadegh Rasooli, Noura Farra, Axinia Radeva, Tao Yu, and Kathleen McKeown. 2018. Cross-lingual Sentiment Transfer with Limited Resources. Machine Translation, 32(1-2):143–165. [find](https://scholar.google.com/scholar?q=Mohammad+Sadegh+Rasooli%2C+Noura+Farra%2C+Axinia+Radeva%2C+Tao+Yu%2C+and+Kathleen+McKeown%2E+2018%2E+Cross%2Dlingual+Sentiment+Transfer+with+Limited+Resources%2E+Machine+Translation%2C+32%281%2D2%29%3A143%E2%80%93165%2E)
18. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding Abuse: A Typology of Abusive Language Detection Subtasks. Proceedings of ALW. [find](https://scholar.google.com/scholar?q=Zeerak+Waseem%2C+Thomas+Davidson%2C+Dana+Warmsley%2C+and+Ingmar+Weber%2E+2017%2E+Understanding+Abuse%3A+A+Typology+of+Abusive+Language+Detection+Subtasks%2E+Proceedings+of+ALW%2E) [our card](https://md.research.elamur.ai/free/en/papers/waseem-2017-understanding-abuse-typology-abusive-language/)