--- title: "Hate Speech Dataset from a White Supremacy — Gibert 2018" description: "Automatic hate speech detection lacks shared, openly released training data." authors: ["Ona de Gibert", "Naiara Perez", "Aitor García-Pablos", "Montse Cuadros"] year: 2018 venue: "Proceedings of the 2nd Workshop on Abusive Language Online (ALW2)" source: "https://aclanthology.org/W18-5102/" doi: "10.18653/v1/w18-5102" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/gibert-2018-hate-speech-dataset-white-supremacy/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gibert-2018-hate-speech-dataset-white-supremacy/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / Hate Speech Dataset from a White Supremacy Forum # Hate Speech Dataset from a White Supremacy Forum **Authors:** [Ona de Gibert](https://md.research.elamur.ai/free/en/authors/ona-de-gibert/), [Naiara Perez](https://md.research.elamur.ai/free/en/authors/naiara-perez/), Aitor García-Pablos, [Montse Cuadros](https://md.research.elamur.ai/free/en/authors/montse-cuadros/) · **Year:** 2018 · **Venue:** Proceedings of the 2nd Workshop on Abusive Language Online (ALW2) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/W18-5102/) · [PDF](https://aclanthology.org/W18-5102.pdf) · [DOI 10.18653/v1/w18-5102](https://doi.org/10.18653/v1/w18-5102) · [OpenAlex](https://openalex.org/W2890727387) **Citations:** 425 citations (openalex-snapshot · 2026-10-14) · [152](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Automatic hate speech detection lacks shared, openly released training data. Earlier annotation efforts often used loosely specified criteria, which produced inconsistent labels. The Vicomtech authors address this with a public English corpus drawn from Stormfront, a large white nationalist forum. They sampled posts from 22 sub-forums, filtered out non-English text, and split the posts into sentences with ixa-pipes. Annotators then labelled each sentence with a custom tool that lets them open the surrounding post when meaning is unclear. The annotators wrote the guidelines together and refined them over the first batches. A sentence counts as hate only if it is a deliberate attack, targets a group, and is motivated by that group's identity. The paper analyses the resulting data and reports baseline classifiers, including SVM and CNN models. Its error analysis shows two common failure types: sentences that need context or world knowledge, and offensive vocabulary used without hateful intent. The dataset matters because it offers sentence-level labels plus conversation metadata, giving a reusable benchmark. It also shows how subjective hate annotation remains. ## Key points - 10,568 sentences were annotated in 10 batches, each built from 500 posts (about 1,000 sentences). Forum content dates from 2002 to 2017. - Cleaning removed sentences under 3 or over 50 words, discarding 652 sentences (6.17%). Both the raw and the "clean" versions are released on GitHub (aitor-garcia-p/hate-speech-dataset). - The labels are HATE and NOHATE plus two auxiliary classes. One of these is RELATION, used when several sentences together form hate; it was rarely applied. - Each sentence comes with anonymised post, user and sub-forum IDs, its position in the post, and the number of context posts the annotator consulted. - Annotators needed less and less context in later batches as they gained experience. The authors suggest active learning to cope with class imbalance in future annotation. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 152 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 139 not yet. - [Toward Gender-Inclusive Coreference Resolution](https://md.research.elamur.ai/free/en/papers/cao-2020-gender-inclusive-coreference-resolution/) - [Toward Gender-Inclusive Coreference Resolution: An Analysis of Gender and Bias Throughout the Machine Learning Lifecycle*](https://md.research.elamur.ai/free/en/papers/cao-2021-gender-inclusive-coreference-resolution-analysis/) - [CONAN - COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech](https://md.research.elamur.ai/free/en/papers/chung-2019-conan-counter-narratives-through-nichesourcing/) - [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://md.research.elamur.ai/free/en/papers/elsherief-2021-latent-hatred-benchmark-understanding-implicit/) - [Hate Towards the Political Opponent: A Twitter Corpus Study of the 2020 US Elections on the Basis of Offensive Speech and Stance Detection](https://md.research.elamur.ai/free/en/papers/grimminger-2021-hate-political-opponent-twitter-corpus/) - [ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection](https://md.research.elamur.ai/free/en/papers/hartvigsen-2022-toxigen-large-scale-machine-generated/) - [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://md.research.elamur.ai/free/en/papers/inan-2023-llama-guard-llm-based-input/) - [Contextualizing Hate Speech Classifiers with Post-hoc Explanation](https://md.research.elamur.ai/free/en/papers/kennedy-2020-contextualizing-hate-speech-classifiers-post/) - [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/) - [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://md.research.elamur.ai/free/en/papers/qi-2021-mind-style-text-adversarial-backdoor/) - [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/) - [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://md.research.elamur.ai/free/en/papers/sap-2020-social-bias-frames-reasoning-about/) - [Large Language Model Alignment: A Survey](https://md.research.elamur.ai/free/en/papers/shen-2023-large-language-model-alignment-survey/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{gibert2018hate, title = {{Hate Speech Dataset from a White Supremacy Forum}}, author = {Gibert, Ona de and Perez, Naiara and García-Pablos, Aitor and Cuadros, Montse}, booktitle = {Proceedings of the 2nd Workshop on Abusive Language Online (ALW2)}, year = {2018}, doi = {10.18653/v1/w18-5102}, url = {https://doi.org/10.18653/v1/w18-5102} } ``` - [← previous Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation](https://md.research.elamur.ai/free/en/papers/laubli-2018-has-machine-translation-achieved-human/) - [next → Higher-Order Coreference Resolution with Coarse-to-Fine Inference](https://md.research.elamur.ai/free/en/papers/lee-2018-higher-order-coreference-resolution-coarse/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Conclusions and Future Work R. Agerri, J. Bermudez, and G. Rigau. 2014. IXA pipeline: Efficient and Ready to Use Multilingual NLP tools. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 3823–3828. [find](https://scholar.google.com/scholar?q=Conclusions+and+Future+Work+R%2E+Agerri%2C+J%2E+Bermudez%2C+and+G%2E+Rigau%2E+2014%2E+IXA+pipeline%3A+Efficient+and+Ready+to+Use+Multilingual+NLP+tools%2E+In+Proceedings+of+the+Ninth+International+Conference+on+Languag)