---
title: "The Role of Computational Stylometry in Identifying"
description: "This paper reports the UniOr ExpSys team's entry in the TRAC-2 shared task at LREC… 0.78 accuracy. 42 references, cited by 2 papers here."
authors: ["Antonio Pascucci", "Raffaele Manna", "Vincenzo Masucci", "Johanna Monti"]
year: 2020
venue: "Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying"
source: "https://aclanthology.org/2020.trac-1.11/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pascucci-2020-role-computational-stylometry-identifying-misogynistic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pascucci-2020-role-computational-stylometry-identifying-misogynistic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/9/) / The Role of Computational Stylometry in Identifying (Misogynistic) Aggression in English Social Media Texts

# The Role of Computational Stylometry in Identifying (Misogynistic) Aggression in English Social Media Texts

**Authors:** Antonio Pascucci, Raffaele Manna, Vincenzo Masucci and [Johanna Monti](https://md.research.elamur.ai/free/en/authors/johanna-monti/) · **Year:** 2020 · **Venue:** [Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying](https://md.research.elamur.ai/free/en/venues/trac/) ([Workshop on Trolling, Aggression and Cyberbullying 2020](https://md.research.elamur.ai/free/en/venues/trac/2020/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.trac-1.11/) · [PDF](https://aclanthology.org/2020.trac-1.11.pdf) · [OpenAlex](https://openalex.org/W3030082272)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/pascucci-2020-role-computational-stylometry-identifying-misogynistic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This paper reports the UniOr ExpSys team's entry in the TRAC-2 shared task at LREC 2020. The team built systems for two English sub-tasks. The first sorts social media posts into overtly aggressive, covertly aggressive or non-aggressive. The second decides whether a post is gendered (misogynistic) or not. The team skipped the Bangla and Hindi tracks because their method relies on linguistic expertise they lack for those languages.

Their pipeline combines three parts:
- **Computational stylometry**: style cues that reflect who an author is and how they write.
- **Hand-written linguistic rules**: authored in Expert System's COGITO, which supports word-sense disambiguation through its semantic network.
- **A machine-learned classifier**: Sequential Minimal Optimization, an SVM training algorithm.

The paper also surveys earlier hate-speech and aggression benchmarks, including the TRAC-1 top scores. The provided extract omits the methodology details and the results section, so this summary cannot report the team's own scores. The work is relevant because it treats aggression detection as partly a question of authorial style, not only of vocabulary.

## Key points
- TRAC-2 has two sub-tasks: a 3-way OAG/CAG/NAG aggression task and a binary gendered/non-gendered misogyny task, offered in Bangla, Hindi and English. The authors entered English only.
- The method is hybrid. Stylometric features and COGITO rules feed an SMO classifier (Platt, 1998). The bibliography also cites Weka, random forests and logistic regression.
- The stylometric cues listed include:
  - sentence length and word-length distributions
  - vocabulary richness and punctuation
  - first- vs third-person usage and specific verb/adjective classes
  - n-grams, readability indices and metaphor use
- TRAC-1 English top systems scored the following weighted F1 (Facebook test / surprise tweet test):
  - saroyehun: 0.642 / 0.592
  - Julian: 0.601 / 0.599
  - vista.ue: 0.581 / 0.600
- TRAC-1 English test data comprised 916 Facebook comments plus a surprise set of 1,257 tweets. In a related study, Malmasi & Zampieri reached 0.78 accuracy with character 4-grams on the 14,509-tweet Davidson dataset.

## Interesting ideas
- Aggression is split into overt and covert, so hostility can be flagged even when no insulting word appears; that separates tone from vocabulary.
- Interpretable style measurements (person usage, sentence length, punctuation, readability) are treated as signals about the writer rather than about the topic.
- Rules are written on top of word-sense disambiguation, so a rule matches a meaning, not a surface string, which avoids false hits on ambiguous slurs or neutral homographs.
- The TRAC-1 organizers added a surprise test set from a different platform (tweets instead of Facebook comments), which exposes systems that overfit one source.
- A cited WhatsApp corpus labels participant roles in bullying (harasser, victim, bystander-defender, bystander-assistant), which casts abuse as a structured event.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): A "covert aggression" speech-act label in prag could be triggered by implicature and irony cues over the frame, instead of by lexical lists.
- **I2** (world): Each text's world could store an "author profile" node built from such measures, giving explainable evidence about the speaker for later reasoning.
- **I3** (frames): Sense-keyed WordNet/VerbNet triggers in prag rules would let offensive-usage detection stay explainable and robust to polysemy.
- **I4** (learn): Evaluation could routinely include an out-of-domain held-out set as a control, flagging repair rules or silver data that only help in-domain.
- **I5** (frames): These roles map naturally onto who-did-what-to-whom frames, letting Mova answer "who defended whom" questions from dialogue.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Antonio Pascucci, Raffaele Manna, Vincenzo Masucci and Johanna Monti. 2020. The Role of Computational Stylometry in Identifying (Misogynistic) Aggression in English Social Media Texts. Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pascucci2020role,
  title = {{The Role of Computational Stylometry in Identifying (Misogynistic) Aggression in English Social Media Texts}},
  author = {Antonio Pascucci, Raffaele Manna, Vincenzo Masucci and Johanna Monti},
  booktitle = {Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying},
  year = {2020},
  url = {https://aclanthology.org/2020.trac-1.11/}
}
```

- [← previous The article <i>a(n)</i> in English quantifying expressions: A default marker of cardinality](https://md.research.elamur.ai/free/en/papers/klockmann-2020-article-i-n-i-english/)
- [next → The role of context in neural pitch accent detection in English](https://md.research.elamur.ai/free/en/papers/nielsen-2020-role-context-neural-pitch-accent/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/9/)

## References

42 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Argamon, S., Šarić, M., and Stein, S. S. (2003). Style mining of electronic messages for multiple authorship discrimination: first results. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 475–480. ACM. [find](https://scholar.google.com/scholar?q=Argamon%2C+S%2E%2C+S%CC%8Caric%CC%81%2C+M%2E%2C+and+Stein%2C+S%2E+S%2E+%282003%29%2E+Style+mining+of+electronic+messages+for+multiple+authorship+discrimination%3A+first+results%2E+In+Proceedings+of+the+ninth+ACM+SIGKDD+international+confe)
2. Aroyehun, S. T. and Gelbukh, A. (2018). Aggression detection in social media: Using deep neural networks, data augmentation, and pseudo labeling. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pages 90–97. [find](https://scholar.google.com/scholar?q=Aroyehun%2C+S%2E+T%2E+and+Gelbukh%2C+A%2E+%282018%29%2E+Aggression+detection+in+social+media%3A+Using+deep+neural+networks%2C+data+augmentation%2C+and+pseudo+labeling%2E+In+Proceedings+of+the+First+Workshop+on+Trolling%2C+Aggr)
3. Baayen, H., Van Halteren, H., and Tweedie, F. (1996). Outside the cave of shadows: Using syntactic annotation to enhance authorship attribution. Literary and Linguistic Computing, 11(3):121–132. [↗](https://doi.org/10.1093/llc/11.3.121)
4. Basile, V., Bosco, C., Fersini, E., Nozza, D., Patti, V., Pardo, F. M. R., Rosso, P., and Sanguinetti, M. (2019). [Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter](https://md.research.elamur.ai/free/en/papers/basile-2019-semeval-2019-task-5-multilingual/). In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 54–63.
5. Bhattacharya, S., Singh, S., Kumar, R., Bansal, A., Bhagat, A., Dawer, Y., Lahiri, B., and Ojha, A. K. (2020). Developing a multilingual annotated corpus of misogyny and aggression. [find](https://scholar.google.com/scholar?q=Bhattacharya%2C+S%2E%2C+Singh%2C+S%2E%2C+Kumar%2C+R%2E%2C+Bansal%2C+A%2E%2C+Bhagat%2C+A%2E%2C+Dawer%2C+Y%2E%2C+Lahiri%2C+B%2E%2C+and+Ojha%2C+A%2E+K%2E+%282020%29%2E+Developing+a+multilingual+annotated+corpus+of+misogyny+and+aggression%2E)
6. Bosco, C., Dell’Orletta, F., Poletto, F., Sanguinetti, M., and Tesconi, M. (2018). [Overview of the evalita 2018 hate speech detection task](https://md.research.elamur.ai/free/en/papers/bosco-2018-overview-evalita-2018-hate-speech/). In EVALITA 2018-Sixth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian, volume 2263, pages 1–9. CEUR. [↗](https://doi.org/10.4000/books.aaccademia.4503)
7. Britannica, E. (2015). Britannica academic. Encyclopædia Britannica Inc. [find](https://scholar.google.com/scholar?q=Britannica%2C+E%2E+%282015%29%2E+Britannica+academic%2E+Encyclop%C3%A6dia+Britannica+Inc%2E)
8. Burnap, P. and Williams, M. L. (2015). Cyber hate speech on twitter: An application of machine classification and statistical modeling for policy and decision making. Policy & Internet, 7(2):223–242. 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 [↗](https://doi.org/10.1002/poi3.85)
9. Capozzi, A. T., Lai, M., Basile, V., Poletto, F., Sanguinetti, M., Bosco, C., Patti, V., Ruffo, G., Musto, C., Polignano, M., et al. (2019). Computational linguistics against hate: Hate speech detection and visualization on social media in the” contro l’odio” project. In 6th Italian Conference on Computational Linguistics, CLiC-it 2019, volume 2481, pages 1–6. CEUR-WS. [find](https://scholar.google.com/scholar?q=Capozzi%2C+A%2E+T%2E%2C+Lai%2C+M%2E%2C+Basile%2C+V%2E%2C+Poletto%2C+F%2E%2C+Sanguinetti%2C+M%2E%2C+Bosco%2C+C%2E%2C+Patti%2C+V%2E%2C+Ruffo%2C+G%2E%2C+Musto%2C+C%2E%2C+Polignano%2C+M%2E%2C+et+al%2E+%282019%29%2E+Computational+linguistics+against+hate%3A+Hate+speech+detecti)
10. Dadvar, M., Trieschnigg, D., Ordelman, R., and de Jong, F. (2013). Improving cyberbullying detection with user context. In Advances in Information Retrieval, pages 693–696. Springer. [find](https://scholar.google.com/scholar?q=Dadvar%2C+M%2E%2C+Trieschnigg%2C+D%2E%2C+Ordelman%2C+R%2E%2C+and+de+Jong%2C+F%2E+%282013%29%2E+Improving+cyberbullying+detection+with+user+context%2E+In+Advances+in+Information+Retrieval%2C+pages+693%E2%80%93696%2E+Springer%2E)
11. Daelemans, W. (2013). Explanation in computational stylometry. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 451– 462. Springer. [find](https://scholar.google.com/scholar?q=Daelemans%2C+W%2E+%282013%29%2E+Explanation+in+computational+stylometry%2E+In+International+Conference+on+Intelligent+Text+Processing+and+Computational+Linguistics%2C+pages+451%E2%80%93+462%2E+Springer%2E)
12. Davidson, T., Warmsley, D., Macy, M., and Weber, I. (2017). [Automated hate speech detection and the problem of offensive language](https://md.research.elamur.ai/free/en/papers/davidson-2017-automated-hate-speech-detection-problem/). In Eleventh international aaai conference on web and social media. [↗](https://doi.org/10.48550/arxiv.1703.04009)
13. De Smedt, T., Jaki, S., Kotzé, E., Saoud, L., Gwóźdź, M., De Pauw, G., and Daelemans, W. (2018). Multilingual cross-domain perspectives on online hate speech. arXiv preprint arXiv:1809.03944. [↗](https://arxiv.org/abs/1809.03944)
14. De Vel, O., Anderson, A., Corney, M., and Mohay, G. (2001). Mining e-mail content for author identification forensics. ACM Sigmod Record, 30(4):55–64. [↗](https://doi.org/10.1145/604264.604272)
15. Del Vigna, F., Cimino, A., Dell’Orletta, F., Petrocchi, M., and Tesconi, M. (2017). Hate me, hate me not: Hate speech detection on facebook. In Proceedings of the First Italian Conference on Cybersecurity (ITASEC17), pages 86–95. [find](https://scholar.google.com/scholar?q=Del+Vigna%2C+F%2E%2C+Cimino%2C+A%2E%2C+Dell%E2%80%99Orletta%2C+F%2E%2C+Petrocchi%2C+M%2E%2C+and+Tesconi%2C+M%2E+%282017%29%2E+Hate+me%2C+hate+me+not%3A+Hate+speech+detection+on+facebook%2E+In+Proceedings+of+the+First+Italian+Conference+on+Cybersecu)
16. Dinakar, K., Reichart, R., and Lieberman, H. (2011). Modeling the detection of textual cyberbullying. In The Social Mobile Web, pages 11–17. [find](https://scholar.google.com/scholar?q=Dinakar%2C+K%2E%2C+Reichart%2C+R%2E%2C+and+Lieberman%2C+H%2E+%282011%29%2E+Modeling+the+detection+of+textual+cyberbullying%2E+In+The+Social+Mobile+Web%2C+pages+11%E2%80%9317%2E)
17. Fersini, E., Nozza, D., and Rosso, P. (2018). [Overview of the evalita 2018 task on automatic misogyny identification (ami)](https://md.research.elamur.ai/free/en/papers/fersini-2018-overview-evalita-2018-task-automatic/). EVALITA Evaluation of NLP and Speech Tools for Italian, 12:59. [↗](https://doi.org/10.4000/books.aaccademia.4497)
18. Fortuna, P. and Nunes, S. (2018). A survey on automatic detection of hate speech in text. ACM Computing Surveys (CSUR), 51(4):1–30. [↗](https://doi.org/10.1145/3232676)
19. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. (2009). The weka data mining software: an update. ACM SIGKDD explorations newsletter, 11(1):10–18. [↗](https://doi.org/10.1145/1656274.1656278)
20. Kumar, R., Ojha, A. K., Malmasi, S., and Zampieri, M. (2018a). [Benchmarking aggression identification in social media](https://md.research.elamur.ai/free/en/papers/kumar-2018-benchmarking-aggression-identification-social-media/). In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pages 1–11, Santa Fe, New Mexico, USA, August. Association for Computational Linguistics.
21. Kumar, R., Ojha, A. K., Malmasi, S., and Zampieri, M. (2018b). [Benchmarking Aggression Identification in Social Media](https://md.research.elamur.ai/free/en/papers/kumar-2018-benchmarking-aggression-identification-social-media/). In Proceedings of the First Workshop on Trolling, Aggression and Cyberbulling (TRAC), Santa Fe, USA.
22. Liaw, A., Wiener, M., et al. (2002). Classification and regression by randomforest. R news, 2(3):18–22. [find](https://scholar.google.com/scholar?q=Liaw%2C+A%2E%2C+Wiener%2C+M%2E%2C+et+al%2E+%282002%29%2E+Classification+and+regression+by+randomforest%2E+R+news%2C+2%283%29%3A18%E2%80%9322%2E)
23. Lucisano, P. and Piemontese, M. E. (1988). Gulpease: una formula per la predizione della difficoltà dei testi in lingua italiana. Scuola e città, 3(31):110–124. [find](https://scholar.google.com/scholar?q=Lucisano%2C+P%2E+and+Piemontese%2C+M%2E+E%2E+%281988%29%2E+Gulpease%3A+una+formula+per+la+predizione+della+difficolta%CC%80+dei+testi+in+lingua+italiana%2E+Scuola+e+citta%CC%80%2C+3%2831%29%3A110%E2%80%93124%2E)
24. Malmasi, S. and Zampieri, M. (2017). Detecting hate speech in social media. arXiv preprint arXiv:1712.06427. [↗](https://arxiv.org/abs/1712.06427)
25. Menini, S., Moretti, G., Corazza, M., Cabrio, E., Tonelli, S., and Villata, S. (2019). A system to monitor cyberbullying based on message classification and social network analysis. In Proceedings of the Third Workshop on Abusive Language Online, pages 105–110. [find](https://scholar.google.com/scholar?q=Menini%2C+S%2E%2C+Moretti%2C+G%2E%2C+Corazza%2C+M%2E%2C+Cabrio%2C+E%2E%2C+Tonelli%2C+S%2E%2C+and+Villata%2C+S%2E+%282019%29%2E+A+system+to+monitor+cyberbullying+based+on+message+classification+and+social+network+analysis%2E+In+Proceedings+of+)
26. Musto, C., Semeraro, G., de Gemmis, M., and Lops, P. (2016). Modeling community behavior through semantic analysis of social data: The italian hate map experience. In Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization, pages 307– 308. [find](https://scholar.google.com/scholar?q=Musto%2C+C%2E%2C+Semeraro%2C+G%2E%2C+de+Gemmis%2C+M%2E%2C+and+Lops%2C+P%2E+%282016%29%2E+Modeling+community+behavior+through+semantic+analysis+of+social+data%3A+The+italian+hate+map+experience%2E+In+Proceedings+of+the+2016+Conferenc)
27. Pascucci, A., Masucci, V., and Monti, J. (2019). Computational stylometry and machine learning for gender and age detection in cyberbullying texts. In 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), pages 1–6. IEEE. [find](https://scholar.google.com/scholar?q=Pascucci%2C+A%2E%2C+Masucci%2C+V%2E%2C+and+Monti%2C+J%2E+%282019%29%2E+Computational+stylometry+and+machine+learning+for+gender+and+age+detection+in+cyberbullying+texts%2E+In+2019+8th+International+Conference+on+Affective+Co)
28. Peng, C.-Y. J., Lee, K. L., and Ingersoll, G. M. (2002). An introduction to logistic regression analysis and reporting. The journal of educational research, 96(1):3–14. [↗](https://doi.org/10.1080/00220670209598786)
29. Platt, J. (1998). Sequential minimal optimization: A fast algorithm for training support vector machines. [find](https://scholar.google.com/scholar?q=Platt%2C+J%2E+%281998%29%2E+Sequential+minimal+optimization%3A+A+fast+algorithm+for+training+support+vector+machines%2E)
30. Poletto, F., Stranisci, M., Sanguinetti, M., Patti, V., and Bosco, C. (2017). Hate speech annotation: Analysis of an italian twitter corpus. In 4th Italian Conference on Computational Linguistics, CLiC-it 2017, volume 2006, pages 1–6. CEUR-WS. [find](https://scholar.google.com/scholar?q=Poletto%2C+F%2E%2C+Stranisci%2C+M%2E%2C+Sanguinetti%2C+M%2E%2C+Patti%2C+V%2E%2C+and+Bosco%2C+C%2E+%282017%29%2E+Hate+speech+annotation%3A+Analysis+of+an+italian+twitter+corpus%2E+In+4th+Italian+Conference+on+Computational+Linguistics%2C+CLi)
31. Raiyani, K., Gonçalves, T., Quaresma, P., and Nogueira, V. B. (2018). Fully connected neural network with advance preprocessor to identify aggression over facebook and twitter. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pages 28–41. [find](https://scholar.google.com/scholar?q=Raiyani%2C+K%2E%2C+Gonc%CC%A7alves%2C+T%2E%2C+Quaresma%2C+P%2E%2C+and+Nogueira%2C+V%2E+B%2E+%282018%29%2E+Fully+connected+neural+network+with+advance+preprocessor+to+identify+aggression+over+facebook+and+twitter%2E+In+Proceedings+of+the+)
32. Ramakrishnan, M., Zadrozny, W., and Tabari, N. (2019). Uva wahoos at semeval-2019 task 6: Hate speech identification using ensemble machine learning. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 806–811. [find](https://scholar.google.com/scholar?q=Ramakrishnan%2C+M%2E%2C+Zadrozny%2C+W%2E%2C+and+Tabari%2C+N%2E+%282019%29%2E+Uva+wahoos+at+semeval%2D2019+task+6%3A+Hate+speech+identification+using+ensemble+machine+learning%2E+In+Proceedings+of+the+13th+International+Workshop+)
33. Risch, J. and Krestel, R. (2018). Aggression identification using deep learning and data augmentation. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pages 150–158. Ritesh Kumar, Atul Kr. [find](https://scholar.google.com/scholar?q=Risch%2C+J%2E+and+Krestel%2C+R%2E+%282018%29%2E+Aggression+identification+using+deep+learning+and+data+augmentation%2E+In+Proceedings+of+the+First+Workshop+on+Trolling%2C+Aggression+and+Cyberbullying+%28TRAC%2D2018%29%2C+pages)
34. Ojha, S. M. and Zampieri, M. (2020). Evaluating aggression identification in social media. In Ritesh Kumar, et al., editors, Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC-2020), Paris, France, may. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Ojha%2C+S%2E+M%2E+and+Zampieri%2C+M%2E+%282020%29%2E+Evaluating+aggression+identification+in+social+media%2E+In+Ritesh+Kumar%2C+et+al%2E%2C+editors%2C+Proceedings+of+the+Second+Workshop+on+Trolling%2C+Aggression+and+Cyberbullyin)
35. Sanguinetti, M., Poletto, F., Bosco, C., Patti, V., and Stranisci, M. (2018). [An italian twitter corpus of hate speech against immigrants](https://md.research.elamur.ai/free/en/papers/sanguinetti-2018-italian-twitter-corpus-hate-speech/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
36. Schmidt, A. and Wiegand, M. (2017). [A Survey on Hate Speech Detection Using Natural Language Processing](https://md.research.elamur.ai/free/en/papers/schmidt-2017-survey-hate-speech-detection-natural/). In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media. Association for Computational Linguistics, pages 1–10, Valencia, Spain.
37. Sprugnoli, R., Menini, S., Tonelli, S., Oncini, F., and Piras, E. (2018). Creating a whatsapp dataset to study pre-teen cyberbullying. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pages 51–59. [find](https://scholar.google.com/scholar?q=Sprugnoli%2C+R%2E%2C+Menini%2C+S%2E%2C+Tonelli%2C+S%2E%2C+Oncini%2C+F%2E%2C+and+Piras%2C+E%2E+%282018%29%2E+Creating+a+whatsapp+dataset+to+study+pre%2Dteen+cyberbullying%2E+In+Proceedings+of+the+2nd+Workshop+on+Abusive+Language+Online+%28AL)
38. Wiegand, M., Siegel, M., and Ruppenhofer, J. (2018). Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language. In Proceedings of GermEval. [find](https://scholar.google.com/scholar?q=Wiegand%2C+M%2E%2C+Siegel%2C+M%2E%2C+and+Ruppenhofer%2C+J%2E+%282018%29%2E+Overview+of+the+GermEval+2018+Shared+Task+on+the+Identification+of+Offensive+Language%2E+In+Proceedings+of+GermEval%2E)
39. Xu, J.-M., Jun, K.-S., Zhu, X., and Bellmore, A. (2012). [Learning from bullying traces in social media](https://md.research.elamur.ai/free/en/papers/xu-2012-learning-bullying-traces-social-media/). In Proceedings of the 2012 conference of the North American chapter of the association for computational linguistics: Human language technologies, pages 656–666. Association for Computational Linguistics.
40. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., and Kumar, R. (2019a). [Predicting the type and target of offensive posts in social media](https://md.research.elamur.ai/free/en/papers/zampieri-2019-predicting-type-target-offensive-posts/). arXiv preprint arXiv:1902.09666. [↗](https://arxiv.org/abs/1902.09666)
41. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., and Kumar, R. (2019b). [Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval)](https://md.research.elamur.ai/free/en/papers/zampieri-2019-semeval-2019-task-6-identifying/). arXiv preprint arXiv:1903.08983. [↗](https://arxiv.org/abs/1903.08983)
42. Zheng, R., Li, J., Chen, H., and Huang, Z. (2006). A framework for authorship identification of online messages: Writing-style features and classification techniques. Journal of the American society for information science and technology, 57(3):378–393. [↗](https://doi.org/10.1002/asi.20316)

*Updated 1 November 2026*