---
title: "System Demo — paper by Grace et al. (2022)"
description: "The authors present OLEA, an open-source Python library ( pip install olea ) for fine-grained error analysis of such classifiers… 17 references."
authors: ["Marie Grace", "Xajavion \"Jay\" Seabrum", "Dananjay Srinivas", "Alexis Palmer"]
year: 2022
venue: "arXiv:2210.16398"
source: "https://arxiv.org/abs/2210.16398"
doi: "10.48550/arxiv.2210.16398"
arxiv: "2210.16398"
licence: "CC BY-SA 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/grace-2022-system-demo-tool-infrastructure-offensive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/grace-2022-system-demo-tool-infrastructure-offensive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/9/) / System Demo: Tool and Infrastructure for Offensive Language Error Analysis (OLEA) in English

# System Demo: Tool and Infrastructure for Offensive Language Error Analysis (OLEA) in English

**Authors:** [Marie Grace](https://md.research.elamur.ai/free/en/authors/marie-grace/), Xajavion "Jay" Seabrum, [Dananjay Srinivas](https://md.research.elamur.ai/free/en/authors/dananjay-srinivas/) and [Alexis Palmer](https://md.research.elamur.ai/free/en/authors/alexis-palmer/) · **Year:** 2022 · **Venue:** arXiv:2210.16398 · **Licence:** [CC BY-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-sa/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2210.16398) · [PDF](https://arxiv.org/pdf/2210.16398) · [DOI 10.48550/arxiv.2210.16398](https://doi.org/10.48550/arxiv.2210.16398) · [arXiv 2210.16398](https://arxiv.org/abs/2210.16398) · [OpenAlex](https://openalex.org/W4307926930)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Offensive language detectors handle overt cases with explicit slurs or keywords well. They do much worse on implicit, negated or counter-speech messages, and they tend to over-flag African American English. One F1 score hides all of this. The authors present OLEA, an open-source Python library (`pip install olea`) for fine-grained error analysis of such classifiers in English. Users pass in model predictions as a Pandas dataframe, along with a mapping from their labels to the dataset's labels. OLEA then reports per-subset metrics and Matplotlib plots, with subsets defined by linguistic or annotation features. It comes with interfaces to two diagnostic sets, HateCheck and COLD, hosted on the HuggingFace Hub. Users can also add their own datasets and analysis functions by subclassing a single class. A demo on roBERTa-offensive and HateXplain shows large performance drops when the models move to HateCheck, and it shows that counter-speech is mostly misclassified. The tool is meant for model developers rather than end users. It moves offensive-language evaluation away from a single binary score and toward diagnosis by phenomenon.

## Key points
- Generic analysis functions:
  - `analyze_on`: metrics per category of any column.
  - `check_anno_agreement`: compares items with full and partial annotator agreement.
  - `aave`: uses dialect scores from the TwitterAAE model.
  - `check_substring`: compares items with and without a given string.
  - `str_len_analysis`: histogram of performance by length.
- Preloaded diagnostic datasets:
  - **HateCheck** has functional test categories such as negation, derogation, spelling obfuscation and threats, plus the identity being targeted.
  - **COLD** has labels from several annotators for slurs, adjectival nominalization and linguistic distancing.
- Cross-dataset results:
  - roBERTa-offensive falls from an F1 of 0.78 on OLID to 0.62 on HateCheck.
  - HateXplain falls from 0.69 on its own dataset to 0.37 on HateCheck.
- Extending and sharing:
  - OLID (level-A labels only) is used to demonstrate adding a new dataset through `Dataset` and `Analysis` subclasses.
  - A text preprocessor turns user mentions into USER and URLs into a placeholder, and replaces emoji with their text names.
  - The code is under the MIT license.
- Limitations and safeguards:
  - Substring analyses assume space-delimited text.
  - Datasets depend on HuggingFace hosting.
  - Displaying examples is off by default to limit exposure to hateful text.

## Interesting ideas
- Score the model separately on every slice defined by a feature column, instead of reporting only one overall number; this shows exactly which phenomena break it.
- Treat full agreement among annotators as a sign that an item is "easy", and partial agreement as a sign that it is hard, then compare scores on the two groups.
- Run the same model on a diagnostic set from another source to expose large drops caused by domain shift.
- Counter-speech, where hateful words are quoted or argued against, is mostly misclassified; this suggests surface words are being confused with the speaker's stance.
- Let each new dataset ship as a small subclass paired with its own analysis class, so benchmarks plug into a shared harness with little code.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Tag evaluation sentences with phenomena such as negation, coordination or passives, and report LAS and answer accuracy per tag, so regressions point to a specific construction.
- **I2** (learn): Split silver data by how many witness parsers agree, and track accuracy on the low-agreement portion separately, since that is where repair rules matter most.
- **I3** (tree): Test the perceptron parser and gparse on held-out treebanks in a different UD dialect, to check that gains are not tied to one corpus.
- **I4** (frames): Make prag mark quoted and reported content as not asserted by the speaker, so speech-act and stance frames do not take on the attitude of the embedded text.
- **I5** (answer): Package each school-test or fable QA suite as a plugin with suite-specific error breakdowns, so new readers are scored uniformly and with explanations.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Colorado Boulder](https://md.research.elamur.ai/free/en/universities/university-of-colorado-boulder/) — University, Boulder, United States · 69 works · 51,069 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Communication](https://md.research.elamur.ai/free/en/shelves/social-sciences/communication/) · 97 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works

## Cite

Cite as: Marie Grace, Xajavion "Jay" Seabrum, Dananjay Srinivas and Alexis Palmer. 2022. System Demo: Tool and Infrastructure for Offensive Language Error Analysis (OLEA) in English. arXiv:2210.16398.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{grace2022system,
  title = {{System Demo: Tool and Infrastructure for Offensive Language Error Analysis (OLEA) in English}},
  author = {Marie Grace, Xajavion "Jay" Seabrum, Dananjay Srinivas and Alexis Palmer},
  year = {2022},
  doi = {10.48550/arxiv.2210.16398},
  eprint = {2210.16398},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2210.16398}
}
```

- [← previous Survey on Vision-language Pre-training](https://md.research.elamur.ai/free/en/papers/yin-2022-vision-language-pre/)
- [next → Teaching Arabic Grammar to Non-Native Speaker, between Curriculum and Acting on it (Silsiletu Abjad Series)](https://md.research.elamur.ai/free/en/papers/jankir-2022-teaching-arabic-grammar-non-native/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/9/)

## References

17 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Francesco Barbieri, José Camacho-Collados, Leonardo Neves, and Luis Espinosa Anke. 2020. [Tweeteval: Unified benchmark and comparative evaluation for tweet classification](https://md.research.elamur.ai/free/en/papers/barbieri-2020-tweeteval-unified-benchmark-comparative-evaluation/). CoRR, abs/2010.12421. [↗](https://arxiv.org/abs/2010.12421)
2. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. [Demographic dialectal variation in social media: A case study of African-American English](https://md.research.elamur.ai/free/en/papers/blodgett-2016-demographic-dialectal-variation-social-media/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1119–1130, Austin, Texas. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d16-1120)
3. Tommaso Caselli, Valerio Basile, Jelena Mitrović, Inga Kartoziya, and Michael Granitzer. 2020. [I feel offended, don’t be abusive!](https://md.research.elamur.ai/free/en/papers/caselli-2020-i-feel-offended-don-t/) implicit/explicit messages in offensive and abusive language. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 6193–6202, Marseille, France. European Language Resources Association.
4. Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, and Diyi Yang. 2021. [Latent hatred: A benchmark for understanding implicit hate speech](https://md.research.elamur.ai/free/en/papers/elsherief-2021-latent-hatred-benchmark-understanding-implicit/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 345– 363, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2109.05322)
5. Komal Florio, Valerio Basile, Marco Polignano, Pierpaolo Basile, and Viviana Patti. 2020. Time of your hate: The challenge of time in hate speech detection on social media. Applied Sciences, 10(12):4180. [↗](https://doi.org/10.3390/app10124180)
6. Eduard Hovy. 2022. Rediscovering the need for representation and knowledge. ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Eduard+Hovy%2E+2022%2E+Rediscovering+the+need+for+representation+and+knowledge%2E+ACL+2022+%2D+60th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2E)
7. J. D. Hunter. 2007. Matplotlib: A 2d graphics environment. Computing in Science & Engineering, 9(3):90–95. [↗](https://doi.org/10.1109/mcse.2007.55)
8. Chris J Kennedy, Geoff Bacon, Alexander Sahn, and Claudia von Vacano. 2020. Constructing interval variables via faceted rasch measurement and multi-task deep learning: a hate speech application. arXiv preprint arXiv:2009.10277. [↗](https://arxiv.org/abs/2009.10277)
9. Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. Hatexplain: A benchmark dataset for explainable hate speech detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 14867–14875. [find](https://scholar.google.com/scholar?q=Binny+Mathew%2C+Punyajoy+Saha%2C+Seid+Muhie+Yimam%2C+Chris+Biemann%2C+Pawan+Goyal%2C+and+Animesh+Mukherjee%2E+2021%2E+Hatexplain%3A+A+benchmark+dataset+for+explainable+hate+speech+detection%2E+In+Proceedings+of+the+AAA)
10. Angelina McMillan-Major, Amandalynne Paullada, and Yacine Jernite. 2022. An interactive exploratory tool for the task of hate speech detection. In Proceedings of the Second Workshop on Bridging Human–Computer Interaction and Natural Language Processing, pages 11–20, Seattle, Washington. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Angelina+McMillan%2DMajor%2C+Amandalynne+Paullada%2C+and+Yacine+Jernite%2E+2022%2E+An+interactive+exploratory+tool+for+the+task+of+hate+speech+detection%2E+In+Proceedings+of+the+Second+Workshop+on+Bridging+Human%E2%80%93)
11. Alexis Palmer, Christine Carr, Melissa Robinson, and Jordan Sanders. 2020. COLD: Annotation scheme and evaluation data set for complex offensive language in English. Journal for Language Technology and Computational Linguistics, 34(1):1–28. 2019b. [SemEval-2019 task 6: Identifying and categorizing offensive language in social media (OffensEval)](https://md.research.elamur.ai/free/en/papers/zampieri-2019-semeval-2019-task-6-identifying/). In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 75– 86, Minneapolis, Minnesota, USA. Association for Computational Linguistics. [↗](https://doi.org/10.21248/jlcl.34.2020.222)
12. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. [↗](https://doi.org/10.48550/arxiv.1201.0490)
13. A Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. 2021. Resources and benchmark corpora for hate speech detection: a systematic review. Language Resources and Evaluation, 55(2):477–523. In this appendix, we provide code snippets corresponding to Section 4. A.1 [↗](https://doi.org/10.1007/s10579-020-09502-8)
14. Paul Röttger, Bertram Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet B Pierrehumbert. 2020. [Hatecheck: Functional tests for hate speech detection models](https://md.research.elamur.ai/free/en/papers/rottger-2021-hatecheck-functional-tests-hate-speech/). arXiv preprint arXiv:2012.15606. [↗](https://arxiv.org/abs/2012.15606)
15. Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A Smith. 2021. [Annotators with attitudes: How annotator beliefs and identities bias toxic language detection](https://md.research.elamur.ai/free/en/papers/sap-2022-annotators-attitudes-how-annotator-beliefs/). arXiv preprint arXiv:2111.07997. [↗](https://arxiv.org/abs/2111.07997)
16. Anna Schmidt and Michael Wiegand. 2017. [A survey on hate speech detection using natural language processing](https://md.research.elamur.ai/free/en/papers/schmidt-2017-survey-hate-speech-detection-natural/). In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, pages 1–10, Valencia, Spain. Association for Computational Linguistics.
17. Bertie Vidgen, Dong Nguyen, Helen Margetts, Patricia Rossini, and Rebekah Tromble. 2021. [Introducing CAD: the contextual abuse dataset](https://md.research.elamur.ai/free/en/papers/vidgen-2021-introducing-cad-contextual-abuse-dataset/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2289–2303, Online. Association for Computational Linguistics. Code Examples Preliminaries Import Statements

*Updated 1 November 2026*