---
title: "Representing and Clustering Errors in Offensive Language"
description: "Offensive language classifiers used for moderation fail in systematic ways. 28 references."
authors: ["Jood Otey", "Laura Biester", "Steven R Wilson"]
year: 2025
venue: "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop)"
source: "https://aclanthology.org/2025.naacl-srw.36/"
doi: "10.18653/v1/2025.naacl-srw.36"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/otey-2025-representing-clustering-errors-offensive-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/otey-2025-representing-clustering-errors-offensive-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/18/) / Representing and Clustering Errors in Offensive Language Detection

# Representing and Clustering Errors in Offensive Language Detection

**Authors:** Jood Otey, Laura Biester and [Steven R Wilson](https://md.research.elamur.ai/free/en/authors/steven-r-wilson/) · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2025](https://md.research.elamur.ai/free/en/venues/naacl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.naacl-srw.36/) · [PDF](https://aclanthology.org/2025.naacl-srw.36.pdf) · [DOI 10.18653/v1/2025.naacl-srw.36](https://doi.org/10.18653/v1/2025.naacl-srw.36) · [OpenAlex](https://openalex.org/W4411119373)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Offensive language classifiers used for moderation fail in systematic ways. Error analysis tools usually group the misclassified items by clustering their vector representations, but few studies check whether the chosen representation produces groups a person can actually make sense of. The authors fine-tune DistilBERT on the English Measuring Hate Speech (MHS) corpus and MARBERT on the Levantine Arabic L-HSAB corpus. They then collect the test-set mistakes and cluster them with K-Means under four encodings:

- the classifier's [CLS] last hidden state (LHS)
- SBERT sentence embeddings
- SBERT embeddings of linguistic descriptions written by an LLM
- a concatenation of the last two

Humans judge interpretability through an intruder-spotting task borrowed from topic model evaluation. For English, plain SBERT and the concatenated vectors work best, and their clusters are organized mostly around the targeted group. LHS clusters barely beat chance and largely mirror the predicted label. For Arabic, only the LLM-feature encoding beats the baseline, plausibly because the descriptions are written in MSA. The authors caution that the default representation in error analysis tools may hide failure patterns rather than reveal them.

## Key points
- Data after filtering:
  - MHS: 12,289 English items (7,497 supportive, 4,792 hatespeech; the neutral class was dropped).
  - L-HSAB: 5,754 Arabic tweets (3,576 normal, 1,713 abusive, 465 hate).
  - Both corpora use an 85/15 fine-tune/test split.
- Classifier accuracy is 89.3% for DistilBERT and 87.9% for MARBERT. Clustering uses only their errors: 196 English and 106 Arabic.
- Setup details:
  - KMeans++ from scikit-learn, with the number of clusters chosen by the inertia elbow.
  - SBERT models: all-distilroberta-v1 for English and distiluse-base-multilingual-cased-v1 for Arabic.
  - Feature-generating LLMs: Mixtral 8x7b (4-bit quantized) for English and Command-R for Arabic.
- Intruder-detection accuracy against a 20% baseline:

  | Language | LHS | SBERT | Features | Concatenated |
  |---|---|---|---|---|
  | English | 27.8% | 67.6% | 34.3% | 67.6% |
  | Arabic | 15.8% | 12.5% | 31.6% | 17.6% |

  Cohen's Kappa ranged from 0.176 to 0.538 for English and from 0.130 to 0.314 for Arabic.
- Within LHS clusters, 96.59% (English) and 91.43% (Arabic) of points share their cluster's majority label. LHS groupings therefore mostly reproduce the classifier's own decision.

## Interesting ideas
- Cluster quality is measured by humans finding the odd item out among five examples, one drawn from a different cluster, which gives a cheap and comparable interpretability score instead of geometric indices.
- The authors check how strongly clusters align with the predicted label, which exposes representations that just recover the model's decision rather than any deeper failure cause.
- Errors are rewritten as LLM-produced linguistic descriptions before embedding, which normalizes noisy dialect input into a register the encoder handles well.
- Only the misclassified items are clustered, and a meaningful latent axis (the targeted group) emerged from them without any subgroup labels.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Clusters of parser or reader failures, or groups of self-induced repair rules, could be validated with an intruder test given to an LLM teacher tier, so we keep only error groupings that are genuinely coherent.
- **I2** (learn): Label purity of error clusters works as a negative control; if parse-error clusters split mainly by predicted relation, the representation is uninformative and should be replaced.
- **I3** (tree): Instead of raw text, we could describe each misparsed sentence by explainable structural traits (construction, attachment site, UD dialect quirk) and cluster those descriptions to find systematic gparse or perceptron failure slices.
- **I4** (answer): Clustering wrong answers on school-course tests or fable QA could surface unlabeled failure themes, such as negation, quantities, or particular entity types, to prioritize fixes.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Middlebury College](https://md.research.elamur.ai/free/en/universities/middlebury-college/) — University, Middlebury (village), United States · 3 works · 421 citations
- [Oakland University](https://md.research.elamur.ai/free/en/universities/oakland-university/) — University, Rochester, United States · 7 works · 180 citations
- [University of Michigan–Flint](https://md.research.elamur.ai/free/en/universities/university-of-michigan-flint/) — University, Flint, United States · 3 works · 1 citation

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Communication](https://md.research.elamur.ai/free/en/shelves/social-sciences/communication/) · 104 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 1,035 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works

## Cite

Cite as: Jood Otey, Laura Biester and Steven R Wilson. 2025. Representing and Clustering Errors in Offensive Language Detection. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{otey2025representing,
  title = {{Representing and Clustering Errors in Offensive Language Detection}},
  author = {Jood Otey, Laura Biester and Steven R Wilson},
  booktitle = {Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop)},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-srw.36},
  url = {https://doi.org/10.18653/v1/2025.naacl-srw.36}
}
```

- [← previous Reinforcement learning for aligning LLM agents with interactive environments](https://md.research.elamur.ai/free/en/papers/aissi-2025-apprentissage-par-renforcement-pour-l/)
- [next → Research Progress on Large Models for Edge Intelligence](https://md.research.elamur.ai/free/en/papers/wang-2025-large-models-edge/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/18/)

## References

28 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.acl-long.551)
2. Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. 1993. Signature verification using a "siamese" time delay neural network. In Advances in Neural Information Processing Systems, volume 6. Morgan-Kaufmann. [find](https://scholar.google.com/scholar?q=Jane+Bromley%2C+Isabelle+Guyon%2C+Yann+LeCun%2C+Eduard+S%C3%A4ckinger%2C+and+Roopak+Shah%2E+1993%2E+Signature+verification+using+a+%22siamese%22+time+delay+neural+network%2E+In+Advances+in+Neural+Information+Processing+Syst)
3. Jonathan Chang, Sean Gerrish, Chong Wang, Jordan Boyd-graber, and David Blei. 2009. [Reading tea leaves: How humans interpret topic models](https://md.research.elamur.ai/free/en/papers/chang-2009-reading-tea-leaves-humans-interpret-topic/). In Advances in Neural Information Processing Systems, volume 22. Curran Associates, Inc.
4. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46. [↗](https://doi.org/10.1177/001316446002000104)
5. Greg d’Eon, Jason d’Eon, James R. Wright, and Kevin Leyton-Brown. 2022. The spotlight: A general method for discovering systematic errors in deep learning models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, page 1962–1981, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Greg+d%E2%80%99Eon%2C+Jason+d%E2%80%99Eon%2C+James+R%2E+Wright%2C+and+Kevin+Leyton%2DBrown%2E+2022%2E+The+spotlight%3A+A+general+method+for+discovering+systematic+errors+in+deep+learning+models%2E+In+Proceedings+of+the+2022+ACM+Confer)
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
7. Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Ré. 2022. Domino: Discovering systematic errors with cross-modal embeddings. Preprint, arXiv:2203.14960. [↗](https://arxiv.org/abs/2203.14960)
8. Gabrielle Gauthier-melancon, Orlando Marquez Ayala, Lindsay Brin, Chris Tyler, Frederic Branchaud-charron, Joseph Marinier, Karine Grande, and Di Le. 2022. Azimuth: Systematic error analysis for text classification. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 298–310, Abu Dhabi, UAE. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Gabrielle+Gauthier%2Dmelancon%2C+Orlando+Marquez+Ayala%2C+Lindsay+Brin%2C+Chris+Tyler%2C+Frederic+Branchaud%2Dcharron%2C+Joseph+Marinier%2C+Karine+Grande%2C+and+Di+Le%2E+2022%2E+Azimuth%3A+Systematic+error+analysis+for+text+)
9. Marie Grace, Jay Seabrum, Dananjay Srinivas, and Alexis Palmer. 2023. [OLEA: Tool and infrastructure for offensive language error analysis in English](https://md.research.elamur.ai/free/en/papers/grace-2023-olea-tool-infrastructure-offensive-language/). In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 209–218, Dubrovnik, Croatia. Association for Computational Linguistics.
10. Camille Harris, Matan Halevy, Ayanna Howard, Amy Bruckman, and Diyi Yang. 2022. Exploring the role of grammar and word choice in bias toward african american english (aae) in hate speech classification. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, page 789–798, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Camille+Harris%2C+Matan+Halevy%2C+Ayanna+Howard%2C+Amy+Bruckman%2C+and+Diyi+Yang%2E+2022%2E+Exploring+the+role+of+grammar+and+word+choice+in+bias+toward+african+american+english+%28aae%29+in+hate+speech+classificatio)
11. Wenyue Hua, Lifeng Jin, Linfeng Song, Haitao Mi, Yongfeng Zhang, and Dong Yu. 2023. Discover, explain, improve: An automatic slice detection benchmark for natural language processing. Transactions of the Association for Computational Linguistics, 11:1537–1552. [find](https://scholar.google.com/scholar?q=Wenyue+Hua%2C+Lifeng+Jin%2C+Linfeng+Song%2C+Haitao+Mi%2C+Yongfeng+Zhang%2C+and+Dong+Yu%2E+2023%2E+Discover%2C+explain%2C+improve%3A+An+automatic+slice+detection+benchmark+for+natural+language+processing%2E+Transactions+of+)
12. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024. Mixtral of experts. Preprint, arxiv:2401.04088 [cs]. [↗](https://arxiv.org/abs/2401.04088)
13. Chris J. Kennedy, Geoff Bacon, Alexander Sahn, and Claudia von Vacano. 2020. Constructing interval variables via faceted rasch measurement and multitask deep learning: a hate speech application. Preprint, arxiv:2009.10277 [cs]. [↗](https://arxiv.org/abs/2009.10277)
14. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [RoBERTa: A robustly optimized BERT pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). Preprint, arxiv:1907.11692 [cs]. [↗](https://arxiv.org/abs/1907.11692)
15. Angelina McMillan-Major, Amandalynne Paullada, and Yacine Jernite. 2022. An interactive exploratory tool for the task of hate speech detection. In Proceedings of the Second Workshop on Bridging Human– Computer Interaction and Natural Language Processing, pages 11–20, Seattle, Washington. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Angelina+McMillan%2DMajor%2C+Amandalynne+Paullada%2C+and+Yacine+Jernite%2E+2022%2E+An+interactive+exploratory+tool+for+the+task+of+hate+speech+detection%2E+In+Proceedings+of+the+Second+Workshop+on+Bridging+Human%E2%80%93)
16. Nimit S. Sohoni, Jared A. Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. 2022. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. Preprint, arXiv:2011.12945. [↗](https://arxiv.org/abs/2011.12945)
17. Julia Mendelsohn, Ronan Le Bras, Yejin Choi, and Maarten Sap. 2023. From dogwhistles to bullhorns: Unveiling coded rhetoric with language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15162–15180, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Julia+Mendelsohn%2C+Ronan+Le+Bras%2C+Yejin+Choi%2C+and+Maarten+Sap%2E+2023%2E+From+dogwhistles+to+bullhorns%3A+Unveiling+coded+rhetoric+with+language+models%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Assoc)
18. Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, and Ann Yuan. 2020. The language interpretability tool: Extensible, interactive visualizations and analysis for NLP models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 107–118, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ian+Tenney%2C+James+Wexler%2C+Jasmijn+Bastings%2C+Tolga+Bolukbasi%2C+Andy+Coenen%2C+Sebastian+Gehrmann%2C+Ellen+Jiang%2C+Mahima+Pushkarna%2C+Carey+Radebaugh%2C+Emily+Reif%2C+and+Ann+Yuan%2E+2020%2E+The+language+interpretabil)
19. Hala Mulki, Hatem Haddad, Chedi Bechikh Ali, and Halima Alshabani. 2019. [L-HSAB: A Levantine Twitter dataset for hate speech and abusive language](https://md.research.elamur.ai/free/en/papers/mulki-2019-l-hsab-levantine-twitter-dataset/). In Proceedings of the Third Workshop on Abusive Language Online, pages 111–118, Florence, Italy. Association for Computational Linguistics.
20. Rakesh R Menon and Shashank Srivastava. 2024. DIS-CERN: Decoding systematic errors in natural language for text classifiers. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19565–19583, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rakesh+R+Menon+and+Shashank+Srivastava%2E+2024%2E+DIS%2DCERN%3A+Decoding+systematic+errors+in+natural+language+for+text+classifiers%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Langu)
21. Nazneen Rajani, Weixin Liang, Lingjiao Chen, Margaret Mitchell, and James Zou. 2022. SEAL: Interactive tool for systematic error analysis and labeling. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 359–370, Abu Dhabi, UAE. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nazneen+Rajani%2C+Weixin+Liang%2C+Lingjiao+Chen%2C+Margaret+Mitchell%2C+and+James+Zou%2E+2022%2E+SEAL%3A+Interactive+tool+for+systematic+error+analysis+and+labeling%2E+In+Proceedings+of+the+2022+Conference+on+Empiric)
22. Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld. 2019. Errudite: Scalable, reproducible, and testable error analysis. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 747–763, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tongshuang+Wu%2C+Marco+Tulio+Ribeiro%2C+Jeffrey+Heer%2C+and+Daniel+Weld%2E+2019%2E+Errudite%3A+Scalable%2C+reproducible%2C+and+testable+error+analysis%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for)
23. Mengzhou Xia, Anjalie Field, and Yulia Tsvetkov. 2020. Demoting racial bias in hate speech detection. In Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media, pages 7–14, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mengzhou+Xia%2C+Anjalie+Field%2C+and+Yulia+Tsvetkov%2E+2020%2E+Demoting+racial+bias+in+hate+speech+detection%2E+In+Proceedings+of+the+Eighth+International+Workshop+on+Natural+Language+Processing+for+Social+Medi)
24. Jun Yuan, Jesse Vig, and Nazneen Rajani. 2022. isea: An interactive pipeline for semantic error analysis of nlp models. IUI ’22, page 878–888, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Jun+Yuan%2C+Jesse+Vig%2C+and+Nazneen+Rajani%2E+2022%2E+isea%3A+An+interactive+pipeline+for+semantic+error+analysis+of+nlp+models%2E+IUI+%E2%80%9922%2C+page+878%E2%80%93888%2C+New+York%2C+NY%2C+USA%2E+Association+for+Computing+Machinery%2E)
25. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d19-1410)
26. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2020. [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/). Preprint, arxiv:1910.01108 [cs]. [↗](https://arxiv.org/abs/1910.01108)
27. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. [The risk of racial bias in hate speech detection](https://md.research.elamur.ai/free/en/papers/sap-2019-risk-racial-bias-hate-speech/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1668–1678, Florence, Italy. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/p19-1163)
28. Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. 2020. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. In Advances in Neural Information Processing Systems, volume 33, pages 19339– 19352. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Nimit+Sohoni%2C+Jared+Dunnmon%2C+Geoffrey+Angus%2C+Albert+Gu%2C+and+Christopher+R%C3%A9%2E+2020%2E+No+subclass+left+behind%3A+Fine%2Dgrained+robustness+in+coarse%2Dgrained+classification+problems%2E+In+Advances+in+Neural+Info)

*Updated 2 November 2026*