--- title: "Mitigating Racial Biases in Toxic Language Detection with an" description: "The authors then propose HxEnsemble, which pairs a general toxicity classifier… 55.9% agreement. 13 references, cited by 18 papers here." authors: ["Matan Halevy", "Camille Harris", "Amy Bruckman", "Diyi Yang", "Ayanna Howard"] year: 2021 venue: "arXiv:2109.13137" source: "https://arxiv.org/abs/2109.13137" doi: "10.1145/3465416.3483299" arxiv: "2109.13137" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/halevy-2021-mitigating-racial-biases-toxic-language/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/halevy-2021-mitigating-racial-biases-toxic-language/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/) / Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework # Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework **Authors:** Matan Halevy, [Camille Harris](https://md.research.elamur.ai/free/en/authors/camille-harris/), Amy Bruckman, [Diyi Yang](https://md.research.elamur.ai/free/en/authors/diyi-yang/) and Ayanna Howard · **Year:** 2021 · **Venue:** arXiv:2109.13137 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://doi.org/10.1145/3465416.3483299) · [PDF](https://arxiv.org/pdf/2109.13137) · [DOI 10.1145/3465416.3483299](https://doi.org/10.1145/3465416.3483299) · [arXiv 2109.13137](https://arxiv.org/abs/2109.13137) · [OpenAlex](https://openalex.org/W3202038256) **Citations:** 18 citations (OpenAlex, 2026-10-14) · [18](https://md.research.elamur.ai/free/en/papers/halevy-2021-mitigating-racial-biases-toxic-language/cited-by/) in our library, of the 141,840 papers whose bibliographies we read ## Summary Toxic-language classifiers trained on Twitter data tend to flag African American English (AAE) posts as offensive more often than Standard American English (SAE) posts. This happens because annotators labeled AAE text as toxic far more readily. Earlier debiasing work judged success mainly by false-positive rate (FPR). The authors show this is a narrow lens, since very few AAE samples in these datasets are labeled non-toxic. They estimate dialect with Blodgett et al.'s model and study four Twitter corpora: DWMW17, FDCL18, Golbeck and WH16. They use these both individually and in two merged sets, Hate and Toxic. On this data they benchmark several models: - logistic regression with n-grams, TF-IDF and GloVe features - a vanilla BERT classifier - two in-process debiasing methods All of these keep reproducing the racial bias, even when trained on the larger merged corpus. The authors then propose HxEnsemble, which pairs a general toxicity classifier with a second classifier fine-tuned on the AAE dialect. HxEnsemble improves fairness metrics on every dataset while barely hurting classification quality. This suggests a practical route for low-resource groups that annotation bias treats unfairly. ## Key points - AAE tweets were identified with Blodgett et al.'s dialect model at Pr(AAE) ≥ 0.6, looser than the original 0.8, to get more AAE samples. An appendix check at 0.8 showed similar patterns. - Label skew is stark: - DWMW17: about 0.98 of AAE tweets are labeled Toxic, versus about 0.80 of SAE tweets. - FDCL18: about 0.84 for AAE versus 0.26 for SAE. - Source datasets: - DWMW17: 24,802 tweets, 92% annotator agreement. - FDCL18: 79,996 tweets, 55.9% agreement. - WH16: 16,849 tweets. Only its positive instances were used, because of strong author and topic bias. - Golbeck: used only inside the Toxic aggregate, since it has too few AAE samples on its own. - Merging corpora into larger Hate and Toxic sets did not remove the bias. It also exposed inconsistencies between the datasets' labeling schemes. - The authors call for new datasets where AAE samples are labeled by in-group annotators, or labeled with racial priming or the CASM method. ## Interesting ideas - **Dialect routing.** A dialect detector sends inputs to a general model plus a specialist tuned on the under-represented variety, instead of forcing one model to cover every variety. This lets a minority variety be handled well without retraining everything. - **Multiple fairness metrics.** FPR alone says little when a subgroup has almost no true negatives, so several descriptive metrics should be reported per group. This avoids declaring success based on a near-empty slice. - **Threshold sensitivity check.** Loosen a detector threshold to collect more minority-class data, then re-run the analysis at the strict threshold to confirm the conclusions hold. This trades sample size for noise while keeping the result honest. - **Bigger corpora don't fix bias.** Merging corpora built with different label definitions adds data, but it also adds label inconsistency and leaves the bias in place. - **Reclaimed terms depend on context.** Error analysis notes that in-group communities, such as AAE and LGBTQIA+ speakers, reclaim slurs in neutral or positive senses. Toxicity therefore depends on speaker and context, not on the word itself. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (tree): a register/dialect detector could route AAE or informal social-media sentences to a parser adapted on that variety, while standard text stays on the default gparse or perceptron model. - **I2** (learn): evaluation reports could break results down by text variety and list several error rates per slice, flagging any slice whose negative examples are too few to trust. - **I3** (learn): when silver data is admitted by a witness-agreement threshold, conclusions could be re-checked at a stricter threshold before being trusted. - **I4** (tree): when mixing treebanks or converting between UD dialects, cross-corpus label agreement should be measured first, so conflicting conventions don't simply get averaged in. - **I5** (frames): speech-act and implicature logic in prag could weigh the speaker's group and stance before marking an utterance as insulting, rather than reacting to the lexical item alone. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Georgia Institute of Technology](https://md.research.elamur.ai/free/en/universities/georgia-institute-of-technology/) — University, Atlanta, United States · 69 works · 22,788 citations - [The Ohio State University](https://md.research.elamur.ai/free/en/universities/the-ohio-state-university/) — University, Columbus, United States · 57 works · 17,793 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 744 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 221 works 🧭 Topics (1): [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 209 works ## Cite Cite as: Matan Halevy, Camille Harris, Amy Bruckman, Diyi Yang and Ayanna Howard. 2021. Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework. arXiv:2109.13137. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{halevy2021mitigating, title = {{Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework}}, author = {Matan Halevy, Camille Harris, Amy Bruckman, Diyi Yang and Ayanna Howard}, year = {2021}, doi = {10.1145/3465416.3483299}, eprint = {2109.13137}, archivePrefix = {arXiv}, url = {https://doi.org/10.1145/3465416.3483299} } ``` - [← previous Integrating Deep Event-Level and Script-Level Information for Script Event Prediction](https://md.research.elamur.ai/free/en/papers/bai-2021-integrating-deep-event-level-script/) - [next → Sarcasm and Sentiment Detection In Arabic Tweets Using BERT-based Models and Data Augmentation](https://md.research.elamur.ai/free/en/papers/abuzayed-2021-sarcasm-sentiment-detection-arabic-tweets/) - [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/) ## References 13 references, 6 of them with a card of ours. ↗ leads to the work. 1. Anti-Defamation League. 2021. Online Hate and Harassment: The American Experience 2021. Technical Report. [https://www.adl.org/online-hate-2021](https://www.adl.org/online-hate-2021) [↗](https://www.adl.org/online-hate-2021) 2. M. Awal, Rui Cao, Roy Ka-Wei Lee, and Sandra Mitrovic. 2020. On Analyzing Annotation Consistency in Online Abusive Behavior Datasets. CoRR abs/2006.13507 (2020). arXiv:2006.13507 https://arxiv.org/abs/2006.13507 [↗](https://arxiv.org/abs/2006.13507) 3. Dan Biddle. 2006. Adverse impact and test validation : a practitioner’s guide to valid and defensible employment testing. (2006). https://doi.org/10.4324/ 9781315263298 "A Gower Book"–Cover. [↗](https://doi.org/10.4324/) 4. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://md.research.elamur.ai/free/en/papers/blodgett-2016-demographic-dialectal-variation-social-media/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, 1119–1130. https://doi.org/10.18653/v1/D16-1120 [↗](https://doi.org/10.18653/v1/d16-1120) 5. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://md.research.elamur.ai/free/en/papers/bolukbasi-2016-man-computer-programmer-woman-homemaker/). In Proceedings of the 30th International Conference on Neural Information Processing Systems (Barcelona, Spain) (NIPS’16). Curran Associates Inc., Red Hook, NY, USA, 4356–4364. 6. Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 77–91. [http://proceedings.mlr.press/v81/buolamwini18a.html](http://proceedings.mlr.press/v81/buolamwini18a.html) [↗](http://proceedings.mlr.press/v81/buolamwini18a.html) 7. Jeremy Calder. 2020. Language and Sexuality: Language and LGBTQ+ Communities. American Cancer Society, 1–7. https://doi.org/10.1002/9781118786093.iela0206 [↗](https://doi.org/10.1002/9781118786093.iela0206) 8. Stevie Chancellor, Jessica Annette Pater, Trustin Clear, Eric Gilbert, and Munmun De Choudhury. 2016. #thyghgapp: Instagram Content Moderation and Lexical Variation in Pro-Eating Disorder Communities. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (San Francisco, California, USA) (CSCW ’16). Association for Computing Machinery, New York, NY, USA, 1201–1213. https://doi.org/10.1145/2818048.2819963 [↗](https://doi.org/10.1145/2818048.2819963) 9. Chris Collins, Simanique Moody, and Paul M. Postal. 2008. An AAE Camouflage Construction. Language 84, 1 (2008), 29–68. http://www.jstor.org/stable/40071011 [↗](https://doi.org/10.1353/lan.2008.0059) 10. Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019. [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://md.research.elamur.ai/free/en/papers/davidson-2019-racial-bias-hate-speech-abusive/). In Proceedings of the Third Workshop on Abusive Language Online. Association for Computational Linguistics, Florence, Italy, 25–35. https://doi.org/10.18653/v1/W19-3504 [↗](https://doi.org/10.18653/v1/w19-3504) 11. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. [Automated Hate Speech Detection and the Problem of Offensive Language](https://md.research.elamur.ai/free/en/papers/davidson-2017-automated-hate-speech-detection-problem/). [https://aaai.org/ocs/index.php/ICWSM/ICWSM17/paper/view/15665](https://aaai.org/ocs/index.php/ICWSM/ICWSM17/paper/view/15665) [↗](https://aaai.org/ocs/index.php/ICWSM/ICWSM17/paper/view/15665) 12. Ona de Gibert, Naiara Perez, Aitor García-Pablos, and Montse Cuadros. 2018. [Hate Speech Dataset from a White Supremacy Forum](https://md.research.elamur.ai/free/en/papers/gibert-2018-hate-speech-dataset-white-supremacy/). In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2). Association for Computational Linguistics, Brussels, Belgium, 11–20. https://doi.org/10.18653/v1/W18-5102 [↗](https://doi.org/10.18653/v1/w18-5102) 13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association *Updated 24 October 2026*