---
title: "Black is to Criminal as Caucasian is to — Manzini 2019"
description: "Word embeddings trained on web text absorb social stereotypes and pass them on to downstream models."
authors: ["Thomas Manzini", "Lim Yao Chong", "Alan W Black", "Yulia Tsvetkov"]
year: 2019
venue: "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)"
source: "https://aclanthology.org/N19-1062/"
doi: "10.18653/v1/n19-1062"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/manzini-2019-black-is-criminal-as-caucasian/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/manzini-2019-black-is-criminal-as-caucasian/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/) / Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings

# Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings

**Authors:** Thomas Manzini, Lim Yao Chong, [Alan W Black](https://md.research.elamur.ai/free/en/authors/alan-w-black/), [Yulia Tsvetkov](https://md.research.elamur.ai/free/en/authors/yulia-tsvetkov/) · **Year:** 2019 · **Venue:** Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N19-1062/) · [PDF](https://aclanthology.org/N19-1062.pdf) · [DOI 10.18653/v1/n19-1062](https://doi.org/10.18653/v1/n19-1062) · [OpenAlex](https://openalex.org/W2929071855)

**Citations:** 255 citations (openalex-snapshot · 2026-10-14) · [77](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary

Word embeddings trained on web text absorb social stereotypes and pass them on to downstream models. Earlier debiasing work (Bolukbasi et al., 2016) only handled binary attributes such as male/female. This paper generalizes that approach to attributes with more than two categories, namely race and religion. The authors build the bias subspace with PCA: each defining set holds one term per class, and the vectors used are the differences between each word and its set's mean. They then apply the existing hard ("Neutralize and Equalize") and soft debiasing procedures. They train word2vec on US-user posts from the L2-Reddit corpus and draw bias lexicons from social-science and NLP studies. They also propose MAC, a WEAT-inspired metric based on mean cosine distance, paired with a t-test. After debiasing, MAC scores move toward 1.0, meaning target terms become less associated with stereotyped attributes, and the changes are statistically significant. Performance on CoNLL 2003 tasks changes only slightly. The work matters because it gives a simple, reusable recipe for reducing non-binary demographic bias.

## Key points

- **Data and setup:** word2vec embeddings were trained on about 56 million sentences from L2-Reddit, restricted to US authors. Lexicons cover three attributes:
  - gender
  - race (Caucasian, African American, Asian American)
  - religion (Christian, Jewish, Muslim)
- **MAC results** (biased → hard-debiased / soft-debiased with λ = 0.2):

  | Attribute | Biased | Hard | Soft |
  |---|---|---|---|
  | Gender | 0.623 | 1.000 | 0.747 |
  | Race | 0.892 | 1.009 | 0.985 |
  | Religion | 0.859 | 1.004 | 0.894 |

- **Downstream evaluation:** the authors tested NER, POS tagging, and POS chunking on CoNLL 2003, both by swapping in the debiased embedding matrix and by retraining the model. NER and chunking F1 rose slightly (about +0.004 to +0.015). POS tagging usually dropped, by up to about 0.035 F1.
- **Remaining "cluster bias":** following Gonen and Goldberg (2019), the authors found that words with similar bias still cluster together after component removal. Using a larger bias subspace weakens this effect.
- **Limitations:**
  - Per-class extremes cannot be measured in the shared linear subspace.
  - The lexicons are US-specific and partly subjective.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

77 papers of our library cite it (of the 140884 whose bibliographies we read); 11 of them have a page here, 66 not yet.

- [Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training](https://md.research.elamur.ai/free/en/papers/agarwal-2021-knowledge-graph-based-synthetic-corpus/)
- [Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings](https://md.research.elamur.ai/free/en/papers/bommasani-2020-interpreting-pretrained-contextualized-representations-reductions/)
- [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://md.research.elamur.ai/free/en/papers/elsherief-2021-latent-hatred-benchmark-understanding-implicit/)
- [Bias and Fairness in Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/gallegos-2023-bias-fairness-large-language-models/)
- [Towards Debiasing Sentence Representations](https://md.research.elamur.ai/free/en/papers/liang-2020-debiasing-sentence-representations/)
- [Towards Understanding and Mitigating Social Biases in Language Models](https://md.research.elamur.ai/free/en/papers/liang-2021-towards-understanding-mitigating-social-biases/)
- [An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models](https://md.research.elamur.ai/free/en/papers/meade-2022-empirical-survey-effectiveness-debiasing-techniques/)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://md.research.elamur.ai/free/en/papers/nadeem-2021-stereoset-measuring-stereotypical-bias-pretrained/)
- [Gender Bias in Machine Translation](https://md.research.elamur.ai/free/en/papers/savoldi-2021-gender-bias-machine-translation/)
- [Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview](https://md.research.elamur.ai/free/en/papers/shah-2019-predictive-biases-natural-language-processing/)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{manzini2019black,
  title = {{Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings}},
  author = {Manzini, Thomas and Chong, Lim Yao and Black, Alan W and Tsvetkov, Yulia},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1062},
  url = {https://doi.org/10.18653/v1/n19-1062}
}
```

- [← previous BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://md.research.elamur.ai/free/en/papers/lee-2019-biobert-pre-trained-biomedical-language/)
- [next → BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/)

## References

26 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Evelyn Alsultany. 2012. Arabs and Muslims in the Media: Race and Representation after 9/11. nyu Press. [find](https://scholar.google.com/scholar?q=Evelyn+Alsultany%2E+2012%2E+Arabs+and+Muslims+in+the+Media%3A+Race+and+Representation+after+9%2F11%2E+nyu+Press%2E)
2. Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. Calif. L. Rev., 104:671. [find](https://scholar.google.com/scholar?q=Solon+Barocas+and+Andrew+D+Selbst%2E+2016%2E+Big+data%E2%80%99s+disparate+impact%2E+Calif%2E+L%2E+Rev%2E%2C+104%3A671%2E)
3. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Proc. of NIPS, pages 4349–4357. [find](https://scholar.google.com/scholar?q=Tolga+Bolukbasi%2C+Kai%2DWei+Chang%2C+James+Y+Zou%2C+Venkatesh+Saligrama%2C+and+Adam+T+Kalai%2E+2016%2E+Man+is+to+computer+programmer+as+woman+is+to+homemaker%3F+Debiasing+word+embeddings%2E+In+Proc%2E+of+NIPS%2C+pages+434) [our card](https://md.research.elamur.ai/free/en/papers/bolukbasi-2016-man-computer-programmer-woman-homemaker/)
4. Narissra M Punyanunt-Carter. 2008. The perceived realism of african american portrayals on television. The Howard Journal of Communications, 19(3):241–257. [find](https://scholar.google.com/scholar?q=Narissra+M+Punyanunt%2DCarter%2E+2008%2E+The+perceived+realism+of+african+american+portrayals+on+television%2E+The+Howard+Journal+of+Communications%2C+19%283%29%3A241%E2%80%93257%2E)
5. Sonja M Brown Givens and Jennifer L Monahan. 2005. Priming mammies, jezebels, and other controlling images: An examination of the influence of mediated stereotypes on perceptions of an african american woman. Media Psychology, 7(1):87–106. [find](https://scholar.google.com/scholar?q=Sonja+M+Brown+Givens+and+Jennifer+L+Monahan%2E+2005%2E+Priming+mammies%2C+jezebels%2C+and+other+controlling+images%3A+An+examination+of+the+influence+of+mediated+stereotypes+on+perceptions+of+an+african+america)
6. Ella Rabinovich, Yulia Tsvetkov, and Shuly Wintner. 2018. Native language cognate effects on second language lexical choice. In Proc. of the Transactions of Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ella+Rabinovich%2C+Yulia+Tsvetkov%2C+and+Shuly+Wintner%2E+2018%2E+Native+language+cognate+effects+on+second+language+lexical+choice%2E+In+Proc%2E+of+the+Transactions+of+Association+for+Computational+Linguistics%2E)
7. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. [find](https://scholar.google.com/scholar?q=Aylin+Caliskan%2C+Joanna+J+Bryson%2C+and+Arvind+Narayanan%2E+2017%2E+Semantics+derived+automatically+from+language+corpora+contain+human%2Dlike+biases%2E+Science%2C+356%286334%29%3A183%E2%80%93186%2E)
8. Kimberly Rios, Zhen Hadassah Cheng, Rebecca R Tot-ton, and Azim F Shariff. 2015. Negative stereotypes cause christians to underperform in and disidentify with science. Social Psychological and Personality Science, 6(8):959–967. [find](https://scholar.google.com/scholar?q=Kimberly+Rios%2C+Zhen+Hadassah+Cheng%2C+Rebecca+R+Tot%2Dton%2C+and+Azim+F+Shariff%2E+2015%2E+Negative+stereotypes+cause+christians+to+underperform+in+and+disidentify+with+science%2E+Social+Psychological+and+Persona)
9. Beth G Chung-Herrera and Melenie J Lankau. 2005. Are we there yet? an assessment of fit between stereotypes of minority managers and the successful-manager prototype. Journal of Applied Social Psychology, 35(10):2029–2056. [find](https://scholar.google.com/scholar?q=Beth+G+Chung%2DHerrera+and+Melenie+J+Lankau%2E+2005%2E+Are+we+there+yet%3F+an+assessment+of+fit+between+stereotypes+of+minority+managers+and+the+successful%2Dmanager+prototype%2E+Journal+of+Applied+Social+Psychol)
10. Jack G Shaheen. 1997. Arab and Muslim stereotyping in American popular culture. Center for Muslim-Christian Understanding, History and International Affairs . [find](https://scholar.google.com/scholar?q=Jack+G+Shaheen%2E+1997%2E+Arab+and+Muslim+stereotyping+in+American+popular+culture%2E+Center+for+Muslim%2DChristian+Understanding%2C+History+and+International+Affairs+%2E)
11. Alan Dundes. 1971. A study of ethnic slurs: The jew and the polack in the united states. The Journal of American Folklore, 84(332):186–203. [find](https://scholar.google.com/scholar?q=Alan+Dundes%2E+1971%2E+A+study+of+ethnic+slurs%3A+The+jew+and+the+polack+in+the+united+states%2E+The+Journal+of+American+Folklore%2C+84%28332%29%3A186%E2%80%93203%2E)
12. Joel S Fetzer. 2000. Public attitudes toward immigration in the United States, France, and Germany. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Joel+S+Fetzer%2E+2000%2E+Public+attitudes+toward+immigration+in+the+United+States%2C+France%2C+and+Germany%2E+Cambridge+University+Press%2E)
13. Andrew Shryock. 2010. Islamophobia/Islamophilia: Beyond the politics of enemy and friend. Indiana University Press. [find](https://scholar.google.com/scholar?q=Andrew+Shryock%2E+2010%2E+Islamophobia%2FIslamophilia%3A+Beyond+the+politics+of+enemy+and+friend%2E+Indiana+University+Press%2E)
14. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proc. of the National Academy of Sciences, 115(16). [find](https://scholar.google.com/scholar?q=Nikhil+Garg%2C+Londa+Schiebinger%2C+Dan+Jurafsky%2C+and+James+Zou%2E+2018%2E+Word+embeddings+quantify+100+years+of+gender+and+ethnic+stereotypes%2E+Proc%2E+of+the+National+Academy+of+Sciences%2C+115%2816%29%2E)
15. Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4, pages 142–147. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Erik+F+Tjong+Kim+Sang+and+Fien+De+Meulder%2E+2003%2E+Introduction+to+the+conll%2D2003+shared+task%3A+Language%2Dindependent+named+entity+recognition%2E+In+Proceedings+of+the+seventh+conference+on+Natural+language)
16. Jim Goad. 1998. The Redneck Manifesto: How Hillbillies Hicks and White Trash Becames America’s Scapegoats. Simon and Schuster. [find](https://scholar.google.com/scholar?q=Jim+Goad%2E+1998%2E+The+Redneck+Manifesto%3A+How+Hillbillies+Hicks+and+White+Trash+Becames+America%E2%80%99s+Scapegoats%2E+Simon+and+Schuster%2E)
17. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. [find](https://scholar.google.com/scholar?q=Hila+Gonen+and+Yoav+Goldberg%2E+2019%2E+Lipstick+on+a+pig%3A+Debiasing+methods+cover+up+systematic+gender+biases+in+word+embeddings+but+do+not+remove+them%2E) [our card](https://md.research.elamur.ai/free/en/papers/gonen-2019-lipstick-pig-debiasing-methods-cover/)
18. James D Unnever, Francis T Cullen, and Brandon K Applegate. 2005. Turning the other cheek: Reassessing the impact of religion on punitive ideology. Justice Quarterly, 22(3):304–339. [find](https://scholar.google.com/scholar?q=James+D+Unnever%2C+Francis+T+Cullen%2C+and+Brandon+K+Applegate%2E+2005%2E+Turning+the+other+cheek%3A+Reassessing+the+impact+of+religion+on+punitive+ideology%2E+Justice+Quarterly%2C+22%283%29%3A304%E2%80%93339%2E)
19. Ernest A Hakanen. 1995. Emotional use of music by african american adolescents. Howard Journal of Communications, 5(3):214–222. [find](https://scholar.google.com/scholar?q=Ernest+A+Hakanen%2E+1995%2E+Emotional+use+of+music+by+african+american+adolescents%2E+Howard+Journal+of+Communications%2C+5%283%29%3A214%E2%80%93222%2E)
20. Kelly Welch. 2007. Black criminal stereotypes and racial profiling. Journal of Contemporary Criminal Justice, 23(3):276–288. [find](https://scholar.google.com/scholar?q=Kelly+Welch%2E+2007%2E+Black+criminal+stereotypes+and+racial+profiling%2E+Journal+of+Contemporary+Criminal+Justice%2C+23%283%29%3A276%E2%80%93288%2E)
21. Yuko Kawai. 2005. Stereotyping asian americans: The dialectic of the model minority and the yellow peril. The Howard Journal of Communications, 16(2):109–130. [find](https://scholar.google.com/scholar?q=Yuko+Kawai%2E+2005%2E+Stereotyping+asian+americans%3A+The+dialectic+of+the+model+minority+and+the+yellow+peril%2E+The+Howard+Journal+of+Communications%2C+16%282%29%3A109%E2%80%93130%2E)
22. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proc. of EMNLP, pages 2979–2989. [find](https://scholar.google.com/scholar?q=Jieyu+Zhao%2C+Tianlu+Wang%2C+Mark+Yatskar%2C+Vicente+Ordonez%2C+and+Kai%2DWei+Chang%2E+2017%2E+Men+also+like+shopping%3A+Reducing+gender+bias+amplification+using+corpus%2Dlevel+constraints%2E+In+Proc%2E+of+EMNLP%2C+pages+297) [our card](https://md.research.elamur.ai/free/en/papers/zhao-2017-men-also-like-shopping-reducing/)
23. Frederick TL Leong and Thomas J Hayes. 1990. Occupational stereotyping of asian americans. The Career Development Quarterly, 39(2):143–154. [find](https://scholar.google.com/scholar?q=Frederick+TL+Leong+and+Thomas+J+Hayes%2E+1990%2E+Occupational+stereotyping+of+asian+americans%2E+The+Career+Development+Quarterly%2C+39%282%29%3A143%E2%80%93154%2E)
24. Phil Zuckerman. 2009. Atheism, secularity, and well-being: How the findings of social science counter negative stereotypes and assumptions. Sociology Compass, 3(6):949–971. [find](https://scholar.google.com/scholar?q=Phil+Zuckerman%2E+2009%2E+Atheism%2C+secularity%2C+and+well%2Dbeing%3A+How+the+findings+of+social+science+counter+negative+stereotypes+and+assumptions%2E+Sociology+Compass%2C+3%286%29%3A949%E2%80%93971%2E)
25. Monica H Lin, Virginia SY Kwan, Anna Cheung, and Susan T Fiske. 2005. Stereotype content model explains prejudice for an envied outgroup: Scale of anti-asian american stereotypes. Personality and Social Psychology Bulletin, 31(1):34–47. [find](https://scholar.google.com/scholar?q=Monica+H+Lin%2C+Virginia+SY+Kwan%2C+Anna+Cheung%2C+and+Susan+T+Fiske%2E+2005%2E+Stereotype+content+model+explains+prejudice+for+an+envied+outgroup%3A+Scale+of+anti%2Dasian+american+stereotypes%2E+Personality+and+Soci)
26. A Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. [find](https://scholar.google.com/scholar?q=A+Tomas+Mikolov%2C+Kai+Chen%2C+Greg+Corrado%2C+and+Jeffrey+Dean%2E+2013%2E+Efficient+estimation+of+word+representations+in+vector+space%2E+arXiv+preprint+arXiv%3A1301%2E3781%2E)