--- title: "Measuring Bias in Contextualized Word — Kurita 2019" description: "Static embeddings like word2vec and GloVe are known to absorb social stereotypes." authors: ["Keita Kurita", "Nidhi Vyas", "Ayush Pareek", "Alan W Black", "Yulia Tsvetkov"] year: 2019 venue: "Proceedings of the First Workshop on Gender Bias in Natural Language Processing" source: "https://aclanthology.org/W19-3823/" doi: "10.18653/v1/w19-3823" arxiv: "1906.07337" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/kurita-2019-measuring-bias-contextualized-word-representations/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kurita-2019-measuring-bias-contextualized-word-representations/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/) / Measuring Bias in Contextualized Word Representations # Measuring Bias in Contextualized Word Representations **Authors:** [Keita Kurita](https://md.research.elamur.ai/free/en/authors/keita-kurita/), [Nidhi Vyas](https://md.research.elamur.ai/free/en/authors/nidhi-vyas/), [Ayush Pareek](https://md.research.elamur.ai/free/en/authors/ayush-pareek/), [Alan W Black](https://md.research.elamur.ai/free/en/authors/alan-w-black/), [Yulia Tsvetkov](https://md.research.elamur.ai/free/en/authors/yulia-tsvetkov/) · **Year:** 2019 · **Venue:** Proceedings of the First Workshop on Gender Bias in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/W19-3823/) · [PDF](https://aclanthology.org/W19-3823.pdf) · [DOI 10.18653/v1/w19-3823](https://doi.org/10.18653/v1/w19-3823) · [arXiv 1906.07337](https://arxiv.org/abs/1906.07337) · [OpenAlex](https://openalex.org/W2972413484) **Citations:** 337 citations (openalex-snapshot · 2026-10-14) · [247](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Static embeddings like word2vec and GloVe are known to absorb social stereotypes. However, the usual ways to detect this, such as analogies and the cosine-based WEAT, do not transfer well to contextual models like BERT, where a word's vector changes with its sentence. The authors instead probe BERT's masked language model directly. A short template such as "[TARGET] is a [ATTRIBUTE]" is filled with a target like he or she. The target's probability when the attribute is visible is normalized by its probability when the attribute is also masked, giving an increased log probability score. The difference between two targets' scores is the log probability bias score. On the WEAT stimuli, using uncased BERT BASE, WEAT applied to final-layer embeddings found no significant bias, while the new score found significant bias in all five tested categories. A case study on Gendered Pronoun Resolution and an analysis of jobs, traits and skills suggest these biases affect downstream behavior. This raises concerns for uses such as automated resume filtering. ## Key points - **WEAT comparison (Table 3):** - The proposed score gave effect sizes of 0.85–1.13, all significant at p \< 0.01. - WEAT on BERT gave 0.50–1.00, with none significant. - WEAT on GloVe was significant in 3 of 5 categories. The EA/AA names test failed partly because only 5 African American names were in BERT's vocabulary. - **GPR case study (Webster et al., 2018):** - The training data is gender-balanced, yet the model predicted that a female pronoun refers to no entity more often than a male one (average probability 13.9% vs 11.5%, against priors of 9.8% vs 10.3%; p = 0.007). - The authors also scored sentence topics extracted with NMF (20 topics) and related them to this behavior. - **Employment attributes:** BERT associated most attributes more strongly with he than with she: - 88.5% of the top-1000 job titles from the Montgomery County salary data - 80.0% of positive traits - 78.9% of negative traits - 84.0% of O*NET 23.2 technology skills (template "TARGET can do ATTRIBUTE") - **Code, data and extensions:** Code and data are publicly released. The method is claimed to extend to multiclass biases such as race and religion. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 247 papers of our library cite it (of the 140884 whose bibliographies we read); 30 of them have a page here, 217 not yet. - [Language (Technology) is Power: A Critical Survey of “Bias” in NLP](https://md.research.elamur.ai/free/en/papers/blodgett-2020-language-technology-is-power-critical/) - [Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings](https://md.research.elamur.ai/free/en/papers/bommasani-2020-interpreting-pretrained-contextualized-representations-reductions/) - [On the Opportunities and Risks of Foundation Models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/) - [RobBERT: a Dutch RoBERTa-based Language Model](https://md.research.elamur.ai/free/en/papers/delobelle-2020-robbert-dutch-roberta-based-language/) - [Toxicity in chatgpt: Analyzing persona-assigned language models](https://md.research.elamur.ai/free/en/papers/deshpande-2023-toxicity-chatgpt-analyzing-persona-assigned/) - [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://md.research.elamur.ai/free/en/papers/feng-2023-pretraining-data-language-models-downstream/) - [Bias and Fairness in Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/gallegos-2023-bias-fairness-large-language-models/) - [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://md.research.elamur.ai/free/en/papers/ganguli-2022-red-teaming-language-models-reduce/) - [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/) - [Auto-Debias: Debiasing Masked Language Models with Automated Biased Prompts](https://md.research.elamur.ai/free/en/papers/guo-2022-auto-debias-debiasing-masked-language/) - [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://md.research.elamur.ai/free/en/papers/hutchinson-2020-social-biases-nlp-models-as/) - [Debiasing Pre-trained Contextualised Embeddings](https://md.research.elamur.ai/free/en/papers/kaneko-2021-debiasing-pre-trained-contextualised-embeddings/) - [Sustainable Modular Debiasing of Language Models](https://md.research.elamur.ai/free/en/papers/lauscher-2021-sustainable-modular-debiasing-language-models/) - [Towards Debiasing Sentence Representations](https://md.research.elamur.ai/free/en/papers/liang-2020-debiasing-sentence-representations/) - [Towards Understanding and Mitigating Social Biases in Language Models](https://md.research.elamur.ai/free/en/papers/liang-2021-towards-understanding-mitigating-social-biases/) - [An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models](https://md.research.elamur.ai/free/en/papers/meade-2022-empirical-survey-effectiveness-debiasing-techniques/) - [StereoSet: Measuring stereotypical bias in pretrained language models](https://md.research.elamur.ai/free/en/papers/nadeem-2021-stereoset-measuring-stereotypical-bias-pretrained/) - [HONEST: Measuring Hurtful Sentence Completion in Language Models](https://md.research.elamur.ai/free/en/papers/nozza-2021-honest-measuring-hurtful-sentence-completion/) - [Investigating How LLMs Propagate Female Stereotypes: Comparing What Models Say via Prompts with What They Represent in Their Embeddings](https://md.research.elamur.ai/free/en/papers/nunez-2026-investigating-how-llms-propagate-female/) - [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://md.research.elamur.ai/free/en/papers/penedo-2024-fineweb-datasets-decanting-web-finest/) - [Simple Entity-Centric Questions Challenge Dense Retrievers](https://md.research.elamur.ai/free/en/papers/sciavolino-2021-simple-entity-centric-questions-challenge/) - [Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview](https://md.research.elamur.ai/free/en/papers/shah-2019-predictive-biases-natural-language-processing/) - [Towards Controllable Biases in Language Generation](https://md.research.elamur.ai/free/en/papers/sheng-2020-controllable-biases-language-generation/) - [“I'm sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://md.research.elamur.ai/free/en/papers/smith-2022-i-m-sorry-hear-that/) - [Release Strategies and the Social Impacts of Language Models](https://md.research.elamur.ai/free/en/papers/solaiman-2019-release-strategies-social-impacts-language/) - [Galactica: A Large Language Model for Science](https://md.research.elamur.ai/free/en/papers/taylor-2022-galactica-large-language-model-science/) - [LaMDA: Language Models for Dialog Applications](https://md.research.elamur.ai/free/en/papers/thoppilan-2022-lamda-language-models-dialog-applications/) - [Measuring and Reducing Gendered Correlations in Pre-trained Models](https://md.research.elamur.ai/free/en/papers/webster-2020-measuring-reducing-gendered-correlations-pre/) - [Ethical and social risks of harm from Language Models](https://md.research.elamur.ai/free/en/papers/weidinger-2021-ethical-social-risks-harm-language/) - [Are Social Biases in LLMs Consistent across Generative Tasks? A Case Study for Basque](https://md.research.elamur.ai/free/en/papers/zulaika-2026-are-social-biases-llms-consistent/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{kurita2019measuring, title = {{Measuring Bias in Contextualized Word Representations}}, author = {Kurita, Keita and Vyas, Nidhi and Pareek, Ayush and Black, Alan W and Tsvetkov, Yulia}, booktitle = {Proceedings of the First Workshop on Gender Bias in Natural Language Processing}, year = {2019}, doi = {10.18653/v1/w19-3823}, eprint = {1906.07337}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/w19-3823} } ``` - [← previous MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms](https://md.research.elamur.ai/free/en/papers/amini-2019-mathqa-interpretable-math-word-problem/) - [next → Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/dowmunt-2019-microsoft-translator-wmt-2019-large/) - [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/) ## References 13 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Evelin Amorim, Marcia Cançado, and Adriano Veloso. 2018. Automated essay scoring in the presence of biased ratings. In Proc. of NAACL, pages 229–237. [find](https://scholar.google.com/scholar?q=Evelin+Amorim%2C+Marcia+Canc%CC%A7ado%2C+and+Adriano+Veloso%2E+2018%2E+Automated+essay+scoring+in+the+presence+of+biased+ratings%2E+In+Proc%2E+of+NAACL%2C+pages+229%E2%80%93237%2E) 2. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proc.of NIPS, pages 3111–3119. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+S+Corrado%2C+and+Jeff+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Proc%2Eof+NIPS%2C+pages+3111%E2%80%933119%2E) 3. Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. Calif. L. Rev., 104:671. [find](https://scholar.google.com/scholar?q=Solon+Barocas+and+Andrew+D+Selbst%2E+2016%2E+Big+data%E2%80%99s+disparate+impact%2E+Calif%2E+L%2E+Rev%2E%2C+104%3A671%2E) 4. Christine Basta, Marta R Costa-jussà, and Noe Casas. 2019. Evaluating the underlying gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.08783. [find](https://scholar.google.com/scholar?q=Christine+Basta%2C+Marta+R+Costa%2Djussa%CC%80%2C+and+Noe+Casas%2E+2019%2E+Evaluating+the+underlying+gender+bias+in+contextualized+word+embeddings%2E+arXiv+preprint+arXiv%3A1904%2E08783%2E) [our card](https://md.research.elamur.ai/free/en/papers/basta-2019-evaluating-underlying-gender-bias-contextualized/) 5. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proce. of EMNLP, pages 1532– 1543. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+Manning%2E+2014%2E+GloVe%3A+Global+vectors+for+word+representation%2E+In+Proce%2E+of+EMNLP%2C+pages+1532%E2%80%93+1543%2E) 6. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Proc. of NIPS, pages 4349–4357. [find](https://scholar.google.com/scholar?q=Tolga+Bolukbasi%2C+Kai%2DWei+Chang%2C+James+Y+Zou%2C+Venkatesh+Saligrama%2C+and+Adam+T+Kalai%2E+2016%2E+Man+is+to+computer+programmer+as+woman+is+to+homemaker%3F+debiasing+word+embeddings%2E+In+Proc%2E+of+NIPS%2C+pages+434) [our card](https://md.research.elamur.ai/free/en/papers/bolukbasi-2016-man-computer-programmer-woman-homemaker/) 7. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Matthew+E+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E+In+Proc%2E+of+NAACL%2E) 8. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Rachel+Rudinger%2C+Jason+Naradowsky%2C+Brian+Leonard%2C+and+Benjamin+Van+Durme%2E+2018%2E+Gender+bias+in+coreference+resolution%2E+In+Proc%2E+of+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/rudinger-2018-gender-bias-coreference-resolution/) 9. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. [find](https://scholar.google.com/scholar?q=Aylin+Caliskan%2C+Joanna+J+Bryson%2C+and+Arvind+Narayanan%2E+2017%2E+Semantics+derived+automatically+from+language+corpora+contain+human%2Dlike+biases%2E+Science%2C+356%286334%29%3A183%E2%80%93186%2E) 10. J Segal. 2014. Social media use in hiring: Assessing the risks. HR Magazine, 59(9). [find](https://scholar.google.com/scholar?q=J+Segal%2E+2014%2E+Social+media+use+in+hiring%3A+Assessing+the+risks%2E+HR+Magazine%2C+59%289%29%2E) 11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proc%2E+of+NAACL%2E) 12. Nathaniel Swinger, Maria De-Arteaga, Neil Heffernan IV, Mark Leiserson, and Adam Kalai. 2019. What are the biases in my word embedding? In Proc. of the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES). [find](https://scholar.google.com/scholar?q=Nathaniel+Swinger%2C+Maria+De%2DArteaga%2C+Neil+Heffernan+IV%2C+Mark+Leiserson%2C+and+Adam+Kalai%2E+2019%2E+What+are+the+biases+in+my+word+embedding%3F+In+Proc%2E+of+the+AAAI%2FACM+Conference+on+Artificial+Intelligence%2C+) 13. Anjalie Field and Yulia Tsvetkov. 2019. Entity-centric contextual affective analysis. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Anjalie+Field+and+Yulia+Tsvetkov%2E+2019%2E+Entity%2Dcentric+contextual+affective+analysis%2E+In+Proc%2E+of+ACL%2E)