---
title: "Understanding Undesirable Word Embedding Associations (2019)"
description: "Ethayarajh, Duvenaud and Hirst study these tools for embedding models that implicitly factorize a… 14 references, cited by 62 papers here."
authors: ["Kawin Ethayarajh", "David Duvenaud", "Graeme Hirst"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1166/"
doi: "10.18653/v1/p19-1166"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ethayarajh-2019-understanding-undesirable-word-embedding-associations/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ethayarajh-2019-understanding-undesirable-word-embedding-associations/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / Understanding Undesirable Word Embedding Associations

# Understanding Undesirable Word Embedding Associations

**Authors:** Kawin Ethayarajh, [David Duvenaud](https://md.research.elamur.ai/free/en/authors/david-duvenaud/) and [Graeme Hirst](https://md.research.elamur.ai/free/en/authors/graeme-hirst/) · **Year:** 2019 · **Venue:** [Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2019](https://md.research.elamur.ai/free/en/venues/acl/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1166/) · [PDF](https://aclanthology.org/P19-1166.pdf) · [DOI 10.18653/v1/p19-1166](https://doi.org/10.18653/v1/p19-1166) · [OpenAlex](https://openalex.org/W2952349219)

**Citations:** 109 citations (OpenAlex, 2026-10-14) · [62](https://md.research.elamur.ai/free/en/papers/ethayarajh-2019-understanding-undesirable-word-embedding-associations/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Word vectors are known to encode socially unwanted associations such as gender stereotypes, but the tools for detecting and removing them lack theoretical grounding. Ethayarajh, Duvenaud and Hirst study these tools for embedding models that implicitly factorize a word-context matrix, such as SGNS and GloVe. They prove that, when certain conditions hold, debiasing by post-hoc subspace projection (Bolukbasi et al., 2016) has the same effect as training on a corpus that carries no bias toward the defining word pairs. This holds only if vectors are not normalized beforehand and the bias subspace is the full span of the difference vectors. They also show that WEAT systematically overstates bias and can be manipulated through the choice of attribute sets. As an alternative, they derive RIPA: the inner product of a word vector with the first principal component of the pair difference vectors. RIPA has an information-theoretic reading, so embedding-space association can be compared with corpus statistics. With it, they find that SGNS generally does not add gender association beyond the corpus, but it amplifies association for stereotyped and gender-specific words. An unsupervised rule for choosing which words to debias also clearly beats the original supervised word list.

## Key points
- **Debiasing theorem:** for matrix-factorizing models (GloVe, SGNS), projecting out the span of difference vectors yields a reconstructed matrix that is unbiased with respect to those word pairs, assuming no reconstruction error.
- **WEAT flaws:** for SGNS and GloVe, WEAT implicitly assumes both attribute sets occur equally often in the corpus. Carefully chosen attribute sets can make even a neutral word like ‘door’ look gender-biased.
- **RIPA results on SGNS:** most words are not more gendered than in the training corpus. Stereotyped words are amplified, e.g. ‘nurse’ moves from −0.190 in the corpus to −1.047 in the embedding space. The amplification is stronger for gender-specific words, and changes are significant (p \< 0.001); the authors attribute this to second-order similarity.
- **Unsupervised word selection:** a word is debiased only if its RIPA with a bias-defining vector exceeds its RIPA with a gender-defining vector. For analogies of strength ≥ 0.5, this keeps 94.9% of gender-appropriate analogies and 36.7% of biased ones.
- **Comparison with Bolukbasi et al.:** their bootstrapped list (github.com/tolga-b/debiaswe) keeps only 16.5% of appropriate analogies and 80.0% of biased ones. The list contains errors such as ‘wolf_cub’ and misses words like ‘godfather’.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Toronto](https://md.research.elamur.ai/free/en/universities/university-of-toronto/) — University, Toronto, Canada · 47 works · 50,254 citations
- [Vector Institute](https://md.research.elamur.ai/free/en/labs/vector-institute/) — Research institute, Toronto, Canada · 19 works · 4,281 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 744 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 221 works

## Cite

Cite as: Kawin Ethayarajh, David Duvenaud and Graeme Hirst. 2019. Understanding Undesirable Word Embedding Associations. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ethayarajh2019understanding,
  title = {{Understanding Undesirable Word Embedding Associations}},
  author = {Kawin Ethayarajh, David Duvenaud and Graeme Hirst},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1166},
  url = {https://doi.org/10.18653/v1/p19-1166}
}
```

- [← previous Soft Contextual Data Augmentation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/gao-2019-soft-contextual-data-augmentation-neural/)
- [next → Curriculum Learning for Domain Adaptation in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/zhang-2019-curriculum-learning-domain-adaptation-neural/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

14 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. 2016. [A latent variable model approach to PMI-based word embeddings](https://md.research.elamur.ai/free/en/papers/arora-2016-latent-variable-model-approach-pmi/). Transactions of the Association for Computational Linguistics, 4:385–399.
2. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. [A neural probabilistic language model](https://md.research.elamur.ai/free/en/papers/bengio-2003-neural-probabilistic-language-model/). Journal of Machine Learning Research, 3(Feb):1137–1155.
3. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. [Man is to computer programmer as woman is to homemaker? debiasing word embeddings](https://md.research.elamur.ai/free/en/papers/bolukbasi-2016-man-computer-programmer-woman-homemaker/). In Advances in Neural Information Processing Systems, pages 4349–4357.
4. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. [find](https://scholar.google.com/scholar?q=Aylin+Caliskan%2C+Joanna+J+Bryson%2C+and+Arvind+Narayanan%2E+2017%2E+Semantics+derived+automatically+from+language+corpora+contain+human%2Dlike+biases%2E+Science%2C+356%286334%29%3A183%E2%80%93186%2E)
5. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, pages 160–167. ACM. [find](https://scholar.google.com/scholar?q=Ronan+Collobert+and+Jason+Weston%2E+2008%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+In+Proceedings+of+the+25th+International+Conference+on+Mac)
6. Kawin Ethayarajh, David Duvenaud, and Graeme Hirst. 2018. Towards understanding linear word analogies. arXiv preprint arXiv:1810.04882. [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2C+David+Duvenaud%2C+and+Graeme+Hirst%2E+2018%2E+Towards+understanding+linear+word+analogies%2E+arXiv+preprint+arXiv%3A1810%2E04882%2E)
7. Hila Gonen and Yoav Goldberg. 2019. [Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them](https://md.research.elamur.ai/free/en/papers/gonen-2019-lipstick-pig-debiasing-methods-cover/). arXiv preprint arXiv:1903.03862.
8. Thomas K Landauer and Susan T Dumais. 1997. A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological review, 104(2):211. [find](https://scholar.google.com/scholar?q=Thomas+K+Landauer+and+Susan+T+Dumais%2E+1997%2E+A+solution+to+Plato%E2%80%99s+problem%3A+The+latent+semantic+analysis+theory+of+acquisition%2C+induction%2C+and+representation+of+knowledge%2E+Psychological+review%2C+104%282%29%3A)
9. Omer Levy and Yoav Goldberg. 2014. Neural word embedding as implicit matrix factorization. In Advances in Neural Information Processing Systems, pages 2177–2185. [find](https://scholar.google.com/scholar?q=Omer+Levy+and+Yoav+Goldberg%2E+2014%2E+Neural+word+embedding+as+implicit+matrix+factorization%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+2177%E2%80%932185%2E)
10. Omer Levy, Yoav Goldberg, and Ido Dagan. 2015. [Improving distributional similarity with lessons learned from word embeddings](https://md.research.elamur.ai/free/en/papers/levy-2015-improving-distributional-similarity-lessons-learned/). Transactions of the Association for Computational Linguistics, 3:211–225.
11. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In Advances in Neural Information Processing Systems, pages 3111–3119.
12. David Mimno and Laure Thompson. 2017. The strange geometry of skip-gram with negative sampling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2873–2878. [find](https://scholar.google.com/scholar?q=David+Mimno+and+Laure+Thompson%2E+2017%2E+The+strange+geometry+of+skip%2Dgram+with+negative+sampling%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+2873%E2%80%9328)
13. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. [GloVe: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543.
14. Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018. [Learning gender-neutral word embeddings](https://md.research.elamur.ai/free/en/papers/zhao-2018-learning-gender-neutral-word-embeddings/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847–4853.

*Updated 24 October 2026*