---
title: "UNQOVERing Stereotyping Biases via Underspecified — Li 2020"
description: "Stereotyping biases are known to exist in language embeddings, but how they carry over into downstream question answering (QA) systems had not been…"
authors: ["Tao Li", "Daniel Khashabi", "Tushar Khot", "Ashish Sabharwal", "Vivek Srikumar"]
year: 2020
venue: "Findings of the Association for Computational Linguistics: EMNLP 2020"
source: "https://aclanthology.org/2020.findings-emnlp.311/"
doi: "10.18653/v1/2020.findings-emnlp.311"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/li-2020-unqovering-stereotyping-biases-underspecified-questions/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/li-2020-unqovering-stereotyping-biases-underspecified-questions/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/) / UNQOVERing Stereotyping Biases via Underspecified Questions

# UNQOVERing Stereotyping Biases via Underspecified Questions

**Authors:** Tao Li, [Daniel Khashabi](https://md.research.elamur.ai/free/en/authors/daniel-khashabi/), [Tushar Khot](https://md.research.elamur.ai/free/en/authors/tushar-khot/), [Ashish Sabharwal](https://md.research.elamur.ai/free/en/authors/ashish-sabharwal/), [Vivek Srikumar](https://md.research.elamur.ai/free/en/authors/vivek-srikumar/) · **Institutions:** [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) · [University of Utah](https://md.research.elamur.ai/free/en/universities/university-of-utah/) · **Year:** 2020 · **Venue:** Findings of the Association for Computational Linguistics: EMNLP 2020 · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.findings-emnlp.311/) · [PDF](https://aclanthology.org/2020.findings-emnlp.311.pdf) · [DOI 10.18653/v1/2020.findings-emnlp.311](https://doi.org/10.18653/v1/2020.findings-emnlp.311) · [OpenAlex](https://openalex.org/W3092475477)

**Citations:** 57 citations (OpenAlex, 2026-10-14) · [87](https://md.research.elamur.ai/free/en/papers/li-2020-unqovering-stereotyping-biases-underspecified-questions/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Stereotyping biases are known to exist in language embeddings, but how they carry over into downstream question answering (QA) systems had not been studied. The authors propose UNQOVER, a framework that probes QA models and masked LMs with underspecified questions. Each template-generated paragraph mentions two subjects from protected groups, and the question asks about an attribute the context gives no evidence for. Any consistent preference for one subject therefore reflects an association learned by the model.

The authors find that raw model scores are misleading because of two reasoning errors. Answers shift with the order in which subjects appear (positional dependence), and they often stay the same when the attribute is negated (question independence). They design a metric that cancels out both effects. They then apply it to gender, nationality, ethnicity and religion across 15 transformer models. All models show clear stereotyping biases. Bigger models tend to be more biased, and fine-tuning changes bias in ways that depend on both the dataset and the model size. The work offers a reusable tool for evaluating, and eventually mitigating, bias in QA.

## Key points
- Five architectures were probed: RoBERTa in its base and large sizes, and BERT in its distilled, base and large versions. Each was tested fine-tuned on SQuAD, fine-tuned on NewsQA, and as a plain masked LM, for 15 models in total. Code is released at github.com/allenai/unqover.
- Fine-tuning effects:
  - Fine-tuning on SQuAD tended to raise bias, while NewsQA tended to lower it.
  - Fine-tuning reduced bias in the distilled model but could amplify it in larger models.
- Nationality results:
  - Rankings show a clear split between Western and non-Western countries.
  - Negative attributes are disproportionately tied to non-Western nations.
- Ethnicity and religion results:
  - Arab and African-American subjects rank as most negatively associated, and European as least.
  - Muslim ranks most negative among religions, though the overall intensity is small (|γ(x)| ≤ 0.03).
- The reasoning errors themselves are substantial:
  - RoBERTa models show more positional error than BERT models of similar size.
  - Neither QA models nor masked LMs handle attribute negation reliably.
  - Gender is treated as binary, and the attributes are partly drawn from StereoSet, which may reflect a Western perspective.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{li2020unqovering,
  title = {{UNQOVERing Stereotyping Biases via Underspecified Questions}},
  author = {Li, Tao and Khashabi, Daniel and Khot, Tushar and Sabharwal, Ashish and Srikumar, Vivek},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.311},
  url = {https://doi.org/10.18653/v1/2020.findings-emnlp.311}
}
```

- [← previous SIMULEVAL: An Evaluation Toolkit for Simultaneous Translation](https://md.research.elamur.ai/free/en/papers/ma-2020-simuleval-evaluation-toolkit-simultaneous-translation/)
- [next → Rethinking embedding coupling in pre-trained language models](https://md.research.elamur.ai/free/en/papers/chung-2020-rethinking-embedding-coupling-pre-trained/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/)

## References

36 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mohsen Abbasi, Sorelle A Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2019. Fairness in Representation: Quantifying Stereotyping as a Representational Harm. In Proceedings of the 2019 SIAM International Conference on Data Mining. [find](https://scholar.google.com/scholar?q=Mohsen+Abbasi%2C+Sorelle+A+Friedler%2C+Carlos+Scheidegger%2C+and+Suresh+Venkatasubramanian%2E+2019%2E+Fairness+in+Representation%3A+Quantifying+Stereotyping+as+a+Representational+Harm%2E+In+Proceedings+of+the+2019+)
2. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Su+Lin+Blodgett%2C+Solon+Barocas%2C+Hal+Daume%CC%81+III%2C+and+Hanna+Wallach%2E+2020%2E+Language+%28technology%29+is+power%3A+A+critical+survey+of+%E2%80%9Cbias%E2%80%9D+in+NLP%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Associatio) [our card](https://md.research.elamur.ai/free/en/papers/blodgett-2020-language-technology-is-power-critical/)
3. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016a. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Tolga+Bolukbasi%2C+Kai%2DWei+Chang%2C+James+Zou%2C+Venkatesh+Saligrama%2C+and+Adam+Kalai%2E+2016a%2E+Man+is+to+Computer+Programmer+as+Woman+is+to+Homemaker%3F+Debiasing+Word+Embeddings%2E+In+Advances+in+Neural+Informat) [our card](https://md.research.elamur.ai/free/en/papers/bolukbasi-2016-man-computer-programmer-woman-homemaker/)
4. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016b. Quantifying and Reducing Bias in Word Embeddings. In International Conference on Machine Learning Workshop on \#Data4Good. [find](https://scholar.google.com/scholar?q=Tolga+Bolukbasi%2C+Kai%2DWei+Chang%2C+James+Zou%2C+Venkatesh+Saligrama%2C+and+Adam+Kalai%2E+2016b%2E+Quantifying+and+Reducing+Bias+in+Word+Embeddings%2E+In+International+Conference+on+Machine+Learning+Workshop+on+%5C%23D)
5. Shikha Bordia and Samuel Bowman. 2019. Identifying and Reducing Gender Bias in Word-Level Language Models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop. [find](https://scholar.google.com/scholar?q=Shikha+Bordia+and+Samuel+Bowman%2E+2019%2E+Identifying+and+Reducing+Gender+Bias+in+Word%2DLevel+Language+Models%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Association+for+Co) [our card](https://md.research.elamur.ai/free/en/papers/bordia-2019-identifying-reducing-gender-bias-word/)
6. Kaytlin Chaloner and Alfredo Maldonado. 2019. Measuring gender bias in word embeddings across domains and discovering new gender bias word categories. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Kaytlin+Chaloner+and+Alfredo+Maldonado%2E+2019%2E+Measuring+gender+bias+in+word+embeddings+across+domains+and+discovering+new+gender+bias+word+categories%2E+In+Proceedings+of+the+First+Workshop+on+Gender+Bi)
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N) [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
8. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences. [find](https://scholar.google.com/scholar?q=Nikhil+Garg%2C+Londa+Schiebinger%2C+Dan+Jurafsky%2C+and+James+Zou%2E+2018%2E+Word+embeddings+quantify+100+years+of+gender+and+ethnic+stereotypes%2E+Proceedings+of+the+National+Academy+of+Sciences%2E)
9. Joseph Henrich, Steven J Heine, and Ara Norenzayan. 2010. Most people are not WEIRD. Nature. [find](https://scholar.google.com/scholar?q=Joseph+Henrich%2C+Steven+J+Heine%2C+and+Ara+Norenzayan%2E+2010%2E+Most+people+are+not+WEIRD%2E+Nature%2E)
10. Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Quantifying Social Biases in Contextual Word Representations. In 1st ACL Workshop on Gender Bias for Natural Language Processing. [find](https://scholar.google.com/scholar?q=Keita+Kurita%2C+Nidhi+Vyas%2C+Ayush+Pareek%2C+Alan+W+Black%2C+and+Yulia+Tsvetkov%2E+2019%2E+Quantifying+Social+Biases+in+Contextual+Word+Representations%2E+In+1st+ACL+Workshop+on+Gender+Bias+for+Natural+Language+Pr)
11. Brian Larson. 2017. Gender as a Variable in Natural-Language Processing: Ethical Considerations. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Brian+Larson%2E+2017%2E+Gender+as+a+Variable+in+Natural%2DLanguage+Processing%3A+Ethical+Considerations%2E+In+Proceedings+of+the+First+ACL+Workshop+on+Ethics+in+Natural+Language+Processing%2E)
12. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar S. Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke S. Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized bert pretraining approach. arXiv. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+S%2E+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+S%2E+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+RoBERTa%3A+A+robustly+optimized+bert+pretraining+appro)
13. Moin Nadeem, Anna Bethke, and Siva Reddy. 2020. StereoSet: Measuring stereotypical bias in pre-trained language models. arXiv. [find](https://scholar.google.com/scholar?q=Moin+Nadeem%2C+Anna+Bethke%2C+and+Siva+Reddy%2E+2020%2E+StereoSet%3A+Measuring+stereotypical+bias+in+pre%2Dtrained+language+models%2E+arXiv%2E)
14. Won Ik Cho, Ji Won Kim, Seok Min Kim, and Nam Soo Kim. 2019. On measuring gender bias in translation of gender-neutral pronouns. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Won+Ik+Cho%2C+Ji+Won+Kim%2C+Seok+Min+Kim%2C+and+Nam+Soo+Kim%2E+2019%2E+On+measuring+gender+bias+in+translation+of+gender%2Dneutral+pronouns%2E+In+Proceedings+of+the+First+Workshop+on+Gender+Bias+in+Natural+Language)
15. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Proce) [our card](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/)
16. Kate Crawford. 2017. The Trouble with Bias. In Conference on Neural Information Processing Systems, invited speaker. [find](https://scholar.google.com/scholar?q=Kate+Crawford%2E+2017%2E+The+Trouble+with+Bias%2E+In+Conference+on+Neural+Information+Processing+Systems%2C+invited+speaker%2E)
17. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol… [find](https://scholar.google.com/scholar?q=Matthew+E+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E+In+Proceedings+of+the+2018+Conference+of) [our card](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/)
18. Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Kalai. 2019. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In Proceedings of the Conference on Fairness, Acco… [find](https://scholar.google.com/scholar?q=Maria+De%2DArteaga%2C+Alexey+Romanov%2C+Hanna+Wallach%2C+Jennifer+Chayes%2C+Christian+Borgs%2C+Alexandra+Chouldechova%2C+Sahin+Geyik%2C+Krishnaram+Kenthapadi%2C+and+Adam+Kalai%2E+2019%2E+Bias+in+bios%3A+A+case+study+of+seman)
19. Sunipa Dev, Tao Li, Jeff Philips, and Vivek Srikumar. 2020a. On Measuring and Mitigating Biased Inferences of Word Embeddings. In the Thirty-Fourth AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Sunipa+Dev%2C+Tao+Li%2C+Jeff+Philips%2C+and+Vivek+Srikumar%2E+2020a%2E+On+Measuring+and+Mitigating+Biased+Inferences+of+Word+Embeddings%2E+In+the+Thirty%2DFourth+AAAI+Conference+on+Artificial+Intelligence%2E)
20. Sunipa Dev, Tao Li, Jeff M Phillips, and Vivek Srikumar. 2020b. OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings. arXiv. [find](https://scholar.google.com/scholar?q=Sunipa+Dev%2C+Tao+Li%2C+Jeff+M+Phillips%2C+and+Vivek+Srikumar%2E+2020b%2E+OSCaR%3A+Orthogonal+Subspace+Correction+and+Rectification+of+Biases+in+Word+Embeddings%2E+arXiv%2E)
21. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Jian+Zhang%2C+Konstantin+Lopyrev%2C+and+Percy+Liang%2E+2016%2E+SQuAD%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natu) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
22. Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 3484 10 [find](https://scholar.google.com/scholar?q=Shauli+Ravfogel%2C+Yanai+Elazar%2C+Hila+Gonen%2C+Michael+Twiton%2C+and+Yoav+Goldberg%2E+2020%2E+Null+It+Out%3A+Guarding+Protected+Attributes+by+Iterative+Nullspace+Projection%2E+In+Proceedings+of+the+58th+Annual+Meet) [our card](https://md.research.elamur.ai/free/en/papers/ravfogel-2020-null-it-out-guarding-protected/)
23. Marta Recasens, Cristian Danescu-Niculescu-Mizil, and Dan Jurafsky. 2013. Linguistic Models for Analyzing and Detecting Biased Language. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Marta+Recasens%2C+Cristian+Danescu%2DNiculescu%2DMizil%2C+and+Dan+Jurafsky%2E+2013%2E+Linguistic+Models+for+Analyzing+and+Detecting+Biased+Language%2E+In+Proceedings+of+the+51st+Annual+Meeting+of+the+Association+fo) [our card](https://md.research.elamur.ai/free/en/papers/recasens-2013-linguistic-models-analyzing-detecting-biased/)
24. Haoran Zhang, Amy X Lu, Mohamed Abdalla, Matthew McDermott, and Marzyeh Ghassemi. 2020. Hurtful words: Quantifying biases in clinical contextual word embeddings. In Proceedings of the ACM Conference on Health, Inference, and Learning. [find](https://scholar.google.com/scholar?q=Haoran+Zhang%2C+Amy+X+Lu%2C+Mohamed+Abdalla%2C+Matthew+McDermott%2C+and+Marzyeh+Ghassemi%2E+2020%2E+Hurtful+words%3A+Quantifying+biases+in+clinical+contextual+word+embeddings%2E+In+Proceedings+of+the+ACM+Conference+o)
25. Marco Tulio Ribeiro, Carlos Guestrin, and Sameer Singh. 2019. Are Red Roses Red? Evaluating Consistency of Question-Answering Models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+Carlos+Guestrin%2C+and+Sameer+Singh%2E+2019%2E+Are+Red+Roses+Red%3F+Evaluating+Consistency+of+Question%2DAnswering+Models%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+C)
26. Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang, and Ahmed Awadallah. 2020. Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jieyu+Zhao%2C+Subhabrata+Mukherjee%2C+Saghar+Hosseini%2C+Kai%2DWei+Chang%2C+and+Ahmed+Awadallah%2E+2020%2E+Gender+Bias+in+Multilingual+Embeddings+and+Cross%2DLingual+Transfer%2E+In+Proceedings+of+the+58th+Annual+Meetin)
27. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. [find](https://scholar.google.com/scholar?q=Rachel+Rudinger%2C+Jason+Naradowsky%2C+Brian+Leonard%2C+and+Benjamin+Van+Durme%2E+2018%2E+Gender+bias+in+coreference+resolution%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter+of+the+Associ) [our card](https://md.research.elamur.ai/free/en/papers/rudinger-2018-gender-bias-coreference-resolution/)
28. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender Bias in Contextualized Word Embeddings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. [find](https://scholar.google.com/scholar?q=Jieyu+Zhao%2C+Tianlu+Wang%2C+Mark+Yatskar%2C+Ryan+Cotterell%2C+Vicente+Ordonez%2C+and+Kai%2DWei+Chang%2E+2019%2E+Gender+Bias+in+Contextualized+Word+Embeddings%2E+In+Proceedings+of+the+2019+Conference+of+the+North+Ameri) [our card](https://md.research.elamur.ai/free/en/papers/zhao-2019-gender-bias-contextualized-word-embeddings/)
29. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBert, a distilled version of BERT: Smaller, faster, cheaper and lighter. In the Thirty-third Conference on Neural Information Processing Systems, 5th Workshop on Energy Efficient Machine Learning and Cognitive Computing. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+2019%2E+DistilBert%2C+a+distilled+version+of+BERT%3A+Smaller%2C+faster%2C+cheaper+and+lighter%2E+In+the+Thirty%2Dthird+Conference+on+Neural+Information)
30. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. [find](https://scholar.google.com/scholar?q=Jieyu+Zhao%2C+Tianlu+Wang%2C+Mark+Yatskar%2C+Vicente+Ordonez%2C+and+Kai%2DWei+Chang%2E+2018%2E+Gender+bias+in+coreference+resolution%3A+Evaluation+and+debiasing+methods%2E+In+Proceedings+of+the+2018+Conference+of+the+N) [our card](https://md.research.elamur.ai/free/en/papers/zhao-2018-gender-bias-coreference-resolution-evaluation/)
31. Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2019. The Woman Worked as a Babysitter: On Biases in Language Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. [find](https://scholar.google.com/scholar?q=Emily+Sheng%2C+Kai%2DWei+Chang%2C+Prem+Natarajan%2C+and+Nanyun+Peng%2E+2019%2E+The+Woman+Worked+as+a+Babysitter%3A+On+Biases+in+Language+Generation%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Nat) [our card](https://md.research.elamur.ai/free/en/papers/sheng-2019-woman-worked-as-babysitter-biases/)
32. Gabriel Stanovsky, Noah A Smith, and Luke Zettlemoyer. 2019. Evaluating gender bias in machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Gabriel+Stanovsky%2C+Noah+A+Smith%2C+and+Luke+Zettlemoyer%2E+2019%2E+Evaluating+gender+bias+in+machine+translation%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/stanovsky-2019-evaluating-gender-bias-machine-translation/)
33. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. In Proceedings of the 57th Annual Meeting of the Association for Computati… [find](https://scholar.google.com/scholar?q=Tony+Sun%2C+Andrew+Gaut%2C+Shirlyn+Tang%2C+Yuxin+Huang%2C+Mai+ElSherief%2C+Jieyu+Zhao%2C+Diba+Mirza%2C+Elizabeth+Belding%2C+Kai%2DWei+Chang%2C+and+William+Yang+Wang%2E+2019%2E+Mitigating+gender+bias+in+natural+language+proce) [our card](https://md.research.elamur.ai/free/en/papers/sun-2019-mitigating-gender-bias-natural-language/)
34. Yi Chern Tan and L Elisa Celis. 2019. Assessing Social and Intersectional Biases in Contextualized Word Representations. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Yi+Chern+Tan+and+L+Elisa+Celis%2E+2019%2E+Assessing+Social+and+Intersectional+Biases+in+Contextualized+Word+Representations%2E+In+Advances+in+Neural+Information+Processing+Systems%2E)
35. Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017. NewsQA: A machine comprehension dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP. [find](https://scholar.google.com/scholar?q=Adam+Trischler%2C+Tong+Wang%2C+Xingdi+Yuan%2C+Justin+Harris%2C+Alessandro+Sordoni%2C+Philip+Bachman%2C+and+Kaheer+Suleman%2E+2017%2E+NewsQA%3A+A+machine+comprehension+dataset%2E+In+Proceedings+of+the+2nd+Workshop+on+Repr) [our card](https://md.research.elamur.ai/free/en/papers/trischler-2017-newsqa-machine-comprehension-dataset/)
36. Thomas Wolf, L Debut, V Sanh, J Chaumond, C Delangue, A Moi, P Cistac, T Rault, R Louf, M Funtowicz, et al. 2019. Huggingface’s transformers: State-of-the-art natural language processing. arXiv. 3485 11 [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+L+Debut%2C+V+Sanh%2C+J+Chaumond%2C+C+Delangue%2C+A+Moi%2C+P+Cistac%2C+T+Rault%2C+R+Louf%2C+M+Funtowicz%2C+et+al%2E+2019%2E+Huggingface%E2%80%99s+transformers%3A+State%2Dof%2Dthe%2Dart+natural+language+processing%2E+arXiv%2E+3485+) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)