---
title: "Social Bias Frames: Reasoning about Social and Power Implications of Language — Sap 2020"
description: "The authors propose Social Bias Frames, a formalism for the social bias a post projects. 80% F1. 75 references, cited by 348 papers here."
authors: ["Maarten Sap", "Saadia Gabriel", "Lianhui Qin", "Dan Jurafsky", "Noah A. Smith", "Yejin Choi"]
year: 2020
venue: "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/2020.acl-main.486/"
doi: "10.18653/v1/2020.acl-main.486"
arxiv: "1911.03891"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sap-2020-social-bias-frames-reasoning-about/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sap-2020-social-bias-frames-reasoning-about/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/) / Social Bias Frames: Reasoning about Social and Power Implications of Language

# Social Bias Frames: Reasoning about Social and Power Implications of Language

**Authors:** [Maarten Sap](https://md.research.elamur.ai/free/en/authors/maarten-sap/), [Saadia Gabriel](https://md.research.elamur.ai/free/en/authors/saadia-gabriel/), [Lianhui Qin](https://md.research.elamur.ai/free/en/authors/lianhui-qin/), [Dan Jurafsky](https://md.research.elamur.ai/free/en/authors/dan-jurafsky/), [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/) and [Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/) · **Year:** 2020 · **Venue:** Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.acl-main.486/) · [PDF](https://aclanthology.org/2020.acl-main.486.pdf) · [DOI 10.18653/v1/2020.acl-main.486](https://doi.org/10.18653/v1/2020.acl-main.486) · [arXiv 1911.03891](https://arxiv.org/abs/1911.03891) · [OpenAlex](https://openalex.org/W2984662393)

**Citations:** 332 citations (OpenAlex, 2026-10-14) · [348](https://md.research.elamur.ai/free/en/papers/sap-2020-social-bias-frames-reasoning-about/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
Toxicity detectors usually give a post one label, such as offensive or not. A label like that doesn't say why a post is harmful, and harm often comes from what a post implies rather than what it says. The authors propose Social Bias Frames, a formalism for the social bias a post projects. It mixes categorical judgments with free-text explanations: whether the post is offensive, whether it was meant to offend, whether it is lewd, whether it targets a group or only an individual, which group it targets, what stereotype it implies, and whether the speaker might belong to that group. To support the formalism, they crowdsourced the Social Bias Inference Corpus (SBIC). It covers posts from Reddit, Twitter and hate sites such as Gab and Stormfront. They then trained baseline models built on pretrained language models, including GPT-2 variants. The models can decide fairly well whether a post is offensive, at about 80% F1. They do poorly at writing the implied stereotype and tend to produce generic statements about a group. The work moves bias detection toward explainable reasoning that people can check.

## Key points
- SBIC has about 150k structured annotations on 44,671 posts. They include over 34k implied statements about roughly a thousand demographic groups (the related-work section gives about 1.4k).
- Each frame has seven variables. Offensiveness, intent, lewdness, group implication and in-group language are categorical. The targeted group and the implied statement are free text, with implications written as simple Hearst-style patterns (for example, "GROUP does ___").
- The posts come from r/darkJokes, r/meanJokes, r/offensiveJokes and Microaggressions on Reddit, from existing Twitter toxicity datasets (Founta et al., Davidson et al., Waseem and Hovy), and from Gab, Stormfront and banned subreddits (r/Incels, r/MensRights).
- Error analysis of SBF-GPT2-gdy-constr shows it usually gets the stereotype right only when the stereotype shares many words with the post.
- The authors frame bias using a U.S. sociocultural lens. To protect annotators, they capped how many posts each person could label per day, paid $7–12, and pointed workers to crisis-support resources.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/)

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 248 works · 214,804 citations
- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 206 works · 158,423 citations

## Cite

Cite as: Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith and Yejin Choi. 2020. Social Bias Frames: Reasoning about Social and Power Implications of Language. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sap2020social,
  title = {{Social Bias Frames: Reasoning about Social and Power Implications of Language}},
  author = {Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith and Yejin Choi},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.486},
  eprint = {1911.03891},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.acl-main.486}
}
```

- [← previous Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/)
- [next → Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings](https://md.research.elamur.ai/free/en/papers/saxena-2020-improving-multi-hop-question-answering/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/)

## References

75 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Maria Antoniak, David Mimno, and Karen Levy. 2019. Narrative paths and negotiation of power in birth stories. In CSCW. [find](https://scholar.google.com/scholar?q=Maria+Antoniak%2C+David+Mimno%2C+and+Karen+Levy%2E+2019%2E+Narrative+paths+and+negotiation+of+power+in+birth+stories%2E+In+CSCW%2E)
2. John W Ayers, Theodore L Caputi, Camille Nebeker, and Mark Dredze. 2018. Don’t quote me: reverse identification of research participants in social media studies. NPJ digital medicine, 1(1):1–2. [find](https://scholar.google.com/scholar?q=John+W+Ayers%2C+Theodore+L+Caputi%2C+Camille+Nebeker%2C+and+Mark+Dredze%2E+2018%2E+Don%E2%80%99t+quote+me%3A+reverse+identification+of+research+participants+in+social+media+studies%2E+NPJ+digital+medicine%2C+1%281%29%3A1%E2%80%932%2E)
3. Maria Becker, Michael Staniek, Vivi Nastase, and Anette Frank. 2017. Enriching argumentative texts with implicit knowledge. In NLDB. [find](https://scholar.google.com/scholar?q=Maria+Becker%2C+Michael+Staniek%2C+Vivi+Nastase%2C+and+Anette+Frank%2E+2017%2E+Enriching+argumentative+texts+with+implicit+knowledge%2E+In+NLDB%2E)
4. Jeanette Bicknell. 2007. What is offensive about offensive jokes? Philosophy Today, 51(4):458–465. [find](https://scholar.google.com/scholar?q=Jeanette+Bicknell%2E+2007%2E+What+is+offensive+about+offensive+jokes%3F+Philosophy+Today%2C+51%284%29%3A458%E2%80%93465%2E)
5. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. Demographic dialectal variation in social media: a case study of African-American English. In EMNLP. [our card](https://md.research.elamur.ai/free/en/papers/blodgett-2016-demographic-dialectal-variation-social-media/)
6. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: commonsense transformers for automatic knowledge graph construction. In ACL. [our card](https://md.research.elamur.ai/free/en/papers/bosselut-2019-comet-commonsense-transformers-automatic-knowledge/)
7. Lorraine Bowman-Grieve. 2009. Exploring “Stormfront”: a virtual community of the radical right. Studies in conflict & terrorism, 32(11):989–1007. [find](https://scholar.google.com/scholar?q=Lorraine+Bowman%2DGrieve%2E+2009%2E+Exploring+%E2%80%9CStormfront%E2%80%9D%3A+a+virtual+community+of+the+radical+right%2E+Studies+in+conflict+%26+terrorism%2C+32%2811%29%3A989%E2%80%931007%2E)
8. Luke M Breitfeller, Emily Ahn, David Jurgens, and Yulia Tsvetkov. 2019. Finding microaggressions in the wild: a case for locating elusive phenomena in social media posts. In EMNLP. [find](https://scholar.google.com/scholar?q=Luke+M+Breitfeller%2C+Emily+Ahn%2C+David+Jurgens%2C+and+Yulia+Tsvetkov%2E+2019%2E+Finding+microaggressions+in+the+wild%3A+a+case+for+locating+elusive+phenomena+in+social+media+posts%2E+In+EMNLP%2E)
9. Adrian Bussone, Simone Stumpf, and Dympna O’Sullivan. 2015. The role of explanations on trust and reliance in clinical decision support systems. In 2015 International Conference on Healthcare Informatics, pages 160–169. IEEE. [find](https://scholar.google.com/scholar?q=Adrian+Bussone%2C+Simone+Stumpf%2C+and+Dympna+O%E2%80%99Sullivan%2E+2015%2E+The+role+of+explanations+on+trust+and+reliance+in+clinical+decision+support+systems%2E+In+2015+International+Conference+on+Healthcare+Informat)
10. Southern Poverty Law Center. 2012. Misogyny: the sites. Intelligence Report, 145. [find](https://scholar.google.com/scholar?q=Southern+Poverty+Law+Center%2E+2012%2E+Misogyny%3A+the+sites%2E+Intelligence+Report%2C+145%2E)
11. Yi-Ling Chung, Elizaveta Kuzmenko, Serra Sinem Tekiroglu, and Marco Guerini. 2019. CONAN -COunter NArratives through nichesourcing: a multilingual dataset of responses to fight online hate speech. In ACL. [our card](https://md.research.elamur.ai/free/en/papers/chung-2019-conan-counter-narratives-through-nichesourcing/)
12. Raphael Cohen-Almagor. 2014. Countering hate on the internet. Annual review of law and ethics, 22:431–443. [find](https://scholar.google.com/scholar?q=Raphael+Cohen%2DAlmagor%2E+2014%2E+Countering+hate+on+the+internet%2E+Annual+review+of+law+and+ethics%2C+22%3A431%E2%80%93443%2E)
13. Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In KDD. [find](https://scholar.google.com/scholar?q=Sam+Corbett%2DDavies%2C+Emma+Pierson%2C+Avi+Feller%2C+Sharad+Goel%2C+and+Aziz+Huq%2E+2017%2E+Algorithmic+decision+making+and+the+cost+of+fairness%2E+In+KDD%2E)
14. Adam M Croom. 2011. Slurs. Language Sciences, 33(3):343–358. [find](https://scholar.google.com/scholar?q=Adam+M+Croom%2E+2011%2E+Slurs%2E+Language+Sciences%2C+33%283%29%3A343%E2%80%93358%2E)
15. Helen L Daly. 2018. On insults. Journal of the American Philosophical Association, 4(4):510–524. [find](https://scholar.google.com/scholar?q=Helen+L+Daly%2E+2018%2E+On+insults%2E+Journal+of+the+American+Philosophical+Association%2C+4%284%29%3A510%E2%80%93524%2E)
16. Cristian Danescu-Niculescu-Mizil, Lillian Lee, Bo Pang, and Jon Kleinberg. 2012. Echoes of power: language effects and power differences in social interaction. In WWW. [find](https://scholar.google.com/scholar?q=Cristian+Danescu%2DNiculescu%2DMizil%2C+Lillian+Lee%2C+Bo+Pang%2C+and+Jon+Kleinberg%2E+2012%2E+Echoes+of+power%3A+language+effects+and+power+differences+in+social+interaction%2E+In+WWW%2E)
17. Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019. Racial bias in hate speech and abusive language detection datasets. In Abusive Language Workshop. [our card](https://md.research.elamur.ai/free/en/papers/davidson-2019-racial-bias-hate-speech-abusive/)
18. Thomas Davidson, Dana Warmsley, Michael W Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In ICWSM. [our card](https://md.research.elamur.ai/free/en/papers/davidson-2017-automated-hate-speech-detection-problem/)
19. Serena Does, Belle Derks, and Naomi Ellemers. 2011. Thou shalt not discriminate: how emphasizing moral ideals rather than obligations increases whites’ support for social equality. Journal of Experimental Social Psychology, 47(3):562–571. [find](https://scholar.google.com/scholar?q=Serena+Does%2C+Belle+Derks%2C+and+Naomi+Ellemers%2E+2011%2E+Thou+shalt+not+discriminate%3A+how+emphasizing+moral+ideals+rather+than+obligations+increases+whites%E2%80%99+support+for+social+equality%2E+Journal+of+Experime)
20. Marta Dynel. 2015. The landscape of impoliteness research. Journal of Politeness Research, 11(2):383. [find](https://scholar.google.com/scholar?q=Marta+Dynel%2E+2015%2E+The+landscape+of+impoliteness+research%2E+Journal+of+Politeness+Research%2C+11%282%29%3A383%2E)
21. Connie C Eble. 1996. Slang & sociability: in-group language among college students. Univ of North Carolina Press. [find](https://scholar.google.com/scholar?q=Connie+C+Eble%2E+1996%2E+Slang+%26+sociability%3A+in%2Dgroup+language+among+college+students%2E+Univ+of+North+Carolina+Press%2E)
22. Anjalie Field, Gayatri Bhat, and Yulia Tsvetkov. 2019. Contextual affective analysis: a case study of people portrayals in online \#MeToo stories. In ICWSM. [find](https://scholar.google.com/scholar?q=Anjalie+Field%2C+Gayatri+Bhat%2C+and+Yulia+Tsvetkov%2E+2019%2E+Contextual+affective+analysis%3A+a+case+study+of+people+portrayals+in+online+%5C%23MeToo+stories%2E+In+ICWSM%2E)
23. Charles J Fillmore and Collin F Baker. 2001. Frame semantics for text understanding. In Proceedings of WordNet and Other Lexical Resources Workshop, NAACL. [find](https://scholar.google.com/scholar?q=Charles+J+Fillmore+and+Collin+F+Baker%2E+2001%2E+Frame+semantics+for+text+understanding%2E+In+Proceedings+of+WordNet+and+Other+Lexical+Resources+Workshop%2C+NAACL%2E)
24. Jon Fingas. 2017. Reddit bans misogynist community as part of anti-violence crackdown. [https://www.engadget.com/2017/11/08/](https://www.engadget.com/2017/11/08/) reddit-bans-misogynist-community-in-anti-violence-crackdown/. Accessed: 2019-12-06. [↗](https://www.engadget.com/2017/11/08/)
25. Susan T Fiske. 1993. Controlling other people. the impact of power on stereotyping. American psychologist, 48(6):621–628. [find](https://scholar.google.com/scholar?q=Susan+T+Fiske%2E+1993%2E+Controlling+other+people%2E+the+impact+of+power+on+stereotyping%2E+American+psychologist%2C+48%286%29%3A621%E2%80%93628%2E)
26. Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of Twitter abusive behavior. In ICWSM. [find](https://scholar.google.com/scholar?q=Antigoni%2DMaria+Founta%2C+Constantinos+Djouvas%2C+Despoina+Chatzakou%2C+Ilias+Leontiadis%2C+Jeremy+Blackburn%2C+Gianluca+Stringhini%2C+Athena+Vakali%2C+Michael+Sirivianos%2C+and+Nicolas+Kourtellis%2E+2018%2E+Large+scale+c)
27. Yiannis Gabriel. 1998. An introduction to the social psychology of insults in organizations. Human Relations, 51(11):1329–1354. [find](https://scholar.google.com/scholar?q=Yiannis+Gabriel%2E+1998%2E+An+introduction+to+the+social+psychology+of+insults+in+organizations%2E+Human+Relations%2C+51%2811%29%3A1329%E2%80%931354%2E)
28. Adam D Galinsky, Cynthia S Wang, Jennifer A Whitson, Eric M Anicich, Kurt Hugenberg, and Galen V Bodenhausen. 2013. The reappropriation of stigmatizing labels: the reciprocal relationship between power and self-labeling. Psychol. Sci., 24(10):2020–2029. [find](https://scholar.google.com/scholar?q=Adam+D+Galinsky%2C+Cynthia+S+Wang%2C+Jennifer+A+Whitson%2C+Eric+M+Anicich%2C+Kurt+Hugenberg%2C+and+Galen+V+Bodenhausen%2E+2013%2E+The+reappropriation+of+stigmatizing+labels%3A+the+reciprocal+relationship+between+powe)
29. Michel Galley, Chris Brockett, Alessandro Sordoni, Yangfeng Ji, Michael Auli, Chris Quirk, Margaret Mitchell, Jianfeng Gao, and William B. Dolan. 2015. deltaBLEU: a discriminative metric for generation tasks with intrinsically diverse targets. In ACL. [find](https://scholar.google.com/scholar?q=Michel+Galley%2C+Chris+Brockett%2C+Alessandro+Sordoni%2C+Yangfeng+Ji%2C+Michael+Auli%2C+Chris+Quirk%2C+Margaret+Mitchell%2C+Jianfeng+Gao%2C+and+William+B%2E+Dolan%2E+2015%2E+deltaBLEU%3A+a+discriminative+metric+for+generatio)
30. Ona de Gibert, Naiara Pérez, Aitor Garcı́a-Pablos, and Montse Cuadros. 2018. Hate speech dataset from a white supremacy forum. In Abusive Language Workshop at EMNLP. [our card](https://md.research.elamur.ai/free/en/papers/gibert-2018-hate-speech-dataset-white-supremacy/)
31. Gil Greengross and Geoffrey F Miller. 2008. Dissing oneself versus dissing rivals: effects of status, personality, and sex on the Short-Term and Long-Term attractiveness of Self-Deprecating and Other-Deprecating humor. Evolutionary Psychology, 6(3). [find](https://scholar.google.com/scholar?q=Gil+Greengross+and+Geoffrey+F+Miller%2E+2008%2E+Dissing+oneself+versus+dissing+rivals%3A+effects+of+status%2C+personality%2C+and+sex+on+the+Short%2DTerm+and+Long%2DTerm+attractiveness+of+Self%2DDeprecating+and+Other%2D)
32. Shirley Gregor and Izak Benbasat. 1999. Explanations from intelligent systems: Theoretical foundations and implications for practice. MIS quarterly, pages 497–530. [find](https://scholar.google.com/scholar?q=Shirley+Gregor+and+Izak+Benbasat%2E+1999%2E+Explanations+from+intelligent+systems%3A+Theoretical+foundations+and+implications+for+practice%2E+MIS+quarterly%2C+pages+497%E2%80%93530%2E)
33. Ivan Habernal and Iryna Gurevych. 2016. What makes a convincing argument? empirical analysis and detecting attributes of convincingness in web argumentation. In EMNLP, pages 1214–1223. [find](https://scholar.google.com/scholar?q=Ivan+Habernal+and+Iryna+Gurevych%2E+2016%2E+What+makes+a+convincing+argument%3F+empirical+analysis+and+detecting+attributes+of+convincingness+in+web+argumentation%2E+In+EMNLP%2C+pages+1214%E2%80%931223%2E)
34. Tatsunori B Hashimoto, Hugh Zhang, and Percy Liang. 2019. Unifying human and statistical evaluation for natural language generation. In NAACL-HLT. [find](https://scholar.google.com/scholar?q=Tatsunori+B+Hashimoto%2C+Hugh+Zhang%2C+and+Percy+Liang%2E+2019%2E+Unifying+human+and+statistical+evaluation+for+natural+language+generation%2E+In+NAACL%2DHLT%2E)
35. Marti A Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In ACL, pages 539– 545. [find](https://scholar.google.com/scholar?q=Marti+A+Hearst%2E+1992%2E+Automatic+acquisition+of+hyponyms+from+large+text+corpora%2E+In+ACL%2C+pages+539%E2%80%93+545%2E)
36. Amanda Hess. 2016. The far right has a new digital safe space. [https://www.nytimes.com/2016/11/30/arts/the-far-right-has-a-new-digital-safe-space.html](https://www.nytimes.com/2016/11/30/arts/the-far-right-has-a-new-digital-safe-space.html). Accessed: 2019-12-06. [↗](https://www.nytimes.com/2016/11/30/arts/the-far-right-has-a-new-digital-safe-space.html)
37. Gabriele Kasper. 1990. Linguistic politeness: current research issues. Journal of Pragmatics, 14(2):193– 218. [find](https://scholar.google.com/scholar?q=Gabriele+Kasper%2E+1990%2E+Linguistic+politeness%3A+current+research+issues%2E+Journal+of+Pragmatics%2C+14%282%29%3A193%E2%80%93+218%2E)
38. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: a conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858. [our card](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/)
39. Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell me more? The effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1–10. ACM. [find](https://scholar.google.com/scholar?q=Todd+Kulesza%2C+Simone+Stumpf%2C+Margaret+Burnett%2C+and+Irwin+Kwan%2E+2012%2E+Tell+me+more%3F+The+effects+of+mental+model+soundness+on+personalizing+an+intelligent+agent%2E+In+Proceedings+of+the+SIGCHI+Conference+)
40. Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. 2015. From word embeddings to document distances. In ICML, pages 957–966. [find](https://scholar.google.com/scholar?q=Matt+Kusner%2C+Yu+Sun%2C+Nicholas+Kolkin%2C+and+Kilian+Weinberger%2E+2015%2E+From+word+embeddings+to+document+distances%2E+In+ICML%2C+pages+957%E2%80%93966%2E)
41. Robin Lakoff. 1973. Language and woman’s place. Language in society, 2(1):45–79. [find](https://scholar.google.com/scholar?q=Robin+Lakoff%2E+1973%2E+Language+and+woman%E2%80%99s+place%2E+Language+in+society%2C+2%281%29%3A45%E2%80%9379%2E)
42. Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How NOT to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation. In ACL. [our card](https://md.research.elamur.ai/free/en/papers/liu-2016-how-not-evaluate-your-dialogue/)
43. Marie-Catherine de Marneffe, Christopher D Manning, and Christopher Potts. 2012. Did it happen? the pragmatic complexity of veridicality assessment. Computational Linguistics, 38(2):301–333. [our card](https://md.research.elamur.ai/free/en/papers/marneffe-2012-did-it-happen-pragmatic-complexity/)
44. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In FAccT. [find](https://scholar.google.com/scholar?q=Margaret+Mitchell%2C+Simone+Wu%2C+Andrew+Zaldivar%2C+Parker+Barnes%2C+Lucy+Vasserman%2C+Ben+Hutchinson%2C+Elena+Spitzer%2C+Inioluwa+Deborah+Raji%2C+and+Timnit+Gebru%2E+2019%2E+Model+cards+for+model+reporting%2E+In+FAccT%2E)
45. An T Nguyen, Aditya Kharosekar, Saumyaa Krishnan, Siddhesh Krishnan, Elizabeth Tate, Byron C Wallace, and Matthew Lease. 2018. Believe it or not: designing a human-AI partnership for mixed-initiative fact-checking. In The 31st Annual ACM Symposium on User Interface Software and Technology, pages 189… [find](https://scholar.google.com/scholar?q=An+T+Nguyen%2C+Aditya+Kharosekar%2C+Saumyaa+Krishnan%2C+Siddhesh+Krishnan%2C+Elizabeth+Tate%2C+Byron+C+Wallace%2C+and+Matthew+Lease%2E+2018%2E+Believe+it+or+not%3A+designing+a+human%2DAI+partnership+for+mixed%2Dinitiative+)
46. Conor J O’Dea, Stuart S Miller, Emma B Andres, Madelyn H Ray, Derrick F Till, and Donald A Saucier. 2015. Out of bounds: Factors affecting the perceived offensiveness of racial slurs. Language Sciences, 52:155–164. [find](https://scholar.google.com/scholar?q=Conor+J+O%E2%80%99Dea%2C+Stuart+S+Miller%2C+Emma+B+Andres%2C+Madelyn+H+Ray%2C+Derrick+F+Till%2C+and+Donald+A+Saucier%2E+2015%2E+Out+of+bounds%3A+Factors+affecting+the+perceived+offensiveness+of+racial+slurs%2E+Language+Science)
47. Nedjma Ousidhoum, Zizheng Lin, Hongming Zhang, Yangqiu Song, and Dit-Yan Yeung. 2019. Multilingual and Multi-Aspect hate speech analysis. In EMNLP. [our card](https://md.research.elamur.ai/free/en/papers/ousidhoum-2019-multilingual-multi-aspect-hate-speech/)
48. Gonçalo Pereira, Rui Prada, and Pedro A Santos. 2016. Integrating social power into the decision-making of cognitive agents. Artificial Intelligence, 241:1–44. [find](https://scholar.google.com/scholar?q=Gonc%CC%A7alo+Pereira%2C+Rui+Prada%2C+and+Pedro+A+Santos%2E+2016%2E+Integrating+social+power+into+the+decision%2Dmaking+of+cognitive+agents%2E+Artificial+Intelligence%2C+241%3A1%E2%80%9344%2E)
49. Vinodkumar Prabhakaran, Prabhakaran Vinodkumar, and Rambow Owen. 2014. Predicting power relations between participants in written dialog from a single thread. In ACL. [find](https://scholar.google.com/scholar?q=Vinodkumar+Prabhakaran%2C+Prabhakaran+Vinodkumar%2C+and+Rambow+Owen%2E+2014%2E+Predicting+power+relations+between+participants+in+written+dialog+from+a+single+thread%2E+In+ACL%2E)
50. Jing Qian, Anna Bethke, Yinyin Liu, Elizabeth Belding, and William Yang Wang. 2019. A benchmark dataset for learning to intervene in online hate speech. In EMNLP. [our card](https://md.research.elamur.ai/free/en/papers/qian-2019-benchmark-dataset-learning-intervene-online/)
51. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Unpublished. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+and+Ilya+Sutskever%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E+Unpublished%2E)
52. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. Unpublished. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+Unpublished%2E)
53. Hannah Rashkin, Maarten Sap, Emily Allaway, Noah A. Smith, and Yejin Choi. 2018. Event2mind: commonsense inference on events, intents, and reactions. In ACL. [find](https://scholar.google.com/scholar?q=Hannah+Rashkin%2C+Maarten+Sap%2C+Emily+Allaway%2C+Noah+A%2E+Smith%2C+and+Yejin+Choi%2E+2018%2E+Event2mind%3A+commonsense+inference+on+events%2C+intents%2C+and+reactions%2E+In+ACL%2E)
54. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why should I trust you?”: Explaining the predictions of any classifier. In KDD. [our card](https://md.research.elamur.ai/free/en/papers/ribeiro-2016-why-should-i-trust-explaining/)
55. Sarah T Roberts. 2016. Commercial content moderation: digital laborers’ dirty work. In Safiya Umoja Noble and Brendesha M Tynes, editors, The Intersectional Internet: Race, Sex, Class and Culture Online, Media Studies Publications. Peter Lang Publishing. [find](https://scholar.google.com/scholar?q=Sarah+T+Roberts%2E+2016%2E+Commercial+content+moderation%3A+digital+laborers%E2%80%99+dirty+work%2E+In+Safiya+Umoja+Noble+and+Brendesha+M+Tynes%2C+editors%2C+The+Intersectional+Internet%3A+Race%2C+Sex%2C+Class+and+Culture+Onli)
56. Björn Ross, Michael Rist, Guillermo Carbonell, Benjamin Cabrera, Nils Kurowsky, and Michael Wojatzki. 2017. Measuring the reliability of hate speech annotations: the case of the european refugee crisis. In NLP 4 CMC Workshop. [find](https://scholar.google.com/scholar?q=Bjo%CC%88rn+Ross%2C+Michael+Rist%2C+Guillermo+Carbonell%2C+Benjamin+Cabrera%2C+Nils+Kurowsky%2C+and+Michael+Wojatzki%2E+2017%2E+Measuring+the+reliability+of+hate+speech+annotations%3A+the+case+of+the+european+refugee+cris)
57. RWJF. 2017. Discrimination in america: experiences and views. [https://www.rwjf.org/en/library/research/2017/](https://www.rwjf.org/en/library/research/2017/) 10/discrimination-in-america-- experiences-and-views.html. Accessed: 2019-11-5. [↗](https://www.rwjf.org/en/library/research/2017/)
58. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Winogrande: an adversarial winograd schema challenge at scale. In AAAI. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2020%2E+Winogrande%3A+an+adversarial+winograd+schema+challenge+at+scale%2E+In+AAAI%2E)
59. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019a. The risk of racial bias in hate speech detection. In ACL. [our card](https://md.research.elamur.ai/free/en/papers/sap-2019-risk-racial-bias-hate-speech/)
60. Maarten Sap, Ronan LeBras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. 2019b. ATOMIC: an atlas of machine commonsense for if-then reasoning. In AAAI. [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Ronan+LeBras%2C+Emily+Allaway%2C+Chandra+Bhagavatula%2C+Nicholas+Lourie%2C+Hannah+Rashkin%2C+Brendan+Roof%2C+Noah+A+Smith%2C+and+Yejin+Choi%2E+2019b%2E+ATOMIC%3A+an+atlas+of+machine+commonsense+for+if%2Dthen+r)
61. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Predicting the type and target of offensive posts in social media. In NAACL. [our card](https://md.research.elamur.ai/free/en/papers/zampieri-2019-predicting-type-target-offensive-posts/)
62. Michael Zimmer. 2018. Addressing conceptual gaps in big data research ethics: an application of contextual integrity. Social Media + Society, 4(2). [find](https://scholar.google.com/scholar?q=Michael+Zimmer%2E+2018%2E+Addressing+conceptual+gaps+in+big+data+research+ethics%3A+an+application+of+contextual+integrity%2E+Social+Media+%2B+Society%2C+4%282%29%2E)
63. Maarten Sap, Marcella Cindy Prasetio, Ariel Holtzman, Hannah Rashkin, and Yejin Choi. 2017. Connotation frames of power and agency in modern films. In EMNLP. [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Marcella+Cindy+Prasetio%2C+Ariel+Holtzman%2C+Hannah+Rashkin%2C+and+Yejin+Choi%2E+2017%2E+Connotation+frames+of+power+and+agency+in+modern+films%2E+In+EMNLP%2E)
64. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019c. Social IQa: commonsense reasoning about social interactions. In EMNLP. [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Hannah+Rashkin%2C+Derek+Chen%2C+Ronan+LeBras%2C+and+Yejin+Choi%2E+2019c%2E+Social+IQa%3A+commonsense+reasoning+about+social+interactions%2E+In+EMNLP%2E)
65. Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In Workshop on NLP for Social Media at EACL. [our card](https://md.research.elamur.ai/free/en/papers/schmidt-2017-survey-hate-speech-detection-natural/)
66. Robyn Speer and Catherine Havasi. 2012. Representing general relational knowledge in ConceptNet 5. In LREC. [find](https://scholar.google.com/scholar?q=Robyn+Speer+and+Catherine+Havasi%2E+2012%2E+Representing+general+relational+knowledge+in+ConceptNet+5%2E+In+LREC%2E)
67. Whitney Strub. 2008. The clearly obscene and the queerly obscene: heteronormativity and obscenity in cold war los angeles. American Quarterly, 60(2):373–398. [find](https://scholar.google.com/scholar?q=Whitney+Strub%2E+2008%2E+The+clearly+obscene+and+the+queerly+obscene%3A+heteronormativity+and+obscenity+in+cold+war+los+angeles%2E+American+Quarterly%2C+60%282%29%3A373%E2%80%93398%2E)
68. Stefanie Ullmann and Marcus Tomalin. 2019. Quaran-tining online hate speech: technical and ethical perspectives. Ethics and Information Technology. [find](https://scholar.google.com/scholar?q=Stefanie+Ullmann+and+Marcus+Tomalin%2E+2019%2E+Quaran%2Dtining+online+hate+speech%3A+technical+and+ethical+perspectives%2E+Ethics+and+Information+Technology%2E)
69. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. [our card](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
70. James Vincent. 2016. Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day. [https://www.theverge.com/2016/3/](https://www.theverge.com/2016/3/) 24/11297050/tay-microsoft-chatbot-racist. Accessed: 2019-10-26. [↗](https://www.theverge.com/2016/3/)
71. Andrew J Vonasch and Roy F Baumeister. 2017. Unjustified side effects were strongly intended: taboo tradeoffs and the side-effect effect. Journal of Experimental Social Psychology, 68:83–92. [find](https://scholar.google.com/scholar?q=Andrew+J+Vonasch+and+Roy+F+Baumeister%2E+2017%2E+Unjustified+side+effects+were+strongly+intended%3A+taboo+tradeoffs+and+the+side%2Deffect+effect%2E+Journal+of+Experimental+Social+Psychology%2C+68%3A83%E2%80%9392%2E)
72. Zijian Wang and Christopher Potts. 2019. TalkDown: a corpus for condescension detection in context. In EMNLP. [find](https://scholar.google.com/scholar?q=Zijian+Wang+and+Christopher+Potts%2E+2019%2E+TalkDown%3A+a+corpus+for+condescension+detection+in+context%2E+In+EMNLP%2E)
73. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter. In NAACL Student Research Workshop. [our card](https://md.research.elamur.ai/free/en/papers/waseem-2016-hateful-symbols-or-hateful-people/)
74. Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: personal attacks seen at scale. In WWW. model offensive 42.2% pos. (dev.) F 1 pr. rec. intent 44.8% pos. (dev.) F 1 pr. rec. lewd 3.0% pos. (dev.) F 1 pr. rec. group 66.6% pos. (dev.) F 1 pr. rec. in-group 5.1% pos. (dev.) F 1 pr. rec.… [find](https://scholar.google.com/scholar?q=Ellery+Wulczyn%2C+Nithum+Thain%2C+and+Lucas+Dixon%2E+2017%2E+Ex+machina%3A+personal+attacks+seen+at+scale%2E+In+WWW%2E+model+offensive+42%2E2%25+pos%2E+%28dev%2E%29+F+1+pr%2E+rec%2E+intent+44%2E8%25+pos%2E+%28dev%2E%29+F+1+pr%2E+rec%2E+lewd+3%2E0%25+)
75. SBF-GPT 1 -gdy 00 -constr SBF-GPT 2 -gdy 00 -constr SBF-GPT 2 -smp 00 -constr 75.2 88.3 75.2 88.3 77.2 88.3 77.2 88.3 80.5 84.3 80.4 84.3 74.4 89.8 74.4 89.8 76.3 89.5 76.3 89.5 75.3 89.9 75.3 89.9 63.6 63.6 66.5 66.5 64.7 64.7 75.2 78.2 72.5 75.2 78.2 72.5 77.6 81.2 74.3 77.6 81.2 74.3 78.6 80.6 76… [find](https://scholar.google.com/scholar?q=SBF%2DGPT+1+%2Dgdy+00+%2Dconstr+SBF%2DGPT+2+%2Dgdy+00+%2Dconstr+SBF%2DGPT+2+%2Dsmp+00+%2Dconstr+75%2E2+88%2E3+75%2E2+88%2E3+77%2E2+88%2E3+77%2E2+88%2E3+80%2E5+84%2E3+80%2E4+84%2E3+74%2E4+89%2E8+74%2E4+89%2E8+76%2E3+89%2E5+76%2E3+89%2E5+75%2E3+89%2E9+75%2E3+89%2E9+63)