---
title: "QueerGen — paper by Sosto et al. (2026) — summary and ideas"
description: "The authors build an English template dataset for sentence completion and give these prompts to Masked Language Models… 58 references."
authors: ["Mae Sosto", "Delfina S. Martinez Pandiani", "Laura Hollink"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: EACL 2026"
source: "https://aclanthology.org/2026.findings-eacl.225/"
doi: "10.18653/v1/2026.findings-eacl.225"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sosto-2026-queergen-how-llms-reflect-societal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sosto-2026-queergen-how-llms-reflect-societal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/40/) / QueerGen: How LLMs Reflect Societal Norms on Gender and Sexuality in Sentence Completion Task

# QueerGen: How LLMs Reflect Societal Norms on Gender and Sexuality in Sentence Completion Task

**Authors:** Mae Sosto, Delfina S. Martinez Pandiani and [Laura Hollink](https://md.research.elamur.ai/free/en/authors/laura-hollink/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.225/) · [PDF](https://aclanthology.org/2026.findings-eacl.225.pdf) · [DOI 10.18653/v1/2026.findings-eacl.225](https://doi.org/10.18653/v1/2026.findings-eacl.225) · [OpenAlex](https://openalex.org/W7140115195)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Most work on gender bias in large language models treats gender as binary and assumes that non-queer identities are the neutral baseline. QueerGen addresses this with a three-way comparison: subjects without any identity term ("unmarked"), subjects with non-queer markers, and subjects with queer markers. The authors build an English template dataset for sentence completion and give these prompts to Masked Language Models (MLMs) and to open- and closed-access autoregressive models (ARLMs). They score the completions on four dimensions: sentiment, regard, toxicity and diversity of predictions.

The results differ by model family:
- **MLMs** are consistently less favourable to queer-marked subjects.
- **Open-access ARLMs** reduce this gap only in part.
- **Closed-access ARLMs**, which are likely shaped by alignment such as RLHF, shift the harms rather than remove them. In some cases unmarked subjects receive the more toxic outputs.

The study shows that heterocisnormative defaults appear in model behaviour both when identity markers are present and when they are absent. It also offers an extensible, openly released framework for auditing representational harms.

## Key points
- The dataset combines 10 unmarked subjects (e.g., cousin, neighbor, employee), 30 markers and 10 templates. The markers are 10 non-queer terms and 20 queer terms, split into 10 gender identities and 10 sexual/romantic orientations. This yields 3,100 prompts: 100 unmarked and 3,000 marked.
- Bias is measured as differences between identity groups in sentiment, regard, toxicity and prediction diversity. Identity markers are excluded from the text before scoring.
- In MLMs, queer-marked subjects receive lower sentiment, more toxicity and more negative regard. In closed-access ARLMs, unmarked subjects get more diverse completions, while marked identities get more constrained ones.
- Judging only top-1 predictions can overstate bias, because less biased alternatives often appear among the top-5.
- The study covers English only and does not address intersectional factors. The code, templates and data are available at github.com/MaeSosto/QueerGen.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Amsterdam](https://md.research.elamur.ai/free/en/universities/university-of-amsterdam/) — University, Amsterdam, Netherlands · 144 works · 173,906 citations
- [Centrum Wiskunde & Informatica](https://md.research.elamur.ai/free/en/labs/centrum-wiskunde-informatica/) — Research institute, Amsterdam, Netherlands · 3 works · 5,881 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works · [💭 Social Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/social-psychology/) · 306 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Mae Sosto, Delfina S. Martinez Pandiani and Laura Hollink. 2026. QueerGen: How LLMs Reflect Societal Norms on Gender and Sexuality in Sentence Completion Task. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sosto2026queergen,
  title = {{QueerGen: How LLMs Reflect Societal Norms on Gender and Sexuality in Sentence Completion Task}},
  author = {Mae Sosto, Delfina S. Martinez Pandiani and Laura Hollink},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.225},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.225}
}
```

- [← previous Que ao mestre vai matá-lo? The evolution of prepositional accusatives in Portuguese across time](https://md.research.elamur.ai/free/en/papers/vaz-2026-que-ao-mestre-vai-mata/)
- [next → Query-Focused Individual Simulation with Progressive Persona Completion](https://md.research.elamur.ai/free/en/papers/su-2026-query-focused-individual-simulation-progressive/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/40/)

## References

58 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Robert Adragna, Elliot Creager, David Madras, and Richard Zemel. 2020. Fairness and robustness in invariant learning: A case study in toxicity classification. arXiv preprint arXiv:2011.06485. [find](https://scholar.google.com/scholar?q=Robert+Adragna%2C+Elliot+Creager%2C+David+Madras%2C+and+Richard+Zemel%2E+2020%2E+Fairness+and+robustness+in+invariant+learning%3A+A+case+study+in+toxicity+classification%2E+arXiv+preprint+arXiv%3A2011%2E06485%2E)
3. Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. [Flair: An easy-to-use framework for state-of-the-art nlp](https://md.research.elamur.ai/free/en/papers/akbik-2019-flair-easy-use-framework-state/). In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics (demonstrations), pages…
4. Mohammed Al-Shabi. 2020. Evaluating the performance of the most important lexicons used to sentiment analysis and opinions mining. IJCSNS, 20(1):1. [find](https://scholar.google.com/scholar?q=Mohammed+Al%2DShabi%2E+2020%2E+Evaluating+the+performance+of+the+most+important+lexicons+used+to+sentiment+analysis+and+opinions+mining%2E+IJCSNS%2C+20%281%29%3A1%2E)
5. Soumya Barikeri, Anne Lauscher, Ivan Vulić, and Goran Glavaš. 2021. Redditbias: A real-world resource for bias evaluation and debiasing of conversational language models. arXiv preprint arXiv:2106.03521. [find](https://scholar.google.com/scholar?q=Soumya+Barikeri%2C+Anne+Lauscher%2C+Ivan+Vulic%CC%81%2C+and+Goran+Glava%C5%A1%2E+2021%2E+Redditbias%3A+A+real%2Dworld+resource+for+bias+evaluation+and+debiasing+of+conversational+language+models%2E+arXiv+preprint+arXiv%3A2106%2E03)
6. Marion Bartl, Abhishek Mandal, Susan Leavy, and Suzanne Little. 2025. Gender bias in natural language processing and computer vision: A comparative survey. ACM Computing Surveys, 57(6):1–36. [find](https://scholar.google.com/scholar?q=Marion+Bartl%2C+Abhishek+Mandal%2C+Susan+Leavy%2C+and+Suzanne+Little%2E+2025%2E+Gender+bias+in+natural+language+processing+and+computer+vision%3A+A+comparative+survey%2E+ACM+Computing+Surveys%2C+57%286%29%3A1%E2%80%9336%2E)
7. Elisa Bassignana, Valerio Basile, Viviana Patti, and 1 others. 2018. [Hurtlex: A multilingual lexicon of words to hurt](https://md.research.elamur.ai/free/en/papers/bassignana-2018-hurtlex-multilingual-lexicon-words-hurt/). In CEUR Workshop proceedings, volume 2253, pages 1–6. CEUR-WS.
8. Edwin L Battistella. 1996. The logic of markedness. Oxford University Press. [find](https://scholar.google.com/scholar?q=Edwin+L+Battistella%2E+1996%2E+The+logic+of+markedness%2E+Oxford+University+Press%2E)
9. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623. [find](https://scholar.google.com/scholar?q=Emily+M+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+2021%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+conference+)
10. Selma Bergstrand and Björn Gambäck. 2024. Detecting and mitigating lgbtqia+ bias in large norwegian language models. In Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 351–364. [find](https://scholar.google.com/scholar?q=Selma+Bergstrand+and+Bj%C3%B6rn+Gamb%C3%A4ck%2E+2024%2E+Detecting+and+mitigating+lgbtqia%2B+bias+in+large+norwegian+language+models%2E+In+Proceedings+of+the+5th+Workshop+on+Gender+Bias+in+Natural+Language+Processing+%28G)
11. Judith Butler. 2020. Critically queer. In Playing with fire, pages 11–29. Routledge. [find](https://scholar.google.com/scholar?q=Judith+Butler%2E+2020%2E+Critically+queer%2E+In+Playing+with+fire%2C+pages+11%E2%80%9329%2E+Routledge%2E)
12. Bharathi Raja Chakravarthi, Ruba Priyadharshini, Rahul Ponnusamy, Prasanna Kumar Kumaresan, Kayalvizhi Sampath, Durairaj Thenmozhi, Sathiyaraj Thangasamy, Rajendran Nallathambi, and John Phillip McCrae. 2021. Dataset for identification of homophobia and transophobia in multilingual youtube comments.… — [arXiv:2109.00227](https://arxiv.org/abs/2109.00227) [find](https://scholar.google.com/scholar?q=Bharathi+Raja+Chakravarthi%2C+Ruba+Priyadharshini%2C+Rahul+Ponnusamy%2C+Prasanna+Kumar+Kumaresan%2C+Kayalvizhi+Sampath%2C+Durairaj+Thenmozhi%2C+Sathiyaraj+Thangasamy%2C+Rajendran+Nallathambi%2C+and+John+Phillip+McCra)
13. Marta R Costa-jussà, Christian Hardmeier, Will Radford, and Kellie Webster. 2020. Proceedings of the second workshop on gender bias in natural language processing. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Marta+R+Costa%2Djuss%C3%A0%2C+Christian+Hardmeier%2C+Will+Radford%2C+and+Kellie+Webster%2E+2020%2E+Proceedings+of+the+second+workshop+on+gender+bias+in+natural+language+processing%2E+In+Proceedings+of+the+Second+Worksho)
14. Jamell Dacon, Harry Shomer, Shaylynn Crum-Dacon, and Jiliang Tang. 2022. Detecting harmful online conversational content towards lgbtqia+ individuals. arXiv preprint arXiv:2207.10032. — [arXiv:2207.10032](https://arxiv.org/abs/2207.10032) [find](https://scholar.google.com/scholar?q=Jamell+Dacon%2C+Harry+Shomer%2C+Shaylynn+Crum%2DDacon%2C+and+Jiliang+Tang%2E+2022%2E+Detecting+harmful+online+conversational+content+towards+lgbtqia%2B+individuals%2E+arXiv+preprint+arXiv%3A2207%2E10032%2E+%E2%80%94+%5BarXiv%3A2207%2E10)
15. Hannah Devinney, Jenny Björklund, and Henrik Björklund. 2022. Theories of “gender” in nlp bias research. In Proceedings of the 2022 ACM conference on fairness, accountability, and transparency, pages 2083– 2102. [find](https://scholar.google.com/scholar?q=Hannah+Devinney%2C+Jenny+Bj%C3%B6rklund%2C+and+Henrik+Bj%C3%B6rklund%2E+2022%2E+Theories+of+%E2%80%9Cgender%E2%80%9D+in+nlp+bias+research%2E+In+Proceedings+of+the+2022+ACM+conference+on+fairness%2C+accountability%2C+and+transparency%2C+pages+)
16. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
17. Harnoor Dhingra, Preetiha Jayashanker, Sayali Moghe, and Emma Strubell. 2023. Queer people are people first: Deconstructing sexual identity stereotypes in large language models. arXiv preprint arXiv:2307.00101. — [arXiv:2307.00101](https://arxiv.org/abs/2307.00101) [find](https://scholar.google.com/scholar?q=Harnoor+Dhingra%2C+Preetiha+Jayashanker%2C+Sayali+Moghe%2C+and+Emma+Strubell%2E+2023%2E+Queer+people+are+people+first%3A+Deconstructing+sexual+identity+stereotypes+in+large+language+models%2E+arXiv+preprint+arXiv%3A2)
18. Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. arXiv preprint arXiv:2104.08758. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Maarten+Sap%2C+Ana+Marasovic%CC%81%2C+William+Agnew%2C+Gabriel+Ilharco%2C+Dirk+Groeneveld%2C+Margaret+Mitchell%2C+and+Matt+Gardner%2E+2021%2E+Documenting+large+webtext+corpora%3A+A+case+study+on+the+colossal+cl)
19. Xiangjue Dong, Yibo Wang, Philip S Yu, and James Caverlee. 2024. Disclosure and mitigation of gender bias in llms. arXiv preprint arXiv:2402.11190. — [arXiv:2402.11190](https://arxiv.org/abs/2402.11190) [find](https://scholar.google.com/scholar?q=Xiangjue+Dong%2C+Yibo+Wang%2C+Philip+S+Yu%2C+and+James+Caverlee%2E+2024%2E+Disclosure+and+mitigation+of+gender+bias+in+llms%2E+arXiv+preprint+arXiv%3A2402%2E11190%2E+%E2%80%94+%5BarXiv%3A2402%2E11190%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E1119)
20. Satyam Dwivedi, Sanjukta Ghosh, and Shivam Dwivedi. 2023. Breaking the bias: Gender fairness in llms using prompt engineering and in-context learning. Rupkatha Journal on Interdisciplinary Studies in Humanities, 15(4). [find](https://scholar.google.com/scholar?q=Satyam+Dwivedi%2C+Sanjukta+Ghosh%2C+and+Shivam+Dwivedi%2E+2023%2E+Breaking+the+bias%3A+Gender+fairness+in+llms+using+prompt+engineering+and+in%2Dcontext+learning%2E+Rupkatha+Journal+on+Interdisciplinary+Studies+in+)
21. Virginia K Felkner, Ho-Chun Herbert Chang, Eugene Jang, and Jonathan May. 2023. Winoqueer: A community-in-the-loop benchmark for anti-lgbtq+ bias in large language models. arXiv preprint arXiv:2306.15087. [find](https://scholar.google.com/scholar?q=Virginia+K+Felkner%2C+Ho%2DChun+Herbert+Chang%2C+Eugene+Jang%2C+and+Jonathan+May%2E+2023%2E+Winoqueer%3A+A+community%2Din%2Dthe%2Dloop+benchmark+for+anti%2Dlgbtq%2B+bias+in+large+language+models%2E+arXiv+preprint+arXiv%3A2306%2E15)
22. Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. [Bias and fairness in large language models: A survey](https://md.research.elamur.ai/free/en/papers/gallegos-2023-bias-fairness-large-language-models/). Computational Linguistics, 50(3):1097– 1179.
23. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, and 1 others. 2022. [Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned](https://md.research.elamur.ai/free/en/papers/ganguli-2022-red-teaming-language-models-reduce/). arXiv preprint arXiv:2209.07858.
24. Tarleton Gillespie. 2024. Generative ai and the politics of visibility. Big Data & Society, 11(2):20539517241252131. [find](https://scholar.google.com/scholar?q=Tarleton+Gillespie%2E+2024%2E+Generative+ai+and+the+politics+of+visibility%2E+Big+Data+%26+Society%2C+11%282%29%3A20539517241252131%2E)
25. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
26. Laura Hanu and Unitary team. 2020. Detoxify. Github. [https://github.com/unitaryai/detoxify](https://github.com/unitaryai/detoxify). [↗](https://github.com/unitaryai/detoxify)
27. Donna Haraway. 2013. Situated knowledges: The science question in feminism and the privilege of partial perspective 1. In Women, science, and technology, pages 455–472. Routledge. [find](https://scholar.google.com/scholar?q=Donna+Haraway%2E+2013%2E+Situated+knowledges%3A+The+science+question+in+feminism+and+the+privilege+of+partial+perspective+1%2E+In+Women%2C+science%2C+and+technology%2C+pages+455%E2%80%93472%2E+Routledge%2E)
28. Saad Hassan, Matt Huenerfauth, and Cecilia Ovesdotter Alm. 2021. Unpacking the interdependent systems of discrimination: Ableist bias in nlp systems through an intersectional lens. arXiv preprint arXiv:2110.00521. [find](https://scholar.google.com/scholar?q=Saad+Hassan%2C+Matt+Huenerfauth%2C+and+Cecilia+Ovesdotter+Alm%2E+2021%2E+Unpacking+the+interdependent+systems+of+discrimination%3A+Ableist+bias+in+nlp+systems+through+an+intersectional+lens%2E+arXiv+preprint+arXi)
29. J Henry Hinnefeld, Peter Cooman, Nat Mammo, and Rupert Deese. 2018. Evaluating fairness metrics in the presence of dataset bias. arXiv preprint arXiv:1809.09245. — [arXiv:1809.09245](https://arxiv.org/abs/1809.09245) [find](https://scholar.google.com/scholar?q=J+Henry+Hinnefeld%2C+Peter+Cooman%2C+Nat+Mammo%2C+and+Rupert+Deese%2E+2018%2E+Evaluating+fairness+metrics+in+the+presence+of+dataset+bias%2E+arXiv+preprint+arXiv%3A1809%2E09245%2E+%E2%80%94+%5BarXiv%3A1809%2E09245%5D%28https%3A%2F%2Farxiv%2Eorg)
30. Tamanna Hossain, Sunipa Dev, and Sameer Singh. 2023. Misgendered: Limits of large language models in understanding pronouns. arXiv preprint arXiv:2306.03950. [find](https://scholar.google.com/scholar?q=Tamanna+Hossain%2C+Sunipa+Dev%2C+and+Sameer+Singh%2E+2023%2E+Misgendered%3A+Limits+of+large+language+models+in+understanding+pronouns%2E+arXiv+preprint+arXiv%3A2306%2E03950%2E)
31. Hossein Hosseini, Sreeram Kannan, Baosen Zhang, and Radha Poovendran. 2017. Deceiving google’s perspective api built for detecting toxic comments. arXiv preprint arXiv:1702.08138. — [arXiv:1702.08138](https://arxiv.org/abs/1702.08138) [find](https://scholar.google.com/scholar?q=Hossein+Hosseini%2C+Sreeram+Kannan%2C+Baosen+Zhang%2C+and+Radha+Poovendran%2E+2017%2E+Deceiving+google%E2%80%99s+perspective+api+built+for+detecting+toxic+comments%2E+arXiv+preprint+arXiv%3A1702%2E08138%2E+%E2%80%94+%5BarXiv%3A1702%2E08138%5D)
32. Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack Rae, Vishal Maini, Dani Yogatama, and Pushmeet Kohli. 2019. Reducing sentiment bias in language models via counterfactual evaluation. arXiv preprint arXiv:1911.03064. [find](https://scholar.google.com/scholar?q=Po%2DSen+Huang%2C+Huan+Zhang%2C+Ray+Jiang%2C+Robert+Stanforth%2C+Johannes+Welbl%2C+Jack+Rae%2C+Vishal+Maini%2C+Dani+Yogatama%2C+and+Pushmeet+Kohli%2E+2019%2E+Reducing+sentiment+bias+in+language+models+via+counterfactual+ev)
33. Clayton Hutto and Eric Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the international AAAI conference on web and social media, volume 8, pages 216–225. [find](https://scholar.google.com/scholar?q=Clayton+Hutto+and+Eric+Gilbert%2E+2014%2E+Vader%3A+A+parsimonious+rule%2Dbased+model+for+sentiment+analysis+of+social+media+text%2E+In+Proceedings+of+the+international+AAAI+conference+on+web+and+social+media%2C+v)
34. Urban Knupleš, Agnieszka Falenska, and Filip Miletić. 2024. Gender identity in pretrained language models: An inclusive approach to data creation and probing. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11612–11631. [find](https://scholar.google.com/scholar?q=Urban+Knuple%C5%A1%2C+Agnieszka+Falenska%2C+and+Filip+Miletic%CC%81%2E+2024%2E+Gender+identity+in+pretrained+language+models%3A+An+inclusive+approach+to+data+creation+and+probing%2E+In+Findings+of+the+Association+for+Compu)
35. Anne Lauscher, Archie Crowley, and Dirk Hovy. 2022. Welcome to the modern world of pronouns: Identity-inclusive natural language processing beyond gender. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1221– 1232, Gyeongju, Republic of Korea. International Co… [find](https://scholar.google.com/scholar?q=Anne+Lauscher%2C+Archie+Crowley%2C+and+Dirk+Hovy%2E+2022%2E+Welcome+to+the+modern+world+of+pronouns%3A+Identity%2Dinclusive+natural+language+processing+beyond+gender%2E+In+Proceedings+of+the+29th+International+Conf)
36. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692.
37. Steven Loria and 1 others. 2018. textblob documentation. Release 0.15, 2(8):269. [find](https://scholar.google.com/scholar?q=Steven+Loria+and+1+others%2E+2018%2E+textblob+documentation%2E+Release+0%2E15%2C+2%288%29%3A269%2E)
38. Elizabeth A McConnell, Antonia Clifford, Aaron K Korpak, Gregory Phillips II, and Michelle Birkett. 2017. Identity, victimization, and support: Facebook experiences and mental health among lgbtq youth. Computers in Human Behavior, 76:237–244. [find](https://scholar.google.com/scholar?q=Elizabeth+A+McConnell%2C+Antonia+Clifford%2C+Aaron+K+Korpak%2C+Gregory+Phillips+II%2C+and+Michelle+Birkett%2E+2017%2E+Identity%2C+victimization%2C+and+support%3A+Facebook+experiences+and+mental+health+among+lgbtq+youth)
39. Heiko Motschenbacher. 2011. Taking queer linguistics further: Sociolinguistics and critical heteronormativity research. International Journal of the Sociology of Language, 2011(212):149–169. [find](https://scholar.google.com/scholar?q=Heiko+Motschenbacher%2E+2011%2E+Taking+queer+linguistics+further%3A+Sociolinguistics+and+critical+heteronormativity+research%2E+International+Journal+of+the+Sociology+of+Language%2C+2011%28212%29%3A149%E2%80%93169%2E)
40. Finn Årup Nielsen. 2017. Afinn project. DTU Compute Technical University of Denmark. [find](https://scholar.google.com/scholar?q=Finn+%C3%85rup+Nielsen%2E+2017%2E+Afinn+project%2E+DTU+Compute+Technical+University+of+Denmark%2E)
41. Debora Nozza, Federico Bianchi, and Dirk Hovy. 2021. [HONEST: Measuring hurtful sentence completion in language models](https://md.research.elamur.ai/free/en/papers/nozza-2021-honest-measuring-hurtful-sentence-completion/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2398–2406, Online. Associati…
42. Debora Nozza, Federico Bianchi, Anne Lauscher, Dirk Hovy, and 1 others. 2022. Measuring harmful sentence completion in language models for lgbtqia+ individuals. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Debora+Nozza%2C+Federico+Bianchi%2C+Anne+Lauscher%2C+Dirk+Hovy%2C+and+1+others%2E+2022%2E+Measuring+harmful+sentence+completion+in+language+models+for+lgbtqia%2B+individuals%2E+In+Proceedings+of+the+Second+Workshop+o)
43. Nedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song, and Dit-Yan Yeung. 2021. Probing toxic content in large pre-trained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language… [find](https://scholar.google.com/scholar?q=Nedjma+Ousidhoum%2C+Xinran+Zhao%2C+Tianqing+Fang%2C+Yangqiu+Song%2C+and+Dit%2DYan+Yeung%2E+2021%2E+Probing+toxic+content+in+large+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Associ)
44. Anaelia Ovalle, Palash Goyal, Jwala Dhamala, Zachary Jaggers, Kai-Wei Chang, Aram Galstyan, Richard Zemel, and Rahul Gupta. 2023. “i’m fully who i am”: Towards centering transgender and non-binary voices to measure biases in open language generation. In Proceedings of the 2023 ACM Conference on Fair… [find](https://scholar.google.com/scholar?q=Anaelia+Ovalle%2C+Palash+Goyal%2C+Jwala+Dhamala%2C+Zachary+Jaggers%2C+Kai%2DWei+Chang%2C+Aram+Galstyan%2C+Richard+Zemel%2C+and+Rahul+Gupta%2E+2023%2E+%E2%80%9Ci%E2%80%99m+fully+who+i+am%E2%80%9D%3A+Towards+centering+transgender+and+non%2Dbinary+voi)
45. Sarah O’Hara. 2015. Heterosexism in secondary schools: The case of ireland. Sex Education, 15(2):163–179. [find](https://scholar.google.com/scholar?q=Sarah+O%E2%80%99Hara%2E+2015%2E+Heterosexism+in+secondary+schools%3A+The+case+of+ireland%2E+Sex+Education%2C+15%282%29%3A163%E2%80%93179%2E)
46. Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. Ai and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366. [find](https://scholar.google.com/scholar?q=Inioluwa+Deborah+Raji%2C+Emily+M+Bender%2C+Amandalynne+Paullada%2C+Emily+Denton%2C+and+Alex+Hanna%2E+2021%2E+Ai+and+the+everything+in+the+whole+wide+world+benchmark%2E+arXiv+preprint+arXiv%3A2111%2E15366%2E)
47. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. [The woman worked as a babysitter: On biases in language generation](https://md.research.elamur.ai/free/en/papers/sheng-2019-woman-worked-as-babysitter-biases/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process…
48. Mae Sosto and Alberto Barrón-Cedeño. 2024. Queer-bench: Quantifying discrimination in language models toward queer identities. arXiv preprint arXiv:2406.12399. — [arXiv:2406.12399](https://arxiv.org/abs/2406.12399) [find](https://scholar.google.com/scholar?q=Mae+Sosto+and+Alberto+Barr%C3%B3n%2DCede%C3%B1o%2E+2024%2E+Queer%2Dbench%3A+Quantifying+discrimination+in+language+models+toward+queer+identities%2E+arXiv+preprint+arXiv%3A2406%2E12399%2E+%E2%80%94+%5BarXiv%3A2406%2E12399%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
49. Michelle F Wright and Sebastian Wachs. 2021. Does empathy and toxic online disinhibition moderate the longitudinal association between witnessing and perpetrating homophobic cyberbullying? International journal of bullying prevention, 3:66–74. [find](https://scholar.google.com/scholar?q=Michelle+F+Wright+and+Sebastian+Wachs%2E+2021%2E+Does+empathy+and+toxic+online+disinhibition+moderate+the+longitudinal+association+between+witnessing+and+perpetrating+homophobic+cyberbullying%3F+Internation)
50. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. [Men also like shopping: Reducing gender bias amplification using corpus-level constraints](https://md.research.elamur.ai/free/en/papers/zhao-2017-men-also-like-shopping-reducing/). arXiv preprint arXiv:1707.09457.
51. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019a. [Mitigating gender bias in natural language processing: Literature review](https://md.research.elamur.ai/free/en/papers/sun-2019-mitigating-gender-bias-natural-language/). In Proceedings of the 57th Annual Meeting of the Association for Computat…
52. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019b. [Mitigating gender bias in natural language processing: Literature review](https://md.research.elamur.ai/free/en/papers/sun-2019-mitigating-gender-bias-natural-language/). arXiv preprint arXiv:1906.08976.
53. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805.
54. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, and 1 others. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Thomas+Mesnard%2C+Cassidy+Hardin%2C+Robert+Dadashi%2C+Surya+Bhupatiraju%2C+Shreya+Pathak%2C+Laurent+Sifre%2C+Morgane+Rivi%C3%A8re%2C+Mihir+Sanjay+Kale%2C+Juliette+Love%2C+and+1+others%2E+2024%2E+Gemma%3A+Open+models+b)
55. Eddie L Ungless, Sunipa Dev, Cynthia L Bennett, Rebecca Gulotta, Jasmijn Bastings, and Remi Denton. 2025. Amplifying trans and nonbinary voices: A community-centred harm taxonomy for llms. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape… [find](https://scholar.google.com/scholar?q=Eddie+L+Ungless%2C+Sunipa+Dev%2C+Cynthia+L+Bennett%2C+Rebecca+Gulotta%2C+Jasmijn+Bastings%2C+and+Remi+Denton%2E+2025%2E+Amplifying+trans+and+nonbinary+voices%3A+A+community%2Dcentred+harm+taxonomy+for+llms%2E+In+Proceedi)
56. Juan Vásquez, Scott Andersen, Gemma Bel-Enguix, Helena Gómez-Adorno, and Sergio-Luis Ojeda-Trueba. 2023. Homo-mex: A mexican spanish annotated corpus for lgbt+ phobia detection on twitter. In The 7th Workshop on Online Abuse and Harms (WOAH), pages 202–214. [find](https://scholar.google.com/scholar?q=Juan+V%C3%A1squez%2C+Scott+Andersen%2C+Gemma+Bel%2DEnguix%2C+Helena+G%C3%B3mez%2DAdorno%2C+and+Sergio%2DLuis+Ojeda%2DTrueba%2E+2023%2E+Homo%2Dmex%3A+A+mexican+spanish+annotated+corpus+for+lgbt%2B+phobia+detection+on+twitter%2E+In+The+7th+)
57. Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. 2021. [Challenges in detoxifying language models](https://md.research.elamur.ai/free/en/papers/welbl-2021-challenges-detoxifying-language-models/). arXiv preprint arXiv:2109.07445.
58. Model BERT Large Subject Category Unmarked Non Queer Queer DeepSeek R1 Unmarked Non Queer Queer GPT-4o Unmarked Non Queer Queer Input sentence [find](https://scholar.google.com/scholar?q=Model+BERT+Large+Subject+Category+Unmarked+Non+Queer+Queer+DeepSeek+R1+Unmarked+Non+Queer+Queer+GPT%2D4o+Unmarked+Non+Queer+Queer+Input+sentence)

*Updated 1 November 2026*