---
title: "Reflexive Research with LLMs — paper by Smith et al. (2026)"
description: "The authors argue that bias enters from two sides. 66 references. Among them: 0."
authors: ["Eleanor L.T. Smith", "Luis Morgado da Costa", "Antske Fokkens"]
year: 2026
venue: "Proceedings of Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities (LLMs4SSH) @ LREC 2026"
source: "https://aclanthology.org/2026.llms4ssh-1.19/"
doi: "10.63317/3ycn8grfuwtx"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/smith-2026-reflexive-research-llms-considering-positionality/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/smith-2026-reflexive-research-llms-considering-positionality/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/41/) / Reflexive Research with LLMs: Considering the Positionality of Users and Systems

# Reflexive Research with LLMs: Considering the Positionality of Users and Systems

**Authors:** Eleanor L.T. Smith, [Luis Morgado da Costa](https://md.research.elamur.ai/free/en/authors/luis-morgado-da-costa/) and [Antske Fokkens](https://md.research.elamur.ai/free/en/authors/antske-fokkens/) · **Year:** 2026 · **Venue:** [Proceedings of Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities (LLMs4SSH) @ LREC 2026](https://md.research.elamur.ai/free/en/venues/llms4ssh/) ([Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities @ LREC 2026](https://md.research.elamur.ai/free/en/venues/llms4ssh/2026/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.llms4ssh-1.19/) · [PDF](https://aclanthology.org/2026.llms4ssh-1.19.pdf) · [DOI 10.63317/3ycn8grfuwtx](https://doi.org/10.63317/3ycn8grfuwtx) · [OpenAlex](https://openalex.org/W7166554466)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
This position paper addresses a methodological gap in digital social science and humanities (DSSH): researchers increasingly use LLMs while often treating them as neutral instruments. The authors argue that bias enters from two sides. The first is the researcher, whose interaction with a machine-like or anthropomorphized chat interface can trigger unwarranted trust. The second is the system, whose behaviour reflects its training data and design choices. Drawing on literature from HCI, media studies and NLP, they propose a conceptual model of the factors shaping LLM use in research. The model has three levels: user, system and external (how LLMs are presented). They review evidence on mechanical objectivity, the machine heuristic, shifting mental models, and hype in academic and news discourse. They conclude that established SSH practices should be applied to both researchers and the tools they use. These practices are positionality statements, source criticism and tool criticism. Combined with transparent, case-specific evaluation, they offer a practical lever for more methodologically sound LLM-based research.

## Key points
- The model (Figure 1) separates three kinds of factors:
  - **User-level factors:** beliefs about capabilities, mental model, and behaviour during interaction.
  - **System-level factors:** inherent attributes, bias learned from data, and structural bias from design.
  - **External factors:** commercial, scientific and journalistic presentation, plus the values of the technical community.
- In the model, upward arrows show that changing user-level factors can influence the wider system of interactions.
- The paper cites Schneider (2025), who analysed 200,000 human–LLM interactions. Users began adopting human-style conversational behaviour from their second turn onward.
- In the study by Cabrero-Daniel and Sanagustín Cabrero, about half of the participants considered gender bias intrinsic to text generation rather than an error. This suggests a tendency not to classify such bias as a system fault.
- Other cited evidence includes the following:
  - Dutch survey respondents rated automated decisions as equal to or better than expert judgments (Araujo et al., 2019).
  - Williams and Huckle's (2024) benchmark shows LLMs failing on tasks that are easy for humans.
  - The GPT-4 technical report acknowledges the model's capacity for confident errors.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Vrije Universiteit Amsterdam](https://md.research.elamur.ai/free/en/universities/vrije-universiteit-amsterdam/) — University, Amsterdam, Netherlands · 42 works · 3,372 citations

📚 Shelves (6): [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 227 works · [📚 Literature and Literary Theory](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/literature-and-literary-theory/) · 519 works

## Cite

Cite as: Eleanor L.T. Smith, Luis Morgado da Costa and Antske Fokkens. 2026. Reflexive Research with LLMs: Considering the Positionality of Users and Systems. Proceedings of Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities (LLMs4SSH) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{smith2026reflexive,
  title = {{Reflexive Research with LLMs: Considering the Positionality of Users and Systems}},
  author = {Eleanor L.T. Smith, Luis Morgado {da Costa} and Antske Fokkens},
  booktitle = {Proceedings of Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities (LLMs4SSH) @ LREC 2026},
  year = {2026},
  doi = {10.63317/3ycn8grfuwtx},
  url = {https://doi.org/10.63317/3ycn8grfuwtx}
}
```

- [← previous ReflectRM: Boosting Generative Reward Models via Self-Reflection within a Unified Judgment Framework](https://md.research.elamur.ai/free/en/papers/qin-2026-reflectrm-boosting-generative-reward-models/)
- [next → ReFreeKV: Towards Threshold-Free KV Cache Compression](https://md.research.elamur.ai/free/en/papers/ni-2026-refreekv-threshold-free-kv-cache/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/41/)

## References

66 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abubakar Abid, Maheen Farooqi, and James Zou. 2021. [Persistent anti-muslim bias in large language models](https://md.research.elamur.ai/free/en/papers/abid-2021-persistent-anti-muslim-bias-large/). In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 298–306.
2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
3. Veronika Alexander, Collin Blinder, and Paul J. Zak. 2018. Why trust an algorithm? performance, cognition, and neurophysiology. Computers in Human Behavior, 89:279–288. [find](https://scholar.google.com/scholar?q=Veronika+Alexander%2C+Collin+Blinder%2C+and+Paul+J%2E+Zak%2E+2018%2E+Why+trust+an+algorithm%3F+performance%2C+cognition%2C+and+neurophysiology%2E+Computers+in+Human+Behavior%2C+89%3A279%E2%80%93288%2E)
4. Theo Araujo, Natali Helberger, Sanne Kruikemeier, and Claes H. de Vreese1. 2019. In ai we trust? perceptions about automated decision-making by artificial intelligence. AI and Society. [find](https://scholar.google.com/scholar?q=Theo+Araujo%2C+Natali+Helberger%2C+Sanne+Kruikemeier%2C+and+Claes+H%2E+de+Vreese1%2E+2019%2E+In+ai+we+trust%3F+perceptions+about+automated+decision%2Dmaking+by+artificial+intelligence%2E+AI+and+Society%2E)
5. Arnav Arora, Lucie-aimée Kaffee, and Isabelle Augenstein. 2023. [Probing pre-trained language models for cross-cultural differences in values](https://md.research.elamur.ai/free/en/papers/arora-2023-probing-pre-trained-language-models/). In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 114–130, Dubrovnik, Croatia. Association for Computational Linguis…
6. Katherine Atwell, Mandy Simons, and Malihe Alikhani. 2025. Measuring bias and agreement in large language model presupposition judgments. In Findings of the Association for Computational Linguistics: ACL 2025, pages 2096–2107, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Katherine+Atwell%2C+Mandy+Simons%2C+and+Malihe+Alikhani%2E+2025%2E+Measuring+bias+and+agreement+in+large+language+model+presupposition+judgments%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+)
7. Charlotte Backerra. 2024. Source criticism for cultural history. Rethinking History, 28(2):194–216. [find](https://scholar.google.com/scholar?q=Charlotte+Backerra%2E+2024%2E+Source+criticism+for+cultural+history%2E+Rethinking+History%2C+28%282%29%3A194%E2%80%93216%2E)
8. Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. California Law Review. [find](https://scholar.google.com/scholar?q=Solon+Barocas+and+Andrew+D+Selbst%2E+2016%2E+Big+data%E2%80%99s+disparate+impact%2E+California+Law+Review%2E)
9. David Beer. 2017. The social power of algorithms. Information, Communication & Society, 20(1):1– 13. [find](https://scholar.google.com/scholar?q=David+Beer%2E+2017%2E+The+social+power+of+algorithms%2E+Information%2C+Communication+%26+Society%2C+20%281%29%3A1%E2%80%93+13%2E)
10. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 610–623, New York, NY, USA. Associat… [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+2021%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+Conference)
11. Emily M. Bender and Alexander Koller. 2020. [Climbing towards NLU: On meaning, form, and understanding in the age of data](https://md.research.elamur.ai/free/en/papers/bender-2020-climbing-nlu-meaning-form-understanding/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185–5198, Online. Association for Computational Linguistics.
12. Abeba Birhane, Pratyusha Kalluri, Dallas Card, William Agnew, Ravit Dotan, and Michelle Bao. 2022. The values encoded in machine learning research. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, page 173–184, New York, NY, USA. Association for Com… [find](https://scholar.google.com/scholar?q=Abeba+Birhane%2C+Pratyusha+Kalluri%2C+Dallas+Card%2C+William+Agnew%2C+Ravit+Dotan%2C+and+Michelle+Bao%2E+2022%2E+The+values+encoded+in+machine+learning+research%2E+In+Proceedings+of+the+2022+ACM+Conference+on+Fairnes)
13. Mitchell Bosley, Musashi Jacobs-Harukawa, Hauke Licht, and Alexander Hoyle. 2023. Do we still need bert in the age of gpt? comparing the benefits of domain-adaptation and in-context-learning approaches to using llms for political science research. In 2023 Annual Meeting of the Midwest Political Scie… [find](https://scholar.google.com/scholar?q=Mitchell+Bosley%2C+Musashi+Jacobs%2DHarukawa%2C+Hauke+Licht%2C+and+Alexander+Hoyle%2E+2023%2E+Do+we+still+need+bert+in+the+age+of+gpt%3F+comparing+the+benefits+of+domain%2Dadaptation+and+in%2Dcontext%2Dlearning+approache)
14. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
15. Beatriz Cabrero-Daniel and Andrea Sanagustín Cabrero. 2023. Perceived trustworthiness of natural language generators. In Proceedings of the First International Symposium on Trustworthy Autonomous Systems, TAS ’23, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Beatriz+Cabrero%2DDaniel+and+Andrea+Sanagust%C3%ADn+Cabrero%2E+2023%2E+Perceived+trustworthiness+of+natural+language+generators%2E+In+Proceedings+of+the+First+International+Symposium+on+Trustworthy+Autonomous+Syst)
16. Matt Carlson. 2019. News algorithms, photojournalism and the assumption of mechanical objectivity in journalism. Digital Journalism, 7(8):1117– 1133. [find](https://scholar.google.com/scholar?q=Matt+Carlson%2E+2019%2E+News+algorithms%2C+photojournalism+and+the+assumption+of+mechanical+objectivity+in+journalism%2E+Digital+Journalism%2C+7%288%29%3A1117%E2%80%93+1133%2E)
17. Myra Cheng, Esin Durmus, and Dan Jurafsky. 2023. [Marked personas: Using natural language prompts to measure stereotypes in language models](https://md.research.elamur.ai/free/en/papers/cheng-2023-marked-personas-natural-language-prompts/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1504–1532, Toronto, Canada. Associatio…
18. Rachel Doherty. 2024. Deified ai: The relationship between gods and artificial intelligence. Mid-Atlantic Humanities Review. [find](https://scholar.google.com/scholar?q=Rachel+Doherty%2E+2024%2E+Deified+ai%3A+The+relationship+between+gods+and+artificial+intelligence%2E+Mid%2DAtlantic+Humanities+Review%2E)
19. Esin Durmus, Karina Nguyen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. [Towards measuring the representation of subjective global opinions in language models](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/). arXiv preprint arXiv:2306.16388.
20. Eva Eigner and Thorsten Händler. 2024. Determinants of llm-assisted decision-making. arXiv preprint arXiv:2402.17385. — [arXiv:2402.17385](https://arxiv.org/abs/2402.17385) [find](https://scholar.google.com/scholar?q=Eva+Eigner+and+Thorsten+H%C3%A4ndler%2E+2024%2E+Determinants+of+llm%2Dassisted+decision%2Dmaking%2E+arXiv+preprint+arXiv%3A2402%2E17385%2E+%E2%80%94+%5BarXiv%3A2402%2E17385%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E17385%29)
21. Aaron Schecter Eric Bogert and Richard T. Watson. 2021. Humans rely more on algorithms than social influence as a task becomes more difficult. Scientific Reports. [find](https://scholar.google.com/scholar?q=Aaron+Schecter+Eric+Bogert+and+Richard+T%2E+Watson%2E+2021%2E+Humans+rely+more+on+algorithms+than+social+influence+as+a+task+becomes+more+difficult%2E+Scientific+Reports%2E)
22. Emilio Ferrara. 2023. Should chatgpt be biased? challenges and risks of bias in large language models. First Monday. [find](https://scholar.google.com/scholar?q=Emilio+Ferrara%2E+2023%2E+Should+chatgpt+be+biased%3F+challenges+and+risks+of+bias+in+large+language+models%2E+First+Monday%2E)
23. Junius Gunaratne, Lior Zalmanson, and Oded Nov. 2018. The persuasive power of algorithmic and crowdsourced advice. Journal of Management Information Systems, 35(4):1092–1120. [find](https://scholar.google.com/scholar?q=Junius+Gunaratne%2C+Lior+Zalmanson%2C+and+Oded+Nov%2E+2018%2E+The+persuasive+power+of+algorithmic+and+crowdsourced+advice%2E+Journal+of+Management+Information+Systems%2C+35%284%29%3A1092%E2%80%931120%2E)
24. Desta Haileselassie Hagos, Rick Battle, and Danda B Rawat. 2024. Recent advances in generative ai and large language models: Current status, challenges, and perspectives. IEEE transactions on artificial intelligence, 5(12):5873– 5893. [find](https://scholar.google.com/scholar?q=Desta+Haileselassie+Hagos%2C+Rick+Battle%2C+and+Danda+B+Rawat%2E+2024%2E+Recent+advances+in+generative+ai+and+large+language+models%3A+Current+status%2C+challenges%2C+and+perspectives%2E+IEEE+transactions+on+artifici)
25. Jochen Hartmann, Jasper Schwenzow, and Maximilian Witte. 2023. The political ideology of conversational ai: Converging evidence on chatgpt’s pro-environmental, left-libertarian orientation. arXiv preprint arXiv:2301.01768. [find](https://scholar.google.com/scholar?q=Jochen+Hartmann%2C+Jasper+Schwenzow%2C+and+Maximilian+Witte%2E+2023%2E+The+political+ideology+of+conversational+ai%3A+Converging+evidence+on+chatgpt%E2%80%99s+pro%2Denvironmental%2C+left%2Dlibertarian+orientation%2E+arXiv+prep)
26. Dirk Hovy and Shrimai Prabhumoye. 2021. Five sources of bias in natural language processing. Language and Linguistics Compass, 15(8):e12432. [find](https://scholar.google.com/scholar?q=Dirk+Hovy+and+Shrimai+Prabhumoye%2E+2021%2E+Five+sources+of+bias+in+natural+language+processing%2E+Language+and+Linguistics+Compass%2C+15%288%29%3Ae12432%2E)
27. Rebecca L Johnson, Giada Pistilli, Natalia Menédez-González, Leslye Denisse Dias Duran, Enrico Panai, Julija Kalpokiene, and Donald Jay Bertulfo. 2022. The ghost in the machine has an american accent: value conflict in gpt-3. arXiv preprint arXiv:2203.07785. [find](https://scholar.google.com/scholar?q=Rebecca+L+Johnson%2C+Giada+Pistilli%2C+Natalia+Men%C3%A9dez%2DGonz%C3%A1lez%2C+Leslye+Denisse+Dias+Duran%2C+Enrico+Panai%2C+Julija+Kalpokiene%2C+and+Donald+Jay+Bertulfo%2E+2022%2E+The+ghost+in+the+machine+has+an+american+accent%3A)
28. Shivani Kapania, Oliver Siy, Gabe Clapper, Azhagu Meena SP, and Nithya Sambasivan. 2022. ”because ai is 100% right and safe”: User attitudes and sources of ai authority in india. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, New York, NY, USA. Association… [find](https://scholar.google.com/scholar?q=Shivani+Kapania%2C+Oliver+Siy%2C+Gabe+Clapper%2C+Azhagu+Meena+SP%2C+and+Nithya+Sambasivan%2E+2022%2E+%E2%80%9Dbecause+ai+is+100%25+right+and+safe%E2%80%9D%3A+User+attitudes+and+sources+of+ai+authority+in+india%2E+In+Proceedings+of+the)
29. Mustafa Karataş and Keisha M. Cutright. 2023. Thinking about god increases acceptance of artificial intelligence in decision-making. Proceedings of the National Academy of Sciences, 120(33):e2218961120. [find](https://scholar.google.com/scholar?q=Mustafa+Karata%C5%9F+and+Keisha+M%2E+Cutright%2E+2023%2E+Thinking+about+god+increases+acceptance+of+artificial+intelligence+in+decision%2Dmaking%2E+Proceedings+of+the+National+Academy+of+Sciences%2C+120%2833%29%3Ae221896112)
30. Artur Klingbeil, Cassandra Grützner, and Philipp Schreck. 2024. Trust and reliance on ai — an experimental study on the extent and costs of overreliance on ai. Computers in Human Behavior, 160:108352. [find](https://scholar.google.com/scholar?q=Artur+Klingbeil%2C+Cassandra+Gr%C3%BCtzner%2C+and+Philipp+Schreck%2E+2024%2E+Trust+and+reliance+on+ai+%E2%80%94+an+experimental+study+on+the+extent+and+costs+of+overreliance+on+ai%2E+Computers+in+Human+Behavior%2C+160%3A108352%2E)
31. Marijn Koolen, Jasmijn van Gorp, and Jacco van Ossenbruggen. 2018. Toward a model for digital tool criticism: Reflection as integrative practice. Digital Scholarship in the Humanities, 34(2):368– 385. [find](https://scholar.google.com/scholar?q=Marijn+Koolen%2C+Jasmijn+van+Gorp%2C+and+Jacco+van+Ossenbruggen%2E+2018%2E+Toward+a+model+for+digital+tool+criticism%3A+Reflection+as+integrative+practice%2E+Digital+Scholarship+in+the+Humanities%2C+34%282%29%3A368%E2%80%93+385%2E)
32. Hadas Kotek, Rikker Dockum, and David Sun. 2023. Gender bias and stereotypes in large language models. In Proceedings of The ACM Collective Intelligence Conference, page 12–24. ACM. [find](https://scholar.google.com/scholar?q=Hadas+Kotek%2C+Rikker+Dockum%2C+and+David+Sun%2E+2023%2E+Gender+bias+and+stereotypes+in+large+language+models%2E+In+Proceedings+of+The+ACM+Collective+Intelligence+Conference%2C+page+12%E2%80%9324%2E+ACM%2E)
33. Jessica B Kuntz and Elise C Silva. 2023. Who authors the internet. Analyzing Gender Diversity in ChatGPT-3 Training Data. Pitt Cyber: University of Pittsburgh. [find](https://scholar.google.com/scholar?q=Jessica+B+Kuntz+and+Elise+C+Silva%2E+2023%2E+Who+authors+the+internet%2E+Analyzing+Gender+Diversity+in+ChatGPT%2D3+Training+Data%2E+Pitt+Cyber%3A+University+of+Pittsburgh%2E)
34. Messi Lee, Jacob Montgomery, and Calvin Lai. 2023. The effect of group status on the variability of group representations in LLM-generated text. In Socially Responsible Language Modelling Research. [find](https://scholar.google.com/scholar?q=Messi+Lee%2C+Jacob+Montgomery%2C+and+Calvin+Lai%2E+2023%2E+The+effect+of+group+status+on+the+variability+of+group+representations+in+LLM%2Dgenerated+text%2E+In+Socially+Responsible+Language+Modelling+Research%2E)
35. Bryan Li, Samar Haider, and Chris Callison-Burch. 2024. This land is Your, My land: Evaluating geopolitical bias in language models through territorial disputes. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techn… [find](https://scholar.google.com/scholar?q=Bryan+Li%2C+Samar+Haider%2C+and+Chris+Callison%2DBurch%2E+2024%2E+This+land+is+Your%2C+My+land%3A+Evaluating+geopolitical+bias+in+language+models+through+territorial+disputes%2E+In+Proceedings+of+the+2024+Conference+)
36. Yotam Liel and Lior Zalmanson. 2020. What if an ai told you that 2 + 2 is 5? conformity to algorithmic recommendations. In International Conference on Information Systems 2020. [find](https://scholar.google.com/scholar?q=Yotam+Liel+and+Lior+Zalmanson%2E+2020%2E+What+if+an+ai+told+you+that+2+%2B+2+is+5%3F+conformity+to+algorithmic+recommendations%2E+In+International+Conference+on+Information+Systems+2020%2E)
37. Zhaoming Liu. 2023. Cultural bias in large language models: A comprehensive analysis and mitigation strategies. Journal of Transcultural Communication, 3(2):224–244. [find](https://scholar.google.com/scholar?q=Zhaoming+Liu%2E+2023%2E+Cultural+bias+in+large+language+models%3A+A+comprehensive+analysis+and+mitigation+strategies%2E+Journal+of+Transcultural+Communication%2C+3%282%29%3A224%E2%80%93244%2E)
38. Jennifer M. Logg, Julia A. Minson, and Don A. Moore. 2019. Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151:90–103. [find](https://scholar.google.com/scholar?q=Jennifer+M%2E+Logg%2C+Julia+A%2E+Minson%2C+and+Don+A%2E+Moore%2E+2019%2E+Algorithm+appreciation%3A+People+prefer+algorithmic+to+human+judgment%2E+Organizational+Behavior+and+Human+Decision+Processes%2C+151%3A90%E2%80%93103%2E)
39. Li Lucy and David Bamman. 2021. [Gender and representation bias in GPT-3 generated stories](https://md.research.elamur.ai/free/en/papers/lucy-2021-gender-representation-bias-gpt-3/). In Proceedings of the Third Workshop on Narrative Understanding, pages 48–55, Virtual. Association for Computational Linguistics.
40. Timothy R. McIntosh, Teo Susnjak, Tong Liu, Paul Watters, and Malka N. Halgamuge. 2024. The inadequacy of reinforcement learning from human feedback—radicalizing large language models via semantic vulnerabilities. IEEE Transactions on Cognitive and Developmental Systems, 16(4):1561–1574. [find](https://scholar.google.com/scholar?q=Timothy+R%2E+McIntosh%2C+Teo+Susnjak%2C+Tong+Liu%2C+Paul+Watters%2C+and+Malka+N%2E+Halgamuge%2E+2024%2E+The+inadequacy+of+reinforcement+learning+from+human+feedback%E2%80%94radicalizing+large+language+models+via+semantic+vul)
41. Simon Münker. 2025. Cultural bias in large language models: Evaluating ai agents through moral questionnaires. arXiv preprint arXiv:2507.10073. — [arXiv:2507.10073](https://arxiv.org/abs/2507.10073) [find](https://scholar.google.com/scholar?q=Simon+M%C3%BCnker%2E+2025%2E+Cultural+bias+in+large+language+models%3A+Evaluating+ai+agents+through+moral+questionnaires%2E+arXiv+preprint+arXiv%3A2507%2E10073%2E+%E2%80%94+%5BarXiv%3A2507%2E10073%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2507%2E10073%29)
42. Arvind Narayanan and Sayash Kapoor. 2024. AI Snake Oil : What Artificial Intelligence Can Do, What It Cant, and How to Tell the Difference. Princeton University Press. [find](https://scholar.google.com/scholar?q=Arvind+Narayanan+and+Sayash+Kapoor%2E+2024%2E+AI+Snake+Oil+%3A+What+Artificial+Intelligence+Can+Do%2C+What+It+Cant%2C+and+How+to+Tell+the+Difference%2E+Princeton+University+Press%2E)
43. Pranav Narayanan Venkit, Sanjana Gautam, Ruchi Panchanadikar, Ting-Hao Huang, and Shomir Wilson. 2023. Nationality bias in text generation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 116–122, Dubrovnik, Croatia. Association f… [find](https://scholar.google.com/scholar?q=Pranav+Narayanan+Venkit%2C+Sanjana+Gautam%2C+Ruchi+Panchanadikar%2C+Ting%2DHao+Huang%2C+and+Shomir+Wilson%2E+2023%2E+Nationality+bias+in+text+generation%2E+In+Proceedings+of+the+17th+Conference+of+the+European+Chapte)
44. Ilana Nguyen, Harini Suresh, and Evan Shieh. 2025. Representational harms in llm-generated narratives against nationalities located in the global south. In HEAL Workshop, CHI. [find](https://scholar.google.com/scholar?q=Ilana+Nguyen%2C+Harini+Suresh%2C+and+Evan+Shieh%2E+2025%2E+Representational+harms+in+llm%2Dgenerated+narratives+against+nationalities+located+in+the+global+south%2E+In+HEAL+Workshop%2C+CHI%2E)
45. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
46. Raja Parasuraman and Dietrich H. Manzey. 2010. Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3):381–410. PMID: 21077562. [find](https://scholar.google.com/scholar?q=Raja+Parasuraman+and+Dietrich+H%2E+Manzey%2E+2010%2E+Complacency+and+bias+in+human+use+of+automation%3A+An+attentional+integration%2E+Human+Factors%2C+52%283%29%3A381%E2%80%93410%2E+PMID%3A+21077562%2E)
47. Bernhard Rieder and Theo Röhle. 2012. Understanding Digital Humanities, chapter Digital Methods: Five Challeges. Houndmills, Basingstoke, Hampshire: Palgrave Macmillan. [find](https://scholar.google.com/scholar?q=Bernhard+Rieder+and+Theo+R%C3%B6hle%2E+2012%2E+Understanding+Digital+Humanities%2C+chapter+Digital+Methods%3A+Five+Challeges%2E+Houndmills%2C+Basingstoke%2C+Hampshire%3A+Palgrave+Macmillan%2E)
48. Thorsten Ries, Karina van Dalen-Oskam, and Fabian Offert. 2024. Reproducibility and explainability in digital humanities. International Journal of Digital Humanities, 6(1):1–7. [find](https://scholar.google.com/scholar?q=Thorsten+Ries%2C+Karina+van+Dalen%2DOskam%2C+and+Fabian+Offert%2E+2024%2E+Reproducibility+and+explainability+in+digital+humanities%2E+International+Journal+of+Digital+Humanities%2C+6%281%29%3A1%E2%80%937%2E)
49. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. [Whose opinions do language models reflect?](https://md.research.elamur.ai/free/en/papers/santurkar-2023-whose-opinions-language-models-reflect/) In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 29971–30004. PMLR.
50. Johannes Schneider. 2025. Mental model shifts in human-llm interactions. Journal of Intelligent Information Systems. [find](https://scholar.google.com/scholar?q=Johannes+Schneider%2E+2025%2E+Mental+model+shifts+in+human%2Dllm+interactions%2E+Journal+of+Intelligent+Information+Systems%2E)
51. Stephen Selka. 2022. Positionality: Identity, standpoint and the limits (and possibilities) of fieldwork. Fieldwork in Religion, 17(1):92–100. [find](https://scholar.google.com/scholar?q=Stephen+Selka%2E+2022%2E+Positionality%3A+Identity%2C+standpoint+and+the+limits+%28and+possibilities%29+of+fieldwork%2E+Fieldwork+in+Religion%2C+17%281%29%3A92%E2%80%93100%2E)
52. Chirag Shah and Emily M. Bender. 2022. Situating search. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval, CHIIR ’22, page 221–232, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Chirag+Shah+and+Emily+M%2E+Bender%2E+2022%2E+Situating+search%2E+In+Proceedings+of+the+2022+Conference+on+Human+Information+Interaction+and+Retrieval%2C+CHIIR+%E2%80%9922%2C+page+221%E2%80%93232%2C+New+York%2C+NY%2C+USA%2E+Association+f)
53. Noel Sharkey and Amanda Sharkey. 2007. Artificial intelligence and natural magic. Artifical Intelligence Review. [find](https://scholar.google.com/scholar?q=Noel+Sharkey+and+Amanda+Sharkey%2E+2007%2E+Artificial+intelligence+and+natural+magic%2E+Artifical+Intelligence+Review%2E)
54. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. [The woman worked as a babysitter: On biases in language generation](https://md.research.elamur.ai/free/en/papers/sheng-2019-woman-worked-as-babysitter-biases/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process…
55. Anders Søgaard, Barbara Plank, and Dirk Hovy. 2014. Selection bias, label bias, and bias in ground truth. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Tutorial Abstracts, pages 11– 13, Dublin, Ireland. Dublin City University and Association for Compu… [find](https://scholar.google.com/scholar?q=Anders+S%C3%B8gaard%2C+Barbara+Plank%2C+and+Dirk+Hovy%2E+2014%2E+Selection+bias%2C+label+bias%2C+and+bias+in+ground+truth%2E+In+Proceedings+of+COLING+2014%2C+the+25th+International+Conference+on+Computational+Linguistics%3A)
56. Pia Sommerauer, Giulia Rambelli, and Tommaso Caselli. 2025. Simulating identity, propagating bias: Abstraction and stereotypes in llm-generated text. In Findings of the Association for Computational Linguistics: EMNLP 2025. [find](https://scholar.google.com/scholar?q=Pia+Sommerauer%2C+Giulia+Rambelli%2C+and+Tommaso+Caselli%2E+2025%2E+Simulating+identity%2C+propagating+bias%3A+Abstraction+and+stereotypes+in+llm%2Dgenerated+text%2E+In+Findings+of+the+Association+for+Computational+L)
57. Nicolas Spatola and Karolina Urbanska. 2019. Godlike robots: the semantic overlap between representation of divine and artificial entities. AI and Society. [find](https://scholar.google.com/scholar?q=Nicolas+Spatola+and+Karolina+Urbanska%2E+2019%2E+Godlike+robots%3A+the+semantic+overlap+between+representation+of+divine+and+artificial+entities%2E+AI+and+Society%2E)
58. S. Shyam Sundar. 2008. The main model : A heuristic approach to understanding technology effects on credibility. In Digital Media,Youth, and Credibility. [find](https://scholar.google.com/scholar?q=S%2E+Shyam+Sundar%2E+2008%2E+The+main+model+%3A+A+heuristic+approach+to+understanding+technology+effects+on+credibility%2E+In+Digital+Media%2CYouth%2C+and+Credibility%2E)
59. Michael Townsen Hicks, James Humphries, and Joe Slater. 2024. Chatgpt is bullshit. Ethics and Information Technology. [find](https://scholar.google.com/scholar?q=Michael+Townsen+Hicks%2C+James+Humphries%2C+and+Joe+Slater%2E+2024%2E+Chatgpt+is+bullshit%2E+Ethics+and+Information+Technology%2E)
60. Karin van Es. 2023. Unpacking tool criticism as practice, in practice. Digital Humanities Quarterly, 17(2). Publisher Copyright: © 2023, Alliance of Digital Humanities Organisations. All rights reserved. [find](https://scholar.google.com/scholar?q=Karin+van+Es%2E+2023%2E+Unpacking+tool+criticism+as+practice%2C+in+practice%2E+Digital+Humanities+Quarterly%2C+17%282%29%2E+Publisher+Copyright%3A+%C2%A9+2023%2C+Alliance+of+Digital+Humanities+Organisations%2E+All+rights+reserv)
61. Karin van Es, Maranke Wieringa, and Mirko Tobias Schäfer. 2018. Tool criticism: From digital methods to digital methodology. In Proceedings of the 2nd International Conference on Web Studies, WS.2 2018, page 24–27, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Karin+van+Es%2C+Maranke+Wieringa%2C+and+Mirko+Tobias+Sch%C3%A4fer%2E+2018%2E+Tool+criticism%3A+From+digital+methods+to+digital+methodology%2E+In+Proceedings+of+the+2nd+International+Conference+on+Web+Studies%2C+WS%2E2+201)
62. Stella Verkijk, Piek Vossen, and Pia Sommerauer. 2025. Language models lack temporal generalization and bigger is not better. In Findings of the Association for Computational Linguistics: ACL 2025, pages 20629–20637, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Stella+Verkijk%2C+Piek+Vossen%2C+and+Pia+Sommerauer%2E+2025%2E+Language+models+lack+temporal+generalization+and+bigger+is+not+better%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2025%2C+pa)
63. Lu Wang, Max Song, Rezvaneh Rezapour, Bum Chul Kwon, and Jina Huh-Yoo. 2023. People’s perceptions toward bias and related concepts in large language models: a systematic review. arXiv preprint arXiv:2309.14504. — [arXiv:2309.14504](https://arxiv.org/abs/2309.14504) [find](https://scholar.google.com/scholar?q=Lu+Wang%2C+Max+Song%2C+Rezvaneh+Rezapour%2C+Bum+Chul+Kwon%2C+and+Jina+Huh%2DYoo%2E+2023%2E+People%E2%80%99s+perceptions+toward+bias+and+related+concepts+in+large+language+models%3A+a+systematic+review%2E+arXiv+preprint+arXiv%3A2)
64. Zeerak Waseem, Smarika Lulz, Joachim Bingel, and Isabelle Augenstein. 2021. Disembodied machine learning: On the illusion of objectivity in nlp. arXiv preprint arXiv:2101.11974. — [arXiv:2101.11974](https://arxiv.org/abs/2101.11974) [find](https://scholar.google.com/scholar?q=Zeerak+Waseem%2C+Smarika+Lulz%2C+Joachim+Bingel%2C+and+Isabelle+Augenstein%2E+2021%2E+Disembodied+machine+learning%3A+On+the+illusion+of+objectivity+in+nlp%2E+arXiv+preprint+arXiv%3A2101%2E11974%2E+%E2%80%94+%5BarXiv%3A2101%2E11974%5D%28h)
65. Sean Williams and James Huckle. 2024. Easy problems that llms get wrong. arXiv preprint arXiv:2405.19616. — [arXiv:2405.19616](https://arxiv.org/abs/2405.19616) [find](https://scholar.google.com/scholar?q=Sean+Williams+and+James+Huckle%2E+2024%2E+Easy+problems+that+llms+get+wrong%2E+arXiv+preprint+arXiv%3A2405%2E19616%2E+%E2%80%94+%5BarXiv%3A2405%2E19616%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2405%2E19616%29)
66. Jiancong Xiao, Ziniu Li, Xingyu Xie, Emily Getzen, Cong Fang, Qi Long, and Weijie J Su. 2025. On the algorithmic bias of aligning large language models with rlhf: Preference collapse and matching regularization. Journal of the American Statistical Association, 120(552):2154–2164. [find](https://scholar.google.com/scholar?q=Jiancong+Xiao%2C+Ziniu+Li%2C+Xingyu+Xie%2C+Emily+Getzen%2C+Cong+Fang%2C+Qi+Long%2C+and+Weijie+J+Su%2E+2025%2E+On+the+algorithmic+bias+of+aligning+large+language+models+with+rlhf%3A+Preference+collapse+and+matching+regu)

*Updated 1 November 2026*