---
title: "InsideOut — paper by Wan et al. (2026) — summary and ideas"
description: "The authors call this insider–outsider bias and build the InsideOut benchmark to measure it. 53 references."
authors: ["Yixin Wan", "Xingrun Chen", "Kai-Wei Chang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1094/"
doi: "10.18653/v1/2026.acl-long.1094"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wan-2026-insideout-measuring-mitigating-insider-outsider/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wan-2026-insideout-measuring-mitigating-insider-outsider/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/33/) / InsideOut: Measuring and Mitigating Insider–Outsider Bias in Interview Script Generation

# InsideOut: Measuring and Mitigating Insider–Outsider Bias in Interview Script Generation

**Authors:** [Yixin Wan](https://md.research.elamur.ai/free/en/authors/yixin-wan/), Xingrun Chen and [Kai-Wei Chang](https://md.research.elamur.ai/free/en/authors/kai-wei-chang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1094/) · [PDF](https://aclanthology.org/2026.acl-long.1094.pdf) · [DOI 10.18653/v1/2026.acl-long.1094](https://doi.org/10.18653/v1/2026.acl-long.1094) · [OpenAlex](https://openalex.org/W4414790658)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper examines a subtle cultural bias in LLMs: when writing text "about" a culture, models tend to speak as members of dominant cultures, especially the US, while treating other cultures as foreign objects of curiosity. The authors call this insider–outsider bias and build the InsideOut benchmark to measure it. The benchmark has 4,000 prompts in which a model plays a journalist interviewing locals in 10 cultures. An LLM judge then decides whether each generated script sounds like an insider or an outsider. The authors also propose three metrics (CEP, CPD, CAG) that capture how this stance shifts across cultures.

They tested five LLMs. On average, more than 88% of US-set scripts took an insider stance, while non-Western contexts such as Papua New Guinea mostly received an outsider framing.

For mitigation, they compare a prompt-based baseline (FIP) with agent-based pipelines (MFA). The agent-based pipelines cut the bias substantially and consistently outperformed FIP. This suggests that inference-time agentic refinement is a practical way to reduce culturally hegemonic framing without retraining.

## Key points
- **InsideOut benchmark:** 4,000 interview-script prompts covering 10 cultures. The planned release is code at github.com/LyannaChen/InsideOut and data at huggingface.co/datasets/elaine1wan/insideout_data.
- **Metrics and models:** Three metrics are used: CEP, which stands for Cultural Externality Percentage; CPD, short for Cultural Perspective Deviation; and CAG, the Cultural Alignment Gap. The models evaluated are ChatGPT, Llama, Mistral, Deepseek and Qwen.
- **Mitigation methods:**
  - FIP injects fairness guidelines through the prompt.
  - MFA-SA has a single agent critique and refine its own output.
  - MFA-HA splits the work between a Critique Agent and a Refinement Agent.
  - MFA-Plan runs an iterative loop of critique, change plan, rewrite and validation.
- **Average reductions across the five models:** MFA-SA lowered CPD by 53.81% and CAG by 60.84%. MFA-HA lowered CPD by 59.70% and CAG by 62.52%. MFA-Plan was reported only for ChatGPT, where it gave the best scores (CPD −77.00%, CAG −77.78%).
- **Best single-model CAG results:** MFA-SA on Llama reduced CAG by 89.70%, and MFA-HA on Qwen by 82.54%. The mitigation experiments used a 500-item subset of InsideOut. The authors note limitations: a small set of cultures and English-only output.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of California, Los Angeles](https://md.research.elamur.ai/free/en/universities/university-of-california-los-angeles/) — University, Los Angeles, United States · 87 works · 54,961 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works

## Cite

Cite as: Yixin Wan, Xingrun Chen and Kai-Wei Chang. 2026. InsideOut: Measuring and Mitigating Insider–Outsider Bias in Interview Script Generation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wan2026insideout,
  title = {{InsideOut: Measuring and Mitigating Insider–Outsider Bias in Interview Script Generation}},
  author = {Yixin Wan, Xingrun Chen and Kai-Wei Chang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1094},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1094}
}
```

- [← previous InsAT: Instance-aware Semantic Alignment and Transfer from Human–Object Keypoints for Zero-to-Few-shot Action Understanding](https://md.research.elamur.ai/free/en/papers/tsutsukawa-2026-insat-instance-aware-semantic-alignment/)
- [next → Insights from Romanized Manipuri Social Media Text: A Transliteration Corpus and Variation Analysis](https://md.research.elamur.ai/free/en/papers/salice-2026-insights-romanized-manipuri-social-media/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/33/)

## References

53 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Badr AlKhamissi, Muhammad ElNokrashy, Mai Alkhamissi, and Mona Diab. 2024. [Investigating cultural alignment of large language models](https://md.research.elamur.ai/free/en/papers/alkhamissi-2024-investigating-cultural-alignment-large-language/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12404–12422, Bangkok, Thailand. Association…
2. Samir Amin. 1989. Eurocentrism. NYU Press. [find](https://scholar.google.com/scholar?q=Samir+Amin%2E+1989%2E+Eurocentrism%2E+NYU+Press%2E)
3. Bushra Asseri, Estabrag Abdelaziz, and Areej Al-Wabil. 2025. Prompt engineering techniques for mitigating cultural bias against arabs and muslims in large language models: A systematic review. arXiv preprint arXiv:2506.18199. — [arXiv:2506.18199](https://arxiv.org/abs/2506.18199) [find](https://scholar.google.com/scholar?q=Bushra+Asseri%2C+Estabrag+Abdelaziz%2C+and+Areej+Al%2DWabil%2E+2025%2E+Prompt+engineering+techniques+for+mitigating+cultural+bias+against+arabs+and+muslims+in+large+language+models%3A+A+systematic+review%2E+arXiv+p)
4. Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The problem with bias: From allocative to representational harms in machine learning. In Proceedings of the 9th Annual Conference of the Special Interest Group for Computing, Information and Society (SIGCIS), Philadelphia, PA. Ass… [find](https://scholar.google.com/scholar?q=Solon+Barocas%2C+Kate+Crawford%2C+Aaron+Shapiro%2C+and+Hanna+Wallach%2E+2017%2E+The+problem+with+bias%3A+From+allocative+to+representational+harms+in+machine+learning%2E+In+Proceedings+of+the+9th+Annual+Conference+)
5. Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. 2024. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954. [find](https://scholar.google.com/scholar?q=Xiao+Bi%2C+Deli+Chen%2C+Guanting+Chen%2C+Shanhuang+Chen%2C+Damai+Dai%2C+Chengqi+Deng%2C+Honghui+Ding%2C+Kai+Dong%2C+Qiushi+Du%2C+Zhe+Fu%2C+et+al%2E+2024%2E+Deepseek+llm%3A+Scaling+open%2Dsource+language+models+with+longtermism%2E+)
6. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. [Latent dirichlet allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/). J. Mach. Learn. Res., 3(null):993–1022.
7. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. [Language (technology) is power: A critical survey of “bias” in NLP](https://md.research.elamur.ai/free/en/papers/blodgett-2020-language-technology-is-power-critical/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454–5476, Online. Association for Computational Linguistics.
8. Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. [Assessing cross-cultural alignment between chatgpt and human societies: An empirical study](https://md.research.elamur.ai/free/en/papers/cao-2023-assessing-cross-cultural-alignment-between/). arXiv preprint arXiv:2303.17466.
9. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46. [find](https://scholar.google.com/scholar?q=Jacob+Cohen%2E+1960%2E+A+coefficient+of+agreement+for+nominal+scales%2E+Educational+and+Psychological+Measurement%2C+20%281%29%3A37%E2%80%9346%2E)
10. Gerard Delanty. 2006. Modernity and the escape from eurocentrism. In Handbook of contemporary European social theory, pages 266–278. Routledge. [find](https://scholar.google.com/scholar?q=Gerard+Delanty%2E+2006%2E+Modernity+and+the+escape+from+eurocentrism%2E+In+Handbook+of+contemporary+European+social+theory%2C+pages+266%E2%80%93278%2E+Routledge%2E)
11. Enrique Dussel. 1993. Eurocentrism and modernity (introduction to the frankfurt lectures). boundary 2, 20(3):65–76. [find](https://scholar.google.com/scholar?q=Enrique+Dussel%2E+1993%2E+Eurocentrism+and+modernity+%28introduction+to+the+frankfurt+lectures%29%2E+boundary+2%2C+20%283%29%3A65%E2%80%9376%2E)
12. Ruixiang Feng, Shen Gao, Xiuying Chen, Lisi Chen, and Shuo Shang. 2025. Culfit: A fine-grained cultural-aware llm training paradigm via multilingual critique data synthesis. arXiv preprint arXiv:2505.19484. [find](https://scholar.google.com/scholar?q=Ruixiang+Feng%2C+Shen+Gao%2C+Xiuying+Chen%2C+Lisi+Chen%2C+and+Shuo+Shang%2E+2025%2E+Culfit%3A+A+fine%2Dgrained+cultural%2Daware+llm+training+paradigm+via+multilingual+critique+data+synthesis%2E+arXiv+preprint+arXiv%3A2505%2E)
13. Joseph Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin, 76:378–. [find](https://scholar.google.com/scholar?q=Joseph+Fleiss%2E+1971%2E+Measuring+nominal+scale+agreement+among+many+raters%2E+Psychological+Bulletin%2C+76%3A378%E2%80%93%2E)
14. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. 2025. A survey on llm-as-a-judge. Preprint, arXiv:2411.15594. [find](https://scholar.google.com/scholar?q=Jiawei+Gu%2C+Xuhui+Jiang%2C+Zhichao+Shi%2C+Hexiang+Tan%2C+Xuehao+Zhai%2C+Chengjin+Xu%2C+Wei+Li%2C+Yinghan+Shen%2C+Shengjie+Ma%2C+Honghao+Liu%2C+Saizhuo+Wang%2C+Kun+Zhang%2C+Yuanzhuo+Wang%2C+Wen+Gao%2C+Lionel+Ni%2C+and+Jian+Guo%2E+20)
15. Ilana Heintz, Ryan Gabbard, Mahesh Srivastava, Dave Barner, Donald Black, Majorie Friedman, and Ralph Weischedel. 2013. Automatic extraction of linguistic metaphors with LDA topic modeling. In Proceedings of the First Workshop on Metaphor in NLP, pages 58– 66, Atlanta, Georgia. Association for Compu… [find](https://scholar.google.com/scholar?q=Ilana+Heintz%2C+Ryan+Gabbard%2C+Mahesh+Srivastava%2C+Dave+Barner%2C+Donald+Black%2C+Majorie+Friedman%2C+and+Ralph+Weischedel%2E+2013%2E+Automatic+extraction+of+linguistic+metaphors+with+LDA+topic+modeling%2E+In+Proceed)
16. William Held, Camille Harris, Michael Best, and Diyi Yang. 2023. A material lens on coloniality in nlp. arXiv preprint arXiv:2311.08391. [find](https://scholar.google.com/scholar?q=William+Held%2C+Camille+Harris%2C+Michael+Best%2C+and+Diyi+Yang%2E+2023%2E+A+material+lens+on+coloniality+in+nlp%2E+arXiv+preprint+arXiv%3A2311%2E08391%2E)
17. Ronald Inglehart and Wayne E. Baker. 2000. Modernization, cultural change, and the persistence of traditional values*. American Sociological Review, 65(1):19–51. [find](https://scholar.google.com/scholar?q=Ronald+Inglehart+and+Wayne+E%2E+Baker%2E+2000%2E+Modernization%2C+cultural+change%2C+and+the+persistence+of+traditional+values%2A%2E+American+Sociological+Review%2C+65%281%29%3A19%E2%80%9351%2E)
18. Hamed Jelodar, Yongli Wang, Chi Yuan, Xia Feng, Xiahui Jiang, Yanchao Li, and Liang Zhao. 2019. Latent dirichlet allocation (lda) and topic modeling: models, applications, a survey. Multimedia tools and applications, 78:15169–15211. [find](https://scholar.google.com/scholar?q=Hamed+Jelodar%2C+Yongli+Wang%2C+Chi+Yuan%2C+Xia+Feng%2C+Xiahui+Jiang%2C+Yanchao+Li%2C+and+Liang+Zhao%2E+2019%2E+Latent+dirichlet+allocation+%28lda%29+and+topic+modeling%3A+models%2C+applications%2C+a+survey%2E+Multimedia+tools+a)
19. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
20. Rebecca L Johnson, Giada Pistilli, Natalia Menédez-González, Leslye Denisse Dias Duran, Enrico Panai, Julija Kalpokiene, and Donald Jay Bertulfo. 2022. The ghost in the machine has an american accent: value conflict in gpt-3. Preprint, arXiv:2203.07785. [find](https://scholar.google.com/scholar?q=Rebecca+L+Johnson%2C+Giada+Pistilli%2C+Natalia+Men%C3%A9dez%2DGonz%C3%A1lez%2C+Leslye+Denisse+Dias+Duran%2C+Enrico+Panai%2C+Julija+Kalpokiene%2C+and+Donald+Jay+Bertulfo%2E+2022%2E+The+ghost+in+the+machine+has+an+american+accent%3A)
21. Julia Kharchenko, Tanya Roosta, Aman Chadha, and Chirag Shah. 2024. How well do llms represent values across cultures? empirical analysis of llm responses based on hofstede cultural dimensions. Preprint, arXiv:2406.14805. — [arXiv:2406.14805](https://arxiv.org/abs/2406.14805) [find](https://scholar.google.com/scholar?q=Julia+Kharchenko%2C+Tanya+Roosta%2C+Aman+Chadha%2C+and+Chirag+Shah%2E+2024%2E+How+well+do+llms+represent+values+across+cultures%3F+empirical+analysis+of+llm+responses+based+on+hofstede+cultural+dimensions%2E+Prepri)
22. Dayeon Ki, Rachel Rudinger, Tianyi Zhou, and Marine Carpuat. 2025. [Multiple LLM agents debate for equitable cultural alignment](https://md.research.elamur.ai/free/en/papers/ki-2025-multiple-llm-agents-debate-equitable/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 24841–24877, Vienna, Austria. Association for Comp…
23. Haein Kong, Yongsu Ahn, Sangyub Lee, and Yunho Maeng. 2024. Gender bias in llm-generated interview responses. arXiv preprint arXiv:2410.20739. — [arXiv:2410.20739](https://arxiv.org/abs/2410.20739) [find](https://scholar.google.com/scholar?q=Haein+Kong%2C+Yongsu+Ahn%2C+Sangyub+Lee%2C+and+Yunho+Maeng%2E+2024%2E+Gender+bias+in+llm%2Dgenerated+interview+responses%2E+arXiv+preprint+arXiv%3A2410%2E20739%2E+%E2%80%94+%5BarXiv%3A2410%2E20739%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2410%2E20739%29)
24. Cheng Li, Mengzhou Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. 2024a. [Culturellm: Incorporating cultural differences into large language models](https://md.research.elamur.ai/free/en/papers/li-2024-culturellm-incorporating-cultural-differences-large/). Preprint, arXiv:2402.10946.
25. Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. 2025. From generation to judgment: Opportunities and challenges of llm-as-a-judge. Preprint, arXiv:2411.16594. [find](https://scholar.google.com/scholar?q=Dawei+Li%2C+Bohan+Jiang%2C+Liangjie+Huang%2C+Alimohammad+Beigi%2C+Chengshuai+Zhao%2C+Zhen+Tan%2C+Amrita+Bhattacharjee%2C+Yuxuan+Jiang%2C+Canyu+Chen%2C+Tianhao+Wu%2C+Kai+Shu%2C+Lu+Cheng%2C+and+Huan+Liu%2E+2025%2E+From+generation+)
26. Huihan Li, Liwei Jiang, Jena D. Hwang, Hyunwoo Kim, Sebastin Santy, Taylor Sorensen, Bill Yuchen Lin, Nouha Dziri, Xiang Ren, and Yejin Choi. 2024b. [Culture-gen: Revealing global cultural perception in language models through natural language prompting](https://md.research.elamur.ai/free/en/papers/li-2024-culture-gen-revealing-global-cultural/). Preprint, arXiv:2404.10199.
27. Reem I. Masoud, Ziquan Liu, Martin Ferianc, Philip Treleaven, and Miguel Rodrigues. 2024. [Cultural alignment in large language models: An explanatory analysis based on hofstede’s cultural dimensions](https://md.research.elamur.ai/free/en/papers/masoud-2025-cultural-alignment-large-language-models/). Preprint, arXiv:2309.12342.
28. Meta. 2024. Llama 3.1 model card. [find](https://scholar.google.com/scholar?q=Meta%2E+2024%2E+Llama+3%2E1+model+card%2E)
29. Burt Monroe, Michael Colaresi, and Kevin Quinn. 2009. Fightin’ words: Lexical feature selection and evaluation for identifying the content of political conflict. Political Analysis, 16. [find](https://scholar.google.com/scholar?q=Burt+Monroe%2C+Michael+Colaresi%2C+and+Kevin+Quinn%2E+2009%2E+Fightin%E2%80%99+words%3A+Lexical+feature+selection+and+evaluation+for+identifying+the+content+of+political+conflict%2E+Political+Analysis%2C+16%2E)
30. Simon Münker. 2025. Cultural bias in large language models: Evaluating ai agents through moral questionnaires. arXiv preprint arXiv:2507.10073. — [arXiv:2507.10073](https://arxiv.org/abs/2507.10073) [find](https://scholar.google.com/scholar?q=Simon+M%C3%BCnker%2E+2025%2E+Cultural+bias+in+large+language+models%3A+Evaluating+ai+agents+through+moral+questionnaires%2E+arXiv+preprint+arXiv%3A2507%2E10073%2E+%E2%80%94+%5BarXiv%3A2507%2E10073%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2507%2E10073%29)
31. Tarek Naous, Michael J. Ryan, Alan Ritter, and Wei Xu. 2024. [Having beer after prayer? measuring cultural bias in large language models](https://md.research.elamur.ai/free/en/papers/naous-2023-having-beer-after-prayer-measuring/). Preprint, arXiv:2305.14456.
32. OpenAI. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/).
33. OpenAI. 2025. Openai o4 mini systen card. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025%2E+Openai+o4+mini+systen+card%2E)
34. Bo Pang, Tingrui Qiao, Caroline Walker, Chris Cunningham, and Yun Sing Koh. 2025. Libra: Measuring bias of large language model from a local context. Preprint, arXiv:2502.01679. — [arXiv:2502.01679](https://arxiv.org/abs/2502.01679) [find](https://scholar.google.com/scholar?q=Bo+Pang%2C+Tingrui+Qiao%2C+Caroline+Walker%2C+Chris+Cunningham%2C+and+Yun+Sing+Koh%2E+2025%2E+Libra%3A+Measuring+bias+of+large+language+model+from+a+local+context%2E+Preprint%2C+arXiv%3A2502%2E01679%2E+%E2%80%94+%5BarXiv%3A2502%2E01679%5D%28h)
35. Philip M. Parker. 1997. National Cultures of the World: A Statistical Reference. Praeger, Westport, CT. [find](https://scholar.google.com/scholar?q=Philip+M%2E+Parker%2E+1997%2E+National+Cultures+of+the+World%3A+A+Statistical+Reference%2E+Praeger%2C+Westport%2C+CT%2E)
36. Siddhesh Pawar, Arnav Arora, Lucie-Aimée Kaffee, and Isabelle Augenstein. 2025. Presumed cultural identity: How names shape llm responses. arXiv preprint arXiv:2502.11995. [find](https://scholar.google.com/scholar?q=Siddhesh+Pawar%2C+Arnav+Arora%2C+Lucie%2DAim%C3%A9e+Kaffee%2C+and+Isabelle+Augenstein%2E+2025%2E+Presumed+cultural+identity%3A+How+names+shape+llm+responses%2E+arXiv+preprint+arXiv%3A2502%2E11995%2E)
37. Richard Peet. 2005. From eurocentrism to americentrism. Antipode, 37(5). [find](https://scholar.google.com/scholar?q=Richard+Peet%2E+2005%2E+From+eurocentrism+to+americentrism%2E+Antipode%2C+37%285%29%2E)
38. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng… [find](https://scholar.google.com/scholar?q=Qwen%2C+%3A%2C+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+)
39. Victor Roudmetof. 1994. Globalization or modernity? Comparative Civilizations Review, 31(31):3. [find](https://scholar.google.com/scholar?q=Victor+Roudmetof%2E+1994%2E+Globalization+or+modernity%3F+Comparative+Civilizations+Review%2C+31%2831%29%3A3%2E)
40. Jonathan Rystrøm, Hannah Rose Kirk, and Scott Hale. 2025. Multilingual != multicultural: Evaluating gaps between multilingual capabilities and cultural alignment in llms. Preprint, arXiv:2502.16534. [find](https://scholar.google.com/scholar?q=Jonathan+Rystr%C3%B8m%2C+Hannah+Rose+Kirk%2C+and+Scott+Hale%2E+2025%2E+Multilingual+%21%3D+multicultural%3A+Evaluating+gaps+between+multilingual+capabilities+and+cultural+alignment+in+llms%2E+Preprint%2C+arXiv%3A2502%2E16534%2E)
41. Shahnewaz Karim Sakib and Anindya Bijoy Das. 2024. Challenging fairness: A comprehensive exploration of bias in llm-based recommendations. In 2024 IEEE International Conference on Big Data (BigData), pages 1585–1592. [find](https://scholar.google.com/scholar?q=Shahnewaz+Karim+Sakib+and+Anindya+Bijoy+Das%2E+2024%2E+Challenging+fairness%3A+A+comprehensive+exploration+of+bias+in+llm%2Dbased+recommendations%2E+In+2024+IEEE+International+Conference+on+Big+Data+%28BigData%29%2C+)
42. Hari Shankar, Vedanta S P, Tejas Cavale, Ponnurangam Kumaraguru, and Abhijnan Chakraborty. 2025. Sometimes the model doth preach: Quantifying religious bias in open llms through demographic analysis in asian nations. Preprint, arXiv:2503.07510. — [arXiv:2503.07510](https://arxiv.org/abs/2503.07510) [find](https://scholar.google.com/scholar?q=Hari+Shankar%2C+Vedanta+S+P%2C+Tejas+Cavale%2C+Ponnurangam+Kumaraguru%2C+and+Abhijnan+Chakraborty%2E+2025%2E+Sometimes+the+model+doth+preach%3A+Quantifying+religious+bias+in+open+llms+through+demographic+analysis+i)
43. Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023. [Judgelm: Fine-tuned large language models are scalable judges](https://md.research.elamur.ai/free/en/papers/zhu-2023-judgelm-fine-tuned-large-language/). arXiv preprint arXiv:2310.17631.
44. Shreya Shankar, J.D. Zamfirescu-Pereira, Bjoern Hartmann, Aditya Parameswaran, and Ian Arawjo. 2024. Who validates the validators? aligning llm-assisted evaluation of llm outputs with human preferences. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, UIST ’… [find](https://scholar.google.com/scholar?q=Shreya+Shankar%2C+J%2ED%2E+Zamfirescu%2DPereira%2C+Bjoern+Hartmann%2C+Aditya+Parameswaran%2C+and+Ian+Arawjo%2E+2024%2E+Who+validates+the+validators%3F+aligning+llm%2Dassisted+evaluation+of+llm+outputs+with+human+preference)
45. Ella Shohat and Robert Stam. 2014. Unthinking Eurocentrism: Multiculturalism and the media. Routledge. [find](https://scholar.google.com/scholar?q=Ella+Shohat+and+Robert+Stam%2E+2014%2E+Unthinking+Eurocentrism%3A+Multiculturalism+and+the+media%2E+Routledge%2E)
46. Nicholas Sukiennik, Chen Gao, Fengli Xu, and Yong Li. 2025. An evaluation of cultural value alignment in llm. Preprint, arXiv:2504.08863. — [arXiv:2504.08863](https://arxiv.org/abs/2504.08863) [find](https://scholar.google.com/scholar?q=Nicholas+Sukiennik%2C+Chen+Gao%2C+Fengli+Xu%2C+and+Yong+Li%2E+2025%2E+An+evaluation+of+cultural+value+alignment+in+llm%2E+Preprint%2C+arXiv%3A2504%2E08863%2E+%E2%80%94+%5BarXiv%3A2504%2E08863%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2504%2E08863%29)
47. Yan Tao, Olga Viberg, Ryan S Baker, and René F Kizilcec. 2024. Cultural bias and cultural alignment of large language models. PNAS Nexus, 3(9):pgae346. [find](https://scholar.google.com/scholar?q=Yan+Tao%2C+Olga+Viberg%2C+Ryan+S+Baker%2C+and+Ren%C3%A9+F+Kizilcec%2E+2024%2E+Cultural+bias+and+cultural+alignment+of+large+language+models%2E+PNAS+Nexus%2C+3%289%29%3Apgae346%2E)
48. Manuel Tonneau, Diyi Liu, Samuel Fraiberger, Ralph Schroeder, Scott A. Hale, and Paul Röttger. 2024. From languages to geographies: Towards evaluating cultural bias in hate speech datasets. Preprint, arXiv:2404.17874. [find](https://scholar.google.com/scholar?q=Manuel+Tonneau%2C+Diyi+Liu%2C+Samuel+Fraiberger%2C+Ralph+Schroeder%2C+Scott+A%2E+Hale%2C+and+Paul+R%C3%B6ttger%2E+2024%2E+From+languages+to+geographies%3A+Towards+evaluating+cultural+bias+in+hate+speech+datasets%2E+Preprint%2C+)
49. Yixin Wan and Kai-Wei Chang. 2025. White men lead, black women help? benchmarking and mitigating language agency social biases in LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9082–9108, Vienna, Austria. Association fo… [find](https://scholar.google.com/scholar?q=Yixin+Wan+and+Kai%2DWei+Chang%2E+2025%2E+White+men+lead%2C+black+women+help%3F+benchmarking+and+mitigating+language+agency+social+biases+in+LLMs%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Association+for)
50. Yixin Wan, George Pu, Jiao Sun, Aparna Garimella, Kai-Wei Chang, and Nanyun Peng. 2023. ["kelly is a warm person, joseph is a role model": Gender biases in llm-generated reference letters](https://md.research.elamur.ai/free/en/papers/wan-2023-kelly-is-warm-person-joseph/). Preprint, arXiv:2310.09219.
51. Hui Wei, Shenghua He, Tian Xia, Fei Liu, Andy Wong, Jingyang Lin, and Mei Han. 2025. Systematic evaluation of llm-as-a-judge in llm alignment tasks: Explainable metrics and diverse prompt templates. Preprint, arXiv:2408.13006. — [arXiv:2408.13006](https://arxiv.org/abs/2408.13006) [find](https://scholar.google.com/scholar?q=Hui+Wei%2C+Shenghua+He%2C+Tian+Xia%2C+Fei+Liu%2C+Andy+Wong%2C+Jingyang+Lin%2C+and+Mei+Han%2E+2025%2E+Systematic+evaluation+of+llm%2Das%2Da%2Djudge+in+llm+alignment+tasks%3A+Explainable+metrics+and+diverse+prompt+templates%2E+P)
52. Weiqi Wu, Hongqiu Wu, Lai Jiang, Xingyuan Liu, Hai Zhao, and Min Zhang. 2024. From role-play to drama-interaction: An llm solution. In Findings of the Association for Computational Linguistics ACL 2024, pages 3271–3290. [find](https://scholar.google.com/scholar?q=Weiqi+Wu%2C+Hongqiu+Wu%2C+Lai+Jiang%2C+Xingyuan+Liu%2C+Hai+Zhao%2C+and+Min+Zhang%2E+2024%2E+From+role%2Dplay+to+drama%2Dinteraction%3A+An+llm+solution%2E+In+Findings+of+the+Association+for+Computational+Linguistics+ACL+202)
53. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in Neural Information Processing Systems, 36:46595–46623.

*Updated 1 November 2026*