---
title: "Towards Measuring and Modeling “Culture” in LLMs: A Survey"
description: "The survey asks how NLP research actually operationalizes \"culture\" when it evaluates cultural… 138 references, cited by 64 papers here."
authors: ["Muhammad Farid Adilazuarda", "Sagnik Mukherjee", "Pradhyumna Lavania", "Siddhant Shivdutt Singh", "Alham Fikri Aji", "Jacki O'Neill", "Ashutosh Modi", "Monojit Choudhury"]
year: 2024
venue: "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2024.emnlp-main.882/"
doi: "10.18653/v1/2024.emnlp-main.882"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/adilazuarda-2024-measuring-modeling-culture-llms-survey/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/adilazuarda-2024-measuring-modeling-culture-llms-survey/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/3/) / Towards Measuring and Modeling “Culture” in LLMs: A Survey

# Towards Measuring and Modeling “Culture” in LLMs: A Survey

**Authors:** Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Shivdutt Singh, [Alham Fikri Aji](https://md.research.elamur.ai/free/en/authors/alham-fikri-aji/), Jacki O'Neill, [Ashutosh Modi](https://md.research.elamur.ai/free/en/authors/ashutosh-modi/) and [Monojit Choudhury](https://md.research.elamur.ai/free/en/authors/monojit-choudhury/) · **Year:** 2024 · **Venue:** [Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2024](https://md.research.elamur.ai/free/en/venues/emnlp/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.emnlp-main.882/) · [PDF](https://aclanthology.org/2024.emnlp-main.882.pdf) · [DOI 10.18653/v1/2024.emnlp-main.882](https://doi.org/10.18653/v1/2024.emnlp-main.882) · [OpenAlex](https://openalex.org/W4404782944)

**Citations:** 38 citations (OpenAlex, 2026-10-14) · [64](https://md.research.elamur.ai/free/en/papers/adilazuarda-2024-measuring-modeling-culture-llms-survey/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The survey asks how NLP research actually operationalizes "culture" when it evaluates cultural awareness, representation and bias in large language models. The authors gathered more than 90 papers from 2020–2024 through keyword searches of the ACL Anthology and Google Scholar, plus NeurIPS and the Web Conference. Work on speech, multimodal models and non-LLM systems was excluded. Each paper was hand-annotated for how it defines culture, how it probes the model, and which languages and cultures it covers. From these labels the authors built bottom-up taxonomies of cultural "proxies" and probing methods.

The central finding is that no surveyed paper explicitly defines culture. Instead, each dataset implicitly stands in for some facet of it, such as food, religion, or norms and values. Research concentrates on values and objectives, while many semantic domains and the notion of aboutness remain untouched. Probing techniques are also fragile.

This matters because it gives practitioners a vocabulary for stating which aspect of culture a benchmark measures. It also exposes a heavy English and Western bias in the evaluation resources.

## Key points
- The corpus contains about 90 papers (2020–2024), with a curated list released on GitHub as `faridlazuarda/cultural-llm-papers`.
- Proxies of culture fall into 12 labels, grouped into demographic proxies (community-level groupings) and semantic proxies (content domains). These are cross-referenced with Hershcovich et al.'s (2022) axes: common ground, aboutness, and objectives and values.
- The authors found no work on several semantic domains from Thompson et al. (2020), including quantity, time, kinship, and pronouns or function words. Aboutness has not been operationalized at all.
- Black-box, prompt-based probing is sensitive to prompt wording and syntax. White-box methods such as gradient-based attribution (e.g., Wichers et al., 2024) have not yet been applied to culture.
- The authors call for:
  - multilingual datasets built natively rather than translated, since most current datasets are English;
  - situated studies of real LLM applications;
  - engagement with anthropology (Geertz's thick description) and HCI.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Indian Institute of Technology Kanpur](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-kanpur/) — University, Kanpur, India · 7 works · 605 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,343 works

## Cite

Cite as: Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Shivdutt Singh, Alham Fikri Aji, Jacki O'Neill, Ashutosh Modi and Monojit Choudhury. 2024. Towards Measuring and Modeling “Culture” in LLMs: A Survey. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{adilazuarda2024measuring,
  title = {{Towards Measuring and Modeling “Culture” in LLMs: A Survey}},
  author = {Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Shivdutt Singh, Alham Fikri Aji, Jacki O'Neill, Ashutosh Modi and Monojit Choudhury},
  booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.882},
  url = {https://doi.org/10.18653/v1/2024.emnlp-main.882}
}
```

- [← previous Tokenizer Choice For LLM Training: Negligible or Crucial?](https://md.research.elamur.ai/free/en/papers/ali-2024-tokenizer-choice-llm-training-negligible/)
- [next → Arcee's MergeKit: A Toolkit for Merging Large Language Models](https://md.research.elamur.ai/free/en/papers/goddard-2024-arcee-s-mergekit-toolkit-merging/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/3/)

## References

138 references, 36 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gati Aher, Rosa I. Arriaga, and Adam Tauman Kalai. 2023. Using large language models to simulate multiple humans and replicate human subject studies. [find](https://scholar.google.com/scholar?q=Gati+Aher%2C+Rosa+I%2E+Arriaga%2C+and+Adam+Tauman+Kalai%2E+2023%2E+Using+large+language+models+to+simulate+multiple+humans+and+replicate+human+subject+studies%2E)
2. Syed Ishtiaque Ahmed, Nicola J. Bidwell, Himanshu Zade, Srihari H. Muralidhar, Anupama Dhareshwar, Baneen Karachiwala, Cedrick N. Tandong, and Jacki O’Neill. 2016. Peer-to-peer in the workplace: A view from the road. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CH… [find](https://scholar.google.com/scholar?q=Syed+Ishtiaque+Ahmed%2C+Nicola+J%2E+Bidwell%2C+Himanshu+Zade%2C+Srihari+H%2E+Muralidhar%2C+Anupama+Dhareshwar%2C+Baneen+Karachiwala%2C+Cedrick+N%2E+Tandong%2C+and+Jacki+O%E2%80%99Neill%2E+2016%2E+Peer%2Dto%2Dpeer+in+the+workplace%3A+A+vie)
3. Badr AlKhamissi, Muhammad ElNokrashy, Mai AlKhamissi, and Mona Diab. 2024. [Investigating cultural alignment of large language models](https://md.research.elamur.ai/free/en/papers/alkhamissi-2024-investigating-cultural-alignment-large-language/).
4. Haozhe An, Zongxia Li, Jieyu Zhao, and Rachel Rudinger. 2023. SODAPOP: Open-ended discovery of social biases in social commonsense reasoning models. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1573–1596, Dubrovnik, Croatia. As… [find](https://scholar.google.com/scholar?q=Haozhe+An%2C+Zongxia+Li%2C+Jieyu+Zhao%2C+and+Rachel+Rudinger%2E+2023%2E+SODAPOP%3A+Open%2Dended+discovery+of+social+biases+in+social+commonsense+reasoning+models%2E+In+Proceedings+of+the+17th+Conference+of+the+Europe)
5. Yannis Assael, Thea Sommerschield, Brendan Shillingford, Mahyar Bordbar, John Pavlopoulos, Maria Chatzipanagiotou, Ion Androutsopoulos, Jonathan Prag, and Nando Freitas. 2022. Restoring and attributing ancient texts using deep neural networks. Nature, 603:280–283. [find](https://scholar.google.com/scholar?q=Yannis+Assael%2C+Thea+Sommerschield%2C+Brendan+Shillingford%2C+Mahyar+Bordbar%2C+John+Pavlopoulos%2C+Maria+Chatzipanagiotou%2C+Ion+Androutsopoulos%2C+Jonathan+Prag%2C+and+Nando+Freitas%2E+2022%2E+Restoring+and+attributin)
6. Mohammad Atari, Mona J. Xue, Peter S. Park, Damián E. Blasi, and Joseph Henrich. Working Paper. Which humans? [find](https://scholar.google.com/scholar?q=Mohammad+Atari%2C+Mona+J%2E+Xue%2C+Peter+S%2E+Park%2C+Dami%C3%A1n+E%2E+Blasi%2C+and+Joseph+Henrich%2E+Working+Paper%2E+Which+humans%3F)
7. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
8. [Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared…](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
9. Lisa Bauer, Hanna Tischer, and Mohit Bansal. 2023. Social commonsense for explanation and cultural bias discovery. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3745–3760, Dubrovnik, Croatia. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Lisa+Bauer%2C+Hanna+Tischer%2C+and+Mohit+Bansal%2E+2023%2E+Social+commonsense+for+explanation+and+cultural+bias+discovery%2E+In+Proceedings+of+the+17th+Conference+of+the+European+Chapter+of+the+Association+for+)
10. Tilman Beck, Hendrik Schuff, Anne Lauscher, and Iryna Gurevych. 2024a. [Sensitivity, performance, robustness: Deconstructing the effect of sociodemographic prompting](https://md.research.elamur.ai/free/en/papers/beck-2024-sensitivity-performance-robustness-deconstructing-effect/).
11. Tilman Beck, Hendrik Schuff, Anne Lauscher, and Iryna Gurevych. 2024b. [Sensitivity, performance, robustness: Deconstructing the effect of sociodemographic prompting](https://md.research.elamur.ai/free/en/papers/beck-2024-sensitivity-performance-robustness-deconstructing-effect/). In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)…
12. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 610–623, New York, NY, USA. Associat… [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+2021%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+Conference)
13. Mehar Bhatia and Vered Shwartz. 2023. GD-COMET: A geo-diverse commonsense inference model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7993–8001, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mehar+Bhatia+and+Vered+Shwartz%2E+2023%2E+GD%2DCOMET%3A+A+geo%2Ddiverse+commonsense+inference+model%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+7993%E2%80%938001%2C+S)
14. Janet Blake. 2000. On defining the cultural heritage. The International and Comparative Law Quarterly, 49(1):61–85. [find](https://scholar.google.com/scholar?q=Janet+Blake%2E+2000%2E+On+defining+the+cultural+heritage%2E+The+International+and+Comparative+Law+Quarterly%2C+49%281%29%3A61%E2%80%9385%2E)
15. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. [Language (technology) is power: A critical survey of “bias” in NLP](https://md.research.elamur.ai/free/en/papers/blodgett-2020-language-technology-is-power-critical/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454– 5476, Online. Association for Computational Linguistics.
16. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. [Man is to computer programmer as woman is to homemaker? debiasing word embeddings](https://md.research.elamur.ai/free/en/papers/bolukbasi-2016-man-computer-programmer-woman-homemaker/). In Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc.
17. P. Bourdieu. 1972. Outline of a Theory of Practice. Cambridge University Press. [find](https://scholar.google.com/scholar?q=P%2E+Bourdieu%2E+1972%2E+Outline+of+a+Theory+of+Practice%2E+Cambridge+University+Press%2E)
18. John Bowers, Graham Button, and Wes Sharrock. 1995. Workflow from within and without: Technology and cooperative work on the print industry shopfloor. In European Conference on Computer Supported Cooperative Work. [find](https://scholar.google.com/scholar?q=John+Bowers%2C+Graham+Button%2C+and+Wes+Sharrock%2E+1995%2E+Workflow+from+within+and+without%3A+Technology+and+cooperative+work+on+the+print+industry+shopfloor%2E+In+European+Conference+on+Computer+Supported+Coop)
19. Nicholas Buttrick. 2024. Studying large language models as compression algorithms for human culture. Trends in Cognitive Sciences, 28(3):187–189. [find](https://scholar.google.com/scholar?q=Nicholas+Buttrick%2E+2024%2E+Studying+large+language+models+as+compression+algorithms+for+human+culture%2E+Trends+in+Cognitive+Sciences%2C+28%283%29%3A187%E2%80%93189%2E)
20. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. [find](https://scholar.google.com/scholar?q=Aylin+Caliskan%2C+Joanna+J%2E+Bryson%2C+and+Arvind+Narayanan%2E+2017%2E+Semantics+derived+automatically+from+language+corpora+contain+human%2Dlike+biases%2E+Science%2C+356%286334%29%3A183%E2%80%93186%2E)
21. Yong Cao, Min Chen, and Daniel Hershcovich. 2024a. [Bridging cultural nuances in dialogue agents through cultural value surveys](https://md.research.elamur.ai/free/en/papers/cao-2024-bridging-cultural-nuances-dialogue-agents/). In Findings of the Association for Computational Linguistics: EACL 2024, pages 929–945, St. Julian’s, Malta. Association for Computational Linguistics.
22. Yong Cao, Yova Kementchedjhieva, Ruixiang Cui, Antonia Karamolegkou, Li Zhou, Megan Dare, Lucia Donatelli, and Daniel Hershcovich. 2024b. Cultural Adaptation of Recipes. Transactions of the Association for Computational Linguistics, 12:80–99. [find](https://scholar.google.com/scholar?q=Yong+Cao%2C+Yova+Kementchedjhieva%2C+Ruixiang+Cui%2C+Antonia+Karamolegkou%2C+Li+Zhou%2C+Megan+Dare%2C+Lucia+Donatelli%2C+and+Daniel+Hershcovich%2E+2024b%2E+Cultural+Adaptation+of+Recipes%2E+Transactions+of+the+Associatio)
23. Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. [Assessing cross-cultural alignment between ChatGPT and human societies: An empirical study](https://md.research.elamur.ai/free/en/papers/cao-2023-assessing-cross-cultural-alignment-between/). In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 53–67, Dubrovnik, Croatia. As…
24. Michael Castelle. 2022. Sapir’s thought-grooves and whorf’s tensors: Reconciling transformer architectures with cultural anthropology. In Cultures in AI/AI in Culture, A NeurIPS 2022 Workshop. University of Warwick, Centre for Interdisciplinary Methodologies. [find](https://scholar.google.com/scholar?q=Michael+Castelle%2E+2022%2E+Sapir%E2%80%99s+thought%2Dgrooves+and+whorf%E2%80%99s+tensors%3A+Reconciling+transformer+architectures+with+cultural+anthropology%2E+In+Cultures+in+AI%2FAI+in+Culture%2C+A+NeurIPS+2022+Workshop%2E+Univers)
25. Sky CH-Wang, Arkadiy Saakyan, Oliver Li, Zhou Yu, and Smaranda Muresan. 2023. Sociocultural norm similarities and differences via situational alignment and explainable textual entailment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3548–3564, Sing… [find](https://scholar.google.com/scholar?q=Sky+CH%2DWang%2C+Arkadiy+Saakyan%2C+Oliver+Li%2C+Zhou+Yu%2C+and+Smaranda+Muresan%2E+2023%2E+Sociocultural+norm+similarities+and+differences+via+situational+alignment+and+explainable+textual+entailment%2E+In+Proceedin)
26. Ana Paula Chaves and Marco Aurélio Gerosa. 2019. How should my chatbot interact? a survey on social characteristics in human–chatbot interaction design. International Journal of Human–Computer Interaction, 37:729 – 758. [find](https://scholar.google.com/scholar?q=Ana+Paula+Chaves+and+Marco+Aur%C3%A9lio+Gerosa%2E+2019%2E+How+should+my+chatbot+interact%3F+a+survey+on+social+characteristics+in+human%E2%80%93chatbot+interaction+design%2E+International+Journal+of+Human%E2%80%93Computer+Interac)
27. Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024. [Benchmarking large language models in retrieval-augmented generation](https://md.research.elamur.ai/free/en/papers/chen-2023-benchmarking-large-language-models-retrieval/). Proceedings of the AAAI Conference on Artificial Intelligence, 38(16):17754–17762.
28. Yu Ying Chiu, Liwei Jiang, Maria Antoniak, Chan Young Park, Shuyue Stella Li, Mehar Bhatia, Sahithya Ravi, Yulia Tsvetkov, Vered Shwartz, and Yejin Choi. 2024. Culturalteaming: Ai-assisted interactive red-teaming for challenging llms’ (lack of) multicultural knowledge. pvq–40. Journal of Personality… [find](https://scholar.google.com/scholar?q=Yu+Ying+Chiu%2C+Liwei+Jiang%2C+Maria+Antoniak%2C+Chan+Young+Park%2C+Shuyue+Stella+Li%2C+Mehar+Bhatia%2C+Sahithya+Ravi%2C+Yulia+Tsvetkov%2C+Vered+Shwartz%2C+and+Yejin+Choi%2E+2024%2E+Culturalteaming%3A+Ai%2Dassisted+interactive)
29. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does BERT look at? an analysis of BERT’s attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276–286, Florence, Italy. Association for Computationa… [find](https://scholar.google.com/scholar?q=Kevin+Clark%2C+Urvashi+Khandelwal%2C+Omer+Levy%2C+and+Christopher+D%2E+Manning%2E+2019%2E+What+does+BERT+look+at%3F+an+analysis+of+BERT%E2%80%99s+attention%2E+In+Proceedings+of+the+2019+ACL+Workshop+BlackboxNLP%3A+Analyzing+an)
30. Ned Cooper, Courtney Heldreth, and Ben Hutchinson. 2024. ”it’s how you do things that matters”: Attending to process to better serve indigenous communities with language technologies. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volu… [find](https://scholar.google.com/scholar?q=Ned+Cooper%2C+Courtney+Heldreth%2C+and+Ben+Hutchinson%2E+2024%2E+%E2%80%9Dit%E2%80%99s+how+you+do+things+that+matters%E2%80%9D%3A+Attending+to+process+to+better+serve+indigenous+communities+with+language+technologies%2E+In+Proceedings+o)
31. Preetam Prabhu Srikar Dammu, Hayoung Jung, Anjali Singh, Monojit Choudhury, and Tanushree Mitra. 2024. " they are uncultured": Unveiling covert harms and social threats in llm generated conversations. arXiv preprint arXiv:2405.05378. [find](https://scholar.google.com/scholar?q=Preetam+Prabhu+Srikar+Dammu%2C+Hayoung+Jung%2C+Anjali+Singh%2C+Monojit+Choudhury%2C+and+Tanushree+Mitra%2E+2024%2E+%22+they+are+uncultured%22%3A+Unveiling+covert+harms+and+social+threats+in+llm+generated+conversations%2E)
32. Dipto Das, Shion Guha, and Bryan Semaan. 2023. Toward cultural bias evaluation datasets: The case of Bengali gender, religious, and national identity. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 68– 83, Dubrovnik, Croatia. Association for Computational… [find](https://scholar.google.com/scholar?q=Dipto+Das%2C+Shion+Guha%2C+and+Bryan+Semaan%2E+2023%2E+Toward+cultural+bias+evaluation+datasets%3A+The+case+of+Bengali+gender%2C+religious%2C+and+national+identity%2E+In+Proceedings+of+the+First+Workshop+on+Cross%2DCul)
33. Sunipa Dev, Jaya Goyal, Dinesh Tewari, Shachi Dave, and Vinodkumar Prabhakaran. 2023. Building socio-culturally inclusive stereotype resources with community engagement. [find](https://scholar.google.com/scholar?q=Sunipa+Dev%2C+Jaya+Goyal%2C+Dinesh+Tewari%2C+Shachi+Dave%2C+and+Vinodkumar+Prabhakaran%2E+2023%2E+Building+socio%2Dculturally+inclusive+stereotype+resources+with+community+engagement%2E)
34. Esin Durmus, Karina Nguyen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2024. [Towards measuring t…](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/)
35. Esin Durmus, Karina Nyugen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2023. [Towards measuring t…](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/)
36. Rochelle Choenni, Anne Lauscher, and Ekaterina Shutova. 2024. The echoes of multilinguality: Tracing cultural value shifts during lm fine-tuning. [find](https://scholar.google.com/scholar?q=Rochelle+Choenni%2C+Anne+Lauscher%2C+and+Ekaterina+Shutova%2E+2024%2E+The+echoes+of+multilinguality%3A+Tracing+cultural+value+shifts+during+lm+fine%2Dtuning%2E)
37. Ashutosh Dwivedi, Pradhyumna Lavania, and Ashutosh Modi. 2023. EtiCor: Corpus for analyzing LLMs for etiquettes. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6921–6931, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ashutosh+Dwivedi%2C+Pradhyumna+Lavania%2C+and+Ashutosh+Modi%2E+2023%2E+EtiCor%3A+Corpus+for+analyzing+LLMs+for+etiquettes%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Processi)
38. Jan Cieciuch and Shalom Schwartz. 2012. The number of distinct basic values and their structure assessed by Lance Eliot. 2022. Ai ethics and the future of where large language models are heading. Forbes. [find](https://scholar.google.com/scholar?q=Jan+Cieciuch+and+Shalom+Schwartz%2E+2012%2E+The+number+of+distinct+basic+values+and+their+structure+assessed+by+Lance+Eliot%2E+2022%2E+Ai+ethics+and+the+future+of+where+large+language+models+are+heading%2E+Forb)
39. EVS/WVS. 2022. Joint evs/wvs 2017-2022 dataset (joint evs/wvs). GESIS, Cologne. ZA7505 Data file Version 4.0.0, https://doi.org/10.4232/1.14023. — [doi:10.4232/1.14023](https://doi.org/10.4232/1.14023) [↗](https://doi.org/10.4232/1.14023)
40. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. [From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models](https://md.research.elamur.ai/free/en/papers/feng-2023-pretraining-data-language-models-downstream/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume…
41. Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi. 2020. Social chemistry 101: Learning to reason about social and moral norms. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 653–670, Online. Association for Computatio… [find](https://scholar.google.com/scholar?q=Maxwell+Forbes%2C+Jena+D%2E+Hwang%2C+Vered+Shwartz%2C+Maarten+Sap%2C+and+Yejin+Choi%2E+2020%2E+Social+chemistry+101%3A+Learning+to+reason+about+social+and+moral+norms%2E+In+Proceedings+of+the+2020+Conference+on+Empiric)
42. Simona Frenda, Alessandro Pedrani, Valerio Basile, Soda Marem Lo, Alessandra Teresa Cignarella, Raffaella Panizzon, Cristina Marco, Bianca Scarlini, Viviana Patti, Cristina Bosco, and Davide Bernardi. 2023. [EPIC: Multi-perspective annotation of a corpus of irony](https://md.research.elamur.ai/free/en/papers/frenda-2023-epic-multi-perspective-annotation-corpus/). In Proceedings of the 61st Annual Me…
43. Felix Friedrich, Wolfgang Stammer, Patrick Schramowski, and Kristian Kersting. 2023. Revision Transformers: Instructing Language Models to Change Their Values. [find](https://scholar.google.com/scholar?q=Felix+Friedrich%2C+Wolfgang+Stammer%2C+Patrick+Schramowski%2C+and+Kristian+Kersting%2E+2023%2E+Revision+Transformers%3A+Instructing+Language+Models+to+Change+Their+Values%2E)
44. Yi Fung, Tuhin Chakrabarty, Hao Guo, Owen Rambow, Smaranda Muresan, and Heng Ji. 2023. NORM-SAGE: Multi-lingual multi-cultural norm discovery from conversations on-the-fly. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15217–15230, Singapore. Associ… [find](https://scholar.google.com/scholar?q=Yi+Fung%2C+Tuhin+Chakrabarty%2C+Hao+Guo%2C+Owen+Rambow%2C+Smaranda+Muresan%2C+and+Heng+Ji%2E+2023%2E+NORM%2DSAGE%3A+Multi%2Dlingual+multi%2Dcultural+norm+discovery+from+conversations+on%2Dthe%2Dfly%2E+In+Proceedings+of+the+2023+)
45. Yi Fung, Ruining Zhao, Jae Doo, Chenkai Sun, and Heng Ji. 2024. Massively multi-cultural knowledge acquisition and lm benchmarking. [find](https://scholar.google.com/scholar?q=Yi+Fung%2C+Ruining+Zhao%2C+Jae+Doo%2C+Chenkai+Sun%2C+and+Heng+Ji%2E+2024%2E+Massively+multi%2Dcultural+knowledge+acquisition+and+lm+benchmarking%2E)
46. Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. 2024. [Bias and Fairness in Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/gallegos-2023-bias-fairness-large-language-models/). Computational Linguistics, 50(3):1097– 1179.
47. C. Geertz. 1973. The Interpretation Of Cultures. ACLS Humanities E-Book. Basic Books. [find](https://scholar.google.com/scholar?q=C%2E+Geertz%2E+1973%2E+The+Interpretation+Of+Cultures%2E+ACLS+Humanities+E%2DBook%2E+Basic+Books%2E)
48. [Amelia Glaese, Nat McAleese, Maja Tr˛ebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug F…](https://md.research.elamur.ai/free/en/papers/glaese-2022-improving-alignment-dialogue-agents-targeted/)
49. Greg Gondwe. 2023. Chatgpt and the global south: how are journalists in sub-saharan africa engaging with generative ai? Online Media and Global Communication, 2. [find](https://scholar.google.com/scholar?q=Greg+Gondwe%2E+2023%2E+Chatgpt+and+the+global+south%3A+how+are+journalists+in+sub%2Dsaharan+africa+engaging+with+generative+ai%3F+Online+Media+and+Global+Communication%2C+2%2E)
50. Akshat Gupta, Xiaoyang Song, and Gopala Anumanchipalli. 2024. Self-assessment tests are unreliable measures of llm personality. [find](https://scholar.google.com/scholar?q=Akshat+Gupta%2C+Xiaoyang+Song%2C+and+Gopala+Anumanchipalli%2E+2024%2E+Self%2Dassessment+tests+are+unreliable+measures+of+llm+personality%2E)
51. Luna Luan Haoyue and Hichang Cho. 2024. Factors influencing intention to engage in human–chatbot interaction: examining user perceptions and context culture orientation. Universal Access in the Information Society. [find](https://scholar.google.com/scholar?q=Luna+Luan+Haoyue+and+Hichang+Cho%2E+2024%2E+Factors+influencing+intention+to+engage+in+human%E2%80%93chatbot+interaction%3A+examining+user+perceptions+and+context+culture+orientation%2E+Universal+Access+in+the+Inform)
52. Shreya Havaldar, Bhumika Singhal, Sunny Rai, Langchen Liu, Sharath Chandra Guntuku, and Lyle Ungar. 2023. Multilingual language models are not multicultural: A case study in emotion. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis,… [find](https://scholar.google.com/scholar?q=Shreya+Havaldar%2C+Bhumika+Singhal%2C+Sunny+Rai%2C+Langchen+Liu%2C+Sharath+Chandra+Guntuku%2C+and+Lyle+Ungar%2E+2023%2E+Multilingual+language+models+are+not+multicultural%3A+A+case+study+in+emotion%2E+In+Proceedings+of)
53. Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, and Anders Søgaard. 2022. [Challenges and strategies in cross-cultural N…](https://md.research.elamur.ai/free/en/papers/hershcovich-2022-challenges-strategies-cross-cultural-nlp/)
54. Jing Huang and Diyi Yang. 2023. [Culturally aware natural language inference](https://md.research.elamur.ai/free/en/papers/huang-2023-culturally-aware-natural-language-inference/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7591–7609, Singapore. Association for Computational Linguistics.
55. EunJeong Hwang, Bodhisattwa Majumder, and Niket Tandon. 2023. [Aligning language models to user opinions](https://md.research.elamur.ai/free/en/papers/hwang-2023-aligning-language-models-user-opinions/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5906– 5919, Singapore. Association for Computational Linguistics.
56. Akshita Jha, Aida Mostafazadeh Davani, Chandan K Reddy, Shachi Dave, Vinodkumar Prabhakaran, and Sunipa Dev. 2023. SeeGULL: A stereotype benchmark with broad geo-cultural coverage leveraging generative models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Akshita+Jha%2C+Aida+Mostafazadeh+Davani%2C+Chandan+K+Reddy%2C+Shachi+Dave%2C+Vinodkumar+Prabhakaran%2C+and+Sunipa+Dev%2E+2023%2E+SeeGULL%3A+A+stereotype+benchmark+with+broad+geo%2Dcultural+coverage+leveraging+generativ)
57. Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini, and Yejin Choi. 2022. Can machines learn morality? the delphi experiment. [find](https://scholar.google.com/scholar?q=Liwei+Jiang%2C+Jena+D%2E+Hwang%2C+Chandra+Bhagavatula%2C+Ronan+Le+Bras%2C+Jenny+Liang%2C+Jesse+Dodge%2C+Keisuke+Sakaguchi%2C+Maxwell+Forbes%2C+Jon+Borchardt%2C+Saadia+Gabriel%2C+Yulia+Tsvetkov%2C+Oren+Etzioni%2C+Maarten+Sap%2C+R)
58. Jiho Jin, Jiseon Kim, Nayeon Lee, Haneul Yoo, Alice Oh, and Hwaran Lee. 2024. [Kobbq: Korean bias benchmark for question answering](https://md.research.elamur.ai/free/en/papers/jin-2024-kobbq-korean-bias-benchmark-question/).
59. Rebecca L Johnson, Giada Pistilli, Natalia Menédez-González, Leslye Denisse Dias Duran, Enrico Panai, Julija Kalpokiene, and Donald Jay Bertulfo. 2022. The ghost in the machine has an american accent: value conflict in gpt-3. [find](https://scholar.google.com/scholar?q=Rebecca+L+Johnson%2C+Giada+Pistilli%2C+Natalia+Men%C3%A9dez%2DGonz%C3%A1lez%2C+Leslye+Denisse+Dias+Duran%2C+Enrico+Panai%2C+Julija+Kalpokiene%2C+and+Donald+Jay+Bertulfo%2E+2022%2E+The+ghost+in+the+machine+has+an+american+accent%3A)
60. Anubha Kabra, Emmy Liu, Simran Khanuja, Alham Fikri Aji, Genta Winata, Samuel Cahyawijaya, Anuoluwapo Aremu, Perez Ogayo, and Graham Neubig. 2023. Multi-lingual and multi-cultural figurative language understanding. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8269–82… [find](https://scholar.google.com/scholar?q=Anubha+Kabra%2C+Emmy+Liu%2C+Simran+Khanuja%2C+Alham+Fikri+Aji%2C+Genta+Winata%2C+Samuel+Cahyawijaya%2C+Anuoluwapo+Aremu%2C+Perez+Ogayo%2C+and+Graham+Neubig%2E+2023%2E+Multi%2Dlingual+and+multi%2Dcultural+figurative+language+)
61. Naveena Karusala, Ding Wang, and Jacki O’Neill. 2020. Making chat at home in the hospital: Exploring chat use by nurses. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. [find](https://scholar.google.com/scholar?q=Naveena+Karusala%2C+Ding+Wang%2C+and+Jacki+O%E2%80%99Neill%2E+2020%2E+Making+chat+at+home+in+the+hospital%3A+Exploring+chat+use+by+nurses%2E+Proceedings+of+the+2020+CHI+Conference+on+Human+Factors+in+Computing+Systems%2E)
62. Amr Keleg and Walid Magdy. 2023. DLAMA: A framework for curating culturally diverse facts for probing the knowledge of pretrained language models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6245–6266, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Amr+Keleg+and+Walid+Magdy%2E+2023%2E+DLAMA%3A+A+framework+for+curating+culturally+diverse+facts+for+probing+the+knowledge+of+pretrained+language+models%2E+In+Findings+of+the+Association+for+Computational+Ling)
63. Simran Khanuja, Sebastian Ruder, and Partha Talukdar. 2023. Evaluating the diversity, equity, and inclusion of NLP technology: A case study for Indian languages. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1763–1777, Dubrovnik, Croatia. Association for Computationa… [find](https://scholar.google.com/scholar?q=Simran+Khanuja%2C+Sebastian+Ruder%2C+and+Partha+Talukdar%2E+2023%2E+Evaluating+the+diversity%2C+equity%2C+and+inclusion+of+NLP+technology%3A+A+case+study+for+Indian+languages%2E+In+Findings+of+the+Association+for+Com)
64. Eunsu Kim, Juyoung Suk, Philhoon Oh, Haneul Yoo, James Thorne, and Alice Oh. 2024. CLIcK: A benchmark dataset of cultural and linguistic intelligence in Korean. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024… [find](https://scholar.google.com/scholar?q=Eunsu+Kim%2C+Juyoung+Suk%2C+Philhoon+Oh%2C+Haneul+Yoo%2C+James+Thorne%2C+and+Alice+Oh%2E+2024%2E+CLIcK%3A+A+benchmark+dataset+of+cultural+and+linguistic+intelligence+in+Korean%2E+In+Proceedings+of+the+2024+Joint+Intern)
65. Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew Bean, Katerina Margatina, Juan Ciro, Rafael Mosquera, Max Bartolo, Adina Williams, He He, Bertie Vidgen, and Scott A. Hale. 2024. The prism alignment project: What participatory, representative and individualised human feedback reveals abo… [find](https://scholar.google.com/scholar?q=Hannah+Rose+Kirk%2C+Alexander+Whitefield%2C+Paul+R%C3%B6ttger%2C+Andrew+Bean%2C+Katerina+Margatina%2C+Juan+Ciro%2C+Rafael+Mosquera%2C+Max+Bartolo%2C+Adina+Williams%2C+He+He%2C+Bertie+Vidgen%2C+and+Scott+A%2E+Hale%2E+2024%2E+The+prism)
66. Fajri Koto, Nurul Aisyah, Haonan Li, and Timothy Baldwin. 2023. Large language models only pass primary school exams in Indonesia: A comprehensive test on IndoMMLU. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12359–12374, Singapore. Association fo… [find](https://scholar.google.com/scholar?q=Fajri+Koto%2C+Nurul+Aisyah%2C+Haonan+Li%2C+and+Timothy+Baldwin%2E+2023%2E+Large+language+models+only+pass+primary+school+exams+in+Indonesia%3A+A+comprehensive+test+on+IndoMMLU%2E+In+Proceedings+of+the+2023+Conferen)
67. Fajri Koto, Rahmad Mahendra, Nurul Aisyah, and Timothy Baldwin. 2024. Indoculture: Exploring geographically-influenced cultural commonsense reasoning across eleven indonesian provinces. [find](https://scholar.google.com/scholar?q=Fajri+Koto%2C+Rahmad+Mahendra%2C+Nurul+Aisyah%2C+and+Timothy+Baldwin%2E+2024%2E+Indoculture%3A+Exploring+geographically%2Dinfluenced+cultural+commonsense+reasoning+across+eleven+indonesian+provinces%2E)
68. Grgur Kovač, Masataka Sawayama, Rémy Portelas, Cédric Colas, Peter Ford Dominey, and Pierre-Yves Oudeyer. 2023. Large language models as superpositions of cultural perspectives. [find](https://scholar.google.com/scholar?q=Grgur+Kovac%CC%8C%2C+Masataka+Sawayama%2C+R%C3%A9my+Portelas%2C+C%C3%A9dric+Colas%2C+Peter+Ford+Dominey%2C+and+Pierre%2DYves+Oudeyer%2E+2023%2E+Large+language+models+as+superpositions+of+cultural+perspectives%2E)
69. Hwaran Lee, Seokhee Hong, Joonsuk Park, Takyoung Kim, Gunhee Kim, and Jung-woo Ha. 2023. KoSBI: A dataset for mitigating social bias risks towards safer large language model applications. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry T… [find](https://scholar.google.com/scholar?q=Hwaran+Lee%2C+Seokhee+Hong%2C+Joonsuk+Park%2C+Takyoung+Kim%2C+Gunhee+Kim%2C+and+Jung%2Dwoo+Ha%2E+2023%2E+KoSBI%3A+A+dataset+for+mitigating+social+bias+risks+towards+safer+large+language+model+applications%2E+In+Proceedin)
70. Cheng Li, Mengzhou Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. 2024a. [Culturellm: Incorporating cultural differences into large language models](https://md.research.elamur.ai/free/en/papers/li-2024-culturellm-incorporating-cultural-differences-large/).
71. Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. 2024b. [Cmmlu: Measuring massive multitask language understanding in chinese](https://md.research.elamur.ai/free/en/papers/li-2023-cmmlu-measuring-massive-multitask-language/).
72. Huihan Li, Liwei Jiang, Jena D. Huang, Hyunwoo Kim, Sebastin Santy, Taylor Sorensen, Bill Yuchen Lin, Nouha Dziri, Xiang Ren, and Yejin Choi. 2024c. Culture-gen: Revealing global cultural perception in language models through natural language prompting. [find](https://scholar.google.com/scholar?q=Huihan+Li%2C+Liwei+Jiang%2C+Jena+D%2E+Huang%2C+Hyunwoo+Kim%2C+Sebastin+Santy%2C+Taylor+Sorensen%2C+Bill+Yuchen+Lin%2C+Nouha+Dziri%2C+Xiang+Ren%2C+and+Yejin+Choi%2E+2024c%2E+Culture%2Dgen%3A+Revealing+global+cultural+perception+i)
73. Chen Cecilia Liu, Iryna Gurevych, and Anna Korhonen. 2024a. [Culturally aware and adapted nlp: A taxonomy and a survey of the state of the art](https://md.research.elamur.ai/free/en/papers/liu-2025-culturally-aware-adapted-nlp-taxonomy/). arXiv e-prints, pages arXiv–2406.
74. Chen Cecilia Liu, Fajri Koto, Timothy Baldwin, and Iryna Gurevych. 2024b. Are multilingual llms culturally-diverse reasoners? an investigation into multicultural proverbs and sayings. [find](https://scholar.google.com/scholar?q=Chen+Cecilia+Liu%2C+Fajri+Koto%2C+Timothy+Baldwin%2C+and+Iryna+Gurevych%2E+2024b%2E+Are+multilingual+llms+culturally%2Ddiverse+reasoners%3F+an+investigation+into+multicultural+proverbs+and+sayings%2E)
75. Queenie Luo, Michael J. Puett, and Michael D. Smith. 2024. A "perspectival" mirror of the elephant: Investigating language bias on google, chatgpt, youtube, and wikipedia. Queue, 22(1):23–47. [find](https://scholar.google.com/scholar?q=Queenie+Luo%2C+Michael+J%2E+Puett%2C+and+Michael+D%2E+Smith%2E+2024%2E+A+%22perspectival%22+mirror+of+the+elephant%3A+Investigating+language+bias+on+google%2C+chatgpt%2C+youtube%2C+and+wikipedia%2E+Queue%2C+22%281%29%3A23%E2%80%9347%2E)
76. Reem I. Masoud, Ziquan Liu, Martin Ferianc, Philip Treleaven, and Miguel Rodrigues. 2024. [Cultural alignment in large language models: An explanatory analysis based on hofstede’s cultural dimensions](https://md.research.elamur.ai/free/en/papers/masoud-2025-cultural-alignment-large-language-models/).
77. Alessio Miaschi, Dominique Brunato, Felice Dell’Orletta, and Giulia Venturi. 2020. Linguistic profiling of a neural language model. In Proceedings of the 28th International Conference on Computational Linguistics, pages 745–756, Barcelona, Spain (Online). International Committee on Computational Lin… [find](https://scholar.google.com/scholar?q=Alessio+Miaschi%2C+Dominique+Brunato%2C+Felice+Dell%E2%80%99Orletta%2C+and+Giulia+Venturi%2E+2020%2E+Linguistic+profiling+of+a+neural+language+model%2E+In+Proceedings+of+the+28th+International+Conference+on+Computational)
78. Nusrat Jahan Mim, Dipannita Nandi, Sadaf Sumyia Khan, Arundhuti Dey, and Syed Ishtiaque Ahmed. 2024. In-between visuals and visible: The impacts of text-to-image generative ai tools on digital image-making practices in the global south. In Proceedings of the CHI Conference on Human Factors in Comput… [find](https://scholar.google.com/scholar?q=Nusrat+Jahan+Mim%2C+Dipannita+Nandi%2C+Sadaf+Sumyia+Khan%2C+Arundhuti+Dey%2C+and+Syed+Ishtiaque+Ahmed%2E+2024%2E+In%2Dbetween+visuals+and+visible%3A+The+impacts+of+text%2Dto%2Dimage+generative+ai+tools+on+digital+image%2Dm)
79. Farhad Moghimifar, Shilin Qu, Tongtong Wu, Yuan-Fang Li, and Gholamreza Haffari. 2023. Norm-Mark: A weakly supervised Markov model for sociocultural norm discovery. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5081–5089, Toronto, Canada. Association for Computational… [find](https://scholar.google.com/scholar?q=Farhad+Moghimifar%2C+Shilin+Qu%2C+Tongtong+Wu%2C+Yuan%2DFang+Li%2C+and+Gholamreza+Haffari%2E+2023%2E+Norm%2DMark%3A+A+weakly+supervised+Markov+model+for+sociocultural+norm+discovery%2E+In+Findings+of+the+Association+for+)
80. Youssef Mohamed, Mohamed Abdelfattah, Shyma Alhuwaider, Feifan Li, Xiangliang Zhang, Kenneth Church, and Mohamed Elhoseiny. 2022. ArtELingo: A million emotion annotations of WikiArt with emphasis on diversity over language and culture. In Proceedings of the 2022 Conference on Empirical Methods in Na… [find](https://scholar.google.com/scholar?q=Youssef+Mohamed%2C+Mohamed+Abdelfattah%2C+Shyma+Alhuwaider%2C+Feifan+Li%2C+Xiangliang+Zhang%2C+Kenneth+Church%2C+and+Mohamed+Elhoseiny%2E+2022%2E+ArtELingo%3A+A+million+emotion+annotations+of+WikiArt+with+emphasis+on+d)
81. L. Monaghan, J.E. Goodman, and J. Robinson. 2012. A Cultural Approach to Interpersonal Communication: Essential Readings. Wiley. [find](https://scholar.google.com/scholar?q=L%2E+Monaghan%2C+J%2EE%2E+Goodman%2C+and+J%2E+Robinson%2E+2012%2E+A+Cultural+Approach+to+Interpersonal+Communication%3A+Essential+Readings%2E+Wiley%2E)
82. Cristina Mora. 2013. Cultures and organizations: Software of the mind intercultural cooperation and its importance for survival. Journal of Media Research, 6(1):65. [find](https://scholar.google.com/scholar?q=Cristina+Mora%2E+2013%2E+Cultures+and+organizations%3A+Software+of+the+mind+intercultural+cooperation+and+its+importance+for+survival%2E+Journal+of+Media+Research%2C+6%281%29%3A65%2E)
83. Anjishnu Mukherjee, Chahat Raj, Ziwei Zhu, and Antonios Anastasopoulos. 2023. Global Voices, local biases: Socio-cultural prejudices across languages. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15828–15845, Singapore. Association for Computationa… [find](https://scholar.google.com/scholar?q=Anjishnu+Mukherjee%2C+Chahat+Raj%2C+Ziwei+Zhu%2C+and+Antonios+Anastasopoulos%2E+2023%2E+Global+Voices%2C+local+biases%3A+Socio%2Dcultural+prejudices+across+languages%2E+In+Proceedings+of+the+2023+Conference+on+Empirica)
84. R. Münch, N.J. Smelser, American Sociological Association. Theory Section, Deutsche Gesellschaft fur Soziologie. Sektion Soziologische Theorien, and Deutsche Gesellschaft für Soziologie. Sektion Soziologische Theorien. 1992. Theory of Culture. New directions in cultural analysis. University of Calif… [find](https://scholar.google.com/scholar?q=R%2E+M%C3%BCnch%2C+N%2EJ%2E+Smelser%2C+American+Sociological+Association%2E+Theory+Section%2C+Deutsche+Gesellschaft+fur+Soziologie%2E+Sektion+Soziologische+Theorien%2C+and+Deutsche+Gesellschaft+f%C3%BCr+Soziologie%2E+Sektion+Sozio)
85. Sheshera Mysore, Zhuoran Lu, Mengting Wan, Longqi Yang, Steve Menezes, Tina Baghaee, Emmanuel Barajas Gonzalez, Jennifer Neville, and Tara Safavi. 2023. Pearl: Personalizing large language model writing assistants with generation-calibrated retrievers. [find](https://scholar.google.com/scholar?q=Sheshera+Mysore%2C+Zhuoran+Lu%2C+Mengting+Wan%2C+Longqi+Yang%2C+Steve+Menezes%2C+Tina+Baghaee%2C+Emmanuel+Barajas+Gonzalez%2C+Jennifer+Neville%2C+and+Tara+Safavi%2E+2023%2E+Pearl%3A+Personalizing+large+language+model+writi)
86. Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. [StereoSet: Measuring stereotypical bias in pretrained language models](https://md.research.elamur.ai/free/en/papers/nadeem-2021-stereoset-measuring-stereotypical-bias-pretrained/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long…
87. Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020. [CrowS-pairs: A challenge dataset for measuring social biases in masked language models](https://md.research.elamur.ai/free/en/papers/nangia-2020-crows-pairs-challenge-dataset-measuring/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1953–1967, Online. Association for…
88. Tarek Naous, Michael J. Ryan, Alan Ritter, and Wei Xu. 2023. [Having beer after prayer? measuring cultural bias in large language models](https://md.research.elamur.ai/free/en/papers/naous-2023-having-beer-after-prayer-measuring/).
89. Tuan-Phong Nguyen, Simon Razniewski, Aparna Varde, and Gerhard Weikum. 2023. Extracting cultural commonsense knowledge at scale. In Proceedings of the ACM Web Conference 2023, WWW ’23. ACM. [find](https://scholar.google.com/scholar?q=Tuan%2DPhong+Nguyen%2C+Simon+Razniewski%2C+Aparna+Varde%2C+and+Gerhard+Weikum%2E+2023%2E+Extracting+cultural+commonsense+knowledge+at+scale%2E+In+Proceedings+of+the+ACM+Web+Conference+2023%2C+WWW+%E2%80%9923%2E+ACM%2E)
90. Tuan-Phong Nguyen, Simon Razniewski, and Gerhard Weikum. 2024. Multi-cultural commonsense knowledge distillation. [find](https://scholar.google.com/scholar?q=Tuan%2DPhong+Nguyen%2C+Simon+Razniewski%2C+and+Gerhard+Weikum%2E+2024%2E+Multi%2Dcultural+commonsense+knowledge+distillation%2E)
91. Jon O’Brien, Tom Rodden, Mark Rouncefield, and John A. Hughes. 1999. At home with the technology: an ethnographic study of a set-top-box trial. ACM Trans. Comput. Hum. Interact., 6(3):282–308. [find](https://scholar.google.com/scholar?q=Jon+O%E2%80%99Brien%2C+Tom+Rodden%2C+Mark+Rouncefield%2C+and+John+A%2E+Hughes%2E+1999%2E+At+home+with+the+technology%3A+an+ethnographic+study+of+a+set%2Dtop%2Dbox+trial%2E+ACM+Trans%2E+Comput%2E+Hum%2E+Interact%2E%2C+6%283%29%3A282%E2%80%93308%2E)
92. Millicent Ochieng, Varun Gumma, Sunayana Sitaram, Jindong Wang, Vishrav Chaudhary, Keshet Ronen, Kalika Bali, and Jacki O’Neill. 2024. Beyond metrics: Evaluating llms’ effectiveness in culturally nuanced, low-resource real-world scenarios. [find](https://scholar.google.com/scholar?q=Millicent+Ochieng%2C+Varun+Gumma%2C+Sunayana+Sitaram%2C+Jindong+Wang%2C+Vishrav+Chaudhary%2C+Keshet+Ronen%2C+Kalika+Bali%2C+and+Jacki+O%E2%80%99Neill%2E+2024%2E+Beyond+metrics%3A+Evaluating+llms%E2%80%99+effectiveness+in+culturally+nuan)
93. Shramay Palta and Rachel Rudinger. 2023. [FORK: A bite-sized test set for probing culinary cultural biases in commonsense reasoning models](https://md.research.elamur.ai/free/en/papers/palta-2023-fork-bite-sized-test-set/). In Findings of the Association for Computational Linguistics: ACL 2023, pages 9952–9962, Toronto, Canada. Association for Computational Linguistics.
94. Talcott Parsons. 1972. Culture and social system revisited. Social Science Quarterly, pages 253–266. [find](https://scholar.google.com/scholar?q=Talcott+Parsons%2E+1972%2E+Culture+and+social+system+revisited%2E+Social+Science+Quarterly%2C+pages+253%E2%80%93266%2E)
95. Jiaxin Pei and David Jurgens. 2023. When do annotator demographics matter? measuring the influence of annotator demographics with the POPQUORN dataset. In Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII), pages 252–265, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jiaxin+Pei+and+David+Jurgens%2E+2023%2E+When+do+annotator+demographics+matter%3F+measuring+the+influence+of+annotator+demographics+with+the+POPQUORN+dataset%2E+In+Proceedings+of+the+17th+Linguistic+Annotation)
96. Rifki Afina Putri, Faiz Ghifari Haznitrama, Dea Adhista, and Alice Oh. 2024. Can llm generate culturally relevant commonsense qa data? case study in indonesian and sundanese. [find](https://scholar.google.com/scholar?q=Rifki+Afina+Putri%2C+Faiz+Ghifari+Haznitrama%2C+Dea+Adhista%2C+and+Alice+Oh%2E+2024%2E+Can+llm+generate+culturally+relevant+commonsense+qa+data%3F+case+study+in+indonesian+and+sundanese%2E)
97. Jun Quan, Shian Zhang, Qian Cao, Zizhong Li, and Deyi Xiong. 2020. RiSAWOZ: A large-scale multi-domain Wizard-of-Oz dataset with rich semantic annotations for task-oriented dialogue modeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 930… [find](https://scholar.google.com/scholar?q=Jun+Quan%2C+Shian+Zhang%2C+Qian+Cao%2C+Zizhong+Li%2C+and+Deyi+Xiong%2E+2020%2E+RiSAWOZ%3A+A+large%2Dscale+multi%2Ddomain+Wizard%2Dof%2DOz+dataset+with+rich+semantic+annotations+for+task%2Doriented+dialogue+modeling%2E+In+Proce)
98. Sunny Rai, Khushang Jilesh Zaveri, Shreya Havaldar, Soumna Nema, Lyle Ungar, and Sharath Chandra Guntuku. 2024. A cross-cultural analysis of social norms in bollywood and hollywood movies. [find](https://scholar.google.com/scholar?q=Sunny+Rai%2C+Khushang+Jilesh+Zaveri%2C+Shreya+Havaldar%2C+Soumna+Nema%2C+Lyle+Ungar%2C+and+Sharath+Chandra+Guntuku%2E+2024%2E+A+cross%2Dcultural+analysis+of+social+norms+in+bollywood+and+hollywood+movies%2E)
99. Aida Ramezani and Yang Xu. 2023. [Knowledge of cultural moral norms in large language models](https://md.research.elamur.ai/free/en/papers/ramezani-2023-knowledge-cultural-moral-norms-large/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 428–446, Toronto, Canada. Association for Computational Linguistics.
100. Abhinav Rao, Aditi Khandelwal, Kumar Tanmay, Utkarsh Agarwal, and Monojit Choudhury. 2023a. Ethical reasoning over moral alignment: A case and framework for in-context ethical policies in LLMs. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13370–13388, Singapore. As… [find](https://scholar.google.com/scholar?q=Abhinav+Rao%2C+Aditi+Khandelwal%2C+Kumar+Tanmay%2C+Utkarsh+Agarwal%2C+and+Monojit+Choudhury%2E+2023a%2E+Ethical+reasoning+over+moral+alignment%3A+A+case+and+framework+for+in%2Dcontext+ethical+policies+in+LLMs%2E+In+Fin)
101. Kavel Rao, Liwei Jiang, Valentina Pyatkin, Yuling Gu, Niket Tandon, Nouha Dziri, Faeze Brahman, and Yejin Choi. 2023b. What makes it ok to set a fire? iterative self-distillation of contexts and rationales for disambiguating defeasible social and moral situations. [find](https://scholar.google.com/scholar?q=Kavel+Rao%2C+Liwei+Jiang%2C+Valentina+Pyatkin%2C+Yuling+Gu%2C+Niket+Tandon%2C+Nouha+Dziri%2C+Faeze+Brahman%2C+and+Yejin+Choi%2E+2023b%2E+What+makes+it+ok+to+set+a+fire%3F+iterative+self%2Ddistillation+of+contexts+and+ratio)
102. J.R. Rest and L. Kohlberg. 1979. Development in Judging Moral Issues. University of Minnesota Press. [find](https://scholar.google.com/scholar?q=J%2ER%2E+Rest+and+L%2E+Kohlberg%2E+1979%2E+Development+in+Judging+Moral+Issues%2E+University+of+Minnesota+Press%2E)
103. Dor Ringel, Gal Lavee, Ido Guy, and Kira Radinsky. 2019. Cross-cultural transfer learning for text classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages… [find](https://scholar.google.com/scholar?q=Dor+Ringel%2C+Gal+Lavee%2C+Ido+Guy%2C+and+Kira+Radinsky%2E+2019%2E+Cross%2Dcultural+transfer+learning+for+text+classification%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Proces)
104. Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, Tulsee Doshi, and Vinodkumar Prabhakaran. 2021. Re-imagining algorithmic fairness in india and beyond. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 315–328, New York, NY, USA. Association for… [find](https://scholar.google.com/scholar?q=Nithya+Sambasivan%2C+Erin+Arnesen%2C+Ben+Hutchinson%2C+Tulsee+Doshi%2C+and+Vinodkumar+Prabhakaran%2E+2021%2E+Re%2Dimagining+algorithmic+fairness+in+india+and+beyond%2E+In+Proceedings+of+the+2021+ACM+Conference+on+Fai)
105. Sandra Sandoval, Jieyu Zhao, Marine Carpuat, and Hal Daumé III. 2023. A rose by any other name would not smell as sweet: Social bias in names mistranslation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3933–3945, Singapore. Association for Computa… [find](https://scholar.google.com/scholar?q=Sandra+Sandoval%2C+Jieyu+Zhao%2C+Marine+Carpuat%2C+and+Hal+Daum%C3%A9+III%2E+2023%2E+A+rose+by+any+other+name+would+not+smell+as+sweet%3A+Social+bias+in+names+mistranslation%2E+In+Proceedings+of+the+2023+Conference+on+E)
106. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. [Whose opinions do language models reflect?](https://md.research.elamur.ai/free/en/papers/santurkar-2023-whose-opinions-language-models-reflect/)
107. Sebastin Santy, Jenny Liang, Ronan Le Bras, Katharina Reinecke, and Maarten Sap. 2023. [NLPositionality: Characterizing design biases of datasets and models](https://md.research.elamur.ai/free/en/papers/santy-2023-nlpositionality-characterizing-design-biases-datasets/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9080–9102, Toronto, C…
108. Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022. [Annotators with attitudes: How annotator beliefs and identities bias toxic language detection](https://md.research.elamur.ai/free/en/papers/sap-2022-annotators-attitudes-how-annotator-beliefs/). In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Ling…
109. Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin A. Rothkopf, and Kristian Kersting. 2021. Large pre-trained language models contain human-like biases of what is right and wrong to do. Nature Machine Intelligence, 4:258 – 268. [find](https://scholar.google.com/scholar?q=Patrick+Schramowski%2C+Cigdem+Turan%2C+Nico+Andersen%2C+Constantin+A%2E+Rothkopf%2C+and+Kristian+Kersting%2E+2021%2E+Large+pre%2Dtrained+language+models+contain+human%2Dlike+biases+of+what+is+right+and+wrong+to+do%2E+Nat)
110. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2023. Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. [find](https://scholar.google.com/scholar?q=Melanie+Sclar%2C+Yejin+Choi%2C+Yulia+Tsvetkov%2C+and+Alane+Suhr%2E+2023%2E+Quantifying+language+models%E2%80%99+sensitivity+to+spurious+features+in+prompt+design+or%3A+How+i+learned+to+start+worrying+about+prompt+formatt)
111. Omar Shaikh, Caleb Ziems, William Held, Aryan Pariani, Fred Morstatter, and Diyi Yang. 2023. [Modeling cross-cultural pragmatic inference with codenames duet](https://md.research.elamur.ai/free/en/papers/shaikh-2023-modeling-cross-cultural-pragmatic-inference/). In Findings of the Association for Computational Linguistics: ACL 2023, pages 6550–6569, Toronto, Canada. Association for Computational Lingui…
112. Weiyan Shi, Ryan Li, Yutong Zhang, Caleb Ziems, Chunhua yu, Raya Horesh, Rogério Abreu de Paula, and Diyi Yang. 2024. [Culturebank: An online community-driven knowledge base towards culturally aware language technologies](https://md.research.elamur.ai/free/en/papers/shi-2024-culturebank-online-community-driven-knowledge/).
113. Richard Shweder, Jacqueline Goodnow, Giyoo Hatano, Robert LeVine, Hazel Markus, and Peggy Miller. 2007. The Cultural Psychology of Development: One Mind, Many Mentalities, volume 1. [find](https://scholar.google.com/scholar?q=Richard+Shweder%2C+Jacqueline+Goodnow%2C+Giyoo+Hatano%2C+Robert+LeVine%2C+Hazel+Markus%2C+and+Peggy+Miller%2E+2007%2E+The+Cultural+Psychology+of+Development%3A+One+Mind%2C+Many+Mentalities%2C+volume+1%2E)
114. Sunayana Sitaram, Monojit Choudhury, Barun Patra, Vishrav Chaudhary, Kabir Ahuja, and Kalika Bali. 2023. Everything you need to know about multilingual LLMs: Towards fair, performant and reliable models for languages of the world. In Proceedings of the 61st Annual Meeting of the Association for Comp… [find](https://scholar.google.com/scholar?q=Sunayana+Sitaram%2C+Monojit+Choudhury%2C+Barun+Patra%2C+Vishrav+Chaudhary%2C+Kabir+Ahuja%2C+and+Kalika+Bali%2E+2023%2E+Everything+you+need+to+know+about+multilingual+LLMs%3A+Towards+fair%2C+performant+and+reliable+mode)
115. Guijin Son, Hanwool Lee, Sungdong Kim, Seungone Kim, Niklas Muennighoff, Taekyoon Choi, Cheonbok Park, Kang Min Yoo, and Stella Biderman. 2024. [Kmmlu: Measuring massive multitask language understanding in korean](https://md.research.elamur.ai/free/en/papers/son-2025-kmmlu-measuring-massive-multitask-language/).
116. Taylor Sorensen, Liwei Jiang, Jena Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, Maarten Sap, John Tasioulas, and Yejin Choi. 2023. Value kaleidoscope: Engaging ai with pluralistic human values, rights, and duties. [find](https://scholar.google.com/scholar?q=Taylor+Sorensen%2C+Liwei+Jiang%2C+Jena+Hwang%2C+Sydney+Levine%2C+Valentina+Pyatkin%2C+Peter+West%2C+Nouha+Dziri%2C+Ximing+Lu%2C+Kavel+Rao%2C+Chandra+Bhagavatula%2C+Maarten+Sap%2C+John+Tasioulas%2C+and+Yejin+Choi%2E+2023%2E+Value)
117. Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, and Adina Williams. 2021. A word on machine ethics: A response to jiang et al. (2021). [find](https://scholar.google.com/scholar?q=Zeerak+Talat%2C+Hagen+Blix%2C+Josef+Valvoda%2C+Maya+Indira+Ganesh%2C+Ryan+Cotterell%2C+and+Adina+Williams%2E+2021%2E+A+word+on+machine+ethics%3A+A+response+to+jiang+et+al%2E+%282021%29%2E)
118. Kumar Tanmay, Aditi Khandelwal, Utkarsh Agarwal, and Monojit Choudhury. 2023. Probing the moral development of large language models through defining issues test. [find](https://scholar.google.com/scholar?q=Kumar+Tanmay%2C+Aditi+Khandelwal%2C+Utkarsh+Agarwal%2C+and+Monojit+Choudhury%2E+2023%2E+Probing+the+moral+development+of+large+language+models+through+defining+issues+test%2E)
119. Bill Thompson, Se’an G. Roberts, and Gary Lupyan. 2020. Cultural influences on word meanings revealed through large-scale semantic alignment. Nature Human Behaviour, 4(10):1029–1038. [find](https://scholar.google.com/scholar?q=Bill+Thompson%2C+Se%E2%80%99an+G%2E+Roberts%2C+and+Gary+Lupyan%2E+2020%2E+Cultural+influences+on+word+meanings+revealed+through+large%2Dscale+semantic+alignment%2E+Nature+Human+Behaviour%2C+4%2810%29%3A1029%E2%80%931038%2E)
120. Silvia Vaccino-Salvadore. 2023. Exploring the ethical dimensions of using chatgpt in language learning and beyond. Languages, 8(3). [find](https://scholar.google.com/scholar?q=Silvia+Vaccino%2DSalvadore%2E+2023%2E+Exploring+the+ethical+dimensions+of+using+chatgpt+in+language+learning+and+beyond%2E+Languages%2C+8%283%29%2E)
121. Mor Ventura, Eyal Ben-David, Anna Korhonen, and Roi Reichart. 2023. Navigating cultural chasms: Exploring and unlocking the cultural pov of text-to-image models. [find](https://scholar.google.com/scholar?q=Mor+Ventura%2C+Eyal+Ben%2DDavid%2C+Anna+Korhonen%2C+and+Roi+Reichart%2E+2023%2E+Navigating+cultural+chasms%3A+Exploring+and+unlocking+the+cultural+pov+of+text%2Dto%2Dimage+models%2E)
122. Yixin Wan, Jieyu Zhao, Aman Chadha, Nanyun Peng, and Kai-Wei Chang. 2023. Are personalized stochastic parrots more dangerous? evaluating persona biases in dialogue systems. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9677–9705, Singapore. Association for Computati… [find](https://scholar.google.com/scholar?q=Yixin+Wan%2C+Jieyu+Zhao%2C+Aman+Chadha%2C+Nanyun+Peng%2C+and+Kai%2DWei+Chang%2E+2023%2E+Are+personalized+stochastic+parrots+more+dangerous%3F+evaluating+persona+biases+in+dialogue+systems%2E+In+Findings+of+the+Associat)
123. Bin Wang, Zhengyuan Liu, Xin Huang, Fangkai Jiao, Yang Ding, Ai Ti Aw, and Nancy F. Chen. 2023. [Seaeval for multilingual foundation models: From cross-lingual alignment to cultural reasoning](https://md.research.elamur.ai/free/en/papers/wang-2024-seaeval-multilingual-foundation-models-cross/).
124. Leslie A. White. 1959. The concept of culture*. American Anthropologist, 61(2):227–251. [find](https://scholar.google.com/scholar?q=Leslie+A%2E+White%2E+1959%2E+The+concept+of+culture%2A%2E+American+Anthropologist%2C+61%282%29%3A227%E2%80%93251%2E)
125. Haryo Akbarianto Wibowo, Erland Hilman Fuadi, Made Nindyatama Nityasya, Radityo Eko Prasojo, and Alham Fikri Aji. 2023. Copal-id: Indonesian language reasoning with local culture and nuances. [find](https://scholar.google.com/scholar?q=Haryo+Akbarianto+Wibowo%2C+Erland+Hilman+Fuadi%2C+Made+Nindyatama+Nityasya%2C+Radityo+Eko+Prasojo%2C+and+Alham+Fikri+Aji%2E+2023%2E+Copal%2Did%3A+Indonesian+language+reasoning+with+local+culture+and+nuances%2E)
126. Nevan Wichers, Carson Denison, and Ahmad Beirami. 2024. Gradient-based language model red teaming. [find](https://scholar.google.com/scholar?q=Nevan+Wichers%2C+Carson+Denison%2C+and+Ahmad+Beirami%2E+2024%2E+Gradient%2Dbased+language+model+red+teaming%2E)
127. Winston Wu, Lu Wang, and Rada Mihalcea. 2023. Cross-cultural analysis of human values, morals, and biases in folk tales. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5113–5125, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Winston+Wu%2C+Lu+Wang%2C+and+Rada+Mihalcea%2E+2023%2E+Cross%2Dcultural+analysis+of+human+values%2C+morals%2C+and+biases+in+folk+tales%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+)
128. Charles Yu, Sullam Jeoung, Anish Kasi, Pengfei Yu, and Heng Ji. 2023. Unlearning bias in language models by partitioning gradients. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6032–6048, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Charles+Yu%2C+Sullam+Jeoung%2C+Anish+Kasi%2C+Pengfei+Yu%2C+and+Heng+Ji%2E+2023%2E+Unlearning+bias+in+language+models+by+partitioning+gradients%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+20)
129. Haolan Zhan, Zhuang Li, Xiaoxi Kang, Tao Feng, Yuncheng Hua, Lizhen Qu, Yi Ying, Mei Rianto Chandra, Kelly Rosalin, Jureynolds Jureynolds, Suraj Sharma, Shilin Qu, Linhao Luo, Lay-Ki Soon, Zhaleh Semnani Azad, Ingrid Zukerman, and Gholamreza Haffari. 2024. Renovi: A benchmark towards remediating nor… [find](https://scholar.google.com/scholar?q=Haolan+Zhan%2C+Zhuang+Li%2C+Xiaoxi+Kang%2C+Tao+Feng%2C+Yuncheng+Hua%2C+Lizhen+Qu%2C+Yi+Ying%2C+Mei+Rianto+Chandra%2C+Kelly+Rosalin%2C+Jureynolds+Jureynolds%2C+Suraj+Sharma%2C+Shilin+Qu%2C+Linhao+Luo%2C+Lay%2DKi+Soon%2C+Zhaleh+Semn)
130. Haolan Zhan, Zhuang Li, Yufei Wang, Linhao Luo, Tao Feng, Xiaoxi Kang, Yuncheng Hua, Lizhen Qu, Lay-Ki Soon, Suraj Sharma, Ingrid Zukerman, Zhaleh Semnani-Azad, and Gholamreza Haffari. 2023. Socialdial: A benchmark for socially-aware dialogue systems. In Proceedings of the 46th International ACM SIG… [find](https://scholar.google.com/scholar?q=Haolan+Zhan%2C+Zhuang+Li%2C+Yufei+Wang%2C+Linhao+Luo%2C+Tao+Feng%2C+Xiaoxi+Kang%2C+Yuncheng+Hua%2C+Lizhen+Qu%2C+Lay%2DKi+Soon%2C+Suraj+Sharma%2C+Ingrid+Zukerman%2C+Zhaleh+Semnani%2DAzad%2C+and+Gholamreza+Haffari%2E+2023%2E+Socialdia)
131. Chiyu Zhang, Khai Doan, Qisheng Liao, and Muhammad Abdul-Mageed. 2023. The skipped beat: A study of sociopragmatic understanding in LLMs for 64 languages. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2630–2662, Singapore. Association for Computatio… [find](https://scholar.google.com/scholar?q=Chiyu+Zhang%2C+Khai+Doan%2C+Qisheng+Liao%2C+and+Muhammad+Abdul%2DMageed%2E+2023%2E+The+skipped+beat%3A+A+study+of+sociopragmatic+understanding+in+LLMs+for+64+languages%2E+In+Proceedings+of+the+2023+Conference+on+Empi)
132. Wenlong Zhao, Debanjan Mondal, Niket Tandon, Danica Dillion, Kurt Gray, and Yuling Gu. 2024. World-ValuesBench: A large-scale benchmark dataset for multi-cultural value awareness of language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Reso… [find](https://scholar.google.com/scholar?q=Wenlong+Zhao%2C+Debanjan+Mondal%2C+Niket+Tandon%2C+Danica+Dillion%2C+Kurt+Gray%2C+and+Yuling+Gu%2E+2024%2E+World%2DValuesBench%3A+A+large%2Dscale+benchmark+dataset+for+multi%2Dcultural+value+awareness+of+language+models%2E+I)
133. Li Zhou, Laura Cabello, Yong Cao, and Daniel Hershcovich. 2023a. Cross-cultural transfer learning for Chinese offensive language detection. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 8–15, Dubrovnik, Croatia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Li+Zhou%2C+Laura+Cabello%2C+Yong+Cao%2C+and+Daniel+Hershcovich%2E+2023a%2E+Cross%2Dcultural+transfer+learning+for+Chinese+offensive+language+detection%2E+In+Proceedings+of+the+First+Workshop+on+Cross%2DCultural+Consi)
134. Binwei Yao, Ming Jiang, Diyi Yang, and Junjie Hu. 2024. Benchmarking llm-based machine translation on cultural awareness. [find](https://scholar.google.com/scholar?q=Binwei+Yao%2C+Ming+Jiang%2C+Diyi+Yang%2C+and+Junjie+Hu%2E+2024%2E+Benchmarking+llm%2Dbased+machine+translation+on+cultural+awareness%2E)
135. Li Zhou, Antonia Karamolegkou, Wenyu Chen, and Daniel Hershcovich. 2023b. Cultural compass: Predicting transfer learning success in offensive language detection with cultural features. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12684–12702, Singapore. Association… [find](https://scholar.google.com/scholar?q=Li+Zhou%2C+Antonia+Karamolegkou%2C+Wenyu+Chen%2C+and+Daniel+Hershcovich%2E+2023b%2E+Cultural+compass%3A+Predicting+transfer+learning+success+in+offensive+language+detection+with+cultural+features%2E+In+Findings+of+)
136. Jing Yao, Xiaoyuan Yi, Xiting Wang, Jindong Wang, and Xing Xie. 2023. From instructions to intrinsic human values – a survey of alignment goals for big models. [find](https://scholar.google.com/scholar?q=Jing+Yao%2C+Xiaoyuan+Yi%2C+Xiting+Wang%2C+Jindong+Wang%2C+and+Xing+Xie%2E+2023%2E+From+instructions+to+intrinsic+human+values+%E2%80%93+a+survey+of+alignment+goals+for+big+models%2E)
137. Li Zhou, Taelin Karidi, Nicolas Garneau, Yong Cao, Wanlong Liu, Wenyu Chen, and Daniel Hershcovich. 2024. Does mapo tofu contain coffee? probing llms for food-related cultural knowledge. [find](https://scholar.google.com/scholar?q=Li+Zhou%2C+Taelin+Karidi%2C+Nicolas+Garneau%2C+Yong+Cao%2C+Wanlong+Liu%2C+Wenyu+Chen%2C+and+Daniel+Hershcovich%2E+2024%2E+Does+mapo+tofu+contain+coffee%3F+probing+llms+for+food%2Drelated+cultural+knowledge%2E)
138. Caleb Ziems, Jane Dwivedi-Yu, Yi-Chia Wang, Alon Halevy, and Diyi Yang. 2023. [NormBank: A knowledge bank of situational social norms](https://md.research.elamur.ai/free/en/papers/ziems-2023-normbank-knowledge-bank-situational-social/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7756–7776, Toronto, Canada. Association for…

*Updated 24 October 2026*