---
title: "CultureBank — paper by Shi et al. (2024) — summary and ideas"
description: "The authors build a scalable, bottom-up pipeline that turns noisy personal stories from online… 63 references, cited by 69 papers here."
authors: ["Weiyan Shi", "Ryan Li", "Yutong Zhang", "Caleb Ziems", "Sunny Yu", "Raya Horesh", "Rogério Abreu De Paula", "Diyi Yang"]
year: 2024
venue: "Findings of the Association for Computational Linguistics: EMNLP 2024"
source: "https://aclanthology.org/2024.findings-emnlp.288/"
doi: "10.18653/v1/2024.findings-emnlp.288"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shi-2024-culturebank-online-community-driven-knowledge/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shi-2024-culturebank-online-community-driven-knowledge/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies

# CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies

**Authors:** [Weiyan Shi](https://md.research.elamur.ai/free/en/authors/weiyan-shi/), Ryan Li, Yutong Zhang, [Caleb Ziems](https://md.research.elamur.ai/free/en/authors/caleb-ziems/), Sunny Yu, Raya Horesh, Rogério Abreu De Paula and [Diyi Yang](https://md.research.elamur.ai/free/en/authors/diyi-yang/) · **Year:** 2024 · **Venue:** [Findings of the Association for Computational Linguistics: EMNLP 2024](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2024](https://md.research.elamur.ai/free/en/venues/findings/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.findings-emnlp.288/) · [PDF](https://aclanthology.org/2024.findings-emnlp.288.pdf) · [DOI 10.18653/v1/2024.findings-emnlp.288](https://doi.org/10.18653/v1/2024.findings-emnlp.288) · [OpenAlex](https://openalex.org/W4404782164)

**Citations:** 19 citations (OpenAlex, 2026-10-14) · [69](https://md.research.elamur.ai/free/en/papers/shi-2024-culturebank-online-community-driven-knowledge/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
LLMs tend to reflect Western-centric values. Existing cultural knowledge resources draw mostly on formal sources such as Wikipedia or C4, state norms as fixed facts, and are evaluated through classification or QA tasks that do not resemble real use. The authors build a scalable, bottom-up pipeline that turns noisy personal stories from online communities into structured cultural knowledge. The pipeline has three stages. It first filters culture-related comments and extracts descriptors into an 11-field taxonomy. It then clusters and summarizes those descriptors. Finally, it computes how many commenters agree and removes toxic content and personal information. Human checks validate each stage. The result, CultureBank, holds about 12K descriptors from TikTok and 11K from Reddit. Each descriptor comes with an agreement score and a realistic scenario for grounded testing. Evaluations using these scenarios show that current LLMs still have gaps in cultural awareness. A model fine-tuned on CultureBank improves zero-shot on two downstream cultural tasks. The work matters because it adds descriptive, community-level cultural knowledge alongside existing normative resources.

## Key points
- **CultureBank size:** roughly 12K descriptors of culture drawn from TikTok, plus about 11K drawn from Reddit. The Reddit set demonstrates that the pipeline transfers across platforms.
- **Taxonomy (11 fields):** cultural group, context, goal, actor, recipient, relation, actor's behavior, recipient's behavior, other description, topic, and agreement (the share of commenters in a cluster who agree).
- **Pipeline steps:**
  - a cultural relevance classifier and a descriptor extractor;
  - clustering with a cluster summarizer (Mixtral);
  - an agreement calculator;
  - moderation using Perspective API (score \< 0.2) plus a classifier, and PII removal.
- **Evaluation design:** each descriptor is paired with a grounded scenario, persona, and question. This supports realistic evaluation, for example a travel consultation.
- **Limitations and availability:**
  - The data is English-only.
  - Sampling bias comes from hashtags such as "culture shock".
  - Data, code, and models are available at github.com/SALT-NLP/CultureBank.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 310 works · 284,480 citations
- [Northeastern University](https://md.research.elamur.ai/free/en/universities/northeastern-university/) — University, Boston, United States · 28 works · 6,164 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🕸️ Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/) · 103 works

## Cite

Cite as: Weiyan Shi, Ryan Li, Yutong Zhang, Caleb Ziems, Sunny Yu, Raya Horesh, Rogério Abreu De Paula and Diyi Yang. 2024. CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies. Findings of the Association for Computational Linguistics: EMNLP 2024.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shi2024culturebank,
  title = {{CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies}},
  author = {Weiyan Shi, Ryan Li, Yutong Zhang, Caleb Ziems, Sunny Yu, Raya Horesh, Rogério Abreu {De Paula} and Diyi Yang},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2024},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.288},
  url = {https://doi.org/10.18653/v1/2024.findings-emnlp.288}
}
```

- [← previous “My Answer is C”: First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models](https://md.research.elamur.ai/free/en/papers/wang-2024-my-answer-is-c-first/)
- [next → Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models](https://md.research.elamur.ai/free/en/papers/lu-2024-not-all-experts-are-equal/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

63 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Sodiq Adewole, Erfaneh Gharavi, Benjamin Shpringer, Martin Bolger, Vaibhav Sharma, Sung Ming Yang, and Donald E. Brown. 2021. Dialogue-based simulation for cultural awareness training. [find](https://scholar.google.com/scholar?q=Sodiq+Adewole%2C+Erfaneh+Gharavi%2C+Benjamin+Shpringer%2C+Martin+Bolger%2C+Vaibhav+Sharma%2C+Sung+Ming+Yang%2C+and+Donald+E%2E+Brown%2E+2021%2E+Dialogue%2Dbased+simulation+for+cultural+awareness+training%2E)
3. Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Ashutosh Dwivedi, Alham Fikri Aji, Jacki O’Neill, Ashutosh Modi, and Monojit Choudhury. 2024. [Towards measuring and modeling" culture" in llms: A survey](https://md.research.elamur.ai/free/en/papers/adilazuarda-2024-measuring-modeling-culture-llms-survey/). arXiv preprint arXiv:2403.15412.
4. Rifki Afina Putri, Faiz Ghifari Haznitrama, Dea Adhista, and Alice Oh. 2024. Can llm generate culturally relevant commonsense qa data? case study in indonesian and sundanese. arXiv e-prints, pages arXiv–2402. [find](https://scholar.google.com/scholar?q=Rifki+Afina+Putri%2C+Faiz+Ghifari+Haznitrama%2C+Dea+Adhista%2C+and+Alice+Oh%2E+2024%2E+Can+llm+generate+culturally+relevant+commonsense+qa+data%3F+case+study+in+indonesian+and+sundanese%2E+arXiv+e%2Dprints%2C+pages+arX)
5. Arnav Arora, Lucie-Aimée Kaffee, and Isabelle Augenstein. 2022. [Probing pre-trained language models for cross-cultural differences in values](https://md.research.elamur.ai/free/en/papers/arora-2023-probing-pre-trained-language-models/). arXiv preprint arXiv:2203.13722.
6. Fredrik Barth. 2010. Introduction to ethnic groups and boundaries: The social organization of cultural difference. Selected studies in international migration and immigrant incorporation, 1:407. [find](https://scholar.google.com/scholar?q=Fredrik+Barth%2E+2010%2E+Introduction+to+ethnic+groups+and+boundaries%3A+The+social+organization+of+cultural+difference%2E+Selected+studies+in+international+migration+and+immigrant+incorporation%2C+1%3A407%2E)
7. Aliaksandr Birukou, Enrico Blanzieri, Paolo Giorgini, and Fausto Giunchiglia. 2013. A formal definition of culture. Models for intercultural collaboration and negotiation, pages 1–26. [find](https://scholar.google.com/scholar?q=Aliaksandr+Birukou%2C+Enrico+Blanzieri%2C+Paolo+Giorgini%2C+and+Fausto+Giunchiglia%2E+2013%2E+A+formal+definition+of+culture%2E+Models+for+intercultural+collaboration+and+negotiation%2C+pages+1%E2%80%9326%2E)
8. Robert Boyd and Peter J Richerson. 1988. Culture and the evolutionary process. University of Chicago press. [find](https://scholar.google.com/scholar?q=Robert+Boyd+and+Peter+J+Richerson%2E+1988%2E+Culture+and+the+evolutionary+process%2E+University+of+Chicago+press%2E)
9. Adda Bruemmer Bozeman. 1992. Strategic intelligence & statecraft: Selected essays. (No Title). [find](https://scholar.google.com/scholar?q=Adda+Bruemmer+Bozeman%2E+1992%2E+Strategic+intelligence+%26+statecraft%3A+Selected+essays%2E+%28No+Title%29%2E)
10. Marilynn B Brewer. 1999. The psychology of prejudice: Ingroup love and outgroup hate? Journal of social issues, 55(3):429–444. [find](https://scholar.google.com/scholar?q=Marilynn+B+Brewer%2E+1999%2E+The+psychology+of+prejudice%3A+Ingroup+love+and+outgroup+hate%3F+Journal+of+social+issues%2C+55%283%29%3A429%E2%80%93444%2E)
11. Christoph Brumann. 1999. Writing for culture: Why a successful concept should not be discarded. Current anthropology, 40(S1):S1–S27. [find](https://scholar.google.com/scholar?q=Christoph+Brumann%2E+1999%2E+Writing+for+culture%3A+Why+a+successful+concept+should+not+be+discarded%2E+Current+anthropology%2C+40%28S1%29%3AS1%E2%80%93S27%2E)
12. Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. [Assessing cross-cultural alignment between chatgpt and human societies: An empirical study](https://md.research.elamur.ai/free/en/papers/cao-2023-assessing-cross-cultural-alignment-between/).
13. Awantee Deshpande, Dana Ruiter, Marius Mosbach, and Dietrich Klakow. 2022. Stereokg: Data-driven knowledge graph construction for cultural knowledge and stereotypes. arXiv preprint arXiv:2205.14036. [find](https://scholar.google.com/scholar?q=Awantee+Deshpande%2C+Dana+Ruiter%2C+Marius+Mosbach%2C+and+Dietrich+Klakow%2E+2022%2E+Stereokg%3A+Data%2Ddriven+knowledge+graph+construction+for+cultural+knowledge+and+stereotypes%2E+arXiv+preprint+arXiv%3A2205%2E14036%2E)
14. Esin Durmus, Karina Nyugen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2023a. [Towards measuring…](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/)
15. Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023b. [Towards measuring the representation of subjective global opinions in language models](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/). arXiv preprint arXiv:2306.16388.
16. Ashutosh Dwivedi, Pradhyumna Lavania, and Ashutosh Modi. 2023. EtiCor: Corpus for analyzing LLMs for etiquettes. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6921–6931, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ashutosh+Dwivedi%2C+Pradhyumna+Lavania%2C+and+Ashutosh+Modi%2E+2023%2E+EtiCor%3A+Corpus+for+analyzing+LLMs+for+etiquettes%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Processi)
17. Wenqi Fan, Zihuai Zhao, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Jiliang Tang, and Qing Li. 2023. Recommender systems in the era of large language models (llms). arXiv preprint arXiv:2307.02046. [find](https://scholar.google.com/scholar?q=Wenqi+Fan%2C+Zihuai+Zhao%2C+Jiatong+Li%2C+Yunqing+Liu%2C+Xiaowei+Mei%2C+Yiqi+Wang%2C+Jiliang+Tang%2C+and+Qing+Li%2E+2023%2E+Recommender+systems+in+the+era+of+large+language+models+%28llms%29%2E+arXiv+preprint+arXiv%3A2307%2E0204)
18. Yi Fung, Tuhin Chakrabarty, Hao Guo, Owen Rambow, Smaranda Muresan, and Heng Ji. 2023. NORM-SAGE: Multi-lingual multi-cultural norm discovery from conversations on-the-fly. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15217–15230, Singapore. Associ… [find](https://scholar.google.com/scholar?q=Yi+Fung%2C+Tuhin+Chakrabarty%2C+Hao+Guo%2C+Owen+Rambow%2C+Smaranda+Muresan%2C+and+Heng+Ji%2E+2023%2E+NORM%2DSAGE%3A+Multi%2Dlingual+multi%2Dcultural+norm+discovery+from+conversations+on%2Dthe%2Dfly%2E+In+Proceedings+of+the+2023+)
19. Yi Fung, Ruining Zhao, Jae Doo, Chenkai Sun, and Heng Ji. 2024. Massively multi-cultural knowledge acquisition and lm benchmarking. arXiv preprint arXiv:2402.09369. [find](https://scholar.google.com/scholar?q=Yi+Fung%2C+Ruining+Zhao%2C+Jae+Doo%2C+Chenkai+Sun%2C+and+Heng+Ji%2E+2024%2E+Massively+multi%2Dcultural+knowledge+acquisition+and+lm+benchmarking%2E+arXiv+preprint+arXiv%3A2402%2E09369%2E)
20. Denise R Gehring and Lindsey Sinnott. 2018. ehraf world cultures. The Charleston Advisor, 20(1):36– 40. [find](https://scholar.google.com/scholar?q=Denise+R+Gehring+and+Lindsey+Sinnott%2E+2018%2E+ehraf+world+cultures%2E+The+Charleston+Advisor%2C+20%281%29%3A36%E2%80%93+40%2E)
21. Erving Goffman et al. 2002. The presentation of self in everyday life. 1959. Garden City, NY, 259. [find](https://scholar.google.com/scholar?q=Erving+Goffman+et+al%2E+2002%2E+The+presentation+of+self+in+everyday+life%2E+1959%2E+Garden+City%2C+NY%2C+259%2E)
22. E Dario Gutiérrez, Ekaterina Shutova, Patricia Lichtenstein, Gerard De Melo, and Luca Gilardi. 2016. Detecting cross-cultural differences using a multilingual topic model. Transactions of the Association for Computational Linguistics, 4:47–60. [find](https://scholar.google.com/scholar?q=E+Dario+Guti%C3%A9rrez%2C+Ekaterina+Shutova%2C+Patricia+Lichtenstein%2C+Gerard+De+Melo%2C+and+Luca+Gilardi%2E+2016%2E+Detecting+cross%2Dcultural+differences+using+a+multilingual+topic+model%2E+Transactions+of+the+Associat)
23. Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, et al. 2022. [Challenges and strategies in cross-cultural nlp](https://md.research.elamur.ai/free/en/papers/hershcovich-2022-challenges-strategies-cross-cultural-nlp/). arXiv preprint arXiv:2203.10020.
24. Dirk Hovy and Diyi Yang. 2021. [The importance of modeling social factors of language: Theory and practice](https://md.research.elamur.ai/free/en/papers/hovy-2021-importance-modeling-social-factors-language/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 588–602, Online. Association for Computa…
25. Jing Huang and Diyi Yang. 2023. [Culturally aware natural language inference](https://md.research.elamur.ai/free/en/papers/huang-2023-culturally-aware-natural-language-inference/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7591–7609.
26. Katharina Hämmerl, Björn Deiseroth, Patrick Schramowski, Jindřich Libovický, Alexander Fraser, and Kristian Kersting. 2022. Do multilingual language models capture differing moral norms? [find](https://scholar.google.com/scholar?q=Katharina+H%C3%A4mmerl%2C+Bj%C3%B6rn+Deiseroth%2C+Patrick+Schramowski%2C+Jindr%CC%8Cich+Libovick%C3%BD%2C+Alexander+Fraser%2C+and+Kristian+Kersting%2E+2022%2E+Do+multilingual+language+models+capture+differing+moral+norms%3F)
27. Ronald Inglehart, Miguel Basanez, Jaime Diez-Medrano, Loek Halman, and Ruud Luijkx. 2000. World values surveys and european values surveys, 1981-1984, 1990-1993, and 1995-1997. Ann Arbor-Michigan, Institute for Social Research, ICPSR version. [find](https://scholar.google.com/scholar?q=Ronald+Inglehart%2C+Miguel+Basanez%2C+Jaime+Diez%2DMedrano%2C+Loek+Halman%2C+and+Ruud+Luijkx%2E+2000%2E+World+values+surveys+and+european+values+surveys%2C+1981%2D1984%2C+1990%2D1993%2C+and+1995%2D1997%2E+Ann+Arbor%2DMichigan%2C+Ins)
28. Sheena S Iyengar, Mark R Lepper, and Lee Ross. 1999. Independence from whom? interdependence with whom? cultural perspectives on ingroups versus out-groups. Cultural divides: Understanding and overcoming group conflict, pages 273–301. [find](https://scholar.google.com/scholar?q=Sheena+S+Iyengar%2C+Mark+R+Lepper%2C+and+Lee+Ross%2E+1999%2E+Independence+from+whom%3F+interdependence+with+whom%3F+cultural+perspectives+on+ingroups+versus+out%2Dgroups%2E+Cultural+divides%3A+Understanding+and+overcom)
29. Akshita Jha, Aida Davani, Chandan K Reddy, Shachi Dave, Vinodkumar Prabhakaran, and Sunipa Dev. 2023. Seegull: A stereotype benchmark with broad geo-cultural coverage leveraging generative models. arXiv preprint arXiv:2305.11840. [find](https://scholar.google.com/scholar?q=Akshita+Jha%2C+Aida+Davani%2C+Chandan+K+Reddy%2C+Shachi+Dave%2C+Vinodkumar+Prabhakaran%2C+and+Sunipa+Dev%2E+2023%2E+Seegull%3A+A+stereotype+benchmark+with+broad+geo%2Dcultural+coverage+leveraging+generative+models%2E+arX)
30. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+et+al%2E+2024%2E+Mixtral+o)
31. Zhengbao Jiang, Antonios Anastasopoulos, Jun Araki, Haibo Ding, and Graham Neubig. 2020. X-FACTR: Multilingual factual knowledge retrieval from pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5943–5959, Online. Ass… [find](https://scholar.google.com/scholar?q=Zhengbao+Jiang%2C+Antonios+Anastasopoulos%2C+Jun+Araki%2C+Haibo+Ding%2C+and+Graham+Neubig%2E+2020%2E+X%2DFACTR%3A+Multilingual+factual+knowledge+retrieval+from+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+2020+)
32. Jiho Jin, Jiseon Kim, Nayeon Lee, Haneul Yoo, Alice Oh, and Hwaran Lee. 2023. [Kobbq: Korean bias benchmark for question answering](https://md.research.elamur.ai/free/en/papers/jin-2024-kobbq-korean-bias-benchmark-question/). arXiv preprint arXiv:2307.16778.
33. Eunsu Kim, Juyoung Suk, Philhoon Oh, Haneul Yoo, James Thorne, and Alice Oh. 2024. Click: A benchmark dataset of cultural and linguistic intelligence in korean. arXiv preprint arXiv:2403.06412. [find](https://scholar.google.com/scholar?q=Eunsu+Kim%2C+Juyoung+Suk%2C+Philhoon+Oh%2C+Haneul+Yoo%2C+James+Thorne%2C+and+Alice+Oh%2E+2024%2E+Click%3A+A+benchmark+dataset+of+cultural+and+linguistic+intelligence+in+korean%2E+arXiv+preprint+arXiv%3A2403%2E06412%2E)
34. Alfred Louis Kroeber and Clyde Kluckhohn. 1952. Culture: A critical review of concepts and definitions. Papers. Peabody Museum of Archaeology & Ethnology, Harvard University. [find](https://scholar.google.com/scholar?q=Alfred+Louis+Kroeber+and+Clyde+Kluckhohn%2E+1952%2E+Culture%3A+A+critical+review+of+concepts+and+definitions%2E+Papers%2E+Peabody+Museum+of+Archaeology+%26+Ethnology%2C+Harvard+University%2E)
35. Abdullatif Köksal, Omer Faruk Yalcin, Ahmet Akbiyik, M. Tahir Kilavuz, Anna Korhonen, and Hinrich Schütze. 2023. Language-agnostic bias detection in language models with bias probing. [find](https://scholar.google.com/scholar?q=Abdullatif+K%C3%B6ksal%2C+Omer+Faruk+Yalcin%2C+Ahmet+Akbiyik%2C+M%2E+Tahir+Kilavuz%2C+Anna+Korhonen%2C+and+Hinrich+Sch%C3%BCtze%2E+2023%2E+Language%2Dagnostic+bias+detection+in+language+models+with+bias+probing%2E)
36. Nayeon Lee, Chani Jung, Junho Myung, Jiho Jin, Juho Kim, and Alice Oh. 2023. Crehate: Cross-cultural re-annotation of english hate speech dataset. arXiv preprint arXiv:2308.16705. [find](https://scholar.google.com/scholar?q=Nayeon+Lee%2C+Chani+Jung%2C+Junho+Myung%2C+Jiho+Jin%2C+Juho+Kim%2C+and+Alice+Oh%2E+2023%2E+Crehate%3A+Cross%2Dcultural+re%2Dannotation+of+english+hate+speech+dataset%2E+arXiv+preprint+arXiv%3A2308%2E16705%2E)
37. Cheng Li, Mengzhou Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. 2024. [Culturellm: Incorporating cultural differences into large language models](https://md.research.elamur.ai/free/en/papers/li-2024-culturellm-incorporating-cultural-differences-large/). arXiv preprint arXiv:2402.10946.
38. Lei Li, Yongfeng Zhang, and Li Chen. 2023. Prompt distillation for efficient llm-based recommendation. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 1348–1357. [find](https://scholar.google.com/scholar?q=Lei+Li%2C+Yongfeng+Zhang%2C+and+Li+Chen%2E+2023%2E+Prompt+distillation+for+efficient+llm%2Dbased+recommendation%2E+In+Proceedings+of+the+32nd+ACM+International+Conference+on+Information+and+Knowledge+Management%2C+)
39. Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott. 2021. Visually grounded reasoning across languages and cultures. arXiv preprint arXiv:2109.13238. [find](https://scholar.google.com/scholar?q=Fangyu+Liu%2C+Emanuele+Bugliarello%2C+Edoardo+Maria+Ponti%2C+Siva+Reddy%2C+Nigel+Collier%2C+and+Desmond+Elliott%2E+2021%2E+Visually+grounded+reasoning+across+languages+and+cultures%2E+arXiv+preprint+arXiv%3A2109%2E13238%2E)
40. Montgomery McFate. 2005. The military utility of understanding adversary culture. Joint Force Quarterly, 38:42. [find](https://scholar.google.com/scholar?q=Montgomery+McFate%2E+2005%2E+The+military+utility+of+understanding+adversary+culture%2E+Joint+Force+Quarterly%2C+38%3A42%2E)
41. Margaret Mead, Rhoda Bubendey Métraux, and Rhoda Métraux. 2000. The study of culture at a distance, volume 1. Berghahn Books. [find](https://scholar.google.com/scholar?q=Margaret+Mead%2C+Rhoda+Bubendey+M%C3%A9traux%2C+and+Rhoda+M%C3%A9traux%2E+2000%2E+The+study+of+culture+at+a+distance%2C+volume+1%2E+Berghahn+Books%2E)
42. Anjishnu Mukherjee, Chahat Raj, Ziwei Zhu, and Antonios Anastasopoulos. 2023. Global voices, local biases: Socio-cultural prejudices across languages. arXiv preprint arXiv:2310.17586. [find](https://scholar.google.com/scholar?q=Anjishnu+Mukherjee%2C+Chahat+Raj%2C+Ziwei+Zhu%2C+and+Antonios+Anastasopoulos%2E+2023%2E+Global+voices%2C+local+biases%3A+Socio%2Dcultural+prejudices+across+languages%2E+arXiv+preprint+arXiv%3A2310%2E17586%2E)
43. Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R Bowman. 2020. [Crows-pairs: A challenge dataset for measuring social biases in masked language models](https://md.research.elamur.ai/free/en/papers/nangia-2020-crows-pairs-challenge-dataset-measuring/). arXiv preprint arXiv:2010.00133.
44. Tarek Naous, Michael J Ryan, and Wei Xu. 2023. [Having beer after prayer? measuring cultural bias in large language models](https://md.research.elamur.ai/free/en/papers/naous-2023-having-beer-after-prayer-measuring/). arXiv preprint arXiv:2305.14456.
45. Tuan-Phong Nguyen, Simon Razniewski, Aparna Varde, and Gerhard Weikum. 2023. Extracting cultural commonsense knowledge at scale. In Proceedings of the ACM Web Conference 2023, WWW ’23. ACM. [find](https://scholar.google.com/scholar?q=Tuan%2DPhong+Nguyen%2C+Simon+Razniewski%2C+Aparna+Varde%2C+and+Gerhard+Weikum%2E+2023%2E+Extracting+cultural+commonsense+knowledge+at+scale%2E+In+Proceedings+of+the+ACM+Web+Conference+2023%2C+WWW+%E2%80%9923%2E+ACM%2E)
46. Keivalya Pandya and Mehfuza Holia. 2023. Automating customer service using langchain: Building custom open-source gpt chatbot for organizations. arXiv preprint arXiv:2310.05421. — [arXiv:2310.05421](https://arxiv.org/abs/2310.05421) [find](https://scholar.google.com/scholar?q=Keivalya+Pandya+and+Mehfuza+Holia%2E+2023%2E+Automating+customer+service+using+langchain%3A+Building+custom+open%2Dsource+gpt+chatbot+for+organizations%2E+arXiv+preprint+arXiv%3A2310%2E05421%2E+%E2%80%94+%5BarXiv%3A2310%2E05421%5D%28h)
47. Antonio Penta, Nigel Shadbolt, Paul Smart, and Winston R. Sieck. 2011. Detection of cognitive features from web resources in support of cultural modeling and analysis. In Proceedings of the International Conference on Management of Emergent Digital EcoSystems, MEDES ’11, page 53–60, New York, NY, US… [find](https://scholar.google.com/scholar?q=Antonio+Penta%2C+Nigel+Shadbolt%2C+Paul+Smart%2C+and+Winston+R%2E+Sieck%2E+2011%2E+Detection+of+cognitive+features+from+web+resources+in+support+of+cultural+modeling+and+analysis%2E+In+Proceedings+of+the+Internatio)
48. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Advances in Neural Information Processing Systems, 36.
49. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67.
50. Aida Ramezani and Yang Xu. 2023. [Knowledge of cultural moral norms in large language models](https://md.research.elamur.ai/free/en/papers/ramezani-2023-knowledge-cultural-moral-norms-large/).
51. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). arXiv preprint arXiv:1908.10084.
52. Jonathan Roberts, Timo Lüddecke, Sowmen Das, Kai Han, and Samuel Albanie. 2023. Gpt4geo: How a language model sees the world’s geography. arXiv preprint arXiv:2306.00020. [find](https://scholar.google.com/scholar?q=Jonathan+Roberts%2C+Timo+L%C3%BCddecke%2C+Sowmen+Das%2C+Kai+Han%2C+and+Samuel+Albanie%2E+2023%2E+Gpt4geo%3A+How+a+language+model+sees+the+world%E2%80%99s+geography%2E+arXiv+preprint+arXiv%3A2306%2E00020%2E)
53. Michael J Ryan, William Held, and Diyi Yang. 2024. Unintended impacts of llm alignment on global representation. arXiv preprint arXiv:2402.15018. [find](https://scholar.google.com/scholar?q=Michael+J+Ryan%2C+William+Held%2C+and+Diyi+Yang%2E+2024%2E+Unintended+impacts+of+llm+alignment+on+global+representation%2E+arXiv+preprint+arXiv%3A2402%2E15018%2E)
54. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+2019%2E+Distilbert%2C+a+distilled+version+of+bert%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+arXiv+preprint+arXiv%3A1910%2E01108%2E)
55. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. [Whose opinions do language models reflect?](https://md.research.elamur.ai/free/en/papers/santurkar-2023-whose-opinions-language-models-reflect/) In International Conference on Machine Learning, pages 29971–30004. PMLR.
56. Sheikh Shafayat, Eunsu Kim, Juhyun Oh, and Alice Oh. 2024. Multi-fact: Assessing multilingual llms’ multi-regional knowledge using factscore. arXiv preprint arXiv:2402.18045. — [arXiv:2402.18045](https://arxiv.org/abs/2402.18045) [find](https://scholar.google.com/scholar?q=Sheikh+Shafayat%2C+Eunsu+Kim%2C+Juhyun+Oh%2C+and+Alice+Oh%2E+2024%2E+Multi%2Dfact%3A+Assessing+multilingual+llms%E2%80%99+multi%2Dregional+knowledge+using+factscore%2E+arXiv+preprint+arXiv%3A2402%2E18045%2E+%E2%80%94+%5BarXiv%3A2402%2E18045%5D%28http)
57. Katérina Stenou. 2002. Unesco universal declaration on cultural diversity: a vision, a conceptual platform, a pool of ideas for implementation, a new paradigm. [find](https://scholar.google.com/scholar?q=Kat%C3%A9rina+Stenou%2E+2002%2E+Unesco+universal+declaration+on+cultural+diversity%3A+a+vision%2C+a+conceptual+platform%2C+a+pool+of+ideas+for+implementation%2C+a+new+paradigm%2E)
58. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+et+al%2E+2023%2E+Llama+2%3A+Open+foundation+and+fi)
59. Wenxuan Wang, Wenxiang Jiao, Jingyuan Huang, Ruyi Dai, Jen tse Huang, Zhaopeng Tu, and Michael R. Lyu. 2023. [Not all countries celebrate thanksgiving: On the cultural dominance in large language models](https://md.research.elamur.ai/free/en/papers/wang-2024-not-all-countries-celebrate-thanksgiving/).
60. David Watkins. 2012. Learning and teaching: A cross-cultural perspective. In School Leadership and Administration, pages 61–76. Routledge. [find](https://scholar.google.com/scholar?q=David+Watkins%2E+2012%2E+Learning+and+teaching%3A+A+cross%2Dcultural+perspective%2E+In+School+Leadership+and+Administration%2C+pages+61%E2%80%9376%2E+Routledge%2E)
61. Binwei Yao, Ming Jiang, Diyi Yang, and Junjie Hu. 2023. Empowering llm-based machine translation with cultural awareness. [find](https://scholar.google.com/scholar?q=Binwei+Yao%2C+Ming+Jiang%2C+Diyi+Yang%2C+and+Junjie+Hu%2E+2023%2E+Empowering+llm%2Dbased+machine+translation+with+cultural+awareness%2E)
62. Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li, and Kai-Wei Chang. 2022. Geomlama: Geo-diverse commonsense probing on multilingual pre-trained language models. arXiv preprint arXiv:2205.12247. [find](https://scholar.google.com/scholar?q=Da+Yin%2C+Hritik+Bansal%2C+Masoud+Monajatipoor%2C+Liunian+Harold+Li%2C+and+Kai%2DWei+Chang%2E+2022%2E+Geomlama%3A+Geo%2Ddiverse+commonsense+probing+on+multilingual+pre%2Dtrained+language+models%2E+arXiv+preprint+arXiv%3A2205)
63. Caleb Ziems, Jane Dwivedi-Yu, Yi-Chia Wang, Alon Halevy, and Diyi Yang. 2023. [NormBank: A knowledge bank of situational social norms](https://md.research.elamur.ai/free/en/papers/ziems-2023-normbank-knowledge-bank-situational-social/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7756–7776, Toronto, Canada. Association for…

*Updated 24 October 2026*