--- title: "Counterfactual Auditing of Cross-Cultural Variation in" description: "Yoo and Rosen propose a counterfactual audit framework. 8 references. Among them: 0." authors: ["Hyunwoo Yoo", "Gail Rosen"] year: 2026 venue: "Proceedings of the 1st Workshop on Stereotypes Across Cultures in Language Technologies (StereACuLT 2026)" source: "https://aclanthology.org/2026.stereacult-1.5/" doi: "10.18653/v1/2026.stereacult-1.5" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/yoo-2026-counterfactual-auditing-cross-cultural-variation/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yoo-2026-counterfactual-auditing-cross-cultural-variation/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) / Counterfactual Auditing of Cross-Cultural Variation in LLM-Generated Medical Advice # Counterfactual Auditing of Cross-Cultural Variation in LLM-Generated Medical Advice **Authors:** [Hyunwoo Yoo](https://md.research.elamur.ai/free/en/authors/hyunwoo-yoo/) and Gail Rosen · **Year:** 2026 · **Venue:** [Proceedings of the 1st Workshop on Stereotypes Across Cultures in Language Technologies (StereACuLT 2026)](https://md.research.elamur.ai/free/en/venues/stereacult/) ([Workshop on Stereotypes Across Cultures in Language Technologies 2026](https://md.research.elamur.ai/free/en/venues/stereacult/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.stereacult-1.5/) · [PDF](https://aclanthology.org/2026.stereacult-1.5.pdf) · [DOI 10.18653/v1/2026.stereacult-1.5](https://doi.org/10.18653/v1/2026.stereacult-1.5) · [OpenAlex](https://openalex.org/W7166869968) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary The paper asks whether LLMs giving patient-facing medical advice change their safety guidance when only cultural identity cues differ. Yoo and Rosen propose a counterfactual audit framework. Clinical vignettes keep symptoms, duration, red flags and history fixed. Each vignette is then paired with a culturally marked variant: a recognition-seeking statement (Arab condition), a stated language difficulty (Korean condition), or limited healthcare access (Nigerian condition). Each condition reflects barriers documented in the health-services literature. Every pair is sampled repeatedly, and outputs are compared on urgency category, escalation, red-flag mentions, empathy and access guidance. The audit covers GPT-4o-mini, Gemini 2.5 Flash and Qwen3-30B-A3B-Thinking-2507. Identity cues moved urgency in a notable share of matched comparisons, but the direction differed by model: GPT-4o-mini mostly escalated, while Gemini moved both ways. A paraphrase-based noise floor shows these shifts are directional rather than random generation noise. The work offers a reproducible method for separating legitimate communication adaptation from clinically unjustified changes in triage advice. ## Key points - Framework components: clinical matching, identity cues, cross-regional localization, repeated sampling, and structured comparison of urgency labels, red flags, empathy, simplification and length. - Urgency-change rates per condition: - GPT-4o-mini: 25–33%, with escalation dominating. - Gemini 2.5 Flash: about 16–22%, leaning toward de-escalation, e.g., 16.7% de-escalation for the Nigerian access cue. - Qwen3: 10–31%. - Roughly 90% of GPT-4o-mini's identity-conditioned changes pushed toward higher urgency. The corresponding share was about 82% for Qwen3 and about 40% for Gemini. - Paraphrase controls show baseline instability exists even without identity cues. However, identity cues produce systematic directional shifts that the controls do not show. - Data collection was incomplete: about 83% of Gemini generations succeeded, and Qwen3 dropped to 50% in the Nigerian condition. The authors therefore frame the results as exploratory and small-scale. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Drexel University](https://md.research.elamur.ai/free/en/universities/drexel-university/) — University, Philadelphia, United States · 6 works · 1,443 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 275 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 214 works ## Cite Cite as: Hyunwoo Yoo and Gail Rosen. 2026. Counterfactual Auditing of Cross-Cultural Variation in LLM-Generated Medical Advice. Proceedings of the 1st Workshop on Stereotypes Across Cultures in Language Technologies (StereACuLT 2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{yoo2026counterfactual, title = {{Counterfactual Auditing of Cross-Cultural Variation in LLM-Generated Medical Advice}}, author = {Hyunwoo Yoo and Gail Rosen}, booktitle = {Proceedings of the 1st Workshop on Stereotypes Across Cultures in Language Technologies (StereACuLT 2026)}, year = {2026}, doi = {10.18653/v1/2026.stereacult-1.5}, url = {https://doi.org/10.18653/v1/2026.stereacult-1.5} } ``` - [← previous Council of LLMs: Evaluating Capability of Large Language Models to Annotate Propaganda](https://md.research.elamur.ai/free/en/papers/sharma-2026-council-llms-evaluating-capability-large/) - [next → Counterspeech Generation using Small Language Models](https://md.research.elamur.ai/free/en/papers/muhammad-2026-counterspeech-generation-small-language-models/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) ## References 8 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Claudia M. Campbell and Robert R. Edwards. 2012. Ethnic differences in pain and pain management. Pain Management, 2(3):219–230. [find](https://scholar.google.com/scholar?q=Claudia+M%2E+Campbell+and+Robert+R%2E+Edwards%2E+2012%2E+Ethnic+differences+in+pain+and+pain+management%2E+Pain+Management%2C+2%283%29%3A219%E2%80%93230%2E) 2. Deborah Dowell, Kathleen R. Ragan, Christopher M. Jones, Grant T. Baldwin, and Roger Chou. 2022. Cdc clinical practice guideline for prescribing opioids for pain—united states, 2022. MMWR. Recommendations and Reports. [find](https://scholar.google.com/scholar?q=Deborah+Dowell%2C+Kathleen+R%2E+Ragan%2C+Christopher+M%2E+Jones%2C+Grant+T%2E+Baldwin%2C+and+Roger+Chou%2E+2022%2E+Cdc+clinical+practice+guideline+for+prescribing+opioids+for+pain%E2%80%94united+states%2C+2022%2E+MMWR%2E+Recommendat) 3. Gemini Team, Google. 2023. [Gemini: A family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805. 4. John J Hanna, Abdi D Wakene, Andrew O Johnson, Christoph U Lehmann, and Richard J Medford. 2025. Assessing racial and ethnic bias in text generation by large language models for health care–related tasks: Cross-sectional study. Journal of Medical Internet Research. [find](https://scholar.google.com/scholar?q=John+J+Hanna%2C+Abdi+D+Wakene%2C+Andrew+O+Johnson%2C+Christoph+U+Lehmann%2C+and+Richard+J+Medford%2E+2025%2E+Assessing+racial+and+ethnic+bias+in+text+generation+by+large+language+models+for+health+care%E2%80%93related+ta) 5. Sou Hyun Jang. 2016. First-generation Korean immigrants’ barriers to healthcare and their coping strategies in the US. Social Science & Medicine, 168:93– 100. [find](https://scholar.google.com/scholar?q=Sou+Hyun+Jang%2E+2016%2E+First%2Dgeneration+Korean+immigrants%E2%80%99+barriers+to+healthcare+and+their+coping+strategies+in+the+US%2E+Social+Science+%26+Medicine%2C+168%3A93%E2%80%93+100%2E) 6. Yuri Jang, Hyunwoo Yoon, Nan Sook Park, and David A. Chiriboga. 2016. Health vulnerability of immigrants with limited English proficiency: A study of older Korean Americans. Journal of the American Geriatrics Society, 64(7):1498–1502. [find](https://scholar.google.com/scholar?q=Yuri+Jang%2C+Hyunwoo+Yoon%2C+Nan+Sook+Park%2C+and+David+A%2E+Chiriboga%2E+2016%2E+Health+vulnerability+of+immigrants+with+limited+English+proficiency%3A+A+study+of+older+Korean+Americans%2E+Journal+of+the+American+Ge) 7. Leilei Jiang, Guixiang Zhu, Jianshan Sun, Jie Cao, and Jia Wu. 2025. Exploring the occupational biases and stereotypes of chinese large language models. Scientific Reports. [find](https://scholar.google.com/scholar?q=Leilei+Jiang%2C+Guixiang+Zhu%2C+Jianshan+Sun%2C+Jie+Cao%2C+and+Jia+Wu%2E+2025%2E+Exploring+the+occupational+biases+and+stereotypes+of+chinese+large+language+models%2E+Scientific+Reports%2E) 8. Yuu Jinnai. 2024. Does cross-cultural alignment change the commonsense morality of language models? pages 48–64. \#\# Ethical Considerations Yeawon Lee, Chia-Hsuan Chang, and Christopher C. Yang. 2025. Enhancing patient-physician communication: Simulating african american vernacular english in medical… [find](https://scholar.google.com/scholar?q=Yuu+Jinnai%2E+2024%2E+Does+cross%2Dcultural+alignment+change+the+commonsense+morality+of+language+models%3F+pages+48%E2%80%9364%2E+%5C%23%5C%23+Ethical+Considerations+Yeawon+Lee%2C+Chia%2DHsuan+Chang%2C+and+Christopher+C%2E+Yang%2E+2025) *Updated 2 November 2026*