---
title: "When Demographic Sensitivity Isn't What It Seems:"
description: "The author proposes a baseline-aware audit. 25 references. Among them: 0."
authors: ["Hyunwoo Yoo"]
year: 2026
venue: "BioNLP 2026"
source: "https://aclanthology.org/2026.bionlp-1.14/"
doi: "10.18653/v1/2026.bionlp-1.14"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yoo-2026-when-demographic-sensitivity-isn-t/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yoo-2026-when-demographic-sensitivity-isn-t/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/56/) / When Demographic Sensitivity Isn't What It Seems: Baseline-Aware Counterfactual Audits for Clinical NLP

# When Demographic Sensitivity Isn't What It Seems: Baseline-Aware Counterfactual Audits for Clinical NLP

**Authors:** [Hyunwoo Yoo](https://md.research.elamur.ai/free/en/authors/hyunwoo-yoo/) · **Year:** 2026 · **Venue:** [BioNLP 2026](https://md.research.elamur.ai/free/en/venues/bionlp/) ([BioNLP 2026](https://md.research.elamur.ai/free/en/venues/bionlp/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.bionlp-1.14/) · [PDF](https://aclanthology.org/2026.bionlp-1.14.pdf) · [DOI 10.18653/v1/2026.bionlp-1.14](https://doi.org/10.18653/v1/2026.bionlp-1.14) · [OpenAlex](https://openalex.org/W7166841458)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Counterfactual fairness audits for clinical language models usually change a patient's age or sex and treat any resulting output shift as evidence of demographic bias. This paper argues that the assumption is unsafe, because modern LLMs can also change their answers when the input is altered in ways that have nothing to do with demographics. The author proposes a baseline-aware audit. Demographic edits are compared against paraphrase controls that keep both the clinical facts and the demographic attributes intact. Differences are then manually labelled as negligible, plausibly justified, or clinically worrying.

The framework was tested on 12 synthetic, triage-oriented EHR-style cases, mostly borderline, using three LLMs. Paraphrase controls often moved outputs as much as, or more than, demographic swaps did. Generated text also proved highly unstable on its own. Age edits had stronger effects than sex edits. The main lesson is that a demographic effect should count as meaningful only when it clearly rises above the model's intrinsic noise. Without that baseline, bias can be overstated or misattributed.

## Key points
- **Models and outputs:** gpt-4o-mini, gemini-2.5-flash and Qwen3-30B-A3B-Thinking-2507 were evaluated. Each produced a structured triage output with a recommendation, urgency, red flags, rationale and immediate action.
- **Test conditions:** each case had four versions (original, age swap, sex swap, paraphrase control). Robustness checks added three extra independent paraphrase controls, plus variation in decoding temperature and prompts.
- **Noise floor:** intrinsic generative instability measured 0.46–0.71 by Jaccard. Action change rates under control vs. demographic edits were:
  - gpt-4o-mini: 0.472 vs. 0.268
  - Qwen3: 0.211 vs. 0.278
  - gemini-2.5-flash: 0.444 vs. 0.727
- **Age vs. sex:** urgency changed more often under age edits than under sex edits:
  - gpt-4o-mini: 0.056 vs. 0.000
  - gemini-2.5-flash: 0.333 vs. 0.154
  - Qwen3: 0.190 vs. 0.000

  Stable labels often concealed shifts in rationales and recommendations.
- **Practical rule:** if the demographic effect is no larger than the control effect (Δdem ≤ Δcontrol), treat it as noise. Main limitations are the small synthetic set of 12 cases and a Jaccard metric that cannot separate stylistic variation from clinical drift.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Drexel University](https://md.research.elamur.ai/free/en/universities/drexel-university/) — University, Philadelphia, United States · 6 works · 1,443 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

## Cite

Cite as: Hyunwoo Yoo. 2026. When Demographic Sensitivity Isn't What It Seems: Baseline-Aware Counterfactual Audits for Clinical NLP. BioNLP 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yoo2026when,
  title = {{When Demographic Sensitivity Isn't What It Seems: Baseline-Aware Counterfactual Audits for Clinical NLP}},
  author = {Hyunwoo Yoo},
  booktitle = {BioNLP 2026},
  year = {2026},
  doi = {10.18653/v1/2026.bionlp-1.14},
  url = {https://doi.org/10.18653/v1/2026.bionlp-1.14}
}
```

- [← previous When Debiasing Backfires: Counterintuitive Side Effects of Preprocessing-Based Stereotype Mitigation](https://md.research.elamur.ai/free/en/papers/zheng-2026-when-debiasing-backfires-counterintuitive-side/)
- [next → When depth is redundant: Efficient transformer-based speech anti-spoofing](https://md.research.elamur.ai/free/en/papers/tran-2026-when-depth-is-redundant-efficient/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/56/)

## References

25 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Amber M. Angell, Yongqiu Li, Jiang Bian, Camille Parchment, Larry Yin, Srikar Chamala, Hesamedin Hakimjavadi, Lindsay Thompson, and Yi Guo. 2025. Algorithmic fairness in machine learning prediction of autism using electronic health records. Studies in Health Technology and Informatics. [find](https://scholar.google.com/scholar?q=Amber+M%2E+Angell%2C+Yongqiu+Li%2C+Jiang+Bian%2C+Camille+Parchment%2C+Larry+Yin%2C+Srikar+Chamala%2C+Hesamedin+Hakimjavadi%2C+Lindsay+Thompson%2C+and+Yi+Guo%2E+2025%2E+Algorithmic+fairness+in+machine+learning+prediction+of)
2. Mingchen Li, Zaifu Zhan, Han Yang, Yongkang Xiao, Huixue Zhou, Jiatan Huang, and Rui Zhang. 2025. Benchmarking retrieval-augmented large language models in biomedical nlp: Application, robustness, and self-awareness. Science Advances. [find](https://scholar.google.com/scholar?q=Mingchen+Li%2C+Zaifu+Zhan%2C+Han+Yang%2C+Yongkang+Xiao%2C+Huixue+Zhou%2C+Jiatan+Huang%2C+and+Rui+Zhang%2E+2025%2E+Benchmarking+retrieval%2Daugmented+large+language+models+in+biomedical+nlp%3A+Application%2C+robustness%2C+and)
3. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. [A survey on bias and fairness in machine learning](https://md.research.elamur.ai/free/en/papers/mehrabi-2019-survey-bias-fairness-machine-learning/). ACM Computing Surveys.
4. OpenAI. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774. Version 6; last revised 4 Mar 2024.
5. Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond accuracy: Behavioral testing of NLP models with CheckList. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902–4912, Online. Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+Tongshuang+Wu%2C+Carlos+Guestrin%2C+and+Sameer+Singh%2E+2020%2E+Beyond+accuracy%3A+Behavioral+testing+of+NLP+models+with+CheckList%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Associat)
6. Varuni Sarwal, Aditya Pimplaskar, Misty Richards, Kunmi Sobowale, Jeffrey N. Chiang, and Loes Olde Loohuis. 2025. Early prediction and fairness evaluation of perinatal depression using ehr: A study of 18,000+ pregnancies. medRxiv. [find](https://scholar.google.com/scholar?q=Varuni+Sarwal%2C+Aditya+Pimplaskar%2C+Misty+Richards%2C+Kunmi+Sobowale%2C+Jeffrey+N%2E+Chiang%2C+and+Loes+Olde+Loohuis%2E+2025%2E+Early+prediction+and+fairness+evaluation+of+perinatal+depression+using+ehr%3A+A+study+of)
7. Miguel Contreras, Parisa Rashidi, and Sumit Kapoor. 2025. 353: Large language models for mortality prediction using structured ehr and unstructured clinical notes. Critical Care Medicine. [find](https://scholar.google.com/scholar?q=Miguel+Contreras%2C+Parisa+Rashidi%2C+and+Sumit+Kapoor%2E+2025%2E+353%3A+Large+language+models+for+mortality+prediction+using+structured+ehr+and+unstructured+clinical+notes%2E+Critical+Care+Medicine%2E)
8. Ravi Shankar, Janani Kannan, Yih Hng Tan, and Qian Xu. 2025. Natural language processing techniques to detect delirium in hospitalized patients from clinical notes: a systematic review. npj Digital Medicine. [find](https://scholar.google.com/scholar?q=Ravi+Shankar%2C+Janani+Kannan%2C+Yih+Hng+Tan%2C+and+Qian+Xu%2E+2025%2E+Natural+language+processing+techniques+to+detect+delirium+in+hospitalized+patients+from+clinical+notes%3A+a+systematic+review%2E+npj+Digital+Me)
9. Amanda Coston, Alan Mishler, Edward H. Kennedy, and Alexandra Chouldechova. 2020. Counterfactual risk assessments, evaluation, and fairness. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. [find](https://scholar.google.com/scholar?q=Amanda+Coston%2C+Alan+Mishler%2C+Edward+H%2E+Kennedy%2C+and+Alexandra+Chouldechova%2E+2020%2E+Counterfactual+risk+assessments%2C+evaluation%2C+and+fairness%2E+Proceedings+of+the+2020+Conference+on+Fairness%2C+Accountabil)
10. Zhecheng Sheng. 2022. Nlp system for mining social determinant of health from clinical notes and its fairness evaluations. 2022 IEEE 10th International Conference on Healthcare Informatics (ICHI). [find](https://scholar.google.com/scholar?q=Zhecheng+Sheng%2E+2022%2E+Nlp+system+for+mining+social+determinant+of+health+from+clinical+notes+and+its+fairness+evaluations%2E+2022+IEEE+10th+International+Conference+on+Healthcare+Informatics+%28ICHI%29%2E)
11. Gemini Team, Google. 2023. [Gemini: A family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805.
12. Abhishek Shivanna, Adam Spannaus, Jordan Tschida, John Gounley, Patrycja Krawczuk, and Heidi Hanson. 2026. Mitigating algorithmic bias in cancer site classification models. JCO Clinical Cancer Informatics. [find](https://scholar.google.com/scholar?q=Abhishek+Shivanna%2C+Adam+Spannaus%2C+Jordan+Tschida%2C+John+Gounley%2C+Patrycja+Krawczuk%2C+and+Heidi+Hanson%2E+2026%2E+Mitigating+algorithmic+bias+in+cancer+site+classification+models%2E+JCO+Clinical+Cancer+Informa)
13. Dong-Gil Ko. 2026. Advancing fairness in clinical ai decision-making through a sociotechnical threshold bias audit. Proceedings of the 19th International Joint Conference on Biomedical Engineering Systems and Technologies. [find](https://scholar.google.com/scholar?q=Dong%2DGil+Ko%2E+2026%2E+Advancing+fairness+in+clinical+ai+decision%2Dmaking+through+a+sociotechnical+threshold+bias+audit%2E+Proceedings+of+the+19th+International+Joint+Conference+on+Biomedical+Engineering+Sys)
14. Katherine Krieger, Irbaz Hameed, Giorgio Quer, Charles Mack, Marco Savic, Polina Mantaj, Aina Hirofuji, Alexander Gregg, Giovanni Soletti, Camilla S Rossi, Mohamed Rahouma, and Mario Gaudino. 2025. Generative pre-trained transformer reinforces historical gender bias in diagnosing women’s cardiovascu… [find](https://scholar.google.com/scholar?q=Katherine+Krieger%2C+Irbaz+Hameed%2C+Giorgio+Quer%2C+Charles+Mack%2C+Marco+Savic%2C+Polina+Mantaj%2C+Aina+Hirofuji%2C+Alexander+Gregg%2C+Giovanni+Soletti%2C+Camilla+S+Rossi%2C+Mohamed+Rahouma%2C+and+Mario+Gaudino%2E+2025%2E+Ge)
15. Ida Surakka, Brooke N Wolford, Scott C Ritchie, Whitney E Hornsby, Nadia R Sutton, Maiken Elvenstad Gabrielsen, Anne Heidi Skogholt, Laurent Thomas, Michael Inouye, Kristian Hveem, and Cristen J Willer. 2023. Sex-specific survival bias and interaction modeling in coronary artery disease risk predict… [find](https://scholar.google.com/scholar?q=Ida+Surakka%2C+Brooke+N+Wolford%2C+Scott+C+Ritchie%2C+Whitney+E+Hornsby%2C+Nadia+R+Sutton%2C+Maiken+Elvenstad+Gabrielsen%2C+Anne+Heidi+Skogholt%2C+Laurent+Thomas%2C+Michael+Inouye%2C+Kristian+Hveem%2C+and+Cristen+J+Wille)
16. Helena Kudiabor. 2026. Sex bias in autism drops as age at diagnosis rises. [find](https://scholar.google.com/scholar?q=Helena+Kudiabor%2E+2026%2E+Sex+bias+in+autism+drops+as+age+at+diagnosis+rises%2E)
17. Simon Suster, Timothy Baldwin, and Karin Verspoor. 2023. Promoting fairness in classification of quality of medical evidence. In Proceedings of the 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pages 413–426, Toronto, Canada. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Simon+Suster%2C+Timothy+Baldwin%2C+and+Karin+Verspoor%2E+2023%2E+Promoting+fairness+in+classification+of+quality+of+medical+evidence%2E+In+Proceedings+of+the+22nd+Workshop+on+Biomedical+Natural+Language+Process)
18. Matt J. Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual fairness. In Advances in Neural Information Processing Systems 30 (NeurIPS 2017). [find](https://scholar.google.com/scholar?q=Matt+J%2E+Kusner%2C+Joshua+Loftus%2C+Chris+Russell%2C+and+Ricardo+Silva%2E+2017%2E+Counterfactual+fairness%2E+In+Advances+in+Neural+Information+Processing+Systems+30+%28NeurIPS+2017%29%2E)
19. Dandan Wang and Shiqing Zhang. 2024. Large language models in medical and healthcare fields: applications, advances, and challenges. Artificial Intelligence Review. [find](https://scholar.google.com/scholar?q=Dandan+Wang+and+Shiqing+Zhang%2E+2024%2E+Large+language+models+in+medical+and+healthcare+fields%3A+applications%2C+advances%2C+and+challenges%2E+Artificial+Intelligence+Review%2E)
20. Xuezhi Wang, Haohan Wang, and Diyi Yang. 2022. [Measure and improve robustness in NLP models: A survey](https://md.research.elamur.ai/free/en/papers/wang-2022-measure-improve-robustness-nlp-models/). In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4569–4586, Seattle, United States. Associati…
21. Yifan Wang, Laura Sikstrom, Robert Xiao, Zoe Findlay, Juveria Zaheer, Sean L. Hill, and Marta M. Maslej. 2026. Fairness analysis of machine learning predictions of aggression in acute psychiatric care. npj Mental Health Research. [find](https://scholar.google.com/scholar?q=Yifan+Wang%2C+Laura+Sikstrom%2C+Robert+Xiao%2C+Zoe+Findlay%2C+Juveria+Zaheer%2C+Sean+L%2E+Hill%2C+and+Marta+M%2E+Maslej%2E+2026%2E+Fairness+analysis+of+machine+learning+predictions+of+aggression+in+acute+psychiatric+care)
22. Site Xu and Mu Sun. 2025. Natural language processing (nlp): Identifying linguistic gender bias in electronic medical records (emrs). Journal of Patient Experience. [find](https://scholar.google.com/scholar?q=Site+Xu+and+Mu+Sun%2E+2025%2E+Natural+language+processing+%28nlp%29%3A+Identifying+linguistic+gender+bias+in+electronic+medical+records+%28emrs%29%2E+Journal+of+Patient+Experience%2E)
23. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
24. Kunyu Yu, Rui Yang, Jingchi Liao, Siqi Li, Huitao Li, Irene Li, Yifan Peng, Rishikesan Kamaleswaran, and Nan Liu. 2025. Benchmarking foundation models with multimodal public electronic health records. IEEE Journal of Biomedical and Health Informatics. [find](https://scholar.google.com/scholar?q=Kunyu+Yu%2C+Rui+Yang%2C+Jingchi+Liao%2C+Siqi+Li%2C+Huitao+Li%2C+Irene+Li%2C+Yifan+Peng%2C+Rishikesan+Kamaleswaran%2C+and+Nan+Liu%2E+2025%2E+Benchmarking+foundation+models+with+multimodal+public+electronic+health+records%2E)
25. Yubo Zhang, Shudi Hou, Mingyu Derek Ma, Wei Wang, Muhao Chen, and Jieyu Zhao. 2024. Climb: A benchmark of clinical bias in large language models. Preprint, arXiv:2407.05250. — [arXiv:2407.05250](https://arxiv.org/abs/2407.05250) [find](https://scholar.google.com/scholar?q=Yubo+Zhang%2C+Shudi+Hou%2C+Mingyu+Derek+Ma%2C+Wei+Wang%2C+Muhao+Chen%2C+and+Jieyu+Zhao%2E+2024%2E+Climb%3A+A+benchmark+of+clinical+bias+in+large+language+models%2E+Preprint%2C+arXiv%3A2407%2E05250%2E+%E2%80%94+%5BarXiv%3A2407%2E05250%5D%28https)

*Updated 2 November 2026*