---
title: "Convergent Demographic Utility Hierarchies: Geometry of"
description: "This paper asks whether the coherent utility functions that LLMs develop contain systematic rankings of demographic groups… 26 references."
authors: ["Pravish Sainath"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)"
source: "https://aclanthology.org/2026.acl-srw.122/"
doi: "10.18653/v1/2026.acl-srw.122"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sainath-2026-convergent-demographic-utility-hierarchies-geometry/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sainath-2026-convergent-demographic-utility-hierarchies-geometry/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) / Convergent Demographic Utility Hierarchies: Geometry of Intersectional Values in LLMs

# Convergent Demographic Utility Hierarchies: Geometry of Intersectional Values in LLMs

**Authors:** Pravish Sainath · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-srw.122/) · [PDF](https://aclanthology.org/2026.acl-srw.122.pdf) · [DOI 10.18653/v1/2026.acl-srw.122](https://doi.org/10.18653/v1/2026.acl-srw.122) · [OpenAlex](https://openalex.org/W7166848366)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
This paper asks whether the coherent utility functions that LLMs develop contain systematic rankings of demographic groups, especially intersectional groups that combine race and gender. The author builds on the forced-choice utility framework of Mazeika et al. (2025) and tests Qwen-2.5-7B-Instruct, Mistral-7B-Instruct-v0.3 and LLaMA-3.1-8B-Instruct. Each model chooses between identical policy improvements that differ only in who benefits. The setup covers 15 groups (a baseline, two genders, four US racial groups and eight intersections) across 8 policy domains.

Preferences are read from A/B next-token logits, balanced over orderings and templates, and fitted with a Thurstonian model. A compositionality residual checks whether intersectional utilities are additive. Ridge probes on hidden states of neutral sentences produce a Principal Utility Direction (PUD), which is then compared with gender and race directions.

All three models agree on a hierarchy that reverses real-world advantage, placing Black women first and White men last. Intersectional effects are sub-additive. The ranking already appears in base models and is amplified by alignment. The findings argue for intersectional fairness audits and point to pre-training data as the source.

## Key points
- **Scale of the experiment:** 105 group pairs per domain, each sampled 30 times (3 templates × 2 orders × 5 repetitions), for 25,200 prompts per model.
- **Agreement across models:** the three models' rankings correlate at Spearman ρ > 0.93.
- **Non-additive intersections:** the residual ε is positive for all 24 group–model combinations, with a 2.2× gendered asymmetry. Adding separate gender and race gaps overstates the Black women vs. White men gap by 26–40%.
- **Geometry of the PUD:** the PUD reaches LOO R² up to 0.89 (Qwen, layer 4). Its cosine with the gender direction is about 0.45, but with the race direction it is below 0.07. The gender direction correlates with ε at r ≈ −0.95 to −0.97.
- **Origins and robustness:** base and instruct rankings correlate at ρ > 0.94, and alignment amplifies the hierarchy 3.6–7.3×. The ranking holds with indirect group descriptions (ρ ≈ 0.66–0.74).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Université de Montréal](https://md.research.elamur.ai/free/en/universities/universite-de-montreal/) — University, Montreal, Canada · 87 works · 91,597 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Pravish Sainath. 2026. Convergent Demographic Utility Hierarchies: Geometry of Intersectional Values in LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sainath2026convergent,
  title = {{Convergent Demographic Utility Hierarchies: Geometry of Intersectional Values in LLMs}},
  author = {Pravish Sainath},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.122},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.122}
}
```

- [← previous Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences](https://md.research.elamur.ai/free/en/papers/ramirez-2026-controlling-what-you-share-assessing/)
- [next → CONVERSE: Annotation Scheme and Dataset for Multimodal Conversational Engagement Analysis in Human-Human and Human-Robot Interaction](https://md.research.elamur.ai/free/en/papers/torubarova-2026-converse-annotation-scheme-dataset-multimodal/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/)

## References

26 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Sadasivam, and Adam Kalai. 2016. [Man is to computer programmer as woman is to homemaker? Debiasing word embeddings](https://md.research.elamur.ai/free/en/papers/bolukbasi-2016-man-computer-programmer-woman-homemaker/). In Advances in Neural Information Processing Systems, volume 29.
2. T. Tony Cai, Jianqing Fan, and Tiefeng Jiang. 2013. Distributions of angles in random packing on spheres. Journal of Machine Learning Research, 14:1837– 1864. [find](https://scholar.google.com/scholar?q=T%2E+Tony+Cai%2C+Jianqing+Fan%2C+and+Tiefeng+Jiang%2E+2013%2E+Distributions+of+angles+in+random+packing+on+spheres%2E+Journal+of+Machine+Learning+Research%2C+14%3A1837%E2%80%93+1864%2E)
3. Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. 2018. [What you can cram into a single $&!\#* vector: Probing sentence embeddings for linguistic properties](https://md.research.elamur.ai/free/en/papers/conneau-2018-what-you-can-cram-into/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pages 2126– 213…
4. Kimberlé Crenshaw. 1989. Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. In University of Chicago Legal Forum, volume 1989, pages 139–167. [find](https://scholar.google.com/scholar?q=Kimberl%C3%A9+Crenshaw%2E+1989%2E+Demarginalizing+the+intersection+of+race+and+sex%3A+A+black+feminist+critique+of+antidiscrimination+doctrine%2C+feminist+theory+and+antiracist+politics%2E+In+University+of+Chicago+L)
5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. [The Llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
6. Bradley Efron. 1987. Better bootstrap confidence intervals. Journal of the American Statistical Association, 82(397):171–185. [find](https://scholar.google.com/scholar?q=Bradley+Efron%2E+1987%2E+Better+bootstrap+confidence+intervals%2E+Journal+of+the+American+Statistical+Association%2C+82%28397%29%3A171%E2%80%93185%2E)
7. Peter C. Fishburn. 1970. Utility Theory for Decision Making. John Wiley & Sons, New York. [find](https://scholar.google.com/scholar?q=Peter+C%2E+Fishburn%2E+1970%2E+Utility+Theory+for+Decision+Making%2E+John+Wiley+%26+Sons%2C+New+York%2E)
8. Phillip I. Good. 2005. Permutation, Parametric, and Bootstrap Tests of Hypotheses, 3rd edition. Springer, New York. [find](https://scholar.google.com/scholar?q=Phillip+I%2E+Good%2E+2005%2E+Permutation%2C+Parametric%2C+and+Bootstrap+Tests+of+Hypotheses%2C+3rd+edition%2E+Springer%2C+New+York%2E)
9. Wei Guo and Aylin Caliskan. 2021. Detecting emergent intersectional biases: Contextualized word embeddings contain a distribution of human-like biases. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 122–133. ACM. [find](https://scholar.google.com/scholar?q=Wei+Guo+and+Aylin+Caliskan%2E+2021%2E+Detecting+emergent+intersectional+biases%3A+Contextualized+word+embeddings+contain+a+distribution+of+human%2Dlike+biases%2E+In+Proceedings+of+the+2021+AAAI%2FACM+Conference+o)
10. Dan Hendrycks, Mantas Mazeika, and Thomas Woodside. 2023. An overview of catastrophic AI risks. arXiv preprint arXiv:2306.12001. — [arXiv:2306.12001](https://arxiv.org/abs/2306.12001) [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Mantas+Mazeika%2C+and+Thomas+Woodside%2E+2023%2E+An+overview+of+catastrophic+AI+risks%2E+arXiv+preprint+arXiv%3A2306%2E12001%2E+%E2%80%94+%5BarXiv%3A2306%2E12001%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2306%2E12001%29)
11. John Hewitt and Christopher D. Manning. 2019. [A structural probe for finding syntax in word representations](https://md.research.elamur.ai/free/en/papers/hewitt-2019-structural-probe-finding-syntax-word/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4129–4138. Association for Computation…
12. Phillip Howard, Avinash Madasu, Tiep Le, Gustavo Lujan Moreno, Anahita Bhiwandiwalla, and Vasudev Lal. 2024. Probing and mitigating intersectional social biases in vision-language models with counterfactual examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio… [find](https://scholar.google.com/scholar?q=Phillip+Howard%2C+Avinash+Madasu%2C+Tiep+Le%2C+Gustavo+Lujan+Moreno%2C+Anahita+Bhiwandiwalla%2C+and+Vasudev+Lal%2E+2024%2E+Probing+and+mitigating+intersectional+social+biases+in+vision%2Dlanguage+models+with+counterf)
13. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, and 1 others. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825. [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+and+1+others%2E+20)
14. Jeongbin Kim, Matthew Kovach, Kyu-Min Lee, Eun-cheol Shin, and Hector Tzavellas. 2024. Learning to be homo economicus: Can an LLM learn preferences from choice. arXiv preprint arXiv:2401.07230. — [arXiv:2401.07230](https://arxiv.org/abs/2401.07230) [find](https://scholar.google.com/scholar?q=Jeongbin+Kim%2C+Matthew+Kovach%2C+Kyu%2DMin+Lee%2C+Eun%2Dcheol+Shin%2C+and+Hector+Tzavellas%2E+2024%2E+Learning+to+be+homo+economicus%3A+Can+an+LLM+learn+preferences+from+choice%2E+arXiv+preprint+arXiv%3A2401%2E07230%2E+%E2%80%94+%5BarX)
15. Weicheng Ma, Brian Chiang, Tong Wu, Lili Wang, and Soroush Vosoughi. 2023. Intersectional stereotypes in large language models: Dataset and analysis. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8589–8597, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Weicheng+Ma%2C+Brian+Chiang%2C+Tong+Wu%2C+Lili+Wang%2C+and+Soroush+Vosoughi%2E+2023%2E+Intersectional+stereotypes+in+large+language+models%3A+Dataset+and+analysis%2E+In+Findings+of+the+Association+for+Computational+L)
16. Chandler May, Alex Wang, Shikha Borber, Samuel R. Bowman, and Rachel Rudinger. 2019. [On measuring social biases in sentence encoders](https://md.research.elamur.ai/free/en/papers/may-2019-measuring-social-biases-sentence-encoders/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 622–628.
17. Mantas Mazeika, Xuwang Yin, Rishub Tamirisa, Jaehyuk Lim, Bruce W. Lee, Richard Ren, Long Phan, Norman Mu, Adam Khoja, Oliver Zhang, and Dan Hendrycks. 2025. Utility engineering: Analyzing and controlling emergent value systems in AIs. arXiv preprint arXiv:2502.08640. — [arXiv:2502.08640](https://arxiv.org/abs/2502.08640) [find](https://scholar.google.com/scholar?q=Mantas+Mazeika%2C+Xuwang+Yin%2C+Rishub+Tamirisa%2C+Jaehyuk+Lim%2C+Bruce+W%2E+Lee%2C+Richard+Ren%2C+Long+Phan%2C+Norman+Mu%2C+Adam+Khoja%2C+Oliver+Zhang%2C+and+Dan+Hendrycks%2E+2025%2E+Utility+engineering%3A+Analyzing+and+control)
18. Jared Moore, Tanvi Deshpande, and Diyi Yang. 2024. Are large language models consistent over value-laden questions? In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15185–15221, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jared+Moore%2C+Tanvi+Deshpande%2C+and+Diyi+Yang%2E+2024%2E+Are+large+language+models+consistent+over+value%2Dladen+questions%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2024%2C+pages+1518)
19. Alexander Pan, Jun Shern Chan, Andy Zou, Nathaniel Li, Steven Basart, Thomas Woodside, Hanlin Zhang, Scott Emmons, and Dan Hendrycks. 2023. Do the rewards justify the means? Measuring trade-offs between rewards and ethical behavior in the MACHI-AVELLI benchmark. Proceedings of the 40th International… [find](https://scholar.google.com/scholar?q=Alexander+Pan%2C+Jun+Shern+Chan%2C+Andy+Zou%2C+Nathaniel+Li%2C+Steven+Basart%2C+Thomas+Woodside%2C+Hanlin+Zhang%2C+Scott+Emmons%2C+and+Dan+Hendrycks%2E+2023%2E+Do+the+rewards+justify+the+means%3F+Measuring+trade%2Doffs+betwe)
20. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. [Whose opinions do language models reflect?](https://md.research.elamur.ai/free/en/papers/santurkar-2023-whose-opinions-language-models-reflect/) In Proceedings of the 40th International Conference on Machine Learning, pages 29971–30004. PMLR.
21. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying language models’ sensitivity to spurious features in prompt design with a focus on faithfulness. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics. Associat… [find](https://scholar.google.com/scholar?q=Melanie+Sclar%2C+Yejin+Choi%2C+Yulia+Tsvetkov%2C+and+Alane+Suhr%2E+2024%2E+Quantifying+language+models%E2%80%99+sensitivity+to+spurious+features+in+prompt+design+with+a+focus+on+faithfulness%2E+In+Proceedings+of+the+2024)
22. Badr Souani and 1 others. 2025. HInter: Exposing hidden intersectional bias in large language models. arXiv preprint arXiv:2503.11962. — [arXiv:2503.11962](https://arxiv.org/abs/2503.11962) [find](https://scholar.google.com/scholar?q=Badr+Souani+and+1+others%2E+2025%2E+HInter%3A+Exposing+hidden+intersectional+bias+in+large+language+models%2E+arXiv+preprint+arXiv%3A2503%2E11962%2E+%E2%80%94+%5BarXiv%3A2503%2E11962%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2503%2E11962%29)
23. Louis Leon Thurstone. 1927. A law of comparative judgment. Psychological Review, 34(4):273–286. [find](https://scholar.google.com/scholar?q=Louis+Leon+Thurstone%2E+1927%2E+A+law+of+comparative+judgment%2E+Psychological+Review%2C+34%284%29%3A273%E2%80%93286%2E)
24. Peiyi Wang, Lei Li, Liang Chen, Feifan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2024. Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926. [find](https://scholar.google.com/scholar?q=Peiyi+Wang%2C+Lei+Li%2C+Liang+Chen%2C+Feifan+Cai%2C+Dawei+Zhu%2C+Binghuai+Lin%2C+Yunbo+Cao%2C+Qi+Liu%2C+Tianyu+Liu%2C+and+Zhifang+Sui%2E+2024%2E+Large+language+models+are+not+fair+evaluators%2E+arXiv+preprint+arXiv%3A2305%2E1792)
25. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, and 1 others. 2023. [Judging LLM-as-a-judge with MT-Bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In Advances in Neural Information Processing Systems, volume 36.
26. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, and 1 others. 2023. [Representation engineering: A top-down approach to AI transparency](https://md.research.elamur.ai/free/en/papers/zou-2023-representation-engineering-top-down-approach/). arXiv preprint arXiv:2310.01405.

*Updated 1 November 2026*