---
title: "Value Profiles for Encoding Human Variation — Sorensen 2025"
description: "They propose value profiles: short free-text descriptions of a rater's underlying values, which an LLM… 57 references, cited by 11 papers here."
authors: ["Taylor Sorensen", "Pushkar Mishra", "Roma Patel", "Michael Henry Tessler", "Michiel A. Bakker", "Georgina Evans", "Iason Gabriel", "Noah Goodman", "Verena Rieser"]
year: 2025
venue: "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2025.emnlp-main.106/"
doi: "10.18653/v1/2025.emnlp-main.106"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sorensen-2025-value-profiles-encoding-human-variation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sorensen-2025-value-profiles-encoding-human-variation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/) / Value Profiles for Encoding Human Variation

# Value Profiles for Encoding Human Variation

**Authors:** [Taylor Sorensen](https://md.research.elamur.ai/free/en/authors/taylor-sorensen/), [Pushkar Mishra](https://md.research.elamur.ai/free/en/authors/pushkar-mishra/), [Roma Patel](https://md.research.elamur.ai/free/en/authors/roma-patel/), [Michael Henry Tessler](https://md.research.elamur.ai/free/en/authors/michael-henry-tessler/), Michiel A. Bakker, Georgina Evans, Iason Gabriel, [Noah Goodman](https://md.research.elamur.ai/free/en/authors/noah-d-goodman/) and [Verena Rieser](https://md.research.elamur.ai/free/en/authors/verena-rieser/) · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2025](https://md.research.elamur.ai/free/en/venues/emnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.emnlp-main.106/) · [PDF](https://aclanthology.org/2025.emnlp-main.106.pdf) · [DOI 10.18653/v1/2025.emnlp-main.106](https://doi.org/10.18653/v1/2025.emnlp-main.106) · [OpenAlex](https://openalex.org/W4416037047)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [11](https://md.research.elamur.ai/free/en/papers/sorensen-2025-value-profiles-encoding-human-variation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Annotators often disagree on subjective rating tasks such as toxicity, hate speech or chat preferences. This disagreement is usually discarded as noise. The authors instead model each rater individually and ask how a person is best represented. They propose value profiles: short free-text descriptions of a rater's underlying values, which an LLM encoder distils from that person's example ratings. A trained, steerable decoder then predicts how the described individual would rate new items. To compare representations, they introduce an information-theoretic measure of how much predictive signal each one carries. Experiments cover six tasks and datasets, with models from the Gemma-2 and Gemini families. Raw demonstrations carry the most information, value profiles come second and demographics come last. Profiles still keep more than 70% of the useful signal in demonstrations, and unlike demonstrations they are readable and editable by people. This makes them an interpretable alternative to demographic grouping for personalization, pluralistic alignment and computational social science.

## Key points
- The paper describes three ways to model variation: population distributions, grouping by traits, and individual modelling. It argues for individual modelling, because group or population views can be recovered by marginalizing and raters need not label overlapping items.
- Value profiles preserve more than 70% of the predictive information found in in-context demonstrations.
- Raters grouped by clustering their value profiles account for disagreement better than the most predictive demographic groupings.
- Decoder outputs shift in step with semantic edits to a profile and are well-calibrated. Simulating a population of annotators helps explain disagreement on individual items.
- The study ran more than 650 training runs across more than twenty demographic distributions, using only Gemma-2 and Gemini models. The profiles are in English only, and the authors raise privacy and misrepresentation risks.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 353 works · 200,043 citations
- [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) — Company, London, United Kingdom · 56 works · 78,880 citations
- [Google (United Kingdom)](https://md.research.elamur.ai/free/en/companies/google-united-kingdom/) — Company, London, United Kingdom · 10 works · 1,009 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 870 works

## Cite

Cite as: Taylor Sorensen, Pushkar Mishra, Roma Patel, Michael Henry Tessler, Michiel A. Bakker, Georgina Evans, Iason Gabriel, Noah Goodman and Verena Rieser. 2025. Value Profiles for Encoding Human Variation. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sorensen2025value,
  title = {{Value Profiles for Encoding Human Variation}},
  author = {Taylor Sorensen, Pushkar Mishra, Roma Patel, Michael Henry Tessler, Michiel A. Bakker, Georgina Evans, Iason Gabriel, Noah Goodman and Verena Rieser},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.106},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.106}
}
```

- [← previous URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Base](https://md.research.elamur.ai/free/en/papers/khan-2025-uriel-enhancing-linguistic-inclusion-usability/)
- [next → What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practices](https://md.research.elamur.ai/free/en/papers/chen-2025-what-are-essential-factors-crafting/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/)

## References

57 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hanna, Alexander Koller, André F. T. Martins, Philipp Mondorf, Vera Neplenbroek, Sandro Pezzelle, Barbara Plank, David Schlangen, Alessandro Suglia, Aditya K. Surikuchi, Ece Takmaz, and Alberto Testoni. 2024. [Llms instead of human judges? a large scale empirical study across 20 nlp evaluation tasks](https://md.research.elamur.ai/free/en/papers/bavaresco-2024-llms-instead-human-judges-large/).…
2. Gavin Abercrombie, Verena Rieser, and Dirk Hovy. 2023. Consistency is key: Disentangling label variation in natural language processing with intra-annotator agreement. Preprint, arXiv:2301.10684. [find](https://scholar.google.com/scholar?q=Gavin+Abercrombie%2C+Verena+Rieser%2C+and+Dirk+Hovy%2E+2023%2E+Consistency+is+key%3A+Disentangling+label+variation+in+natural+language+processing+with+intra%2Dannotator+agreement%2E+Preprint%2C+arXiv%3A2301%2E10684%2E)
3. William Agnew, A. Stevie Bergman, Jennifer Chien, Mark Díaz, Seliem El-Sayed, Jaylen Pittman, Shakir Mohamed, and Kevin R. McKee. 2024. The illusion of artificial inclusion. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24, New York, NY, USA. Association for C… [find](https://scholar.google.com/scholar?q=William+Agnew%2C+A%2E+Stevie+Bergman%2C+Jennifer+Chien%2C+Mark+D%C3%ADaz%2C+Seliem+El%2DSayed%2C+Jaylen+Pittman%2C+Shakir+Mohamed%2C+and+Kevin+R%2E+McKee%2E+2024%2E+The+illusion+of+artificial+inclusion%2E+In+Proceedings+of+the+2024)
4. Carlos Aguirre, Kuleen Sasse, Isabel Cachola, and Mark Dredze. 2023. Selecting shots for demographic fairness in few-shot learning with large language models. Preprint, arXiv:2311.08472. [find](https://scholar.google.com/scholar?q=Carlos+Aguirre%2C+Kuleen+Sasse%2C+Isabel+Cachola%2C+and+Mark+Dredze%2E+2023%2E+Selecting+shots+for+demographic+fairness+in+few%2Dshot+learning+with+large+language+models%2E+Preprint%2C+arXiv%3A2311%2E08472%2E)
5. Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. 2023. [Out of one, many: Using language models to simulate human samples](https://md.research.elamur.ai/free/en/papers/argyle-2022-out-one-many-using-language/). Political Analysis, page 1–15.
6. Lora Aroyo, Alex S. Taylor, Mark Diaz, Christopher M. Homan, Alicia Parrish, Greg Serapio-Garcia, Vinodkumar Prabhakaran, and Ding Wang. 2023. Dices dataset: Diversity in conversational ai evaluation for safety. Preprint, arXiv:2306.11247. — [arXiv:2306.11247](https://arxiv.org/abs/2306.11247) [find](https://scholar.google.com/scholar?q=Lora+Aroyo%2C+Alex+S%2E+Taylor%2C+Mark+Diaz%2C+Christopher+M%2E+Homan%2C+Alicia+Parrish%2C+Greg+Serapio%2DGarcia%2C+Vinodkumar+Prabhakaran%2C+and+Ding+Wang%2E+2023%2E+Dices+dataset%3A+Diversity+in+conversational+ai+evaluation+)
7. Lora Aroyo and Chris Welty. 2015. [Truth is a lie: Crowd truth and the seven myths of human annotation](https://md.research.elamur.ai/free/en/papers/aroyo-2015-truth-lie-crowd-truth-seven/). AI Magazine, 36(1):15–24.
8. Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2019. Explainable artificial intelligence (xai): Concepts, taxonomies, opportuni… [find](https://scholar.google.com/scholar?q=Alejandro+Barredo+Arrieta%2C+Natalia+D%C3%ADaz%2DRodr%C3%ADguez%2C+Javier+Del+Ser%2C+Adrien+Bennetot%2C+Siham+Tabik%2C+Alberto+Barbado%2C+Salvador+Garc%C3%ADa%2C+Sergio+Gil%2DL%C3%B3pez%2C+Daniel+Molina%2C+Richard+Benjamins%2C+Raja+Chatila%2C+and)
9. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, and 32 others. 20… — [Constitutional AI: Harmlessness from AI Feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
10. Krisztian Balog, Filip Radlinski, and Shushan Arakelyan. 2019. Transparent, scrutable and explainable user models for personalized recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’19, page 265–274, New York, NY,… [find](https://scholar.google.com/scholar?q=Krisztian+Balog%2C+Filip+Radlinski%2C+and+Shushan+Arakelyan%2E+2019%2E+Transparent%2C+scrutable+and+explainable+user+models+for+personalized+recommendation%2E+In+Proceedings+of+the+42nd+International+ACM+SIGIR+Co)
11. Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Quan Ze Chen, K. J. Kevin Feng, Chan Young Park, and Amy X. Zhang. 2024. Spica: Retrieving scenarios for pluralistic in-context alignment. Preprint, arXiv:… [find](https://scholar.google.com/scholar?q=Anna+Bavaresco%2C+Raffaella+Bernardi%2C+Leonardo+Bertolazzi%2C+Desmond+Elliott%2C+Raquel+Fern%C3%A1ndez%2C+Albert+Gatt%2C+Esam+Ghaleb%2C+Mario+Giulianelli%2C+Michael+Quan+Ze+Chen%2C+K%2E+J%2E+Kevin+Feng%2C+Chan+Young+Park%2C+and+Am)
12. Myra Cheng, Esin Durmus, and Dan Jurafsky. 2023. [Marked personas: Using natural language prompts to measure stereotypes in language models](https://md.research.elamur.ai/free/en/papers/cheng-2023-marked-personas-natural-language-prompts/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1504–1532, Toronto, Canada. Associatio…
13. Soham De, Michiel A Bakker, Jay Baxter, and Martin Saveski. 2024. Supernotes: Driving consensus in crowd-sourced fact-checking. arXiv preprint arXiv:2411.06116. — [arXiv:2411.06116](https://arxiv.org/abs/2411.06116) [find](https://scholar.google.com/scholar?q=Soham+De%2C+Michiel+A+Bakker%2C+Jay+Baxter%2C+and+Martin+Saveski%2E+2024%2E+Supernotes%3A+Driving+consensus+in+crowd%2Dsourced+fact%2Dchecking%2E+arXiv+preprint+arXiv%3A2411%2E06116%2E+%E2%80%94+%5BarXiv%3A2411%2E06116%5D%28https%3A%2F%2Farxiv%2Eorg%2F)
14. Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, and Kai-Wei Chang. 2022. On measures of biases and harms in NLP. In Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, pages 246–267, Online only… [find](https://scholar.google.com/scholar?q=Sunipa+Dev%2C+Emily+Sheng%2C+Jieyu+Zhao%2C+Aubrie+Amstutz%2C+Jiao+Sun%2C+Yu+Hou%2C+Mattie+Sanseverino%2C+Jiin+Kim%2C+Akihiro+Nishi%2C+Nanyun+Peng%2C+and+Kai%2DWei+Chang%2E+2022%2E+On+measures+of+biases+and+harms+in+NLP%2E+In+Fin)
15. R. M. Dworkin. 2002. Sovereign virtue: The theory and practice of equality. Philosophical Quarterly, 52(208):377–389. [find](https://scholar.google.com/scholar?q=R%2E+M%2E+Dworkin%2E+2002%2E+Sovereign+virtue%3A+The+theory+and+practice+of+equality%2E+Philosophical+Quarterly%2C+52%28208%29%3A377%E2%80%93389%2E)
16. Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. 2022. Understanding dataset difficulty with V-usable information. Preprint, arXiv:2110.08420. [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2C+Yejin+Choi%2C+and+Swabha+Swayamdipta%2E+2022%2E+Understanding+dataset+difficulty+with+V%2Dusable+information%2E+Preprint%2C+arXiv%3A2110%2E08420%2E)
17. Arduin Findeis, Timo Kaufmann, Eyke Hüllermeier, Samuel Albanie, and Robert Mullins. 2024. Inverse constitutional ai: Compressing preferences into principles. Preprint, arXiv:2406.06560. — [arXiv:2406.06560](https://arxiv.org/abs/2406.06560) [find](https://scholar.google.com/scholar?q=Arduin+Findeis%2C+Timo+Kaufmann%2C+Eyke+H%C3%BCllermeier%2C+Samuel+Albanie%2C+and+Robert+Mullins%2E+2024%2E+Inverse+constitutional+ai%3A+Compressing+preferences+into+principles%2E+Preprint%2C+arXiv%3A2406%2E06560%2E+%E2%80%94+%5BarXiv%3A2406)
18. Iason Gabriel. 2020. Artificial intelligence, values, and alignment. Minds and Machines, 30(3):411–437. [find](https://scholar.google.com/scholar?q=Iason+Gabriel%2E+2020%2E+Artificial+intelligence%2C+values%2C+and+alignment%2E+Minds+and+Machines%2C+30%283%29%3A411%E2%80%93437%2E)
19. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, and 179… [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+Cassidy+Hardin%2C+Surya+Bhupatiraju%2C+L%C3%A9onard+Hussenot%2C+Thomas+Mesnard%2C+Bobak+Shahriari%2C+Alexandre+Ram%C3%A9%2C+Johan+Ferret%2C+Peter+Liu%2C+Pouya+Ta)
20. Mitchell L. Gordon, Michelle S. Lam, Joon Sung Park, Kayur Patel, Jeff Hancock, Tatsunori Hashimoto, and Michael S. Bernstein. 2022. Jury learning: Integrating dissenting voices into machine learning models. In CHI Conference on Human Factors in Computing Systems, CHI ’22, page 1–19. ACM. [find](https://scholar.google.com/scholar?q=Mitchell+L%2E+Gordon%2C+Michelle+S%2E+Lam%2C+Joon+Sung+Park%2C+Kayur+Patel%2C+Jeff+Hancock%2C+Tatsunori+Hashimoto%2C+and+Michael+S%2E+Bernstein%2E+2022%2E+Jury+learning%3A+Integrating+dissenting+voices+into+machine+learning+)
21. Tiancheng Hu and Nigel Collier. 2024. [Quantifying the persona effect in llm simulations](https://md.research.elamur.ai/free/en/papers/hu-2024-quantifying-persona-effect-llm-simulations/). Preprint, arXiv:2402.10811.
22. EunJeong Hwang, Bodhisattwa Prasad Majumder, and Niket Tandon. 2023. [Aligning language models to user opinions](https://md.research.elamur.ai/free/en/papers/hwang-2023-aligning-language-models-user-opinions/). Preprint, arXiv:2305.14929.
23. Ziwei Ji, Justin D. Li, and Matus Telgarsky. 2021. Early-stopped neural networks are consistent. Preprint, arXiv:2106.05932. — [arXiv:2106.05932](https://arxiv.org/abs/2106.05932) [find](https://scholar.google.com/scholar?q=Ziwei+Ji%2C+Justin+D%2E+Li%2C+and+Matus+Telgarsky%2E+2021%2E+Early%2Dstopped+neural+networks+are+consistent%2E+Preprint%2C+arXiv%3A2106%2E05932%2E+%E2%80%94+%5BarXiv%3A2106%2E05932%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2106%2E05932%29)
24. Liwei Jiang, Taylor Sorensen, Sydney Levine, and Yejin Choi. 2024. [Can language models reason about individualistic human values and preferences?](https://md.research.elamur.ai/free/en/papers/jiang-2025-can-language-models-reason-about/) Preprint, arXiv:2410.03868.
25. Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, and Scott A Hale. 2024a. The benefits, risks and bounds of personalizing the alignment of large language models to individuals. Nature Machine Intelligence, pages 1–10. [find](https://scholar.google.com/scholar?q=Hannah+Rose+Kirk%2C+Bertie+Vidgen%2C+Paul+R%C3%B6ttger%2C+and+Scott+A+Hale%2E+2024a%2E+The+benefits%2C+risks+and+bounds+of+personalizing+the+alignment+of+large+language+models+to+individuals%2E+Nature+Machine+Intelligen)
26. Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew Bean, Katerina Margatina, Juan Ciro, Rafael Mosquera, Max Bartolo, Adina Williams, He He, Bertie Vidgen, and Scott A. Hale. 2024b. The prism alignment dataset: What participatory, representative and individualised human feedback reveals ab… [find](https://scholar.google.com/scholar?q=Hannah+Rose+Kirk%2C+Alexander+Whitefield%2C+Paul+R%C3%B6ttger%2C+Andrew+Bean%2C+Katerina+Margatina%2C+Juan+Ciro%2C+Rafael+Mosquera%2C+Max+Bartolo%2C+Adina+Williams%2C+He+He%2C+Bertie+Vidgen%2C+and+Scott+A%2E+Hale%2E+2024b%2E+The+pris)
27. Oliver Klingefjord, Ryan Lowe, and Joe Edelman. 2024. What are human values, and how do we align ai to them? Preprint, arXiv:2404.10636. — [arXiv:2404.10636](https://arxiv.org/abs/2404.10636) [find](https://scholar.google.com/scholar?q=Oliver+Klingefjord%2C+Ryan+Lowe%2C+and+Joe+Edelman%2E+2024%2E+What+are+human+values%2C+and+how+do+we+align+ai+to+them%3F+Preprint%2C+arXiv%3A2404%2E10636%2E+%E2%80%94+%5BarXiv%3A2404%2E10636%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2404%2E10636%29)
28. Carl Knight. 2025. Reflective Equilibrium. In Edward N. Zalta and Uri Nodelman, editors, The Stanford Encyclopedia of Philosophy, Spring 2025 edition. Metaphysics Research Lab, Stanford University. [find](https://scholar.google.com/scholar?q=Carl+Knight%2E+2025%2E+Reflective+Equilibrium%2E+In+Edward+N%2E+Zalta+and+Uri+Nodelman%2C+editors%2C+The+Stanford+Encyclopedia+of+Philosophy%2C+Spring+2025+edition%2E+Metaphysics+Research+Lab%2C+Stanford+University%2E)
29. Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. 2020. Concept bottleneck models. Preprint, arXiv:2007.04612. [find](https://scholar.google.com/scholar?q=Pang+Wei+Koh%2C+Thao+Nguyen%2C+Yew+Siang+Tang%2C+Stephen+Mussmann%2C+Emma+Pierson%2C+Been+Kim%2C+and+Percy+Liang%2E+2020%2E+Concept+bottleneck+models%2E+Preprint%2C+arXiv%3A2007%2E04612%2E)
30. Deepak Kumar, Patrick Gage Kelley, Sunny Consolvo, Joshua Mason, Elie Bursztein, Zakir Durumeric, Kurt Thomas, and Michael Bailey. 2021. Designing toxic content classification for a diversity of perspectives. Preprint, arXiv:2106.04511. — [arXiv:2106.04511](https://arxiv.org/abs/2106.04511) [find](https://scholar.google.com/scholar?q=Deepak+Kumar%2C+Patrick+Gage+Kelley%2C+Sunny+Consolvo%2C+Joshua+Mason%2C+Elie+Bursztein%2C+Zakir+Durumeric%2C+Kurt+Thomas%2C+and+Michael+Bailey%2E+2021%2E+Designing+toxic+content+classification+for+a+diversity+of+persp)
31. Seth Lazar, Luke Thorburn, Tian Jin, and Luca Belli. 2024. The moral case for using language model agents for recommendation. Preprint, arXiv:2410.12123. — [arXiv:2410.12123](https://arxiv.org/abs/2410.12123) [find](https://scholar.google.com/scholar?q=Seth+Lazar%2C+Luke+Thorburn%2C+Tian+Jin%2C+and+Luca+Belli%2E+2024%2E+The+moral+case+for+using+language+model+agents+for+recommendation%2E+Preprint%2C+arXiv%3A2410%2E12123%2E+%E2%80%94+%5BarXiv%3A2410%2E12123%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F241)
32. Junyi Li, Charith Peris, Ninareh Mehrabi, Palash Goyal, Kai-Wei Chang, Aram Galstyan, Richard Zemel, and Rahul Gupta. 2024. The steerability of large language models toward data-driven personas. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational… [find](https://scholar.google.com/scholar?q=Junyi+Li%2C+Charith+Peris%2C+Ninareh+Mehrabi%2C+Palash+Goyal%2C+Kai%2DWei+Chang%2C+Aram+Galstyan%2C+Richard+Zemel%2C+and+Rahul+Gupta%2E+2024%2E+The+steerability+of+large+language+models+toward+data%2Ddriven+personas%2E+In+Pr)
33. Nicole Meister, Carlos Guestrin, and Tatsunori Hashimoto. 2024. [Benchmarking distributional alignment of large language models](https://md.research.elamur.ai/free/en/papers/meister-2025-benchmarking-distributional-alignment-large-language/). Preprint, arXiv:2411.05403.
34. Christopher Nguyen. 2024. Value capture. Journal of Ethics and Social Philosophy, 27(3). [find](https://scholar.google.com/scholar?q=Christopher+Nguyen%2E+2024%2E+Value+capture%2E+Journal+of+Ethics+and+Social+Philosophy%2C+27%283%29%2E)
35. Matthias Orlikowski, Jiaxin Pei, Paul Röttger, Philipp Cimiano, David Jurgens, and Dirk Hovy. 2025. [Beyond demographics: Fine-tuning large language models to predict individuals’ subjective text perceptions](https://md.research.elamur.ai/free/en/papers/orlikowski-2025-beyond-demographics-fine-tuning-large/). Preprint, arXiv:2502.20897.
36. Matthias Orlikowski, Paul Röttger, Philipp Cimiano, and Dirk Hovy. 2023. The ecological fallacy in annotation: Modeling human label variation goes beyond sociodemographics. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1017… [find](https://scholar.google.com/scholar?q=Matthias+Orlikowski%2C+Paul+R%C3%B6ttger%2C+Philipp+Cimiano%2C+and+Dirk+Hovy%2E+2023%2E+The+ecological+fallacy+in+annotation%3A+Modeling+human+label+variation+goes+beyond+sociodemographics%2E+In+Proceedings+of+the+61st+)
37. Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. Preprint, arXiv:2304.03442. [find](https://scholar.google.com/scholar?q=Joon+Sung+Park%2C+Joseph+C%2E+O%E2%80%99Brien%2C+Carrie+J%2E+Cai%2C+Meredith+Ringel+Morris%2C+Percy+Liang%2C+and+Michael+S%2E+Bernstein%2E+2023%2E+Generative+agents%3A+Interactive+simulacra+of+human+behavior%2E+Preprint%2C+arXiv%3A2304%2E)
38. Joon Sung Park, Carolyn Q. Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S. Bernstein. 2024. Generative agent simulations of 1,000 people. Preprint, arXiv:2411.10109. [find](https://scholar.google.com/scholar?q=Joon+Sung+Park%2C+Carolyn+Q%2E+Zou%2C+Aaron+Shaw%2C+Benjamin+Mako+Hill%2C+Carrie+Cai%2C+Meredith+Ringel+Morris%2C+Robb+Willer%2C+Percy+Liang%2C+and+Michael+S%2E+Bernstein%2E+2024%2E+Generative+agent+simulations+of+1%2C000+peop)
39. Barbara Plank. 2022. [The “problem” of human label variation: On ground truth in data, modeling and evaluation](https://md.research.elamur.ai/free/en/papers/plank-2022-problem-human-label-variation-ground/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10671–10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
40. Sriyash Poddar, Yanming Wan, Hamish Ivison, Abhishek Gupta, and Natasha Jaques. 2024. Personalizing reinforcement learning from human feedback with variational preference learning. In Advances in Neural Information Processing Systems, volume 37, pages 52516–52544. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Sriyash+Poddar%2C+Yanming+Wan%2C+Hamish+Ivison%2C+Abhishek+Gupta%2C+and+Natasha+Jaques%2E+2024%2E+Personalizing+reinforcement+learning+from+human+feedback+with+variational+preference+learning%2E+In+Advances+in+Neur)
41. Vinodkumar Prabhakaran, Christopher Homan, Lora Aroyo, Aida Mostafazadeh Davani, Alicia Parrish, Alex Taylor, Mark Diaz, Ding Wang, and Gregory Serapio-García. 2024. GRASP: A disagreement analysis framework to assess group associations in perspectives. In Proceedings of the 2024 Conference of the No… [find](https://scholar.google.com/scholar?q=Vinodkumar+Prabhakaran%2C+Christopher+Homan%2C+Lora+Aroyo%2C+Aida+Mostafazadeh+Davani%2C+Alicia+Parrish%2C+Alex+Taylor%2C+Mark+Diaz%2C+Ding+Wang%2C+and+Gregory+Serapio%2DGarc%C3%ADa%2E+2024%2E+GRASP%3A+A+disagreement+analysis+fra)
42. Filip Radlinski, Krisztian Balog, Fernando Diaz, Lucas Dixon, and Ben Wedin. 2022. On natural language user profiles for transparent and scrutable recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’22, page 2863–… [find](https://scholar.google.com/scholar?q=Filip+Radlinski%2C+Krisztian+Balog%2C+Fernando+Diaz%2C+Lucas+Dixon%2C+and+Ben+Wedin%2E+2022%2E+On+natural+language+user+profiles+for+transparent+and+scrutable+recommendation%2E+In+Proceedings+of+the+45th+Internatio)
43. Nikolas Vitsakis, Amit Parekh, and Ioannis Konstas. 2024. Voices in a crowd: Searching for clusters of unique perspectives. Preprint, arXiv:2407.14259. [find](https://scholar.google.com/scholar?q=Nikolas+Vitsakis%2C+Amit+Parekh%2C+and+Ioannis+Konstas%2E+2024%2E+Voices+in+a+crowd%3A+Searching+for+clusters+of+unique+perspectives%2E+Preprint%2C+arXiv%3A2407%2E14259%2E)
44. Jerome Ramos, Hossein A. Rahmani, Xi Wang, Xiao Fu, and Aldo Lipani. 2024. Transparent and scrutable recommendations using natural language user profiles. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13971–13984, Bangkok, T… [find](https://scholar.google.com/scholar?q=Jerome+Ramos%2C+Hossein+A%2E+Rahmani%2C+Xi+Wang%2C+Xiao+Fu%2C+and+Aldo+Lipani%2E+2024%2E+Transparent+and+scrutable+recommendations+using+natural+language+user+profiles%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+)
45. Angelina Wang, Jamie Morgenstern, and John P. Dickerson. 2025. Large language models that replace human participants can harmfully misportray and flatten identity groups. Nature Machine Intelligence. [find](https://scholar.google.com/scholar?q=Angelina+Wang%2C+Jamie+Morgenstern%2C+and+John+P%2E+Dickerson%2E+2025%2E+Large+language+models+that+replace+human+participants+can+harmfully+misportray+and+flatten+identity+groups%2E+Nature+Machine+Intelligence%2E)
46. John Rawls. 2005. Political Liberalism, expanded edition edition. Columbia University Press, New York. [find](https://scholar.google.com/scholar?q=John+Rawls%2E+2005%2E+Political+Liberalism%2C+expanded+edition+edition%2E+Columbia+University+Press%2C+New+York%2E)
47. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. [Whose opinions do language models reflect?](https://md.research.elamur.ai/free/en/papers/santurkar-2023-whose-opinions-language-models-reflect/) Preprint, arXiv:2303.17548.
48. Anand Siththaranjan, Cassidy Laidlaw, and Dylan Hadfield-Menell. 2024. Distributional preference learning: Understanding and accounting for hidden context in rlhf. Preprint, arXiv:2312.08358. — [arXiv:2312.08358](https://arxiv.org/abs/2312.08358) [find](https://scholar.google.com/scholar?q=Anand+Siththaranjan%2C+Cassidy+Laidlaw%2C+and+Dylan+Hadfield%2DMenell%2E+2024%2E+Distributional+preference+learning%3A+Understanding+and+accounting+for+hidden+context+in+rlhf%2E+Preprint%2C+arXiv%3A2312%2E08358%2E+%E2%80%94+%5BarXiv)
49. Taylor Sorensen, Liwei Jiang, Jena D. Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, Maarten Sap, John Tasioulas, and Yejin Choi. 2024a. [Value kaleidoscope: Engaging ai with pluralistic human values, rights, and duties](https://md.research.elamur.ai/free/en/papers/sorensen-2023-value-kaleidoscope-engaging-ai-pluralistic/). Proceedings of the…
50. Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. 2024b. [A roadmap to pluralistic alignment](https://md.research.elamur.ai/free/en/papers/sorensen-2024-roadmap-pluralistic-alignment/). Preprint, arXiv:2402.05070.
51. Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. 2022. [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing in…](https://md.research.elamur.ai/free/en/papers/wortsman-2022-model-soups-averaging-weights-multiple/)
52. Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, and Stefano Ermon. 2020. A theory of usable information under computational constraints. Preprint, arXiv:2002.10689. [find](https://scholar.google.com/scholar?q=Yilun+Xu%2C+Shengjia+Zhao%2C+Jiaming+Song%2C+Russell+Stewart%2C+and+Stefano+Ermon%2E+2020%2E+A+theory+of+usable+information+under+computational+constraints%2E+Preprint%2C+arXiv%3A2002%2E10689%2E)
53. Michael JQ Zhang, Zhilin Wang, Jena D. Hwang, Yi Dong, Olivier Delalleau, Yejin Choi, Eunsol Choi, Xiang Ren, and Valentina Pyatkin. 2024. Diverging preferences: When do annotators disagree and do models know? Preprint, arXiv:2410.14632. [find](https://scholar.google.com/scholar?q=Michael+JQ+Zhang%2C+Zhilin+Wang%2C+Jena+D%2E+Hwang%2C+Yi+Dong%2C+Olivier+Delalleau%2C+Yejin+Choi%2C+Eunsol+Choi%2C+Xiang+Ren%2C+and+Valentina+Pyatkin%2E+2024%2E+Diverging+preferences%3A+When+do+annotators+disagree+and+do+mod)
54. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. [Personalizing dialogue agents: I have a dog, do you have pets too?](https://md.research.elamur.ai/free/en/papers/zhang-2018-personalizing-dialogue-agents-i-have/) In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbou…
55. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, Xinyang Geng, Fred Alcober, Roy Frostig, Mark Omernick, Lexi Walker, Cosmin Paduraru, Christina Sorokin, and 1118 others. 2024. G… — [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/)
56. Michael Henry Tessler, Michiel A. Bakker, Daniel Jarrett, Hannah Sheahan, Martin J. Chadwick, Raphael Koster, Georgina Evans, Lucy Campbell-Gillingham, Tantum Collins, David C. Parkes, Matthew Botvinick, and Christopher Summerfield. 2024. Ai can help humans find common ground in democratic deliberat… [find](https://scholar.google.com/scholar?q=Michael+Henry+Tessler%2C+Michiel+A%2E+Bakker%2C+Daniel+Jarrett%2C+Hannah+Sheahan%2C+Martin+J%2E+Chadwick%2C+Raphael+Koster%2C+Georgina+Evans%2C+Lucy+Campbell%2DGillingham%2C+Tantum+Collins%2C+David+C%2E+Parkes%2C+Matthew+Botvini)
57. Nenad Tomasev, Kevin R. McKee, Jackie Kay, and Shakir Mohamed. 2021. Fairness for unobserved characteristics: Insights from technological impacts on queer communities. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’21, page 254–265, New York, NY, USA. Association fo… [find](https://scholar.google.com/scholar?q=Nenad+Tomasev%2C+Kevin+R%2E+McKee%2C+Jackie+Kay%2C+and+Shakir+Mohamed%2E+2021%2E+Fairness+for+unobserved+characteristics%3A+Insights+from+technological+impacts+on+queer+communities%2E+In+Proceedings+of+the+2021+AAAI%2F)

*Updated 30 October 2026*