---
title: "Do Language Models Think Consistently? A Study of Value"
description: "This paper asks whether value preferences measured in short-form tests match those that appear in long-form generations… 52 references."
authors: ["Inderjeet Jayakumar Nair", "Lu Wang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.514/"
doi: "10.18653/v1/2026.findings-acl.514"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nair-2026-do-language-models-think-consistently/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nair-2026-do-language-models-think-consistently/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/27/) / Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengths

# Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengths

**Authors:** Inderjeet Jayakumar Nair and [Lu Wang](https://md.research.elamur.ai/free/en/authors/lu-wang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.514/) · [PDF](https://aclanthology.org/2026.findings-acl.514.pdf) · [DOI 10.18653/v1/2026.findings-acl.514](https://doi.org/10.18653/v1/2026.findings-acl.514) · [OpenAlex](https://openalex.org/W4415132985)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Most audits of LLM values use multiple-choice questionnaires or short answers, but real users get long, open-ended replies. This paper asks whether value preferences measured in short-form tests match those that appear in long-form generations, and whether long-form outputs agree with each other when the requested number of arguments changes. The authors prompt 10 models from five families, covering both pre-trained and aligned checkpoints, on DailyDilemmas and on OpinionQA. Short-form preferences come from the action a model picks in a dilemma. Long-form preferences come from the order in which value-laden arguments appear. Both are turned into value vectors with a TrueSkill-style Gaussian belief update. The two formats correlate only weakly, and so do pairs of long-form settings. Alignment helps consistency only slightly. Models are also more stable on everyday dilemmas than on contentious social questions. These results suggest that short-form value benchmarks may misrepresent how a model behaves in deployment.

## Key points
- **Models:** the five main models are llama3-8b, gemma2-9b, mistral-7b, qwen2-7b and olmo-7b. All are under 10B parameters, with only limited checks on larger models (Appendix C.8).
- **Data:**
  - DailyDilemmas: 1360 everyday dilemmas, each with two actions annotated with values. Its 301 values serve as the value system.
  - OpinionQA: 63 open-ended questions on topics such as health, automation and crime. It is used to test a domain shift and has no value annotations.
- **Method:** each value gets a Gaussian belief N(μ, σ²) that TrueSkill updates after every decision, so values within an action set can carry unequal weight. The μ parameter is used as the preference score.
- **Generation attributes:** values a model prefers more strongly appear across more varied scenarios (higher diversity) but in less specific arguments (lower specificity).
- **Scope:** the study covers English only. It proposes, but does not test, consistency-oriented training and interpretability analyses as future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Michigan](https://md.research.elamur.ai/free/en/universities/university-of-michigan/) — University, Ann Arbor, United States · 70 works · 13,693 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Inderjeet Jayakumar Nair and Lu Wang. 2026. Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengths. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nair2026do,
  title = {{Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengths}},
  author = {Inderjeet Jayakumar Nair and Lu Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.514},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.514}
}
```

- [← previous Do I know what I want to say? Modeling meaning uncertainty in RSA](https://md.research.elamur.ai/free/en/papers/wang-2026-do-i-know-what-i/)
- [next → Do Large Language Models Understand Double Mismatches? Evidence from Farsi](https://md.research.elamur.ai/free/en/papers/mohammadi-2026-do-large-language-models-understand/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/27/)

## References

52 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Arnav Arora, Lucie-aimée Kaffee, and Isabelle Augenstein. 2023. [Probing pre-trained language models for cross-cultural differences in values](https://md.research.elamur.ai/free/en/papers/arora-2023-probing-pre-trained-language-models/). In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 114–130, Dubrovnik, Croatia. Association for Computational Linguis…
2. Pablo Biedma, Xiaoyuan Yi, Linus Huang, Maosong Sun, and Xing Xie. 2024. Beyond human norms: Unveiling unique values of large language models through interdisciplinary approaches. arXiv preprint arXiv:2404.12744. — [arXiv:2404.12744](https://arxiv.org/abs/2404.12744) [find](https://scholar.google.com/scholar?q=Pablo+Biedma%2C+Xiaoyuan+Yi%2C+Linus+Huang%2C+Maosong+Sun%2C+and+Xing+Xie%2E+2024%2E+Beyond+human+norms%3A+Unveiling+unique+values+of+large+language+models+through+interdisciplinary+approaches%2E+arXiv+preprint+arXiv)
3. Vamshi Krishna Bonagiri, Sreeram Vennam, Priyanshul Govil, Ponnurangam Kumaraguru, and Manas Gaur. 2024. Sage: Evaluating moral consistency in large language models. arXiv preprint arXiv:2402.13709. [find](https://scholar.google.com/scholar?q=Vamshi+Krishna+Bonagiri%2C+Sreeram+Vennam%2C+Priyanshul+Govil%2C+Ponnurangam+Kumaraguru%2C+and+Manas+Gaur%2E+2024%2E+Sage%3A+Evaluating+moral+consistency+in+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E13709%2E)
4. Angana Borah and Rada Mihalcea. 2024. Towards implicit bias detection and mitigation in multi-agent llm interactions. arXiv preprint arXiv:2410.02584. [find](https://scholar.google.com/scholar?q=Angana+Borah+and+Rada+Mihalcea%2E+2024%2E+Towards+implicit+bias+detection+and+mitigation+in+multi%2Dagent+llm+interactions%2E+arXiv+preprint+arXiv%3A2410%2E02584%2E)
5. Winston Carlile, Nishant Gurrapadi, Zixuan Ke, and Vincent Ng. 2018. Give me more feedback: Annotating argument persuasiveness and related attributes in student essays. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 621–631,… [find](https://scholar.google.com/scholar?q=Winston+Carlile%2C+Nishant+Gurrapadi%2C+Zixuan+Ke%2C+and+Vincent+Ng%2E+2018%2E+Give+me+more+feedback%3A+Annotating+argument+persuasiveness+and+related+attributes+in+student+essays%2E+In+Proceedings+of+the+56th+Annu)
6. Yu Ying Chiu, Liwei Jiang, and Yejin Choi. 2024. Dailydilemmas: Revealing value preferences of llms with quandaries of daily life. arXiv preprint arXiv:2410.02683. — [arXiv:2410.02683](https://arxiv.org/abs/2410.02683) [find](https://scholar.google.com/scholar?q=Yu+Ying+Chiu%2C+Liwei+Jiang%2C+and+Yejin+Choi%2E+2024%2E+Dailydilemmas%3A+Revealing+value+preferences+of+llms+with+quandaries+of+daily+life%2E+arXiv+preprint+arXiv%3A2410%2E02683%2E+%E2%80%94+%5BarXiv%3A2410%2E02683%5D%28https%3A%2F%2Farxiv%2Eo)
7. Rochelle Choenni, Anne Lauscher, and Ekaterina Shutova. 2024. The echoes of multilinguality: Tracing cultural value shifts during lm fine-tuning. arXiv preprint arXiv:2405.12744. — [arXiv:2405.12744](https://arxiv.org/abs/2405.12744) [find](https://scholar.google.com/scholar?q=Rochelle+Choenni%2C+Anne+Lauscher%2C+and+Ekaterina+Shutova%2E+2024%2E+The+echoes+of+multilinguality%3A+Tracing+cultural+value+shifts+during+lm+fine%2Dtuning%2E+arXiv+preprint+arXiv%3A2405%2E12744%2E+%E2%80%94+%5BarXiv%3A2405%2E12744%5D%28)
8. Esin Durmus, Faisal Ladhak, and Claire Cardie. 2019. Determining relative argument specificity and stance for complex argumentative structures. arXiv preprint arXiv:1906.11313. [find](https://scholar.google.com/scholar?q=Esin+Durmus%2C+Faisal+Ladhak%2C+and+Claire+Cardie%2E+2019%2E+Determining+relative+argument+specificity+and+stance+for+complex+argumentative+structures%2E+arXiv+preprint+arXiv%3A1906%2E11313%2E)
9. Emilio Ferrara. 2023. Should chatgpt be biased? challenges and risks of bias in large language models. arXiv preprint arXiv:2304.03738. [find](https://scholar.google.com/scholar?q=Emilio+Ferrara%2E+2023%2E+Should+chatgpt+be+biased%3F+challenges+and+risks+of+bias+in+large+language+models%2E+arXiv+preprint+arXiv%3A2304%2E03738%2E)
10. Iason Gabriel. 2020. Artificial intelligence, values, and alignment. Minds and machines, 30(3):411–437. [find](https://scholar.google.com/scholar?q=Iason+Gabriel%2E+2020%2E+Artificial+intelligence%2C+values%2C+and+alignment%2E+Minds+and+machines%2C+30%283%29%3A411%E2%80%93437%2E)
11. Christian Haerpfer, Ronald Inglehart, Alejandro Moreno, Christian Welzel, Kseniya Kizilova, Jaime Diez-Medrano, Marta Lagos, Pippa Norris, Eduard Ponarin, Bjorn Puranen, et al. 2022. World values survey: Round seven-country-pooled datafile version 5.0. Madrid, Spain & Vienna, Austria: JD Systems Ins… [find](https://scholar.google.com/scholar?q=Christian+Haerpfer%2C+Ronald+Inglehart%2C+Alejandro+Moreno%2C+Christian+Welzel%2C+Kseniya+Kizilova%2C+Jaime+Diez%2DMedrano%2C+Marta+Lagos%2C+Pippa+Norris%2C+Eduard+Ponarin%2C+Bjorn+Puranen%2C+et+al%2E+2022%2E+World+values+surv)
12. Ralf Herbrich, Tom Minka, and Thore Graepel. 2006. Trueskill™: a bayesian skill rating system. Advances in neural information processing systems, 19. [find](https://scholar.google.com/scholar?q=Ralf+Herbrich%2C+Tom+Minka%2C+and+Thore+Graepel%2E+2006%2E+Trueskill%E2%84%A2%3A+a+bayesian+skill+rating+system%2E+Advances+in+neural+information+processing+systems%2C+19%2E)
13. Liwei Jiang, Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, et al. 2021. Can machines learn morality? the delphi experiment. arXiv preprint arXiv:2110.07574. [find](https://scholar.google.com/scholar?q=Liwei+Jiang%2C+Jena+D+Hwang%2C+Chandra+Bhagavatula%2C+Ronan+Le+Bras%2C+Jenny+Liang%2C+Jesse+Dodge%2C+Keisuke+Sakaguchi%2C+Maxwell+Forbes%2C+Jon+Borchardt%2C+Saadia+Gabriel%2C+et+al%2E+2021%2E+Can+machines+learn+morality%3F+the)
14. Zhijing Jin, Sydney Levine, Fernando Gonzalez Adauto, Ojasv Kamal, Maarten Sap, Mrinmaya Sachan, Rada Mihalcea, Josh Tenenbaum, and Bernhard Schölkopf. 2022. When to make exceptions: Exploring language models as accounts of human moral judgment. Advances in neural information processing systems, 35:… [find](https://scholar.google.com/scholar?q=Zhijing+Jin%2C+Sydney+Levine%2C+Fernando+Gonzalez+Adauto%2C+Ojasv+Kamal%2C+Maarten+Sap%2C+Mrinmaya+Sachan%2C+Rada+Mihalcea%2C+Josh+Tenenbaum%2C+and+Bernhard+Sch%C3%B6lkopf%2E+2022%2E+When+to+make+exceptions%3A+Exploring+languag)
15. Grgur Kovač, Masataka Sawayama, Rémy Portelas, Cédric Colas, Peter Ford Dominey, and Pierre-Yves Oudeyer. 2023. Large language models as superpositions of cultural perspectives. arXiv preprint arXiv:2307.07870. — [arXiv:2307.07870](https://arxiv.org/abs/2307.07870) [find](https://scholar.google.com/scholar?q=Grgur+Kovac%CC%8C%2C+Masataka+Sawayama%2C+R%C3%A9my+Portelas%2C+C%C3%A9dric+Colas%2C+Peter+Ford+Dominey%2C+and+Pierre%2DYves+Oudeyer%2E+2023%2E+Large+language+models+as+superpositions+of+cultural+perspectives%2E+arXiv+preprint+arXiv%3A)
16. Lucio La Cava and Andrea Tagarelli. 2024. Open models, closed minds? on agents capabilities in mimicking human personalities through open large language models. arXiv preprint arXiv:2401.07115. — [arXiv:2401.07115](https://arxiv.org/abs/2401.07115) [find](https://scholar.google.com/scholar?q=Lucio+La+Cava+and+Andrea+Tagarelli%2E+2024%2E+Open+models%2C+closed+minds%3F+on+agents+capabilities+in+mimicking+human+personalities+through+open+large+language+models%2E+arXiv+preprint+arXiv%3A2401%2E07115%2E+%E2%80%94+%5BarX)
17. Thom Lake, Eunsol Choi, and Greg Durrett. 2024. From distributional to overton pluralism: Investigating large language model alignment. arXiv preprint arXiv:2406.17692. [find](https://scholar.google.com/scholar?q=Thom+Lake%2C+Eunsol+Choi%2C+and+Greg+Durrett%2E+2024%2E+From+distributional+to+overton+pluralism%3A+Investigating+large+language+model+alignment%2E+arXiv+preprint+arXiv%3A2406%2E17692%2E)
18. Yuan Li, Yue Huang, Hongyi Wang, Xiangliang Zhang, James Zou, and Lichao Sun. 2024. Quantifying ai psychology: A psychometrics benchmark for large language models. arXiv preprint arXiv:2406.17675. — [arXiv:2406.17675](https://arxiv.org/abs/2406.17675) [find](https://scholar.google.com/scholar?q=Yuan+Li%2C+Yue+Huang%2C+Hongyi+Wang%2C+Xiangliang+Zhang%2C+James+Zou%2C+and+Lichao+Sun%2E+2024%2E+Quantifying+ai+psychology%3A+A+psychometrics+benchmark+for+large+language+models%2E+arXiv+preprint+arXiv%3A2406%2E17675%2E+%E2%80%94+%5B)
19. Chenyang Lyu, Minghao Wu, and Alham Fikri Aji. 2024. Beyond probabilities: Unveiling the misalignment in evaluating large language models. arXiv preprint arXiv:2402.13887. [find](https://scholar.google.com/scholar?q=Chenyang+Lyu%2C+Minghao+Wu%2C+and+Alham+Fikri+Aji%2E+2024%2E+Beyond+probabilities%3A+Unveiling+the+misalignment+in+evaluating+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E13887%2E)
20. Justin K Miller and Wenjia Tang. 2025. Evaluating llm metrics through real-world capabilities. arXiv preprint arXiv:2505.08253. — [arXiv:2505.08253](https://arxiv.org/abs/2505.08253) [find](https://scholar.google.com/scholar?q=Justin+K+Miller+and+Wenjia+Tang%2E+2025%2E+Evaluating+llm+metrics+through+real%2Dworld+capabilities%2E+arXiv+preprint+arXiv%3A2505%2E08253%2E+%E2%80%94+%5BarXiv%3A2505%2E08253%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2505%2E08253%29)
21. Jared Moore, Tanvi Deshpande, and Diyi Yang. 2024. Are large language models consistent over value-laden questions? arXiv preprint arXiv:2407.02996. [find](https://scholar.google.com/scholar?q=Jared+Moore%2C+Tanvi+Deshpande%2C+and+Diyi+Yang%2E+2024%2E+Are+large+language+models+consistent+over+value%2Dladen+questions%3F+arXiv+preprint+arXiv%3A2407%2E02996%2E)
22. Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, et al. 2024. 2 olmo 2 furious. arXiv preprint arXiv:2501.00656. [find](https://scholar.google.com/scholar?q=Team+OLMo%2C+Pete+Walsh%2C+Luca+Soldaini%2C+Dirk+Groeneveld%2C+Kyle+Lo%2C+Shane+Arora%2C+Akshita+Bhagia%2C+Yuling+Gu%2C+Shengyi+Huang%2C+Matt+Jordan%2C+et+al%2E+2024%2E+2+olmo+2+furious%2E+arXiv+preprint+arXiv%3A2501%2E00656%2E)
23. Vishakh Padmakumar and He He. 2023. Does writing with language models reduce content diversity? arXiv preprint arXiv:2309.05196. [find](https://scholar.google.com/scholar?q=Vishakh+Padmakumar+and+He+He%2E+2023%2E+Does+writing+with+language+models+reduce+content+diversity%3F+arXiv+preprint+arXiv%3A2309%2E05196%2E)
24. R Plutchik. 1982. A psycho evolutionary theory of emotions. Social Science Information. [find](https://scholar.google.com/scholar?q=R+Plutchik%2E+1982%2E+A+psycho+evolutionary+theory+of+emotions%2E+Social+Science+Information%2E)
25. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Advances in Neural Information Processing Systems, 36.
26. Alexandre Rame, Guillaume Couairon, Corentin Dancette, Jean-Baptiste Gaya, Mustafa Shukor, Laure Soulier, and Matthieu Cord. 2023. Rewarded soups: towards pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards. Advances in Neural Information Processing Systems, 36:71095–7113… [find](https://scholar.google.com/scholar?q=Alexandre+Rame%2C+Guillaume+Couairon%2C+Corentin+Dancette%2C+Jean%2DBaptiste+Gaya%2C+Mustafa+Shukor%2C+Laure+Soulier%2C+and+Matthieu+Cord%2E+2023%2E+Rewarded+soups%3A+towards+pareto%2Doptimal+alignment+by+interpolating+wei)
27. Yuanyi Ren, Haoran Ye, Hanjun Fang, Xin Zhang, and Guojie Song. 2024. [Valuebench: Towards comprehensively evaluating value orientations and understanding of large language models](https://md.research.elamur.ai/free/en/papers/ren-2024-valuebench-comprehensively-evaluating-value-orientations/). arXiv preprint arXiv:2406.04214.
28. Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Rose Kirk, Hinrich Schütze, and Dirk Hovy. 2024. [Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2024-political-compass-or-spinning-arrow/). arXiv preprint arXiv:2402.16786.
29. Keita Saito, Akifumi Wachi, Koki Wataoka, and Youhei Akimoto. 2023. Verbosity bias in preference labeling by large language models. arXiv preprint arXiv:2310.10076. — [arXiv:2310.10076](https://arxiv.org/abs/2310.10076) [find](https://scholar.google.com/scholar?q=Keita+Saito%2C+Akifumi+Wachi%2C+Koki+Wataoka%2C+and+Youhei+Akimoto%2E+2023%2E+Verbosity+bias+in+preference+labeling+by+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E10076%2E+%E2%80%94+%5BarXiv%3A2310%2E10076%5D%28https%3A%2F%2Farxiv%2E)
30. Nikita Salkar, Thomas Trikalinos, Byron Wallace, and Ani Nenkova. 2022. Self-repetition in abstractive neural summarizers. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language… [find](https://scholar.google.com/scholar?q=Nikita+Salkar%2C+Thomas+Trikalinos%2C+Byron+Wallace%2C+and+Ani+Nenkova%2E+2022%2E+Self%2Drepetition+in+abstractive+neural+summarizers%2E+In+Proceedings+of+the+2nd+Conference+of+the+Asia%2DPacific+Chapter+of+the+Assoc)
31. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. [Whose opinions do language models reflect?](https://md.research.elamur.ai/free/en/papers/santurkar-2023-whose-opinions-language-models-reflect/) In International Conference on Machine Learning, pages 29971–30004. PMLR.
32. Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2024. Evaluating the moral beliefs encoded in llms. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Nino+Scherrer%2C+Claudia+Shi%2C+Amir+Feder%2C+and+David+Blei%2E+2024%2E+Evaluating+the+moral+beliefs+encoded+in+llms%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E)
33. Chantal Shaib, Joe Barrow, Jiuding Sun, Alexa F Siu, Byron C Wallace, and Ani Nenkova. 2024. Standardizing the measurement of text diversity: A tool and a comparative analysis of scores. arXiv preprint arXiv:2403.00553. — [arXiv:2403.00553](https://arxiv.org/abs/2403.00553) [find](https://scholar.google.com/scholar?q=Chantal+Shaib%2C+Joe+Barrow%2C+Jiuding+Sun%2C+Alexa+F+Siu%2C+Byron+C+Wallace%2C+and+Ani+Nenkova%2E+2024%2E+Standardizing+the+measurement+of+text+diversity%3A+A+tool+and+a+comparative+analysis+of+scores%2E+arXiv+preprin)
34. Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, and David Jurgens. 2023. You don’t need a personality test to know these models are unreliable: Assessing the reliability of large language models on psychometric instruments. arXiv preprint arXi… [find](https://scholar.google.com/scholar?q=Bangzhao+Shu%2C+Lechen+Zhang%2C+Minje+Choi%2C+Lavinia+Dunagan%2C+Lajanugen+Logeswaran%2C+Moontae+Lee%2C+Dallas+Card%2C+and+David+Jurgens%2E+2023%2E+You+don%E2%80%99t+need+a+personality+test+to+know+these+models+are+unreliable%3A)
35. Gabriel Simmons. 2022. Moral mimicry: Large language models produce moral rationalizations tailored to political identity. arXiv preprint arXiv:2209.12106. [find](https://scholar.google.com/scholar?q=Gabriel+Simmons%2E+2022%2E+Moral+mimicry%3A+Large+language+models+produce+moral+rationalizations+tailored+to+political+identity%2E+arXiv+preprint+arXiv%3A2209%2E12106%2E)
36. Xiaoyang Song, Akshat Gupta, Kiyan Mohebbizadeh, Shujie Hu, and Anant Singh. 2023. Have large language models developed a personality?: Applicability of self-assessment tests in measuring personality in llms. arXiv preprint arXiv:2305.14693. — [arXiv:2305.14693](https://arxiv.org/abs/2305.14693) [find](https://scholar.google.com/scholar?q=Xiaoyang+Song%2C+Akshat+Gupta%2C+Kiyan+Mohebbizadeh%2C+Shujie+Hu%2C+and+Anant+Singh%2E+2023%2E+Have+large+language+models+developed+a+personality%3F%3A+Applicability+of+self%2Dassessment+tests+in+measuring+personality+)
37. Charles Spearman. 1961. The proof and measurement of association between two things. [find](https://scholar.google.com/scholar?q=Charles+Spearman%2E+1961%2E+The+proof+and+measurement+of+association+between+two+things%2E)
38. Christian Stab and Iryna Gurevych. 2017. [Parsing argumentation structures in persuasive essays](https://md.research.elamur.ai/free/en/papers/stab-2017-parsing-argumentation-structures-persuasive-essays/). Computational Linguistics, 43(3):619–659.
39. James Alexander Kerr Thomson. 1956. The ethics of aristotle. Philosophy, 31(119). [find](https://scholar.google.com/scholar?q=James+Alexander+Kerr+Thomson%2E+1956%2E+The+ethics+of+aristotle%2E+Philosophy%2C+31%28119%29%2E)
40. Adithya V Ganesan, Yash Kumar Lal, August Nilsson, and H. Andrew Schwartz. 2023. Systematic evaluation of GPT-3 for zero-shot personality estimation. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 390–400, Toronto, Canada. A… [find](https://scholar.google.com/scholar?q=Adithya+V+Ganesan%2C+Yash+Kumar+Lal%2C+August+Nilsson%2C+and+H%2E+Andrew+Schwartz%2E+2023%2E+Systematic+evaluation+of+GPT%2D3+for+zero%2Dshot+personality+estimation%2E+In+Proceedings+of+the+13th+Workshop+on+Computation)
41. Haoxiang Wang, Yong Lin, Wei Xiong, Rui Yang, Shizhe Diao, Shuang Qiu, Han Zhao, and Tong Zhang. 2024. [Arithmetic control of llms for diverse user preferences: Directional preference alignment with multi-objective rewards](https://md.research.elamur.ai/free/en/papers/wang-2024-arithmetic-control-llms-diverse-user/). arXiv preprint arXiv:2402.18571.
42. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. [Ethical and social risks of harm from language models](https://md.research.elamur.ai/free/en/papers/weidinger-2021-ethical-social-risks-harm-language/). arXiv preprint arXiv:2112.04359.
43. Ruoxi Xu, Hongyu Lin, Xianpei Han, Jia Zheng, Weixiang Zhou, Le Sun, and Yingfei Sun. 2025. Large language models often say one thing and do another. arXiv preprint arXiv:2503.07003. — [arXiv:2503.07003](https://arxiv.org/abs/2503.07003) [find](https://scholar.google.com/scholar?q=Ruoxi+Xu%2C+Hongyu+Lin%2C+Xianpei+Han%2C+Jia+Zheng%2C+Weixiang+Zhou%2C+Le+Sun%2C+and+Yingfei+Sun%2E+2025%2E+Large+language+models+often+say+one+thing+and+do+another%2E+arXiv+preprint+arXiv%3A2503%2E07003%2E+%E2%80%94+%5BarXiv%3A2503%2E070)
44. Jing Yao, Xiaoyuan Yi, Shitong Duan, Jindong Wang, Yuzhuo Bai, Muhua Huang, Peng Zhang, Tun Lu, Zhicheng Dou, Maosong Sun, et al. 2025. Value compass leaderboard: A platform for fundamental and validated evaluation of llms values. arXiv preprint arXiv:2501.07071. — [arXiv:2501.07071](https://arxiv.org/abs/2501.07071) [find](https://scholar.google.com/scholar?q=Jing+Yao%2C+Xiaoyuan+Yi%2C+Shitong+Duan%2C+Jindong+Wang%2C+Yuzhuo+Bai%2C+Muhua+Huang%2C+Peng+Zhang%2C+Tun+Lu%2C+Zhicheng+Dou%2C+Maosong+Sun%2C+et+al%2E+2025%2E+Value+compass+leaderboard%3A+A+platform+for+fundamental+and+valida)
45. Jing Yao, Xiaoyuan Yi, Yifan Gong, Xiting Wang, and Xing Xie. 2024. Value FULCRA: Mapping large language models to the multidimensional spectrum of basic human value. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… [find](https://scholar.google.com/scholar?q=Jing+Yao%2C+Xiaoyuan+Yi%2C+Yifan+Gong%2C+Xiting+Wang%2C+and+Xing+Xie%2E+2024%2E+Value+FULCRA%3A+Mapping+large+language+models+to+the+multidimensional+spectrum+of+basic+human+value%2E+In+Proceedings+of+the+2024+Confer)
46. Haoran Ye, Yuhang Xie, Yuanyi Ren, Hanjun Fang, Xin Zhang, and Guojie Song. 2025. Measuring human and ai values based on generative psychometrics with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 26400–26408. [find](https://scholar.google.com/scholar?q=Haoran+Ye%2C+Yuhang+Xie%2C+Yuanyi+Ren%2C+Hanjun+Fang%2C+Xin+Zhang%2C+and+Guojie+Song%2E+2025%2E+Measuring+human+and+ai+values+based+on+generative+psychometrics+with+large+language+models%2E+In+Proceedings+of+the+AAAI)
47. Wentao Ye, Mingfeng Ou, Tianyi Li, Xuetao Ma, Yifan Yanggong, Sai Wu, Jie Fu, Gang Chen, Haobo Wang, Junbo Zhao, et al. 2023. Assessing hidden risks of llms: an empirical study on robustness, consistency, and credibility. arXiv preprint arXiv:2305.10235. — [arXiv:2305.10235](https://arxiv.org/abs/2305.10235) [find](https://scholar.google.com/scholar?q=Wentao+Ye%2C+Mingfeng+Ou%2C+Tianyi+Li%2C+Xuetao+Ma%2C+Yifan+Yanggong%2C+Sai+Wu%2C+Jie+Fu%2C+Gang+Chen%2C+Haobo+Wang%2C+Junbo+Zhao%2C+et+al%2E+2023%2E+Assessing+hidden+risks+of+llms%3A+an+empirical+study+on+robustness%2C+consiste)
48. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Varsha+Kishore%2C+Felix+Wu%2C+Kilian+Q+Weinberger%2C+and+Yoav+Artzi%2E+2019%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+arXiv+preprint+arXiv%3A1904%2E09675%2E)
49. Wenlong Zhao, Debanjan Mondal, Niket Tandon, Danica Dillion, Kurt Gray, and Yuling Gu. 2024. World-ValuesBench: A large-scale benchmark dataset for multi-cultural value awareness of language models. [find](https://scholar.google.com/scholar?q=Wenlong+Zhao%2C+Debanjan+Mondal%2C+Niket+Tandon%2C+Danica+Dillion%2C+Kurt+Gray%2C+and+Yuling+Gu%2E+2024%2E+World%2DValuesBench%3A+A+large%2Dscale+benchmark+dataset+for+multi%2Dcultural+value+awareness+of+language+models%2E)
50. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 17696–17706, Torino, Italia. ELRA and ICCL. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+2024+Joint+International+Conference+on+Computational+Linguistics%2C+Language+Resources+and+Evaluation+%28LREC%2DCOLING+2024%29%2C+pages+17696%E2%80%9317706%2C+Torino%2C+Italia%2E+ELRA+and+ICCL%2E)
51. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. [Texygen: A benchmarking platform for text generation models](https://md.research.elamur.ai/free/en/papers/zhu-2018-texygen-benchmarking-platform-text-generation/). In The 41st international ACM SIGIR conference on research & development in information retrieval, pages 1097–1100.
52. A preferences computed after applying the above procedure, refer to the Table 1. This table consists of two scenarios that are processed sequentially and the belief parameters associated with each value are shown after every processing. [find](https://scholar.google.com/scholar?q=A+preferences+computed+after+applying+the+above+procedure%2C+refer+to+the+Table+1%2E+This+table+consists+of+two+scenarios+that+are+processed+sequentially+and+the+belief+parameters+associated+with+each+val)

*Updated 1 November 2026*