---
title: "Too Open for Opinion? Embracing Open-Endedness in Large"
description: "The paper presents no experiments or benchmarks. 132 references. Among them: 0."
authors: ["Bolei Ma", "Yong Cao", "Indira Sen", "Anna-Carolina Haensch", "Frauke Kreuter", "Barbara Plank", "Daniel Hershcovich"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.89/"
doi: "10.18653/v1/2026.eacl-long.89"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ma-2026-too-open-opinion-embracing-open/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ma-2026-too-open-opinion-embracing-open/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/53/) / Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation

# Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation

**Authors:** [Bolei Ma](https://md.research.elamur.ai/free/en/authors/bolei-ma/), [Yong Cao](https://md.research.elamur.ai/free/en/authors/yong-cao/), [Indira Sen](https://md.research.elamur.ai/free/en/authors/indira-sen/), [Anna-Carolina Haensch](https://md.research.elamur.ai/free/en/authors/anna-carolina-haensch/), [Frauke Kreuter](https://md.research.elamur.ai/free/en/authors/frauke-kreuter/), [Barbara Plank](https://md.research.elamur.ai/free/en/authors/barbara-plank/) and [Daniel Hershcovich](https://md.research.elamur.ai/free/en/authors/daniel-hershcovich/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.89/) · [PDF](https://aclanthology.org/2026.eacl-long.89.pdf) · [DOI 10.18653/v1/2026.eacl-long.89](https://doi.org/10.18653/v1/2026.eacl-long.89) · [OpenAlex](https://openalex.org/W7140077113)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Research that uses LLMs to stand in for survey respondents or social agents usually restricts the models to multiple-choice or short-answer formats, because such outputs are easy to score and aggregate. This position paper by Ma et al. (EACL 2026) argues that this restriction wastes the generative nature of LLMs. It can flatten nuanced opinions, push answers into categories the researcher chose in advance, and hide how the model reasons. The authors draw on decades of survey methodology on open-ended questions and connect it to recent NLP work. They map the known advantages of free-text answers onto LLM-based simulation, including better measurement, discovery of unexpected and minority views, less directive bias, more individual expression, and support for pretesting. They also discuss methods for coding and analysing free-form outputs, along with the challenges these raise. The paper presents no experiments or benchmarks. Its contribution is a call for evaluation frameworks that use open-ended diversity instead of suppressing it, which would strengthen the link between NLP and the social sciences.

## Key points
- The authors re-examined the 53 simulation studies catalogued by Anthis et al. (2025). Only 11 (21%) contain any open-text element, and just 4 (8%) rely mainly on free-form outputs for evaluation.
- The paper defines open-endedness by how the task input is designed, not by how text is generated. Prompts that offer no fixed answer set count as open-ended, unlike outputs that are mapped back to labels through string matching or token probabilities.
- Benefits of open-ended questions are grouped into eight dimensions taken from survey research, such as measurement, design, unanticipated views, directive bias, expressiveness/individuality, pretesting, and methodological utility.
- The authors list several risks to address:
  - known LLM biases, such as left-leaning and WEIRD framings, found mostly in closed-ended setups;
  - LLM survey answers that are more positive than human ones;
  - fluent but shallow outputs being mistaken for authentic responses;
  - analytic flexibility that could lead to "silicon-only" findings that cannot be replicated.
- The paper is purely conceptual and relies on a non-exhaustive literature synthesis. It notes that "coding" in social science means annotating free responses into analytic categories, not programming.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Maryland, College Park](https://md.research.elamur.ai/free/en/universities/university-of-maryland-college-park/) — University, College Park, United States · 140 works · 29,011 citations
- [University of Copenhagen](https://md.research.elamur.ai/free/en/universities/university-of-copenhagen/) — University, Copenhagen, Denmark · 116 works · 15,292 citations
- [Ludwig-Maximilians-Universität München](https://md.research.elamur.ai/free/en/universities/ludwig-maximilians-universitat-munchen/) — University, Munich, Germany · 113 works · 56,817 citations
- [University of Tübingen](https://md.research.elamur.ai/free/en/universities/university-of-tubingen/) — University, Tübingen, Germany · 78 works · 41,280 citations
- [University of Mannheim](https://md.research.elamur.ai/free/en/universities/university-of-mannheim/) — University, Mannheim, Germany · 29 works · 8,347 citations
- [Munich Center for Machine Learning](https://md.research.elamur.ai/free/en/universities/munich-center-for-machine-learning/) — University, Munich, Germany · 62 works · 1,167 citations
- [Tübingen AI Center](https://md.research.elamur.ai/free/en/universities/tubingen-ai-center/) — University, Tübingen, Germany · 3 works · 30 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Bolei Ma, Yong Cao, Indira Sen, Anna-Carolina Haensch, Frauke Kreuter, Barbara Plank and Daniel Hershcovich. 2026. Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ma2026too,
  title = {{Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation}},
  author = {Bolei Ma, Yong Cao, Indira Sen, Anna-Carolina Haensch, Frauke Kreuter, Barbara Plank and Daniel Hershcovich},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.89},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.89}
}
```

- [← previous Too Nice to Tell the Truth: Quantifying Agreeableness-Driven Sycophancy in Role-Playing Language Models](https://md.research.elamur.ai/free/en/papers/shah-2026-too-nice-tell-truth-quantifying/)
- [next → Tool-Aware Planning for Contact-Center Analytics: Evaluating LLMs through Lineage-Guided Query Decomposition](https://md.research.elamur.ai/free/en/papers/nathan-2026-tool-aware-planning-contact-center/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/53/)

## References

132 references, 33 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Divya Mani Adhikari, Alexander Hartland, Ingmar Weber, and Vikram Kamath Cannanure. 2025. Exploring llms for automated generation and adaptation of questionnaires. In Proceedings of the 7th ACM Conference on Conversational User Interfaces, CUI ’25, New York, NY, USA. Association for Computing Machin… [find](https://scholar.google.com/scholar?q=Divya+Mani+Adhikari%2C+Alexander+Hartland%2C+Ingmar+Weber%2C+and+Vikram+Kamath+Cannanure%2E+2025%2E+Exploring+llms+for+automated+generation+and+adaptation+of+questionnaires%2E+In+Proceedings+of+the+7th+ACM+Confer)
2. Dalal Albudaiwi. 2017. Survey: Open-ended questions. In The SAGE Encyclopedia of Communication Research Methods, volume 4, pages 1716–1717. SAGE Publications, Inc, Thousand Oaks, California. [find](https://scholar.google.com/scholar?q=Dalal+Albudaiwi%2E+2017%2E+Survey%3A+Open%2Dended+questions%2E+In+The+SAGE+Encyclopedia+of+Communication+Research+Methods%2C+volume+4%2C+pages+1716%E2%80%931717%2E+SAGE+Publications%2C+Inc%2C+Thousand+Oaks%2C+California%2E)
3. Jacy Reese Anthis, Ryan Liu, Sean M Richardson, Austin C. Kozlowski, Bernard Koch, Erik Brynjolfsson, James Evans, and Michael S. Bernstein. 2025. Position: LLM social simulations are a promising research method. In Forty-second International Conference on Machine Learning Position Paper Track. [find](https://scholar.google.com/scholar?q=Jacy+Reese+Anthis%2C+Ryan+Liu%2C+Sean+M+Richardson%2C+Austin+C%2E+Kozlowski%2C+Bernard+Koch%2C+Erik+Brynjolfsson%2C+James+Evans%2C+and+Michael+S%2E+Bernstein%2E+2025%2E+Position%3A+LLM+social+simulations+are+a+promising+rese)
4. Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. 2023. [Out of one, many: Using language models to simulate human samples](https://md.research.elamur.ai/free/en/papers/argyle-2022-out-one-many-using-language/). Political Analysis, 31(3):337–351.
5. Joshua Ashkinaze, Emily Fry, Narendra Edara, Eric Gilbert, and Ceren Budak. 2025. Plurals: A system for guiding llms via simulated social ensembles. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI ’25, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Joshua+Ashkinaze%2C+Emily+Fry%2C+Narendra+Edara%2C+Eric+Gilbert%2C+and+Ceren+Budak%2E+2025%2E+Plurals%3A+A+system+for+guiding+llms+via+simulated+social+ensembles%2E+In+Proceedings+of+the+2025+CHI+Conference+on+Human+)
6. Nishant Balepur, Rachel Rudinger, and Jordan Lee Boyd-Graber. 2025. [Which of these best describes multiple choice evaluation with LLMs? a) forced B) flawed C) fixable D) all of the above](https://md.research.elamur.ai/free/en/papers/balepur-2025-which-these-best-describes-multiple/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper…
7. Franck Bardol. 2025. Chatgpt reads your tone and responds accordingly – until it does not – emotional framing induces bias in llm outputs. Preprint, arXiv:2507.21083. — [arXiv:2507.21083](https://arxiv.org/abs/2507.21083) [find](https://scholar.google.com/scholar?q=Franck+Bardol%2E+2025%2E+Chatgpt+reads+your+tone+and+responds+accordingly+%E2%80%93+until+it+does+not+%E2%80%93+emotional+framing+induces+bias+in+llm+outputs%2E+Preprint%2C+arXiv%3A2507%2E21083%2E+%E2%80%94+%5BarXiv%3A2507%2E21083%5D%28https%3A%2F%2Farxi)
8. Jacob Beck, Stephanie Eckman, Christoph Kern, and Frauke Kreuter. 2025. Bias in the loop: How humans evaluate ai-generated suggestions. Preprint, arXiv:2509.08514. — [arXiv:2509.08514](https://arxiv.org/abs/2509.08514) [find](https://scholar.google.com/scholar?q=Jacob+Beck%2C+Stephanie+Eckman%2C+Christoph+Kern%2C+and+Frauke+Kreuter%2E+2025%2E+Bias+in+the+loop%3A+How+humans+evaluate+ai%2Dgenerated+suggestions%2E+Preprint%2C+arXiv%3A2509%2E08514%2E+%E2%80%94+%5BarXiv%3A2509%2E08514%5D%28https%3A%2F%2Farxiv%2Eo)
9. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 610–623, New York, NY, USA. Associat… [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+2021%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+Conference)
10. Herbert Bless and Norbert Schwarz. 2010. Chapter 6 - mental construal and the emergence of assimilation and contrast effects: The inclusion/exclusion model. In Advances in Experimental Social Psychology, volume 42 of Advances in Experimental Social Psychology, pages 319–373. Academic Press. [find](https://scholar.google.com/scholar?q=Herbert+Bless+and+Norbert+Schwarz%2E+2010%2E+Chapter+6+%2D+mental+construal+and+the+emergence+of+assimilation+and+contrast+effects%3A+The+inclusion%2Fexclusion+model%2E+In+Advances+in+Experimental+Social+Psycholo)
11. Norman M. Bradburn, Seymour Sudman, and Brian Wansink. 2004. Asking Questions: The Definitive Guide to Questionnaire Design – For Market Research, Political Polls, and Social and Health Questionnaires, 2nd, revised edition. Jossey-Bass, San Francisco, CA. [find](https://scholar.google.com/scholar?q=Norman+M%2E+Bradburn%2C+Seymour+Sudman%2C+and+Brian+Wansink%2E+2004%2E+Asking+Questions%3A+The+Definitive+Guide+to+Questionnaire+Design+%E2%80%93+For+Market+Research%2C+Political+Polls%2C+and+Social+and+Health+Questionnaires)
12. James Brand, Ayelet Israeli, and Donald Ngwe. 2023. Using llms for market research. Working Paper 23-062, Harvard Business School Marketing Unit. Last revised: July 8, 2023. [find](https://scholar.google.com/scholar?q=James+Brand%2C+Ayelet+Israeli%2C+and+Donald+Ngwe%2E+2023%2E+Using+llms+for+market+research%2E+Working+Paper+23%2D062%2C+Harvard+Business+School+Marketing+Unit%2E+Last+revised%3A+July+8%2C+2023%2E)
13. Yong Cao, Haijiang Liu, Arnav Arora, Isabelle Augenstein, Paul Röttger, and Daniel Hershcovich. 2025. [Specializing large language models to simulate survey response distributions for global populations](https://md.research.elamur.ai/free/en/papers/cao-2025-specializing-large-language-models-simulate/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association f…
14. Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. [Assessing cross-cultural alignment between ChatGPT and human societies: An empirical study](https://md.research.elamur.ai/free/en/papers/cao-2023-assessing-cross-cultural-alignment-between/). In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 53–67, Dubrovnik, Croatia. As…
15. Anthony Chemero. 2023. Llms differ from human cognition because they are not embodied. Nature Human Behaviour, 7(11):1828–1829. [find](https://scholar.google.com/scholar?q=Anthony+Chemero%2E+2023%2E+Llms+differ+from+human+cognition+because+they+are+not+embodied%2E+Nature+Human+Behaviour%2C+7%2811%29%3A1828%E2%80%931829%2E)
16. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Paul+F+Christiano%2C+Jan+Leike%2C+Tom+Brown%2C+Miljan+Martic%2C+Shane+Legg%2C+and+Dario+Amodei%2E+2017%2E+Deep+reinforcement+learning+from+human+preferences%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+vo)
17. Saoirse Connor Desai and Stian Reimers. 2019. Comparing the use of open and closed questions for web-based measures of the continued-influence effect. Behavior Research Methods, 51(3):1426–1440. [find](https://scholar.google.com/scholar?q=Saoirse+Connor+Desai+and+Stian+Reimers%2E+2019%2E+Comparing+the+use+of+open+and+closed+questions+for+web%2Dbased+measures+of+the+continued%2Dinfluence+effect%2E+Behavior+Research+Methods%2C+51%283%29%3A1426%E2%80%931440%2E)
18. Don A. Dillman, Jolene D. Smyth, and Leah Melani Christian. 2014. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method, 4th edition. Wiley, Hoboken, NJ. [find](https://scholar.google.com/scholar?q=Don+A%2E+Dillman%2C+Jolene+D%2E+Smyth%2C+and+Leah+Melani+Christian%2E+2014%2E+Internet%2C+Phone%2C+Mail%2C+and+Mixed%2DMode+Surveys%3A+The+Tailored+Design+Method%2C+4th+edition%2E+Wiley%2C+Hoboken%2C+NJ%2E)
19. Zackary Okun Dunivin. 2025. Scaling hermeneutics: a guide to qualitative coding with llms for reflexive content analysis. EPJ Data Science, 14(1):28. [find](https://scholar.google.com/scholar?q=Zackary+Okun+Dunivin%2E+2025%2E+Scaling+hermeneutics%3A+a+guide+to+qualitative+coding+with+llms+for+reflexive+content+analysis%2E+EPJ+Data+Science%2C+14%281%29%3A28%2E)
20. Esin Durmus, Karina Nguyen, Thomas Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2024. [Towards measuring the…](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/)
21. Stephanie Eckman, Barbara Plank, and Frauke Kreuter. 2024. Position: Insights from survey methodology can improve training data. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 12268–12283. PMLR. [find](https://scholar.google.com/scholar?q=Stephanie+Eckman%2C+Barbara+Plank%2C+and+Frauke+Kreuter%2E+2024%2E+Position%3A+Insights+from+survey+methodology+can+improve+training+data%2E+In+Proceedings+of+the+41st+International+Conference+on+Machine+Learning)
22. Aparna Elangovan, Ling Liu, Lei Xu, Sravan Babu Bodapati, and Dan Roth. 2024. ConSiDERS-the-human evaluation framework: Rethinking human evaluation for generative large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)… [find](https://scholar.google.com/scholar?q=Aparna+Elangovan%2C+Ling+Liu%2C+Lei+Xu%2C+Sravan+Babu+Bodapati%2C+and+Dan+Roth%2E+2024%2E+ConSiDERS%2Dthe%2Dhuman+evaluation+framework%3A+Rethinking+human+evaluation+for+generative+large+language+models%2E+In+Proceedings)
23. Andrew Estornell and Yang Liu. 2024. Multi-llm debate: Framework, principals, and interventions. In Advances in Neural Information Processing Systems, volume 37, pages 28938–28964. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Andrew+Estornell+and+Yang+Liu%2E+2024%2E+Multi%2Dllm+debate%3A+Framework%2C+principals%2C+and+interventions%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+37%2C+pages+28938%E2%80%9328964%2E+Curran+Associates%2C+)
24. Frederick G. Conrad, Mick P. Couper, and Joseph W. Sakshaug. 2016. Classifying open-ended reports: Factors affecting the reliability of occupation codes. Journal of Official Statistics, 32(1):75–92. [find](https://scholar.google.com/scholar?q=Frederick+G%2E+Conrad%2C+Mick+P%2E+Couper%2C+and+Joseph+W%2E+Sakshaug%2E+2016%2E+Classifying+open%2Dended+reports%3A+Factors+affecting+the+reliability+of+occupation+codes%2E+Journal+of+Official+Statistics%2C+32%281%29%3A75%E2%80%9392%2E)
25. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. [Hierarchical neural story generation](https://md.research.elamur.ai/free/en/papers/fan-2018-hierarchical-neural-story-generation/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898, Melbourne, Australia. Association for Computational Linguistics.
26. Ziyan Cui, Ning Li, and Huaikang Zhou. 2025. A large-scale replication of scenario-based experiments in psychology and management using large language models. Nature Computational Science, 5(8):627– 634. [find](https://scholar.google.com/scholar?q=Ziyan+Cui%2C+Ning+Li%2C+and+Huaikang+Zhou%2E+2025%2E+A+large%2Dscale+replication+of+scenario%2Dbased+experiments+in+psychology+and+management+using+large+language+models%2E+Nature+Computational+Science%2C+5%288%29%3A627%E2%80%93+6)
27. Beatrice Ferrario and Stefanie Stantcheva. 2022. Eliciting people’s first-order concerns: Text analysis of open-ended survey questions. AEA Papers and Proceedings, 112:163–169. [find](https://scholar.google.com/scholar?q=Beatrice+Ferrario+and+Stefanie+Stantcheva%2E+2022%2E+Eliciting+people%E2%80%99s+first%2Dorder+concerns%3A+Text+analysis+of+open%2Dended+survey+questions%2E+AEA+Papers+and+Proceedings%2C+112%3A163%E2%80%93169%2E)
28. Jamie Cummins. 2025. The threat of analytic flexibility in using large language models to simulate human data: A call to attention. Preprint, arXiv:2509.13397. — [arXiv:2509.13397](https://arxiv.org/abs/2509.13397) [find](https://scholar.google.com/scholar?q=Jamie+Cummins%2E+2025%2E+The+threat+of+analytic+flexibility+in+using+large+language+models+to+simulate+human+data%3A+A+call+to+attention%2E+Preprint%2C+arXiv%3A2509%2E13397%2E+%E2%80%94+%5BarXiv%3A2509%2E13397%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
29. Angus Deaton and Arthur A. Stone. 2016. Understanding context effects for a measure of life evaluation: how responses matter. Oxford Economic Papers, 68(4):861–870. [find](https://scholar.google.com/scholar?q=Angus+Deaton+and+Arthur+A%2E+Stone%2E+2016%2E+Understanding+context+effects+for+a+measure+of+life+evaluation%3A+how+responses+matter%2E+Oxford+Economic+Papers%2C+68%284%29%3A861%E2%80%93870%2E)
30. Stefan Feuerriegel, Abdurahman Maarouf, Dominik Bär, Dominique Geissler, Jonas Schweisthal, Nicolas Pröllochs, Claire E. Robertson, Steve Rathje, Jochen Hartmann, Saif M. Mohammad, Oded Netzer, Alexandra A. Siegel, Barbara Plank, and Jay J. Van Bavel. 2025. Using natural language processing to analy… [find](https://scholar.google.com/scholar?q=Stefan+Feuerriegel%2C+Abdurahman+Maarouf%2C+Dominik+B%C3%A4r%2C+Dominique+Geissler%2C+Jonas+Schweisthal%2C+Nicolas+Pr%C3%B6llochs%2C+Claire+E%2E+Robertson%2C+Steve+Rathje%2C+Jochen+Hartmann%2C+Saif+M%2E+Mohammad%2C+Oded+Netzer%2C+Alexan)
31. James Flamino, Mohammed Shahid Modi, Boleslaw K. Szymanski, Brendan Cross, and Colton Mikolajczyk. 2025. Testing the limits of large language models in debating humans. Scientific Reports, 15(1):13852. [find](https://scholar.google.com/scholar?q=James+Flamino%2C+Mohammed+Shahid+Modi%2C+Boleslaw+K%2E+Szymanski%2C+Brendan+Cross%2C+and+Colton+Mikolajczyk%2E+2025%2E+Testing+the+limits+of+large+language+models+in+debating+humans%2E+Scientific+Reports%2C+15%281%29%3A13852)
32. Zhoushanyue He and Matthias Schonlau. 2021a. Coding text answers to open-ended questions: Human coders and statistical learning algorithms make similar mistakes. methods, data, analyses, 15(1):17. [find](https://scholar.google.com/scholar?q=Zhoushanyue+He+and+Matthias+Schonlau%2E+2021a%2E+Coding+text+answers+to+open%2Dended+questions%3A+Human+coders+and+statistical+learning+algorithms+make+similar+mistakes%2E+methods%2C+data%2C+analyses%2C+15%281%29%3A17%2E)
33. Mirta Galesic and Michael Bosnjak. 2009. Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73(2):349–360. [find](https://scholar.google.com/scholar?q=Mirta+Galesic+and+Michael+Bosnjak%2E+2009%2E+Effects+of+questionnaire+length+on+participation+and+indicators+of+response+quality+in+a+web+survey%2E+Public+Opinion+Quarterly%2C+73%282%29%3A349%E2%80%93360%2E)
34. Zhoushanyue He and Matthias Schonlau. 2021b. A model-assisted approach for finding coding errors in manual coding of open-ended questions. Journal of Survey Statistics and Methodology, 10(2):365–376. [find](https://scholar.google.com/scholar?q=Zhoushanyue+He+and+Matthias+Schonlau%2E+2021b%2E+A+model%2Dassisted+approach+for+finding+coding+errors+in+manual+coding+of+open%2Dended+questions%2E+Journal+of+Survey+Statistics+and+Methodology%2C+10%282%29%3A365%E2%80%93376%2E)
35. John G. Geer. 1988. What do open-ended questions measure? Public Opinion Quarterly, 52(3):365–367. [find](https://scholar.google.com/scholar?q=John+G%2E+Geer%2E+1988%2E+What+do+open%2Dended+questions+measure%3F+Public+Opinion+Quarterly%2C+52%283%29%3A365%E2%80%93367%2E)
36. Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A. Lemley, and Percy Liang. 2023. Foundation models and fair use. Journal of Machine Learning Research, 24(400):1–79. [find](https://scholar.google.com/scholar?q=Peter+Henderson%2C+Xuechen+Li%2C+Dan+Jurafsky%2C+Tatsunori+Hashimoto%2C+Mark+A%2E+Lemley%2C+and+Percy+Liang%2E+2023%2E+Foundation+models+and+fair+use%2E+Journal+of+Machine+Learning+Research%2C+24%28400%29%3A1%E2%80%9379%2E)
37. Julian Iñaki Goñi. 2025. Citizen participation and technology: lessons from the fields of deliberative democracy and science and technology studies. Humanities and Social Sciences Communications, 12(1):287. [find](https://scholar.google.com/scholar?q=Julian+I%C3%B1aki+Go%C3%B1i%2E+2025%2E+Citizen+participation+and+technology%3A+lessons+from+the+fields+of+deliberative+democracy+and+science+and+technology+studies%2E+Humanities+and+Social+Sciences+Communications%2C+12%281)
38. Josh A. Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, and Katerina Sedova. 2023. Generative language models and automated influence operations: Emerging threats and potential mitigations. Preprint, arXiv:2301.04246. — [arXiv:2301.04246](https://arxiv.org/abs/2301.04246) [find](https://scholar.google.com/scholar?q=Josh+A%2E+Goldstein%2C+Girish+Sastry%2C+Micah+Musser%2C+Renee+DiResta%2C+Matthew+Gentzel%2C+and+Katerina+Sedova%2E+2023%2E+Generative+language+models+and+automated+influence+operations%3A+Emerging+threats+and+potential)
39. Robert M. Groves, Floyd J. Fowler, Jr., Mick P. Couper, James M. Lepkowski, Eleanor Singer, and Roger Tourangeau. 2009. Survey methodology, 2nd edition. John Wiley & Sons, Hoboken, NJ. [find](https://scholar.google.com/scholar?q=Robert+M%2E+Groves%2C+Floyd+J%2E+Fowler%2C+Jr%2E%2C+Mick+P%2E+Couper%2C+James+M%2E+Lepkowski%2C+Eleanor+Singer%2C+and+Roger+Tourangeau%2E+2009%2E+Survey+methodology%2C+2nd+edition%2E+John+Wiley+%26+Sons%2C+Hoboken%2C+NJ%2E)
40. Jairo F Gudiño, Umberto Grandi, and César Hidalgo. 2024. Large language models (llms) as agents for augmented democracy. Philosophical Transactions A, 382(2285):20240100. [find](https://scholar.google.com/scholar?q=Jairo+F+Gudi%C3%B1o%2C+Umberto+Grandi%2C+and+C%C3%A9sar+Hidalgo%2E+2024%2E+Large+language+models+%28llms%29+as+agents+for+augmented+democracy%2E+Philosophical+Transactions+A%2C+382%282285%29%3A20240100%2E)
41. Hyukjun Gweon and Matthias Schonlau. 2023. Automated classification for open-ended questions with bert. Journal of Survey Statistics and Methodology, 12(2):493–504. [find](https://scholar.google.com/scholar?q=Hyukjun+Gweon+and+Matthias+Schonlau%2E+2023%2E+Automated+classification+for+open%2Dended+questions+with+bert%2E+Journal+of+Survey+Statistics+and+Methodology%2C+12%282%29%3A493%E2%80%93504%2E)
42. Anna-Carolina Haensch, Bernd Weiß, Patricia Steins, Priscilla Chyrva, and Katja Bitz. 2022. The semi-automatic classification of an open-ended question on panel survey motivation and its application in attrition analysis. Frontiers in Big Data, 5:880554. [find](https://scholar.google.com/scholar?q=Anna%2DCarolina+Haensch%2C+Bernd+Wei%C3%9F%2C+Patricia+Steins%2C+Priscilla+Chyrva%2C+and+Katja+Bitz%2E+2022%2E+The+semi%2Dautomatic+classification+of+an+open%2Dended+question+on+panel+survey+motivation+and+its+application+i)
43. C. Haerpfer, R. Inglehart, A. Moreno, C. Welzel, K. Kizilova, J. Diez-Medrano, M. Lagos, P. Norris, E. Ponarin, and B. Puranen, editors. 2022. World Values Survey: Round Seven - Country-Pooled Datafile Version 5.0. JD Systems Institute & WVSA Secretariat, Madrid, Spain & Vienna, Austria. [find](https://scholar.google.com/scholar?q=C%2E+Haerpfer%2C+R%2E+Inglehart%2C+A%2E+Moreno%2C+C%2E+Welzel%2C+K%2E+Kizilova%2C+J%2E+Diez%2DMedrano%2C+M%2E+Lagos%2C+P%2E+Norris%2C+E%2E+Ponarin%2C+and+B%2E+Puranen%2C+editors%2E+2022%2E+World+Values+Survey%3A+Round+Seven+%2D+Country%2DPooled+Datafil)
44. Tyler Hamby and Wyn Taylor. 2016. Survey satisficing inflates reliability and validity measures: An experimental comparison of college and amazon mechanical turk samples. Educational and Psychological Measurement, 76(6):912–932. [find](https://scholar.google.com/scholar?q=Tyler+Hamby+and+Wyn+Taylor%2E+2016%2E+Survey+satisficing+inflates+reliability+and+validity+measures%3A+An+experimental+comparison+of+college+and+amazon+mechanical+turk+samples%2E+Educational+and+Psychological)
45. Zhoushanyue He and Matthias Schonlau. 2020. Automatic coding of text answers to open-ended questions: Should you double code the training data? Social Science Computer Review, 38(6):754–765. [find](https://scholar.google.com/scholar?q=Zhoushanyue+He+and+Matthias+Schonlau%2E+2020%2E+Automatic+coding+of+text+answers+to+open%2Dended+questions%3A+Should+you+double+code+the+training+data%3F+Social+Science+Computer+Review%2C+38%286%29%3A754%E2%80%93765%2E)
46. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In International Conference on Learning Representations.
47. John J. Horton. 2023. Large language models as simulated economic agents: What can we learn from homo silicus? Preprint, arXiv:2301.07543. — [arXiv:2301.07543](https://arxiv.org/abs/2301.07543) [find](https://scholar.google.com/scholar?q=John+J%2E+Horton%2E+2023%2E+Large+language+models+as+simulated+economic+agents%3A+What+can+we+learn+from+homo+silicus%3F+Preprint%2C+arXiv%3A2301%2E07543%2E+%E2%80%94+%5BarXiv%3A2301%2E07543%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2301%2E07543%29)
48. Dirk Hovy and Diyi Yang. 2021. [The importance of modeling social factors of language: Theory and practice](https://md.research.elamur.ai/free/en/papers/hovy-2021-importance-modeling-social-factors-language/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 588–602, Online. Association for Computa…
49. Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru, Arman Cohan, and Bhuwan Dhingra. 2024. Calibrating long-form generations from large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13441–13460, Miami, Florida, USA. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Yukun+Huang%2C+Yixin+Liu%2C+Raghuveer+Thirukovalluru%2C+Arman+Cohan%2C+and+Bhuwan+Dhingra%2E+2024%2E+Calibrating+long%2Dform+generations+from+large+language+models%2E+In+Findings+of+the+Association+for+Computational+)
50. Shivani Kapania, William Agnew, Motahhare Eslami, Hoda Heidari, and Sarah E Fox. 2025. Simulacrum of stories: Examining large language models as qualitative research participants. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI ’25, New York, NY, USA. Association… [find](https://scholar.google.com/scholar?q=Shivani+Kapania%2C+William+Agnew%2C+Motahhare+Eslami%2C+Hoda+Heidari%2C+and+Sarah+E+Fox%2E+2025%2E+Simulacrum+of+stories%3A+Examining+large+language+models+as+qualitative+research+participants%2E+In+Proceedings+of+th)
51. Antonia Karamolegkou, Angana Borah, Eunjung Cho, Sagnik Ray Choudhury, Martina Galletti, Rajarshi Ghosh, Pranav Gupta, Oana Ignat, Priyanka Kargupta, Neema Kotonya, Hemank Lamba, Sun-Joo Lee, Arushi Mangla, Ishani Mondal, Deniz Nazarova, Poli Nemkova, Dina Pisarevskaya, Naquee Rizwan, Nazanin Sabri,… — [arXiv:2505.22327](https://arxiv.org/abs/2505.22327) [find](https://scholar.google.com/scholar?q=Antonia+Karamolegkou%2C+Angana+Borah%2C+Eunjung+Cho%2C+Sagnik+Ray+Choudhury%2C+Martina+Galletti%2C+Rajarshi+Ghosh%2C+Pranav+Gupta%2C+Oana+Ignat%2C+Priyanka+Kargupta%2C+Neema+Kotonya%2C+Hemank+Lamba%2C+Sun%2DJoo+Lee%2C+Arushi+M)
52. Rabimba Karanjai, Boris Shor, Amanda Austin, Ryan Kennedy, Yang Lu, Lei Xu, and Weidong Shi. 2025. Synthesizing public opinions with llms: Role creation, impacts, and the future to edemorcacy. Preprint, arXiv:2504.00241. — [arXiv:2504.00241](https://arxiv.org/abs/2504.00241) [find](https://scholar.google.com/scholar?q=Rabimba+Karanjai%2C+Boris+Shor%2C+Amanda+Austin%2C+Ryan+Kennedy%2C+Yang+Lu%2C+Lei+Xu%2C+and+Weidong+Shi%2E+2025%2E+Synthesizing+public+opinions+with+llms%3A+Role+creation%2C+impacts%2C+and+the+future+to+edemorcacy%2E+Preprin)
53. Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke Hüllermeier. 2024. [A survey of reinforcement learning from human feedback](https://md.research.elamur.ai/free/en/papers/kaufmann-2023-survey-reinforcement-learning-human-feedback/). Preprint, arXiv:2312.14925.
54. Christoph Kern, Stephanie Eckman, Jacob Beck, Rob Chew, Bolei Ma, and Frauke Kreuter. 2023. Annotation sensitivity: Training data collection methods affect model performance. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14874–14886, Singapore. Association for Compu… [find](https://scholar.google.com/scholar?q=Christoph+Kern%2C+Stephanie+Eckman%2C+Jacob+Beck%2C+Rob+Chew%2C+Bolei+Ma%2C+and+Frauke+Kreuter%2E+2023%2E+Annotation+sensitivity%3A+Training+data+collection+methods+affect+model+performance%2E+In+Findings+of+the+Associ)
55. Eunsu Kim, Juyoung Suk, Seungone Kim, Niklas Muennighoff, Dongkwan Kim, and Alice Oh. 2025. LLM-as-an-interviewer: Beyond static testing through dynamic LLM evaluation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 26456–26493, Vienna, Austria. Association for Computa… [find](https://scholar.google.com/scholar?q=Eunsu+Kim%2C+Juyoung+Suk%2C+Seungone+Kim%2C+Niklas+Muennighoff%2C+Dongkwan+Kim%2C+and+Alice+Oh%2E+2025%2E+LLM%2Das%2Dan%2Dinterviewer%3A+Beyond+static+testing+through+dynamic+LLM+evaluation%2E+In+Findings+of+the+Association+)
56. Sunwoong Kim, Jongho Jeong, Jin Soo Han, and Donghyuk Shin. 2024. Llm-mirror: A generated-persona approach for survey pre-testing. Preprint, arXiv:2412.03162. — [arXiv:2412.03162](https://arxiv.org/abs/2412.03162) [find](https://scholar.google.com/scholar?q=Sunwoong+Kim%2C+Jongho+Jeong%2C+Jin+Soo+Han%2C+and+Donghyuk+Shin%2E+2024%2E+Llm%2Dmirror%3A+A+generated%2Dpersona+approach+for+survey+pre%2Dtesting%2E+Preprint%2C+arXiv%3A2412%2E03162%2E+%E2%80%94+%5BarXiv%3A2412%2E03162%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
57. Austin C. Kozlowski and James Evans. 2025. Simulating subjects: The promise and peril of artificial intelligence stand-ins for social agents and interactions. Sociological Methods & Research, 54(3):1017–1073. [find](https://scholar.google.com/scholar?q=Austin+C%2E+Kozlowski+and+James+Evans%2E+2025%2E+Simulating+subjects%3A+The+promise+and+peril+of+artificial+intelligence+stand%2Dins+for+social+agents+and+interactions%2E+Sociological+Methods+%26+Research%2C+54%283%29%3A10)
58. Fabienne Kraemer, Henning Silber, Bella Struminskaya, Matthias Sand, Michael Bosnjak, Joanna Koßmann, and Bernd Weiß. 2023. Satisficing response behavior across time: Assessing negative panel conditioning using an experimental design with six repetitions. Survey Research Methods, 17(3). [find](https://scholar.google.com/scholar?q=Fabienne+Kraemer%2C+Henning+Silber%2C+Bella+Struminskaya%2C+Matthias+Sand%2C+Michael+Bosnjak%2C+Joanna+Ko%C3%9Fmann%2C+and+Bernd+Wei%C3%9F%2E+2023%2E+Satisficing+response+behavior+across+time%3A+Assessing+negative+panel+conditio)
59. Kalpesh Krishna, Aurko Roy, and Mohit Iyyer. 2021. [Hurdles to progress in long-form question answering](https://md.research.elamur.ai/free/en/papers/krishna-2021-hurdles-progress-long-form-question/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4940–4957, Online. Association for Computat…
60. Jon A. Krosnick. 1999. Survey research. Annual Review of Psychology, 50:537–567. [find](https://scholar.google.com/scholar?q=Jon+A%2E+Krosnick%2E+1999%2E+Survey+research%2E+Annual+Review+of+Psychology%2C+50%3A537%E2%80%93567%2E)
61. Jon A. Krosnick and Stanley Presser. 2010. Question and questionnaire design. In Peter V. Marsden and James D. Wright, editors, Handbook of Survey Research, 2 edition, pages 264–313. Emerald, Bingley, UK. [find](https://scholar.google.com/scholar?q=Jon+A%2E+Krosnick+and+Stanley+Presser%2E+2010%2E+Question+and+questionnaire+design%2E+In+Peter+V%2E+Marsden+and+James+D%2E+Wright%2C+editors%2C+Handbook+of+Survey+Research%2C+2+edition%2C+pages+264%E2%80%93313%2E+Emerald%2C+Bingley%2C)
62. Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. 2018. [Scalable agent alignment via reward modeling: a research direction](https://md.research.elamur.ai/free/en/papers/leike-2018-scalable-agent-alignment-reward-modeling/). Preprint, arXiv:1811.07871.
63. Timo Lenzner, Cornelia Neuert, and Wanda Otto. 2015. Kognitives Pretesting. GESIS – Leibniz-Institut für Sozialwissenschaften, Mannheim. [find](https://scholar.google.com/scholar?q=Timo+Lenzner%2C+Cornelia+Neuert%2C+and+Wanda+Otto%2E+2015%2E+Kognitives+Pretesting%2E+GESIS+%E2%80%93+Leibniz%2DInstitut+f%C3%BCr+Sozialwissenschaften%2C+Mannheim%2E)
64. Chance Jiajie Li, Jiayi Wu, Zhenze Mo, Ao Qu, Yuhan Tang, Kaiya Ivy Zhao, Yulu Gan, Jie Fan, Jiangbo Yu, Jinhua Zhao, Paul Liang, Luis Alonso, and Kent Larson. 2025. Position: Simulating society requires simulating thought. Preprint, arXiv:2506.06958. — [arXiv:2506.06958](https://arxiv.org/abs/2506.06958) [find](https://scholar.google.com/scholar?q=Chance+Jiajie+Li%2C+Jiayi+Wu%2C+Zhenze+Mo%2C+Ao+Qu%2C+Yuhan+Tang%2C+Kaiya+Ivy+Zhao%2C+Yulu+Gan%2C+Jie+Fan%2C+Jiangbo+Yu%2C+Jinhua+Zhao%2C+Paul+Liang%2C+Luis+Alonso%2C+and+Kent+Larson%2E+2025%2E+Position%3A+Simulating+society+requi)
65. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. [Encouraging divergent thinking in large language models through multi-agent debate](https://md.research.elamur.ai/free/en/papers/liang-2023-encouraging-divergent-thinking-large-language/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17…
66. Chin-Yew Lin. 2004. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
67. Haijiang Liu, Qiyuan Li, Chao Gao, Yong Cao, Xiangyu Xu, Xun Wu, Daniel Hershcovich, and Jinguang Gu. 2025. Beyond demographics: Enhancing cultural value survey simulation with multi-stage personality-driven cognitive reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural L… [find](https://scholar.google.com/scholar?q=Haijiang+Liu%2C+Qiyuan+Li%2C+Chao+Gao%2C+Yong+Cao%2C+Xiangyu+Xu%2C+Xun+Wu%2C+Daniel+Hershcovich%2C+and+Jinguang+Gu%2E+2025%2E+Beyond+demographics%3A+Enhancing+cultural+value+survey+simulation+with+multi%2Dstage+personality)
68. Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, and Andrew M. Dai. 2024. [Best practices and lessons learned on synthetic data](https://md.research.elamur.ai/free/en/papers/liu-2024-best-practices-lessons-learned-synthetic/). In First Conference on Language Modeling.
69. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. [G-eval: NLG evaluation using gpt-4 with better human alignment](https://md.research.elamur.ai/free/en/papers/liu-2023-g-eval-nlg-evaluation-gpt/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore. Association for Computational Lin…
70. Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. [On LLMs-driven synthetic data generation, curation, and evaluation: A survey](https://md.research.elamur.ai/free/en/papers/long-2024-llms-driven-synthetic-data-generation/). In Findings of the Association for Computational Linguistics ACL 2024, pages 11065–11082, Bangkok, Thailand and virtual meeting. Ass…
71. Chenyang Lyu, Minghao Wu, and Alham Aji. 2024. Beyond probabilities: Unveiling the misalignment in evaluating large language models. In Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024), pages 109–131, Bangkok, Thailand. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Chenyang+Lyu%2C+Minghao+Wu%2C+and+Alham+Aji%2E+2024%2E+Beyond+probabilities%3A+Unveiling+the+misalignment+in+evaluating+large+language+models%2E+In+Proceedings+of+the+1st+Workshop+on+Towards+Knowledgeable+Languag)
72. Bolei Ma, Xinpeng Wang, Tiancheng Hu, Anna-Carolina Haensch, Michael A. Hedderich, Barbara Plank, and Frauke Kreuter. 2024. The potential and challenges of evaluating attitudes, opinions, and values in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, p… [find](https://scholar.google.com/scholar?q=Bolei+Ma%2C+Xinpeng+Wang%2C+Tiancheng+Hu%2C+Anna%2DCarolina+Haensch%2C+Michael+A%2E+Hedderich%2C+Barbara+Plank%2C+and+Frauke+Kreuter%2E+2024%2E+The+potential+and+challenges+of+evaluating+attitudes%2C+opinions%2C+and+values+i)
73. Bolei Ma, Berk Yoztyurk, Anna-Carolina Haensch, Xinpeng Wang, Markus Herklotz, Frauke Kreuter, Barbara Plank, and Matthias Aßenmacher. 2025. [Algorithmic fidelity of large language models in generating synthetic German public opinions: A case study](https://md.research.elamur.ai/free/en/papers/ma-2025-algorithmic-fidelity-large-language-models/). In Proceedings of the 63rd Annual Meeting of the As…
74. Nicole Meister, Carlos Guestrin, and Tatsunori Hashimoto. 2025. [Benchmarking distributional alignment of large language models](https://md.research.elamur.ai/free/en/papers/meister-2025-benchmarking-distributional-alignment-large-language/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long P…
75. Mark-Alec Mellor and 1 others. 2023. Interview with ChatGPT. The Irenaut, 1(1):152–164. [find](https://scholar.google.com/scholar?q=Mark%2DAlec+Mellor+and+1+others%2E+2023%2E+Interview+with+ChatGPT%2E+The+Irenaut%2C+1%281%29%3A152%E2%80%93164%2E)
76. Keiichi Namikoshi, Alex Filipowicz, David A. Shamma, Rumen Iliev, Candice L. Hogan, and Nikos Arechiga. 2024. Using llms to model the beliefs and preferences of targeted populations. Preprint, arXiv:2403.20252. — [arXiv:2403.20252](https://arxiv.org/abs/2403.20252) [find](https://scholar.google.com/scholar?q=Keiichi+Namikoshi%2C+Alex+Filipowicz%2C+David+A%2E+Shamma%2C+Rumen+Iliev%2C+Candice+L%2E+Hogan%2C+and+Nikos+Arechiga%2E+2024%2E+Using+llms+to+model+the+beliefs+and+preferences+of+targeted+populations%2E+Preprint%2C+arXiv%3A2)
77. Cornelia Neuert, Katharina Meitinger, Dorothée Behr, and Matthias Schonlau. 2021. Editorial: The use of open-ended questions in surveys. Methods, data, analyses : a journal for quantitative methods and survey methodology (mda), 15(1):3–6. [find](https://scholar.google.com/scholar?q=Cornelia+Neuert%2C+Katharina+Meitinger%2C+Doroth%C3%A9e+Behr%2C+and+Matthias+Schonlau%2E+2021%2E+Editorial%3A+The+use+of+open%2Dended+questions+in+surveys%2E+Methods%2C+data%2C+analyses+%3A+a+journal+for+quantitative+methods+an)
78. Shiwen Ni, Guhong Chen, Shuaimin Li, Xuanang Chen, Siyi Li, Bingli Wang, Qiyao Wang, Xingjian Wang, Yifan Zhang, Liyang Fan, Chengming Li, Ruifeng Xu, Le Sun, and Min Yang. 2025. A survey on large language model benchmarks. Preprint, arXiv:2508.15361. — [arXiv:2508.15361](https://arxiv.org/abs/2508.15361) [find](https://scholar.google.com/scholar?q=Shiwen+Ni%2C+Guhong+Chen%2C+Shuaimin+Li%2C+Xuanang+Chen%2C+Siyi+Li%2C+Bingli+Wang%2C+Qiyao+Wang%2C+Xingjian+Wang%2C+Yifan+Zhang%2C+Liyang+Fan%2C+Chengming+Li%2C+Ruifeng+Xu%2C+Le+Sun%2C+and+Min+Yang%2E+2025%2E+A+survey+on+large+lan)
79. Jacopo Nudo, Mario Edoardo Pandolfo, Edoardo Loru, Mattia Samory, Matteo Cinelli, and Walter Quattrociocchi. 2026. Generative exaggeration in llm social agents: Consistency, bias, and toxicity. Online Social Networks and Media, 51:100344. [find](https://scholar.google.com/scholar?q=Jacopo+Nudo%2C+Mario+Edoardo+Pandolfo%2C+Edoardo+Loru%2C+Mattia+Samory%2C+Matteo+Cinelli%2C+and+Walter+Quattrociocchi%2E+2026%2E+Generative+exaggeration+in+llm+social+agents%3A+Consistency%2C+bias%2C+and+toxicity%2E+Online)
80. Yfke P. Ongena and Wil Dijkstra. 2006. Methods of behavior coding of survey interviews. Journal of Official Statistics, 22(3):1–34. [find](https://scholar.google.com/scholar?q=Yfke+P%2E+Ongena+and+Wil+Dijkstra%2E+2006%2E+Methods+of+behavior+coding+of+survey+interviews%2E+Journal+of+Official+Statistics%2C+22%283%29%3A1%E2%80%9334%2E)
81. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. [Train…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
82. Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23, New York, NY, USA. Ass… [find](https://scholar.google.com/scholar?q=Joon+Sung+Park%2C+Joseph+O%E2%80%99Brien%2C+Carrie+Jun+Cai%2C+Meredith+Ringel+Morris%2C+Percy+Liang%2C+and+Michael+S%2E+Bernstein%2E+2023%2E+Generative+agents%3A+Interactive+simulacra+of+human+behavior%2E+In+Proceedings+of+the+3)
83. Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2022. [Social simulacra: Creating populated prototypes for social computing systems](https://md.research.elamur.ai/free/en/papers/park-2022-social-simulacra-creating-populated-prototypes/). In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST ’22, New Yo…
84. Joon Sung Park, Carolyn Q. Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S. Bernstein. 2024. [Generative agent simulations of 1,000 people](https://md.research.elamur.ai/free/en/papers/park-2024-llm-agents-grounded-self-reports/). Preprint, arXiv:2411.10109.
85. Pouya Pezeshkpour and Estevam Hruschka. 2024. [Large language models sensitivity to the order of options in multiple-choice questions](https://md.research.elamur.ai/free/en/papers/pezeshkpour-2023-large-language-models-sensitivity-order/). In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2006–2017, Mexico City, Mexico. Association for Computational Linguistics.
86. Chau Minh Pham, Alexander Hoyle, Simeng Sun, Philip Resnik, and Mohit Iyyer. 2024. [TopicGPT: A prompt-based topic modeling framework](https://md.research.elamur.ai/free/en/papers/pham-2024-topicgpt-prompt-based-topic-modeling/). In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pape…
87. Jinghua Piao, Yuwei Yan, Jun Zhang, Nian Li, Junbo Yan, Xiaochong Lan, Zhihong Lu, Zhiheng Zheng, Jing Yi Wang, Di Zhou, Chen Gao, Fengli Xu, Fang Zhang, Ke Rong, Jun Su, and Yong Li. 2025. Agentsociety: Large-scale simulation of llm-driven generative agents advances understanding of human behaviors… — [arXiv:2502.08691](https://arxiv.org/abs/2502.08691) [find](https://scholar.google.com/scholar?q=Jinghua+Piao%2C+Yuwei+Yan%2C+Jun+Zhang%2C+Nian+Li%2C+Junbo+Yan%2C+Xiaochong+Lan%2C+Zhihong+Lu%2C+Zhiheng+Zheng%2C+Jing+Yi+Wang%2C+Di+Zhou%2C+Chen+Gao%2C+Fengli+Xu%2C+Fang+Zhang%2C+Ke+Rong%2C+Jun+Su%2C+and+Yong+Li%2E+2025%2E+Agentsocie)
88. Natalia Pinzón, Vikram Koundinya, Ryan E. Galt, William O’R. Dowling, Marcela Baukloh, Namah C. Taku-Forchu, Tracy Schohr, Leslie M. Roche, Samuel Ikendi, Mark Cooper, Lauren E. Parker, and Tapan B. Pathak. 2024. Ai-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud de… [find](https://scholar.google.com/scholar?q=Natalia+Pinz%C3%B3n%2C+Vikram+Koundinya%2C+Ryan+E%2E+Galt%2C+William+O%E2%80%99R%2E+Dowling%2C+Marcela+Baukloh%2C+Namah+C%2E+Taku%2DForchu%2C+Tracy+Schohr%2C+Leslie+M%2E+Roche%2C+Samuel+Ikendi%2C+Mark+Cooper%2C+Lauren+E%2E+Parker%2C+and+Tapan+B%2E+P)
89. Rolf Porst. 2014. Fragebogen, 4 edition. Studien-skripten zur Soziologie. Springer VS, Wiesbaden. [find](https://scholar.google.com/scholar?q=Rolf+Porst%2E+2014%2E+Fragebogen%2C+4+edition%2E+Studien%2Dskripten+zur+Soziologie%2E+Springer+VS%2C+Wiesbaden%2E)
90. Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy. 2024. [Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2024-political-compass-or-spinning-arrow/). In Proceedings of the 62nd Annual Meeting of the Associatio…
91. Jessie Rouder, Olivia Saucier, Rachel Kinder, and Matt Jans. 2021. What to Do With All Those Open-Ended Responses? Data Visualization Techniques for Survey Researchers. Survey Practice. [find](https://scholar.google.com/scholar?q=Jessie+Rouder%2C+Olivia+Saucier%2C+Rachel+Kinder%2C+and+Matt+Jans%2E+2021%2E+What+to+Do+With+All+Those+Open%2DEnded+Responses%3F+Data+Visualization+Techniques+for+Survey+Researchers%2E+Survey+Practice%2E)
92. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. [Whose opinions do language models reflect?](https://md.research.elamur.ai/free/en/papers/santurkar-2023-whose-opinions-language-models-reflect/) In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org.
93. Matthias Schonlau and Mick P. Couper. 2016. Semi-automated categorization of open-ended questions. Survey Research Methods, 10(2):143–152. [find](https://scholar.google.com/scholar?q=Matthias+Schonlau+and+Mick+P%2E+Couper%2E+2016%2E+Semi%2Dautomated+categorization+of+open%2Dended+questions%2E+Survey+Research+Methods%2C+10%282%29%3A143%E2%80%93152%2E)
94. Matthias Schonlau, Julia Weiß, and Jan Marquardt. 2023. Multi-label classification of open-ended questions with bert. In 2023 Big Data Meets Survey Science (BigSurv), pages 1–8. [find](https://scholar.google.com/scholar?q=Matthias+Schonlau%2C+Julia+Wei%C3%9F%2C+and+Jan+Marquardt%2E+2023%2E+Multi%2Dlabel+classification+of+open%2Dended+questions+with+bert%2E+In+2023+Big+Data+Meets+Survey+Science+%28BigSurv%29%2C+pages+1%E2%80%938%2E)
95. Jonathon P. Schuldt and Sungjong Roh. 2014. Media frames and cognitive accessibility: What do “global warming” and “climate change” evoke in partisan minds? Environmental Communication, 8(4):529– 548. [find](https://scholar.google.com/scholar?q=Jonathon+P%2E+Schuldt+and+Sungjong+Roh%2E+2014%2E+Media+frames+and+cognitive+accessibility%3A+What+do+%E2%80%9Cglobal+warming%E2%80%9D+and+%E2%80%9Cclimate+change%E2%80%9D+evoke+in+partisan+minds%3F+Environmental+Communication%2C+8%284%29%3A529%E2%80%93+548%2E)
96. Sander Schulhoff, Michael Ilie, Nishant Balepur, Konstantine Kahadze, Amanda Liu, Chenglei Si, Yinheng Li, Aayush Gupta, HyoJung Han, Sevien Schulhoff, Pranav Sandeep Dulepet, Saurav Vidyadhara, Dayeon Ki, Sweta Agrawal, Chau Pham, Gerson Kroiz, Feileen Li, Hudson Tao, Ashay Srivastava, and 12 other… — [The Prompt Report: A Systematic Survey of Prompt Engineering Techniques](https://md.research.elamur.ai/free/en/papers/schulhoff-2024-prompt-report-systematic-survey-prompt/)
97. Howard Schuman and Stanley Presser. 1979. The open and closed question. American Sociological Review, 44(5):692–712. [find](https://scholar.google.com/scholar?q=Howard+Schuman+and+Stanley+Presser%2E+1979%2E+The+open+and+closed+question%2E+American+Sociological+Review%2C+44%285%29%3A692%E2%80%93712%2E)
98. Howard Schuman and Stanley Presser. 1996. Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context. SAGE Publications, Inc, Thousand Oaks, CA. [find](https://scholar.google.com/scholar?q=Howard+Schuman+and+Stanley+Presser%2E+1996%2E+Questions+and+Answers+in+Attitude+Surveys%3A+Experiments+on+Question+Form%2C+Wording%2C+and+Context%2E+SAGE+Publications%2C+Inc%2C+Thousand+Oaks%2C+CA%2E)
99. Indira Sen, Marlene Lutz, Elisa Rogers, David Garcia, and Markus Strohmaier. 2025a. Missing the margins: A systematic literature review on the demographic representativeness of LLMs. In Findings of the Association for Computational Linguistics: ACL 2025, pages 24263–24289, Vienna, Austria. Associati… [find](https://scholar.google.com/scholar?q=Indira+Sen%2C+Marlene+Lutz%2C+Elisa+Rogers%2C+David+Garcia%2C+and+Markus+Strohmaier%2E+2025a%2E+Missing+the+margins%3A+A+systematic+literature+review+on+the+demographic+representativeness+of+LLMs%2E+In+Findings+of+th)
100. Indira Sen, Bolei Ma, Georg Ahnert, Anna-Carolina Haensch, Tobias Holtdirk, Frauke Kreuter, and Markus Strohmaier. 2025b. Connecting natural language processing and survey methodology: Potentials, challenges, and open questions. [find](https://scholar.google.com/scholar?q=Indira+Sen%2C+Bolei+Ma%2C+Georg+Ahnert%2C+Anna%2DCarolina+Haensch%2C+Tobias+Holtdirk%2C+Frauke+Kreuter%2C+and+Markus+Strohmaier%2E+2025b%2E+Connecting+natural+language+processing+and+survey+methodology%3A+Potentials%2C+cha)
101. Burr Settles. 2009. [Active learning literature survey](https://md.research.elamur.ai/free/en/papers/settles-2009-active-learning-literature-survey/). Technical report, University of Wisconsin-Madison Department of Computer Sciences.
102. Joongi Shin, Michael A. Hedderich, Bartłomiej Jakub Rey, Andrés Lucero, and Antti Oulasvirta. 2024. Understanding human-ai workflows for generating personas. In Proceedings of the 2024 ACM Designing Interactive Systems Conference, DIS ’24, page 757–781, New York, NY, USA. Association for Computing M… [find](https://scholar.google.com/scholar?q=Joongi+Shin%2C+Michael+A%2E+Hedderich%2C+Bart%C5%82omiej+Jakub+Rey%2C+Andr%C3%A9s+Lucero%2C+and+Antti+Oulasvirta%2E+2024%2E+Understanding+human%2Dai+workflows+for+generating+personas%2E+In+Proceedings+of+the+2024+ACM+Designing+I)
103. Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, and David Jurgens. 2024. You don’t need a personality test to know these models are unreliable: Assessing the reliability of large language models on psychometric instruments. In Proceedings of t… [find](https://scholar.google.com/scholar?q=Bangzhao+Shu%2C+Lechen+Zhang%2C+Minje+Choi%2C+Lavinia+Dunagan%2C+Lajanugen+Logeswaran%2C+Moontae+Lee%2C+Dallas+Card%2C+and+David+Jurgens%2E+2024%2E+You+don%E2%80%99t+need+a+personality+test+to+know+these+models+are+unreliable%3A)
104. Eleanor Singer and Mick P. Couper. 2017. Some methodological uses of responses to open questions and other verbatim comments in quantitative surveys. methods, data, analyses, 11(2):115–134. [find](https://scholar.google.com/scholar?q=Eleanor+Singer+and+Mick+P%2E+Couper%2E+2017%2E+Some+methodological+uses+of+responses+to+open+questions+and+other+verbatim+comments+in+quantitative+surveys%2E+methods%2C+data%2C+analyses%2C+11%282%29%3A115%E2%80%93134%2E)
105. Oliver Slumbers, Joel Z. Leibo, and Marco A. Janssen. 2025. Using large language models to simulate human behavioural experiments: Port of mars. Preprint, arXiv:2506.05555. — [arXiv:2506.05555](https://arxiv.org/abs/2506.05555) [find](https://scholar.google.com/scholar?q=Oliver+Slumbers%2C+Joel+Z%2E+Leibo%2C+and+Marco+A%2E+Janssen%2E+2025%2E+Using+large+language+models+to+simulate+human+behavioural+experiments%3A+Port+of+mars%2E+Preprint%2C+arXiv%3A2506%2E05555%2E+%E2%80%94+%5BarXiv%3A2506%2E05555%5D%28https%3A)
106. Kenneth O. Stanley and Joel Lehman. 2015. Why Greatness Cannot Be Planned, 1 edition. Springer Cham, Cham, Switzerland. [find](https://scholar.google.com/scholar?q=Kenneth+O%2E+Stanley+and+Joel+Lehman%2E+2015%2E+Why+Greatness+Cannot+Be+Planned%2C+1+edition%2E+Springer+Cham%2C+Cham%2C+Switzerland%2E)
107. Amir Taubenfeld, Yaniv Dover, Roi Reichart, and Ariel Goldstein. 2024. [Systematic biases in LLM simulations of debates](https://md.research.elamur.ai/free/en/papers/taubenfeld-2024-systematic-biases-llm-simulations-debates/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 251–267, Miami, Florida, USA. Association for Computational Linguistics.
108. Kea Tijdens. 2014. Dropout rates and response times of an occupation search tree in a web survey. Journal of Official Statistics, 30(1):23–43. [find](https://scholar.google.com/scholar?q=Kea+Tijdens%2E+2014%2E+Dropout+rates+and+response+times+of+an+occupation+search+tree+in+a+web+survey%2E+Journal+of+Official+Statistics%2C+30%281%29%3A23%E2%80%9343%2E)
109. Petter Törnberg, Diliara Valeeva, Justus Uitermark, and Christopher Bail. 2023. Simulating social media using large language models to evaluate alternative news feed algorithms. arXiv preprint arXiv:2310.05984. — [arXiv:2310.05984](https://arxiv.org/abs/2310.05984) [find](https://scholar.google.com/scholar?q=Petter+T%C3%B6rnberg%2C+Diliara+Valeeva%2C+Justus+Uitermark%2C+and+Christopher+Bail%2E+2023%2E+Simulating+social+media+using+large+language+models+to+evaluate+alternative+news+feed+algorithms%2E+arXiv+preprint+arXiv%3A2)
110. Roger Tourangeau, Lance J. Rips, and Kenneth Rasin-ski. 2000. The Psychology of Survey Response. Cambridge University Press, Cambridge, UK. [find](https://scholar.google.com/scholar?q=Roger+Tourangeau%2C+Lance+J%2E+Rips%2C+and+Kenneth+Rasin%2Dski%2E+2000%2E+The+Psychology+of+Survey+Response%2E+Cambridge+University+Press%2C+Cambridge%2C+UK%2E)
111. Amos Tversky and Daniel Kahneman. 1981. The framing of decisions and the psychology of choice. Science, 211(4481):453–458. [find](https://scholar.google.com/scholar?q=Amos+Tversky+and+Daniel+Kahneman%2E+1981%2E+The+framing+of+decisions+and+the+psychology+of+choice%2E+Science%2C+211%284481%29%3A453%E2%80%93458%2E)
112. Petter Törnberg. 2025. Large language models outperform expert coders and supervised classifiers at annotating political social media messages. Social Science Computer Review, 43(6):1181–1195. [find](https://scholar.google.com/scholar?q=Petter+T%C3%B6rnberg%2E+2025%2E+Large+language+models+outperform+expert+coders+and+supervised+classifiers+at+annotating+political+social+media+messages%2E+Social+Science+Computer+Review%2C+43%286%29%3A1181%E2%80%931195%2E)
113. Leah von der Heyde, Anna-Carolina Haensch, Bernd Weiß, and Jessica Daikeler. 2025a. Using large language models for coding german open-ended survey responses on survey motivation. Survey Research Methods, 19(4):355–370. [find](https://scholar.google.com/scholar?q=Leah+von+der+Heyde%2C+Anna%2DCarolina+Haensch%2C+Bernd+Wei%C3%9F%2C+and+Jessica+Daikeler%2E+2025a%2E+Using+large+language+models+for+coding+german+open%2Dended+survey+responses+on+survey+motivation%2E+Survey+Research+Meth)
114. Leah von der Heyde, Anna-Carolina Haensch, and Alexander Wenz. 2025b. Vox populi, vox ai? using large language models to estimate german vote choice. Social Science Computer Review, 0(0):08944393251337014. [find](https://scholar.google.com/scholar?q=Leah+von+der+Heyde%2C+Anna%2DCarolina+Haensch%2C+and+Alexander+Wenz%2E+2025b%2E+Vox+populi%2C+vox+ai%3F+using+large+language+models+to+estimate+german+vote+choice%2E+Social+Science+Computer+Review%2C+0%280%29%3A0894439325133)
115. Qian Wang, Jiaying Wu, Zhenheng Tang, Bingqiao Luo, Nuo Chen, Wei Chen, and Bingsheng He. 2025a. What limits llm-based human simulation: Llms or our design? Preprint, arXiv:2501.08579. — [arXiv:2501.08579](https://arxiv.org/abs/2501.08579) [find](https://scholar.google.com/scholar?q=Qian+Wang%2C+Jiaying+Wu%2C+Zhenheng+Tang%2C+Bingqiao+Luo%2C+Nuo+Chen%2C+Wei+Chen%2C+and+Bingsheng+He%2E+2025a%2E+What+limits+llm%2Dbased+human+simulation%3A+Llms+or+our+design%3F+Preprint%2C+arXiv%3A2501%2E08579%2E+%E2%80%94+%5BarXiv%3A2501%2E0)
116. Xinpeng Wang, Chengzhi Hu, Bolei Ma, Paul Rottger, and Barbara Plank. 2024a. Look at the text: Instruction-tuned language models are more robust multiple choice selectors than you think. In First Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=Xinpeng+Wang%2C+Chengzhi+Hu%2C+Bolei+Ma%2C+Paul+Rottger%2C+and+Barbara+Plank%2E+2024a%2E+Look+at+the+text%3A+Instruction%2Dtuned+language+models+are+more+robust+multiple+choice+selectors+than+you+think%2E+In+First+Conf)
117. Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy, and Barbara Plank. 2024b. [“my answer is C”: First-token probabilities do not match text answers in instruction-tuned language models](https://md.research.elamur.ai/free/en/papers/wang-2024-my-answer-is-c-first/). In Findings of the Association for Computational Linguistics ACL 2024…
118. Xunzhi Wang, Zhuowei Zhang, Gaonan Chen, Qiongyu Li, Bitong Luo, Zhixin Han, Haotian Wang, Zhiyu Li, Hang Gao, and Mengting Hu. 2025b. UBench: Benchmarking uncertainty in large language models with multiple choice questions. In Findings of the Association for Computational Linguistics: ACL 2025, pag… [find](https://scholar.google.com/scholar?q=Xunzhi+Wang%2C+Zhuowei+Zhang%2C+Gaonan+Chen%2C+Qiongyu+Li%2C+Bitong+Luo%2C+Zhixin+Han%2C+Haotian+Wang%2C+Zhiyu+Li%2C+Hang+Gao%2C+and+Mengting+Hu%2E+2025b%2E+UBench%3A+Benchmarking+uncertainty+in+large+language+models+with+mu)
119. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc.
120. Sheng-Lun Wei, Cheng-Kuang Wu, Hen-Hsen Huang, and Hsin-Hsi Chen. 2024. Unveiling selection biases: Exploring order and token sensitivity in large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5598– 5621, Bangkok, Thailand. Association for Computation… [find](https://scholar.google.com/scholar?q=Sheng%2DLun+Wei%2C+Cheng%2DKuang+Wu%2C+Hen%2DHsen+Huang%2C+and+Hsin%2DHsi+Chen%2E+2024%2E+Unveiling+selection+biases%3A+Exploring+order+and+token+sensitivity+in+large+language+models%2E+In+Findings+of+the+Association+for+C)
121. Joseph Wilson, Benjamin Pollard, John M. Aiken, Marcos D. Caballero, and H. J. Lewandowski. 2022. Classification of open-ended responses to a research-based assessment using natural language processing. Phys. Rev. Phys. Educ. Res., 18:010141. [find](https://scholar.google.com/scholar?q=Joseph+Wilson%2C+Benjamin+Pollard%2C+John+M%2E+Aiken%2C+Marcos+D%2E+Caballero%2C+and+H%2E+J%2E+Lewandowski%2E+2022%2E+Classification+of+open%2Dended+responses+to+a+research%2Dbased+assessment+using+natural+language+processin)
122. Alexander Wuttke, Matthias Aßenmacher, Christopher Klamm, Max M. Lang, Quirin Würschinger, and Frauke Kreuter. 2025. AI conversational interviewing: Transforming surveys with LLMs as adaptive interviewers. In Proceedings of the 9th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heri… [find](https://scholar.google.com/scholar?q=Alexander+Wuttke%2C+Matthias+A%C3%9Fenmacher%2C+Christopher+Klamm%2C+Max+M%2E+Lang%2C+Quirin+W%C3%BCrschinger%2C+and+Frauke+Kreuter%2E+2025%2E+AI+conversational+interviewing%3A+Transforming+surveys+with+LLMs+as+adaptive+intervie)
123. Weijia Xu, Nebojsa Jojic, Sudha Rao, Chris Brockett, and Bill Dolan. 2025. Echoes in ai: Quantifying lack of plot diversity in llm outputs. Proceedings of the National Academy of Sciences, 122(35):e2504966122. [find](https://scholar.google.com/scholar?q=Weijia+Xu%2C+Nebojsa+Jojic%2C+Sudha+Rao%2C+Chris+Brockett%2C+and+Bill+Dolan%2E+2025%2E+Echoes+in+ai%3A+Quantifying+lack+of+plot+diversity+in+llm+outputs%2E+Proceedings+of+the+National+Academy+of+Sciences%2C+122%2835%29%3Ae25)
124. Diyi Yang, Dirk Hovy, David Jurgens, and Barbara Plank. 2025. [Socially aware language technologies: Perspectives and practices](https://md.research.elamur.ai/free/en/papers/yang-2025-socially-aware-language-technologies-perspectives/). Computational Linguistics, 51:689–703.
125. Simone Zhang, Janet Xu, and AJ Alvero. 2025. Generative ai meets open-ended survey responses: Research participant use of ai and homogenization. Sociological Methods & Research, 54(3):1197–1242. [find](https://scholar.google.com/scholar?q=Simone+Zhang%2C+Janet+Xu%2C+and+AJ+Alvero%2E+2025%2E+Generative+ai+meets+open%2Dended+survey+responses%3A+Research+participant+use+of+ai+and+homogenization%2E+Sociological+Methods+%26+Research%2C+54%283%29%3A1197%E2%80%931242%2E)
126. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020a. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Varsha+Kishore%2C+Felix+Wu%2C+Kilian+Q%2E+Weinberger%2C+and+Yoav+Artzi%2E+2020a%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+In+International+Conference+on+Learning+Representations%2E)
127. Yiqun Zhang, Xiaocui Yang, Shi Feng, Daling Wang, Yifei Zhang, and Kaisong Song. 2024. Can llms beat humans in debating? a dynamic multi-agent framework for competitive debate. Preprint, arXiv:2408.04472. — [arXiv:2408.04472](https://arxiv.org/abs/2408.04472) [find](https://scholar.google.com/scholar?q=Yiqun+Zhang%2C+Xiaocui+Yang%2C+Shi+Feng%2C+Daling+Wang%2C+Yifei+Zhang%2C+and+Kaisong+Song%2E+2024%2E+Can+llms+beat+humans+in+debating%3F+a+dynamic+multi%2Dagent+framework+for+competitive+debate%2E+Preprint%2C+arXiv%3A2408%2E04)
128. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020b. [DIALOGPT : Large-scale generative pre-training for conversational response generation](https://md.research.elamur.ai/free/en/papers/zhang-2019-dialogpt-large-scale-generative-pre/). In Proceedings of the 58th Annual Meeting of the Association for Computational Lin…
129. Zhisong Zhang, Emma Strubell, and Eduard Hovy. 2022. A survey of active learning for natural language processing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6166–6190, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhisong+Zhang%2C+Emma+Strubell%2C+and+Eduard+Hovy%2E+2022%2E+A+survey+of+active+learning+for+natural+language+processing%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natural+Language+Process)
130. Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2024. Large language models are not robust multiple choice selectors. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Chujie+Zheng%2C+Hao+Zhou%2C+Fandong+Meng%2C+Jie+Zhou%2C+and+Minlie+Huang%2E+2024%2E+Large+language+models+are+not+robust+multiple+choice+selectors%2E+In+The+Twelfth+International+Conference+on+Learning+Representati)
131. Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. 2024. [Can large language models transform computational social science?](https://md.research.elamur.ai/free/en/papers/ziems-2023-large-language-models-transform-computational/) Computational Linguistics, 50(1):237–291.
132. Cornelia Züll. 2016. Open-Ended Questions. GESIS – Leibniz Institute for the Social Sciences, Mannheim, Germany. [find](https://scholar.google.com/scholar?q=Cornelia+Z%C3%BCll%2E+2016%2E+Open%2DEnded+Questions%2E+GESIS+%E2%80%93+Leibniz+Institute+for+the+Social+Sciences%2C+Mannheim%2C+Germany%2E)

*Updated 2 November 2026*