---
title: "Algorithmic Fidelity of Large Language Models in Generating"
description: "They build persona prompts in German from six respondent attributes. 44 references, cited by 8 papers here."
authors: ["Bolei Ma", "Berk Yoztyurk", "Anna-Carolina Haensch", "Xinpeng Wang", "Markus Herklotz", "Frauke Kreuter", "Barbara Plank", "Matthias Aßenmacher"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.90/"
doi: "10.18653/v1/2025.acl-long.90"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ma-2025-algorithmic-fidelity-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ma-2025-algorithmic-fidelity-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/) / Algorithmic Fidelity of Large Language Models in Generating Synthetic German Public Opinions: A Case Study

# Algorithmic Fidelity of Large Language Models in Generating Synthetic German Public Opinions: A Case Study

**Authors:** [Bolei Ma](https://md.research.elamur.ai/free/en/authors/bolei-ma/), Berk Yoztyurk, [Anna-Carolina Haensch](https://md.research.elamur.ai/free/en/authors/anna-carolina-haensch/), [Xinpeng Wang](https://md.research.elamur.ai/free/en/authors/xinpeng-wang/), Markus Herklotz, [Frauke Kreuter](https://md.research.elamur.ai/free/en/authors/frauke-kreuter/), [Barbara Plank](https://md.research.elamur.ai/free/en/authors/barbara-plank/) and [Matthias Aßenmacher](https://md.research.elamur.ai/free/en/authors/matthias-assenmacher/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.90/) · [PDF](https://aclanthology.org/2025.acl-long.90.pdf) · [DOI 10.18653/v1/2025.acl-long.90](https://doi.org/10.18653/v1/2025.acl-long.90) · [OpenAlex](https://openalex.org/W4412886803)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/ma-2025-algorithmic-fidelity-large-language-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tests whether LLMs show "algorithmic fidelity", meaning whether they can mimic how specific German population groups answer an open-ended survey question. The authors use the GLES Panel and its free-text item on Germany's most important problem. They build persona prompts in German from six respondent attributes. Three open-weight chat models (Llama-2-13b-chat-hf, Gemma-7b-it, Mixtral-8x7B-Instruct-v0.1) answer as those personas.

To compare the free-text outputs with human answers, they hand-label LLM responses and fine-tune a German BERT classifier that maps each answer to one of 16 coarse problem categories. They then compare distributions using JS distance, entropy and predictive measures.

Llama2 matches subgroup opinions best, and it does so most closely when a group's own answers are less varied. Alignment is uneven across parties: it is strongest for supporters of The Greens and The Left and weakest for AfD supporters. Adding more persona variables improves results, and party preference matters most. The work extends opinion-simulation studies beyond English and beyond closed multiple-choice formats.

## Key points
- Data come from the GLES Panel (GESIS), which has 21 waves. The figures cover November 2019 to November 2021, and they show Health Policy becoming the dominant problem category after the COVID-19 outbreak.
- Personas use six variables: age, gender, party leaning, region, school education and vocational degree. Prompts are written in German, the original survey language.
- About 50 fine GLES coding classes were merged into 16 coarse ones. For example, Price Level and Housing Policy were folded into Economic Policy.
- The authors manually annotated 1,500 LLM outputs (500 per model) to train a German BERT classifier. It reached a weighted F1 of 0.93 on the test set.
- Representativeness drops as answer diversity in a survey wave rises. Party affiliation is the most influential prompt variable, and results are sensitive to which variables are included or left out.

## Interesting ideas
- Instead of parsing free-form generations with rules, the authors train a lightweight classifier on hand-labeled *model outputs*, which gives a robust and scalable way to code open-ended text.
- Collapsing a large fine-grained taxonomy into a handful of coarse classes makes evaluation statistically stable, while the fine labels are kept for deeper analysis.
- Comparing whole label distributions with a bounded metric (JS distance) allows fair comparison across time slices and subgroups, which per-item accuracy alone does not reveal.
- The study links model fidelity to the entropy of human answers within a group, showing that difficulty grows with genuine diversity of the target.
- The authors switch individual persona variables on and off to measure how much each one drives the output, which is a simple and interpretable ablation design.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): A small hand-labeled set of LLM teacher answers could train a classifier that maps teacher free text onto Mova's own label inventory, making teacher tiers usable without brittle string matching.
- **I2** (world): Verb-class or ontology evaluations could first be scored at a coarse level (e.g., merged VerbNet families), and fine-grained distinctions could then be checked separately.
- **I3** (learn): Comparing the distribution of dependency relations or frame roles in silver data against gold, per genre, would flag systematic drift even when per-sentence accuracy looks fine.
- **I4** (answer): Test results on school-course and fable QA could be stratified by how much human answers vary, so failures on inherently open questions are not confused with reading errors.
- **I5** (learn): The same include/exclude scheme could be applied to parser features or teacher-prompt context, with the removed-feature runs serving as negative controls.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Ludwig-Maximilians-Universität München](https://md.research.elamur.ai/free/en/universities/ludwig-maximilians-universitat-munchen/) — University, Munich, Germany · 107 works · 56,478 citations
- [Munich Center for Machine Learning](https://md.research.elamur.ai/free/en/universities/munich-center-for-machine-learning/) — University, Munich, Germany · 57 works · 1,094 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Bolei Ma, Berk Yoztyurk, Anna-Carolina Haensch, Xinpeng Wang, Markus Herklotz, Frauke Kreuter, Barbara Plank and Matthias Aßenmacher. 2025. Algorithmic Fidelity of Large Language Models in Generating Synthetic German Public Opinions: A Case Study. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ma2025algorithmic,
  title = {{Algorithmic Fidelity of Large Language Models in Generating Synthetic German Public Opinions: A Case Study}},
  author = {Bolei Ma, Berk Yoztyurk, Anna-Carolina Haensch, Xinpeng Wang, Markus Herklotz, Frauke Kreuter, Barbara Plank and Matthias Aßenmacher},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.90},
  url = {https://doi.org/10.18653/v1/2025.acl-long.90}
}
```

- [← previous A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directions](https://md.research.elamur.ai/free/en/papers/hamed-2025-survey-code-switched-arabic-nlp/)
- [next → Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race](https://md.research.elamur.ai/free/en/papers/sun-2025-aligned-but-blind-alignment-increases/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/)

## References

44 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Utkarsh Agarwal, Kumar Tanmay, Aditi Khandelwal, and Monojit Choudhury. 2024. Ethical reasoning and moral value alignment of LLMs depend on the language we prompt them in. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 6330–6340, Torino, Italia. ELRA and ICCL. [find](https://scholar.google.com/scholar?q=Utkarsh+Agarwal%2C+Kumar+Tanmay%2C+Aditi+Khandelwal%2C+and+Monojit+Choudhury%2E+2024%2E+Ethical+reasoning+and+moral+value+alignment+of+LLMs+depend+on+the+language+we+prompt+them+in%2E+In+Proceedings+of+the+2024+J)
2. Hans-Jürgen Andreß, Katrin Golsch, and Alexander W. Schmidt. 2013. Applied Panel Data Analysis for Economic and Social Surveys. Springer. [↗](https://doi.org/10.1007/978-3-642-32914-2)
3. Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. 2023. [Out of one, many: Using language models to simulate human samples](https://md.research.elamur.ai/free/en/papers/argyle-2022-out-one-many-using-language/). Political Analysis, 31(3):337–351. [↗](https://doi.org/10.1017/pan.2023.2)
4. Arnav Arora, Lucie-aimée Kaffee, and Isabelle Augenstein. 2023. [Probing pre-trained language models for cross-cultural differences in values](https://md.research.elamur.ai/free/en/papers/arora-2023-probing-pre-trained-language-models/). In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 114–130, Dubrovnik, Croatia. Association for Computational Linguistics.
5. Marcel Binz and Eric Schulz. 2023. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences, 120(6):e2218523120. [↗](https://doi.org/10.1073/pnas.2218523120)
6. James Bisbee, Joshua D. Clinton, Cassy Dorff, Brenton Kenkel, and Jennifer M. Larson. 2024. Synthetic replacements for human survey data? the perils of large language models. Political Analysis, 32(4):401–416. [↗](https://doi.org/10.1017/pan.2024.5)
7. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam Mc-Candlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Preprint, arXiv:2005.14165. [↗](https://arxiv.org/abs/2005.14165)
8. Yong Cao, Haijiang Liu, Arnav Arora, Isabelle Augenstein, Paul Röttger, and Daniel Hershcovich. 2025. [Specializing large language models to simulate survey response distributions for global populations](https://md.research.elamur.ai/free/en/papers/cao-2025-specializing-large-language-models-simulate/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3141–3154, Albuquerque, New Mexico. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2502.07068)
9. Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. [Assessing cross-cultural alignment between ChatGPT and human societies: An empirical study](https://md.research.elamur.ai/free/en/papers/cao-2023-assessing-cross-cultural-alignment-between/). In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 53–67, Dubrovnik, Croatia. Association for Computational Linguistics.
10. Madiha Zahrah Choksi, Marianne Aubin Le Quéré, Travis Lloyd, Ruojia Tao, James Grimmelmann, and Mor Naaman. 2024. Under the (neighbor)hood: Hyperlocal surveillance on nextdoor. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI ’24, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Madiha+Zahrah+Choksi%2C+Marianne+Aubin+Le+Qu%C3%A9r%C3%A9%2C+Travis+Lloyd%2C+Ruojia+Tao%2C+James+Grimmelmann%2C+and+Mor+Naaman%2E+2024%2E+Under+the+%28neighbor%29hood%3A+Hyperlocal+surveillance+on+nextdoor%2E+In+Proceedings+of+the+C)
11. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46. [↗](https://doi.org/10.1177/001316446002000104)
12. Harald Cramér. 1999. Mathematical Methods of Statistics (PMS-9). Princeton University Press. Neural Information Processing Systems, volume 37, pages 45850–45878. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Harald+Cram%C3%A9r%2E+1999%2E+Mathematical+Methods+of+Statistics+%28PMS%2D9%29%2E+Princeton+University+Press%2E+Neural+Information+Processing+Systems%2C+volume+37%2C+pages+45850%E2%80%9345878%2E+Curran+Associates%2C+Inc%2E)
13. Esin Durmus, Karina Nguyen, Thomas Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2024. [Towards measuring the representation of subjective global opinions in language models](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/). In First Conference on Language Modeling. [↗](https://doi.org/10.48550/arxiv.2306.16388)
14. Peter Eigenschink, Thomas Reutterer, Stefan Vamosi, Ralf Vamosi, Chang Sun, and Klaudius Kalcher. 2023. Deep Generative Models for Synthetic Data: A Survey. IEEE Access, 11:47304–47320. [↗](https://doi.org/10.1109/access.2023.3275134)
15. GESIS. 2023. Gles panel 2016-2021, wellen 1-21. GESIS, Köln. ZA6838 Datenfile Version 6.0.0, https://doi.org/10.4232/1.14114. [↗](https://doi.org/10.4232/1.14114)
16. GESIS. 2024. Gles codierung des wichtigsten politischen problems (2018-2022). GESIS, Köln. ZA7957 Datenfile Version 2.0.0, https://doi.org/10.4232/1.14241. [↗](https://doi.org/10.4232/1.14241)
17. Perttu Hämäläinen, Mikke Tavast, and Anton Kunnari. 2023. Evaluating large language models in generating synthetic hci research data: a case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Perttu+H%C3%A4m%C3%A4l%C3%A4inen%2C+Mikke+Tavast%2C+and+Anton+Kunnari%2E+2023%2E+Evaluating+large+language+models+in+generating+synthetic+hci+research+data%3A+a+case+study%2E+In+Proceedings+of+the+2023+CHI+Conference+on+Human+F)
18. Thomas M. Holbrook. 2022. An Introduction to Political and Social Data Analysis Using R. Thomas M. Holbrook. [find](https://scholar.google.com/scholar?q=Thomas+M%2E+Holbrook%2E+2022%2E+An+Introduction+to+Political+and+Social+Data+Analysis+Using+R%2E+Thomas+M%2E+Holbrook%2E)
19. Tobias Holtdirk, Dennis Assenmacher, Arnim Bleier, and Claudia Wagner. 2025. Addressing Systematic Non-response Bias with Supervised Fine-Tuning of Large Language Models: A Case Study on German Voting Behaviour. [↗](https://doi.org/10.31219/osf.io/udz28_v2)
20. EunJeong Hwang, Bodhisattwa Majumder, and Niket Tandon. 2023. [Aligning language models to user opinions](https://md.research.elamur.ai/free/en/papers/hwang-2023-aligning-language-models-user-opinions/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5906– 5919, Singapore. Association for Computational Linguistics.
21. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
22. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024a. Mixtral of experts. Preprint, arXiv:2401.04088. [↗](https://arxiv.org/abs/2401.04088)
23. Ricardo Dominguez-Olmedo, Moritz Hardt, and Celestine Mendler-Dünner. 2024. Questioning the survey responses of large language models. In Advances in Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. 2024b. PersonaLLM: Investigating the ability of large language models to express personality traits. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3605–3627, Mexico City, Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ricardo+Dominguez%2DOlmedo%2C+Moritz+Hardt%2C+and+Celestine+Mendler%2DD%C3%BCnner%2E+2024%2E+Questioning+the+survey+responses+of+large+language+models%2E+In+Advances+in+Hang+Jiang%2C+Xiajie+Zhang%2C+Xubo+Cao%2C+Cynthia+Breaze)
24. Dan Jurafsky and James H. Martin. 2024. Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition. [find](https://scholar.google.com/scholar?q=Dan+Jurafsky+and+James+H%2E+Martin%2E+2024%2E+Speech+and+language+processing%3A+an+introduction+to+natural+language+processing%2C+computational+linguistics%2C+and+speech+recognition%2E)
25. S. Kullback and R. A. Leibler. 1951. On Information and Sufficiency. The Annals of Mathematical Statistics, 22(1):79 – 86. [↗](https://doi.org/10.1214/aoms/1177729694)
26. Jianhua Lin. 2006. Divergence measures based on the shannon entropy. IEEE Trans. Inf. Theor., 37(1):145–151. [find](https://scholar.google.com/scholar?q=Jianhua+Lin%2E+2006%2E+Divergence+measures+based+on+the+shannon+entropy%2E+IEEE+Trans%2E+Inf%2E+Theor%2E%2C+37%281%29%3A145%E2%80%93151%2E)
27. Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. [On LLMs-driven synthetic data generation, curation, and evaluation: A survey](https://md.research.elamur.ai/free/en/papers/long-2024-llms-driven-synthetic-data-generation/). In Findings of the Association for Computational Linguistics ACL 2024, pages 11065–11082, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
28. Bolei Ma, Xinpeng Wang, Tiancheng Hu, Anna-Carolina Haensch, Michael A. Hedderich, Barbara Plank, and Frauke Kreuter. 2024. The potential and challenges of evaluating attitudes, opinions, and values in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 8783–8805, Miami, Florida, USA. Association for Computational Linguistics. for Computational Linguistics (Volume 1: Long Papers), pages 15295–15311, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bolei+Ma%2C+Xinpeng+Wang%2C+Tiancheng+Hu%2C+Anna%2DCarolina+Haensch%2C+Michael+A%2E+Hedderich%2C+Barbara+Plank%2C+and+Frauke+Kreuter%2E+2024%2E+The+potential+and+challenges+of+evaluating+attitudes%2C+opinions%2C+and+values+i)
29. Guilherme Sanches de Oliveira and Edward Baggs. 2023. Psychology’s WEIRD Problems. Elements in Psychology and Culture. Cambridge University Press. [↗](https://doi.org/10.1017/9781009303538)
30. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. [Whose opinions do language models reflect?](https://md.research.elamur.ai/free/en/papers/santurkar-2023-whose-opinions-language-models-reflect/) In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org. [↗](https://doi.org/10.48550/arxiv.2303.17548)
31. Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, and David Jurgens. 2024. You don’t need a personality test to know these models are unreliable: Assessing the reliability of large language models on psychometric instruments. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 5263–5281, Mexico City, Mexico. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2311.09718)
32. Peter M. Steiner, Christiane Atzmüller, and Dan Su. 2017. Designing valid and reliable vignette experiments for survey research: A case study on the fair gender income gap. Journal of Methods and Measurement in the Social Sciences, 7:52–94. [↗](https://doi.org/10.2458/v7i2.20321)
33. Jonathan Mellon, Jack Bailey, Ralph Scott, James Breckwoldt, Marta Miori, and Phillip Schmedeman. 2024. Do ais know what the most important issue is? using language models to code open-text social survey responses at scale. Research & Politics, 11(1):20531680241231468. [↗](https://doi.org/10.1177/20531680241231468)
34. Seungjong Sun, Eungu Lee, Dongyan Nan, Xiangying Zhao, Wonbyung Lee, Bernard J. Jansen, and Jang Hyun Kim. 2024. Random silicon sampling: Simulating human sub-population opinion using a large language model based on group-level demographic information. Preprint, arXiv:2402.18144. [↗](https://arxiv.org/abs/2402.18144)
35. Keiichi Namikoshi, Alex Filipowicz, David A. Shamma, Rumen Iliev, Candice L. Hogan, and Nikos Arechiga. 2024. Using llms to model the beliefs and preferences of targeted populations. Preprint, arXiv:2403.20252. [↗](https://arxiv.org/abs/2403.20252)
36. Gemma Team. 2024. Gemma: Open models based on gemini research and technology. Preprint, arXiv:2403.08295. [↗](https://arxiv.org/abs/2403.08295)
37. Joon Sung Park, Carolyn Q. Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S. Bernstein. 2024. Generative agent simulations of 1,000 people. Preprint, arXiv:2411.10109. [↗](https://arxiv.org/abs/2411.10109)
38. Philip Resnik, Bolei Ma, Alexander Hoyle, Pranav Goel, Rupak Sarkar, Maeve Gearing, Anna-Carolina Haensch, and Frauke Kreuter. 2024. Topic-oriented protocol for content analysis of text – a preliminary study. Unpublished manuscript. [find](https://scholar.google.com/scholar?q=Philip+Resnik%2C+Bolei+Ma%2C+Alexander+Hoyle%2C+Pranav+Goel%2C+Rupak+Sarkar%2C+Maeve+Gearing%2C+Anna%2DCarolina+Haensch%2C+and+Frauke+Kreuter%2E+2024%2E+Topic%2Doriented+protocol+for+content+analysis+of+text+%E2%80%93+a+preliminar)
39. Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy. 2024. [Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2024-political-compass-or-spinning-arrow/). In Proceedings of the 62nd Annual Meeting of the Association Lindia Tjuatja, Valerie Chen, Tongshuang Wu, Ameet Talwalkwar, and Graham Neubig. 2024. Do LLMs exhibit human-like response biases? a case study in survey design. Transactions of the Association for Computational Linguistics, 12:1011–1026. [↗](https://doi.org/10.48550/arxiv.2402.16786)
40. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). Preprint, arXiv:2302.13971. [↗](https://arxiv.org/abs/2302.13971)
41. Veniamin Veselovsky, Manoel Horta Ribeiro, Akhil Arora, Martin Josifoski, Ashton Anderson, and Robert West. 2023. Generating faithful synthetic data with large language models: A case study in computational social science. Preprint, arXiv:2305.15041. [↗](https://arxiv.org/abs/2305.15041)
42. Leah von der Heyde, Anna-Carolina Haensch, and Alexander Wenz. 2025. Vox populi, vox ai? using large language models to estimate german vote choice. Social Science Computer Review, 0(0):08944393251337014. [↗](https://doi.org/10.1177/08944393251337014)
43. Xinpeng Wang, Chengzhi Hu, Bolei Ma, Paul Rottger, and Barbara Plank. 2024a. Look at the text: Instruction-tuned language models are more robust multiple choice selectors than you think. In First Conference on Language Modeling. [↗](https://doi.org/10.48550/arxiv.2404.08382)
44. Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy, and Barbara Plank. 2024b. [“my answer is C”: First-token probabilities do not match text answers in instruction-tuned language models](https://md.research.elamur.ai/free/en/papers/wang-2024-my-answer-is-c-first/). In Findings of the Association for Computational Linguistics ACL 2024, pages 7407–7416, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics. Wave Start Date End Date 10 11 12 13 14 15 16 17 18 19 20 21 06-11-2018 28-05-2019 05-11-2019 21-04-2020 03-11-2020 25-02-2021 06-05-2021 07-07-2021 11-08-2021 15-09-2021 29-09-2021 09-12-2021 21-11-2018 12-06-2019 19-11-2019 05-05-2020 17-11-2020 12-03-2021 19-07-2021 20-07-2021 24-08-2021 25-09-2021 12-10-2021 21-12-2021 Age Gender Male Female Leaning Party AfD CDU/CSU FDP Grünen A minor party Linke SPD No party Region East Germany West Germany Completed vocational internship/volunteer work Vocational school diploma University of applied sciences degree Specialist school diploma Completed apprenticeship Vocational Degree Master craftsman or technician qualification University degree In vocational training Commercial or agricultural apprenticeship Commercial apprenticeship No vocational training completed Data and Prompt Template Table 4 shows the dates for the waves involved in the original GLES survey (GESIS, 2023). The six main social demographic variables and their subgroups in the original survey are presented in Table 5. Sub-Groups 18-29 30-44 45-59 60+ High school diploma Higher education entrance qualification Secondary school diploma Education Degree Intermediate school diploma Is still student No school diploma Xuena Wang, Xueting Li, Zi Yin, Yue Wu, and Jia Liu. 2023. Emotional intelligence of large language models. Journal of Pacific Rim Psychology, 17:1–12.

*Updated 1 November 2026*