---
title: "Towards Simulating Social Media Users with LLMs: Evaluating"
description: "To test this, the authors introduce Conditioned Comment Prediction (CCP). 22 references, cited by 1 paper here."
authors: ["Nils Schwager", "Simon Münker", "Alistair Plum", "Achim Rettinger"]
year: 2026
venue: "The Proceedings for the 15th Workshop on Computational Approaches to Subjectivity, Sentiment Social Media Analysis (WASSA 2026)"
source: "https://aclanthology.org/2026.wassa-1.16/"
doi: "10.18653/v1/2026.wassa-1.16"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/schwager-2026-simulating-social-media-users-llms/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/schwager-2026-simulating-social-media-users-llms/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/) / Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction

# Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction

**Authors:** [Nils Schwager](https://md.research.elamur.ai/free/en/authors/nils-schwager/), [Simon Münker](https://md.research.elamur.ai/free/en/authors/simon-munker/), [Alistair Plum](https://md.research.elamur.ai/free/en/authors/alistair-plum/) and [Achim Rettinger](https://md.research.elamur.ai/free/en/authors/achim-rettinger/) · **Year:** 2026 · **Venue:** [The Proceedings for the 15th Workshop on Computational Approaches to Subjectivity, Sentiment Social Media Analysis (WASSA 2026)](https://md.research.elamur.ai/free/en/venues/wassa/) ([Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis 2026](https://md.research.elamur.ai/free/en/venues/wassa/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.wassa-1.16/) · [PDF](https://aclanthology.org/2026.wassa-1.16.pdf) · [DOI 10.18653/v1/2026.wassa-1.16](https://doi.org/10.18653/v1/2026.wassa-1.16) · [OpenAlex](https://openalex.org/W7140079217)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/schwager-2026-simulating-social-media-users-llms/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether LLMs used as stand-ins for real people in social science can reproduce how *specific* individuals behave online, rather than just producing comments that look plausible. To test this, the authors introduce Conditioned Comment Prediction (CCP). In this task, a model receives a stimulus (a post or a news article) and must generate the reply a particular user would write. The output is then scored against that user's real comment with lexical metrics (ROUGE, BLEU) and a semantic one (Embedding Distance). Three open-weight 8B models (Llama3.1, Qwen3, Ministral) are tested on English, German and Luxembourgish data. The comparison covers four conditions: no personalization, a generated persona biography, the user's past replies, and both combined. Each condition is run with and without Supervised Fine-Tuning (SFT). Fine-tuning makes the outputs' length and syntax more realistic, but in low-resource settings it weakens semantic grounding. After SFT, biographies add nothing beyond behavioral history. The authors conclude that simulation studies should rely on authentic user traces rather than descriptive persona prompts.

## Key points
- **Data:**
  - German X corpus: 3.38M tweets on political discourse from the first half of 2023.
  - English X corpus: 7.79M tweets, collected around 100 politically active accounts and their followee networks.
  - Luxembourgish corpus: 1.02M moderated RTL news comments from 21,427 users, posted 2012–2024.
- **Conditioning setups:**
  - Implicit: up to 30 earlier stimulus–reply pairs, given as chat turns.
  - Explicit: a four-part biography (basics, language, worldview, behavior) inferred by Qwen3-235B-A22B-Instruct-2507.
  - Combined: both of the above.
  - Control: neither, only the stimulus and a generic instruction.
- **SFT setup:** one epoch, maximum sequence length of 4,500 tokens, paged 8-bit AdamW, TRL defaults, trained on a single NVIDIA L40S GPU.
- **Form vs. content:** the decoupling of surface form from meaning is strongest for Luxembourgish. The authors suggest testing larger models (≥70B) and semantics-aware training objectives such as DPO.
- **Limitations and ethics:**
  - Fine-tuned models already perform well with only 5–10 history examples, which the authors flag as a dual-use risk for impersonation.
  - The evaluation uses automated metrics only.
  - Results across the three languages are not strictly comparable.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Luxembourg](https://md.research.elamur.ai/free/en/universities/university-of-luxembourg/) — University, Luxembourg, Luxembourg · 27 works · 337 citations
- [Universität Trier](https://md.research.elamur.ai/free/en/universities/universitat-trier/) — University, Trier, Germany · 8 works · 9 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 227 works

## Cite

Cite as: Nils Schwager, Simon Münker, Alistair Plum and Achim Rettinger. 2026. Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction. The Proceedings for the 15th Workshop on Computational Approaches to Subjectivity, Sentiment Social Media Analysis (WASSA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{schwager2026simulating,
  title = {{Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction}},
  author = {Nils Schwager, Simon Münker, Alistair Plum and Achim Rettinger},
  booktitle = {The Proceedings for the 15th Workshop on Computational Approaches to Subjectivity, Sentiment Social Media Analysis (WASSA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.wassa-1.16},
  url = {https://doi.org/10.18653/v1/2026.wassa-1.16}
}
```

- [← previous Towards Proactive Personalization through Profile Customization for Individual Users in Dialogues](https://md.research.elamur.ai/free/en/papers/zhang-2026-proactive-personalization-through-profile-customization/)
- [next → Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts](https://md.research.elamur.ai/free/en/papers/zhang-2026-stable-effective-reinforcement-learning-mixture/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/)

## References

22 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jacob Andreas. 2022. [Language models as agent models](https://md.research.elamur.ai/free/en/papers/andreas-2022-language-models-as-agent-models/). In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5769–5779.
2. Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 2021. [8-bit optimizers via block-wise quantization](https://md.research.elamur.ai/free/en/papers/dettmers-2021-8-bit-optimizers-block-wise/). arXiv preprint arXiv:2110.02861.
3. Noé Durandard, Saurabh Dhawan, and Thierry Poibeau. 2025. Llms stick to the point, humans to style: Semantic and stylistic alignment in human and llm communication. In Proceedings of the 26th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 206–213. [find](https://scholar.google.com/scholar?q=No%C3%A9+Durandard%2C+Saurabh+Dhawan%2C+and+Thierry+Poibeau%2E+2025%2E+Llms+stick+to+the+point%2C+humans+to+style%3A+Semantic+and+stylistic+alignment+in+human+and+llm+communication%2E+In+Proceedings+of+the+26th+Annual+M)
4. Chen Gao, Xiaochong Lan, Zhihong Lu, Jinzhu Mao, Jinghua Piao, Huandong Wang, Depeng Jin, and Yong Li. 2023. S3: Social-network simulation system with large language model-empowered agents. arXiv preprint arXiv:2307.14984. — [arXiv:2307.14984](https://arxiv.org/abs/2307.14984) [find](https://scholar.google.com/scholar?q=Chen+Gao%2C+Xiaochong+Lan%2C+Zhihong+Lu%2C+Jinzhu+Mao%2C+Jinghua+Piao%2C+Huandong+Wang%2C+Depeng+Jin%2C+and+Yong+Li%2E+2023%2E+S3%3A+Social%2Dnetwork+simulation+system+with+large+language+model%2Dempowered+agents%2E+arXiv+prep)
5. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
6. Tiancheng Hu, Joachim Baumann, Lorenzo Lupo, Nigel Collier, Dirk Hovy, and Paul Röttger. 2025. Simbench: Benchmarking the ability of large language models to simulate human behaviors. arXiv preprint arXiv:2510.17516. — [arXiv:2510.17516](https://arxiv.org/abs/2510.17516) [find](https://scholar.google.com/scholar?q=Tiancheng+Hu%2C+Joachim+Baumann%2C+Lorenzo+Lupo%2C+Nigel+Collier%2C+Dirk+Hovy%2C+and+Paul+R%C3%B6ttger%2E+2025%2E+Simbench%3A+Benchmarking+the+ability+of+large+language+models+to+simulate+human+behaviors%2E+arXiv+preprint+a)
7. Akbir Khan, John Hughes, Dan Valentine, Laura Ruis, Kshitij Sachan, Ansh Radhakrishnan, Edward Grefenstette, Samuel R Bowman, Tim Rocktäschel, and Ethan Perez. 2024. Debating with more persuasive llms leads to more truthful answers. In Proceedings of the 41st International Conference on Machine Lear… [find](https://scholar.google.com/scholar?q=Akbir+Khan%2C+John+Hughes%2C+Dan+Valentine%2C+Laura+Ruis%2C+Kshitij+Sachan%2C+Ansh+Radhakrishnan%2C+Edward+Grefenstette%2C+Samuel+R+Bowman%2C+Tim+Rockt%C3%A4schel%2C+and+Ethan+Perez%2E+2024%2E+Debating+with+more+persuasive+llms)
8. Maik Larooij and Petter Törnberg. 2025. Validation is the central challenge for generative social simulation: a critical review of llms in agent-based modeling. Artificial Intelligence Review, 59(1):15. [find](https://scholar.google.com/scholar?q=Maik+Larooij+and+Petter+T%C3%B6rnberg%2E+2025%2E+Validation+is+the+central+challenge+for+generative+social+simulation%3A+a+critical+review+of+llms+in+agent%2Dbased+modeling%2E+Artificial+Intelligence+Review%2C+59%281%29%3A1)
9. Chin-Yew Lin. 2004. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text summarization branches out, pages 74–81.
10. Yuhan Liu, Xiuying Chen, Xiaoqing Zhang, Xing Gao, Ji Zhang, and Rui Yan. 2024. From skepticism to acceptance: simulating the attitude dynamics toward fake news. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 7886–7894. [find](https://scholar.google.com/scholar?q=Yuhan+Liu%2C+Xiuying+Chen%2C+Xiaoqing+Zhang%2C+Xing+Gao%2C+Ji+Zhang%2C+and+Rui+Yan%2E+2024%2E+From+skepticism+to+acceptance%3A+simulating+the+attitude+dynamics+toward+fake+news%2E+In+Proceedings+of+the+Thirty%2DThird+Int)
11. Charles M Macal and Michael J North. 2009. Agent-based modeling and simulation. In Proceedings of the 2009 winter simulation conference (WSC), pages 86–98. IEEE. [find](https://scholar.google.com/scholar?q=Charles+M+Macal+and+Michael+J+North%2E+2009%2E+Agent%2Dbased+modeling+and+simulation%2E+In+Proceedings+of+the+2009+winter+simulation+conference+%28WSC%29%2C+pages+86%E2%80%9398%2E+IEEE%2E)
12. Simon Münker, Nils Schwager, and Achim Rettinger. 2025. [Don’t trust generative agents to mimic communication on social networks unless you benchmarked their empirical realism](https://md.research.elamur.ai/free/en/papers/munker-2026-don-t-trust-generative-agents/). arXiv preprint arXiv:2506.21974.
13. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
14. Fred Philippy, Siwen Guo, Jacques Klein, and Tegawende Bissyande. 2025. [LuxEmbedder: A cross-lingual approach to enhanced Luxembourgish sentence embeddings](https://md.research.elamur.ai/free/en/papers/philippy-2025-luxembedder-cross-lingual-approach-enhanced/). In Proceedings of the 31st International Conference on Computational Linguistics, pages 11369–11379, Abu Dhabi, UAE. Association for Computatio…
15. Qwen Team. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
16. Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schütze, and Dirk Hovy. 2024. [Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2024-political-compass-or-spinning-arrow/). In Proceedings of the 62nd Annual Meeting of the Association…
17. Henrique Schechter Vera, Sahil Dua, Biao Zhang, Daniel Salz, Ryan Mullins, Sindhu Raghuram Panyam, Sara Smoot, Iftekhar Naim, Joe Zou, Feiyang Chen, and 1 others. 2025. Embeddinggemma: Powerful and lightweight text representations. arXiv preprint arXiv:2509.20354. — [arXiv:2509.20354](https://arxiv.org/abs/2509.20354) [find](https://scholar.google.com/scholar?q=Henrique+Schechter+Vera%2C+Sahil+Dua%2C+Biao+Zhang%2C+Daniel+Salz%2C+Ryan+Mullins%2C+Sindhu+Raghuram+Panyam%2C+Sara+Smoot%2C+Iftekhar+Naim%2C+Joe+Zou%2C+Feiyang+Chen%2C+and+1+others%2E+2025%2E+Embeddinggemma%3A+Powerful+and+li)
18. Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. 2020. Trl: Transformer reinforcement learning. [https://github.com/huggingface/trl](https://github.com/huggingface/trl). [↗](https://github.com/huggingface/trl)
19. Yilei Wang, Jiabao Zhao, Deniz S Ones, Liang He, and Xin Xu. 2025. Evaluating the ability of large language models to emulate personality. Scientific Reports, 15(1):519. [find](https://scholar.google.com/scholar?q=Yilei+Wang%2C+Jiabao+Zhao%2C+Deniz+S+Ones%2C+Liang+He%2C+and+Xin+Xu%2E+2025%2E+Evaluating+the+ability+of+large+language+models+to+emulate+personality%2E+Scientific+Reports%2C+15%281%29%3A519%2E)
20. Yangbin Yu, Qin Zhang, Junyou Li, Qiang Fu, and Deheng Ye. 2024. Affordable generative agents. arXiv preprint arXiv:2402.02053. — [arXiv:2402.02053](https://arxiv.org/abs/2402.02053) [find](https://scholar.google.com/scholar?q=Yangbin+Yu%2C+Qin+Zhang%2C+Junyou+Li%2C+Qiang+Fu%2C+and+Deheng+Ye%2E+2024%2E+Affordable+generative+agents%2E+arXiv+preprint+arXiv%3A2402%2E02053%2E+%E2%80%94+%5BarXiv%3A2402%2E02053%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E02053%29)
21. Bowen Zhang, Yi Yang, Fuqiang Niu, Xianghua Fu, Genan Dai, and Hu Huang. 2025a. Spark: Simulating the co-evolution of stance and topic dynamics in online discourse with llm-based agents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 23072–23084. [find](https://scholar.google.com/scholar?q=Bowen+Zhang%2C+Yi+Yang%2C+Fuqiang+Niu%2C+Xianghua+Fu%2C+Genan+Dai%2C+and+Hu+Huang%2E+2025a%2E+Spark%3A+Simulating+the+co%2Devolution+of+stance+and+topic+dynamics+in+online+discourse+with+llm%2Dbased+agents%2E+In+Proceeding)
22. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025b. [Qwen3 embedding: Advancing text embedding and reranking through foundation models](https://md.research.elamur.ai/free/en/papers/zhang-2025-qwen3-embedding-advancing-text/). arXiv preprint arXiv:2506.05176.

*Updated 1 November 2026*