--- title: "Reading Between the Prompts: How Stereotypes Shape LLM's" description: "The authors test how much stereotypes drive these guesses during multi-turn chats. 4 references, cited by 8 papers here." authors: ["Vera Neplenbroek", "Arianna Bisazza", "Raquel Fernández"] year: 2025 venue: "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/2025.emnlp-main.1029/" doi: "10.18653/v1/2025.emnlp-main.1029" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/neplenbroek-2025-reading-between-prompts-how-stereotypes/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/neplenbroek-2025-reading-between-prompts-how-stereotypes/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/7/) / Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalization # Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalization **Authors:** [Vera Neplenbroek](https://md.research.elamur.ai/free/en/authors/vera-neplenbroek/), [Arianna Bisazza](https://md.research.elamur.ai/free/en/authors/arianna-bisazza/) and [Raquel Fernández](https://md.research.elamur.ai/free/en/authors/raquel-fernandez/) · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2025](https://md.research.elamur.ai/free/en/venues/emnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.emnlp-main.1029/) · [PDF](https://aclanthology.org/2025.emnlp-main.1029.pdf) · [DOI 10.18653/v1/2025.emnlp-main.1029](https://doi.org/10.18653/v1/2025.emnlp-main.1029) · [OpenAlex](https://openalex.org/W4416036094) **Citations:** 4 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/neplenbroek-2025-reading-between-prompts-how-stereotypes/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary LLMs often guess who they are talking to from indirect hints, a behavior called implicit personalization. This can lower answer quality for some groups. The authors test how much stereotypes drive these guesses during multi-turn chats. They built template-based synthetic English conversations that separately control two things: whether the user states their demographic identity, and whether they ask about items stereotypically linked to a group. These items cover food, drinks, hobbies and character traits. They examined three LLMs (OLMo, Gemma, Llama) in two ways. First, through model internals, using linear probes and surprisal. Second, through generated answers to direct and indirect questions about the user. All models kept explicitly stated demographics in their internal representations. Without such statements, they inferred identity from stereotypical cues, especially for gender and race. For gender, stereotypes could even override a user's explicit self-description in every model. Steering activations with the probe weights largely restored the stated identity. This matters because hidden, stereotype-based user modeling affects fairness and trust, and it calls for more transparency and control. ## Key points - The setup spans 3 LLMs and 13 demographic groups across 4 attributes: age, gender, race and SES. Conversations last 6 rounds and draw on 404 items. Stereotype associations come from social science literature and from BBQ (for character traits). - Some models avoid naming the user's demographics when asked directly. Internal signals (probes, surprisal) still show that the information is retained. - After probe-based steering toward the stated group, that group had the lowest surprisal in over 77% of conversations, and in 100% for many groups. Steering toward a neutral "no information" class worked poorly and sometimes made stereotyped mentions in indirect answers more frequent. - Steering barely changed MMLU and IFEval scores. The only significant drops (p \< 0.01) were OLMo's IFEval score and Llama's MMLU score. - Code and data: github.com/Veranep/implicit-personalization-stereotypes. Limitations include a U.S.-centered selection of groups and stereotypes, and no analysis of intersectional identities. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Amsterdam](https://md.research.elamur.ai/free/en/universities/university-of-amsterdam/) — University, Amsterdam, Netherlands · 144 works · 173,906 citations - [University of Groningen](https://md.research.elamur.ai/free/en/universities/university-of-groningen/) — University, Groningen, Netherlands · 70 works · 5,142 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works 🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works ## Cite Cite as: Vera Neplenbroek, Arianna Bisazza and Raquel Fernández. 2025. Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalization. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{neplenbroek2025reading, title = {{Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalization}}, author = {Vera Neplenbroek, Arianna Bisazza and Raquel Fernández}, booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing}, year = {2025}, doi = {10.18653/v1/2025.emnlp-main.1029}, url = {https://doi.org/10.18653/v1/2025.emnlp-main.1029} } ``` - [← previous QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions](https://md.research.elamur.ai/free/en/papers/wang-2025-qualispeech-speech-quality-assessment-dataset/) - [next → REARANK: Reasoning Re-ranking Agent via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/zhang-2025-rearank-reasoning-re-ranking-agent/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/7/) ## References 4 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. AI@Meta. 2024. Llama 3.1 model card. [find](https://scholar.google.com/scholar?q=AI%40Meta%2E+2024%2E+Llama+3%2E1+model+card%2E) 2. Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, and Wanli Ouyang. 2024. [MT-bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues](https://md.research.elamur.ai/free/en/papers/bai-2024-mt-bench-101-fine-grained/). In Proceedings of the 62nd Annual Meeting of the Associatio… 3. Jeesoo Bang, Hyungjong Noh, Yonghee Kim, and Gary Geunbae Lee. 2015. Example-based chat-oriented dialogue system with personalized long-term memory. In 2015 International Conference on Big Data and Smart Computing (BIGCOMP), pages 238–243. [find](https://scholar.google.com/scholar?q=Jeesoo+Bang%2C+Hyungjong+Noh%2C+Yonghee+Kim%2C+and+Gary+Geunbae+Lee%2E+2015%2E+Example%2Dbased+chat%2Doriented+dialogue+system+with+personalized+long%2Dterm+memory%2E+In+2015+International+Conference+on+Big+Data+and+Sm) 4. Yonatan Belinkov. 2022. [Probing classifiers: Promises, shortcomings, and advances](https://md.research.elamur.ai/free/en/papers/belinkov-2022-probing-classifiers-promises-shortcomings-advances/). Computational Linguistics, 48(1):207–219. \#\# Ethical Considerations *Updated 1 November 2026*