--- title: "Fairness Beyond Anonymization? Demographic Leakage in" description: "The paper asks whether résumés written by LLMs in German still reveal an applicant's gender or ethnicity after the documents… 4 references." authors: ["Charlotte Leininger", "Helena Veit", "Matthias A\\ssenmacher", "Andreas Bender"] year: 2026 venue: "arXiv:2609.22188" source: "https://arxiv.org/abs/2609.22188" doi: "10.48550/arxiv.2609.22188" arxiv: "2609.22188" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/leininger-2026-fairness-anonymization-demographic-leakage-german/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/leininger-2026-fairness-anonymization-demographic-leakage-german/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/) / Fairness Beyond Anonymization? Demographic Leakage in German LLM-Generated Resumes # Fairness Beyond Anonymization? Demographic Leakage in German LLM-Generated Resumes **Authors:** Charlotte Leininger, Helena Veit, [Matthias A\\ssenmacher](https://md.research.elamur.ai/free/en/authors/matthias-a-ssenmacher/) and Andreas Bender · **Year:** 2026 · **Venue:** arXiv:2609.22188 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2609.22188) · [PDF](https://arxiv.org/pdf/2609.22188) · [DOI 10.48550/arxiv.2609.22188](https://doi.org/10.48550/arxiv.2609.22188) · [arXiv 2609.22188](https://arxiv.org/abs/2609.22188) · [OpenAlex](https://openalex.org/W7213998949) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary The paper asks whether résumés written by LLMs in German still reveal an applicant's gender or ethnicity after the documents are anonymized. The authors built a two-stage audit. In the first stage, they took 30 real, anonymized job-matching questionnaire profiles from the German platform Chemistree. Each profile was combined with synthetic names that signal gender (male/female) and ethnicity (German/Turkish), while the qualifications stayed fixed. Résumés were then generated with GPT-4o-mini, Gemini 2.5 Flash-Lite and Qwen 3 at 4B, 8B and 14B. In the second stage, the résumés were anonymized and gender-neutralized, and TF-IDF features with logistic regression were used to try to recover the demographic attributes. Gender stayed reliably predictable for every model. The signal did not come from obviously gendered words. It came from preferences between near-equivalent neutral and marked terms. Ethnicity signals were weak. The work shows that anonymizing the final document may not be enough to make AI-assisted hiring fair, especially in grammatically gendered languages. ## Key points - Data source: Chemistree holds 5,932 profiles, of which 3,542 are complete. A random sample of 30 profiles was used, each paired with 40 full names. - Names were sampled as 10 per subgroup across gender × {German, Turkish}. First names came from birth-year-1985 frequency statistics, and every applicant was assigned age 40. Each male–female name pair shared a surname. - Gender classifiers succeeded despite neutralization (reported scores around 0.67–0.75). Ethnicity classifiers stayed close to chance, at roughly 0.50–0.58. - The key lexical cue was word choice. The neutral noun *Fachkraft* leaned toward female-named résumés, while the marked *Fachmann/-frau* leaned toward male-named ones. - Leakage did not grow steadily with Qwen 3 size. Accuracy rose with scale, but AUC peaked at 8B. A larger decoding top-k (64 vs 20) generally weakened gender prediction. ## Interesting ideas - The design creates counterfactual minimal pairs: only the name changes while the content stays fixed, so any output difference can be traced to that one variable. - The audit trains a deliberately simple probe on text that has already been "cleaned" to measure what the cleaning missed, turning residual predictability into a measurable leakage score. - Reading the probe's top features revealed that choices between near-synonymous variants, not explicit markers, carried the signal. - Using ten names per subgroup instead of one keeps quirks of a single name from being mistaken for a group effect. - The scale study reported both accuracy and AUC and found they disagreed, so "bigger model" did not mean a monotonic change. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Swapping character names or genders in fables and test texts would make a cheap negative control, since frames and answers that shift under the swap point to a hidden dependency. - **I2** (back): The same probe could check the text → graph → text compressor, testing whether regenerated text still carries source attributes that the meaning graph was not supposed to keep. - **I3** (world): Mapping such variant pairs to one ontology concept, with the surface choice kept as a separate style feature, would keep meaning-level reasoning from silently picking up register or demographic cues. - **I4** (answer): Test suites for tales and word problems could vary entity names across several samples, so that a reader's success is not tied to one memorized name. - **I5** (learn): When comparing LLM teacher tiers, Mova should track more than one metric and avoid assuming the largest teacher gives the cleanest silver data. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Ludwig-Maximilians-Universität München](https://md.research.elamur.ai/free/en/universities/ludwig-maximilians-universitat-munchen/) — University, Munich, Germany · 107 works · 56,478 citations - [Munich Center for Machine Learning](https://md.research.elamur.ai/free/en/universities/munich-center-for-machine-learning/) — University, Munich, Germany · 57 works · 1,094 citations 📚 Shelves (5): [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works · [💼 Business, Management and Accounting](https://md.research.elamur.ai/free/en/shelves/business-management-and-accounting/) · 32 works · [💼 Organizational Behavior and Human Resource Management](https://md.research.elamur.ai/free/en/shelves/business-management-and-accounting/organizational-behavior-and-human-resource-management/) · 12 works 🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works ## Cite Cite as: Charlotte Leininger, Helena Veit, Matthias A\\ssenmacher and Andreas Bender. 2026. Fairness Beyond Anonymization? Demographic Leakage in German LLM-Generated Resumes. arXiv:2609.22188. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{leininger2026fairness, title = {{Fairness Beyond Anonymization? Demographic Leakage in German LLM-Generated Resumes}}, author = {Charlotte Leininger, Helena Veit, Matthias Assenmacher and Andreas Bender}, year = {2026}, doi = {10.48550/arxiv.2609.22188}, eprint = {2609.22188}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2609.22188} } ``` - [← previous FAIRGAMER: Evaluating Social Biases in LLM-Based Video Game NPCs](https://md.research.elamur.ai/free/en/papers/shi-2026-fairgamer-evaluating-social-biases-llm/) - [next → Faithful Medical Dialogue Generation Using Homo-Heterogeneous Exemplar-based In-Context Knowledge Grounding](https://md.research.elamur.ai/free/en/papers/priya-2026-faithful-medical-dialogue-generation-homo/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/) ## References 4 references, 0 of them with a card of ours. ↗ leads to the work. 1. 1. An, J., Huang, D., Lin, C., Tai, M.: Measuring gender and racial biases in large language models: Intersectional evidence from automated resume evaluation. PNAS Nexus p. pgaf089 (2025). https://doi.org/10.1093/pnasnexus/pgaf089 2. Armstrong, L., Liu, A., MacNeil, S., Metaxa, D.: The silicon ceiling: Auditing GPT’s race and gender biases in hiring. In: Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. Association for Computing Machinery (2024). https://doi.org/10.1145/3689904.3694699 3. Behaghel, L., Crépon, B., Le Barbanchon, T.: Unintended effects of anonymous résumés. American Economic Journal: Applied Economics 7(3), 1–27 (2015). https://doi.org/10.1257/app.201401 85 4. Dastin, J.: Amazon scraps secret AI recruiting tool that showed bias against women. In: Ethics of data and analytics. [↗](https://doi.org/10.1093/pnasnexus/pgaf089) 2. Auerbach Publications (2022), https://www.taylorfrancis.com/chapters/e dit/10.1201/9781003278290-44/amazon-scraps-secret-ai-recruiting-tool-showed-bias-w omen-jeffrey-dastin 5. Fabris, A., Baranowska, N., Dennis, M.J., Graus, D., Hacker, P., Saldivar, J., Zuiderveen Borgesius, F., Biega, A.J.: Fairness and bias in algorithmic hiring: A multidisciplinary survey. ACM Trans. Intell. Syst. Technol. 16(1) (2025). https://doi.org/10.1145/3696457 6. Friedman, J.H., Hastie, T., Tibshirani, R.: Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33(1), 1–22 (2010). https://doi.org/10.186 37/jss.v033.i01 7. Gaebler, J.D., Goel, S., Huq, A., Tambe, P.: Auditing the use of language models to guide hiring decisions (2024). https://doi.org/10.48550/arXiv.2404.03086 8. Gaucher, D., Friesen, J., Kay, A.C.: Evidence that gendered wording in job advertisements exists and sustains gender inequality. Journal of personality and social psychology 101(1), 109–128 (2011). https://doi.org/10.1037/a0022530 9. Hastie, T., Tibshirani, R., Friedman, J.: The elements of statistical learning. [↗](https://doi.org/10.1201/9781003278290-44/amazon-scraps-secret-ai-recruiting-tool-showed-bias-w) 3. Springer (2009). https://doi.org/10.1007/978-0-387-84858-7 10. Honnibal, M., Montani, I., Van Landeghem, S., Boyd, A.: spaCy: Industrial-strength natural language processing in python (2020). https://doi.org/10.5281/zenodo.1212303 11. Hu, J.: 99% of fortune 500 companies use applicant tracking systems. [↗](https://doi.org/10.1007/978-0-387-84858-7) 4. Jobscan Blog (2019), https://www.jobscan.co/blog/fortune-500-use-applicant-tracking-systems/ 12. Kaas, L., Manger, C.: Ethnic discrimination in Germany’s labour market: A field experiment. German Economic Review pp. 1–20 (2012). https://doi.org/10.1111/j.1468-0475.2011.00538 .x 13. Koh, N.H., Plata, J., Chai, J.: BAD: BiAs detection for large language models in the context of candidate screening (2023). https://doi.org/10.48550/arXiv.2305.10407 14. Köhler, E.A., Wiemann, D.: Field experimental evidence on hiring discrimination in the German apprenticeship market (2025). https://doi.org/10.2139/ssrn.5368943 15. LinkedIn Talent Solutions: The future of recruiting 2024. Tech. rep., LinkedIn Talent Solutions (2024), https://business.linkedin.com/content/dam/me/business/en-us/talent-solutions /resources/pdfs/future-of-recruiting-2024.pdf, accessed: 2026-05-31 16. Lo, F.P.W., Qiu, J., Wang, Z., Yu, H., Chen, Y., Zhang, G., Lo, B.: AI hiring with LLMs: A context-aware and explainable multi-agent framework for resume screening. In: 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). pp. 4184–4193 (2025). https://doi.org/10.1109/CVPRW67362.2025.00402 17. Motschenbacher, H.: Grammatical gender as a challenge for language policy: The (im)possibility of non-heteronormative language use in German versus English. Language Policy 13(3), 243–261 (2014). https://doi.org/10.1007/s10993-013-9300-0 18. Mujtaba, D.F., Mahapatra, N.R.: Fairness in AI-driven recruitment: Challenges, metrics, methods, and future directions (2025). https://doi.org/10.48550/arXiv.2405.19699 19. Parasurama, P., Sedoc, J.: Gendered language in resumes and its implications for algorithmic bias in hiring. In: Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP). pp. 74–74. Association for Computational Linguistics, Seattle, Washington (2022). https://doi.org/10.18653/v1/2022.gebnlp-1.7 20. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12, 2825–2830 (2011) 21. Qu, Q., Liu, Q.H., Gao, J., Huang, S., Feng, W., Yue, Z., Lu, X., Zhou, T., Lv, J.: Gender differences in resume language and gender gaps in salary expectations. Journal of The Royal Society Interface p. 20240784 (2025). https://doi.org/10.1098/rsif.2024.0784 22. ResumeBuilder.com: 3 in 4 job seekers who used chatgpt to write their resume got an interview (February 2023), https://www.resumebuilder.com/3-in-4-job-seekers-who-used-chatgpt-t o-write-their-resume-got-an-interview/, accessed: 2026-05-31 23. Rhea, A., Markey, K., D’Arinzo, L., Schellmann, H., Sloane, M., Squires, P., Stoyanovich, J.: Resume format, LinkedIn URLs and other unexpected influences on AI personality prediction in hiring: Results of an audit. In: Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. pp. 572–587. AIES ’22, Association for Computing Machinery (2022). https://doi.org/ 10.1145/3514094.3534189 24. Saldivar, J., Gatzioura, A., Castillo, C.: Synthetic CVs to build and test fairness-aware hiring tools (2025). https://doi.org/10.48550/arXiv.2508.21179 25. Tan, B.C.Z., Khoo, S., Doan, B.N., Liu, Z., Chen, N.F., Lee, R.K.W.: Small changes, big impact: Demographic bias in LLM-based hiring through subtle sociocultural markers in anonymised resumes (2026). https://doi.org/10.48550/arXiv.2603.05189 26. Tibshirani, R.: Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology 58(1), 267–288 (1996). https://doi.org/10.1111/j.25 17-6161.1996.tb02080.x 27. Wu, A.H.: Gendered language on the economics job market rumors forum. AEA Papers and Proceedings 108, 175–79 (2018). https://doi.org/10.1257/pandp.20181101 28. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al.: Qwen3 technical report (2025). https://doi.org/10.48550/arXiv.2505.09388 [↗](https://doi.org/10.1111/j.1468-0475.2011.00538) *Updated 30 October 2026*