---
title: "User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness"
description: "Privacy benchmarks such as ConfAIde and PrivacyLens test whether LLMs leak personal data while… 74 references, cited by 1 paper here."
authors: ["Xiaoyuan Wu", "Roshni Kaushik", "Wenkai Li", "Lujo Bauer", "Koichi Onoue"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1645/"
doi: "10.18653/v1/2026.acl-long.1645"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2026-user-perceptions-vs-proxy-llm/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2026-user-perceptions-vs-proxy-llm/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/) / User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios

# User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios

**Authors:** Xiaoyuan Wu, Roshni Kaushik, [Wenkai Li](https://md.research.elamur.ai/free/en/authors/wenkai-li/), Lujo Bauer and Koichi Onoue · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1645/) · [PDF](https://aclanthology.org/2026.acl-long.1645.pdf) · [DOI 10.18653/v1/2026.acl-long.1645](https://doi.org/10.18653/v1/2026.acl-long.1645) · [OpenAlex](https://openalex.org/W4416621899)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Privacy benchmarks such as ConfAIde and PrivacyLens test whether LLMs leak personal data while completing everyday tasks. They score responses with proxy LLM judges rather than real people. This paper asks whether those automated judgments reflect what users actually think about the helpfulness and privacy protection of such responses.

The authors sampled 90 PrivacyLens scenarios, such as email replies and meeting summaries. For each one, GPT-5 produced a single response using the PrivacyLens privacy-enhancing prompt. In a survey, 94 U.S.-based Prolific participants each rated five scenarios and explained their ratings. Five proxy LLMs answered the same survey, and the authors compared the two sets of judgments quantitatively and qualitatively.

People generally rated the responses as helpful and privacy-preserving, but they disagreed considerably with one another. The proxy LLMs were highly consistent but tracked human ratings only loosely, and they missed the spread of individual opinions.

The authors conclude that LLM-only evaluation may misjudge real user risk. They call for human-centered evaluation and for separating objective tasks from preference-sensitive ones. They also suggest personalized or persona-based proxies that reflect the diversity of user views.

## Key points
- Study setup: 90 randomly chosen PrivacyLens scenarios with GPT-5 responses, 94 participants rating five scenarios each, and five proxy LLM judges completing the same survey.
- Human raters agreed only weakly (Krippendorff's α = 0.36). When rerun five times, each proxy LLM gave almost the same judgments every time (α > 0.88), and agreement among the five proxies was fairly strong (α = 0.78).
- Per-scenario correlation between proxy judgments and average human ratings was weak to moderate (Spearman's ρ from 0.24 to 0.68).
- In the qualitative analysis, proxies sometimes overlooked subtle context and missed obviously private data such as credit card details. They also treated some information, like clients' first names, as harmless when participants considered it sensitive.
- Limitations include a single response-generating model (GPT-5) and English-speaking U.S. participants only. Participants spent about 47 minutes on the survey and were paid $10.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 539 works · 179,533 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Xiaoyuan Wu, Roshni Kaushik, Wenkai Li, Lujo Bauer and Koichi Onoue. 2026. User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wu2026user,
  title = {{User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios}},
  author = {Xiaoyuan Wu, Roshni Kaushik, Wenkai Li, Lujo Bauer and Koichi Onoue},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1645},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1645}
}
```

- [← previous USB: A COMPREHENSIVE AND UNIFIED SAFETY EVALUATION BENCHMARK FOR MULTIMODAL LARGE LANGUAGE MODELS](https://md.research.elamur.ai/free/en/papers/zheng-2026-usb-comprehensive-unified-safety-evaluation/)
- [next → Utility-Oriented Visual Evidence Selection for Multimodal Retrieval-Augmented Generation](https://md.research.elamur.ai/free/en/papers/luo-2026-utility-oriented-visual-evidence-selection/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/)

## References

74 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Noura Abdi, Xiao Zhan, Kopo M. Ramokapane, and Jose Such. 2021. Privacy norms for smart home personal assistants. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–14. ACM. [find](https://scholar.google.com/scholar?q=Noura+Abdi%2C+Xiao+Zhan%2C+Kopo+M%2E+Ramokapane%2C+and+Jose+Such%2E+2021%2E+Privacy+norms+for+smart+home+personal+assistants%2E+In+Proceedings+of+the+2021+CHI+Conference+on+Human+Factors+in+Computing+Systems%2C+pages)
2. Alessandro Acquisti, Laura Brandimarte, and George Loewenstein. 2015. Privacy and human behavior in the age of information. Science, 347(6221):509–514. Publisher: American Association for the Advancement of Science. [find](https://scholar.google.com/scholar?q=Alessandro+Acquisti%2C+Laura+Brandimarte%2C+and+George+Loewenstein%2E+2015%2E+Privacy+and+human+behavior+in+the+age+of+information%2E+Science%2C+347%286221%29%3A509%E2%80%93514%2E+Publisher%3A+American+Association+for+the+Advancem)
3. Haldun Akoglu. 2018. User’s guide to correlation coefficients. Turkish Journal of Emergency Medicine, 18(3):91–93. [find](https://scholar.google.com/scholar?q=Haldun+Akoglu%2E+2018%2E+User%E2%80%99s+guide+to+correlation+coefficients%2E+Turkish+Journal+of+Emergency+Medicine%2C+18%283%29%3A91%E2%80%9393%2E)
4. Sumit Asthana, Sagi Hilleli, Pengcheng He, and Aaron Halfaker. 2025. Summaries, highlights, and action items: Design, implementation and evaluation of an LLM-powered meeting recap system. In Proceedings of the ACM on Human-Computer Interaction, volume 9, pages 1–29, Yokohama Japan. ACM. [find](https://scholar.google.com/scholar?q=Sumit+Asthana%2C+Sagi+Hilleli%2C+Pengcheng+He%2C+and+Aaron+Halfaker%2E+2025%2E+Summaries%2C+highlights%2C+and+action+items%3A+Design%2C+implementation+and+evaluation+of+an+LLM%2Dpowered+meeting+recap+system%2E+In+Proceedin)
5. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, and 32 others. 20… — [Constitutional AI: Harmlessness from AI Feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
6. Valerio Basile, Michael Fell, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio, and Alexandra Uma. 2021. [We need to consider disagreement in evaluation](https://md.research.elamur.ai/free/en/papers/basile-2021-we-need-consider-disagreement-evaluation/). In Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future, pages 15–21, Online. Association for Computa…
7. Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tramèr. 2022. What does it mean for a language model to preserve privacy? In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, pages 2280–2292, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Hannah+Brown%2C+Katherine+Lee%2C+Fatemehsadat+Mireshghallah%2C+Reza+Shokri%2C+and+Florian+Tram%C3%A8r%2E+2022%2E+What+does+it+mean+for+a+language+model+to+preserve+privacy%3F+In+Proceedings+of+the+2022+ACM+Conference+on)
8. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini%2C+Daphne+Ippolito%2C+Matthew+Jagielski%2C+Katherine+Lee%2C+Florian+Tramer%2C+and+Chiyuan+Zhang%2E+2023%2E+Quantifying+memorization+across+neural+language+models%2E+In+The+Eleventh+International+Conf)
9. Santiago Castro. 2017. Fast Krippendorff: Fast computation of Krippendorff’s alpha agreement measure. [https://github.com/pln-fing-udelar/fast-k](https://github.com/pln-fing-udelar/fast-k) rippendorff. [↗](https://github.com/pln-fing-udelar/fast-k)
10. Chaoran Chen, Zhiping Zhang, Bingcan Guo, Shang Ma, Ibrahim Khalilov, Simret A. Gebreegziabher, Yanfang Ye, Ziang Xiao, Yaxing Yao, Tianshi Li, and Toby Jia-Jun Li. 2025a. The obvious invisible threat: LLM-powered GUI agents’ vulnerability to fine-print injections. arXiv preprint. ArXiv:2504.11281 [… — [arXiv:2504.11281](https://arxiv.org/abs/2504.11281) [find](https://scholar.google.com/scholar?q=Chaoran+Chen%2C+Zhiping+Zhang%2C+Bingcan+Guo%2C+Shang+Ma%2C+Ibrahim+Khalilov%2C+Simret+A%2E+Gebreegziabher%2C+Yanfang+Ye%2C+Ziang+Xiao%2C+Yaxing+Yao%2C+Tianshi+Li%2C+and+Toby+Jia%2DJun+Li%2E+2025a%2E+The+obvious+invisible+threat)
11. Chaoran Chen, Daodao Zhou, Yanfang Ye, Toby Jia-Jun Li, and Yaxing Yao. 2025b. CLEAR: Towards contextual LLM-empowered privacy policy analysis and risk generation for large language model applications. In Proceedings of the 30th International Conference on Intelligent User Interfaces, IUI ’25, pages… [find](https://scholar.google.com/scholar?q=Chaoran+Chen%2C+Daodao+Zhou%2C+Yanfang+Ye%2C+Toby+Jia%2DJun+Li%2C+and+Yaxing+Yao%2E+2025b%2E+CLEAR%3A+Towards+contextual+LLM%2Dempowered+privacy+policy+analysis+and+risk+generation+for+large+language+model+applications)
12. Mia Xu Chen, Benjamin N. Lee, Gagan Bansal, Yuan Cao, Shuyuan Zhang, Justin Lu, Jackie Tsay, Yinan Wang, Andrew M. Dai, Zhifeng Chen, Timothy Sohn, and Yonghui Wu. 2019. Gmail smart compose: Real-time assisted writing. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Disco… [find](https://scholar.google.com/scholar?q=Mia+Xu+Chen%2C+Benjamin+N%2E+Lee%2C+Gagan+Bansal%2C+Yuan+Cao%2C+Shuyuan+Zhang%2C+Justin+Lu%2C+Jackie+Tsay%2C+Yinan+Wang%2C+Andrew+M%2E+Dai%2C+Zhifeng+Chen%2C+Timothy+Sohn%2C+and+Yonghui+Wu%2E+2019%2E+Gmail+smart+compose%3A+Real%2Dtime)
13. Yubin Choi, Assem Zhunis, Wenchao Dong, Joseph Seering, Sangchul Park, Meeyoung Cha, and Hyojin Chin. 2025. Privacy disclosure to large language models: A large-scale study on awareness, benefits, and concerns in health contexts across three countries. Computers in Human Behavior Reports, 20:100841. [find](https://scholar.google.com/scholar?q=Yubin+Choi%2C+Assem+Zhunis%2C+Wenchao+Dong%2C+Joseph+Seering%2C+Sangchul+Park%2C+Meeyoung+Cha%2C+and+Hyojin+Chin%2E+2025%2E+Privacy+disclosure+to+large+language+models%3A+A+large%2Dscale+study+on+awareness%2C+benefits%2C+and)
14. Samuel Rhys Cox, Rune Møberg Jacobsen, and Niels van Berkel. 2025. The impact of a chatbot’s ephemerality-framing on self-disclosure perceptions. [find](https://scholar.google.com/scholar?q=Samuel+Rhys+Cox%2C+Rune+M%C3%B8berg+Jacobsen%2C+and+Niels+van+Berkel%2E+2025%2E+The+impact+of+a+chatbot%E2%80%99s+ephemerality%2Dframing+on+self%2Ddisclosure+perceptions%2E)
15. In Proceedings of the 7th ACM Conference on Conversational User Interfaces, CUI ’25, pages 1–17, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+7th+ACM+Conference+on+Conversational+User+Interfaces%2C+CUI+%E2%80%9925%2C+pages+1%E2%80%9317%2C+New+York%2C+NY%2C+USA%2E+ACM%2E)
16. Lorrie Faith Cranor. 2008. A framework for reasoning about the human in the loop. In Proceedings of the 1st Conference on Usability, Psychology, and Security, UPSEC’08, pages 1–15. USENIX Association. [find](https://scholar.google.com/scholar?q=Lorrie+Faith+Cranor%2E+2008%2E+A+framework+for+reasoning+about+the+human+in+the+loop%2E+In+Proceedings+of+the+1st+Conference+on+Usability%2C+Psychology%2C+and+Security%2C+UPSEC%E2%80%9908%2C+pages+1%E2%80%9315%2E+USENIX+Association%2E)
17. Alexander R. Fabbri, Wojciech Kryściński, Bryan Mc-Cann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2021. SummEval: Re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics, 9:391–409. [find](https://scholar.google.com/scholar?q=Alexander+R%2E+Fabbri%2C+Wojciech+Krys%CC%81cin%CC%81ski%2C+Bryan+Mc%2DCann%2C+Caiming+Xiong%2C+Richard+Socher%2C+and+Dragomir+Radev%2E+2021%2E+SummEval%3A+Re%2Devaluating+summarization+evaluation%2E+Transactions+of+the+Association+fo)
18. Alisa Frik, Xiao Zhan, Noura Abdi, and Julia Bernd. 2025. Who cares? Contextual privacy judgments from owner and bystander perspectives in different smart home situations. Proceedings on Privacy Enhancing Technologies. [find](https://scholar.google.com/scholar?q=Alisa+Frik%2C+Xiao+Zhan%2C+Noura+Abdi%2C+and+Julia+Bernd%2E+2025%2E+Who+cares%3F+Contextual+privacy+judgments+from+owner+and+bystander+perspectives+in+different+smart+home+situations%2E+Proceedings+on+Privacy+Enhan)
19. Hao Fu, Zizhan Zheng, Sencun Zhu, and Prasant Mohapatra. 2019. Keeping context in mind: Automating mobile app access control with user interface inspection. In IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pages 2089–2097. IEEE. [find](https://scholar.google.com/scholar?q=Hao+Fu%2C+Zizhan+Zheng%2C+Sencun+Zhu%2C+and+Prasant+Mohapatra%2E+2019%2E+Keeping+context+in+mind%3A+Automating+mobile+app+access+control+with+user+interface+inspection%2E+In+IEEE+INFOCOM+2019+%2D+IEEE+Conference+on+C)
20. Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, and 21 ot… — [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/guha-2023-legalbench-collaboratively-built-benchmark-measuring/)
21. Franziska Herbert, Collins W. Munyendo, Jonas Hielscher, Steffen Becker, and Yixin Zou. 2025. Digital security perceptions and practices around the world: A WEIRD versus non-weird comparison. In 34th USENIX Security Symposium, USENIX Security 2025, Seattle, WA, USA, August 13-15, 2025, pages 1455–14… [find](https://scholar.google.com/scholar?q=Franziska+Herbert%2C+Collins+W%2E+Munyendo%2C+Jonas+Hielscher%2C+Steffen+Becker%2C+and+Yixin+Zou%2E+2025%2E+Digital+security+perceptions+and+practices+around+the+world%3A+A+WEIRD+versus+non%2Dweird+comparison%2E+In+34th+)
22. Basra Jehangir, Saravanan Radhakrishnan, and Rahul Agarwal. 2023. A survey on Named Entity Recognition – datasets, tools, and methodologies. Natural Language Processing Journal, 3:100017. [find](https://scholar.google.com/scholar?q=Basra+Jehangir%2C+Saravanan+Radhakrishnan%2C+and+Rahul+Agarwal%2E+2023%2E+A+survey+on+Named+Entity+Recognition+%E2%80%93+datasets%2C+tools%2C+and+methodologies%2E+Natural+Language+Processing+Journal%2C+3%3A100017%2E)
23. Gurusha Juneja, Jayanth Pasupulati, Alon Albalak, Wenyue Hua, and William Yang Wang. 2025. MAG-PIE: A benchmark for multi-AGent contextual privacy evaluation. In Socially Responsible and Trustworthy Foundation Models at NeurIPS 2025. [find](https://scholar.google.com/scholar?q=Gurusha+Juneja%2C+Jayanth+Pasupulati%2C+Alon+Albalak%2C+Wenyue+Hua%2C+and+William+Yang+Wang%2E+2025%2E+MAG%2DPIE%3A+A+benchmark+for+multi%2DAGent+contextual+privacy+evaluation%2E+In+Socially+Responsible+and+Trustworthy+F)
24. Ruchira Dhar, Daniel Hershcovich, and Anders Søgaard. 2025. Evaluating multimodal language models as visual assistants for visually impaired users. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25949–25982, Vienna, Austria.… [find](https://scholar.google.com/scholar?q=Ruchira+Dhar%2C+Daniel+Hershcovich%2C+and+Anders+S%C3%B8gaard%2E+2025%2E+Evaluating+multimodal+language+models+as+visual+assistants+for+visually+impaired+users%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Ass)
25. Klaus Krippendorff. 2011. Computing Krippendorff’s Alpha-Reliability. [https://repository.upenn.edu/handle/20.500.14332/2089](https://repository.upenn.edu/handle/20.500.14332/2089). Accessed 2025-10-08. [↗](https://repository.upenn.edu/handle/20.500.14332/2089)
26. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Lorenz+Kuhn%2C+Yarin+Gal%2C+and+Sebastian+Farquhar%2E+2023%2E+Semantic+uncertainty%3A+Linguistic+invariances+for+uncertainty+estimation+in+natural+language+generation%2E+In+The+Eleventh+International+Conference+o)
27. Josephine Lau, Benjamin Zimmerman, and Florian Schaub. 2018. Alexa, are you listening?: Privacy perceptions, concerns and privacy-seeking behaviors with smart speakers. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW):1–31. [find](https://scholar.google.com/scholar?q=Josephine+Lau%2C+Benjamin+Zimmerman%2C+and+Florian+Schaub%2E+2018%2E+Alexa%2C+are+you+listening%3F%3A+Privacy+perceptions%2C+concerns+and+privacy%2Dseeking+behaviors+with+smart+speakers%2E+Proceedings+of+the+ACM+on+Human)
28. Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, and Tat-Seng Chua. 2025a. [Hello again! LLM-powered personalized agent for long-term dialogue](https://md.research.elamur.ai/free/en/papers/li-2025-hello-again-llm-powered-personalized/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technolo…
29. Haoran Li, Wenbin Hu, Huihao Jing, Yulin Chen, Qi Hu, Sirui Han, Tianshu Chu, Peizhao Hu, and Yangqiu Song. 2025b. [PrivaCI-bench: Evaluating privacy with contextual integrity and legal compliance](https://md.research.elamur.ai/free/en/papers/li-2025-privaci-bench-evaluating-privacy-contextual/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: L…
30. Tianshi Li, Sauvik Das, Hao-Ping Lee, Dakuo Wang, Bingsheng Yao, and Zhiping Zhang. 2024. Human-centered privacy research in the age of large language models. arXiv preprint. ArXiv:2402.01994 [cs]. — [arXiv:2402.01994](https://arxiv.org/abs/2402.01994) [find](https://scholar.google.com/scholar?q=Tianshi+Li%2C+Sauvik+Das%2C+Hao%2DPing+Lee%2C+Dakuo+Wang%2C+Bingsheng+Yao%2C+and+Zhiping+Zhang%2E+2024%2E+Human%2Dcentered+privacy+research+in+the+age+of+large+language+models%2E+arXiv+preprint%2E+ArXiv%3A2402%2E01994+%5Bcs%5D%2E+%E2%80%94+)
31. Weijiang Li, Yinmeng Lai, Sandeep Soni, and Koustuv Saha. 2025c. Emails by LLMs: A comparison of language in AI-generated and human-written emails. In Proceedings of the 17th ACM Web Science Conference 2025, Websci ’25, pages 391–403. ACM. [find](https://scholar.google.com/scholar?q=Weijiang+Li%2C+Yinmeng+Lai%2C+Sandeep+Soni%2C+and+Koustuv+Saha%2E+2025c%2E+Emails+by+LLMs%3A+A+comparison+of+language+in+AI%2Dgenerated+and+human%2Dwritten+emails%2E+In+Proceedings+of+the+17th+ACM+Web+Science+Conferenc)
32. David Jurgens, Agrima Seth, Jackson Sargent, Athena Aghighi, and Michael Geraci. 2023. Your spouse needs professional help: Determining the contextual appropriateness of messages through modeling social relationships. arXiv preprint arXiv:2307.02763. [find](https://scholar.google.com/scholar?q=David+Jurgens%2C+Agrima+Seth%2C+Jackson+Sargent%2C+Athena+Aghighi%2C+and+Michael+Geraci%2E+2023%2E+Your+spouse+needs+professional+help%3A+Determining+the+contextual+appropriateness+of+messages+through+modeling+soci)
33. Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona T. Diab, and Maarten Sap. 2025d. [BIG5-CHAT: Shaping LLM personalities through training on human-grounded data](https://md.research.elamur.ai/free/en/papers/li-2025-big5-chat-shaping-llm-personalities/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 20434– 20471, Vie…
34. Antonia Karamolegkou, Malvina Nikandrou, Georgios Pantazopoulos, Danae Sanchez Villegas, Phillip Rust, Wenkai Li, Liwen Sun, Zhenxiang Guan, Xuhui Zhou, and Maarten Sap. 2025e. 1-2-3 check: Enhancing contextual privacy in LLM via multi-agent reasoning. In Proceedings of the The First Workshop on LLM… [find](https://scholar.google.com/scholar?q=Antonia+Karamolegkou%2C+Malvina+Nikandrou%2C+Georgios+Pantazopoulos%2C+Danae+Sanchez+Villegas%2C+Phillip+Rust%2C+Wenkai+Li%2C+Liwen+Sun%2C+Zhenxiang+Guan%2C+Xuhui+Zhou%2C+and+Maarten+Sap%2E+2025e%2E+1%2D2%2D3+check%3A+Enhancing+)
35. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew A. Hudson, and 31… [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Rishi+Bommasani%2C+Tony+Lee%2C+Dimitris+Tsipras%2C+Dilara+Soylu%2C+Michihiro+Yasunaga%2C+Yian+Zhang%2C+Deepak+Narayanan%2C+Yuhuai+Wu%2C+Ananya+Kumar%2C+Benjamin+Newman%2C+Binhang+Yuan%2C+Bobby+Yan%2C+Ce+Zhang%2C+C)
36. Weiran Lin, Anna Gerchanovsky, Omer Akgul, Lujo Bauer, Matt Fredrikson, and Zifan Wang. 2025. LLM Whisperer: An Inconspicuous Attack to Bias LLM Responses. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI ’25, pages 1–24, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Weiran+Lin%2C+Anna+Gerchanovsky%2C+Omer+Akgul%2C+Lujo+Bauer%2C+Matt+Fredrikson%2C+and+Zifan+Wang%2E+2025%2E+LLM+Whisperer%3A+An+Inconspicuous+Attack+to+Bias+LLM+Responses%2E+In+Proceedings+of+the+2025+CHI+Conference+on)
37. Bin Liu, Mads Schaarup Andersen, Florian Schaub, Hazim Almuhimedi, Shikun (Aerin) Zhang, Norman Sadeh, Yuvraj Agarwal, and Alessandro Acquisti. 2016. Follow my recommendations: A personalized privacy assistant for mobile app permissions. In Twelfth Symposium on Usable Privacy and Security (SOUPS 201… [find](https://scholar.google.com/scholar?q=Bin+Liu%2C+Mads+Schaarup+Andersen%2C+Florian+Schaub%2C+Hazim+Almuhimedi%2C+Shikun+%28Aerin%29+Zhang%2C+Norman+Sadeh%2C+Yuvraj+Agarwal%2C+and+Alessandro+Acquisti%2E+2016%2E+Follow+my+recommendations%3A+A+personalized+privacy+)
38. Kirsten Martin and Helen Nissenbaum. 2016. Measuring privacy: An empirical test using context to expose confounding variables. Colum. Sci. & Tech. L. Rev., 18:176. [find](https://scholar.google.com/scholar?q=Kirsten+Martin+and+Helen+Nissenbaum%2E+2016%2E+Measuring+privacy%3A+An+empirical+test+using+context+to+expose+confounding+variables%2E+Colum%2E+Sci%2E+%26+Tech%2E+L%2E+Rev%2E%2C+18%3A176%2E)
39. Giacomo Marzi, Marco Balzano, and Davide Marchiori. 2024. K-Alpha Calculator–Krippendorff’s Alpha Calculator: A user-friendly tool for computing Krippendorff’s Alpha inter-rater reliability coefficient. MethodsX, 12:102545. [find](https://scholar.google.com/scholar?q=Giacomo+Marzi%2C+Marco+Balzano%2C+and+Davide+Marchiori%2E+2024%2E+K%2DAlpha+Calculator%E2%80%93Krippendorff%E2%80%99s+Alpha+Calculator%3A+A+user%2Dfriendly+tool+for+computing+Krippendorff%E2%80%99s+Alpha+inter%2Drater+reliability+coefficien)
40. Antonio Mastropaolo, Luca Pascarella, Emanuela Guglielmi, Matteo Ciniselli, Simone Scalabrino, Rocco Oliveto, and Gabriele Bavota. 2023. On the robustness of code generation techniques: An empirical study on GitHub copilot. In Proceedings of the 45th International Conference on Software Engineering,… [find](https://scholar.google.com/scholar?q=Antonio+Mastropaolo%2C+Luca+Pascarella%2C+Emanuela+Guglielmi%2C+Matteo+Ciniselli%2C+Simone+Scalabrino%2C+Rocco+Oliveto%2C+and+Gabriele+Bavota%2E+2023%2E+On+the+robustness+of+code+generation+techniques%3A+An+empirical+s)
41. Niloofar Mireshghallah, Maria Antoniak, Yash More, Yejin Choi, and Golnoosh Farnadi. 2024a. Trust no bot: Discovering personal disclosures in human-LLM conversations in the wild. In First Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=Niloofar+Mireshghallah%2C+Maria+Antoniak%2C+Yash+More%2C+Yejin+Choi%2C+and+Golnoosh+Farnadi%2E+2024a%2E+Trust+no+bot%3A+Discovering+personal+disclosures+in+human%2DLLM+conversations+in+the+wild%2E+In+First+Conference+o)
42. Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, and Yejin Choi. 2024b. Can LLMs keep a secret? Testing privacy implications of language models via contextual integrity theory. In International Conference on Representation Learning, volume 2024, pages 1892–… [find](https://scholar.google.com/scholar?q=Niloofar+Mireshghallah%2C+Hyunwoo+Kim%2C+Xuhui+Zhou%2C+Yulia+Tsvetkov%2C+Maarten+Sap%2C+Reza+Shokri%2C+and+Yejin+Choi%2E+2024b%2E+Can+LLMs+keep+a+secret%3F+Testing+privacy+implications+of+language+models+via+contextual)
43. Yusuke Miura, Chi-Lan Yang, Masaki Kuribayashi, Keigo Matsumoto, Hideaki Kuzuoka, and Shigeo Morishima. 2025. Understanding and supporting formal email exchange by answering AI-generated questions. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–20, Yokohama… [find](https://scholar.google.com/scholar?q=Yusuke+Miura%2C+Chi%2DLan+Yang%2C+Masaki+Kuribayashi%2C+Keigo+Matsumoto%2C+Hideaki+Kuzuoka%2C+and+Shigeo+Morishima%2E+2025%2E+Understanding+and+supporting+formal+email+exchange+by+answering+AI%2Dgenerated+questions%2E+In)
44. Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. 2022. [Dealing with disagreements: Looking beyond the majority vote in subjective annotations](https://md.research.elamur.ai/free/en/papers/davani-2022-dealing-disagreements-looking-beyond-majority/). Transactions of the Association for Computational Linguistics, 10:92–110.
45. Collins W. Munyendo. 2026. Security and Privacy Beyond Western Peripheries: Practices, Challenges, and Opportunities in the Majority World. Ph.D. thesis, George Washington University. Copyright - Database copyright ProQuest LLC; ProQuest does not claim copyright in the individual underlying works; L… [find](https://scholar.google.com/scholar?q=Collins+W%2E+Munyendo%2E+2026%2E+Security+and+Privacy+Beyond+Western+Peripheries%3A+Practices%2C+Challenges%2C+and+Opportunities+in+the+Majority+World%2E+Ph%2ED%2E+thesis%2C+George+Washington+University%2E+Copyright+%2D+Data)
46. Sheza Naveed, Hamza Naveed, Mobin Javed, and Maryam Mustafa. 2022. “Ask this from the person who has private stuff”: Privacy perceptions, behaviours and beliefs beyond W.E.I.R.D. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Sheza+Naveed%2C+Hamza+Naveed%2C+Mobin+Javed%2C+and+Maryam+Mustafa%2E+2022%2E+%E2%80%9CAsk+this+from+the+person+who+has+private+stuff%E2%80%9D%3A+Privacy+perceptions%2C+behaviours+and+beliefs+beyond+W%2EE%2EI%2ER%2ED%2E+In+Proceedings+of+the)
47. Helen Nissenbaum. 2004. Privacy as contextual integrity. Wash. L. Rev., 79:119. [find](https://scholar.google.com/scholar?q=Helen+Nissenbaum%2E+2004%2E+Privacy+as+contextual+integrity%2E+Wash%2E+L%2E+Rev%2E%2C+79%3A119%2E)
48. Helen Nissenbaum. 2019. Contextual integrity up and down the data food chain. Theoretical inquiries in law, 20(1):221–256. [find](https://scholar.google.com/scholar?q=Helen+Nissenbaum%2E+2019%2E+Contextual+integrity+up+and+down+the+data+food+chain%2E+Theoretical+inquiries+in+law%2C+20%281%29%3A221%E2%80%93256%2E)
49. Stephen Meisenbacher, Alexandra Klymenko, and Florian Matthes. 2025. LLM-as-a-judge for privacy evaluation? Exploring the alignment of human and LLM perceptions of privacy in textual data. In Proceedings of the 2025 Workshop on Human-Centered AI Privacy and Security, pages 126–138. [find](https://scholar.google.com/scholar?q=Stephen+Meisenbacher%2C+Alexandra+Klymenko%2C+and+Florian+Matthes%2E+2025%2E+LLM%2Das%2Da%2Djudge+for+privacy+evaluation%3F+Exploring+the+alignment+of+human+and+LLM+perceptions+of+privacy+in+textual+data%2E+In+Proceedi)
50. Anna-Marie Ortloff, Matthias Fassl, Alexander Pon-ticello, Florin Martius, Anne Mertens, Katharina Krombholz, and Matthew Smith. 2023. Different researchers, different results? Analyzing the influence of researcher experience and data type during qualitative analysis of an interview and survey study… [find](https://scholar.google.com/scholar?q=Anna%2DMarie+Ortloff%2C+Matthias+Fassl%2C+Alexander+Pon%2Dticello%2C+Florin+Martius%2C+Anne+Mertens%2C+Katharina+Krombholz%2C+and+Matthew+Smith%2E+2023%2E+Different+researchers%2C+different+results%3F+Analyzing+the+influence)
51. Ethan Mendes, Yang Chen, Wei Xu, and Alan Ritter. 2023. Human-in-the-loop evaluation for early misinformation detection: A case study of COVID-19 treatments. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers), pages 15817–15835. [find](https://scholar.google.com/scholar?q=Ethan+Mendes%2C+Yang+Chen%2C+Wei+Xu%2C+and+Alan+Ritter%2E+2023%2E+Human%2Din%2Dthe%2Dloop+evaluation+for+early+misinformation+detection%3A+A+case+study+of+COVID%2D19+treatments%2E+In+Proceedings+of+the+61st+annual+meeting+)
52. Sriyash Poddar, Yanming Wan, Hamish Ivison, Abhishek Gupta, and Natasha Jaques. 2024. Personalizing reinforcement learning from human feedback with variational preference learning. In Advances in Neural Information Processing Systems, volume 37, pages 52516–52544. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Sriyash+Poddar%2C+Yanming+Wan%2C+Hamish+Ivison%2C+Abhishek+Gupta%2C+and+Natasha+Jaques%2E+2024%2E+Personalizing+reinforcement+learning+from+human+feedback+with+variational+preference+learning%2E+In+Advances+in+Neur)
53. Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. [Grad-cam: Visual explanations from deep networks via gradient-based localization](https://md.research.elamur.ai/free/en/papers/selvaraju-2016-grad-cam-visual-explanations-deep/). In Proceedings of the IEEE international conference on computer vision, pages 618–626.
54. Primal Wijesekera, Arjun Baokar, Ashkan Hosseini, Serge Egelman, David Wagner, and Konstantin Beznosov. 2015. Android permissions remystified: A field study on contextual integrity. In 24th USENIX Security Symposium (USENIX Security 15), pages 499–514. [find](https://scholar.google.com/scholar?q=Primal+Wijesekera%2C+Arjun+Baokar%2C+Ashkan+Hosseini%2C+Serge+Egelman%2C+David+Wagner%2C+and+Konstantin+Beznosov%2E+2015%2E+Android+permissions+remystified%3A+A+field+study+on+contextual+integrity%2E+In+24th+USENIX+Sec)
55. Yijia Shao, Tianshi Li, Weiyan Shi, Yanchen Liu, and Diyi Yang. 2024. PrivacyLens: Evaluating privacy norm awareness of language models in action. Advances in Neural Information Processing Systems, 37:89373–89407. [find](https://scholar.google.com/scholar?q=Yijia+Shao%2C+Tianshi+Li%2C+Weiyan+Shi%2C+Yanchen+Liu%2C+and+Diyi+Yang%2E+2024%2E+PrivacyLens%3A+Evaluating+privacy+norm+awareness+of+language+models+in+action%2E+Advances+in+Neural+Information+Processing+Systems%2C+37)
56. Primal Wijesekera, Arjun Baokar, Lynn Tsai, Joel Reardon, Serge Egelman, David Wagner, and Konstantin Beznosov. 2017. The feasibility of dynamically granted permissions: Aligning mobile privacy with user preferences. In 2017 IEEE Symposium on Security and Privacy (SP), pages 1077–1093. IEEE. [find](https://scholar.google.com/scholar?q=Primal+Wijesekera%2C+Arjun+Baokar%2C+Lynn+Tsai%2C+Joel+Reardon%2C+Serge+Egelman%2C+David+Wagner%2C+and+Konstantin+Beznosov%2E+2017%2E+The+feasibility+of+dynamically+granted+permissions%3A+Aligning+mobile+privacy+with+u)
57. Yan Shvartzshnaider, Noah Apthorpe, Nick Feamster, and Helen Nissenbaum. 2018. Analyzing privacy policies using contextual integrity annotations. arXiv preprint arXiv:1809.02236. — [arXiv:1809.02236](https://arxiv.org/abs/1809.02236) [find](https://scholar.google.com/scholar?q=Yan+Shvartzshnaider%2C+Noah+Apthorpe%2C+Nick+Feamster%2C+and+Helen+Nissenbaum%2E+2018%2E+Analyzing+privacy+policies+using+contextual+integrity+annotations%2E+arXiv+preprint+arXiv%3A1809%2E02236%2E+%E2%80%94+%5BarXiv%3A1809%2E02236%5D%28)
58. Yan Shvartzshnaider, Noah Apthorpe, Nick Feamster, and Helen Nissenbaum. 2019. Going against the (appropriate) flow: A contextual integrity approach to privacy policy analysis. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, volume 7, pages 162–170. [find](https://scholar.google.com/scholar?q=Yan+Shvartzshnaider%2C+Noah+Apthorpe%2C+Nick+Feamster%2C+and+Helen+Nissenbaum%2E+2019%2E+Going+against+the+%28appropriate%29+flow%3A+A+contextual+integrity+approach+to+privacy+policy+analysis%2E+In+Proceedings+of+the+A)
59. Tejpalsingh Siledar, Swaroop Nath, Sankara Muddu, Rupasai Rangaraju, Swaprava Nath, Pushpak Bhattacharyya, Suman Banerjee, Amey Patil, Sudhanshu Singh, Muthusamy Chelliah, and Nikesh Garera. 2024. One prompt to rule them all: LLMs for opinion summary evaluation. In Proceedings of the 62nd Annual Mee… [find](https://scholar.google.com/scholar?q=Tejpalsingh+Siledar%2C+Swaroop+Nath%2C+Sankara+Muddu%2C+Rupasai+Rangaraju%2C+Swaprava+Nath%2C+Pushpak+Bhattacharyya%2C+Suman+Banerjee%2C+Amey+Patil%2C+Sudhanshu+Singh%2C+Muthusamy+Chelliah%2C+and+Nikesh+Garera%2E+2024%2E+One)
60. Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Abubakr Babiker, Nathanael Schärli, Aakanksha Chowdhery, Philip Mansfield, Dina Demner-Fushm… — [Large Language Models Encode Clinical Knowledge](https://md.research.elamur.ai/free/en/papers/singhal-2022-large-language-models-encode-clinical/)
61. Alina Stöver, Sara Hahn, Felix Kretschmer, and Nina Gerber. 2023. Investigating how users imagine their personal privacy assistant. Proceedings on Privacy Enhancing Technologies, 2023(2):384–402. [find](https://scholar.google.com/scholar?q=Alina+St%C3%B6ver%2C+Sara+Hahn%2C+Felix+Kretschmer%2C+and+Nina+Gerber%2E+2023%2E+Investigating+how+users+imagine+their+personal+privacy+assistant%2E+Proceedings+on+Privacy+Enhancing+Technologies%2C+2023%282%29%3A384%E2%80%93402%2E)
62. Xiaoyuan Wu, Lydia Hu, Eric Zeng, Hana Habib, and Lujo Bauer. 2025a. Transparency or information overload? Evaluating users’ comprehension and perceptions of the iOS App Privacy Report. In Proceedings 2025 Network and Distributed System Security Symposium, San Diego, CA, USA. Internet Society. [find](https://scholar.google.com/scholar?q=Xiaoyuan+Wu%2C+Lydia+Hu%2C+Eric+Zeng%2C+Hana+Habib%2C+and+Lujo+Bauer%2E+2025a%2E+Transparency+or+information+overload%3F+Evaluating+users%E2%80%99+comprehension+and+perceptions+of+the+iOS+App+Privacy+Report%2E+In+Proceedings)
63. Xiaoyuan Wu, Weiran Lin, Omer Akgul, and Lujo Bauer. 2025b. Estimating LLM consistency: A user baseline vs surrogate metrics. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 30518–30532, Suzhou, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xiaoyuan+Wu%2C+Weiran+Lin%2C+Omer+Akgul%2C+and+Lujo+Bauer%2E+2025b%2E+Estimating+LLM+consistency%3A+A+user+baseline+vs+surrogate+metrics%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+Natural+Lang)
64. Yuankai Xue, Hanlin Chen, Gina R. Bai, Robert Tairas, and Yu Huang. 2024. Does ChatGPT help with introductory programming? An experiment of students using ChatGPT in CS1. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training, pages 3… [find](https://scholar.google.com/scholar?q=Yuankai+Xue%2C+Hanlin+Chen%2C+Gina+R%2E+Bai%2C+Robert+Tairas%2C+and+Yu+Huang%2E+2024%2E+Does+ChatGPT+help+with+introductory+programming%3F+An+experiment+of+students+using+ChatGPT+in+CS1%2E+In+Proceedings+of+the+46th+In)
65. Oleksandr Yermilov, Vipul Raheja, and Artem Chernodub. 2023. Privacy- and utility-preserving NLP with anonymized data: A case study of pseudonymization. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pages 232–241. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Oleksandr+Yermilov%2C+Vipul+Raheja%2C+and+Artem+Chernodub%2E+2023%2E+Privacy%2D+and+utility%2Dpreserving+NLP+with+anonymized+data%3A+A+case+study+of+pseudonymization%2E+In+Proceedings+of+the+3rd+Workshop+on+Trustwort)
66. Miao Yu, Fanci Meng, Xinyun Zhou, Shilong Wang, Junyuan Mao, Linsey Pan, Tianlong Chen, Kun Wang, Xinfeng Li, Yongfeng Zhang, Bo An, and Qingsong Wen. 2025. A survey on trustworthy LLM agents: Threats and countermeasures. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Da… [find](https://scholar.google.com/scholar?q=Miao+Yu%2C+Fanci+Meng%2C+Xinyun+Zhou%2C+Shilong+Wang%2C+Junyuan+Mao%2C+Linsey+Pan%2C+Tianlong+Chen%2C+Kun+Wang%2C+Xinfeng+Li%2C+Yongfeng+Zhang%2C+Bo+An%2C+and+Qingsong+Wen%2E+2025%2E+A+survey+on+trustworthy+LLM+agents%3A+Threats)
67. Jenny Tang, Eleanor Birrell, and Ada Lerner. 2022. Replication: How well do my results generalize now? The external validity of online privacy and security surveys. In Proceedings of the 18th USENIX Symposium on Usable Privacy and Security (SOUPS 2022), pages 367–385. USENIX Association. [find](https://scholar.google.com/scholar?q=Jenny+Tang%2C+Eleanor+Birrell%2C+and+Ada+Lerner%2E+2022%2E+Replication%3A+How+well+do+my+results+generalize+now%3F+The+external+validity+of+online+privacy+and+security+surveys%2E+In+Proceedings+of+the+18th+USENIX+S)
68. Hye Sun Yun and Timothy Bickmore. 2025. Framing health information: The impact of search methods and source types on user trust and satisfaction in the age of LLMs. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA ’25, pages 1–7, New York,… [find](https://scholar.google.com/scholar?q=Hye+Sun+Yun+and+Timothy+Bickmore%2E+2025%2E+Framing+health+information%3A+The+impact+of+search+methods+and+source+types+on+user+trust+and+satisfaction+in+the+age+of+LLMs%2E+In+Proceedings+of+the+Extended+Abst)
69. Shouju Wang, Fenglin Yu, Xirui Liu, Xiaoting Qin, Jue Zhang, Qingwei Lin, Dongmei Zhang, and Saravan Rajmohan. 2025. Privacy in action: Towards realistic privacy mitigation and evaluation for LLM-powered agents. Preprint, arxiv:2509.17488 [cs]. [find](https://scholar.google.com/scholar?q=Shouju+Wang%2C+Fenglin+Yu%2C+Xirui+Liu%2C+Xiaoting+Qin%2C+Jue+Zhang%2C+Qingwei+Lin%2C+Dongmei+Zhang%2C+and+Saravan+Rajmohan%2E+2025%2E+Privacy+in+action%3A+Towards+realistic+privacy+mitigation+and+evaluation+for+LLM%2Dpowe)
70. Zhiping Zhang, Bingcan Guo, and Tianshi Li. 2025. Privacy leakage overshadowed by views of AI: A study on human oversight of privacy in language model agent. arXiv preprint. ArXiv:2411.01344 [cs]. — [arXiv:2411.01344](https://arxiv.org/abs/2411.01344) [find](https://scholar.google.com/scholar?q=Zhiping+Zhang%2C+Bingcan+Guo%2C+and+Tianshi+Li%2E+2025%2E+Privacy+leakage+overshadowed+by+views+of+AI%3A+A+study+on+human+oversight+of+privacy+in+language+model+agent%2E+arXiv+preprint%2E+ArXiv%3A2411%2E01344+%5Bcs%5D%2E+%E2%80%94+%5B)
71. Zhiping Zhang, Michelle Jia, Hao-Ping (Hank) Lee, Bingsheng Yao, Sauvik Das, Ada Lerner, Dakuo Wang, and Tianshi Li. 2024. “It’s a fair game”, or is it? Examining how users navigate disclosure risks and benefits when using LLM-based conversational agents. In Proceedings of the 2024 CHI Conference on… [find](https://scholar.google.com/scholar?q=Zhiping+Zhang%2C+Michelle+Jia%2C+Hao%2DPing+%28Hank%29+Lee%2C+Bingsheng+Yao%2C+Sauvik+Das%2C+Ada+Lerner%2C+Dakuo+Wang%2C+and+Tianshi+Li%2E+2024%2E+%E2%80%9CIt%E2%80%99s+a+fair+game%E2%80%9D%2C+or+is+it%3F+Examining+how+users+navigate+disclosure+risks+a)
72. Siyan Zhao, Mingyi Hong, Yang Liu, Devamanyu Hazarika, and Kaixiang Lin. 2025. Do LLMs recognize your preferences? Evaluating personalized preference following in LLMs. arXiv preprint. ArXiv:2502.09597 [cs]. — [arXiv:2502.09597](https://arxiv.org/abs/2502.09597) [find](https://scholar.google.com/scholar?q=Siyan+Zhao%2C+Mingyi+Hong%2C+Yang+Liu%2C+Devamanyu+Hazarika%2C+and+Kaixiang+Lin%2E+2025%2E+Do+LLMs+recognize+your+preferences%3F+Evaluating+personalized+preference+following+in+LLMs%2E+arXiv+preprint%2E+ArXiv%3A2502%2E0959)
73. Arman Zharmagambetov, Chuan Guo, Ivan Evtimov, Maya Pavlova, Ruslan Salakhutdinov, and Kamalika Chaudhuri. 2025. AgentDAM: Privacy leakage evaluation for autonomous web agents. arXiv preprint. ArXiv:2503.09780 [cs]. [find](https://scholar.google.com/scholar?q=Arman+Zharmagambetov%2C+Chuan+Guo%2C+Ivan+Evtimov%2C+Maya+Pavlova%2C+Ruslan+Salakhutdinov%2C+and+Kamalika+Chaudhuri%2E+2025%2E+AgentDAM%3A+Privacy+leakage+evaluation+for+autonomous+web+agents%2E+arXiv+preprint%2E+ArXiv%3A2)
74. Jijie Zhou, Eryue Xu, Yaoyao Wu, and Tianshi Li. 2025. Rescriber: Smaller-LLM-powered user-led data minimization for LLM-based chatbots. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI ’25, pages 1–28, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Jijie+Zhou%2C+Eryue+Xu%2C+Yaoyao+Wu%2C+and+Tianshi+Li%2E+2025%2E+Rescriber%3A+Smaller%2DLLM%2Dpowered+user%2Dled+data+minimization+for+LLM%2Dbased+chatbots%2E+In+Proceedings+of+the+2025+CHI+Conference+on+Human+Factors+in+C)

*Updated 31 October 2026*