---
title: "Membership Inference Attacks From First Principles — Carlini 2021"
description: "They propose a different test: an attack's true-positive rate at very low false-positive… 59.5% accuracy. 73 references, cited by 82 papers here."
authors: ["Nicholas Carlini", "Steve Chien", "Milad Nasr", "Shuang Song", "Andreas Terzis", "Florian Tramer"]
year: 2021
venue: "arXiv:2112.03570"
source: "https://arxiv.org/abs/2112.03570"
doi: "10.48550/arxiv.2112.03570"
arxiv: "2112.03570"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/carlini-2021-membership-inference-attacks-first-principles/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/carlini-2021-membership-inference-attacks-first-principles/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/) / Membership Inference Attacks From First Principles

# Membership Inference Attacks From First Principles

**Authors:** [Nicholas Carlini](https://md.research.elamur.ai/free/en/authors/nicholas-carlini/), Steve Chien, [Milad Nasr](https://md.research.elamur.ai/free/en/authors/milad-nasr/), Shuang Song, Andreas Terzis and [Florian Tramer](https://md.research.elamur.ai/free/en/authors/florian-tramer/) · **Year:** 2021 · **Venue:** [arXiv:2112.03570](https://md.research.elamur.ai/free/en/venues/proceedings-ieee-symposium-on-security-and-privacy-proceedings-of-the-ieee/) ([Proceedings - IEEE Symposium on Security and Privacy/Proceedings of the ... IEEE Symposium on Security and Privacy 2021](https://md.research.elamur.ai/free/en/venues/proceedings-ieee-symposium-on-security-and-privacy-proceedings-of-the-ieee/2021/)) · **Publisher:** [IEEE Computer Society](https://md.research.elamur.ai/free/en/publishers/ieee-computer-society/) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2112.03570) · [PDF](https://arxiv.org/pdf/2112.03570) · [DOI 10.48550/arxiv.2112.03570](https://doi.org/10.48550/arxiv.2112.03570) · [arXiv 2112.03570](https://arxiv.org/abs/2112.03570) · [OpenAlex](https://openalex.org/W4288057780)

**Citations:** 502 citations (OpenAlex, 2026-10-14) · [82](https://md.research.elamur.ai/free/en/papers/carlini-2021-membership-inference-attacks-first-principles/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Membership inference attacks ask whether a specific example was used to train a model. They are the most common way to empirically audit the privacy of machine learning systems. Carlini et al. argue that the usual yardsticks, balanced accuracy and ROC-AUC averaged over all thresholds, reveal little about whether an attacker can confidently identify even a few individual members. They propose a different test: an attack's true-positive rate at very low false-positive rates (around 0.1% or below), shown on log-scale ROC curves. By that test, most earlier attacks perform poorly.

The authors then design the Likelihood Ratio Attack (LiRA). LiRA trains shadow models that either include or exclude the target example. It fits Gaussian distributions to the logit-scaled confidences in each case and scores membership with a likelihood ratio. A cheaper offline variant uses only models trained without the example.

LiRA was tested on well-regularized models for CIFAR-10, CIFAR-100, WikiText-103, Texas and Purchase. At low FPR it is roughly ten times stronger than prior work, and it still wins on average-case metrics. The result suggests that privacy audits built on weaker attacks may seriously underestimate leakage.

## Key points
- Proposed metric: TPR at low FPR (e.g., ≤0.1%) rather than balanced accuracy or AUC. Aggregate scores can disagree with this metric. The Yeom et al. attack reaches 59.5% accuracy yet fails at low FPR. The Long et al. attack scores only 53.5% accuracy but performs better in that regime.
- On a non-overfit CIFAR-10 model with 92% test accuracy, LiRA reaches 63.7% balanced accuracy, compared with roughly 53–60% for earlier attacks. It improves low-FPR success by about 10×.
- LiRA combines per-example difficulty calibration, logit scaling, a Gaussian likelihood test and multiple augmented queries. Ablations on CIFAR-100, WikiText-103, Texas and Purchase show that logit scaling and the Gaussian test help only when applied together.
- There are two variants: online (in and out shadow models) and offline (out models only). Further experiments examine attackers whose shadow models use a different architecture (CNN, WRN28 sizes), optimizer (SGD, momentum, Adam) or augmentation than the target.
- Code is released in the tensorflow/privacy repository (research/mi_lira_2021).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 492 works · 440,463 citations
- [University of Massachusetts Amherst](https://md.research.elamur.ai/free/en/universities/university-of-massachusetts-amherst/) — University, Amherst Center, United States · 114 works · 68,731 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works

## Cite

Cite as: Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis and Florian Tramer. 2021. Membership Inference Attacks From First Principles. arXiv:2112.03570.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{carlini2021membership,
  title = {{Membership Inference Attacks From First Principles}},
  author = {Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis and Florian Tramer},
  year = {2021},
  doi = {10.48550/arxiv.2112.03570},
  eprint = {2112.03570},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2112.03570}
}
```

- [← previous A Robustly Optimized BERT Pre-training Approach with Post-training](https://md.research.elamur.ai/free/en/papers/zhuang-2021-robustly-optimized-bert-pre-training/)
- [next → OCR-free Document Understanding Transformer](https://md.research.elamur.ai/free/en/papers/kim-2021-ocr-free-document-understanding-transformer/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/)

## References

73 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, page 308–318. ACM, 2016. [find](https://scholar.google.com/scholar?q=Martin+Abadi%2C+Andy+Chu%2C+Ian+Goodfellow%2C+H%2E+Brendan+McMahan%2C+Ilya+Mironov%2C+Kunal+Talwar%2C+and+Li+Zhang%2E+Deep+learning+with+differential+privacy%2E+In+Proceedings+of+the+2016+ACM+SIGSAC+Conference+on+Compu)
2. Gavin Brown, Mark Bun, Vitaly Feldman, Adam Smith, and Kunal Talwar. When is memorization of irrelevant training data necessary for high-accuracy learning? In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, pages 123–132, 2021. [find](https://scholar.google.com/scholar?q=Gavin+Brown%2C+Mark+Bun%2C+Vitaly+Feldman%2C+Adam+Smith%2C+and+Kunal+Talwar%2E+When+is+memorization+of+irrelevant+training+data+necessary+for+high%2Daccuracy+learning%3F+In+Proceedings+of+the+53rd+Annual+ACM+SIGACT)
3. Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. [The secret sharer: Evaluating and testing unintended memorization in neural networks](https://md.research.elamur.ai/free/en/papers/carlini-2018-secret-sharer-evaluating-testing-unintended/). In 28th USENIX Security Symposium (USENIX Security 19), pages 267–284, 2019.
4. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. [Extracting training data from large language models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/). In 30th USENIX Security Symposium (USENIX Security 21), 2021.
5. Mia Xu Chen, Benjamin N Lee, Gagan Bansal, Yuan Cao, Shuyuan Zhang, Justin Lu, Jackie Tsay, Yinan Wang, Andrew M Dai, Zhifeng Chen, et al. Gmail smart compose: Real-time assisted writing. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2287–2295, 2019. [find](https://scholar.google.com/scholar?q=Mia+Xu+Chen%2C+Benjamin+N+Lee%2C+Gagan+Bansal%2C+Yuan+Cao%2C+Shuyuan+Zhang%2C+Justin+Lu%2C+Jackie+Tsay%2C+Yinan+Wang%2C+Andrew+M+Dai%2C+Zhifeng+Chen%2C+et+al%2E+Gmail+smart+compose%3A+Real%2Dtime+assisted+writing%2E+In+ACM+SIGKD)
6. Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label-only membership inference attacks. In International Conference on Machine Learning, pages 1964–1974. PMLR, 2021. [find](https://scholar.google.com/scholar?q=Christopher+A+Choquette%2DChoo%2C+Florian+Tramer%2C+Nicholas+Carlini%2C+and+Nicolas+Papernot%2E+Label%2Donly+membership+inference+attacks%2E+In+International+Conference+on+Machine+Learning%2C+pages+1964%E2%80%931974%2E+PMLR%2C+2)
7. Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le. Autoaugment: Learning augmentation policies from data, 2018. [find](https://scholar.google.com/scholar?q=Ekin+D%2E+Cubuk%2C+Barret+Zoph%2C+Dandelion+Mane%2C+Vijay+Vasudevan%2C+and+Quoc+V%2E+Le%2E+Autoaugment%3A+Learning+augmentation+policies+from+data%2C+2018%2E)
8. Luke N Darlow, Elliot J Crowley, Antreas Antoniou, and Amos J Storkey. CINIC-10 is not Imagenet or CIFAR-10. arXiv preprint arXiv:1810.03505, 2018. — [arXiv:1810.03505](https://arxiv.org/abs/1810.03505) [find](https://scholar.google.com/scholar?q=Luke+N+Darlow%2C+Elliot+J+Crowley%2C+Antreas+Antoniou%2C+and+Amos+J+Storkey%2E+CINIC%2D10+is+not+Imagenet+or+CIFAR%2D10%2E+arXiv+preprint+arXiv%3A1810%2E03505%2C+2018%2E+%E2%80%94+%5BarXiv%3A1810%2E03505%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1810%2E0350)
9. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. [find](https://scholar.google.com/scholar?q=Jia+Deng%2C+Wei+Dong%2C+Richard+Socher%2C+Li%2DJia+Li%2C+Kai+Li%2C+and+Li+Fei%2DFei%2E+ImageNet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+pages+248%E2%80%93255)
10. Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci., 9 (3-4):211–407, 2014. [find](https://scholar.google.com/scholar?q=Cynthia+Dwork+and+Aaron+Roth%2E+The+algorithmic+foundations+of+differential+privacy%2E+Found%2E+Trends+Theor%2E+Comput%2E+Sci%2E%2C+9+%283%2D4%29%3A211%E2%80%93407%2C+2014%2E)
11. Cynthia Dwork, Adam Smith, Thomas Steinke, Jonathan Ullman, and Salil Vadhan. Robust traceability from trace amounts. In 2015 IEEE 56th Annual Symposium on Foundations of Computer Science, pages 650–669. IEEE, 2015. [find](https://scholar.google.com/scholar?q=Cynthia+Dwork%2C+Adam+Smith%2C+Thomas+Steinke%2C+Jonathan+Ullman%2C+and+Salil+Vadhan%2E+Robust+traceability+from+trace+amounts%2E+In+2015+IEEE+56th+Annual+Symposium+on+Foundations+of+Computer+Science%2C+pages+650%E2%80%936)
12. Cynthia Dwork, Adam Smith, Thomas Steinke, and Jonathan Ullman. Exposed! a survey of attacks on private data. Annual Review of Statistics and Its Application, 4:61–84, 2017. [find](https://scholar.google.com/scholar?q=Cynthia+Dwork%2C+Adam+Smith%2C+Thomas+Steinke%2C+and+Jonathan+Ullman%2E+Exposed%21+a+survey+of+attacks+on+private+data%2E+Annual+Review+of+Statistics+and+Its+Application%2C+4%3A61%E2%80%9384%2C+2017%2E)
13. Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639):115–118, 2017. [find](https://scholar.google.com/scholar?q=Andre+Esteva%2C+Brett+Kuprel%2C+Roberto+A+Novoa%2C+Justin+Ko%2C+Susan+M+Swetter%2C+Helen+M+Blau%2C+and+Sebastian+Thrun%2E+Dermatologist%2Dlevel+classification+of+skin+cancer+with+deep+neural+networks%2E+Nature%2C+542%28763)
14. Vitaly Feldman. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pages 954–959, 2020. [find](https://scholar.google.com/scholar?q=Vitaly+Feldman%2E+Does+learning+require+memorization%3F+a+short+tale+about+a+long+tail%2E+In+Proceedings+of+the+52nd+Annual+ACM+SIGACT+Symposium+on+Theory+of+Computing%2C+pages+954%E2%80%93959%2C+2020%2E)
15. Vitaly Feldman and Chiyuan Zhang. [What neural networks memorize and why: Discovering the long tail via influence estimation](https://md.research.elamur.ai/free/en/papers/feldman-2020-neural-networks-memorize-why-discovering/). arXiv preprint arXiv:2008.03703, 2020.
16. Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322–1333, 2015. [find](https://scholar.google.com/scholar?q=Matt+Fredrikson%2C+Somesh+Jha%2C+and+Thomas+Ristenpart%2E+Model+inversion+attacks+that+exploit+confidence+information+and+basic+countermeasures%2E+In+Proceedings+of+the+22nd+ACM+SIGSAC+Conference+on+Computer+)
17. Karan Ganju, Qi Wang, Wei Yang, Carl A Gunter, and Nikita Borisov. Property inference attacks on fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pages 619–633, 2018. [find](https://scholar.google.com/scholar?q=Karan+Ganju%2C+Qi+Wang%2C+Wei+Yang%2C+Carl+A+Gunter%2C+and+Nikita+Borisov%2E+Property+inference+attacks+on+fully+connected+neural+networks+using+permutation+invariant+representations%2E+In+Proceedings+of+the+2018)
18. Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro. LOGAN: Membership inference attacks against generative models. In Proceedings on Privacy Enhancing Technologies (PoPETs), pages 133–152. De Gruyter, 2019. [find](https://scholar.google.com/scholar?q=Jamie+Hayes%2C+Luca+Melis%2C+George+Danezis%2C+and+Emiliano+De+Cristofaro%2E+LOGAN%3A+Membership+inference+attacks+against+generative+models%2E+In+Proceedings+on+Privacy+Enhancing+Technologies+%28PoPETs%29%2C+pages+133)
19. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/), 2015.
20. Xinlei He, Jinyuan Jia, Michael Backes, Neil Zhenqiang Gong, and Yang Zhang. Stealing links from graph neural networks. In 30th USENIX Security Symposium (USENIX Security 21), 2021. [find](https://scholar.google.com/scholar?q=Xinlei+He%2C+Jinyuan+Jia%2C+Michael+Backes%2C+Neil+Zhenqiang+Gong%2C+and+Yang+Zhang%2E+Stealing+links+from+graph+neural+networks%2E+In+30th+USENIX+Security+Symposium+%28USENIX+Security+21%29%2C+2021%2E)
21. Grant Ho, Aashish Sharma, Mobin Javed, Vern Paxson, and David Wagner. Detecting credential spearphishing in enterprise settings. In 26th USENIX Security Symposium (USENIX Security 17), pages 469–485, 2017. [find](https://scholar.google.com/scholar?q=Grant+Ho%2C+Aashish+Sharma%2C+Mobin+Javed%2C+Vern+Paxson%2C+and+David+Wagner%2E+Detecting+credential+spearphishing+in+enterprise+settings%2E+In+26th+USENIX+Security+Symposium+%28USENIX+Security+17%29%2C+pages+469%E2%80%93485%2C+)
22. Nils Homer, Szabolcs Szelinger, Margot Redman, David Duggan, Waibhav Tembe, Jill Muehling, John V Pearson, Dietrich A Stephan, Stanley F Nelson, and David W Craig. Resolving individuals contributing trace amounts of dna to highly complex mixtures using high-density snp genotyping microarrays. PLoS g… [find](https://scholar.google.com/scholar?q=Nils+Homer%2C+Szabolcs+Szelinger%2C+Margot+Redman%2C+David+Duggan%2C+Waibhav+Tembe%2C+Jill+Muehling%2C+John+V+Pearson%2C+Dietrich+A+Stephan%2C+Stanley+F+Nelson%2C+and+David+W+Craig%2E+Resolving+individuals+contributing+t)
23. Robert A Jacobs. Increased rates of convergence through learning rate adaptation. Neural networks, 1(4):295–307, 1988. [find](https://scholar.google.com/scholar?q=Robert+A+Jacobs%2E+Increased+rates+of+convergence+through+learning+rate+adaptation%2E+Neural+networks%2C+1%284%29%3A295%E2%80%93307%2C+1988%2E)
24. Matthew Jagielski, Jonathan Ullman, and Alina Oprea. Auditing differentially private machine learning: How private is private SGD? arXiv preprint arXiv:2006.07709, 2020. — [arXiv:2006.07709](https://arxiv.org/abs/2006.07709) [find](https://scholar.google.com/scholar?q=Matthew+Jagielski%2C+Jonathan+Ullman%2C+and+Alina+Oprea%2E+Auditing+differentially+private+machine+learning%3A+How+private+is+private+SGD%3F+arXiv+preprint+arXiv%3A2006%2E07709%2C+2020%2E+%E2%80%94+%5BarXiv%3A2006%2E07709%5D%28https%3A%2F%2Fa)
25. Bargav Jayaraman, Lingxiao Wang, David Evans, and Quanquan Gu. Revisiting membership inference under realistic assumptions. In Proceedings on Privacy Enhancing Technologies (PoPETs), 2021. [find](https://scholar.google.com/scholar?q=Bargav+Jayaraman%2C+Lingxiao+Wang%2C+David+Evans%2C+and+Quanquan+Gu%2E+Revisiting+membership+inference+under+realistic+assumptions%2E+In+Proceedings+on+Privacy+Enhancing+Technologies+%28PoPETs%29%2C+2021%2E)
26. Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, and Neil Zhenqiang Gong. Memguard: Defending against black-box membership inference attacks via adversarial examples. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 259–274, 2019. [find](https://scholar.google.com/scholar?q=Jinyuan+Jia%2C+Ahmed+Salem%2C+Michael+Backes%2C+Yang+Zhang%2C+and+Neil+Zhenqiang+Gong%2E+Memguard%3A+Defending+against+black%2Dbox+membership+inference+attacks+via+adversarial+examples%2E+In+Proceedings+of+the+2019+A)
27. Alex Kantchelian, Michael Carl Tschantz, Sadia Afroz, Brad Miller, Vaishaal Shankar, Rekha Bachwani, Anthony D Joseph, and J Doug Tygar. Better malware ground truth: Techniques for weighting anti-virus vendor labels. In Proceedings of the 8th ACM Workshop on Artificial Intelligence and Security, pag… [find](https://scholar.google.com/scholar?q=Alex+Kantchelian%2C+Michael+Carl+Tschantz%2C+Sadia+Afroz%2C+Brad+Miller%2C+Vaishaal+Shankar%2C+Rekha+Bachwani%2C+Anthony+D+Joseph%2C+and+J+Doug+Tygar%2E+Better+malware+ground+truth%3A+Techniques+for+weighting+anti%2Dviru)
28. Zico Kolter and Marcus A Maloof. Learning to detect and classify malicious executables in the wild. Journal of Machine Learning Research, 7(12), 2006. [find](https://scholar.google.com/scholar?q=Zico+Kolter+and+Marcus+A+Maloof%2E+Learning+to+detect+and+classify+malicious+executables+in+the+wild%2E+Journal+of+Machine+Learning+Research%2C+7%2812%29%2C+2006%2E)
29. Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images, 2009. [find](https://scholar.google.com/scholar?q=Alex+Krizhevsky%2C+Geoffrey+Hinton%2C+et+al%2E+Learning+multiple+layers+of+features+from+tiny+images%2C+2009%2E)
30. Anders Krogh and John A Hertz. A simple weight decay can improve generalization. In Advances in neural information processing systems, pages 950–957, 1992. [find](https://scholar.google.com/scholar?q=Anders+Krogh+and+John+A+Hertz%2E+A+simple+weight+decay+can+improve+generalization%2E+In+Advances+in+neural+information+processing+systems%2C+pages+950%E2%80%93957%2C+1992%2E)
31. Aleksandar Lazarevic, Levent Ertoz, Vipin Kumar, Aysel Ozgur, and Jaideep Srivastava. A comparative study of anomaly detection schemes in network intrusion detection. In Proceedings of the 2003 SIAM international conference on data mining, pages 25–36. SIAM, 2003. [find](https://scholar.google.com/scholar?q=Aleksandar+Lazarevic%2C+Levent+Ertoz%2C+Vipin+Kumar%2C+Aysel+Ozgur%2C+and+Jaideep+Srivastava%2E+A+comparative+study+of+anomaly+detection+schemes+in+network+intrusion+detection%2E+In+Proceedings+of+the+2003+SIAM+i)
32. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. [find](https://scholar.google.com/scholar?q=Yann+LeCun%2C+Le%CC%81on+Bottou%2C+Yoshua+Bengio%2C+and+Patrick+Haffner%2E+Gradient%2Dbased+learning+applied+to+document+recognition%2E+Proceedings+of+the+IEEE%2C+86%2811%29%3A2278%E2%80%932324%2C+1998%2E)
33. Klas Leino and Matt Fredrikson. Stolen memories: Leveraging model memorization for calibrated white-box membership inference. arXiv preprint arXiv:1906.11798, 2019. — [arXiv:1906.11798](https://arxiv.org/abs/1906.11798) [find](https://scholar.google.com/scholar?q=Klas+Leino+and+Matt+Fredrikson%2E+Stolen+memories%3A+Leveraging+model+memorization+for+calibrated+white%2Dbox+membership+inference%2E+arXiv+preprint+arXiv%3A1906%2E11798%2C+2019%2E+%E2%80%94+%5BarXiv%3A1906%2E11798%5D%28https%3A%2F%2Farxiv%2E)
34. Zheng Li and Yang Zhang. Membership leakage in label-only exposures. arXiv preprint arXiv:2007.15528, 2020. — [arXiv:2007.15528](https://arxiv.org/abs/2007.15528) [find](https://scholar.google.com/scholar?q=Zheng+Li+and+Yang+Zhang%2E+Membership+leakage+in+label%2Donly+exposures%2E+arXiv+preprint+arXiv%3A2007%2E15528%2C+2020%2E+%E2%80%94+%5BarXiv%3A2007%2E15528%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2007%2E15528%29)
35. Yugeng Liu, Rui Wen, Xinlei He, Ahmed Salem, Zhikun Zhang, Michael Backes, Emiliano De Cristofaro, Mario Fritz, and Yang Zhang. ML-Doctor: Holistic risk assessment of inference attacks against machine learning models. arXiv preprint arXiv:2102.02551, 2021. — [arXiv:2102.02551](https://arxiv.org/abs/2102.02551) [find](https://scholar.google.com/scholar?q=Yugeng+Liu%2C+Rui+Wen%2C+Xinlei+He%2C+Ahmed+Salem%2C+Zhikun+Zhang%2C+Michael+Backes%2C+Emiliano+De+Cristofaro%2C+Mario+Fritz%2C+and+Yang+Zhang%2E+ML%2DDoctor%3A+Holistic+risk+assessment+of+inference+attacks+against+machine)
36. Yunhui Long, Vincent Bindschaedler, and Carl A Gunter. Towards measuring membership privacy. arXiv preprint arXiv:1712.09136, 2017. — [arXiv:1712.09136](https://arxiv.org/abs/1712.09136) [find](https://scholar.google.com/scholar?q=Yunhui+Long%2C+Vincent+Bindschaedler%2C+and+Carl+A+Gunter%2E+Towards+measuring+membership+privacy%2E+arXiv+preprint+arXiv%3A1712%2E09136%2C+2017%2E+%E2%80%94+%5BarXiv%3A1712%2E09136%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1712%2E09136%29)
37. Yunhui Long, Lei Wang, Diyue Bu, Vincent Bindschaedler, Xiaofeng Wang, Haixu Tang, Carl A Gunter, and Kai Chen. A pragmatic approach to membership inferences on machine learning models. In 2020 IEEE European Symposium on Security and Privacy (EuroS&P), pages 521–534. IEEE, 2020. [find](https://scholar.google.com/scholar?q=Yunhui+Long%2C+Lei+Wang%2C+Diyue+Bu%2C+Vincent+Bindschaedler%2C+Xiaofeng+Wang%2C+Haixu+Tang%2C+Carl+A+Gunter%2C+and+Kai+Chen%2E+A+pragmatic+approach+to+membership+inferences+on+machine+learning+models%2E+In+2020+IEEE+E)
38. Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+SGDR%3A+Stochastic+gradient+descent+with+warm+restarts%2E+arXiv+preprint+arXiv%3A1608%2E03983%2C+2016%2E)
39. Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE Symposium on Security and Privacy (SP), pages 691–706. IEEE, 2019. [find](https://scholar.google.com/scholar?q=Luca+Melis%2C+Congzheng+Song%2C+Emiliano+De+Cristofaro%2C+and+Vitaly+Shmatikov%2E+Exploiting+unintended+feature+leakage+in+collaborative+learning%2E+In+2019+IEEE+Symposium+on+Security+and+Privacy+%28SP%29%2C+pages+69)
40. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. [Pointer sentinel mixture models](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/). arXiv preprint arXiv:1609.07843, 2016.
41. Vangelis Metsis, Ion Androutsopoulos, and Georgios Paliouras. Spam filtering with naive Bayes–which naive Bayes? In CEAS, volume 17, pages 28–69, 2006. [find](https://scholar.google.com/scholar?q=Vangelis+Metsis%2C+Ion+Androutsopoulos%2C+and+Georgios+Paliouras%2E+Spam+filtering+with+naive+Bayes%E2%80%93which+naive+Bayes%3F+In+CEAS%2C+volume+17%2C+pages+28%E2%80%9369%2C+2006%2E)
42. Sasi Kumar Murakonda and Reza Shokri. ML Privacy Meter: Aiding regulatory compliance by quantifying the privacy risks of machine learning. arXiv preprint arXiv:2007.09339, 2020. — [arXiv:2007.09339](https://arxiv.org/abs/2007.09339) [find](https://scholar.google.com/scholar?q=Sasi+Kumar+Murakonda+and+Reza+Shokri%2E+ML+Privacy+Meter%3A+Aiding+regulatory+compliance+by+quantifying+the+privacy+risks+of+machine+learning%2E+arXiv+preprint+arXiv%3A2007%2E09339%2C+2020%2E+%E2%80%94+%5BarXiv%3A2007%2E09339%5D%28h)
43. Sasi Kumar Murakonda, Reza Shokri, and George Theodorakopoulos. Ultimate power of inference attacks: Privacy risks of learning high-dimensional graphical models. arXiv e-prints, pages arXiv–1905, 2019. [find](https://scholar.google.com/scholar?q=Sasi+Kumar+Murakonda%2C+Reza+Shokri%2C+and+George+Theodorakopoulos%2E+Ultimate+power+of+inference+attacks%3A+Privacy+risks+of+learning+high%2Ddimensional+graphical+models%2E+arXiv+e%2Dprints%2C+pages+arXiv%E2%80%931905%2C+2019)
44. Sasi Kumar Murakonda, Reza Shokri, and George Theodorakopoulos. Quantifying the privacy risks of learning high-dimensional graphical models. In International Conference on Artificial Intelligence and Statistics, pages 2287–2295. PMLR, 2021. [find](https://scholar.google.com/scholar?q=Sasi+Kumar+Murakonda%2C+Reza+Shokri%2C+and+George+Theodorakopoulos%2E+Quantifying+the+privacy+risks+of+learning+high%2Ddimensional+graphical+models%2E+In+International+Conference+on+Artificial+Intelligence+and+)
45. Milad Nasr, Reza Shokri, and Amir Houmansadr. Machine learning with membership privacy using adversarial regularization. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pages 634–646, 2018. [find](https://scholar.google.com/scholar?q=Milad+Nasr%2C+Reza+Shokri%2C+and+Amir+Houmansadr%2E+Machine+learning+with+membership+privacy+using+adversarial+regularization%2E+In+Proceedings+of+the+2018+ACM+SIGSAC+Conference+on+Computer+and+Communications)
46. Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP), pages 739–753. IEEE, 2019. [find](https://scholar.google.com/scholar?q=Milad+Nasr%2C+Reza+Shokri%2C+and+Amir+Houmansadr%2E+Comprehensive+privacy+analysis+of+deep+learning%3A+Passive+and+active+white%2Dbox+inference+attacks+against+centralized+and+federated+learning%2E+In+2019+IEEE+s)
47. Milad Nasr, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, and Nicholas Carlini. Adversary instantiation: Lower bounds for differentially private machine learning. arXiv preprint arXiv:2101.04535, 2021. — [arXiv:2101.04535](https://arxiv.org/abs/2101.04535) [find](https://scholar.google.com/scholar?q=Milad+Nasr%2C+Shuang+Song%2C+Abhradeep+Thakurta%2C+Nicolas+Papernot%2C+and+Nicholas+Carlini%2E+Adversary+instantiation%3A+Lower+bounds+for+differentially+private+machine+learning%2E+arXiv+preprint+arXiv%3A2101%2E04535%2C)
48. Jerzy Neyman and Egon Sharpe Pearson. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London., 231(694-706): 289–337, 1933. [find](https://scholar.google.com/scholar?q=Jerzy+Neyman+and+Egon+Sharpe+Pearson%2E+On+the+problem+of+the+most+efficient+tests+of+statistical+hypotheses%2E+Philosophical+Transactions+of+the+Royal+Society+of+London%2E%2C+231%28694%2D706%29%3A+289%E2%80%93337%2C+1933%2E)
49. Patrick Pantel and Dekang Lin. SpamCop: A spam classification & organization program. In Proceedings of AAAI-98 Workshop on Learning for Text Categorization, pages 95–98, 1998. [find](https://scholar.google.com/scholar?q=Patrick+Pantel+and+Dekang+Lin%2E+SpamCop%3A+A+spam+classification+%26+organization+program%2E+In+Proceedings+of+AAAI%2D98+Workshop+on+Learning+for+Text+Categorization%2C+pages+95%E2%80%9398%2C+1998%2E)
50. Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson. Scalable private learning with PATE. arXiv preprint arXiv:1802.08908, 2018. — [arXiv:1802.08908](https://arxiv.org/abs/1802.08908) [find](https://scholar.google.com/scholar?q=Nicolas+Papernot%2C+Shuang+Song%2C+Ilya+Mironov%2C+Ananth+Raghunathan%2C+Kunal+Talwar%2C+and+U%CC%81lfar+Erlingsson%2E+Scalable+private+learning+with+PATE%2E+arXiv+preprint+arXiv%3A1802%2E08908%2C+2018%2E+%E2%80%94+%5BarXiv%3A1802%2E08908%5D%28h)
51. Huy Phan. huyvnphan/pytorch cifar10, January 2021. URL https://doi.org/10.5281/zenodo.4431043. — [doi:10.5281/zenodo.4431043](https://doi.org/10.5281/zenodo.4431043) [↗](https://doi.org/10.5281/zenodo.4431043)
52. Apostolos Pyrgelis, Carmela Troncoso, and Emiliano De Cristofaro. Knock knock, who’s there? membership inference on aggregate location data. arXiv preprint arXiv:1708.06145, 2017. — [arXiv:1708.06145](https://arxiv.org/abs/1708.06145) [find](https://scholar.google.com/scholar?q=Apostolos+Pyrgelis%2C+Carmela+Troncoso%2C+and+Emiliano+De+Cristofaro%2E+Knock+knock%2C+who%E2%80%99s+there%3F+membership+inference+on+aggregate+location+data%2E+arXiv+preprint+arXiv%3A1708%2E06145%2C+2017%2E+%E2%80%94+%5BarXiv%3A1708%2E06145%5D)
53. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 2019. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+2019%2E)
54. Shadi Rahimian, Tribhuvanesh Orekondy, and Mario Fritz. Sampling attacks: Amplification of membership inference attacks by repeated queries. arXiv preprint arXiv:2009.00395, 2020. — [arXiv:2009.00395](https://arxiv.org/abs/2009.00395) [find](https://scholar.google.com/scholar?q=Shadi+Rahimian%2C+Tribhuvanesh+Orekondy%2C+and+Mario+Fritz%2E+Sampling+attacks%3A+Amplification+of+membership+inference+attacks+by+repeated+queries%2E+arXiv+preprint+arXiv%3A2009%2E00395%2C+2020%2E+%E2%80%94+%5BarXiv%3A2009%2E00395%5D)
55. Md Atiqur Rahman, Tanzila Rahman, Robert Laganière, Noman Mohammed, and Yang Wang. Membership inference attack against differentially private deep learning model. Trans. Data Priv., 11(1):61–79, 2018. [find](https://scholar.google.com/scholar?q=Md+Atiqur+Rahman%2C+Tanzila+Rahman%2C+Robert+Laganie%CC%80re%2C+Noman+Mohammed%2C+and+Yang+Wang%2E+Membership+inference+attack+against+differentially+private+deep+learning+model%2E+Trans%2E+Data+Priv%2E%2C+11%281%29%3A61%E2%80%9379%2C+2018)
56. Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, and Hervé Jégou. White-box vs black-box: Bayes optimal strategies for membership inference. In International Conference on Machine Learning, pages 5558–5567. PMLR, 2019. [find](https://scholar.google.com/scholar?q=Alexandre+Sablayrolles%2C+Matthijs+Douze%2C+Cordelia+Schmid%2C+Yann+Ollivier%2C+and+Herve%CC%81+Je%CC%81gou%2E+White%2Dbox+vs+black%2Dbox%3A+Bayes+optimal+strategies+for+membership+inference%2E+In+International+Conference+on+Mac)
57. Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. ML-Leaks: Model and data independent membership inference attacks and defenses on machine learning models, 2018. [find](https://scholar.google.com/scholar?q=Ahmed+Salem%2C+Yang+Zhang%2C+Mathias+Humbert%2C+Pascal+Berrang%2C+Mario+Fritz%2C+and+Michael+Backes%2E+ML%2DLeaks%3A+Model+and+data+independent+membership+inference+attacks+and+defenses+on+machine+learning+models%2C+20)
58. Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, and Yang Zhang. Updates-leak: Data set inference and reconstruction attacks in online learning. In 29th USENIX Security Symposium (USENIX Security 20), pages 1291–1308, 2020. [find](https://scholar.google.com/scholar?q=Ahmed+Salem%2C+Apratim+Bhattacharya%2C+Michael+Backes%2C+Mario+Fritz%2C+and+Yang+Zhang%2E+Updates%2Dleak%3A+Data+set+inference+and+reconstruction+attacks+in+online+learning%2E+In+29th+USENIX+Security+Symposium+%28USENI)
59. Sriram Sankararaman, Guillaume Obozinski, Michael I Jordan, and Eran Halperin. Genomic privacy and limits of individual detection in a pool. Nature genetics, 41(9):965–967, 2009. [find](https://scholar.google.com/scholar?q=Sriram+Sankararaman%2C+Guillaume+Obozinski%2C+Michael+I+Jordan%2C+and+Eran+Halperin%2E+Genomic+privacy+and+limits+of+individual+detection+in+a+pool%2E+Nature+genetics%2C+41%289%29%3A965%E2%80%93967%2C+2009%2E)
60. Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. [Membership inference attacks against machine learning models](https://md.research.elamur.ai/free/en/papers/shokri-2016-membership-inference-attacks-against-machine/). arXiv preprint arXiv:1610.05820, 2016.
61. Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learning models. In 30th USENIX Security Symposium (USENIX Security 21), 2021. [find](https://scholar.google.com/scholar?q=Liwei+Song+and+Prateek+Mittal%2E+Systematic+evaluation+of+privacy+risks+of+machine+learning+models%2E+In+30th+USENIX+Security+Symposium+%28USENIX+Security+21%29%2C+2021%2E)
62. Liwei Song, Reza Shokri, and Prateek Mittal. Privacy risks of securing machine learning models against adversarial examples. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 241–257, 2019. [find](https://scholar.google.com/scholar?q=Liwei+Song%2C+Reza+Shokri%2C+and+Prateek+Mittal%2E+Privacy+risks+of+securing+machine+learning+models+against+adversarial+examples%2E+In+Proceedings+of+the+2019+ACM+SIGSAC+Conference+on+Computer+and+Communicat)
63. Shuang Song and David Marn. Introducing a new privacy testing library in tensorflow. [https://blog.tensorflow.org/2020/06/introducing-new-privacy-testing-library.html](https://blog.tensorflow.org/2020/06/introducing-new-privacy-testing-library.html), 2020. [↗](https://blog.tensorflow.org/2020/06/introducing-new-privacy-testing-library.html)
64. Shuang Song, Kamalika Chaudhuri, and Anand D Sarwate. Stochastic gradient descent with differentially private updates. In 2013 IEEE Global Conference on Signal and Information Processing, pages 245–248. IEEE, 2013. [find](https://scholar.google.com/scholar?q=Shuang+Song%2C+Kamalika+Chaudhuri%2C+and+Anand+D+Sarwate%2E+Stochastic+gradient+descent+with+differentially+private+updates%2E+In+2013+IEEE+Global+Conference+on+Signal+and+Information+Processing%2C+pages+245%E2%80%9324)
65. Thomas Steinke and Jonathan Ullman. The pitfalls of average-case differential privacy. DifferentialPrivacy.org, 07 2020. [https://differentialprivacy.org/average-case-dp/](https://differentialprivacy.org/average-case-dp/). [↗](https://differentialprivacy.org/average-case-dp/)
66. Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei. Towards demystifying membership inference attacks. arXiv preprint arXiv:1807.09173, 2018. — [arXiv:1807.09173](https://arxiv.org/abs/1807.09173) [find](https://scholar.google.com/scholar?q=Stacey+Truex%2C+Ling+Liu%2C+Mehmet+Emre+Gursoy%2C+Lei+Yu%2C+and+Wenqi+Wei%2E+Towards+demystifying+membership+inference+attacks%2E+arXiv+preprint+arXiv%3A1807%2E09173%2C+2018%2E+%E2%80%94+%5BarXiv%3A1807%2E09173%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
67. David A Van Dyk and Xiao-Li Meng. The art of data augmentation. Journal of Computational and Graphical Statistics, 10 (1):1–50, 2001. [find](https://scholar.google.com/scholar?q=David+A+Van+Dyk+and+Xiao%2DLi+Meng%2E+The+art+of+data+augmentation%2E+Journal+of+Computational+and+Graphical+Statistics%2C+10+%281%29%3A1%E2%80%9350%2C+2001%2E)
68. Lauren Watson, Chuan Guo, Graham Cormode, and Alex Sablayrolles. On the importance of difficulty calibration in membership inference attacks. arXiv preprint arXiv:2111.08440, 2021. — [arXiv:2111.08440](https://arxiv.org/abs/2111.08440) [find](https://scholar.google.com/scholar?q=Lauren+Watson%2C+Chuan+Guo%2C+Graham+Cormode%2C+and+Alex+Sablayrolles%2E+On+the+importance+of+difficulty+calibration+in+membership+inference+attacks%2E+arXiv+preprint+arXiv%3A2111%2E08440%2C+2021%2E+%E2%80%94+%5BarXiv%3A2111%2E08440)
69. Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, and Reza Shokri. Enhanced membership inference attacks against machine learning models. arXiv preprint arXiv:2111.09679, 2021. — [arXiv:2111.09679](https://arxiv.org/abs/2111.09679) [find](https://scholar.google.com/scholar?q=Jiayuan+Ye%2C+Aadyaa+Maddi%2C+Sasi+Kumar+Murakonda%2C+and+Reza+Shokri%2E+Enhanced+membership+inference+attacks+against+machine+learning+models%2E+arXiv+preprint+arXiv%3A2111%2E09679%2C+2021%2E+%E2%80%94+%5BarXiv%3A2111%2E09679%5D%28http)
70. Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. [Privacy risk in machine learning: Analyzing the connection to overfitting](https://md.research.elamur.ai/free/en/papers/yeom-2017-privacy-risk-machine-learning-analyzing/). In 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pages 268–282. IEEE, 2018.
71. Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016. — [arXiv:1605.07146](https://arxiv.org/abs/1605.07146) [find](https://scholar.google.com/scholar?q=Sergey+Zagoruyko+and+Nikos+Komodakis%2E+Wide+residual+networks%2E+arXiv+preprint+arXiv%3A1605%2E07146%2C+2016%2E+%E2%80%94+%5BarXiv%3A1605%2E07146%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1605%2E07146%29)
72. Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3): 107–115, 2021. [find](https://scholar.google.com/scholar?q=Chiyuan+Zhang%2C+Samy+Bengio%2C+Moritz+Hardt%2C+Benjamin+Recht%2C+and+Oriol+Vinyals%2E+Understanding+deep+learning+%28still%29+requires+rethinking+generalization%2E+Communications+of+the+ACM%2C+64%283%29%3A+107%E2%80%93115%2C+2021%2E)
73. Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 13001–13008, 2020. [find](https://scholar.google.com/scholar?q=Zhun+Zhong%2C+Liang+Zheng%2C+Guoliang+Kang%2C+Shaozi+Li%2C+and+Yi+Yang%2E+Random+erasing+data+augmentation%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+34%2C+pages+13001%E2%80%9313008%2C+2020%2E)

*Updated 2 November 2026*