---
title: "Membership Inference Attacks on Sequence-to-Sequence"
description: "The paper asks whether someone with only black-box API access to a machine translation service… 29 references, cited by 23 papers here."
authors: ["Sorami Hisamoto", "Matt Post", "Kevin Duh"]
year: 2020
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2020.tacl-1.4/"
doi: "10.1162/tacl_a_00299"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hisamoto-2020-membership-inference-attacks-sequence-sequence/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hisamoto-2020-membership-inference-attacks-sequence-sequence/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/) / Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?

# Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?

**Authors:** [Sorami Hisamoto](https://md.research.elamur.ai/free/en/authors/sorami-hisamoto/), [Matt Post](https://md.research.elamur.ai/free/en/authors/matt-post/) and [Kevin Duh](https://md.research.elamur.ai/free/en/authors/kevin-duh/) · **Year:** 2020 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2020](https://md.research.elamur.ai/free/en/venues/tacl/2020/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.tacl-1.4/) · [PDF](https://aclanthology.org/2020.tacl-1.4.pdf) · [DOI 10.1162/tacl\_a\_00299](https://doi.org/10.1162/tacl_a_00299) · [OpenAlex](https://openalex.org/W3011159643)

**Citations:** 59 citations (OpenAlex, 2026-10-14) · [23](https://md.research.elamur.ai/free/en/papers/hisamoto-2020-membership-inference-attacks-sequence-sequence/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether someone with only black-box API access to a machine translation service can tell if a specific sentence pair was used to train it. Membership inference had previously been studied for classifiers with fixed label sets. The authors extend it to sequence-to-sequence generation, where outputs are variable-length word sequences. They frame the task as a game. Alice trains and serves the model, Bob attacks it using probes made of a source sentence and its reference translation, and Carol, a neutral organizer, builds disjoint in-probe, out-probe and out-of-domain probe sets from parallel subcorpora. Bob trains shadow MT models on partly shared data to mimic Alice, then builds attack classifiers from Alice's outputs. The main finding favors the defender: simple shadow-model attacks, which succeed on classifiers, barely beat chance at the sentence level. Some leakage remains, however, for unusual inputs and for groups of sentences. The work provides a formal problem definition and an open benchmark for privacy auditing of "ML as a service" translation and other generation systems.

## Key points
- The setup follows Shokri et al. (2017) but adapts it to sequence generation. Bob makes one API call per probe and decides in or out from the source, the reference and Alice's translation (optionally also translation probabilities).
- Data is split by subcorpus (examples given include Europarl and CommonCrawl). One subcorpus acts as Alice's private in-house data, and a third subcorpus supplies the out-of-domain probe. The dataset is released at github.com/sorami/tacl-membership.
- Sentence-level attacks are close to chance. Accuracy is above chance on out-of-domain and out-of-vocabulary data, and when membership is judged over groups of sentences.
- External features did not help significantly. These included sentence scores from a quality estimation model (sentence-level RMSE 0.1745) and fine-tuned BERT used as a classifier.
- Caveats:
  - Alice's and Bob's models reached similar BLEU, which favors Bob.
  - Bob's accuracy on his own shadow models is likely an optimistic upper bound.
  - Stronger attacks (repeated or perturbed queries, watermark sentences, memorization metrics from Carlini et al., 2018) remain future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Johns Hopkins University](https://md.research.elamur.ai/free/en/universities/johns-hopkins-university/) — University, Baltimore, United States · 240 works · 96,487 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 1,087 works

## Cite

Cite as: Sorami Hisamoto, Matt Post and Kevin Duh. 2020. Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{hisamoto2020membership,
  title = {{Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?}},
  author = {Sorami Hisamoto, Matt Post and Kevin Duh},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.1162/tacl\_a\_00299},
  url = {https://doi.org/10.1162/tacl_a_00299}
}
```

- [← previous Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness](https://md.research.elamur.ai/free/en/papers/kim-2020-will-i-sound-like-me/)
- [next → Stance Prediction and Claim Verification: An Arabic Perspective](https://md.research.elamur.ai/free/en/papers/khouja-2020-stance-prediction-claim-verification-arabic/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/)

## References

29 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Kevin Duh. 2018. The multitarget TED talks task. [http://www.cs.jhu.edu/∼kevinduh/](http://www.cs.jhu.edu/∼kevinduh/) a/multitarget-tedtalks/. [↗](http://www.cs.jhu.edu/∼kevinduh/)
2. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. 2017. [find](https://scholar.google.com/scholar?q=Ondr%CC%8Cej+Bojar%2C+Rajen+Chatterjee%2C+Christian+Federmann%2C+Yvette+Graham%2C+Barry+Haddow%2C+Shujian+Huang%2C+Matthias+Huck%2C+Philipp+Koehn%2C+Qun+Liu%2C+Varvara+Logacheva%2C+Christof+Monz%2C+Matteo+Negri%2C+Matt+Post%2C+Raph)
3. Find-Cynthia Dwork. 2008. Differential privacy: A survey of results. Manindra Agrawal, Dingzhu Du, Zhenhua Duan, and Angsheng Li, editors, In Theory and Applications of Models of Computation, pages 1–19. Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Find%2DCynthia+Dwork%2E+2008%2E+Differential+privacy%3A+A+survey+of+results%2E+Manindra+Agrawal%2C+Dingzhu+Du%2C+Zhenhua+Duan%2C+and+Angsheng+Li%2C+editors%2C+In+Theory+and+Applications+of+Models+of+Computation%2C+pages+1%E2%80%93)
4. Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A. Gunter, and Kai Chen. 2018. Understanding membership inferences on well-generalized learning models. CoRR, abs/1802.04889. — [arXiv:1802.04889](https://arxiv.org/abs/1802.04889) [find](https://scholar.google.com/scholar?q=Yunhui+Long%2C+Vincent+Bindschaedler%2C+Lei+Wang%2C+Diyue+Bu%2C+Xiaofeng+Wang%2C+Haixu+Tang%2C+Carl+A%2E+Gunter%2C+and+Kai+Chen%2E+2018%2E+Understanding+membership+inferences+on+well%2Dgeneralized+learning+models%2E+CoRR%2C+ab)
5. Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro. 2017. LOGAN: Evaluating privacy leakage of generative models using generative adversarial networks. CoRR, abs/1705.07663. — [arXiv:1705.07663](https://arxiv.org/abs/1705.07663) [find](https://scholar.google.com/scholar?q=Jamie+Hayes%2C+Luca+Melis%2C+George+Danezis%2C+and+Emiliano+De+Cristofaro%2E+2017%2E+LOGAN%3A+Evaluating+privacy+leakage+of+generative+models+using+generative+adversarial+networks%2E+CoRR%2C+abs%2F1705%2E07663%2E+%E2%80%94+%5BarXiv%3A)
6. Ashwin Machanavajjhala, Xi He, and Michael Hay. 2017. Differential privacy in the wild: A tutorial on current practices & open challenges. In Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD ’17, pages 1727–1730, New York, NY. ACM. [find](https://scholar.google.com/scholar?q=Ashwin+Machanavajjhala%2C+Xi+He%2C+and+Michael+Hay%2E+2017%2E+Differential+privacy+in+the+wild%3A+A+tutorial+on+current+practices+%26+open+challenges%2E+In+Proceedings+of+the+2017+ACM+International+Conference+on+Ma)
7. Felix Hieber, Tobias Domhan, Michael Denkowski, David Vilar, Artem Sokolov, Ann Clifton, and Matt Post. 2017. Sockeye: A toolkit for neural machine translation. CoRR, abs/1712.05690. [find](https://scholar.google.com/scholar?q=Felix+Hieber%2C+Tobias+Domhan%2C+Michael+Denkowski%2C+David+Vilar%2C+Artem+Sokolov%2C+Ann+Clifton%2C+and+Matt+Post%2E+2017%2E+Sockeye%3A+A+toolkit+for+neural+machine+translation%2E+CoRR%2C+abs%2F1712%2E05690%2E)
8. Milad Nasr, Reza Shokri, and Amir Houmansadr. 2018. Machine learning with membership privacy using adversarial regularization. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS ’18, pages 634–646, New York, NY. ACM. [find](https://scholar.google.com/scholar?q=Milad+Nasr%2C+Reza+Shokri%2C+and+Amir+Houmansadr%2E+2018%2E+Machine+learning+with+membership+privacy+using+adversarial+regularization%2E+In+Proceedings+of+the+2018+ACM+SIGSAC+Conference+on+Computer+and+Communic)
9. Marcin Junczys-Dowmunt. 2018. [Dual conditional cross-entropy filtering of noisy parallel corpora](https://md.research.elamur.ai/free/en/papers/dowmunt-2018-dual-conditional-cross-entropy-filtering/). In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 888–895, Belgium, Brussels. Association for Computational Linguistics.
10. Milad Nasr, Reza Shokri, and Amir Houmansadr. 2019. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE Symposium on Security and Privacy (SP). [find](https://scholar.google.com/scholar?q=Milad+Nasr%2C+Reza+Shokri%2C+and+Amir+Houmansadr%2E+2019%2E+Comprehensive+privacy+analysis+of+deep+learning%3A+Passive+and+active+white%2Dbox+inference+attacks+against+centralized+and+federated+learning%2E+In+2019+)
11. Fabio Kepler, Jonay Trénous, Marcos Treviso, Miguel Vera, and André F. T. Martins. 2019. [OpenKiwi: An open source framework for quality estimation](https://md.research.elamur.ai/free/en/papers/kepler-2019-openkiwi-open-source-framework-quality/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 117–122, Florence, Italy. Ass…
12. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002, Jul. [BLEU: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, PA. Association for Computational Linguistics.
13. Hyun Kim, Jong-Hyeok Lee, and Seung-Hoon Na. 2017. [Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation](https://md.research.elamur.ai/free/en/papers/kim-2017-predictor-estimator-multilevel-task-learning/). In Proceedings of the Second Conference on Machine Translation, pages 562–568, Copenhagen, Denmark. Association for Computational Linguistics.
14. Matt Post. 2018, Oct. [A call for clarity in reporting BLEU scores](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/). In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Belgium, Brussels. Association for Computational Linguistics.
15. Diederik P. Kingma and Jimmy Ba. 2015. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). CoRR, abs/1412.6980.
16. Apostolos Pyrgelis, Carmela Troncoso, and Emiliano De Cristofaro. 2018. Knock knock, who’s there? Membership inference on aggregate location data. CoRR, abs/1708.06145. — [arXiv:1708.06145](https://arxiv.org/abs/1708.06145) [find](https://scholar.google.com/scholar?q=Apostolos+Pyrgelis%2C+Carmela+Troncoso%2C+and+Emiliano+De+Cristofaro%2E+2018%2E+Knock+knock%2C+who%E2%80%99s+there%3F+Membership+inference+on+aggregate+location+data%2E+CoRR%2C+abs%2F1708%2E06145%2E+%E2%80%94+%5BarXiv%3A1708%2E06145%5D%28https%3A%2F%2Far)
17. Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation. [find](https://scholar.google.com/scholar?q=Philipp+Koehn+and+Rebecca+Knowles%2E+2017%2E+Six+challenges+for+neural+machine+translation%2E+In+Proceedings+of+the+First+Workshop+on+Neural+Machine+Translation%2E)
18. Md Atiqur Rahman, Tanzila Rahman, Robert Laganiere, Noman Mohammed, and Yang Wang. 2018. Membership inference attack against differentially private deep learning model. Transactions on Data Privacy, 11:61–79. [find](https://scholar.google.com/scholar?q=Md+Atiqur+Rahman%2C+Tanzila+Rahman%2C+Robert+Laganiere%2C+Noman+Mohammed%2C+and+Yang+Wang%2E+2018%2E+Membership+inference+attack+against+differentially+private+deep+learning+model%2E+Transactions+on+Data+Privacy%2C+1)
19. Chin-Yew Lin and Franz Josef Och. 2004. [Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics](https://md.research.elamur.ai/free/en/papers/lin-2004-automatic-evaluation-machine-translation-quality/). In Proceedings of the 42nd Meeting of the Association for Computational Linguistics (ACL’04), Main Volume, pages 605–612, Barcelona, Spain.
20. Ahmed Salem, Yonghui Zhang, Mathias Humbert, Mario Fritz, and Michael Backes. 2018. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. CoRR, abs/1806.01246. — [arXiv:1806.01246](https://arxiv.org/abs/1806.01246) [find](https://scholar.google.com/scholar?q=Ahmed+Salem%2C+Yonghui+Zhang%2C+Mathias+Humbert%2C+Mario+Fritz%2C+and+Michael+Backes%2E+2018%2E+Ml%2Dleaks%3A+Model+and+data+independent+membership+inference+attacks+and+defenses+on+machine+learning+models%2E+CoRR%2C+abs)
21. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in opus. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+opus%2E+In+Proceedings+of+the+Eight+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9912%29%2C+Istanbul%2C+Turkey%2E+European+Lang)
22. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Lin…
23. Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei. 2018. Towards demystifying membership inference attacks. CoRR, abs/1807.09173. — [arXiv:1807.09173](https://arxiv.org/abs/1807.09173) [find](https://scholar.google.com/scholar?q=Stacey+Truex%2C+Ling+Liu%2C+Mehmet+Emre+Gursoy%2C+Lei+Yu%2C+and+Wenqi+Wei%2E+2018%2E+Towards+demystifying+membership+inference+attacks%2E+CoRR%2C+abs%2F1807%2E09173%2E+%E2%80%94+%5BarXiv%3A1807%2E09173%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1807%2E09173%29)
24. R. Shokri, M. Stronati, C. Song, and V. Shmatikov. 2017. [Membership inference attacks against machine learning models](https://md.research.elamur.ai/free/en/papers/shokri-2016-membership-inference-attacks-against-machine/). In 2017 IEEE Symposium on Security and Privacy (SP), pages 3–18.
25. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces…
26. Congzheng Song and Vitaly Shmatikov. 2019. Auditing data provenance in text-generation models. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, pages 196–206. New York, NY, ACM. [find](https://scholar.google.com/scholar?q=Congzheng+Song+and+Vitaly+Shmatikov%2E+2019%2E+Auditing+data+provenance+in+text%2Dgeneration+models%2E+In+Proceedings+of+the+25th+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+%26+Data+Mining%2C+KDD+)
27. Michael Veale, Reuben Binns, and Lilian Edwards. 2018. Algorithms that remember: Model inversion attacks and data protection law. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, 376. [find](https://scholar.google.com/scholar?q=Michael+Veale%2C+Reuben+Binns%2C+and+Lilian+Edwards%2E+2018%2E+Algorithms+that+remember%3A+Model+inversion+attacks+and+data+protection+law%2E+Philosophical+Transactions%2E+Series+A%2C+Mathematical%2C+Physical%2C+and+Engi)
28. Lucia Specia, Varvara Logacheva, Frederic Blain, Ramon Fernandez, and André Martins. 2018. WMT18 quality estimation shared task training and development data. LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles Univer… [find](https://scholar.google.com/scholar?q=Lucia+Specia%2C+Varvara+Logacheva%2C+Frederic+Blain%2C+Ramon+Fernandez%2C+and+Andre%CC%81+Martins%2E+2018%2E+WMT18+quality+estimation+shared+task+training+and+development+data%2E+LINDAT%2FCLARIN+digital+library+at+the+Ins)
29. Samuel Yeom, Matt Fredrikson, and Somesh Jha. 2017. [The unintended consequences of overfitting: Training data inference attacks](https://md.research.elamur.ai/free/en/papers/yeom-2017-privacy-risk-machine-learning-analyzing/). CoRR, abs/1709.01604.

*Updated 30 October 2026*