Name
Abstract | ||
|---|---|---|
Confidentiality of training data induced by releasing machine-learning models, and has recently received increasing attention. Motivated by existing MI attacks and other previous attacks that turn out to be MI "in disguise," this paper initiates a formal study of MI attacks by presenting a game-based methodology. Our methodology uncovers a number of subtle issues, and devising a rigorous game-based definition, analogous to those in cryptography, is an interesting avenue for future work. We describe methodologies for two types of attacks. The first is for black-box attacks, which consider an adversary who infers sensitive values with only oracle access to a model. The second methodology targets the white-box scenario where an adversary has some additional knowledge about the structure of a model. For the restricted class of Boolean models and black-box attacks, we characterize model invertibility using the concept of influence from Boolean analysis in the noiseless case, and connect model invertibility with stable influence in the noisy case. Interestingly, we also discovered an intriguing phenomenon, which we call "invertibility interference," where a highly invertible model quickly becomes highly non-invertible by adding little noise. For the white-box case, we consider a common phenomenon in machine-learning models where the model is a sequential composition of several sub-models. We show, quantitatively, that even very restricted communication between layers could leak a significant amount of information. Perhaps more importantly, our study also unveils unexpected computational power of these restricted communication channels, which, to the best of our knowledge, were not previously known. |
Year | DOI | Venue |
|---|---|---|
2016 | 10.1109/CSF.2016.32 | 2016 IEEE 29th Computer Security Foundations Symposium (CSF) |
Keywords | Field | DocType |
model-inversion attack formalization,training data confidentiality,machine learning models,MI attacks,game-based methodology,rigorous game-based definition,cryptography,black-box attacks,white-box attacks,Boolean models,Boolean analysis,model invertibility,invertibility interference phenomenon | Data modeling,Computer science,Cryptography,Oracle,Theoretical computer science,Phenomenon,Adversary,Correlation attack,Information privacy,Boolean analysis | Conference |
ISSN | ISBN | Citations |
1063-6900 | 978-1-5090-2608-1 | 8 |
PageRank | References | Authors |
0.49 | 17 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Xi Wu | 1 | 419 | 26.88 |
| Matt Fredrikson | 2 | 972 | 48.56 |
| S. Jha | 3 | 7921 | 539.19 |
| Jeffrey F. Naughton | 4 | 8363 | 1913.71 |