Paper Info
Title
Are Labels Required for Improving Adversarial Robustness?
Abstract
Recent work has uncovered the interesting (and somewhat surprising) finding that training models to be invariant to adversarial perturbations requires substantially larger datasets than those required for standard classification. This result is a key hurdle in the deployment of robust machine learning models in many real world applications where labeled data is expensive. Our main insight is that unlabeled data can be a competitive alternative to labeled data for training adversarially robust models. Theoretically, we show that in a simple statistical setting, the sample complexity for learning an adversarially robust model from unlabeled data matches the fully supervised case up to constant factors. On standard datasets like CIFAR-10, a simple Unsupervised Adversarial Training (UAT) approach using unlabeled data improves robust accuracy by 21.7% over using 4K supervised examples alone, and captures over 95% of the improvement from the same number of labeled examples. Finally, we report an improvement of 4% over the previous state-of-the-art on CIFAR-10 against the strongest known attack by using additional unlabeled data from the uncurated 80 Million Tiny Images dataset. This demonstrates that our finding extends as well to the more realistic case where unlabeled data is also uncurated, therefore opening a new avenue for improving adversarial training.
Year
Venue
Field
2019
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019)
Robustness (computer science),Artificial intelligence,Invariant (mathematics),Labeled data,Sample complexity,Machine learning,Mathematics,Adversarial system
DocType
Volume
ISSN
Journal
32
1049-5258
Citations 
PageRank 
References 
6
0.41
0
Authors
6
Name
Order
Citations
PageRank
Jonathan Uesato1150.90
Jean-Baptiste Alayrac2857.47
Po-Sen Huang392644.01
Robert Stanforth4737.08
Alhussein Fawzi576636.80
Pushmeet Kohli67398332.84