Name
Abstract | ||
|---|---|---|
Deep learning-based techniques have achieved state-of-the-art performance on a wide variety of recognition and classification tasks. However, these networks are typically computationally expensive to train, requiring weeks of computation on many GPUs; as a result, many users outsource the training procedure to the cloud or rely on pre-trained models that are then fine-tuned for a specific task. In this paper, we show that the outsourced training introduces new security risks: an adversary can create a maliciously trained network (a backdoored neural network, or a BadNet) that has the state-of-the-art performance on the user's training and validation samples but behaves badly on specific attacker-chosen inputs. We first explore the properties of BadNets in a toy example, by creating a backdoored handwritten digit classifier. Next, we demonstrate backdoors in a more realistic scenario by creating a U.S. street sign classifier that identifies stop signs as speed limits when a special sticker is added to the stop sign; we then show in addition that the backdoor in our U.S. street sign detector can persist even if the network is later retrained for another task and cause a drop in an accuracy of 25% on average when the backdoor trigger is present. These results demonstrate that backdoors in neural networks are both powerful and-because the behavior of neural networks is difficult to explicate-stealthy. This paper provides motivation for further research into techniques for verifying and inspecting neural networks, just as we have developed tools for verifying and debugging software. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/ACCESS.2019.2909068 | IEEE ACCESS |
Keywords | Field | DocType |
Computer security,machine learning,neural networks | Stop sign,Computer science,Software,Backdoor,Artificial intelligence,Deep learning,Artificial neural network,Classifier (linguistics),Machine learning,Distributed computing,Cloud computing,Debugging | Journal |
Volume | ISSN | Citations |
7 | 2169-3536 | 8 |
PageRank | References | Authors |
0.55 | 0 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Tianyu Gu | 1 | 82 | 5.45 |
| Kang Liu | 2 | 8 | 1.57 |
| Brendan Dolan-Gavitt | 3 | 368 | 19.94 |
| Siddharth Garg | 4 | 675 | 55.14 |