Name
Abstract | ||
|---|---|---|
Machine unlearning is the process through which a deployed machine learning model is made to forget about some of its training data points. While naively retraining the model from scratch is an option, it is almost always associated with large computational overheads for deep learning models. Thus, several approaches to approximately unlearn have been proposed along with corresponding metrics that formalize what it means for a model to forget about a data point. In this work, we first taxonomize approaches and metrics of approximate unlearning. As a result, we identify verification error, i.e., the
<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{2}$</tex>
difference between the weights of an approximately unlearned and a naively retrained model, as an approximate unlearning metric that should be optimized for as it subsumes a large class of other metrics. We theoretically analyze the canonical training algorithm, stochastic gradient descent (SGD), to surface the variables which are relevant to reducing the verification error of approximate unlearning for SGD. From this analysis, we first derive an easy-to-compute proxy for verification error (termed unlearning error). The analysis also informs the design of a new training objective penalty that limits the overall change in weights during SGD and as a result facilitates approximate unlearning with lower verification error. We validate our theoretical work through an empirical evaluation on learning with CIFAR-10, CIFAR-100, and IMDB sentiment analysis. |
Year | DOI | Venue |
|---|---|---|
2022 | 10.1109/EuroSP53844.2022.00027 | 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P) |
Keywords | DocType | ISBN |
deployed machine learning model,training data points,deep learning models,corresponding metrics,data point,naively retrained model,approximate unlearning metric,unlearning error,lower verification error,unrolling SGD,factors influencing machine unlearning understanding,stochastic gradient descent,verification error reduction,training objective penalty,CIFAR-10,CIFAR-100,IMDB sentiment analysis | Conference | 978-1-6654-1615-3 |
Citations | PageRank | References |
0 | 0.34 | 9 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Anvith Thudi | 1 | 3 | 0.85 |
| Gabriel Deza | 2 | 0 | 0.34 |
| Varun Chandrasekaran | 3 | 0 | 0.34 |
| Nicolas Papernot | 4 | 1932 | 87.62 |