Name
Title | ||
|---|---|---|
Supervised autoencoders - Improving generalization performance with unsupervised regularizers. |
Abstract | ||
|---|---|---|
Generalization performance is a central goal in machine learning, with explicit generalization strategies needed when training over-parametrized models, like large neural networks. There is growing interest in using multiple, potentially auxiliary tasks, as one strategy towards this goal. In this work, we theoretically and empirically analyze one such model, called a supervised auto-encoder: a neural network that jointly predicts targets and inputs (reconstruction). We provide a novel generalization result for linear auto-encoders, proving uniform stability based on the inclusion of the reconstruction error-particularly as an improvement on simplistic regularization such as norms. We then demonstrate empirically that, across an array of architectures with a different number of hidden units and activation functions, the supervised auto-encoder compared to the corresponding standard neural network never harms performance and can improve generalization. |
Year | Venue | Keywords |
|---|---|---|
2018 | ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018) | machine learning,neural network |
Field | DocType | Volume |
Computer science,Neural network architecture,Reconstruction error,Regularization (mathematics),Artificial intelligence,Artificial neural network,Machine learning | Conference | 31 |
ISSN | Citations | PageRank |
1049-5258 | 3 | 0.39 |
References | Authors | |
0 | 3 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Lei Le | 1 | 8 | 1.86 |
| Andrew Patterson | 2 | 5 | 4.52 |
| Martha White | 3 | 198 | 27.75 |