Name
Abstract | ||
|---|---|---|
Unsupervised learning of feature hierarchies is often a good initialization for supervised training of deep architectures. In existing deep learning methods, these feature hierarchies are built layer by layer in a greedy fashion using auto-encoders or restricted Boltzmann machines. Both yield encoders, which compute linear projections followed by a smooth thresholding function. In this work, we demonstrate that these encoders fail to find stable features when the required computation is in the exclusive-or class. To overcome this limitation, we propose a two-layer encoder which is not restricted in the type of features it can learn. The proposed encoder can be regularized by an extension of previous work on contractive regularization. We demonstrate the advantages of two-layer encoders qualitatively, as well as on commonly used benchmark datasets. |
Year | DOI | Venue |
|---|---|---|
2012 | 10.1007/978-3-642-33269-2_78 | ICANN (1) |
Keywords | Field | DocType |
unsupervised learning,yield encoders,restricted boltzmann machine,two-layer contractive encodings,two-layer encoder,deep architecture,stable feature,two-layer encoders qualitatively,feature hierarchy,previous work,proposed encoder | Restricted Boltzmann machine,Boltzmann machine,Computer science,Theoretical computer science,Unsupervised learning,Regularization (mathematics),Encoder,Artificial intelligence,Thresholding,Initialization,Deep learning,Machine learning | Conference |
Citations | PageRank | References |
2 | 0.38 | 14 |
Authors | ||
2 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Hannes Schulz | 1 | 55 | 10.82 |
| Sven Behnke | 2 | 1672 | 181.84 |