Name
Abstract | ||
|---|---|---|
We describe a novel unsupervised method for learning sparse, overcomplete fea- tures. The model uses a linear encoder, and a linear decoder preceded by a spar- sifying non-linearity that turns a code vector into a quasi- binary sparse code vec- tor. Given an input, the optimal code minimizes the distance between the output of the decoder and the input patch while being as similar as possible to the en- coder output. Learning proceeds in a two-phase EM-like fashion: (1) compute the minimum-energy code vector, (2) adjust the parameters of the encoder and de- coder so as to decrease the energy. The model produces "stroke detectors" when trained on handwritten numerals, and Gabor-like filters whe n trained on natural image patches. Inference and learning are very fast, requiring no preprocessing, and no expensive sampling. Using the proposed unsupervised method to initialize the first layer of a convolutional network, we achieved an err or rate slightly lower than the best reported result on the MNIST dataset. Finally, an extension of the method is described to learn topographical filter maps. |
Year | Venue | Keywords |
|---|---|---|
2006 | NIPS | sparse representation,sparse coding |
Field | DocType | Citations |
MNIST database,Pattern recognition,Computer science,Word error rate,Sparse approximation,Preprocessor,Artificial intelligence,Linear encoder,Encoder,Detector,Code (cryptography),Machine learning | Conference | 365 |
PageRank | References | Authors |
59.48 | 8 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Marc'Aurelio Ranzato | 1 | 5242 | 470.94 |
| Christopher S. Poultney | 2 | 368 | 60.44 |
| Sumit Chopra | 3 | 2835 | 181.37 |
| Yann LeCun | 4 | 26090 | 3771.21 |