Name
Abstract | ||
|---|---|---|
Transfer learning is a process that allows reusing a learning machine trained on a problem to solve a new problem. Transfer learning studies on shallow architectures show low performance as they are generally based on hand-crafted features obtained from experts. It is therefore interesting to study transference on deep architectures, known to directly extract the features from the input data. A Stacked Denoising Autoencoder (SDA) is a deep model able to represent the hierarchical features needed for solving classification problems. In this paper we study the performance of SDAs trained on one problem and reused to solve a different problem not only with different distribution but also with a different tasks. We propose two different approaches: 1) unsupervised feature transference, and 2) supervised feature transference using deep transfer learning. We show that SDAs using the unsupervised feature transference outperform randomly initialized machines on a new problem. We achieved 7% relative improvement on average error rate and 41% on average computation time to classify typed uppercase letters. In the case of supervised feature transference, we achieved 5.7% relative improvement in the average error rate, by reusing the first and second hidden layer, and 8.5% relative improvement for the average error rate and 54% speed up w.r.t the baseline by reusing all three hidden layers for the same data. We also explore transfer learning between geometrical shapes and canonical shapes, we achieved 7.4% relative improvement on average error rate in case of supervised feature transference approach. |
Year | DOI | Venue |
|---|---|---|
2014 | 10.1109/SMC.2014.6974107 | Systems, Man and Cybernetics |
Keywords | DocType | ISSN |
computational geometry,feature extraction,image classification,image representation,neural nets,unsupervised learning,SDA,canonical shapes,deep transfer learning accuracy improvement,feature extraction,geometrical shapes,hierarchical feature representation,learning machine,stacked denoising autoencoder reuse,supervised feature transference,typed uppercase letter classification,unsupervised feature transference,Deep Learning,Transfer Learning | Conference | 1062-922X |
Citations | PageRank | References |
7 | 0.57 | 13 |
Authors | ||
6 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Chetak Kandaswamy | 1 | 46 | 4.51 |
| Luís M. Silva | 2 | 88 | 9.02 |
| Luís A. Alexandre | 3 | 703 | 47.66 |
| Ricardo Sousa | 4 | 95 | 12.68 |
| Jorge M. Santos | 5 | 123 | 11.75 |
| Joaquim Marques de Sá | 6 | 72 | 9.04 |