Name
Abstract | ||
|---|---|---|
Deep neural networks (NNs) have shown high inference performance in the field of machine learning, but at the same time, researchers require their speeding-up and miniaturization methods due to the computational complexity. Distillation is drawing attention as one of the ways to overcome this problem. NNs usually have better expression power than its learning ability. Distillation bridges the gap between expressive power and learnability by training a small NN with additional information obtained from a larger already trained NN. This gap does not exist only in neural networks but also in other machine learning methods such as support vector machine, random forest, and gradient boosting decision tree. In this research, we propose a distillation method using information extracted from NNs for non-NN models. Experimental results show that distillation can improve the accuracies of other machine learning methods, and especially, the accuracy of SVM increases by 2.80%, 90.15% to 92.95%. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/APSIPAASC47483.2019.9023120 | Asia-Pacific Signal and Information Processing Association Annual Summit and Conference |
DocType | ISSN | Citations |
Conference | 2309-9402 | 0 |
PageRank | References | Authors |
0.34 | 0 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Shota Fukui | 1 | 0 | 0.34 |
| Jaehoon Yu | 2 | 28 | 22.44 |
| Masanori Hashimoto | 3 | 462 | 79.39 |