Name
Abstract | ||
|---|---|---|
Neural network pruning offers a promising prospect to facilitate deploying deep neural networks on resource-limited devices. However, existing methods are still challenged by the training inefficiency and labor cost in pruning designs, due to missing theoretical guidance of non-salient network components. In this paper, we propose a novel filter pruning method by exploring the High Rank of feature maps (HRank). Our HRank is inspired by the discovery that the average rank of multiple feature maps generated by a single filter is always the same, regardless of the number of image batches CNNs receive. Based on HRank, we develop a method that is mathematically formulated to prune filters with low-rank feature maps. The principle behind our pruning is that low-rank feature maps contain less information, and thus pruned results can be easily reproduced. Besides, we experimentally show that weights with high-rank feature maps contain more important information, such that even when a portion is not updated, very little damage would be done to the model performance. Without introducing any additional constraints, HRank leads to significant improvements over the state-of-the-arts in terms of FLOPs and parameters reduction, with similar accuracies. For example, with ResNet-110, we achieve a 58.2%-FLOPs reduction by removing 59.2% of the parameters, with only a small loss of 0.14% in top-1 accuracy on CIFAR-10. With Res-50, we achieve a 43.8%-FLOPs reduction by removing 36.7% of the parameters, with only a loss of 1.17% in the top-1 accuracy on ImageNet. The codes can be available at https://github.com/1mbxmu/HRank. |
Year | DOI | Venue |
|---|---|---|
2020 | 10.1109/CVPR42600.2020.00160 | 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) |
DocType | ISSN | Citations |
Conference | 1063-6919 | 4 |
PageRank | References | Authors |
0.42 | 24 | 7 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Mingbao Lin | 1 | 25 | 5.17 |
| Rongrong Ji | 2 | 3616 | 189.98 |
| Yan Wang | 3 | 78 | 7.92 |
| YiChen Zhang | 4 | 26 | 10.72 |
| Baochang Zhang | 5 | 1130 | 93.76 |
| Yonghong Tian | 6 | 1057 | 102.81 |
| Ling Shao | 7 | 5424 | 249.92 |