Paper Info
Title
Discrimination-aware Channel Pruning for Deep Neural Networks.
Abstract
Channel pruning is one of the predominant approaches for deep model compression. Existing pruning methods either train from scratch with sparsity constraints on channels, or minimize the reconstruction error between the pre-trained feature maps and the compressed ones. Both strategies suffer from some limitations: the former kind is computationally expensive and difficult to converge, whilst the latter kind optimizes the reconstruction error but ignores the discriminative power of channels. In this paper, we investigate a simple-yet-effective method called discrimination-aware channel pruning (DCP) to choose those channels that really contribute to discriminative power. To this end, we introduce additional discrimination-aware losses into the network to increase the discriminative power of intermediate layers and then select the most discriminative channels for each layer by considering the additional loss and the reconstruction error. Last, we propose a greedy algorithm to conduct channel selection and parameter optimization in an iterative way. Extensive experiments demonstrate the effectiveness of our method. For example, on ILSVRC-12, our pruned ResNet-50 with 30% reduction of channels outperforms the baseline model by 0.39% in top-1 accuracy.
Year
Venue
Keywords
2018
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018)
deep neural networks,computationally expensive,matrix representation
DocType
Volume
ISSN
Conference
31
1049-5258
Citations 
PageRank 
References 
18
0.55
0
Authors
8
Name
Order
Citations
PageRank
Zhuang, Zhuangwei1180.55
Tan, Mingkui2180.55
Bohan Zhuang318311.01
Liu, Jing4180.88
Yong Guo5455.94
Wu Qingyao625933.46
Junzhou Huang72182141.43
jinhui zhu8272.03