Name
Title | ||
|---|---|---|
Learning Sparse Convolutional Neural Network via Quantization With Low Rank Regularization |
Abstract | ||
|---|---|---|
With the refinement of tasks in artificial intelligence, bringing in exponential level increments in computation cost and storage. Therefore, the augment of computation resource for complicated neural networks severely hinders their applications on limited-power devices in recent years. As a result, there is an impending necessity to compress and accelerate the deep networks by special ways. Considering the different peculiarities of weight quantization and sparse regularization, in this paper, we propose a low rank sparse quantization (LRSQ) method to quantize network weights and regularize the corresponding structures at the same time. Our LRSQ can: 1) obtain low-bit quantized networks to reduce memory and computation cost and 2) learn a compact structure from complex convolutional networks for subsequent channel pruning which has significant reduction on FLOPs. In experimental sections, we evaluate the proposed method on several popular models such as VGG-7/16/19 and ResNet-18/34/50, and results show that this method can dramatically reduce parameters and channels of the network with slight inference accuracy loss. Furthermore, we also visualize and analyze the four-dimensional weight tensors, which shows the low rank and group-sparsity structure of it. Finally, we try pruning unimportant channels which are zero-channels in our quantized model, and finding even a little better precision than the standard full-precision network. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/ACCESS.2019.2911536 | IEEE ACCESS |
Keywords | Field | DocType |
Convolutional neural network (CNN),weight quantization,spectral regularization,sparsity,visualization,channel pruning | Convolutional neural network,Computer science,FLOPS,Algorithm,Communication channel,Regularization (mathematics),Quantization (physics),Quantization (signal processing),Artificial neural network,Distributed computing,Computation | Journal |
Volume | ISSN | Citations |
7 | 2169-3536 | 0 |
PageRank | References | Authors |
0.34 | 0 | 6 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Xin Long | 1 | 0 | 0.34 |
| Zongcheng Ben | 2 | 0 | 0.34 |
| Xiangrong Zeng | 3 | 10 | 5.20 |
| Yan Liu | 4 | 241 | 73.08 |
| Maojun Zhang | 5 | 314 | 48.74 |
| Dianle Zhou | 6 | 9 | 0.92 |