Paper Info
Title
ClickTrain: efficient and accurate end-to-end deep learning training via fine-grained architecture-preserving pruning
Abstract
ABSTRACTConvolutional neural networks (CNNs) are becoming increasingly deeper, wider, and non-linear because of the growing demand on prediction accuracy and analysis quality. The wide and deep CNNs, however, require a large amount of computing resources and processing time. Many previous works have studied model pruning to improve inference performance, but little work has been done for effectively reducing training cost. In this paper, we propose ClickTrain: an efficient and accurate end-to-end training and pruning framework for CNNs. Different from the existing pruning-during-training work, ClickTrain provides higher model accuracy and compression ratio via fine-grained architecture-preserving pruning. By leveraging pattern-based pruning with our proposed novel accurate weight importance estimation, dynamic pattern generation and selection, and compiler-assisted computation optimizations, ClickTrain generates highly accurate and fast pruned CNN models for direct deployment without any time overhead, compared with the baseline training. ClickTrain also reduces the end-to-end time cost of the state-of-the-art pruning-after-training method by up to 2.3x with comparable accuracy and compression ratio. Moreover, compared with the state-of-the-art pruning-during-training approach, ClickTrain provides significant improvements both accuracy and compression ratio on the tested CNN models and datasets, under similar limited training time.
Year
DOI
Venue
2021
10.1145/3447818.3459988
ICS
DocType
Citations 
PageRank 
Conference
0
0.34
References 
Authors
0
11
Name
Order
Citations
PageRank
Chengming Zhang153.10
Geng Yuan273.56
Wei Niu32411.21
Jiannan Tian401.01
Sian Jin583.16
Donglin Zhuang600.34
Zhe Jiang700.34
Yanzhi Wang81082136.11
Bin Ren900.34
Shuaiwen Song1060341.87
Dingwen Tao1101.01