Paper Info
Title
Associative Convolutional Layers
Abstract
We provide a general and easy to implement method for reducing the number of parameters of Convolutional Neural Networks (CNNs) during the training and inference phases. We introduce a simple trainable auxiliary neural network which can generate approximate versions of \slices" of the sets of convolutional filters of any CNN architecture from a low dimensional\code" space. These slices are then concatenated to form the sets of filters in the CNN architecture. The auxiliary neural network, which we call\Convolutional Slice Generator" (CSG), is unique to the network and provides the association among its convolutional layers. We apply our method to various CNN architectures including ResNet, DenseNet, MobileNet and ShueNet. Experiments on CIFAR-10 and ImageNet-1000, without any hyper-parameter tuning, show that our approach reduces the network parameters by approximately 2x while the reduction in accuracy is confined to within one percent and sometimes the accuracy even improves after compression. Interestingly, through our experiments, we show that even when the CSG takes random binary values for its weights that are not learned, still acceptable performances are achieved. To show that our approach generalizes to other tasks, we apply it to an image segmentation architecture, Deeplab V3, on the Pascal VOC 2012 dataset. Results show that without any parameter tuning, there is approximate to 2.3x parameter reduction and the mean Intersection over Union (mIoU) drops by approximate to 3%. Finally, we provide comparisons with several related methods showing the superiority of our method in terms of accuracy.
Year
Venue
DocType
2019
24TH INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS (AISTATS)
Journal
Volume
ISSN
Citations 
130
2640-3498
0
PageRank 
References 
Authors
0.34
0
4
Name
Order
Citations
PageRank
Hamed Omidvar112.08
Akhlaghi, Vahideh2172.63
Massimo Franceschetti32200167.33
Rajesh K. Gupta44570390.84