Title
Functional Hashing for Compressing Neural Networks.
Abstract
As the complexity of deep neural networks (DNNs) trend to grow to absorb the increasing sizes of data, memory and energy consumption has been receiving more and more attentions for industrial applications, especially on mobile devices. This paper presents a novel structure based on functional hashing to compress DNNs, namely FunHashNN. For each entry in a deep net, FunHashNN uses multiple low-cost hash functions to fetch values in the compression space, and then employs a small reconstruction network to recover that entry. The reconstruction network is plugged into the whole network and trained jointly. FunHashNN includes the recently proposed HashedNets as a degenerated case, and benefits from larger value capacity and less reconstruction loss. We further discuss extensions with dual space hashing and multi-hops. On several benchmark datasets, FunHashNN demonstrates high compression ratios with little loss on prediction accuracy.
Year
Venue
Field
2016
arXiv: Learning
Data mining,Computer science,Dual space,Compression ratio,Mobile device,Artificial intelligence,Hash function,Artificial neural network,Energy consumption,Machine learning,Deep neural networks,Fold (higher-order function)
DocType
Volume
Citations 
Journal
abs/1605.06560
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Lei Shi11106.76
Shikun Feng253.78
Zhifan Zhu300.34