Name
Abstract | ||
|---|---|---|
We consider compression of deep neural networks (DNNs) by weight quantization and lossless source coding for memory-efficient deployment. Whereas the previous work addressed non-universal scalar quantization and entropy source coding, we for the first time introduce universal DNN compression by universal vector quantization and universal source coding. In particular, the proposed scheme utilizes universal lattice quantization, which randomizes the source by uniform random dithering before lattice quantization and can perform near-optimally on any source without relying on knowledge of the source distribution. Moreover, we present a method of fine-tuning vector quantized DNNs to recover any accuracy loss due to quantization. From our experiments, we show that the proposed scheme compresses the MobileNet and ShuffleNet models trained on ImageNet with the state-of-the-art compression ratios of 10.7 and 8.8, respectively. |
Year | DOI | Venue |
|---|---|---|
2018 | 10.1109/JSTSP.2020.2975903 | IEEE Journal of Selected Topics in Signal Processing |
Keywords | Field | DocType |
Deep neural networks,lossy compression,universal compression,entropy coded vector quantization,universal quantization | Entropy encoding,Lossy compression,Pattern recognition,Computer science,Vector quantization,Huffman coding,Artificial intelligence,Quantization (physics),Dither,Artificial neural network,Quantization (signal processing) | Journal |
Volume | Issue | ISSN |
14 | 4 | 1932-4553 |
Citations | PageRank | References |
6 | 0.49 | 43 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Yoo-Jin Choi | 1 | 18 | 4.87 |
| El-Khamy Mostafa | 2 | 264 | 28.10 |
| Jungwon Lee | 3 | 890 | 95.15 |