Name
Abstract | ||
|---|---|---|
Recent advances in convolutional neural networks have considered model complexity and hardware efficiency to enable deployment onto embedded systems and mobile devices. For example, it is now well-known that the arithmetic operations of deep networks can be encoded down to 8-bit fixed-point without significant deterioration in performance. However, further reduction in precision down to as low as 3-bit fixed-point results in significant losses in performance. In this paper we propose a new data representation that enables state-of-the-art networks to be encoded to 3 bits with negligible loss in classification performance. To perform this, we take advantage of the fact that the weights and activations in a trained network naturally have non-uniform distributions. Using non-uniform, base-2 logarithmic representation to encode weights, communicate activations, and perform dot-products enables networks to 1) achieve higher classification accuracies than fixed-point at the same resolution and 2) eliminate bulky digital multipliers. Finally, we propose an end-to-end training procedure that uses log representation at 5-bits, which achieves higher final test accuracy than linear at 5-bits. |
Year | Venue | Field |
|---|---|---|
2016 | arXiv: Neural and Evolutionary Computing | ENCODE,External Data Representation,Computer science,Convolutional neural network,Algorithm,Mobile device,Artificial intelligence,Logarithm,Machine learning,Model complexity |
DocType | Volume | Citations |
Journal | abs/1603.01025 | 38 |
PageRank | References | Authors |
1.45 | 19 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Daisuke Miyashita | 1 | 72 | 9.99 |
| Edward Lee | 2 | 66 | 9.90 |
| Boris Murmann | 3 | 594 | 82.64 |