Name
Abstract | ||
|---|---|---|
Low bit-width Quantized Neural Networks (QNNs) enable deployment of complex machine learning models on constrained devices such as microcontrollers (MCUs) by reducing their memory footprint. Fine-grained asymmetric quantization (i.e., different bit-widths assigned to weights and activations on a tensor-by-tensor basis) is a particularly interesting scheme to maximize accuracy under a tight memory constraint. However, the lack of sub-byte instruction set architecture (ISA) support in SoA microprocessors makes it hard to fully exploit this extreme quantization paradigm in embedded MCUs. Support for sub-byte and asymmetric QNNs would require many precision formats and an exorbitant amount of opcode space. In this work, we attack this problem with status-based SIMD instructions: rather than encoding precision explicitly, each operand's precision is set dynamically in a core status register. We propose a novel RISC-V ISA core MPIC (Mixed Precision Inference Core) based on the open-source RI5CY core. Our approach enables full support for mixed-precision QNN inference with 292 different combinations of operands at 16-, 8-, 4- and 2-bit precision, without adding any extra opcode or increasing the complexity of the decode stage. Our results show that MPIC improves both performance and energy efficiency by a factor of 1.1-4.9x when compared to software-based mixed-precision on RI5CY; with respect to commercially available Cortex-M4 and M7 microcontrollers, it delivers 3.6-11.7x better performance and 41-155x higher efficiency. |
Year | DOI | Venue |
|---|---|---|
2020 | 10.1109/ISVLSI49217.2020.000-5 | 2020 IEEE Computer Society Annual Symposium on VLSI (ISVLSI) |
Keywords | DocType | ISSN |
PULP Platform,Embedded-Systems,Deep Neural Networks,Mixed precision,Microcontroller | Conference | 2159-3469 |
ISBN | Citations | PageRank |
978-1-7281-5776-4 | 1 | 0.37 |
References | Authors | |
5 | 6 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Gianmarco Ottavi | 1 | 3 | 0.75 |
| Angelo Garofalo | 2 | 4 | 2.47 |
| Giuseppe Tagliavini | 3 | 1 | 0.37 |
| Francesco Lo Conti | 4 | 5 | 5.20 |
| Luca Benini | 5 | 13116 | 1188.49 |
| Davide Rossi | 6 | 131 | 13.94 |