Name
Title | ||
|---|---|---|
Towards Energy Efficient DNN accelerator via Sparsified Gradual Knowledge Distillation |
Abstract | ||
|---|---|---|
Artificial intelligence (AI) is becoming increasingly popular in many applications. However, the computation cost of deep neural network (DNN) , which is a powerful form of AI, calls for efficient DNN compression technique to make energy efficient networks. In this paper, we proposed SKG, a method to jointly sparsify and quantize DNN models to ultra-low bit-precision using Knowledge Distillation and gradual quantization (SKG). We demonstrated that our method can preserve the accuracy more than 20% for uniform quantization with 2 bit-width compared to the baseline methods on ImageNet and ResNet-18. In addition, our method can achieve up to 2.7x lower energy consumption using compute-in-memory (CIM) architecture compared to a traditional 65nm CMOS architecture for both pruned and unpruned network during inference and eventually enabling using DNN models on resource constrained edge devices. |
Year | DOI | Venue |
|---|---|---|
2022 | 10.1109/VLSI-SoC54400.2022.9939619 | 2022 IFIP/IEEE 30th International Conference on Very Large Scale Integration (VLSI-SoC) |
Keywords | DocType | ISSN |
DNN model,Knowledge distillation,DNN compression,quantization,low bit precision | Conference | 2324-8432 |
ISBN | Citations | PageRank |
978-1-6654-9006-1 | 0 | 0.34 |
References | Authors | |
14 | 2 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Foroozan Karimzadeh | 1 | 0 | 0.68 |
| Arijit Raychowdhury | 2 | 284 | 48.04 |