Paper Info
Title
Accurate Inference with Inaccurate RRAM Devices: Statistical Data, Model Transfer, and On-line Adaptation
Abstract
Resistive random-access memory (RRAM) is a promising technology for in-memory computing with high storage density, fast inference, and good compatibility with CMOS. However, the mapping of a pre-trained deep neural network (DNN) model on RRAM suffers from realistic device issues, especially the variation and quantization error, resulting in a significant reduction in inference accuracy. In this work, we first extract these statistical properties from 65 nm RRAM data on 300mm wafers. The RRAM data present 10-levels in quantization and 50% variance, resulting in an accuracy drop to 31.76% and 10.49% for MNIST and CIFAR-10 datasets, respectively. Based on the experimental data, we propose a combination of machine learning algorithms and on-line adaptation to recover the accuracy with the minimum overhead. The recipe first applies Knowledge Distillation (KD) to transfer an ideal model into a student model with statistical variations and 10 levels. Furthermore, an on-line sparse adaptation (OSA) method is applied to the DNN model mapped on to the RRAM array. Using importance sampling, OSA adds a small SRAM array that is sparsely connected to the main RRAM array; only this SRAM array is updated to recover the accuracy. As demonstrated on MNIST and CIFAR-10 datasets, a 7.86% area cost is sufficient to achieve baseline accuracy for the 65 nm RRAM devices.
Year
DOI
Venue
2020
10.1109/DAC18072.2020.9218605
2020 57th ACM/IEEE Design Automation Conference (DAC)
Keywords
DocType
ISSN
Robustness,in-memory computing,Resistive random access memory (RRAM),Knowledge Distillation,on-line adaptation
Conference
0738-100X
ISBN
Citations 
PageRank 
978-1-7281-1085-1
5
0.46
References 
Authors
0
8
Name
Order
Citations
PageRank
Gouranga Charan191.87
Jubin Hazra262.56
Karsten Beckmann3184.70
Xiaocong Du4195.25
Gokul Krishnan5247.77
Rajiv V. Joshi626064.87
Nathaniel C. Cady751.48
Yu Cao82765245.91