Paper Info
Title
Stateful Neural Networks for Intermittent Systems
Abstract
Deep neural network (DNN) inference on intermittently powered battery-less devices has the potential to unlock new possibilities for sustainable and intelligent edge applications. Existing intermittent inference approaches preserve progress information separate from the computed output features during inference. However, we observe that even in highly specialized approaches, the additional overhead incurred for inference progress preservation still accounts for a significant portion of the inference latency. This work proposes the concept of stateful neural networks, which enables a DNN to indicate the inference progress itself. Our runtime middleware embeds state information into the DNN such that the computed and preserved output features intrinsically contain progress indicators, avoiding the need to preserve them separately. The specific position and representation of the embedded states jointly ensure both output features and states are not corrupted while maintaining model accuracy, and the embedded states allow the latest output feature to be determined, enabling correct inference recovery upon power resumption. Evaluations were conducted on different Texas Instruments devices under varied intermittent power strengths and network models. Compared to the state-of-the-art, our approach can speed up intermittent inference by 1.3 to 5 times, achieving higher performance when executing modern convolutional networks with weaker power.
Year
DOI
Venue
2022
10.1109/TCAD.2022.3197513
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Keywords
DocType
Volume
Battery-less devices,deep neural networks (DNNs),energy harvesting,intermittent systems,stateful neural networks
Journal
41
Issue
ISSN
Citations 
11
0278-0070
0
PageRank 
References 
Authors
0.34
17
4
Name
Order
Citations
PageRank
Chih-Hsuan Yen100.68
Hashan Roshantha Mendis240.73
Tei-Wei Kuo33203326.35
Pi-Cheng Hsiu439834.30