Paper Info
Title
SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud
Abstract
Despite the soaring use of convolutional neural networks (CNNs) in mobile applications, uniformly sustaining high-performance inference on mobile has been elusive due to the excessive computational demands of modern CNNs and the increasing diversity of deployed devices. A popular alternative comprises offloading CNN processing to powerful cloud-based servers. Nevertheless, by relying on the cloud to produce outputs, emerging mission-critical and high-mobility applications, such as drone obstacle avoidance or interactive applications, can suffer from the dynamic connectivity conditions and the uncertain availability of the cloud. In this paper, we propose SPINN, a distributed inference system that employs synergistic device-cloud computation together with a progressive inference method to deliver fast and robust CNN inference across diverse settings. The proposed system introduces a novel scheduler that co-optimises the early-exit policy and the CNN splitting at run time, in order to adapt to dynamic conditions and meet user-defined service-level requirements. Quantitative evaluation illustrates that SPINN outperforms its state-of-the-art collaborative inference counterparts by up to 2× in achieved throughput under varying network conditions, reduces the server cost by up to 6.8× and improves accuracy by 20.7% under latency constraints, while providing robust operation under uncertain connectivity conditions and significant energy savings compared to cloud-centric execution.
Year
DOI
Venue
2020
10.1145/3372224.3419194
MobiCom '20: The 26th Annual International Conference on Mobile Computing and Networking London United Kingdom September, 2020
DocType
ISBN
Citations 
Conference
978-1-4503-7085-1
7
PageRank 
References 
Authors
0.46
27
5
Name
Order
Citations
PageRank
Stefanos Laskaridis1183.21
Stylianos I. Venieris210612.98
Mário Almeida3122.62
Ilias Leontiadis476144.38
Nicholas D. Lane54247248.15