Title
Synergy: A HW/SW Framework for High Throughput CNNs on Embedded Heterogeneous SoC.
Abstract
Convolutional Neural Networks (CNN) have been widely deployed in diverse application domains. There has been significant progress in accelerating both their training and inference using high-performance GPUs, FPGAs, and custom ASICs for datacenter-scale environments. The recent proliferation of mobile and Internet of Things (IoT) devices have necessitated real-time, energy-efficient deep neural network inference on embedded-class, resource-constrained platforms. In this context, we present Synergy, an automated, hardware-software co-designed, pipelined, high-throughput CNN inference framework on embedded heterogeneous system-on-chip (SoC) architectures (Xilinx Zynq). Synergy leverages, through multi-threading, all the available on-chip resources, which includes the dual-core ARM processor along with the FPGA and the NEON Single-Instruction Multiple-Data (SIMD) engines as accelerators. Moreover, Synergy provides a unified abstraction of the heterogeneous accelerators (FPGA and NEON) and can adapt to different network configurations at runtime without changing the underlying hardware accelerator architecture by balancing workload across accelerators through work-stealing. Synergy achieves 7.3X speedup, averaged across seven CNN models, over a well-optimized software-only solution. Synergy demonstrates substantially better throughput and energy-efficiency compared to the contemporary CNN implementations on the same SoC architecture.
Year
DOI
Venue
2018
10.1145/3301278
ACM Transactions on Embedded Computing Systems (TECS)
Keywords
Field
DocType
CNNs, FPGAs, Hardware/software co-design, accelerator abstraction, heterogeneous computing, multi-threading, work stealing
Multithreading,ARM architecture,Computer architecture,Computer science,Parallel computing,Field-programmable gate array,SIMD,Symmetric multiprocessor system,Hardware acceleration,Throughput,Speedup
Journal
Volume
Issue
ISSN
abs/1804.00706
2.0
TECS, 18 (2019) 13-39
Citations 
PageRank 
References 
3
0.36
0
Authors
4
Name
Order
Citations
PageRank
Guanwen Zhong1393.43
Akshat Dubey230.36
Cheng Tan3389.50
Tulika Mitra42714135.99