Name
Abstract | ||
|---|---|---|
Convolutional neural networks (CNNs) have recently become the state-of-the-art in a diversity of AI tasks. Despite their popularity, CNN inference still comes at a high computational cost. A growing body of work aims to alleviate this by exploiting the difference in the classification difficulty among samples and early-exiting at different stages of the network. Nevertheless, existing studies on early exiting have primarily focused on the training scheme, without considering the use-case requirements or the deployment platform. This work presents HAPI, a novel methodology for generating highperformance early-exit networks by co-optimising the placement of intermediate exits together with the early-exit strategy at inference time. Furthermore, we propose an efficient design space exploration algorithm which enables the faster traversal of a large number of alternative architectures and generates the highest-performing design, tailored to the use-case requirements and target hardware. Quantitative evaluation shows that our system consistently outperforms alternative search mechanisms and state-of-the-art early-exit schemes across various latency budgets. Moreover, it pushes further the performance of highly optimised hand-crafted early-exit CNNs, delivering up to 5.11× speedup over lightweight models on imposed latency-driven SLAs for embedded devices. |
Year | DOI | Venue |
|---|---|---|
2020 | 10.1145/3400302.3415698 | 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD) |
Keywords | DocType | ISSN |
training scheme,HAPI,intermediate exits,early-exit strategy,inference time,efficient design space exploration algorithm,highly optimised hand-crafted early-exit CNNs,hardware-aware progressive inference,convolutional neural networks,AI tasks,CNN inference,classification difficulty,high-performance early-exit networks,latency-driven SLA,embedded devices | Conference | 1933-7760 |
Citations | PageRank | References |
1 | 0.37 | 13 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Stefanos Laskaridis | 1 | 18 | 3.21 |
| Stylianos I. Venieris | 2 | 106 | 12.98 |
| Kim, Hyeji | 3 | 23 | 6.94 |
| Nicholas D. Lane | 4 | 4247 | 248.15 |