Name
Title | ||
|---|---|---|
Efficient Dynamic Device Placement For Deep Neural Network Training On Heterogeneous Systems |
Abstract | ||
|---|---|---|
Deep Neural Networks (DNNs) based learning methods have brought revolutionary advances in computer vision and machine learning. However, training a DNN model often requires very intensive computational resources. For edge incremental learning, more energy efficient learning solutions are called for. Heterogeneous computing is more power efficient, and has been increasingly popular for embedded platforms. Therefore, how to deploy training models on heterogeneous platforms to support edge learning is a critical issue.Due to the increasing size of DNNs, it is rather difficult to determine how to dispatch a large number of operations to proper devices. One state-of-art approach uses reinforcement learning to address this device placement issue, but is too costly to apply in an embedded setting. In this paper, our approach leverages the information available from the computational graph of the model, and the dynamic profiles of run time and communication time of each device, to more efficiently deploy operations on heterogeneous systems. We use Critical Earliest Finish Time (CEFT) algorithm together with the Partitioned Boolean Quadratic Assignment Problem (PBQP) solver to find a cost-effective placement, and dynamically adjust assignments during the training process, which makes our method more adaptive and effective for different computational environments. On AlexNet, VGG, Inception, ResNet, RNNLM and other well-known models, our approach significantly outperforms traditional algorithms and reinforcement learning based methods. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1007/978-3-030-27562-4_4 | EMBEDDED COMPUTER SYSTEMS: ARCHITECTURES, MODELING, AND SIMULATION, SAMOS 2019 |
Field | DocType | Volume |
Power efficient,Efficient energy use,Computer science,Incremental learning,Symmetric multiprocessor system,Device placement,Real-time computing,Artificial neural network,Deep neural networks,Distributed computing | Conference | 11733 |
ISSN | Citations | PageRank |
0302-9743 | 0 | 0.34 |
References | Authors | |
0 | 3 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Zi Xuan Huang | 1 | 0 | 0.34 |
| Shen Yu Fu | 2 | 0 | 0.34 |
| Wei-Chung Hsu | 3 | 719 | 58.87 |