Name
Abstract | ||
|---|---|---|
Deep learning systems have been growing in prominence as a way to automatically characterize objects, trends, and anomalies. Researchers have been investigating techniques to optimize such systems. An area of particular interest has been using supercomputing systems to quickly generate effective deep learning networks, a phase referred to as “training” of the deep neural network. As we scale deep learning frameworks-such as Caffe-on large-scale systems, we notice that parallelism can help improve the computation tremendously, leaving data I/O as the major bottleneck limiting the overall system scalability. In this paper, we present a detailed analysis of the performance bottlenecks of Caffe on large supercomputing systems. The analysis shows that Caffe's I/O subsystem-LMDB-relies on memory-mapped I/O, which can be highly inefficient on large-scale systems because of its interaction with the process-scheduling system and the network-based parallel filesystem. Based on this analysis, we present LMDBIO, an optimized I/O plugin for Caffe that takes into account the data access pattern in order to vastly improve I/O performance. Experimental results show that LMDBIO can improve the overall execution time of Caffe by nearly 20-fold in some cases. |
Year | DOI | Venue |
|---|---|---|
2017 | 10.1109/HPCC-SmartCity-DSS.2017.29 | 2017 IEEE 19th International Conference on High Performance Computing and Communications; IEEE 15th International Conference on Smart City; IEEE 3rd International Conference on Data Science and Systems (HPCC/SmartCity/DSS) |
Keywords | Field | DocType |
deep learning systems,supercomputing systems,system scalability,performance bottlenecks,process-scheduling system,deep neural network training,deep learning frameworks,scalable deep learning,deep learning networks,Caffe-on large-scale systems,I/O analysis,optimization | Bottleneck,Supercomputer,Computer science,Caffè,Input/output,Artificial intelligence,Deep learning,Artificial neural network,Data access,Scalability,Distributed computing | Conference |
ISBN | Citations | PageRank |
978-1-5386-2589-7 | 5 | 0.44 |
References | Authors | |
7 | 4 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Sarunya Pumma | 1 | 21 | 2.69 |
| Min Si | 2 | 58 | 9.95 |
| Wu-chun Feng | 3 | 2812 | 232.50 |
| Pavan Balaji | 4 | 1475 | 111.48 |