Name
Title | ||
|---|---|---|
Scaling Distributed Training of Flood-Filling Networks on HPC Infrastructure for Brain Mapping |
Abstract | ||
|---|---|---|
Mapping all the neurons in the brain requires automatic reconstruction of entire cells from volume electron microscopy data. The flood-filling network (FFN) architecture has demonstrated leading performance for segmenting structures from this data. However, the training of the network is computationally expensive. In order to reduce the training time, we implemented synchronous and data-parallel distributed training using the Horovod library, which is different from the asynchronous training scheme used in the published FFN code. We demonstrated that our distributed training scaled well up to 2048 Intel Knights Landing (KNL) nodes on the Theta supercomputer. Our trained models achieved similar level of inference performance, but took less training time compared to previous methods. Our study on the effects of different batch sizes on FFN training suggests ways to further improve training efficiency. Our findings on optimal learning rate and batch sizes agree with previous works. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/DLS49591.2019.00012 | 2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) |
Keywords | Field | DocType |
automatic reconstruction,volume electron microscopy data,flood-filling network architecture,training time,asynchronous training scheme,published FFN code,Intel Knights Landing,trained models,FFN training,HPC infrastructure,brain mapping,FFN architecture,data-parallel distributed training,Theta supercomputer | Brain mapping,Asynchronous communication,Supercomputer,Optimal learning,Computer science,Inference,Scaling,Computer engineering,Flood myth | Conference |
ISBN | Citations | PageRank |
978-1-7281-6012-2 | 1 | 0.35 |
References | Authors | |
4 | 12 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Wushi Dong | 1 | 1 | 0.35 |
| Nicola J. Ferrier | 2 | 181 | 24.79 |
| Narayanan Kasthuri | 3 | 62 | 7.11 |
| Peter Littlewood | 4 | 1 | 0.35 |
| Murat Keçeli | 5 | 8 | 2.59 |
| Rafael Vescovi | 6 | 1 | 1.03 |
| Hanyu Li | 7 | 6 | 5.58 |
| Clive E. Adams | 8 | 13 | 1.37 |
| Elise Jennings | 9 | 1 | 0.35 |
| Samuel Flender | 10 | 1 | 0.35 |
| Thomas D. Uram | 11 | 95 | 11.29 |
| Venkatram Vishwanath | 12 | 507 | 47.27 |