Title | ||
---|---|---|
PASM: Parallelism Aware Space Management strategy for hybrid SSD towards in-storage DNN training acceleration |
Abstract | ||
---|---|---|
With the explosive growth of data volume and great improvement in flash technologies, SSD-based In-Storage Computing (ISC) is becoming one of the most important means to accelerate data-intensive applications, which also provides a possibility for in-storage acceleration of DNN training. However, the continuous write requests involved in DNN training become a big challenge for the reliability and efficiency of the flash-centric computing system. Recently trend in high-density and low-cost NAND flash further aggravates this challenge. To address this problem, we propose a Parallelism Aware Space Management (PASM) strategy to enable the utilization of SLC-TLC hybrid SSD for DNN training acceleration. And two key technologies are included in PASM. Firstly, we proposed the conflict-free parallelized data layout. It distributes data of different lifetimes into different kinds of physical blocks to avoid data migration as far as possible. Meanwhile, it enables parallel read/program and erase operations by exploiting the inherent parallelism in the flash array. As a result, it eliminates the conflicts between continuous I/O requests and garbage collections so as to provide a stable I/O performance during training. Secondly, a novel garbage collection strategy, called lifetime-aware deterministic garbage collection, is proposed. This scheme performs erase operations without disturbing continuous I/O requests by sensing the data lifetime and tightly coordinating the neural network training process, which is able to reclaim invalid blocks in a timely and efficient way to guarantee that the high endurance of SLC can be utilized continuously. Finally, to verify the performance of PASM, we compared PASM with the related state-of-the-art hybrid SSD and TLC-only SSD under Resnet50 training workload. Experimental results show that the PASM improves I/O performance by 20% and increases lifetime by 6.6 times. |
Year | DOI | Venue |
---|---|---|
2022 | 10.1016/j.sysarc.2022.102565 | Journal of Systems Architecture |
Keywords | DocType | Volume |
In-storage computing,DNN training,Hybrid SSD,SLC-TLC | Journal | 128 |
ISSN | Citations | PageRank |
1383-7621 | 0 | 0.34 |
References | Authors | |
0 | 3 |
Name | Order | Citations | PageRank |
---|---|---|---|
Chunhua Xiao | 1 | 0 | 8.45 |
Shi Qiu | 2 | 0 | 0.34 |
Dandan Xu | 3 | 0 | 0.34 |