Paper Info
Title
Dynamic Data Layout Optimization for High Performance Parallel I/O
Abstract
Storage performance bottlenecks are one of the major threats limiting the scalability of I/O intensive applications. Parallel storage systems have the potential to alleviate I/O bottlenecks through concurrent operation of independent storage components if a parallelism-aware data layout can be continuously guaranteed. Existing systems use one-layout-fits-all data placement strategy that frequently results in sub-optimal I/O parallelism. Guided by association rule mining, graph coloring, bin packing, and network flow techniques, this paper proposes a general framework for self-optimizing parallel storage systems, with the goal of continuously providing a high-degree of I/O parallelism that is robust to changes in the parallel access patterns of applications and the coexistence of applications with different parallel access characteristics. Evaluation results indicate that the proposed framework is highly successful in adjusting to skewed parallel access patterns for both traditional hard disk drive (HDD) based storage arrays and solid-state drive (SSD) based all-flash arrays. In addition to the storage arrays, the proposed framework is sufficiently generic to be tailored to various other parallel storage scenarios including but not limited to key-value stores, parallel/distributed file systems, and internal parallelism of SSDs.
Year
DOI
Venue
2016
10.1109/HiPC.2016.024
2016 IEEE 23rd International Conference on High Performance Computing (HiPC)
Keywords
Field
DocType
storage systems,parallel I/O,self-optimization
Flow network,Disk array,Computer science,Parallel computing,Dynamic data,Self-optimization,Data parallelism,Parallel I/O,Bin packing problem,Scalability,Distributed computing
Conference
ISSN
ISBN
Citations 
1094-7256
978-1-5090-5412-1
0
PageRank 
References 
Authors
0.34
19
4
Name
Order
Citations
PageRank
Everett Neil Rush121.04
Bryan Harris201.35
Nihat Altiparmak3316.33
Ali Saman Tosun414418.94