Paper Info
Title
Enabling dynamic file I/O path selection at runtime for parallel file system
Abstract
Parallel file systems are experiencing more and more applications from various fields. Various applications have different I/O workload characteristics, which have diverse requirements on accessing storage resources. However, parallel file systems often adopt the "one-size-fits-all" solution, which fails to meet specific application needs and hinders the full exploitation of potential performance. This paper presents a framework to enable dynamic file I/O path selection with fine granularity at runtime. The framework adopts a file handle-rich scheme to allow file systems choose corresponding optimizations to serve I/O requests. Consistency control algorithms are proposed to ensure data consistency while changing optimizations at runtime. One case study on our prototype shows that choosing proper optimizations can improve the I/O performance for small files and large files by up to 40 and 64.4 %, respectively. Another case study shows that the data prefetch performance for real-world application traces can be improved by up to 193 % by selecting correct prefetch patterns. Simulations in large-scale environment also show that our method is scalable and both the memory consumption and the consistency control overhead can be negligible.
Year
DOI
Venue
2014
10.1007/s11227-013-1077-6
The Journal of Supercomputing
Keywords
Field
DocType
parallel file system,o performance,file system,large file,dynamic file,file handle-rich scheme,o path selection,case study,o request,small file
Virtual file system,File system,SSH File Transfer Protocol,File Control Block,Computer science,Parallel computing,Device file,Unix file types,Versioning file system,File system fragmentation,Operating system,Distributed computing
Journal
Volume
Issue
ISSN
68
2
1573-0484
Citations 
PageRank 
References 
5
0.46
29
Authors
5
Name
Order
Citations
PageRank
Xiuqiao Li1515.74
Limin Xiao210728.51
Meikang Qiu33722246.98
Bin Dong4252.50
Li Ruan512325.10