Title
Accelerating big data analytics on HPC clusters using two-level storage.
Abstract
•Develop a new two-level storage system that integrates an upper-level in-memory file system with a lower-level parallel file system.•Model and compare I/O throughput of two-level storage to HDFS and OrangeFS.•Build a prototype of two-level storage system with Tachyon and OrangeFS,•Conduct experiments on real systems show that the proposed two-level storage delivers higher aggregate I/O throughputs than HDFS and OrangeFS and achieves weak scalability on both read and write.
Year
DOI
Venue
2017
10.1016/j.parco.2016.08.001
Parallel Computing
Keywords
Field
DocType
Two-level storage,In-memory file system,Parallel file system,Data-intensive computing
Cluster (physics),File system,Data-intensive computing,Computer science,Computer data storage,Parallel computing,Systems design,Throughput,Big data,Scalability
Journal
Volume
ISSN
Citations 
61
0167-8191
3
PageRank 
References 
Authors
0.41
26
5
Name
Order
Citations
PageRank
Pengfei Xuan1163.07
Walter B. Ligon III212418.52
Pradip K. Srimani387996.11
Ge, Rong4111978.72
Feng Luo528426.03