Paper Info
Title
Decoupled I/O for Data-Intensive High Performance Computing
Abstract
The I/O bottleneck issue has been acknowledged as one of main performance issues of high performance computing (HPC) systems for data-intensive scientific applications, and has attracted intensive studies in recent years. With the enlarging gap between the computing bandwidth and I/O bandwidth in projected next-generation HPC systems, this issue will become even worse. In this paper, we present a novel decoupled I/O to address the fundamental I/O bottleneck issue. The decoupled I/O is a software stack including MPI extensions, compiler improvements, and runtime library support, based one decoupled HPC system architecture. It allows users to treat the computing of data-intensive operations and the traditional I/O operation as an ensemble and offload them into dedicated data nodes, which are near to the data source, to reduce the overhead of data movement and improve the I/O bandwidth usage. The decoupled I/O is user-friendly and requires little changes in application codes. Experiments were conducted to evaluate the performance of the decoupled I/O, and the results show that it outperforms existing solutions (such as active storage I/O) and provides an attractive I/O solution for data-intensive high performance computing.
Year
DOI
Venue
2014
10.1109/ICPPW.2014.48
International Conference on Parallel Processing Workshops
DocType
ISSN
Citations 
Conference
1530-2016
3
PageRank 
References 
Authors
0.42
22
8
Name
Order
Citations
PageRank
Chao Chen12032185.26
Yong Chen210317.60
Kun Feng3344.32
Yanlong Yin41348.93
Hassan Eslami551.79
Rajeev Thakur63773251.09
Xian-he Sun71987182.64
William D. Gropp85547548.31