Paper Info
Title
A Scalable Message Passing Interface Implementation of an Ad-Hoc Parallel I/o system
Abstract
In this paper we present the novel design, implementation, and evaluation of an ad-hoc parallel I/O system (AHPIOS). AHPIOS is the first scalable parallel I/O system completely implemented in the Message Passing Interface (MPI). The MPI implementation brings the advantages of portability, scalability and high performance. AHPIOS allows MPI applications to dynamically manage and scale distributed partitions in a convenient way. The configuration of both the MPI-IO and the storage management system is unified and allows for a tight integration of the optimizations of these layers. AHPIOS partitions are elastic: they conveniently scale up and down with the number of resources. We develop two collective I/O strategies, which leverage a two-tiered cooperative cache in order to exploit the spatial locality of data-intensive parallel applications. The file access latency is hidden from the applications through an asynchronous data staging strategy. The two-tiered cooperative cache scales with both the number of processors and storage resources. Our experimental section demonstrates that, with various optimizations, integrated AHPIOS offers a substantial performance benefit over the traditional MPI-IO solutions on both PVFS or Lustre parallel file systems.
Year
DOI
Venue
2010
10.1177/1094342009347890
IJHPCA
Keywords
DocType
Volume
o strategy,data-intensive parallel application,o system,scalable parallel,ahpios partition,ad-hoc parallel,lustre parallel file system,scalable message passing interface,mpi implementation,mpi application,integrated ahpios,parallel systems,message passing interface,distributed file system
Journal
24
Issue
ISSN
Citations 
2
1094-3420
8
PageRank 
References 
Authors
0.50
34
5
Name
Order
Citations
PageRank
Florin Isaila123424.01
Francisco Javier Garcia Blas280.50
Jesús Carretero355269.87
Wei-keng Liao4109587.98
Alok N. Choudhary53441326.32