Abstract | ||
---|---|---|
Massively parallel applications often require periodic data checkpointing for program restart and post-run data analysis. Although high performance computing systems provide massive parallelism and computing power to fulfill the crucial requirements of the scientific applications, the I/O tasks of high-end applications do not scale. Strict data consistency semantics adopted from traditional file systems are inadequate for homogeneous parallel computing platforms. For high performance parallel applications independent I/O is critical, particularly if checkpointing data are dynamically created or irregularly partitioned. In particular, parallel programs generating a large number of unrelated I/O accesses on large-scale systems often face serious I/O serializations introduced by lock contention and conflicts at file system layer. As these applications may not be able to utilize the I/O optimizations requiring process synchronization, they pose a great challenge for parallel I/O architecture and software designs. We propose an I/O mechanism to bridge the gap between scientific applications and parallel storage systems. A static file domain partitioning method is developed to align the I/O requests and produce a client-server mapping that minimizes the file lock acquisition costs and eliminates the lock contention. Our performance evaluations of production application I/O kernels demonstrate scalable performance and achieve high I/O bandwidths. |
Year | DOI | Venue |
---|---|---|
2012 | 10.1109/TPDS.2011.166 | Parallel and Distributed Systems, IEEE Transactions |
Keywords | Field | DocType |
client-server systems,data analysis,data integrity,file organisation,information retrieval,input-output programs,optimisation,parallel architectures,parallel programming,performance evaluation,software architecture,I-O optimization,I-O request,I-O serialization,client-server mapping,delegation-based I-O mechanism,file lock acquisition cost,file system layer,high I-O bandwidth,high end application,high performance computing system,high performance parallel application,homogeneous parallel computing platform,large scale system,lock contention,parallel I-O architecture,parallel program,parallel storage system,performance evaluation,periodic data checkpointing,post-run data analysis,process synchronization,production application I-O kernel,software design,static file domain partitioning method,strict data consistency semantics,unrelated I-O access,I/O delegation,MPI-IO,Parallel I/O,collaborative caching,file locking.,non collective I/O,parallel file systems | File system,Supercomputer,Massively parallel,Computer science,Parallel computing,Real-time computing,Input/output,Parallel I/O,Multi-core processor,File locking,Distributed computing,Scalability | Journal |
Volume | Issue | ISSN |
23 | 2 | 1045-9219 |
Citations | PageRank | References |
8 | 0.47 | 17 |
Authors | ||
3 |
Name | Order | Citations | PageRank |
---|---|---|---|
Arifa Nisar | 1 | 74 | 4.08 |
Wei-keng Liao | 2 | 1095 | 87.98 |
Alok N. Choudhary | 3 | 3441 | 326.32 |