Paper Info
Title
ZOID: I/O-forwarding infrastructure for petascale architectures
Abstract
The ZeptoOS project is developing an open-source alternative to the proprietary software stacks available on contemporary massively parallel architectures. The aim is to enable computer science research on these architectures, enhance community collaboration, and foster innovation. In this paper, we introduce a component of ZeptoOS called ZOID---an I/O-forwarding infrastructure for architectures such as IBM Blue Gene that decouple file and socket I/O from the compute nodes, shipping those functions to dedicated I/O nodes. Through the use of optimized network protocols and data paths, as well as a multithreaded daemon running on I/O nodes, ZOID provides greater performance than does the stock infrastructure. We present a set of benchmark results that highlight the improvements. Crucially, the flexibility of our infrastructure is a vast improvement over the stock infrastructure, allowing users to forward data using custom-designed application interfaces, through an easy-to-use plug-in mechanism. This capability is used for real-time telescope data transfers, extensively discussed in the paper. Plug-in--specific threads implement prefetching of data obtained over sockets from an input cluster and merge results from individual compute nodes before sending them out, significantly reducing required network bandwidth. This approach allows a ZOID version of the application to handle a larger number of subbands per I/O node, or even to bypass the input cluster altogether, plugging the input from remote receiver stations directly into the I/O nodes. Using the resources more efficiently can result in considerable savings.
Year
DOI
Venue
2008
10.1145/1345206.1345230
PPOPP
Keywords
Field
DocType
real-time telescope data transfer,input cluster,o node,zeptoos project,stock infrastructure,data path,custom-designed application interface,network bandwidth,zoid version,petascale architecture,o-forwarding infrastructure,programming,petascale,performance,scientific computing,real time,genes,network protocol,data transfer
IBM,Massively parallel,Computer science,Parallel computing,Input/output,Thread (computing),Bandwidth (signal processing),Software,Petascale computing,Distributed computing,Communications protocol
Conference
Citations 
PageRank 
References 
53
2.73
6
Authors
4
Name
Order
Citations
PageRank
Kamil Iskra164246.46
John W. Romein230426.87
Kazutomo Yoshii324918.53
Pete Beckman482248.04