Paper Info
Title
Global-scale distributed I-O with ParaMEDIC
Abstract
Achieving high performance for distributed I-O on a wide-area network continues to be an elusive holy grail. Despite enhancements in network hardware as well as software stacks, achieving high-performance remains a challenge. In this paper, our worldwide team took a completely new and non-traditional approach to distributed I-O, called ParaMEDIC: Parallel Metadata Environment for Distributed I-O and Computing, by utilizing application-specific transformation of data to orders of magnitude smaller metadata before performing the actual I-O. Specifically, this paper details our experiences in deploying a large-scale system to facilitate the discovery of missing genes and constructing a genome similarity tree by encapsulating the mpiBLAST sequence-search algorithm into ParaMEDIC. The overall project involved nine computational sites spread across the U.S. and generated more than a petabyte of data that was ‘teleported’ to a large-scale facility in Tokyo for storage. Copyright © 2010 John Wiley & Sons, Ltd.
Year
DOI
Venue
2010
10.1002/cpe.v22:16
Concurrency and Computation: Practice and Experience
Keywords
DocType
Volume
actual I-O,large-scale facility,large-scale system,network hardware,paper detail,wide-area network,John Wiley,Parallel Metadata Environment,application-specific transformation,computational site
Journal
22
Issue
Citations 
PageRank 
16
1
0.37
References 
Authors
15
15
Name
Order
Citations
PageRank
Pavan Balaji11475111.48
Wu-chun Feng22812232.50
Heshan Lin337523.13
Jeremy S. Archuleta4635.05
Satoshi Matsuoka53773359.36
Andrew S. Warren6514.09
João Carlos Setubal7403.00
Ewing L. Lusk83080472.23
Rajeev Thakur93773251.09
Foster Ian10229382663.24
Daniel S. Katz111496121.04
Shantenu Jha1218832.40
K. Shinpaugh1310.37
Susan Coghlan1429118.09
Daniel A. Reed152024313.41