Paper Info
Title
Programming Abstractions for Data Intensive Computing on Clouds and Grids
Abstract
MapReduce has emerged as an important data-parallel programming model for data-intensive computing – for Clouds and Grids. However most if not all implementations of MapReduce are coupled to a specific infrastructure. SAGA is a high-level programming interface which provides the ability to create distributed applications in an infrastructure independent way. In this paper, we show how MapReduce has been implemented using SAGA and demonstrate its interoperability across different distributed platforms – Grids, Cloud-like infrastructure and Clouds. We discuss the advantages of programmatically developing MapReduce using SAGA, by demonstrating that the SAGA-based implementation is infrastructure independent whilst still providing control over the deployment, distribution and runtime decomposition. The ability to control the distribution and placement of the computation units (workers) is critical in order to implement the ability to move computational work to the data. This is required to keep data network transfer low and in the case of commercial Clouds the monetary cost of computing the solution low. Using data-sets of size up to 10GB, and upto 10 workers, we provide detailed performance analysis of the SAGA-MapReduce implementation, and show how controllingthe distribution of computation and the payload per worker helps enhance performance.
Year
DOI
Venue
2009
10.1109/CCGRID.2009.87
CCGrid
Keywords
Field
DocType
application program interfaces,grid computing,parallel programming,MapReduce data-parallel programming model,SAGA programming interface,cloud computing,data intensive computing,data network transfer,distributed application,grid computing,programming abstraction,SAGA,clouds grids,data intensive,mapreduce
Grid computing,Programming paradigm,Data-intensive computing,Interoperability,Computer science,Implementation,Distributed database,Distributed computing,Cloud computing,Payload
Conference
Citations 
PageRank 
References 
17
1.05
4
Authors
5
Name
Order
Citations
PageRank
Chris Miceli1171.05
Michael Miceli2181.45
S. Jha37921539.19
Hartmut Kaiser417216.63
Andre Merzky513020.45