Paper Info
Title
A Physical and Virtual Compute Cluster Resource Load Balancing Approach to Data-Parallel Scientific Workflow Scheduling
Abstract
To execute workflows on a compute cluster resource, workflow engines can work with cluster resource manager software to distribute jobs into compute nodes on the cluster. We discuss how to interact with traditional Oracle Grid Engine and recent Hadoop cluster resource managers using a dataflow-based scheduling approach to balance compute resource load for data-parallel workflow execution. Our experiments show that: 1) The presented approach can balance computational resource load well by interacting with the resource managers and provides good execution performance on both physical and virtual clusters, 2) Oracle Grid Engine outperforms Hadoop for CPU-intensive applications on small-scale clusters.
Year
DOI
Venue
2011
10.1109/SERVICES.2011.50
SERVICES
Keywords
Field
DocType
computational resource load,data-parallel scientific workflow scheduling,cluster resource manager software,small-scale cluster,resource manager,oracle grid engine,recent hadoop cluster resource,virtual compute cluster resource,data-parallel workflow execution,cluster resource,virtual cluster,resource load,grid computing,engines,load balancing,resource allocation,scheduling,load balance,cloud computing,biochemistry,workflow engine,servers
Resource management,Workflow technology,Grid computing,Load balancing (computing),Computer science,Resource allocation,Workflow engine,Database,Computer cluster,Computational resource,Operating system,Distributed computing
Conference
Citations 
PageRank 
References 
4
0.80
6
Authors
3
Name
Order
Citations
PageRank
Jianwu Wang184.00
Prakashan Korambath2253.16
Ilkay Altintas31191106.09