Paper Info
Title
iCATS: Scheduling Big Data Workflows in the Cloud Using Cultural Algorithms
Abstract
Workflow scheduling has remained a critical functionality of modern data-centric workflow management systems. Cloud computing, which provides practically unlimited computing and storage resources, has enabled a new generation of data-centric workflows, called big data workflows. New big data workflow scheduling algorithms should optimally utilize the characteristics of cloud computing such as heterogeneous virtual machines, the elastic resource provisioning model, and the pay-as-you-go pricing model, as well as the time and monetary cost to transfer large amounts of data. In this paper, we consider one case of the general big data workflow scheduling problem where a deadline, δ, is given for a workflow, W, and the goal is to minimize the monetary cost of running W in the cloud while satisfying the given deadline, δ. To this end, we leverage the power of Evolutionary Algorithms (EA) in order to search for the best solution within a reasonable planning time. More specifically, we introduce an innovative fitness function that combines the time and monetary cost of a workflow in one metric. Based on the EA and the fitness function, we design a deadline-constrained big data workflow scheduling algorithm, called iCATS (Improved Cultural Algorithms-based Task Scheduling). Extensive experiments demonstrate the statistical advantages of iCATS over existing representative EA workflow scheduling algorithms, including random (Rand), Genetic Algorithms (GA), Particle Swarm Optimization (PSO), and Cultural Algorithms (CA).
Year
DOI
Venue
2019
10.1109/BigDataService.2019.00020
2019 IEEE Fifth International Conference on Big Data Computing Service and Applications (BigDataService)
Keywords
Field
DocType
big data workflows, Cultural Algorithms, workflow scheduling, cloud computing, evolutionary algorithms
Evolutionary algorithm,Scheduling (computing),Computer science,Algorithm,Provisioning,Fitness function,Workflow,Workflow management system,Genetic algorithm,Cloud computing
Conference
ISBN
Citations 
PageRank 
978-1-7281-0060-9
0
0.34
References 
Authors
0
4
Name
Order
Citations
PageRank
Seyed Ziae Mousavi Mojab100.34
Mahdi Ebrahimi2173.06
Robert G. Reynolds3610188.20
Lu, Shiyong42022126.17