Paper Info
Title
Cuckoo: Opportunistic MapReduce on Ephemeral and Heterogeneous Cloud Resources
Abstract
Cloud infrastructures are generally over-provisioned for handling load peaks and node failures. However, the drawback of this approach is that a large portion of data center resources remains unused. In this paper, we propose a framework that leverages unused resources of data centers, which are ephemeral by nature, to run MapReduce jobs. Our approach allows: i) to run efficiently Hadoop jobs on top of heterogeneous Cloud resources, thanks to our data placement strategy, ii) to predict accurately the volatility of ephemeral resources, thanks to the quantile regression method, and iii) for avoiding the interference between MapReduce jobs and co-resident workloads, thanks to our reactive QoS controller. We have extended Hadoop implementation with our framework and evaluated it with three different data center workloads. The experimental results show that our approach divides Hadoop job execution time by up to 7 when compared to the standard Hadoop implementation.
Year
DOI
Venue
2019
10.1109/CLOUD.2019.00070
2019 IEEE 12th International Conference on Cloud Computing (CLOUD)
Keywords
DocType
ISSN
cloud, scheduling, unused resources, ephemeral resources, spare resources, data placement, big data, mapreduce, hadoop, quantile regression.
Conference
2159-6182
ISBN
Citations 
PageRank 
978-1-7281-2706-4
0
0.34
References 
Authors
14
4
Name
Order
Citations
PageRank
jeanemile dartois151.81
Heverson B. Ribeiro200.34
Jalil Boukhobza310722.94
Olivier Barais472461.99