Paper Info
Title
Scheduling Distributed Resources in Heterogeneous Private Clouds
Abstract
We first consider the static problem of allocating resources to (i.e., scheduling) multiple distributed application frameworks, possibly with different priorities and server preferences, in a private cloud with heterogeneous servers. Several fair scheduling mechanisms have been proposed for this purpose. We extend prior results on max-min fair (MMF) and proportional fair (PF) scheduling to this constrained multiresource and multiserver case for generic fair scheduling criteria. The task efficiencies (a metric related to proportional fairness) of max-min fair allocations found by progressive filling are compared by illustrative examples. In the second part of this paper, we consider the online problem (with framework churn) by implementing variants of these schedulers in Apache Mesos using progressive filling to dynamically approximate max-min fair allocations. We evaluate the implemented schedulers in terms of overall execution time of realistic distributed Spark workloads. Our experiments show that resource efficiency is improved and execution times are reduced when the scheduler is "server specific" or when it leverages characterized required resources of the workloads (when known).
Year
DOI
Venue
2018
10.1109/MASCOTS.2018.00018
2018 IEEE 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)
Keywords
Field
DocType
private cloud,scheduling,heterogeneity,progressive filling
Spark (mathematics),Resource efficiency,Computer science,Scheduling (computing),Fair scheduling,Server,Execution time,Proportionally fair,Cloud computing,Distributed computing
Conference
ISSN
ISBN
Citations 
1526-7539
978-1-5386-6887-0
1
PageRank 
References 
Authors
0.37
20
6
Name
Order
Citations
PageRank
Kesidis, G.143871.79
Yuquan Shan295.03
Aman Jain321.73
Bhuvan Urgaonkar42309158.10
Jalal Khamse-Ashari5112.23
Ioannis Lambadaris650278.37