Paper Info
Title
DRESS: Dynamic RESource-Reservation Scheme for Congested Data-Intensive Computing Platforms
Abstract
In the past few years, we have envisioned an increasing number of businesses start driving by big data analytics, such as Amazon recommendations and Google Advertisements. At the back-end side, the businesses are powered by big data processing platforms to quickly extract information and make decisions. Running on top of a computing cluster, those platforms utilize scheduling algorithms to allocate resources. An efficient scheduler is crucial to the system performance due to limited resources, e.g. CPU and Memory, and a large number of user demands. However, besides requests from clients and current status of the system, it has limited knowledge about execution length of the running jobs, and incoming jobs' resource demands, which make assigning resources a challenging task. If most of the resources are occupied by a long-running job, other jobs will have to keep waiting until it releases them. This paper presents a new scheduling strategy, named DRESS that particularly aims to optimize the allocation among jobs with various demands. Specifically, it classifies the jobs into two categories based on their requests, reserves a portion of resources for each of category, and dynamically adjusts the reserved ratio by monitoring the pending requests and estimating release patterns of running jobs. The results demonstrate DRESS significantly reduces the completion time for one category, up to 76.1% in our experiments, and in the meanwhile, maintains a stable overall system performance.
Year
DOI
Venue
2018
10.1109/CLOUD.2018.00095
2018 IEEE 11th International Conference on Cloud Computing (CLOUD)
Keywords
DocType
Volume
Big data platforms,Hadoop,Scheduling
Conference
abs/1805.08359
ISBN
Citations 
PageRank 
978-1-5386-7236-5
0
0.34
References 
Authors
11
5
Name
Order
Citations
PageRank
Ying Mao1308.54
Victoria Green200.34
Jiayin Wang3567.69
Haoyi Xiong450544.63
Zhishan Guo532934.04