Paper Info
Title
Real-Time Scheduling in MapReduce Clusters
Abstract
MapReduce has been widely used as a Big Data processing platform. As it gets popular, its scheduling becomes increasingly important. In particular, since many MapReduce applications require real-time data processing, scheduling real time applications in MapReduce environments has become a significant problem. In this paper, we create a novel real-time scheduler for MapReduce, which overcomes the deficiencies of an existing scheduler. It avoids accepting jobs that will lead to deadline misses and improves the cluster utilization. We implement our scheduler in Hadoop system and experimental results show that our scheduler provides deadline guarantees for accepted jobs and achieves good cluster utilization.
Year
DOI
Venue
2013
10.1109/HPCC.and.EUC.2013.216
HPCC/EUC
Keywords
Field
DocType
mapreduce,pattern clustering,real-time data processing,scheduling,mapreduce clusters,hadoop system,distributed programming,real-time scheduling,big data processing platform,cluster utilization,big data,clustering algorithms,estimation,vectors,real time systems,adaptive control
Fixed-priority pre-emptive scheduling,Data processing,Fair-share scheduling,Scheduling (computing),Computer science,Parallel computing,Real-time computing,Adaptive control,Cluster analysis,Dynamic priority scheduling,Earliest deadline first scheduling,Distributed computing
Conference
Citations 
PageRank 
References 
2
0.37
13
Authors
3
Name
Order
Citations
PageRank
Chen He1714101.22
Ying Lu2654.01
David Swanson3483.17