Abstract | ||
---|---|---|
MapReduce has been widely used as a Big Data processing platform. As it gets popular, its scheduling becomes increasingly important. In particular, since many MapReduce applications require real-time data processing, scheduling real time applications in MapReduce environments has become a significant problem. In this paper, we create a novel real-time scheduler for MapReduce, which overcomes the deficiencies of an existing scheduler. It avoids accepting jobs that will lead to deadline misses and improves the cluster utilization. We implement our scheduler in Hadoop system and experimental results show that our scheduler provides deadline guarantees for accepted jobs and achieves good cluster utilization. |
Year | DOI | Venue |
---|---|---|
2013 | 10.1109/HPCC.and.EUC.2013.216 | HPCC/EUC |
Keywords | Field | DocType |
mapreduce,pattern clustering,real-time data processing,scheduling,mapreduce clusters,hadoop system,distributed programming,real-time scheduling,big data processing platform,cluster utilization,big data,clustering algorithms,estimation,vectors,real time systems,adaptive control | Fixed-priority pre-emptive scheduling,Data processing,Fair-share scheduling,Scheduling (computing),Computer science,Parallel computing,Real-time computing,Adaptive control,Cluster analysis,Dynamic priority scheduling,Earliest deadline first scheduling,Distributed computing | Conference |
Citations | PageRank | References |
2 | 0.37 | 13 |
Authors | ||
3 |
Name | Order | Citations | PageRank |
---|---|---|---|
Chen He | 1 | 714 | 101.22 |
Ying Lu | 2 | 65 | 4.01 |
David Swanson | 3 | 48 | 3.17 |