Name
Abstract | ||
|---|---|---|
MapReduce is a powerful platform for large-scale data processing. To achieve good performance, a MapReduce scheduler must avoid unnecessary data transmission by enhancing the data locality (placing tasks on nodes that contain their input data). This paper develops a new MapReduce scheduling technique to enhance map task's data locality. We have integrated this technique into Hadoop default FIFO scheduler and Hadoop fair scheduler. To evaluate our technique, we compare not only MapReduce scheduling algorithms with and without our technique but also with an existing data locality enhancement technique (i.e., the delay algorithm developed by Face book). Experimental results show that our technique often leads to the highest data locality rate and the lowest response time for map tasks. Furthermore, unlike the delay algorithm, it does not require an intricate parameter tuning process. |
Year | DOI | Venue |
|---|---|---|
2011 | 10.1109/CloudCom.2011.16 | CloudCom |
Keywords | Field | DocType |
delay algorithm,existing data,data locality,new mapreduce scheduling technique,input data,large-scale data processing,map task,unnecessary data transmission,highest data,locality enhancement technique,data processing,schedules,data transmission,scheduling algorithm,data handling,scheduling,clustering algorithms | Data processing,Locality,Data transmission,FIFO (computing and electronics),Scheduling (computing),Computer science,Parallel computing,Real-time computing,Schedule,Cluster analysis,Group method of data handling,Distributed computing | Conference |
Citations | PageRank | References |
37 | 1.51 | 8 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Chen He | 1 | 714 | 101.22 |
| Ying Lu | 2 | 65 | 4.01 |
| David Swanson | 3 | 48 | 3.17 |