Name
Abstract | ||
|---|---|---|
Although Hadoop MapReduce provides good programming abstractions and horizontal scalability, it is often blamed for its poor single node performance. In the meantime, MapReduce has already achieved a large install base, thus any performance improvement should keep the compatibility. In this paper, we address the challenges via several approaches guided by low-level performance analysis. And we materialize the approaches via NativeTask, a high-performance, fully compatible MapReduce execution engine. We evaluate its performance with representative HiBench workloads. The results show that the speedup NativeTask achieves ranges from 10% to 160%, and it paves the way for a better MapReduce that excels on both single node performance and scalability. In the future, hardware acceleration can also be applied to further improve the system's efficiency. |
Year | DOI | Venue |
|---|---|---|
2013 | 10.1109/BigData.2013.6691703 | BigData Conference |
Keywords | Field | DocType |
nativetask,high performance,c++ implementation,low-level performance analysis,cache-oblivious sort,hardware acceleration,programming abstractions,representative hibench workloads,cpu-bound application,system efficiency,hadoop compatible framework,software performance evaluation,compatibility,hadoop,horizontal scalability,hadoop mapreduce,distributed processing | Compatibility (mechanics),Computer science,Parallel computing,Hardware acceleration,Speedup,Performance improvement,Scalability | Conference |
Volume | Issue | ISSN |
null | null | 2639-1589 |
Citations | PageRank | References |
2 | 0.39 | 13 |
Authors | ||
9 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Dong Yang | 1 | 59 | 5.36 |
| Xiang Zhong | 2 | 5 | 0.86 |
| Dong Yan | 3 | 17 | 6.40 |
| Fangqin Dai | 4 | 2 | 0.39 |
| Xusen Yin | 5 | 5 | 1.19 |
| Cheng Lian | 6 | 312 | 9.99 |
| Zhongliang Zhu | 7 | 2 | 0.39 |
| Weihua Jiang | 8 | 2 | 0.39 |
| Gansha Wu | 9 | 107 | 9.06 |