Paper Info
Title
Energy-efficient hadoop for big data analytics and computing: A systematic review and research insights.
Abstract
As the demands for big data analytics keep growing rapidly in scientific applications and online services, MapReduce and its open-source implementation Hadoop gained popularity in both academia and enterprises. Hadoop provides a highly feasible solution for building big data analytics platforms. However, defects of Hadoop are also exposed in many aspects including data management, resource management, scheduling policies, etc. These issues usually cause high energy consumption when running MapReduce jobs in Hadoop clusters. In this paper, we review the studies on improving energy efficiency of Hadoop clusters and summarize them in five categories including the energy-aware cluster node management, energy-aware data management, energy-aware resource allocation, energy-aware task scheduling and other energy-saving schemes. For each category, we briefly illustrate its rationale and comparatively analyze the relevant works regarding their advantages and limitations. Moreover, we present our insights and figure out possible research directions including energy-efficient cluster partitioning, data-oriented resource classification and provisioning, resource provisioning based on optimal utilization, EE and locality aware task scheduling, optimizing job profiling with machine learning, elastic power-saving Hadoop with containerization and efficient big data analytics on Hadoop. On one hand, the summary of studies on energy-efficient Hadoop presented in this paper provides useful guidance for the developers and users to better utilize Hadoop. On the other hand, the insights and research trends discussed in this work may inspire the relevant research on improving the energy efficiency of Hadoop in big data analytics. (C) 2017 Elsevier B.V. All rights reserved.
Year
DOI
Venue
2018
10.1016/j.future.2017.11.010
FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE
Keywords
Field
DocType
Energy efficiency,Hadoop,MapReduce,Data centers,Big data analytics
Resource management,Data science,Locality,Scheduling (computing),Efficient energy use,Computer science,Provisioning,Resource allocation,Data management,Big data,Distributed computing
Journal
Volume
ISSN
Citations 
86
0167-739X
3
PageRank 
References 
Authors
0.37
60
4
Name
Order
Citations
PageRank
Wentai Wu1181.91
Weiwei Lin214713.95
Ching-Hsien Hsu31121125.53
Ligang He454256.73