Paper Info
Title
RDP: A storage-tier-aware Robust Data Placement strategy for Hadoop in a Cloud-based Heterogeneous Environment.
Abstract
Cloud computing is a robust technology, which facilitate to resolve many parallel distributed computing issues in the modern Big Data environment. Hadoop is an ecosystem, which process large data-sets in distributed computing environment. The HDFS is a filesystem of Hadoop, which process data blocks to the cluster nodes. The data block placement has become a bottleneck to overall performance in a Hadoop cluster. The current placement policy assumes that, all Datanodes have equal computing capacity to process data blocks. This computing capacity includes availability of same storage media and same processing performances of a node. As a result, Hadoop cluster performance gets effected with unbalanced workloads, inefficient storage-tier, network traffic congestion and HDFS integrity issues. This paper proposes a storage-tier-aware Robust Data Placement (RDP) scheme, which systematically resolves unbalanced workloads, reduces network congestion to an optimal state, utilizes storage-tier in a useful manner and minimizes the HDFS integrity issues. The experimental results show that the proposed approach reduced unbalanced workload issue to 72%. Moreover, the presented approach resolve storage-tier compatibility problem to 81% by predicting storage for block jobs and improved overall data block placement by 78% through pre-calculated computing capacity allocations and execution of map files over respective Namenode and Datanodes.
Year
DOI
Venue
2016
10.3837/tiis.2016.09.003
KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS
Keywords
Field
DocType
MapReduce,Hadoop,Data placement
Bottleneck,Distributed Computing Environment,Workload,Computer science,Block (data storage),Computer network,Network congestion,Big data,Traffic congestion,Distributed computing,Cloud computing
Journal
Volume
Issue
ISSN
10
9
1976-7277
Citations 
PageRank 
References 
0
0.34
0
Authors
2
Name
Order
Citations
PageRank
Nawab Muhammad Faseeh Qureshi100.68
Dong-Ryeol Shin212427.03