Paper Info
Title
Multilevel Data Processing Using Parallel Algorithms for Analyzing Big Data in High-Performance Computing.
Abstract
The growing gap between users and the Big Data analytics requires innovative tools that address the challenges faced by big data volume, variety, and velocity. Therefore, it becomes computationally inefficient to analyze such massive volume of data. Moreover, advancements in the field of Big Data application and data science poses additional challenges, where High-Performance Computing solution has become a key issue and has attracted attention in recent years. However, these systems are either memoryless or computational inefficient. Therefore, keeping in view the aforementioned needs, there is a requirement for a system that can efficiently analyze a stream of Big Data within their requirements. Hence, this paper presents a system architecture that enhances the working of traditional MapReduce by incorporating parallel processing algorithm. Moreover, complete four-tier architecture is also proposed that efficiently aggregate the data, eliminate unnecessary data, and analyze the data by the proposed parallel processing algorithm. The proposed system architecture both read and writes operations that enhance the efficiency of the Input/Output operation. To check the efficiency of the proposed algorithms exploited in the proposed system architecture, we have implemented our proposed system using Hadoop and MapReduce. MapReduce is supported by a parallel algorithm that efficiently processes a huge volume of data sets. The system is implemented using MapReduce tool at the top of the Hadoop parallel nodes to generate and process graphs with near real-time. Moreover, the system is evaluated in terms of efficiency by considering the system throughput and processing time. The results show that the proposed system is more scalable and efficient.
Year
DOI
Venue
2018
10.1007/s10766-017-0498-x
International Journal of Parallel Programming
Keywords
Field
DocType
Big Data, HPC, Parallel Processing algorithm, Four-tier system architecture
Data processing,Data-intensive computing,Supercomputer,Parallel algorithm,Computer science,Parallel computing,Theoretical computer science,Throughput,Systems architecture,Big data,Scalability
Journal
Volume
Issue
ISSN
46
3
0885-7458
Citations 
PageRank 
References 
5
0.45
24
Authors
6
Name
Order
Citations
PageRank
Awais Ahmad137945.85
Anand Paul252746.32
Sadia Din38719.55
muhammad mazhar ullah rathore430121.15
Gyu Sang Choi512120.20
Gwanggil Jeon6596117.99