Paper Info
Title
A real-time big data analysis framework on a CPU/GPU heterogeneous cluster: a meteorological application case study.
Abstract
It is important to analyze and predict meteorological phenomena in real-time. Parallel programming by exploiting thousands of threads in GPUs can be efficiently used to speed up the execution of many applications. However, GPUs have limitations when used for processing big data, which can be better analyzed using distributed computing platforms such as Hadoop and Spark. In this paper, we propose DAMB a system that processes streamed data on a heterogeneous cluster of CPUs and GPUs in real-time. The core of DAMB is SparkGPU, a platform that extends Apache Spark to allow it to manage a heterogeneous cluster that has both CPUs and GPUs and to execute tasks on GPUs. DAMB also provides data visualization tools that present the analyzed data in an interactive way in real-time. As a case study, we focus on a meteorological application that analyzes lightening discharges. We show that DAMB can successfully process and analyze the meteorological data streamed to it and visualize the results in real-time on a cluster of size 12 nodes, each is equipped with one or more GPU cards. This is a speedup of two orders of magnitude as compared to a sequential program implementation for the same application.
Year
DOI
Venue
2016
10.1145/3006299.3006304
Proceedings of the 3rd IEEE/ACM International Conference on Big Data Computing, Applications and Technologies
Keywords
Field
DocType
Heterogeneous clusters, GPU Programming, In-memory cluster computing
Data visualization,Spark (mathematics),Computer science,Parallel computing,Heterogeneous cluster,Thread (computing),General-purpose computing on graphics processing units,Big data,Speedup
Conference
ISBN
Citations 
PageRank 
978-1-5090-4468-9
1
0.36
References 
Authors
10
2
Name
Order
Citations
PageRank
Mohamed Hassaan110.36
Iman Elghandour2564.72