Name
Title | ||
|---|---|---|
Application profiling in hierarchical Hadoop for geo-distributed computing environments |
Abstract | ||
|---|---|---|
In the past two decades there has been a growing interest over the definition of new distributed computational paradigms capable to serve the need of manipulating and analyzing huge amounts of data. Among the others, the MapReduce outstands for popularity. Its open-source implementation Hadoop is widely used in academic environments and is also greatly supported by huge IT players. There are many application scenarios where the data to be manipulated resides on data centers which are heterogeneous in term of computing capacity and are geographically distant from each other's. Unfortunately, in this contexts Hadoop performs very poorly. In this paper we propose to leverage on a hierarchical computing framework to boost the Hadoop performance in geo-distributed computing environments. The framework we propose drains fresh information from the distributed computing context and exploits it to carry out a smart job scheduling strategy. In this work, the focus is put on the study and definition of the application profile of the jobs. We implemented a software prototype of the proposed hierarchical Hadoop framework. Tests run on the prototype proved the capability of the job scheduling system to compute the job's execution path and estimate its completion time. |
Year | DOI | Venue |
|---|---|---|
2016 | 10.1109/ISCC.2016.7543796 | 2016 IEEE Symposium on Computers and Communication (ISCC) |
Keywords | Field | DocType |
application profiling,geo-distributed computing environment,data manipulation,data analysis,open-source implementation,data center,hierarchical computing framework,Hadoop performance,smart job scheduling strategy,software prototype,hierarchical Hadoop framework,job scheduling system,job execution path,job completion time estimation | Application profile,Data-intensive computing,Computer science,Profiling (computer programming),Popularity,Exploit,Software,Job scheduler,Distributed database,Database,Distributed computing | Conference |
ISBN | Citations | PageRank |
978-1-5090-0680-9 | 4 | 0.40 |
References | Authors | |
9 | 4 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Marco Cavallo | 1 | 39 | 5.57 |
| Giuseppe Di Modica | 2 | 268 | 34.98 |
| Carmelo Polito | 3 | 13 | 2.31 |
| Orazio Tomarchio | 4 | 417 | 47.79 |