Paper Info
Title
Towards Hierarchical Autonomous Control for Elastic Data Stream Processing in the Fog.
Abstract
In the Big Data era, Data Stream Processing (DSP) applications should be capable to seamlessly process huge amount of data. Hence, they need to dynamically scale their execution on multiple computing nodes so to adjust to unpredictable data source rate. In this paper, we present a hierarchical and distributed architecture for the autonomous control of elastic DSP applications. It revolves around a two layered approach. At the lower level, distributed components issue requests for adapting the deployment of DSP operations as to adjust to changing workload conditions. At the higher level, a per-application centralized component works on a broader time scale; it oversees the application behavior and grants reconfigurations to control the application performance while limiting the negative effect of their enactment, i.e., application downtime. We have implemented the proposed solution in our distributed Storm prototype and evaluated its behavior adopting simple policies. The experimental results are promising and show that, even with simple policies, it is possible to limit the number of reconfigurations while at the same time guaranteeing an adequate level of application performance.
Year
Venue
Field
2017
Euro-Par Workshops
Digital signal processing,Data stream processing,Software deployment,Workload,Computer science,Downtime,Big data,Limiting,Distributed computing,Autonomous control
DocType
Citations 
PageRank 
Conference
2
0.38
References 
Authors
12
4
Name
Order
Citations
PageRank
Valeria Cardellini11514106.12
Francesco Lo Presti2107378.83
Matteo Nardelli3777.95
Gabriele Russo Russo422.07