Paper Info
Title
Feature-based high-availability mechanism for quantile tasks in real-time data stream processing
Abstract
AbstractUnder distributed Cloud environment, the real-time and continuous data stream makes the availability during processing essential but expensive. For aggregation tasks of data stream processing systems, traditional replica-based high-availability mechanisms require large overheads at run-time and long recovery latency at fail-time, because of specific nature of aggregations. In this paper, we focus on the typical quantile tasks and propose a feature-based high-availability mechanism to reduce related overhead and the latency. With the help of monitor module, quantile feature is maintained incrementally through histogram synopsis over time-based sliding window, and the failed quantile tasks can be recovered precisely with high probability in an efficient way. The effectiveness has been analyzed theoretically, and meanwhile, the acceptable tradeoff between overheads and performance has been demonstrated by comprehensive experiments on both synthetic and real data. Copyright © 2013 John Wiley & Sons, Ltd.
Year
DOI
Venue
2014
10.1002/spe.2244
Periodicals
Keywords
DocType
Volume
Cloud Computing,data stream,quantile feature,high availability,histogram
Journal
44
Issue
ISSN
Citations 
7
0038-0644
6
PageRank 
References 
Authors
0.66
28
4
Name
Order
Citations
PageRank
Weilong Ding12810.14
Yanbo Han250059.74
Jing Wang 0002360.66
Zhuofeng Zhao46615.46