Name
Abstract | ||
|---|---|---|
Many streaming applications demand continuous processing of live data with little or no downtime, therefore, making high-availability a crucial operational requirement. Fault tolerance techniques are generally expensive and when directly applied to streaming systems with stringent throughput and latency requirements, they might incur a prohibitive performance overhead. This paper describes a flexible, light-weight fault tolerance solution in the context of the SPADE language and the System S distributed stream processing engine. We devised language extensions so users can define and parameterize checkpoint policies easily. This configurable fault tolerance solution is implemented through code generation in SPADE, which reduces the overall application fault tolerance costs by incurring them only for the parts of the application that require it. In this paper we focus on the overall design of our checkpoint mechanism and we also describe an incremental checkpointing algorithm that is suitable for on-the-fly processing of high-rate data streams. |
Year | DOI | Venue |
|---|---|---|
2009 | 10.1109/DSN.2009.5270344 | Lisbon |
Keywords | Field | DocType |
checkpointing,data analysis,fault tolerant computing,middleware,program compilers,SPADE language,System S distributed stream processing middleware engine,code generation,cost reduction,data analysis,data stream processing application,flexible light-weight configurable fault tolerance solution,incremental checkpointing algorithm,language-level checkpointing support,latency requirements,prohibitive performance overhead,throughput requirements | Middleware,Data stream mining,Computer science,Software fault tolerance,Code generation,Real-time computing,Fault tolerance,Throughput,Stream processing,Downtime,Distributed computing | Conference |
ISSN | ISBN | Citations |
1530-0889 | 978-1-4244-4421-2 | 18 |
PageRank | References | Authors |
0.85 | 20 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Gabriela Jacques-Silva | 1 | 171 | 11.81 |
| Bugra Gedik | 2 | 2397 | 108.79 |
| Henrique Andrade | 3 | 431 | 23.85 |
| Kun-Lung Wu | 4 | 2849 | 389.90 |