Name
Abstract | ||
|---|---|---|
In a data stream management system (DSMS), users register continuous queries, and receive result updates as data arrive and expire. We focus on applications with real-time constraints, in which the user must receive each result update within a given period after the update occurs. To handle fast data, the DSMS is commonly placed on top of a cloud infrastructure. Because stream properties such as arrival rates can fluctuate unpredictably, cloud resources must be dynamically provisioned and scheduled accordingly to ensure real-time response. It is essential, for the existing systems or future developments, to possess the ability of scheduling resources dynamically according to the current workload, in order to avoid wasting resources, or failing in delivering correct results on time. Motivated by this, we propose DRS, a novel dynamic resource scheduler for cloud-based DSMSs. DRS overcomes three fundamental challenges: (a) how to model the relationship between the provisioned resources and query response time (b) where to best place resources, and (c) how to measure system load with minimal overhead. In particular, DRS includes an accurate performance model based on the theory of Jackson open queueing networks and is capable of handling arbitrary operator topologies, possibly with loops, splits and joins. Extensive experiments with real data confirm that DRS achieves real-time response with close to optimal resource consumption. |
Year | DOI | Venue |
|---|---|---|
2015 | 10.1109/ICDCS.2015.49 | 2015 IEEE 35th International Conference on Distributed Computing Systems |
Keywords | Field | DocType |
resource scheduling,data stream analytics | Data stream management system,Fair-share scheduling,Scheduling (computing),Computer science,Computer network,Provisioning,Queueing theory,Resource allocation,Dynamic priority scheduling,Cloud computing,Distributed computing | Journal |
Volume | ISSN | Citations |
abs/1501.03610 | 1063-6927 | 32 |
PageRank | References | Authors |
0.97 | 31 | 6 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Tom Z. J. Fu | 1 | 76 | 4.24 |
| Jianbing Ding | 2 | 68 | 4.72 |
| Richard T. B. Ma | 3 | 620 | 51.15 |
| Marianne Winslett | 4 | 3519 | 744.78 |
| Yin Yang | 5 | 1003 | 52.10 |
| Zhenjie Zhang | 6 | 1288 | 61.63 |