Name
Abstract | ||
|---|---|---|
Real-time stream processing has become increasingly important in recent years and has led to the development of a multitude of stream processing systems. Given the varying job workloads that characterize stream processing, these systems need to be tuned and adjusted to maintain performance targets in the face of variation in incoming traffic. Current auto-scaling systems adopt a series of trials to approach a job's expected performance due to a lack of performance modelling tools. We find that general traffic trends in most jobs lend themselves well to prediction. Based on this premise, we built a system called Caladrius that forecasts the future traffic load of a stream processing job and predicts its processing performance after a proposed change to the parallelism of its operators. Experimental results show that Caladrius is able to estimate a job's throughput performance and CPU load under a given scaling configuration. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/ICDE.2019.00204 | 2019 IEEE 35th International Conference on Data Engineering (ICDE) |
Keywords | Field | DocType |
Topology,Throughput,Containers,Measurement,Predictive models,Parallel processing,Analytical models | Data mining,Traffic load,Computer science,Parallel processing,Operator (computer programming),Throughput,Cpu load,Stream processing,Scaling,Distributed computing | Conference |
ISSN | ISBN | Citations |
1084-4627 | 978-1-5386-7474-1 | 1 |
PageRank | References | Authors |
0.38 | 0 | 14 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Faria Kalim | 1 | 4 | 2.21 |
| Thomas Cooper | 2 | 3 | 0.76 |
| Huijun Wu | 3 | 8 | 3.19 |
| Yao Li | 4 | 1 | 2.41 |
| Ning Wang | 5 | 230 | 87.46 |
| Neng Lu | 6 | 1 | 0.38 |
| Maosong Fu | 7 | 269 | 8.98 |
| Xiaoyao Qian | 8 | 1 | 0.38 |
| Hao Luo | 9 | 18 | 7.77 |
| Da Cheng | 10 | 1 | 0.38 |
| Yaliang Wang | 11 | 1 | 1.06 |
| Fred Dai | 12 | 1 | 0.38 |
| Mainak Ghosh | 13 | 63 | 4.63 |
| Beinan Wang | 14 | 1 | 2.07 |