Name
Abstract | ||
|---|---|---|
There has been growing interest in building large-scale distributed monitoring systems for sensor, enterprise, and ISP networks. Recent work has proposed using principal component analysis (PCA) over global traffic matrix statistics to effectively isolate network-wide anomalies. To allow such a PCA-based anomaly detection scheme to scale, we propose a novel approximation scheme that dramatically reduces the burden on the production network. Our scheme avoids the expensive step of centralizing all the data by performing intelligent filtering at the distributed monitors. This filtering reduces monitoring bandwidth overheads, but can result in the anomaly detector making incorrect decisions based on a perturbed view of the global data set. We employ stochastic matrix perturbation theory to bound such errors. Our algorithm selects the filtering parameters at local monitors such that the errors made by the detector are guaranteed to lie below a user-specified upper bound. Our algorithm thus allows network operators to explicitly balance the tradeoff between detection accuracy and the amount of data communicated over the network. In addition, our approach enables real-time detection because we exploit continuous monitoring at the distributed monitors. Experiments with traffic data from Abilene backbone network demonstrate that our methods yield significant communication benefits while simultaneously achieving high detection accuracy. |
Year | DOI | Venue |
|---|---|---|
2007 | 10.1109/INFCOM.2007.24 | INFOCOM |
Keywords | Field | DocType |
intelligent filtering,stochastic processes,communication-efficient online detection,stochastic matrix perturbation theory,matrix algebra,realtime detection,anomaly detection,network-wide anomalies,distributed monitoring systems,abilene backbone network,telecommunication security,principal component analysis,distributed processing,security of data,statistical analysis,stochastic matrix,detectors,statistical distributions,real time,remote monitoring,filtering,data reduction,perturbation theory,production,sensor network,distributed system,intelligent control,data collection,time series data | Anomaly detection,Upper and lower bounds,Computer science,Filter (signal processing),Real-time computing,Continuous monitoring,Bandwidth (signal processing),Backbone network,Detector,Principal component analysis | Conference |
ISSN | ISBN | Citations |
0743-166X | 1-4244-1047-9 | 63 |
PageRank | References | Authors |
3.37 | 20 | 7 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Ling Huang | 1 | 2496 | 118.80 |
| Xuanlong Nguyen | 2 | 63 | 3.37 |
| Minos Garofalakis | 3 | 4904 | 664.22 |
| Joseph M. Hellerstein | 4 | 14093 | 1651.14 |
| Michael I. Jordan | 5 | 31220 | 3640.80 |
| D. Joseph | 6 | 5463 | 492.96 |
| Nina Taft | 7 | 2109 | 154.92 |