Title
Compressing Large Amounts of NetFlow Data Using a Pattern Classification Scheme
Abstract
The storage of large amounts of network data is a challenging problem, in particular if it still needs to be actively consulted as for example in the case of network forensics. Here we propose a method to compress NetFlow data while simultaneously adding domain knowledge. Our method is based on a pattern classification scheme by considering all flows from a single source IP address simultaneously. Each pattern can be described by at most 19 attributes that give a good statistical description of the original NetFlow data, while minimising information loss. We estimate that on average a factor of about 300 in storage space can be gained. The process is explained using a real world dataset from a large, high-speed, network, and a formal rationale is provided.
Year
DOI
Venue
2016
10.1109/BigDataSecurity-HPSC-IDS.2016.69
2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC), and IEEE International Conference on Intelligent Data and Security (IDS)
Keywords
DocType
Citations 
pattern classification,data summarisation,compression,storage,netflow
Conference
0
PageRank 
References 
Authors
0.34
0
5
Name
Order
Citations
PageRank
Rémon Cornelisse100.34
Mortaza S. Bargh219921.12
Sunil Choenni3309111.82
Debora Moolenaar400.34
Luc V. De Zeeuw500.34