Paper Info
Title
Real time intrusion detection system for ultra-high-speed big data environments.
Abstract
In recent years, the number of people using the Internet and network services is increasing day by day. On a daily basis, a large amount of data is generated over the Internet from zeta byte to petabytes with a very high speed. On the other hand, we see more security threats on the network, the Internet, websites, and the enterprise network. Therefore, detecting intrusion in such ultra-high-speed environment in real time is a challenging task. Many intrusion detection systems (IDSs) are proposed for various types of network attacks using machine learning approaches. Most of them are unable to detect recent unknown attacks, whereas the others do not provide a real-time solution to overcome the above-mentioned challenges. Therefore, to address these problems, we propose a real-time intrusion detection system for ultra-high-speed big data environment using Hadoop implementation. The proposed system includes four-layered IDS architecture, which consists of the capturing layer, filtration and load balancing layer, processing or Hadoop layer, and the decision-making layer. Furthermore, feature selection scheme is proposed that selects nine parameters for classification using (FSR) and (BER), as well as from the analysis of DARPA datasets. In addition, five major machine learning approaches are used to evaluate the proposed system including J48, REPTree, random forest tree, conjunctive rule, support vector machine, and Naïve Bayes classifiers. Results show that among all these classifiers, REPTree and J48 are the best classifiers in terms of accuracy as well as efficiency. The proposed system architecture is evaluated with respect to accuracy in terms of true positive (TP) and false positive (FP), with respect to efficiency in terms of processing time and by comparing results with traditional techniques. It has more than 99 % TP and less than 0.001 % FP on REPTree and J48. The system has overall higher accuracy than existing IDSs with the capability to work in real time in ultra-high-speed big data environment.
Year
DOI
Venue
2016
10.1007/s11227-015-1615-5
The Journal of Supercomputing
Keywords
Field
DocType
Machine learning,Intrusion detection,Threats,Big data,Network
Data mining,Feature selection,Computer science,Load balancing (computing),Support vector machine,Parallel computing,Systems architecture,Random forest,Intrusion detection system,Big data,The Internet,Distributed computing
Journal
Volume
Issue
ISSN
72
9
0920-8542
Citations 
PageRank 
References 
10
0.56
9
Authors
3
Name
Order
Citations
PageRank
muhammad mazhar ullah rathore130121.15
Awais Ahmad237945.85
Anand Paul352746.32