Paper Info
Title
OILog: An online incremental log keyword extraction approach based on MDP-LSTM neural network
Abstract
Log keyword extraction is an indispensable part of log anomaly detection. There are two main challenges in keyword extraction, one is that the essence of logs is unstructured, and different vendors usually define different log formats, the other one is that the most of the traditional method cannot update the log keywords incrementally to match the newly generated log data, so the extraction accuracy is low. To solve these problems, we introduce an online incremental keyword extraction method OILog. The essential idea of this method is that log templates are usually the longest combination of high-frequency words. OILog builds models by using a deep Long Short-Term Memory network (LSTM) for capturing both high-frequency log keywords in real-time and new log keywords generated by the system, which can transform unstructured raw logs into structured logs quickly. To improve the efficiency and accuracy of the model, we proposed an improved particle swarm optimization algorithm, which changes the traditional topology structure of Particle Swarm Optimization algorithm (PSO) into a multilayer structure and applies a new particle velocity update formula to increase the attraction between particles. We summarized the previous works and validated OILog using real log data collected from four systems. The results show that OILog has superiority in terms of both accuracy and robustness.
Year
DOI
Venue
2021
10.1016/j.is.2020.101618
Information Systems
Keywords
DocType
Volume
Keyword extraction,Optimization algorithm,Text mining,LSTM,Deep learning
Journal
95
ISSN
Citations 
PageRank 
0306-4379
2
0.37
References 
Authors
0
5
Name
Order
Citations
PageRank
Xiaoyu Duan1938.38
Shi Ying233431.11
Hailong Cheng320.71
Wanli Yuan420.71
Xiang Yin541.75