Paper Info
Title
Towards Increasing the Error Handling Time Window in Large-Scale Distributed Systems Using Console and Resource Usage Logs
Abstract
Resource-intensive applications such as scientific applications require the architecture or system on which they execute to display a very high level of dependability to reduce the impact of faults. Typically, the state of the underlying system is captured through messages that are recorded in a log file, which has been proven useful to system administrators in understanding the root-causes of system failures (and for their subsequent debugging). However, the time window between when the first error message is detected in the log file and time of the ensuing failure may not be large enough to allow the administrators to save the state of the running application, which will result in lost execution time. We thus address this fundamental question: Is it possible to extend this time window? The answer is positive: We show that, by using (i) resource usage logs to track anomalous resource usage and (ii) error logs to identify root-causes of system failures, it is possible to increase the time window, on average, by 50 minutes. These files were those obtained for the Ranger Supercomputer from TACC. We achieve this by applying anomaly detection techniques on resource usage data and conducting a root-cause analysis on error log files.
Year
DOI
Venue
2015
10.1109/Trustcom-BigDataSe-ISPA.2015.613
TrustCom/BigDataSE/ISPA
Keywords
Field
DocType
Anomaly detection, Resource usage data, Error logs, PCA, Root-cause analysis, large-scale HPC systems
Anomaly detection,Dependability,Supercomputer,Computer security,Computer science,Root cause analysis,Real-time computing,Execution time,Usage data,Operating system,Debugging
Conference
Volume
ISSN
Citations 
3
2324-9013
0
PageRank 
References 
Authors
0.34
16
5
Name
Order
Citations
PageRank
Nentawe Gurumdimma131.40
Arshad Jhumka236131.79
Maria Liakata337530.40
Edward Chuah4345.04
James C. Browne5998300.57