Title
Combining Tensor Decompositions and Graph Analytics to Provide Cyber Situational Awareness at HPC Scale
Abstract
This paper describes MADHAT (Multidimensional Anomaly Detection fusing HPC, Analytics, and Tensors), an integrated workflow that demonstrates the applicability of HPC resources to the problem of maintaining cyber situational awareness. MADHAT combines two high-performance packages: ENSIGN for large-scale sparse tensor decompositions and HAGGLE for graph analytics. Tensor decompositions isolate coherent patterns of network behavior in ways that common clustering methods based on distance metrics cannot. Parallelized graph analysis then uses directed queries on a representation that combines the elements of identified patterns with other available information (such as additional log fields, domain knowledge, network topology, whitelists and blacklists, prior feedback, and published alerts) to confirm or reject a threat hypothesis, collect context, and raise alerts. MADHAT was developed using the collaborative HPC Architecture for Cyber Situational Awareness (HACSAW) research environment and evaluated on structured network sensor logs collected from Defense Research and Engineering Network (DREN) sites using HPC resources at the U.S. Army Engineer Research and Development Center DoD Supercomputing Resource Center (ERDC DSRC). To date, MADHAT has analyzed logs with over 650 million entries.
Year
DOI
Venue
2019
10.1109/HPEC.2019.8916559
2019 IEEE High Performance Extreme Computing Conference (HPEC)
Keywords
Field
DocType
HPC resources,MADHAT,graph analytics,HPC scale,integrated workflow,high-performance packages,large-scale sparse tensor decompositions,clustering methods,parallelized graph analysis,network topology,structured network sensor logs,HPC architecture for cyber situational awareness,multidimensional anomaly detection fusing HPC Analytics and Tensors
Data science,Tensor,Computer science,Situation awareness,Graph analytics
Conference
ISSN
ISBN
Citations 
2377-6943
978-1-7281-5021-5
2
PageRank 
References 
Authors
0.36
9
11