Title
Automatic Inference and Enforcement of Kernel Data Structure Invariants
Abstract
Kernel-level rootkits affect system security by modifying key kernel data structures to achieve a variety of malicious goals. While early rootkits modified control data structures, such as the system call table and values of function pointers, recent work has demonstrated rootkits that maliciously modify non-control data. Prior techniques for rootkit detection fail to identify such rootkits either because they focus solely on detecting control data modifications or because they require elaborate, manually-supplied specifications to detect modifications of non-control data. This paper presents a novel rootkit detection technique that automatically detects rootkits that modify both control and non-control data. The key idea is to externally observe the execution of the kernel during a training period and hypothesize invariants on kernel data structures. These invariants are used as specifications of data structure integrity during an enforcement phase; violation of these invariants indicates the presence of a rootkit. We present the design and implementation of Gibraltar, a tool that uses the above approach to infer and enforce invariants. In our experiments, we found that Gibraltar can detect rootkits that modify both control and non-control data structures, and that its false positive rate and monitoring overheads are negligible.
Year
DOI
Venue
2008
10.1109/ACSAC.2008.29
ACSAC
Keywords
Field
DocType
key kernel data structure,data structure integrity,automatic inference,kernel data structure invariants,kernel-level rootkits,control data modification,non-control data,detects rootkits,kernel data structure,early rootkits,control data structure,non-control data structure,rootkits,entropy,invariants,data structure,function point,data structures,system security,kernel,watermarking,false positive rate,automatic
False positive rate,Data mining,Digital watermarking,Function pointer,Computer security,Computer science,System call,Artificial intelligence,Kernel (linear algebra),Data structure,Inference,Rootkit,Machine learning
Conference
ISSN
Citations 
PageRank 
1063-9527
82
3.57
References 
Authors
21
3
Name
Order
Citations
PageRank
Arati Baliga127516.48
Vinod Ganapathy271342.69
Liviu Iftode32112148.14