Title
Detecting Kernel-Level Rootkits Using Data Structure Invariants
Abstract
Rootkits affect system security by modifying kernel data structures to achieve a variety of malicious goals. While early rootkits modified control data structures, such as the system call table and values of function pointers, recent work has demonstrated rootkits that maliciously modify noncontrol data. Most prior techniques for rootkit detection have focused solely on detecting control data modifications and, therefore, fail to detect such rootkits. This paper presents a novel technique to detect rootkits that modify both control and noncontrol data. The main idea is to externally observe the execution of the kernel during an inference phase and hypothesize invariants on kernel data structures. A rootkit detection phase uses these invariants as specifications of data structure integrity. During this phase, violation of invariants indicates an infection. We have implemented Gibraltar, a prototype tool that infers kernel data structure invariants and uses them to detect rootkits. Experiments show that Gibraltar can effectively detect previously known rootkits, including those that modify noncontrol data structures.
Year
DOI
Venue
2011
10.1109/TDSC.2010.38
Dependable and Secure Computing, IEEE Transactions
Keywords
Field
DocType
data structures,invasive software,Gibraltar,data structure integrity,data structure invariants,kernel-level rootkits detection,noncontrol data modification,Kernel-level rootkits,invariant inference,noncontrol data attacks,static and dynamic program analysis.
Kernel (linear algebra),Data structure,Data mining,Function pointer,Inference,Computer science,Rootkit,System call,Invariant (mathematics)
Journal
Volume
Issue
ISSN
8
5
1545-5971
Citations 
PageRank 
References 
36
1.24
32
Authors
3
Name
Order
Citations
PageRank
Arati Baliga127516.48
Vinod Ganapathy271342.69
Liviu Iftode32112148.14