Title
HookScout: proactive binary-centric hook detection
Abstract
In order to obtain and maintain control, kernel malware usually makes persistent control flow modifications (i.e., installing hooks). To avoid detection, malware developers have started to target function pointers in kernel data structures, especially those dynamically allocated from heaps and memory pools. Function pointer modification is stealthy and the attack surface is large; thus, this type of attacks is appealing to malware developers. In this paper, we first conduct a systematic study of this problem, and show that the attack surface is vast, with over 18, 000 function pointers (most of them long-lived) existing within the Windows kernel. Moreover, to demonstrate this threat is realistic for closed-source operating systems, we implement two new attacks for Windows by exploiting two function pointers individually. Then, we propose a new proactive hook detection technique, and develop a prototype, called HookScout. Our approach is binary-centric, and thus can generate hook detection policy without access to the OS kernel source code. Our approach is also context-sensitive, and thus can deal with polymorphic data structures. We evaluated HookScout with a set of rootkits which use advanced hooking techniques and show that it detects all of the stealth techniques utilized (including our new attacks). Additionally, we show that our approach is easily deployable, has wide coverage and minimal performance overhead.
Year
DOI
Venue
2010
10.1007/978-3-642-14215-4_1
DIMVA
Keywords
Field
DocType
function pointer,new attack,windows kernel,kernel malware,function pointer modification,os kernel source code,malware developer,hook detection policy,attack surface,proactive binary-centric hook detection,kernel data structure,flow,data bases,information security,operating system,source code,control flow,kernel functions,polymorphism,data structure
Data structure,Attack surface,Function pointer,Computer science,Rootkit,Hypervisor,Memory pool,Malware,Operating system,Kernel (statistics),Distributed computing
Conference
ISBN
Citations 
PageRank 
3-642-14214-1
20
1.12
References 
Authors
17
4
Name
Order
Citations
PageRank
Heng Yin12153111.33
Pongsin Poosankam297539.96
Steve Hanna3126167.87
Dawn Song47084442.36