Title
Operating system interface obfuscation and the revealing of hidden operations
Abstract
Many software security solutions--including malware analyzers, information flow tracking systems, auditing utilities, and host-based intrusion detectors--rely on knowledge of standard system call interfaces to reason about process execution behavior. In this work, we show how a rootkit can obfuscate a commodity kernel's system call interfaces to degrade the effectiveness of these tools. Our attack, called Illusion, allows user-level malware to invoke privileged kernel operations without requiring the malware to call the actual system calls corresponding to the operations. The Illusion interface hides system operations from user-, kernel-, and hypervisor-level monitors mediating the conventional system-call interface. Illusion alters neither static kernel code nor read-only dispatch tables, remaining elusive from tools protecting kernel memory. We then consider the problem of Illusion attacks and augment system call data with kernel-level execution information to expose the hidden kernel operations. We present a Xen-based monitoring system, Sherlock, that adds kernel execution watchpoints to the stream of system calls. Sherlock automatically adapts its sensitivity based on security requirements to remain performant on desktop systems: in normal execution, it adds 1% to 10% overhead to a variety of workloads.
Year
DOI
Venue
2011
10.1007/978-3-642-22424-9_13
DIMVA
Keywords
DocType
Volume
hidden operation,kernel memory,illusion interface hides system,kernel execution watchpoints,actual system,standard system call interface,privileged kernel operation,system interface obfuscation,hidden kernel operation,commodity kernel,xen-based monitoring system,desktop system,hypervisor,intrusion detection systems,malicious software,virtual machine
Conference
6739
ISSN
Citations 
PageRank 
0302-9743
11
0.59
References 
Authors
50
4
Name
Order
Citations
PageRank
Abhinav Srivastava139324.48
Andrea Lanzi284540.99
Jonathon Giffin337915.67
Davide Balzarotti42040113.64