Download Section for Related Work % \section{Related Work} Rootkit detection

%
% Section for Related Work
%
\section{Related Work}
Rootkit detection and prevention techniques have been heavily researched
in the context of servers and personal computers. However, there has been
little research to date with respect to rootkit detection or prevention
methods on smartphones such as Android. But, given the fact that the
Android framework is built on top of the Linux kernel, many of the
proposed methods to detect rootkits on traditional operating systems
still apply to at least some extent.
There are a number of significant differences between servers and PCs and
smartphone platforms that must be accounted for in any security system
designed for a mobile device. For example, unlike servers and desktop
PCs, smartphones rely on battery power and power consumption is a major
concern of any proposed solution. Therefore, any proposed system to
conquer rootkits on smartphones must also be efficient in terms of power
usage or it would not be a usable solution. Also, the unique sensitive
information, such as location and personal data present on a smartphone
is concentrated in fewer places throughout the system than is the case on
a PC, making the information easier to gather by malicious software.
Most current trends in rootkit detection or prevention propose user level
or kernel level inspection methods, use of a VMM, or variations of
additional hardware. Generally, the existing solutions apply no special
or additional protection to sensitive resources and thus seem inadequate
for smartphone use. Individually, we were unable to find an existing
solution that was both sufficient and efficient enough to directly apply
to the Android platform.
%User level Inspection
\subsection{User Level Inspection}
One proposed approach to detecting rootkits on smartphones is to offload
the work of running detection or prevention software on a remote
computer. In the proposed scheme, the mobile phone's file system contents
are hashed and uploaded to the remote checking computer each time it is
connected. The remote computer then verifies the hashes of files that
have changed since the last time they were checked
\cite{dixon2010rootkit}. The problem with this approach is that a rootkit
could modify the hashes or file contents as they are uploaded to the
remote checking authority, and thus this approach does not seem feasible
to detect modern behavior of rootkits.
There also exist many user level tools that check for the presence of
rootkits. One such tool, Tripwire, is a file and directory integrity
checker \cite{tripwire}. Another tool that is in existence is called
chkrootkit, which is a tool that aims to check for many different signs
of rootkits. The chkrootkit tool checks system binaries to detect
modification, looks for network interfaces that are set to promiscuous
mode, and checks various system logs \cite{chkrootkit}. The problem with
any user level inspection method is the fact that there is an implicit
need to trust the underlying operating system’s interface. This is not
sufficient due to kernel level rootkits that modify kernel level system
calls and data structures \cite{adore, knark}. User level inspection is
only applicable to user level rootkits, which simply replace system
binaries with malicious modified versions.
\subsection{Kernel Level Inspection}
Binary analysis has also been utilized to detect kernel level rootkits at
the time of their loading. Kruegel et. al propose checking kernel modules
as they are loaded to see if they appear to contain code that exhibits
common rootkit behaviors. The authors argue that rootkit behavior is
deterministically different at the binary level when compared to the
behavior of non-malicious kernel modules. The method first sets up
criteria to define undesirable behavior and then statically analyzes
instruction sequences in the kernel module binary for traces of the
undesirable behavior \cite{kruegel2004detecting}. The downside of this
approach is that static analysis cannot be used to determine all dynamic
actions taken by a malicious module and the detection is only as good as
the initial criteria setup to identify malicious behavior. This models
typical virus scanning techniques, which are limited in capability due to
a lack of prior knowledge of known attacks, polymorphic code, and
packers.
Gibraltar, another behavior-based kernel inspection rootkit detection
tool, detects kernel level rootkits by using an initial training period
in which invariants on kernel data structures are created
\cite{baliga2008automatic}. Due to the way that Gilbratar monitors the
kernel, it must reside on a separate physical machine from the operating
system under inspection. This approach is not easily applicable to
smartphone environments due to the fact that a separate physical machine
is required. In addition, the initial training period can be quite
extensive, and can lead to false positives if inconsistencies are
encountered during the initial training period.
Many kernel level inspection techniques aim to detect modifications to
the system call table or the underlying kernel data structures themselves
as these are two highly common attack points for rootkits. It is standard
behavior for a kernel-mode rootkit to overwrite system call table
pointers to instead point to rootkit functions. These modified system
service calls perform checks on the system call data passed back to a
user process and hide information about rootkit files and processes or
otherwise compromise and alter the behavior of the system. Levine et al.
propose a methodology to detect rootkits that overwrite the system call
table as well as the code that implements those system calls by comparing
the contents of the system call table and system call implementations to
archived copies of the system call table stored in files
\cite{levine2004methodology}. While the Levine et al. approach is a
potential component of a security system on the Android platform, it does
not take the sensitive resources present on a smartphone into account.
Another similar approach was implemented in a tool called HookScout,
which has an analysis subsystem and a detection subsystem. The analysis
subsystem performs static and dynamic binary analysis on the operating
system in order to generate a policy for hook detection
\cite{yin2010hookscout}. The detection subsystem enforces policies
generated by the earlier analysis to actively detect hooks at runtime.
The HookScout program uses context-sensitive policies to detect kernelwide – not just system call – function pointer changes in an operating
system. This is a very computationally expensive protection, however, and
would thus cause a significant battery drain on a smartphone or mobile
device.
The fundamental problem with methods of rootkit detection tools that
execute as part of the kernel of the operating system being monitored is
that the code and data performing these detection techniques share the
same address space as the malicious code. In other words, kernel-mode
rootkits also execute as an extension to the operating system's kernel,
and any approach that lives solely at this level of the software stack is
also subject to manipulation by the malicious rootkit kernel code. It is
for this reason that we believe rootkit detection mechanisms require
presence in a layer below the target operating system, such as part of
the hardware or as a security extension to a VMM hosting the smartphone
OS.
\subsection{VMM Level Inspection}
One popular approach to detecting rootkits is to make use of a VMM in
order to remove the need to trust the underlying operating system. The
fact that the VMM lives a layer below the guest operating system removes
the need to trust data returned by the OS and provides isolation from the
hostile environment. Along with virtualization comes increased
computational requirements and battery usage. Another concern is that
VMMs are in the early stages of development on smartphones, and supported
open source VMMs do not currently exist for Android. Still, this approach
immediately grabbed our attention as having potential to be at least part
of a solution to our problem.
A technique called state-based control-flow integrity (SBCFI) proposed by
Petroni et al. dynamically monitors the operating system kernel’s
integrity \cite{petroni2007automated}. In this approach, a virtual
machine monitor is used to detect unexpected modifications of the
underlying kernel's control flow graph (CFG). Two virtual machines exist
in the SBCFI mechanism; one VM that runs the target OS, and one that runs
the monitor OS. This way, the monitor VM can periodically detect controlflow integrity violations to the CFG that occur on the target VM. The
results of this technique proved successful, and the authors claimed to
be able to detect 24 of 25 Linux rootkits. However, the technique
incurred significant performance costs and may be inapplicable on
resource limited smartphones. The authors noted that a single check of
the target VM state took close to a second and consumed up to 1 GB of
memory on their desktop environment.
Microsoft also published a VMM based system called HookMap which assumes
that a persistent rootkit will eventually need to replace certain kernel
hook points in order to prevent detection from security programs within
the kernel \cite{wang2008countering}. A virtual machine monitor is used
to map the system calls and internal kernel hook points used by security
programs such as 'ls' and 'netstat' on a traditional Linux operating
system. In essence, each call or jmp instruction is identified as a hook
attack point (HAP) during the execution of a security program on a
traditional operating system. Then, with the help of a VMM, the system
can detect any changes in the HAPs while running programs on the target
OS.
Another proposed method is to use a VMM to enforce policies on kernel
data states. For example, one technique used by rootkits is to
dynamically change kernel data structures by using direct memory access
(DMA). It is known that a kernel function, \_\_generic\_copy\_from\_user
actually copies data from user space to a location in the kernel on
Linux. Rhee et al. propose monitoring kernel functions to ensure that DMA
is not performed on protected kernel structures \cite{rhee2009defeating}.
The problem with this approach is like that of any policy mechanism; the
protection is only as good as the policies themselves. So, the technique
relies on the completeness and enforceability of the provided policies.
A tool called XView makes use of a VMM in order to intercept system calls
from the guest operating system to reconstruct semantic-level process
information. In this way, XView is a dynamic "cross view" approach that
keeps a separate list of processes apart from the guest operating
system's copy. The approach also aims to detect direct kernel object
manipulation (DKOM) events. Basically, through the means of introspection
via the VMM, the tool is able to audit the target kernel without relying
on the target operating system data directly \cite{xudynamic}.
\subsection{Hardware Level Inspection}
Analogous to VMM introspection, hardware based introspection provides
isolation from the operating system being monitored. For example, Copilot
\cite{petroni2004copilot} utilizes an additional PCI card to access the
operating system memory through DMA. This approach takes away the need to
trust the operating system, but PCI cards obviously cannot be added to
smartphones. The Copilot approach also suggests a communication link upon
which to communicate with a remote administrator station. This would
require a large amount of power consumption due to constantly reading
memory contents and transmitting data to a remote host.
Saile et al. propose using a trusted platform module (TPM) to monitor the
integrity of an operating system \cite{sailer2004design}. The TPM chip is
additional hardware that aims to be tamper-proof, perform integrity
measurements on the operating system, and relay those measurements to a
remote authority for verification. The remote authority implements a
challenge-response protocol in order to receive integrity measurements
from the TPM residing on the device to be verified. The remote authority
verifies the software running on the remote device by comparing precomputed hash values to those returned from the TPM. The TPM implemented
by Saile et al, which was based on the TCG standards, was able to detect
a Linux rootkit that replaced the syslogd binary with a trojaned version.
Along with hardware additions, entire system architectures have been
devised that aim to detect rootkit exploits. Secure Hardware support
Against RootKits (SHARK), is an architecture which aims to provide
system-level security against rootkits without needing to trust the
software stack \cite{vasisht2008shark}. SHARK is a process context-aware
architecture that moves the control of operating system processes into a
hardware engine called the SHARK Security Manager (SSM). This way,
knowledge of context and running processes is controlled and revealed by
the hardware, rather than relying on the operating system. The SHARK
system claims to be able to detect malware and rootkits at the user,
kernel, and VMM levels of the software stack due to the architectural
changes and hardware cooperation.
The problem with hardware solutions on mobile devices such as smartphones
is the fact that modifying or redesigning hardware is an expensive
solution with potentially limited scalability. Smartphones are already
very restrictive in terms of cost and hardware footprint. Also, there is
a lack of certain standardized hardware interfaces such as PCI in which
additional cards could be plugged into to gain DMA. Due to the cost and
difficulty of such a hardware-based approach, we determined a solution
that requires no hardware modifications to be a desirable choice.