The document discusses analyzing and improving kernel security. It describes how kernels work and why kernel security is important. Methods for analyzing kernel security like DIGGER are presented, which can identify critical kernel objects like pointers without prior knowledge. The document also discusses approaches for improving kernel security, such as protecting generic pointers with techniques like Sentry that control access to kernel data structures through object partitioning. Future work areas include automatically detecting all kernel data structures and expanding Sentry's protections.

1. Analyzing Kernel Security
and
Approaches for Improving It
Milan Rajpara
IT Systems and Network Security
Gujarat Technological University
C DAC
Ahmedabad
Pune

2. Agenda
• Kernel Introduction
• Necessity for Kernel Security
• Kernel breach
• Analyzing Kernel Security
• Improving Approaches
• Future Work
Milan Rajpara
October 8, 2013
2

3. What is Kernel ?
• A computer program that manages
input/output requests from software
and translates them into data
processing instructions for the
central processing unit and other
electronic components of a
computer. [Wikipedia]
• The kernel is a fundamental part of a
modern computer's operating
system.
• OS rests on a outer ring, and
application above that.
Fig: Privilege rings for the x86 available in protected mode
[Source: Wikipedia]
Milan Rajpara
October 8, 2013
3

4. Necessity for Kernel Security
• Kernel, a vary basic (core) part of the Operating Systems
• Single vulnerability will be exposes large number of systems
• Increasing of Cloud Usage with Virtual Systems
• Smartphones now is in every hand
Milan Rajpara
October 8, 2013
4

5. We talk on ..
• Kernels for General Purpose Operating System
• Some Linux flavor gives Server Optimized Kernel
• Ex. Ubuntu older then 12.04, were gave this option. Since 12.04, linux-image-server is merged into linuximage-generic, there is no difference between Generic and Server kernel. [4]
• Windows do not disclose.
• Kernels which Constructed in C language
• Almost kernels are in C
• Improvement for Monolithic kernels
• All work performed in Virtual environment
• The Xen, and VMware used
Milan Rajpara
October 8, 2013
5

6. How Kernel Affected ?
• By Kernel level rootkits
• Manipulating pointers
• Manipulating data
• Direct Kernel Object Manipulation (DKOM)
• By Boot-kits
• Via hooking techniques
• Direct Hardware or Firmware injection
Milan Rajpara
October 8, 2013
6

7. Effect of this Attacks
• Escalate a process’ privileges by overwriting the process’ credentials
• Hide itself by illicitly removing data structures identifying their presence from
loaded drivers
• Eliding task structures for the processes from the kernel’s process accounting list
• Alter the overall behavior of OS without injecting any malicious code into the
kernel address space, by just pointer manipulating.
Milan Rajpara
October 8, 2013
7

8. How to analyze the Kernel Security
• Find the most critical objects of the kernel, without prior knowledge of the OS
kernel data layout in memory
• Identifying OS Kernel Objects for Run-time Security Analysis
• Sort-out objects which are vulnerable to hijack
• Do Kernel Data Disambiguation
• This will make the system easy to analyze
Milan Rajpara
October 8, 2013
8

9. Most critical objects in Kernel
• Windows and Linux, the core kernel part are mostly written in C
• 40% inter-data structure relations are Pointer based
• 35% of these are Generic Pointers
• Pointers which defines at run time, no initial value or data type is associated
• 28% kernel data structure are well known objects
Milan Rajpara
October 8, 2013
9

10. Generic Pointer Problem
• It is the weak link in kernel security
• Use of void pointers *, assists hackers to point somewhere else
• Use of NULL pointers (to implements linklist), helps hackers to hide / change
runtime objects.
• Use of Casting in C
• Enables the hackers to exploit data structure layout in physical memory
Milan Rajpara
October 8, 2013
10

11. To Find Critical Objects
1. Memory Mapping techniques
• Travers address space from global variables via pointer dereferencing until reaching
running object.
• according to a predefined kernel data definition for each kernel version.
2. Value Invariant Approaches
• Use the value invariants of certain fields or of a whole data structure as a signature to
scan the memory for matching running instances. Ex. DeepScanner, DIMSIM
• Drawbacks of this approaches
- Not very accurate
- Require a predefined definition of the kernel data layout
- Not effective when memory mapping and object reachability information is not available.
- High performance overhead
Milan Rajpara
October 8, 2013
11

12. To Find Critical Objects
3. DIGGER
[1]
• Uncover all system runtime objects without any prior knowledge of the OS kernel
data layout in memory.
• First it performs offline and constructs type-graph (which is used to enable
systematic memory traversal of the object details).
• Then it uses the 4-byte pool memory tagging schema (to uncover kernel runtime
objects from the kernel address space.)
• (+)
• Accurate result
• Low performance overhead
• Fast and nearly complete coverage
Milan Rajpara
October 8, 2013
12

13. DIGGER & KDD
• DIGGER uses the KDD (Kernel Data Disambiguator) to precisely models the
direct and indirect relations between data structures.
• KDD is a static analysis tool that operates offline on an OS kernel’s source code
• Generates a type-graph for the kernel data with direct and indirect relations
between structures, models data structures [2]
• KDD disambiguates pointer-based relations (including generic pointers)
• by performing static points-to analysis on the kernel’s source code.
• Points-to analysis is the problem of determining statically a set of locations to
which a given variable may point to at runtime.
Milan Rajpara
October 8, 2013
13

14. KDD Operation
Source: Ref [2]
AST: Abstract Syntax Tree (high-level intermediate representation for the source code )
Milan Rajpara
October 8, 2013
14

15. KDD Operation
• Interprocedural Analysis 1: Takes AST and differentiate it
• Gets: Variables, Procedure definition, Procedure call, etc.. .
• Interprocedural Analysis 2: Do points-to analysis across different files to perform
whole-program analysis.
• Context Sensitive Analysis:
• It uses Procedure Dependency Graph (PDG) consists of nodes representing the statements of the
data dependency in the program.
• context-sensitive analysis solves two problems: the calling context and the indirect (implicit)
relations between nodes.
Milan Rajpara
October 8, 2013
15

16. Soundness and Precision of KDD
• The points-to analysis algorithm is sound if the points-to set for each variable
contains all its actual runtime targets, and is imprecise if the inferred set is larger
than necessary.
• Check on C programs from the SPEC2000 and SPEC2006 benchmark suites.
• Achieved a high level of precision and 100% of soundness.
• And 96% precision on Windows (WRK*, Vista) and Linux kernel (v3.0.22). [2]
*WRK – Windows Research Kernel, the only available code from windows [6]
Milan Rajpara
October 8, 2013
16

17. DIGGER Approach
Source: Ref [1]
Milan Rajpara
October 8, 2013
17

18. DIGGER Approach
• Static Analysis Component: from KDD
• Signature Extraction Component:
• When the object manager allocates a memory pool block, it associates with a pool tag
(pool tag is a unique four-byte tag for each object type.) Uses this tag to uncover the
kernel objects running instances, and they are static and cannot be changed during
object runtime.
• Dynamic Memory Analysis Component: Extract the object details,
• From Pool Tag, it gets the pool block start memory address and the object’s start
address.
Milan Rajpara
October 8, 2013
18

19. Analyzing Kernel through DIGGER Gives …
• Disambiguate the points-to relations between data structures, all without any
prior knowledge of the OS kernel data layout.
• Robust and quite small signature size to uncover runtime objects, enhancing
performance
• Able to keep track of all critical objects of kernel
Milan Rajpara
October 8, 2013
19

20. Protection of Kernel
• Protect the generic pointers.
• Microsoft added a feature PatchGuard, which blocks kernel mode drivers from
altering sensitive parts of the Windows kernel.
• But TDL (rootkit) manages to circumvent this protection as well, by altering a machine's MBR so
that it can intercept Windows startup routines. [7]
• One approach is use of “Object Partitioning” to protect kernel data structure. [3]
• Uses Sentry, that creates access control protections for security-critical kernel data.
Milan Rajpara
October 8, 2013
20

21. Sentry Architecture
• Sentry protects critical data and
enforces data access restrictions
based upon the origin of the access
within the code of the kernel and its
modules or drivers. [3]
• The data integrity model is
straightforward and matches that of
the Biba ring policy [9]
• The malicious code that modifies
privileges by directly writing to
memory is in a loaded module and
not in the core kernel code, so Sentry
will prevent the write
Milan Rajpara
October 8, 2013
21

22. Kernel Memory Access Control
• Protect data structure from DCOM
• Sentry’s design uses a hypervisor to remain isolated from an untrusted kernel
• To keep the overhead low, Sentry uses memory partitioning to lay out sensitive
data on separate memory pages and protects those pages using the hypervisor
• The policy enforcer mediates attempted writes to protected data and uses the
policy to determine when writes should be permitted.
Milan Rajpara
October 8, 2013
22

23. Working of Sentry
• Identifying Security-Critical Members
• Activation of mediated access
• Instruction emulation
• Secure execution history extraction
Milan Rajpara
October 8, 2013
23

24. Evaluation of Sentry
• Performance
• Low performance overhead
• more performance van be achieved by memory layout optimization
• False Positive Analysis
• There were no instances when security-critical kernel data protected by Sentry was
directly modified by a benign driver.
• Sentry provided a 100% detection rate for DKOM rootkits
Milan Rajpara
October 8, 2013
24

25. Future Work
• Detect all kernel data structures automatically, beyond the kernel version
• The DIGGER can only be used to analyze Windows Kernels.
• The current prototype of Sentry only protects two key structures.
• Other kernel data structures may also require similar protection.
• This may gives versatile performance of Sentry, (if more data structure included)
Milan Rajpara
October 8, 2013
25

26. References
[1] Amani S. Ibrahim, James Hamlyn-Harris, John Grundy, Mohamed Almorsy, "Identifying OS Kernel Objects for
Run-Time Security Analysis", DOI: 10.1007/978-3-642-34601-9_6
[2] Amani S. Ibrahim, John Grundy, James Hamlyn-Harris, Mohamed Almorsy, "Operating System Kernel Data
Disambiguation to Support Security Analysis", DOI: 10.1007/978-3-642-34601-9_20
[3] Abhinav Srivastava, Jonathon Giffin, "Efficient Protection of Kernel Data Structures via Object Partitioning", DOI:
10.1145/2420950.2421012
[4] RFC: Linux kernel merging. https://lists.ubuntu.com/archives/kernel-team/2011-October/017471.html
[5] Rootkits detail by Symantec http://www.symantec.com/avcenter/reference/windows.rootkit.overview.pdf
[6] Windows Research Kernel https://www.facultyresourcecenter.com/curriculum/pfv.aspx?ID=7366&c1=enus&c2=0
[7] TDL Rootkit: http://www.theregister.co.uk/2010/11/16/tdl_rootkit_does_64_bit_windows
[8] Windows hooks: http://msdn.microsoft.com/en-us/library/ms644959(v=vs.85).aspx
[9] K. J. Biba. Integrity considerations for secure computer systems. Technical Report MTR-3153, Mitre, Apr. 1977
Milan Rajpara
October 8, 2013
26

27. Thank you 
Questions
__________________________
- Milan Rajpara
27