DUSK - Develop at Userland Install into Kernel

DUSK: Develop at User level, inStall in Kernel Alexey Smirnov and Tzi-cker Chiueh ECSL Research Seminar 09/13/05

Outline
Introduction
Related Work
Netfilter Overview
System Architecture
Evaluation
Conclusion

Introduction
Kernel is often extended using kernel modules.
Netfilter – hooks in network component.
Development at user-level is convenient: gdb, ddd, etc.
Kernel modules have lower overhead.
DUSK achieves best of both worlds.

User-level Development of Kernel Extensions
Provide kernel API at user-level. A lot of functions, changes from one version to another.
Implement a new API that resembles kernel API. Not convenient for the programmer, requires code rewriting.

DUSK Overview
DUSK is an extension to GCC that compiles the source code of a kernel module into a user-level program.
It links several helper functions to the user-level module.
Same helper functions for different versions of Linux kernel.

DUSK Framework
User-level component: the kernel module compiled as a user level program.
Kernel-level component: handles user-level module’s requests.
The two components are connected using UDP sockets.
Can run on different architectures: user-level module on x86 running in gdb and kernel-level module on MIPS wireless router.

Related Work
VFS layer: UFS uses NFS loopback API. UpcallFS – a stackable file system.
Device drivers: Userdev provides a user-level library.
Network stack: icTCP exposes TCP stack parameters to user programs. Netfilter simulator runs Netfilter hooks at user level.

Netfilter Architecture
Netfilter adds a set of five hooks to kernel network stack:
A Netfilter module verdict: NF_ACCEPT , NF_DROP , NF_STOLEN , NF_QUEUE , NF_REPEAT .

Netfilter User Mode Modules
NF_QUEUE verdict sends sk_buff to user space using a netlink socket. The queue handler is either standard or programmer-defined.
A user module can only read sk_buff but cannot change kernel memory.

iptables
iptables controls parameters of Linux firewall.
iptables -A PREROUTING –p tcp --destination-port 80 –j QUEUE
iptables –A POSTROUTING –p udp --source-port 5678 –j DROP
Has string matching capabilities. Stateless IDS.

Kernel-level Programming at User Level
User-level kernel modules need to:
Read/write kernel memory;
Call kernel functions;
Access global kernel variables, e. g. current
Programmers can modify source code of kernel module using these blocks to run it at user level.

Three Basic Blocks
Kernel memory access: /dev/kmem
Kernel function calls: new system call sys_exec(). System.map for address lookup.
A new system call sys_kvar() for variable access.
sys_exec() and sys_kvar() are implemented in DUSK kernel module.
Kernel module and user module communicate using UDP sockets.

Using Basic Blocks to Run Kernel Module at User Level
Kernel module source code:
sk_buff->len
User-level code:
Define a shadow variable u_sk_buff;
malloc() memory for it;
Use kmem_read() to copy sk_buff from kernel memory to shadow variable;
Commit changes to u_sk_buff using kmem_write().
Our goal: automatic transformation.

DUSK Framework

DUSK Run-Time Support
Kernel extension registration/de-registration. The address of a function is sent to the kernel.
Processing kernel requests. An appropriate extension handles each request.
Module initialization/cleanup. init_module() and cleanup_module() are called when user-level module starts and terminates.

Extension Management
nfho_post_routing.hook=khook_ip_input;
nfho_post_routing.hooknum = NF_IP_POST_ROUTING;
nf_register_hook(&nfho_post_routing);
Kernel-level DUSK module replaces khook_ip_input with a proxy kernel function and returns extension type and sequence number to the user program.
DUSK clones a new process for each extension. They share same address space.

Blocking and Non-blocking Extensions
Extensions that block: the process that called the extension blocks until the user-level module returns a verdict. Exampe: procfs, timer.
Extensions that don’t block: the kernel callback returns as soon as a message is sent to the user space. Example: Netfilter.

Why Netfilter Is Non-blocking?
Hardware interrupt handler receives network packets.
Runs with interrupts disabled. It saves packets in a queue and invokes software interrupt, then returns.
One software interrupt handler per processor. Hardware interrupts enabled.
A packet is re-injected into the network stack when user-level processing is finished.

Extension Types
DUSK supports Netfilter only.
Timers, threads, procfs, ioctl, etc. are required for advanced modules.

DUSK Run-Time Configuration

Compile-time Support
DUSK adds two functions: check() and commit() .
check() ensures that every buffer accessed using a pointer exists in kernel memory and in user-space.
commit() updates the kernel version when a function is called; it refreshes the user version when the function returns.

Function check()
check() is called for every array access, de-referenced pointer, or &.
It maintains the kernel-to-user address mapping buffer.
For a new user address DUSK calls kmalloc() and kmem_write()
For a new kernel address DUSK calls malloc() and kmem_read().
Addresses under PAGE_OFFSET are user addresses, above are kernel addresses.

Type Information
Type information is gathered at compile-time. Type size, offsets of pointer fields are stored.
Type size is used in check() .
commit() uses field information.

Kernel Function Calls
A kernel function is replaced with:
void_kernel_func() for void functions;
int_kernel_func() for int functions;
Function arguments are written into kernel memory when the function is called.
Each pointer-type argument is wrapped into commit() call.

Function commit()
commit() traverses data types recursively.
Issue: array elements accessed using a pointer: *(a+1), *(a+2). If a is a function parameter then commit() ensures that all elements get synchronized.
Issue: a union’s elements of different type have same offset. Types of the argument and that of the address mapping table are compared.

Function commit()

Function Prolog and Epilog
All changes are committed when user-level module returns a verdict.
DUSK adds a function epilog that commits the changes.
depth variable counts the current nesting level.
Incremented in the function prolog, decremented in the epilog.

check() and commit() Example
printk(“test”);
int_kernel_func(addr_of(“printk”), commit(check(“test”)));

Cross-Platform Debugging
Linksys WRT54gs wireless router: 32 MB RAM, 200 MHz MIPS CPU. Runs Linux 2.4.20.
Cannot run gcc, gdb. Supports Netfilter.
printk() debugging possible. Logs are not stored after reboot.

X86 and MIPS Differences
Data structures are different:
PAGE_OFFSETs are different.
DUSK compiles module twice: with the cross-compiler and with x86 compiler.
Addresses in data structures are changed when data from kernel is received.

Evaluation
nfsiff – FTP password sniffer
POSTROUTING: finds USER and PASS commands;
PREROUTING: replies to special ICMP packets;
lwfw – light-weight firewall
PREROUTING: drops certain packets (port, interface).
ipp2p – P2P traffic classifier
POSTROUTING: scans for P2P signatures.

nfsiff
Sniffer on wireless router:

Compile-time Overhead

Saved Work

Conclusion
DUSK allows to debug kernel modules at user level without changing the source code.
Future work: add better debugger support, i.e. integrate DUSK with gdb and ddd.
Support device driver development.

DUSK - Develop at Userland Install into Kernel

by Alexey Smirnov on Nov 13, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 37

Statistics

DUSK - Develop at Userland Install into Kernel — Presentation Transcript

http://research.alexeysmirnov.name	31
http://www.slideshare.net	6