DUSK is a framework that allows kernel modules to be developed at the user level by compiling them into a user-level program while still maintaining the performance of running in the kernel. It uses helper functions to connect the user-level component to the kernel-level component, allowing things like debugging and testing to be done at the user level. DUSK supports Netfilter modules initially and aims to provide an easier development process for kernel modules.

1. DUSK: Develop at User level, inStall in Kernel Alexey Smirnov and Tzi-cker Chiueh ECSL Research Seminar 09/13/05

2. Outline Introduction Related Work Netfilter Overview System Architecture Evaluation Conclusion

3. Introduction Kernel is often extended using kernel modules. Netfilter – hooks in network component. Development at user-level is convenient: gdb, ddd, etc. Kernel modules have lower overhead. DUSK achieves best of both worlds.

4. User-level Development of Kernel Extensions Provide kernel API at user-level. A lot of functions, changes from one version to another. Implement a new API that resembles kernel API. Not convenient for the programmer, requires code rewriting.

5. DUSK Overview DUSK is an extension to GCC that compiles the source code of a kernel module into a user-level program. It links several helper functions to the user-level module. Same helper functions for different versions of Linux kernel.

6. DUSK Framework User-level component: the kernel module compiled as a user level program. Kernel-level component: handles user-level module’s requests. The two components are connected using UDP sockets. Can run on different architectures: user-level module on x86 running in gdb and kernel-level module on MIPS wireless router.

7. Related Work VFS layer: UFS uses NFS loopback API. UpcallFS – a stackable file system. Device drivers: Userdev provides a user-level library. Network stack: icTCP exposes TCP stack parameters to user programs. Netfilter simulator runs Netfilter hooks at user level.

8. Netfilter Architecture Netfilter adds a set of five hooks to kernel network stack: A Netfilter module verdict: NF_ACCEPT , NF_DROP , NF_STOLEN , NF_QUEUE , NF_REPEAT .

9. Netfilter User Mode Modules NF_QUEUE verdict sends sk_buff to user space using a netlink socket. The queue handler is either standard or programmer-defined. A user module can only read sk_buff but cannot change kernel memory.

10. iptables iptables controls parameters of Linux firewall. iptables -A PREROUTING –p tcp --destination-port 80 –j QUEUE iptables –A POSTROUTING –p udp --source-port 5678 –j DROP Has string matching capabilities. Stateless IDS.

11. Kernel-level Programming at User Level User-level kernel modules need to: Read/write kernel memory; Call kernel functions; Access global kernel variables, e. g. current Programmers can modify source code of kernel module using these blocks to run it at user level.

12. Three Basic Blocks Kernel memory access: /dev/kmem Kernel function calls: new system call sys_exec(). System.map for address lookup. A new system call sys_kvar() for variable access. sys_exec() and sys_kvar() are implemented in DUSK kernel module. Kernel module and user module communicate using UDP sockets.

13. Using Basic Blocks to Run Kernel Module at User Level Kernel module source code: sk_buff->len User-level code: Define a shadow variable u_sk_buff; malloc() memory for it; Use kmem_read() to copy sk_buff from kernel memory to shadow variable; Commit changes to u_sk_buff using kmem_write(). Our goal: automatic transformation.

14. DUSK Framework

15. DUSK Run-Time Support Kernel extension registration/de-registration. The address of a function is sent to the kernel. Processing kernel requests. An appropriate extension handles each request. Module initialization/cleanup. init_module() and cleanup_module() are called when user-level module starts and terminates.

16. Extension Management nfho_post_routing.hook=khook_ip_input; nfho_post_routing.hooknum = NF_IP_POST_ROUTING; nf_register_hook(&nfho_post_routing); Kernel-level DUSK module replaces khook_ip_input with a proxy kernel function and returns extension type and sequence number to the user program. DUSK clones a new process for each extension. They share same address space.

17. Blocking and Non-blocking Extensions Extensions that block: the process that called the extension blocks until the user-level module returns a verdict. Exampe: procfs, timer. Extensions that don’t block: the kernel callback returns as soon as a message is sent to the user space. Example: Netfilter.

18. Why Netfilter Is Non-blocking? Hardware interrupt handler receives network packets. Runs with interrupts disabled. It saves packets in a queue and invokes software interrupt, then returns. One software interrupt handler per processor. Hardware interrupts enabled. A packet is re-injected into the network stack when user-level processing is finished.

19. Extension Types DUSK supports Netfilter only. Timers, threads, procfs, ioctl, etc. are required for advanced modules.

20. DUSK Run-Time Configuration

21. Compile-time Support DUSK adds two functions: check() and commit() . check() ensures that every buffer accessed using a pointer exists in kernel memory and in user-space. commit() updates the kernel version when a function is called; it refreshes the user version when the function returns.

22. Function check() check() is called for every array access, de-referenced pointer, or &. It maintains the kernel-to-user address mapping buffer. For a new user address DUSK calls kmalloc() and kmem_write() For a new kernel address DUSK calls malloc() and kmem_read(). Addresses under PAGE_OFFSET are user addresses, above are kernel addresses.

23. Type Information Type information is gathered at compile-time. Type size, offsets of pointer fields are stored. Type size is used in check() . commit() uses field information.

24. Kernel Function Calls A kernel function is replaced with: void_kernel_func() for void functions; int_kernel_func() for int functions; Function arguments are written into kernel memory when the function is called. Each pointer-type argument is wrapped into commit() call.

25. Function commit() commit() traverses data types recursively. Issue: array elements accessed using a pointer: *(a+1), *(a+2). If a is a function parameter then commit() ensures that all elements get synchronized. Issue: a union’s elements of different type have same offset. Types of the argument and that of the address mapping table are compared.

26. Function commit()

27. Function Prolog and Epilog All changes are committed when user-level module returns a verdict. DUSK adds a function epilog that commits the changes. depth variable counts the current nesting level. Incremented in the function prolog, decremented in the epilog.

28. check() and commit() Example printk(“test”); int_kernel_func(addr_of(“printk”), commit(check(“test”)));

29. Cross-Platform Debugging Linksys WRT54gs wireless router: 32 MB RAM, 200 MHz MIPS CPU. Runs Linux 2.4.20. Cannot run gcc, gdb. Supports Netfilter. printk() debugging possible. Logs are not stored after reboot.

30. X86 and MIPS Differences Data structures are different: PAGE_OFFSETs are different. DUSK compiles module twice: with the cross-compiler and with x86 compiler. Addresses in data structures are changed when data from kernel is received.

31. Evaluation nfsiff – FTP password sniffer POSTROUTING: finds USER and PASS commands; PREROUTING: replies to special ICMP packets; lwfw – light-weight firewall PREROUTING: drops certain packets (port, interface). ipp2p – P2P traffic classifier POSTROUTING: scans for P2P signatures.

32. nfsiff Sniffer on wireless router:

33. Compile-time Overhead

34. Saved Work

35. Conclusion DUSK allows to debug kernel modules at user level without changing the source code. Future work: add better debugger support, i.e. integrate DUSK with gdb and ddd. Support device driver development.