DUSK: Develop at User level, inStall in Kernel Alexey Smirnov and Tzi-cker Chiueh ECSL Research Seminar 09/13/05
Outline <ul><li>Introduction </li></ul><ul><li>Related Work </li></ul><ul><li>Netfilter Overview </li></ul><ul><li>System ...
Introduction <ul><li>Kernel is often extended using kernel modules. </li></ul><ul><li>Netfilter – hooks in network compone...
User-level Development of Kernel Extensions <ul><li>Provide kernel API at user-level. A lot of functions, changes from one...
DUSK Overview <ul><li>DUSK is an extension to GCC that compiles the source code of a kernel module into a user-level progr...
DUSK Framework <ul><li>User-level component:  the kernel module compiled as a user level program. </li></ul><ul><li>Kernel...
Related Work <ul><li>VFS layer:  UFS uses NFS loopback API. UpcallFS – a stackable file system. </li></ul><ul><li>Device d...
Netfilter Architecture <ul><li>Netfilter adds a set of five hooks to kernel network stack:  </li></ul><ul><li>A Netfilter ...
Netfilter User Mode Modules <ul><li>NF_QUEUE  verdict sends sk_buff to user space using a netlink socket. The queue handle...
iptables <ul><li>iptables controls parameters of Linux firewall. </li></ul><ul><li>iptables -A PREROUTING –p tcp  --destin...
Kernel-level Programming at User Level <ul><li>User-level kernel modules need to: </li></ul><ul><ul><li>Read/write kernel ...
Three Basic Blocks <ul><li>Kernel memory access:  /dev/kmem </li></ul><ul><li>Kernel function calls:  new system call sys_...
Using Basic Blocks to Run Kernel Module at User Level <ul><li>Kernel module source code: </li></ul><ul><li>sk_buff->len </...
DUSK Framework
DUSK Run-Time Support <ul><li>Kernel extension registration/de-registration. The address of a function is sent to the kern...
Extension Management <ul><li>nfho_post_routing.hook=khook_ip_input; </li></ul><ul><li>nfho_post_routing.hooknum = NF_IP_PO...
Blocking and Non-blocking Extensions <ul><li>Extensions that block:  the process that called the extension blocks until th...
Why Netfilter Is Non-blocking? <ul><li>Hardware interrupt handler receives network packets. </li></ul><ul><li>Runs with in...
Extension Types <ul><li>DUSK supports Netfilter only. </li></ul><ul><li>Timers, threads, procfs, ioctl, etc. are required ...
DUSK Run-Time Configuration
Compile-time Support <ul><li>DUSK adds two functions:  check()  and  commit() . </li></ul><ul><li>check()  ensures that ev...
Function check() <ul><li>check()  is called for every array access, de-referenced pointer, or &. </li></ul><ul><li>It main...
Type Information <ul><li>Type information is gathered at compile-time. Type size, offsets of pointer fields are stored. </...
Kernel Function Calls <ul><li>A kernel function is replaced with: </li></ul><ul><ul><li>void_kernel_func() for void functi...
Function commit() <ul><li>commit()  traverses data types recursively. </li></ul><ul><li>Issue: array elements accessed usi...
Function commit()
Function Prolog and Epilog <ul><li>All changes are committed when user-level module returns a verdict. </li></ul><ul><li>D...
check() and commit() Example <ul><li>printk(“test”); </li></ul><ul><li>int_kernel_func(addr_of(“printk”), commit(check(“te...
Cross-Platform Debugging <ul><li>Linksys WRT54gs wireless router: 32 MB RAM, 200 MHz MIPS CPU. Runs Linux 2.4.20. </li></u...
X86 and MIPS Differences <ul><li>Data structures are different: </li></ul><ul><li>PAGE_OFFSETs are different. </li></ul><u...
Evaluation <ul><li>nfsiff – FTP password sniffer </li></ul><ul><ul><li>POSTROUTING: finds USER and PASS commands; </li></u...
nfsiff <ul><li>Sniffer on wireless router: </li></ul>
Compile-time Overhead
Saved Work
Conclusion <ul><li>DUSK allows to debug kernel modules at user level without changing the source code. </li></ul><ul><li>F...
Upcoming SlideShare
Loading in …5
×

DUSK - Develop at Userland Install into Kernel

4,123 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
4,123
On SlideShare
0
From Embeds
0
Number of Embeds
112
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

DUSK - Develop at Userland Install into Kernel

  1. 1. DUSK: Develop at User level, inStall in Kernel Alexey Smirnov and Tzi-cker Chiueh ECSL Research Seminar 09/13/05
  2. 2. Outline <ul><li>Introduction </li></ul><ul><li>Related Work </li></ul><ul><li>Netfilter Overview </li></ul><ul><li>System Architecture </li></ul><ul><li>Evaluation </li></ul><ul><li>Conclusion </li></ul>
  3. 3. Introduction <ul><li>Kernel is often extended using kernel modules. </li></ul><ul><li>Netfilter – hooks in network component. </li></ul><ul><li>Development at user-level is convenient: gdb, ddd, etc. </li></ul><ul><li>Kernel modules have lower overhead. </li></ul><ul><li>DUSK achieves best of both worlds. </li></ul>
  4. 4. User-level Development of Kernel Extensions <ul><li>Provide kernel API at user-level. A lot of functions, changes from one version to another. </li></ul><ul><li>Implement a new API that resembles kernel API. Not convenient for the programmer, requires code rewriting. </li></ul>
  5. 5. DUSK Overview <ul><li>DUSK is an extension to GCC that compiles the source code of a kernel module into a user-level program. </li></ul><ul><li>It links several helper functions to the user-level module. </li></ul><ul><li>Same helper functions for different versions of Linux kernel. </li></ul>
  6. 6. DUSK Framework <ul><li>User-level component: the kernel module compiled as a user level program. </li></ul><ul><li>Kernel-level component: handles user-level module’s requests. </li></ul><ul><li>The two components are connected using UDP sockets. </li></ul><ul><li>Can run on different architectures: user-level module on x86 running in gdb and kernel-level module on MIPS wireless router. </li></ul>
  7. 7. Related Work <ul><li>VFS layer: UFS uses NFS loopback API. UpcallFS – a stackable file system. </li></ul><ul><li>Device drivers: Userdev provides a user-level library. </li></ul><ul><li>Network stack: icTCP exposes TCP stack parameters to user programs. Netfilter simulator runs Netfilter hooks at user level. </li></ul>
  8. 8. Netfilter Architecture <ul><li>Netfilter adds a set of five hooks to kernel network stack: </li></ul><ul><li>A Netfilter module verdict: NF_ACCEPT , NF_DROP , NF_STOLEN , NF_QUEUE , NF_REPEAT . </li></ul>
  9. 9. Netfilter User Mode Modules <ul><li>NF_QUEUE verdict sends sk_buff to user space using a netlink socket. The queue handler is either standard or programmer-defined. </li></ul><ul><li>A user module can only read sk_buff but cannot change kernel memory. </li></ul>
  10. 10. iptables <ul><li>iptables controls parameters of Linux firewall. </li></ul><ul><li>iptables -A PREROUTING –p tcp --destination-port 80 –j QUEUE </li></ul><ul><li>iptables –A POSTROUTING –p udp --source-port 5678 –j DROP </li></ul><ul><li>Has string matching capabilities. Stateless IDS. </li></ul>
  11. 11. Kernel-level Programming at User Level <ul><li>User-level kernel modules need to: </li></ul><ul><ul><li>Read/write kernel memory; </li></ul></ul><ul><ul><li>Call kernel functions; </li></ul></ul><ul><ul><li>Access global kernel variables, e. g. current </li></ul></ul><ul><li>Programmers can modify source code of kernel module using these blocks to run it at user level. </li></ul>
  12. 12. Three Basic Blocks <ul><li>Kernel memory access: /dev/kmem </li></ul><ul><li>Kernel function calls: new system call sys_exec(). System.map for address lookup. </li></ul><ul><li>A new system call sys_kvar() for variable access. </li></ul><ul><li>sys_exec() and sys_kvar() are implemented in DUSK kernel module. </li></ul><ul><li>Kernel module and user module communicate using UDP sockets. </li></ul>
  13. 13. Using Basic Blocks to Run Kernel Module at User Level <ul><li>Kernel module source code: </li></ul><ul><li>sk_buff->len </li></ul><ul><li>User-level code: </li></ul><ul><ul><li>Define a shadow variable u_sk_buff; </li></ul></ul><ul><ul><li>malloc() memory for it; </li></ul></ul><ul><ul><li>Use kmem_read() to copy sk_buff from kernel memory to shadow variable; </li></ul></ul><ul><ul><li>Commit changes to u_sk_buff using kmem_write(). </li></ul></ul><ul><li>Our goal: automatic transformation. </li></ul>
  14. 14. DUSK Framework
  15. 15. DUSK Run-Time Support <ul><li>Kernel extension registration/de-registration. The address of a function is sent to the kernel. </li></ul><ul><li>Processing kernel requests. An appropriate extension handles each request. </li></ul><ul><li>Module initialization/cleanup. init_module() and cleanup_module() are called when user-level module starts and terminates. </li></ul>
  16. 16. Extension Management <ul><li>nfho_post_routing.hook=khook_ip_input; </li></ul><ul><li>nfho_post_routing.hooknum = NF_IP_POST_ROUTING; </li></ul><ul><li>nf_register_hook(&nfho_post_routing); </li></ul><ul><li>Kernel-level DUSK module replaces khook_ip_input with a proxy kernel function and returns extension type and sequence number to the user program. </li></ul><ul><li>DUSK clones a new process for each extension. They share same address space. </li></ul>
  17. 17. Blocking and Non-blocking Extensions <ul><li>Extensions that block: the process that called the extension blocks until the user-level module returns a verdict. Exampe: procfs, timer. </li></ul><ul><li>Extensions that don’t block: the kernel callback returns as soon as a message is sent to the user space. Example: Netfilter. </li></ul>
  18. 18. Why Netfilter Is Non-blocking? <ul><li>Hardware interrupt handler receives network packets. </li></ul><ul><li>Runs with interrupts disabled. It saves packets in a queue and invokes software interrupt, then returns. </li></ul><ul><li>One software interrupt handler per processor. Hardware interrupts enabled. </li></ul><ul><li>A packet is re-injected into the network stack when user-level processing is finished. </li></ul>
  19. 19. Extension Types <ul><li>DUSK supports Netfilter only. </li></ul><ul><li>Timers, threads, procfs, ioctl, etc. are required for advanced modules. </li></ul>
  20. 20. DUSK Run-Time Configuration
  21. 21. Compile-time Support <ul><li>DUSK adds two functions: check() and commit() . </li></ul><ul><li>check() ensures that every buffer accessed using a pointer exists in kernel memory and in user-space. </li></ul><ul><li>commit() updates the kernel version when a function is called; it refreshes the user version when the function returns. </li></ul>
  22. 22. Function check() <ul><li>check() is called for every array access, de-referenced pointer, or &. </li></ul><ul><li>It maintains the kernel-to-user address mapping buffer. </li></ul><ul><li>For a new user address DUSK calls kmalloc() and kmem_write() </li></ul><ul><li>For a new kernel address DUSK calls malloc() and kmem_read(). </li></ul><ul><li>Addresses under PAGE_OFFSET are user addresses, above are kernel addresses. </li></ul>
  23. 23. Type Information <ul><li>Type information is gathered at compile-time. Type size, offsets of pointer fields are stored. </li></ul><ul><li>Type size is used in check() . </li></ul><ul><li>commit() uses field information. </li></ul>
  24. 24. Kernel Function Calls <ul><li>A kernel function is replaced with: </li></ul><ul><ul><li>void_kernel_func() for void functions; </li></ul></ul><ul><ul><li>int_kernel_func() for int functions; </li></ul></ul><ul><li>Function arguments are written into kernel memory when the function is called. </li></ul><ul><li>Each pointer-type argument is wrapped into commit() call. </li></ul>
  25. 25. Function commit() <ul><li>commit() traverses data types recursively. </li></ul><ul><li>Issue: array elements accessed using a pointer: *(a+1), *(a+2). If a is a function parameter then commit() ensures that all elements get synchronized. </li></ul><ul><li>Issue: a union’s elements of different type have same offset. Types of the argument and that of the address mapping table are compared. </li></ul>
  26. 26. Function commit()
  27. 27. Function Prolog and Epilog <ul><li>All changes are committed when user-level module returns a verdict. </li></ul><ul><li>DUSK adds a function epilog that commits the changes. </li></ul><ul><li>depth variable counts the current nesting level. </li></ul><ul><li>Incremented in the function prolog, decremented in the epilog. </li></ul>
  28. 28. check() and commit() Example <ul><li>printk(“test”); </li></ul><ul><li>int_kernel_func(addr_of(“printk”), commit(check(“test”))); </li></ul>
  29. 29. Cross-Platform Debugging <ul><li>Linksys WRT54gs wireless router: 32 MB RAM, 200 MHz MIPS CPU. Runs Linux 2.4.20. </li></ul><ul><li>Cannot run gcc, gdb. Supports Netfilter. </li></ul><ul><li>printk() debugging possible. Logs are not stored after reboot. </li></ul>
  30. 30. X86 and MIPS Differences <ul><li>Data structures are different: </li></ul><ul><li>PAGE_OFFSETs are different. </li></ul><ul><li>DUSK compiles module twice: with the cross-compiler and with x86 compiler. </li></ul><ul><li>Addresses in data structures are changed when data from kernel is received. </li></ul>
  31. 31. Evaluation <ul><li>nfsiff – FTP password sniffer </li></ul><ul><ul><li>POSTROUTING: finds USER and PASS commands; </li></ul></ul><ul><ul><li>PREROUTING: replies to special ICMP packets; </li></ul></ul><ul><li>lwfw – light-weight firewall </li></ul><ul><ul><li>PREROUTING: drops certain packets (port, interface). </li></ul></ul><ul><li>ipp2p – P2P traffic classifier </li></ul><ul><ul><li>POSTROUTING: scans for P2P signatures. </li></ul></ul>
  32. 32. nfsiff <ul><li>Sniffer on wireless router: </li></ul>
  33. 33. Compile-time Overhead
  34. 34. Saved Work
  35. 35. Conclusion <ul><li>DUSK allows to debug kernel modules at user level without changing the source code. </li></ul><ul><li>Future work: add better debugger support, i.e. integrate DUSK with gdb and ddd. </li></ul><ul><li>Support device driver development. </li></ul>

×