Successfully reported this slideshow.

Virtunoid: Breaking out of KVM

3

Share

Upcoming SlideShare
Kvm
Kvm
Loading in …3
×
1 of 54
1 of 54

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Virtunoid: Breaking out of KVM

  1. 1. Virtunoid: Breaking out of KVM Nelson Elhage DEFCON 19 August 8, 2011 Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 1 / 50
  2. 2. KVM The new hotness for Virtualization on Linux Official virtualization platform for Ubuntu and RHEL. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 2 / 50
  3. 3. Who am I? Kernel engineer at Ksplice (now Oracle). Open-source security hacker in my spare time. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 3 / 50
  4. 4. Outline 1 KVM: Architecture overview Attack Surface 2 CVE-2011-1751: The bug 3 virtunoid.c: The exploit %rip control Getting to shellcode Bypassing ASLR 4 Conclusions and further research 5 Demo Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 4 / 50
  5. 5. KVM: Architecture overview 1 KVM: Architecture overview Attack Surface 2 CVE-2011-1751: The bug 3 virtunoid.c: The exploit %rip control Getting to shellcode Bypassing ASLR 4 Conclusions and further research 5 Demo Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 5 / 50
  6. 6. KVM: Architecture overview KVM: The components kvm.ko kvm-intel.ko / kvm-amd.ko qemu-kvm Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 6 / 50
  7. 7. KVM: Architecture overview kvm.ko The core KVM kernel module Implements the virtual CPU and MMU (with the hardware’s help). Emulates a few devices in-kernel for efficiency. Provides ioctls for communicating with the kernel module. Contains an emulator for a subset of x86 used in handling certain traps (!) Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 7 / 50
  8. 8. KVM: Architecture overview kvm-intel.ko / kvm-amd.ko Provides support for Intel’s VMX and AMD’s SVM virtualization extensions. Relatively small compared to the rest of KVM (one .c file each) Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 8 / 50
  9. 9. KVM: Architecture overview qemu-kvm Provides the most direct user interface to KVM. Based on the classic qemu emulator. Implements the bulk of the virtual devices a VM uses. Implements a wide variety of types of devices. An order of magnitude more code than the kernel module. There is work in progress to replace this component, but it’s a ways out, if ever. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 9 / 50
  10. 10. KVM: Architecture overview Attack Surface kvm.ko A tempting target – successful exploitation gets ring0 on the host without further escalation. Much less code than qemu-kvm, and much of that is dedicated to interfacing with qemu-kvm, not the guest directly. The x86 emulator is an interesting target. A number of bugs have been discovered allowing privesc within the guest. A lot of tricky code that is not often exercised. Not the target of this talk, but I have some ideas for future work. Also, be on the lookout for privesc within either the host or guest. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 10 / 50
  11. 11. KVM: Architecture overview Attack Surface kvm-intel.ko / kvm-amd.ko Not much direct attack surface. Largely straight-line code doing lots of low-level bit twiddling with the hardware structures. Lots of subtlety, possibly some more complex attacks. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 11 / 50
  12. 12. KVM: Architecture overview Attack Surface qemu-kvm A veritable goldmine of targets. Hundreds of thousands of lines of device emulation code. Emulated devices communicate directly with the guest via MMIO or IO ports, lots of attack surface. Much of the code comes straight from qemu and is ancient. qemu-kvm is often sandboxed using SELinux or similar, meaning that successful exploitation will often require a second privesc within the host. (Fortunately, Linux never has any of those) Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 12 / 50
  13. 13. CVE-2011-1751: The bug 1 KVM: Architecture overview Attack Surface 2 CVE-2011-1751: The bug 3 virtunoid.c: The exploit %rip control Getting to shellcode Bypassing ASLR 4 Conclusions and further research 5 Demo Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 13 / 50
  14. 14. CVE-2011-1751: The bug RHSA-2011:0534-1 “It was found that the PIIX4 Power Management emulation layer in qemu-kvm did not properly check for hot plug eligibility during device removals. A privileged guest user could use this flaw to crash the guest or, possibly, execute arbitrary code on the host. (CVE-2011-1751)” Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 14 / 50
  15. 15. CVE-2011-1751: The bug PIIX4 The PIIX4 was a southbridge chip used in many circa-2000 Intel chipsets. The default southbridge emulated by qemu-kvm Includes ACPI support, a PCI-ISA bridge, an embedded MC146818 RTC, and much more. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 15 / 50
  16. 16. CVE-2011-1751: The bug Device Hotplug The PIIX4 supports PCI hotplug, implemented by writing values to IO port 0xae08. qemu-kvm emulates this by calling qdev_free(qdev);, which calls a device’s cleanup function and free()s it. Many devices weren’t implemented with hotplug in mind! Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 16 / 50
  17. 17. CVE-2011-1751: The bug The PCI-ISA bridge In particular, it should not be possible to unplug the ISA bridge. Among other things, the emulated MC146818 RTC hangs off the ISA bridge. KVM’s emulated RTC is not designed to be unplugged; In particular, it leaves around dangling QEMUTimer objects when unplugged. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 17 / 50
  18. 18. CVE-2011-1751: The bug QEMUTimer t y p e d e f v o i d QEMUTimerCB( v o i d * opaque ) ; s t r u c t QEMUTimer { ... i n t 6 4 t e x p i r e t i m e ; /* i n n a n o s e c o n d s */ QEMUTimerCB * cb ; v o i d * opaque ; s t r u c t QEMUTimer * n e x t ; }; t y p e d e f s t r u c t RTCState { ... QEMUTimer * s e c o n d t i m e r ; ... } RTCState ; Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 18 / 50
  19. 19. CVE-2011-1751: The bug Use-after-free Unplugging the virtual RTC free()s the RTCState It doesn’t free() or unregister either of the timers. So we’re left with dangling pointers from the QEMUTimers On the next second, we’ll call rtc_update_second(<freed RTCState> ) Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 19 / 50
  20. 20. CVE-2011-1751: The bug Reproducer #i n c l u d e <s y s / i o . h> i n t main ( v o i d ) { iopl (3); o u t l (2 , 0 xae08 ) ; return 0; } Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 20 / 50
  21. 21. virtunoid.c: The exploit 1 KVM: Architecture overview Attack Surface 2 CVE-2011-1751: The bug 3 virtunoid.c: The exploit %rip control Getting to shellcode Bypassing ASLR 4 Conclusions and further research 5 Demo Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 21 / 50
  22. 22. virtunoid.c: The exploit %rip control High-level TODO 1 Inject a controlled QEMUTimer into qemu-kvm at a known address 2 Eject the emulated ISA bridge 3 Force an allocation into the freed RTCState, with second_timer pointing at our dummy timer. When rtc_update_second next runs, our timer will get scheduled. One second later, boom. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 22 / 50
  23. 23. virtunoid.c: The exploit %rip control 1. Injecting data The guest’s RAM is backed by a simple mmap()ed region inside the qemu-kvm process. So we allocate an object in the guest, and compute hva = physmem_base + gpa gpa = (gva_to_gfn(gva) << PAGE_SHIFT) + page_offset(gva) For now, assume we can guess physmem_base (e.g. no ASLR) hva host virtual address gva guest virtual address gpa guest physical address gfn guest frame (physical page) number Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 23 / 50
  24. 24. virtunoid.c: The exploit %rip control qemu-kvm userspace network stack qemu-kvm contains a user-mode networking stack. Implements a DHCP server, DNS server, and a gateway NAT. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 24 / 50
  25. 25. virtunoid.c: The exploit %rip control Userspace network stack packet delivery The user-mode stack normally handles packets synchronously. To prevent recursion, if a second packet is emitted while handling a first packet, the second packet is queued, using malloc(). Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 25 / 50
  26. 26. virtunoid.c: The exploit %rip control The virtual network gateway responds synchronously to ICMP ping. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 26 / 50
  27. 27. virtunoid.c: The exploit %rip control Putting it together 1 Allocate a fake QEMUTimer Point ->cb at the desired %rip. 2 Calculate its address in the host. 3 Write 2 to IO port 0xae08 to eject the ISA bridge. 4 ping the emulated gateway with ICMP packets containing pointers to your allocated timer in the host. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 27 / 50
  28. 28. virtunoid.c: The exploit Getting to shellcode We’ve got %rip, now what? Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 28 / 50
  29. 29. virtunoid.c: The exploit Getting to shellcode We’ve got %rip, now what? Get EIP = 0x41414141 and declare victory. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 28 / 50
  30. 30. virtunoid.c: The exploit Getting to shellcode We’ve got %rip, now what? Get EIP = 0x41414141 and declare victory. Disable NX in my BIOS and call it good enough for a demo. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 28 / 50
  31. 31. virtunoid.c: The exploit Getting to shellcode We’ve got %rip, now what? Get EIP = 0x41414141 and declare victory. Disable NX in my BIOS and call it good enough for a demo. Do a ROP pivot, ROP to victory. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 28 / 50
  32. 32. virtunoid.c: The exploit Getting to shellcode We’ve got %rip, now what? Get EIP = 0x41414141 and declare victory. Disable NX in my BIOS and call it good enough for a demo. Do a ROP pivot, ROP to victory. Do something else clever. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 28 / 50
  33. 33. virtunoid.c: The exploit Getting to shellcode Another look at QEMUTimer s t r u c t QEMUTimer { ... i n t 6 4 t e x p i r e t i m e ; /* i n n a n o s e c o n d s */ ... s t r u c t QEMUTimer * n e x t ; }; Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 29 / 50
  34. 34. virtunoid.c: The exploit Getting to shellcode qemu_run_timers s t a t i c v o i d q e m u r u n t i m e r s ( QEMUClock * c l o c k ) { QEMUTimer ** p t i m e r h e a d , * t s ; int64 t current time ; current time = qemu get clock ns ( clock ); p t i m e r h e a d = &a c t i v e t i m e r s [ c l o c k −>t y p e ] ; for ( ; ; ) { ts = * ptimer head ; i f ( ! qemu timer expired ns ( ts , c u r r e n t t i m e )) break ; * p t i m e r h e a d = t s −>n e x t ; t s −>n e x t = NULL ; t s −>cb ( t s −>opaque ) ; } } Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 30 / 50
  35. 35. virtunoid.c: The exploit Getting to shellcode Timer chains second_timer f1 −>cb −>opaque X −>next f2 −>cb −>opaque Y −>next f3 −>cb −>opaque Z −>next NULL ⇒ f1(X); f2(Y); f3(Z); Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 31 / 50
  36. 36. virtunoid.c: The exploit Getting to shellcode More arguments amd64 calling convention: %rdi, %rsi, %rdx, . . . Every version of qemu_run_timers I’ve checked leaves %rsi untouched. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 32 / 50
  37. 37. virtunoid.c: The exploit Getting to shellcode More arguments set rsi : movl %r d i , %r s i ret Let f1 = set_rsi f2(Y, X) Same trick doesn’t work with %rdx. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 33 / 50
  38. 38. virtunoid.c: The exploit Getting to shellcode set_rsi v o i d c p u o u t l ( p i o a d d r t addr , u i n t 3 2 t v a l ) { i o p o r t w r i t e ( 2 , addr , v a l ) ; } Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 34 / 50
  39. 39. virtunoid.c: The exploit Getting to shellcode Getting to mprotect int mprotect(const void *addr, size_t len, int prot); #define PROT_EXEC 0x4 s t a t i c u i n t 3 2 t i o p o r t r e a d l t h u n k ( v o i d * opaque , u i n t 3 2 t a d d r ) { IORange * i o p o r t = opaque ; u i n t 6 4 t data ; i o p o r t −>ops−>r e a d ( i o p o r t , a d d r − i o p o r t −>b a s e , 4 , &d a t a ) ; r e t u r n data ; } Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 35 / 50
  40. 40. virtunoid.c: The exploit Getting to shellcode Putting it together Allocate a fake IORangeOps, with fake_ops->read = mprotect. Allocate a page-aligned IORange, with ->ops = fake_ops and ->base = -PAGE_SIZE. Copy shellcode immediately following the IORange. Construct a timer chain that calls cpu_outl(0, *) ioport_readl_thunk(fake_ioport, 0) fake_ioport + 1 Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 36 / 50
  41. 41. virtunoid.c: The exploit Getting to shellcode Why not ROP? Continued execution is dead simple. Reduced dependence on details of compiled code. I’m not that good at ROP :) Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 37 / 50
  42. 42. virtunoid.c: The exploit Bypassing ASLR Addresses For a known qemu-kvm binary, we need two addresses. The base address of the qemu-kvm binary, to find code addresses. physmem_base, the address of the physical memory mapping inside qemu-kvm. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 38 / 50
  43. 43. virtunoid.c: The exploit Bypassing ASLR Option A Find an information leak. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 39 / 50
  44. 44. virtunoid.c: The exploit Bypassing ASLR Option B Assume non-PIE, and be clever. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 40 / 50
  45. 45. virtunoid.c: The exploit Bypassing ASLR fw_cfg Emulated IO ports 0x510 (address) and 0x511 (data) Used to communicate various tables to the qemu BIOS (e820 map, ACPI tables, etc) Also provides support for exporting writable tables to the BIOS. However, fw_cfg_write doesn’t check if the target table is supposed to be writable! Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 41 / 50
  46. 46. virtunoid.c: The exploit Bypassing ASLR Static data Several fw_cfg areas are backed by statically-allocated buffers. Net result: nearly 500 writable bytes inside static variables. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 42 / 50
  47. 47. virtunoid.c: The exploit Bypassing ASLR read4 your way to victory mprotect needs a page-aligned address, so these aren’t suitable for our shellcode. But, we can construct fake timer chains in this space to build a read4() primitive. Follow pointers from static variables to find physmem_base Proceed as before. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 43 / 50
  48. 48. virtunoid.c: The exploit Bypassing ASLR Repeated timer chaining Previously, we ended timer chains with ->next = NULL. Instead, end them with a timer that calls rtc_update_second. The timer we control will be scheduled once a second, and we can change ->cb at any time. Now we can execute a read4, update structures based on the result, and then hijack the list again. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 44 / 50
  49. 49. Conclusions and further research Conclusions VM breakouts aren’t magic. Hypervisors are just as vulnerable as anything else. Device drivers are the weak spot. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 45 / 50
  50. 50. Conclusions and further research Comparing with some past breakouts 2008 “Adventures with a certain Xen vulnerability”, Xen, Invisible Things Lab 2009 “Cloudburst”, Immunity, VMware 2011 “Software attacks against Intel VT-d technology”, Invisible Things Lab, Xen Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 46 / 50
  51. 51. Conclusions and further research Possible hardening directions Sandbox qemu-kvm (work underway well before this talk). Build qemu-kvm as PIE. Lazily mmap/mprotect guest RAM? XOR-encode key function pointers? More auditing and fuzzing of qemu-kvm. Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 47 / 50
  52. 52. Conclusions and further research Future research directions Fuzzing/auditing kvm.ko (That x86 emulator sketches me) Fingerprinting qemu-kvm versions Searching for infoleaks (Rosenbugs?) Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 48 / 50
  53. 53. Demo It’s demo time Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 49 / 50
  54. 54. Demo Questions? nelhage@nelhage.com http://blog.nelhage.com @nelhage Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 50 / 50

×