Robert Clark
Lead Security Architect
HP Cloud
Hypervisor Security
About the Speaker
OpenStack Security Group
• Established 18-24 months ago
• Issues OpenStack Security Notes
• Consults on OpenStack Security...
OpenStack Security Guide
http://docs.openstack.org/security
OpenStack Security Guide
Virtualization
Overview
Virtualization Technologies
• Hosted OS Virtualization – VMware Desktop
Solutions
• Para Virtualization – The guest needs ...
Virtualization Stack
Compute Host
Alice
VM
Alice
VM
Alice
VM
Hardware
Hypervisor
Device Emulation
Simplified KVM
Compute Host
Alice
VM
Alice
VM
Alice
VM
Hardware CPU VIRT
Linux Kernel
KVM
QEMU
Linux OS
Simplified Xen
Compute Host
Dom0 Alice
VM
Hardware
Xen Hypervisor
Alice
VM
QEMU
Generalized Virtualization Stack
Compute Host
Alice
VM
Alice
VM
Alice
VM
Hardware
Hypervisor / Host OS / Dom0
QEMU
Compute...
Attack
Vectors
Introducing ‘Mal’
Mal
VM
Compute Host Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
KVM / XEN
QEMU
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
KVM / XEN
QEMU
Dom0
Linux Kernel
Linux...
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM KVM / XEN
QEMU
Dom0
Linux Kerne...
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Basic VM to VM network Att...
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM KVM / XEN
QEMU
VM to hypervisor...
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM KVM / XEN
QEMU
VM to QEMU / Dev...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Dom0
Compute Instance Attack Vectors
Compute Host [Nova]
Alice
VM
Alice
VM
Bob
VM
Bob
VM
Mal
VM
QEMU
Linux Kernel
Linux OS...
Cloud Issues
Compute Host [Nova]
Alice
VM
Bob
VM
Cloud Issues - Scale
Compute Host [Nova]
Cher
VM
Dave
VM
Compute Host [Nova]
Alice
VM
Bob
VM
Cloud Issues - Scale
Compute Host [Nova]
Cher
VM
Dave
VM
Compute Host [Nova]
Alice
VM
Bob
VM
Compute Manager
Block Storage...
Cloud Issues - Scale
Compute Host [Nova]
Cher
VM
Dave
VM
Compute Host [Nova]
Alice
VM
Bob
VM
Compute Manager
Block Storage...
Cloud Issues – Flat Exploitation
Compute Host [Nova]
Cher
VM
Dave
VM
Compute Host [Nova]
Alice
VM
Bob
VM
Compute Manager
B...
Cloud Issues – Flat Exploitation
Compute Host [Nova]
Cher
VM
Dave
VM
Compute Host [Nova]
Alice
VM
Bob
VM
Compute Manager
B...
Cloud Issues – Service Trust
Compute Host [Nova]
Cher
VM
Dave
VM
Compute Host [Nova]
Alice
VM
Bob
VM
Compute Manager
Block...
Cloud Issues – Service Trust
Compute Host [Nova]
Cher
VM
Dave
VM
Compute Host [Nova]
Alice
VM
Bob
VM
Compute Manager
Block...
Cloud Issues – Nova RPC
Compute Host [Nova]
Cher
VM
Dave
VM
Compute Host [Nova]
Alice
VM
Bob
VM
Compute Manager
Block Stor...
What about
side channels?
Cross-VM Side Channel Attacks
• Web Servers providing SSL
• VOIP providers
• Cloud VPN
• Chat Applications
• Secure File S...
Cross-VM Side Channel Attacks
Alice
Client
Compute Host [Nova]
Bob
VM
TLS/SSL
CPU
L1 Cache
• Disrupting or observing syste...
Cross-VM Side Channel Attacks
Alice
Client
Compute Host [Nova]
Bob
VM
TLS/SSL
Stealing the bits!
Mal
MITM
CPU
L1 Cache
Cross-VM Side Channel Attacks
Alice
Client
Compute Host [Nova]
Bob
VM
Mal
VM
TLS/SSL
Mal
MITM
CPU
L1 Cache
Stealing the bi...
Cross-VM Side Channel Attacks
Alice
Client
Compute Host [Nova]
Bob
VM
Mal
VM
TLS/SSL
Mal
MITM
CPU
L1 Cache
Stealing the bi...
Cross-VM Side Channel Attacks
Alice
Client
Compute Host [Nova]
Bob
VM
Mal
VM
TLS/SSL
Mal
MITM
CPU
L1 Cache
Stealing the bi...
Isn’t this all a bit
theoretical?
CloudBurst
• Date: 2008
• Type: OS Virtualization - VMWare
• Result: Full Breakout
• Author: Kostya Kirtchinsky, Immunity ...
Xen Ownage Trilogy
• Date: 2011
• Type: Xen
• Result: Full Breakout
• Author: Joanna Rutkowska
VirtuNoid
• Date: 2011
• Type: Kernel Side Full Virtualization - KVM
• Result: Full Breakout
• Author: Nelson Elhage
• CVE...
SYSRET-64
• Date: 2012
• Type: Para Virtualization - Xen
• Result: Full Breakout
• Author: Rafal Wojtczuk
• US-CERT #649219
VMDK Has Left The Building
• Date: 2012
• Type: ESXi File Handling Logic
• Result: Data Leakage / Loss
• Author: Friedwart...
KVM IOAPIC, SET MSR, TIME
• Date: 2013
• Type: Full Virtualization - KVM
• Result: Denial of Service, Potential Breakout
•...
Virtualization Security Trends
IBM X-Force 2010 Mid-Term Report
Virtualization Security Trends
Attack Vector Xen KVM
Virtual CPUs 5 (8.5%) 8 (21.1%)
SMP 1 (1.7%) 3 (7.9%)
Software MMU 4 ...
Time to unplug?
Go home cloud, you’re drunk!
Protections – Compiler Hardening
• RELocation Read-Only
• Stack Canaries
• Never eXecute (NX) / (DEP)
• Position Independe...
Protections – Reduce Attack Surface
• Out of the box you probably support
– 3D Graphics
– Multiple Network Devices
– Sound...
Protections – Mandatory Access
Controls
• Limit the capabilities of a successful exploit
• Define and constrain with QEMU ...
Protections – Mandatory Access
Controls
Protection
• Reduce Attack Surface
• Harden Compilation
• Isolate, detect and alert on exploitation
through MAC
• Harden y...
OpenStack Security Guide
• http://docs.openstack.org/sec
• Chapter 26 – Securing OpenStack Networking
Services
• Chapter 4...
Thank You
Please consider contributing to the
OpenStack Security Group
References
• Directly Referenced / Informed This Talk
– http://www.insinuator.net/2013/05/analysis-of-hypervisor-breakouts...
Hypervisor Security - OpenStack Summit Hong Kong
Upcoming SlideShare
Loading in...5
×

Hypervisor Security - OpenStack Summit Hong Kong

552

Published on

Hypervisor Security and steps that must be taken to protect against breakouts

Video here: https://www.youtube.com/watch?v=y8L6B6Q5EdI

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
552
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
39
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Hypervisor Security - OpenStack Summit Hong Kong

  1. 1. Robert Clark Lead Security Architect HP Cloud Hypervisor Security
  2. 2. About the Speaker
  3. 3. OpenStack Security Group • Established 18-24 months ago • Issues OpenStack Security Notes • Consults on OpenStack Security Advisories • Security Initiatives • Nearly 100 members
  4. 4. OpenStack Security Guide http://docs.openstack.org/security
  5. 5. OpenStack Security Guide
  6. 6. Virtualization Overview
  7. 7. Virtualization Technologies • Hosted OS Virtualization – VMware Desktop Solutions • Para Virtualization – The guest needs to know it’s running in a virtualized environment • Full Virtualization – The guest is un-aware that it is running on a virtualized platform.
  8. 8. Virtualization Stack Compute Host Alice VM Alice VM Alice VM Hardware Hypervisor Device Emulation
  9. 9. Simplified KVM Compute Host Alice VM Alice VM Alice VM Hardware CPU VIRT Linux Kernel KVM QEMU Linux OS
  10. 10. Simplified Xen Compute Host Dom0 Alice VM Hardware Xen Hypervisor Alice VM QEMU
  11. 11. Generalized Virtualization Stack Compute Host Alice VM Alice VM Alice VM Hardware Hypervisor / Host OS / Dom0 QEMU Compute Instances Device Emulation / Paravirt Hardware Interfacing / Enabling Hardware Memory, Disk, CPU etc
  12. 12. Attack Vectors
  13. 13. Introducing ‘Mal’ Mal VM
  14. 14. Compute Host Attack Vectors Compute Host [Nova] Alice VM Alice VM
  15. 15. Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM
  16. 16. Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM KVM / XEN QEMU
  17. 17. Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM KVM / XEN QEMU Dom0 Linux Kernel Linux OS
  18. 18. Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM KVM / XEN QEMU Dom0 Linux Kernel Linux OS
  19. 19. Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Basic VM to VM network Attacks KVM / XEN QEMU Dom0 Linux Kernel Linux OS
  20. 20. Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM KVM / XEN QEMU VM to hypervisor attacks KVM / XEN QEMU Dom0 Linux Kernel Linux OS
  21. 21. Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM KVM / XEN QEMU VM to QEMU / Device attacks KVM / XEN QEMU Dom0 Linux Kernel Linux OS
  22. 22. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS VM to QEMU KVM / XEN QEMU
  23. 23. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS VM to QEMU KVM / XEN QEMU
  24. 24. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS 1. VM to QEMU KVM / XEN QEMU 2.
  25. 25. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS 1. VM to QEMU KVM / XEN QEMU 2.
  26. 26. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS VM to hypervisor attacks KVM / XEN QEMU
  27. 27. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS VM to hypervisor attacks KVM / XEN QEMU
  28. 28. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS VM to hypervisor attacks KVM / XEN QEMU
  29. 29. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS VM to OS / Management / Linux Kernel / Dom0 KVM / XEN QEMU
  30. 30. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS VM to OS / Management / Linux Kernel / Dom0 KVM / XEN QEMU
  31. 31. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS 1. 2. VM to OS / Management / Linux Kernel / Dom0 KVM / XEN QEMU
  32. 32. Dom0 Compute Instance Attack Vectors Compute Host [Nova] Alice VM Alice VM Bob VM Bob VM Mal VM QEMU Linux Kernel Linux OS 1. 2. 3. VM to OS / Management / Linux Kernel / Dom0 KVM / XEN QEMU
  33. 33. Cloud Issues Compute Host [Nova] Alice VM Bob VM
  34. 34. Cloud Issues - Scale Compute Host [Nova] Cher VM Dave VM Compute Host [Nova] Alice VM Bob VM
  35. 35. Cloud Issues - Scale Compute Host [Nova] Cher VM Dave VM Compute Host [Nova] Alice VM Bob VM Compute Manager Block Storage Network Nodes Operations Systems Object Storage
  36. 36. Cloud Issues - Scale Compute Host [Nova] Cher VM Dave VM Compute Host [Nova] Alice VM Bob VM Compute Manager Block Storage Network Nodes Operations Systems Object Storage Mal VM
  37. 37. Cloud Issues – Flat Exploitation Compute Host [Nova] Cher VM Dave VM Compute Host [Nova] Alice VM Bob VM Compute Manager Block Storage Network Nodes Operations Systems Object Storage Mal VM
  38. 38. Cloud Issues – Flat Exploitation Compute Host [Nova] Cher VM Dave VM Compute Host [Nova] Alice VM Bob VM Compute Manager Block Storage Network Nodes Operations Systems Object Storage Mal VM
  39. 39. Cloud Issues – Service Trust Compute Host [Nova] Cher VM Dave VM Compute Host [Nova] Alice VM Bob VM Compute Manager Block Storage Network Nodes Operations Systems Object Storage Mal VM
  40. 40. Cloud Issues – Service Trust Compute Host [Nova] Cher VM Dave VM Compute Host [Nova] Alice VM Bob VM Compute Manager Block Storage Network Nodes Operations Systems Object Storage Mal VM
  41. 41. Cloud Issues – Nova RPC Compute Host [Nova] Cher VM Dave VM Compute Host [Nova] Alice VM Bob VM Compute Manager Block Storage Network Nodes Operations Systems Object Storage Mal VM
  42. 42. What about side channels?
  43. 43. Cross-VM Side Channel Attacks • Web Servers providing SSL • VOIP providers • Cloud VPN • Chat Applications • Secure File Storage • Virtually any service doing anything useful
  44. 44. Cross-VM Side Channel Attacks Alice Client Compute Host [Nova] Bob VM TLS/SSL CPU L1 Cache • Disrupting or observing system operation
  45. 45. Cross-VM Side Channel Attacks Alice Client Compute Host [Nova] Bob VM TLS/SSL Stealing the bits! Mal MITM CPU L1 Cache
  46. 46. Cross-VM Side Channel Attacks Alice Client Compute Host [Nova] Bob VM Mal VM TLS/SSL Mal MITM CPU L1 Cache Stealing the bits!
  47. 47. Cross-VM Side Channel Attacks Alice Client Compute Host [Nova] Bob VM Mal VM TLS/SSL Mal MITM CPU L1 Cache Stealing the bits!
  48. 48. Cross-VM Side Channel Attacks Alice Client Compute Host [Nova] Bob VM Mal VM TLS/SSL Mal MITM CPU L1 Cache Stealing the bits!
  49. 49. Isn’t this all a bit theoretical?
  50. 50. CloudBurst • Date: 2008 • Type: OS Virtualization - VMWare • Result: Full Breakout • Author: Kostya Kirtchinsky, Immunity Inc
  51. 51. Xen Ownage Trilogy • Date: 2011 • Type: Xen • Result: Full Breakout • Author: Joanna Rutkowska
  52. 52. VirtuNoid • Date: 2011 • Type: Kernel Side Full Virtualization - KVM • Result: Full Breakout • Author: Nelson Elhage • CVE-2011-1751
  53. 53. SYSRET-64 • Date: 2012 • Type: Para Virtualization - Xen • Result: Full Breakout • Author: Rafal Wojtczuk • US-CERT #649219
  54. 54. VMDK Has Left The Building • Date: 2012 • Type: ESXi File Handling Logic • Result: Data Leakage / Loss • Author: Friedwart Kuhn
  55. 55. KVM IOAPIC, SET MSR, TIME • Date: 2013 • Type: Full Virtualization - KVM • Result: Denial of Service, Potential Breakout • Author: Andrew Honig • IOAPIC: CVE-2013-1798 • TIME: CVE-2013-1797 • SET MSR: CVE-2013-1796
  56. 56. Virtualization Security Trends IBM X-Force 2010 Mid-Term Report
  57. 57. Virtualization Security Trends Attack Vector Xen KVM Virtual CPUs 5 (8.5%) 8 (21.1%) SMP 1 (1.7%) 3 (7.9%) Software MMU 4 (6.8%) 2 (5.3%) Interrupt and Timer Mechanisms 2 (3.4%) 4 (10.5%) I/O and Networking 11 (18.6%) 10 (26.3%) VM Exits 4 (6.8%) 2 (5.3%) Hypercalls 2 (3.4%) 1 (2.6%) VM Management 7 (11.9%) 2 (5.3%) Remote Management Software 9 (15.3%) 1 (2.6%) Hypervisor add-ons 5 (8.5%) 0 (0.0%) TOTAL 59 38
  58. 58. Time to unplug? Go home cloud, you’re drunk!
  59. 59. Protections – Compiler Hardening • RELocation Read-Only • Stack Canaries • Never eXecute (NX) / (DEP) • Position Independent Executable • Address Space Layout Randomization • QEMU: CFLAGS="-arch x86_64 -fstack-protector-all -Wstack-protector --param ssp-buffer-size=4 -pie -fPIE -ftrapv -D_FORTIFY_SOURCE=2 O2 -Wl,-z,relro,- z,now"
  60. 60. Protections – Reduce Attack Surface • Out of the box you probably support – 3D Graphics – Multiple Network Devices – Sound – Bluetooth!? • Compile them out!
  61. 61. Protections – Mandatory Access Controls • Limit the capabilities of a successful exploit • Define and constrain with QEMU should be doing • Provide isolation for VM processes (KVM) • SELinux • AppArmour
  62. 62. Protections – Mandatory Access Controls
  63. 63. Protection • Reduce Attack Surface • Harden Compilation • Isolate, detect and alert on exploitation through MAC • Harden your base OS/Dom0 using the same techniques • Apply MAC to other OpenStack components
  64. 64. OpenStack Security Guide • http://docs.openstack.org/sec • Chapter 26 – Securing OpenStack Networking Services • Chapter 40 – Hypervisor Selection • Chapter 41 – Hardening the Virtualization Layers • Chapter 43 – Security Services for Instances
  65. 65. Thank You Please consider contributing to the OpenStack Security Group
  66. 66. References • Directly Referenced / Informed This Talk – http://www.insinuator.net/2013/05/analysis-of-hypervisor-breakouts/ – https://www.ernw.de/download/ERNW_DCVI-HypervisorsToClouds.pdf – https://www.hashdays.ch/downloads/slides/jonathan_sinclair_vm_state.pdf – ftp://public.dhe.ibm.com/linux/pdfs/LXW03004-USEN-00.pdf – http://blog.cryptographyengineering.com/2012/10/attack-of-week-cross-vm- timing-attacks.html – http://www.vupen.com/blog/20120904.Advanced_Exploitation_of_Xen_Sysre t_VM_Escape_CVE-2012-0217.php – http://www.symantec.com/avcenter/reference/Virtual_Machine_Threats.pdf – http://invisiblethingslab.com/resources/bh08/part1.pdf – http://blogs.gartner.com/neil_macdonald/2011/01/26/yes-hypervisors-are- vulnerable/ – ftp://public.dhe.ibm.com/common/ssi/ecm/en/wgl03003usen/WGL03003USE N.PDF
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×