HARDWARE SUPPORT
FOR EFFICIENT
VIRTUALIZATION
Lennox Wu

1
Outline

• Classifications

• Processor virtualization

Two main Software-based solutions
• Challenges to virtualize Intel...
Classifications

• VMM(virtual machine monitor) = hypervisor
• By techniques

Full Virtualization
• Paravirtualization
• H...
TYPE1/TYPE2 hypervisors

4
PROCESSOR
VIRTUALIZATION

5
Two main Software-based
solutions(1)
• Full virtualization using binary translation
•

Transforming guest OS binaries on-t...
Full virtualization using binary
translation

7
•

Two main Software-based
OS assisted virtualization or paravirtualization
solutions(2) guest OSs help the VMM
OS assist...
OS assisted virtualization or
paravirtualization

9
KVM
KVM

10
Challenges to virtualize Intel
x86(software-only)(1/3)
• Ring Aliasing
•

Guest-OSes run at the Ring3
•

•

Original: OS:R...
Challenges to virtualize Intel
x86(software-only)(2/3)
• Non-Faulting Access to Privileged State
•

Some instructions shou...
Challenges to virtualize Intel
x86(software-only) (3/3)
• Ring Compression
•

Guest OS runs at the same privilege level as...
Intel VT-x overview(1/4)
• VT=virtualization technology
•

Two new form of CPU operation
VMX root operation : for VMM
• VM...
Two new form of CPU operation

15
Intel VT-x overview(2/4)
• Two new transitions
• VM entry
• VMX root operation (VMM) non-root operation(VM)
• VM exit
• V...
Intel VT-x overview(3/4)
• VMCS (Virtual Machine Control Structure)

A new data structure includes guest-state area and ho...
Intel VT-x overview(4/4)
• VMCS supports interrupt virtualization
•

Determine the conditions of VM to cause VM exit
•
•
•...
Intel EPT / AMD NPT(1)
• EPT (Extended Page Tables)
•

•

“EPT provides performance gains of up to 48% for MMU-intensive
b...
Intel EPT / AMD NPT(2)
• Software MMU (software-only)

Hardware uses the shadow page table
• VMM must maintain the shadow ...
Intelmaintains guest page tables
EPT / AMD NPT(3)
Guest-OS

• Hardware MMU
•

VMM maintains PPN->MPN mappings in an additi...
AMD-V(1/2)
• Tagged TLB

Add the ASID
• Hardware features that facilitate efficient switching between virtual
machines for...
AMD-V(2/2)
• Nested page table (NPT)/ Rapid Virtualization Indexing (RVI)
• VMM migration
•

Use the CPUID to identify the...
Hardware-base solution with VTx(1/2)
• Address-Space Compression
•

VM Exits / VM Entries change the linear address space
...
Hardware-base solution with VTx(2/2)
• Guest System Calls
•

a guest OS can run at privilege level 0

• Frequent Access to...
Hardware Assisted Virtualization
of x86

26
I/O VIRTUALIZATION

27
Current I/O virtualization
techniques
• Emulation

The VMM supports virtual devices that guest OS can recognize
• The virt...
DMA on a virtualizing system
• DMA

Driver issue a packet consists of command, physical address, etc.
• DMA controller rea...
Intel VT-d(1/2)
• Need the support of the North bridge
• Two functions
•

Bind devices to a specify VM
•

•

DMA remapping...
DMA remapping

31
Intel VT-d (2/2)
•

Interrupt Remapping
• Assign an interrupt attribute
•

•

Destination processor, vector, etc.

A VMM e...
Intel VT-c
• Virtualization Technology for Connectivity
•

Virtualization on devices

• A collection of technologies that ...
Why VMDq?

34
35
VMDc

36
DO THESE
TECHNIQUES WORK?

37
Ubuntu 11.10: Xen vs. KVM vs.
VirtualBox(1)

38
Ubuntu 11.10: Xen vs. KVM vs.
VirtualBox(2)

39
Ubuntu 11.10: Xen vs. KVM vs.
VirtualBox(3)

40
Ubuntu 12.10: KVM vs. Xen (1)

41
Ubuntu 12.10: KVM vs. Xen (2)

42
Ubuntu 12.10: KVM vs. Xen (3)

43
Ubuntu 12.10: KVM vs. Xen (4)

44
Ubuntu 12.04 KVM/Xen
Virtualization: Intel vs. AMD(1)
• Ubuntu 12.04 LTS, an Intel Core i7 3960X "Sandy Bridge" Extreme

E...
Ubuntu 12.04 KVM/Xen
Virtualization: Intel vs. AMD(2)

46
Intel Ivy Bridge Linux
Virtualization Performance(1)

47
Intel Ivy Bridge Linux
Virtualization Performance(2)

48
Summarization of Hardware
Assisted Virtualization
• Hardware provides some mechanisms to reduce overheads of

virtualizati...
References
•

Performance Evaluation of Intel EPT Hardware Assist, VMware

•

I/O Virtualization and AMD's IOMMU
•

•

Pro...
References
• Ubuntu 11.10: Xen vs. KVM vs. VirtualBox

http://www.phoronix.com/scan.php?page=article&item=ubuntu_11
10_xen...
Q&A

52
THANK YOU

53
Upcoming SlideShare
Loading in...5
×

Hardware support for efficient virtualization

454

Published on

Hardware support for efficient virtualization

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
454
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
28
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • 按照INTEL的說法GOS該放RING3比較好, Ring Compression
  • Hypercalls不是硬體指令,是VMM開放出來的FUNCTION,供GOS和VMM溝通(*)採用一些優點,但CODE不用改
  • Performance of HW-based is based on techniques in 2006. Now(2012) the performance is improved! Look later slides.
  • Ring AliasingThe problem that arise when software is run at a privilege level other than the privilege level for which it was written
  • Adverse 不利的
  • Ring Compression:Ring 0-2 在記憶體方面是沒區分的,為了保護VMM,GOS也要放RING3
  • Address-Space Compression -VM Exits / VM Entries -> switch VMCS->switch address space
  • Device 1 is binded to domain 1
  • Switching over to the computationally-intensive tests, beginning with Google's libvpx VP8 encoding test
  • Hardware support for efficient virtualization

    1. 1. HARDWARE SUPPORT FOR EFFICIENT VIRTUALIZATION Lennox Wu 1
    2. 2. Outline • Classifications • Processor virtualization Two main Software-based solutions • Challenges to virtualize Intel x86(software-only) • Hardware-based Virtualization • Intel VT-x : x86 • Intel VT-I :Itanium (X) • Intel EPT/AMD NPT • AMD-V • Sun SPARC (X) • ARM Virtualization Extensions(X) • IBM Power(X) • • I/O virtualization • Intel VT-d • AMD IOMMU(AMD-V) • Intel VT-c • Dose these techniques work? 2
    3. 3. Classifications • VMM(virtual machine monitor) = hypervisor • By techniques Full Virtualization • Paravirtualization • Hardware Assisted Virtualization • • Robert P. Goldberg(Harvard University,1973) • Type 1/native/bare metal hypervisors Hypervisors run directly on the host's hardware to control the hardware and to manage guest operating systems. • Oracle VM Server for SPARC, the Citrix XenServer, KVM, VMware ESX/ESXi, and Microsoft Hyper-V hypervisor. • • Type 2/hosted hypervisors Hypervisors run within a conventional operating system environment. • VMware Workstation/player, Microsoft Virtual PC and VirtualBox • 3
    4. 4. TYPE1/TYPE2 hypervisors 4
    5. 5. PROCESSOR VIRTUALIZATION 5
    6. 6. Two main Software-based solutions(1) • Full virtualization using binary translation • Transforming guest OS binaries on-the-fly • • Guest applications don’t use privilege instructions Pros Support unmodified OSs (The only way of pure-software solutions) • Offer best isolation and security • Simplify migration and portability of guest OS • Cons: low performance • Examples: VMware, MS Virtual PC, Virtual box • • Disable HW virtualization 6
    7. 7. Full virtualization using binary translation 7
    8. 8. • Two main Software-based OS assisted virtualization or paravirtualization solutions(2) guest OSs help the VMM OS assisted virtualization • • • paravirtualization refers to communication between the guest OS and the VMM to improve performance and efficiency Modify the guest OS to cooperate with the VMM • Modify the OS kernel to replace non-virtualizable instructions with hypercalls(the functions provided by the VMM) Pros: Offer higher performance • Cons: Need the source code of an OS • Example: Xen, KVM(*), VMware(*) • (*) Vmware tool uses some paravirtualization techniques • optimize virtual device drivers • time synchronization • logging and guest shutdown. • Vmxnet is a paravirtualized I/O device driver • 8
    9. 9. OS assisted virtualization or paravirtualization 9
    10. 10. KVM KVM 10
    11. 11. Challenges to virtualize Intel x86(software-only)(1/3) • Ring Aliasing • Guest-OSes run at the Ring3 • • Original: OS:Ring 0, APP:Ring 3 (Ring0>ring3) A guest OS can know its run level • Address-Space Compression VMM must use some of the guest’s virtual-address space to manage transition between guest OS and VMM • VMM’s address spaces must be protected • • Guest could detect that it is running in a VM 11
    12. 12. Challenges to virtualize Intel x86(software-only)(2/3) • Non-Faulting Access to Privileged State • Some instructions should be intercepted by VMM do not involve faults • Adverse Impact on Guest System Calls • VMM must emulate every system calls • Interrupt Virtualization A VMM may manage external interrupts and deny guest to control interrupt masking • Some OS frequently mask and unmask • • VMM must process these requests. 12
    13. 13. Challenges to virtualize Intel x86(software-only) (3/3) • Ring Compression • Guest OS runs at the same privilege level as applications • The guest OS can’t protect guest applications • Frequent Access to Privileged Resources • VMM should deny the accesses • Address translation • Guest OS doesn’t know the physical address, so the VMM must intercepted guest page table updates 13
    14. 14. Intel VT-x overview(1/4) • VT=virtualization technology • Two new form of CPU operation VMX root operation : for VMM • VMX non-root operation: for guest-software • Both forms of operation support all four privilege levels(Ring0~Ring3) • • Guest OS can run at its intended privilege level 14
    15. 15. Two new form of CPU operation 15
    16. 16. Intel VT-x overview(2/4) • Two new transitions • VM entry • VMX root operation (VMM) non-root operation(VM) • VM exit • VMX non-root operation (VM) root operation (VMM) • Under VMX non-root operation, many instructions/events cause VM exits • configurable 16
    17. 17. Intel VT-x overview(3/4) • VMCS (Virtual Machine Control Structure) A new data structure includes guest-state area and host-state area • VM entry: load the guest-state area and save the host-state area • VM exit : load the host-state area and save the guest-state area • The exiting conditions controlled by the VM-execution fields • Switch the structure will switch the address space • 17
    18. 18. Intel VT-x overview(4/4) • VMCS supports interrupt virtualization • Determine the conditions of VM to cause VM exit • • • • • All interrupt Whenever guest OS is ready to receive interrupts Which exception? Which port access attempts? Which Model Specific Register access attempts? 18
    19. 19. Intel EPT / AMD NPT(1) • EPT (Extended Page Tables) • • “EPT provides performance gains of up to 48% for MMU-intensive benchmarks and up to 600% for MMU-intensive microbenchmarks.” – VMware AMD’s nested page table (NPT) is similar to EPT • A.k.a Rapid Virtualization Indexing (RVI) • “RVI provides performance gains of up to 42% for MMU-intensive benchmarks and up to 500% for MMU-intensive microbenchmarks.” -VMware 19
    20. 20. Intel EPT / AMD NPT(2) • Software MMU (software-only) Hardware uses the shadow page table • VMM must maintain the shadow page table • 20
    21. 21. Intelmaintains guest page tables EPT / AMD NPT(3) Guest-OS • Hardware MMU • VMM maintains PPN->MPN mappings in an additional level of page tables • The hardware will find the LPN->MPN with the two pages • 21
    22. 22. AMD-V(1/2) • Tagged TLB Add the ASID • Hardware features that facilitate efficient switching between virtual machines for better application responsiveness • • Two new form of CPU operation Host mode : for VMM (similar to Intel’s VMX root operation) • Guest mode : for guest software (similar to Intel’s VMX non-root operation) • new instructions • • • • • vmrun : host mode  guest mode exit : guest mode  host mode vmcall: it lets the operating system and VMM communicate directly A new structure • Virtual Machine Control Block (VMCB) • Similar to Intel’s VMCS 22
    23. 23. AMD-V(2/2) • Nested page table (NPT)/ Rapid Virtualization Indexing (RVI) • VMM migration • Use the CPUID to identify the ability of the processor where the VMM runs, and the VMM use the supported functions. 23
    24. 24. Hardware-base solution with VTx(1/2) • Address-Space Compression • VM Exits / VM Entries change the linear address space • Ring Aliasing • & Ring Compression VT-x allows guest OS to run at its intended privilege level • Nonfaulting Access to Privileged State Either causes transition to VMM • Or becomes unimportant to VMM • 24
    25. 25. Hardware-base solution with VTx(2/2) • Guest System Calls • a guest OS can run at privilege level 0 • Frequent Access to Privileged Resources • VT-x provides TPR shadow. VMM is only involved when the value drops below the threshold VMM only processes the situation it cares. 25
    26. 26. Hardware Assisted Virtualization of x86 26
    27. 27. I/O VIRTUALIZATION 27
    28. 28. Current I/O virtualization techniques • Emulation The VMM supports virtual devices that guest OS can recognize • The virtual device models are responsible to translate commands and data. • Pros. No requirement to modify guest-OSs • Cons. Low performance • • Paravirtualization Modify the guest software (driver)to enhance the performance • Pros. better performance • Cons. Limited applicability. (modify need the source code) • • Direct assignment Bind a specify device to a VM • VMM allow the owning VM to connect directly • Issue command (go) low overhead • DMA? (back) • 28
    29. 29. DMA on a virtualizing system • DMA Driver issue a packet consists of command, physical address, etc. • DMA controller read/write data from/to the physical address • Challenge? • A physical address that a Guest-OS knows is not really physical ! • The really physical address space is managed by the VMM • The DMA controller will incorrectly write data to an address. • 29
    30. 30. Intel VT-d(1/2) • Need the support of the North bridge • Two functions • Bind devices to a specify VM • • DMA remapping Interrupt virtualization • Interrupt remapping • DMA remapping DVA (DMA Virtual Address), GPA(Guest Physical Address), HPA(Host Physical Address) • A guest-OS issue a DMA request with DVA(=GPA) • The VT-d hardware will translate the DVA to HPA • • The concept: lookup tables 30
    31. 31. DMA remapping 31
    32. 32. Intel VT-d (2/2) • Interrupt Remapping • Assign an interrupt attribute • • Destination processor, vector, etc. A VMM enables the interrupt requests from the I/O device to target the physical CPUs running the appropriate virtual CPUs of the legacy VM • AMD IOMMU is similar to Intel VT-d 32
    33. 33. Intel VT-c • Virtualization Technology for Connectivity • Virtualization on devices • A collection of technologies that improve the performance of network I/O on a virtualized system • VT-c is comprised of two components • VMDq (Virtual Machine Device Queues) A hardware-base enhancement • Target: throughput • • VMDc (Virtual Machine Direct Connect) Virtualizing physical I/O ports of a network controller into multiple virtual I/O ports, and then to map the virtual ports to individual VMs • Target :VT-x + VT-d + VT-c  nearly native performance • 33
    34. 34. Why VMDq? 34
    35. 35. 35
    36. 36. VMDc 36
    37. 37. DO THESE TECHNIQUES WORK? 37
    38. 38. Ubuntu 11.10: Xen vs. KVM vs. VirtualBox(1) 38
    39. 39. Ubuntu 11.10: Xen vs. KVM vs. VirtualBox(2) 39
    40. 40. Ubuntu 11.10: Xen vs. KVM vs. VirtualBox(3) 40
    41. 41. Ubuntu 12.10: KVM vs. Xen (1) 41
    42. 42. Ubuntu 12.10: KVM vs. Xen (2) 42
    43. 43. Ubuntu 12.10: KVM vs. Xen (3) 43
    44. 44. Ubuntu 12.10: KVM vs. Xen (4) 44
    45. 45. Ubuntu 12.04 KVM/Xen Virtualization: Intel vs. AMD(1) • Ubuntu 12.04 LTS, an Intel Core i7 3960X "Sandy Bridge" Extreme Edition and AMD FX-8150 "Bulldozer" systems were used. 45
    46. 46. Ubuntu 12.04 KVM/Xen Virtualization: Intel vs. AMD(2) 46
    47. 47. Intel Ivy Bridge Linux Virtualization Performance(1) 47
    48. 48. Intel Ivy Bridge Linux Virtualization Performance(2) 48
    49. 49. Summarization of Hardware Assisted Virtualization • Hardware provides some mechanisms to reduce overheads of virtualization to improve performance • Pros. The highest performance in theory (a counter example, 2006 VMware) • Support unmodified Oss • Simplify the development of VMM • • Cons. • Need newer processors • Example • KVM(basic requirements) 49
    50. 50. References • Performance Evaluation of Intel EPT Hardware Assist, VMware • I/O Virtualization and AMD's IOMMU • • Processor-Based Virtualization, AMD64 Style, Part I • • http://developer.amd.com/documentation/articles/pages/630200614.aspx Processor-Based Virtualization, AMD64 Style, Part II • • http://developer.amd.com/documentation/articles/pages/892006101.aspx http://developer.amd.com/documentation/articles/pages/630200615.aspx Intel technology Journal, vol 10, issue 3, 2006 Intel virtualization technology: Hardware Support for Efficient processor virtualization • Intel virtualization technology for Directed I/O • • ARM virtualization Extension Architecture Specification • A Comparison of software and hardware techniques for x86 virtualization,Vmware • http://www.intel.com/network/connectivity/solutions/vmdc.htm • http://www.intel.com/network/connectivity/solutions/vmdq.htm • http://software.intel.com/en-us/blogs/2009/09/30/understanding-vt-c-virtualizationtechnology-for-connectivity/ 50
    51. 51. References • Ubuntu 11.10: Xen vs. KVM vs. VirtualBox http://www.phoronix.com/scan.php?page=article&item=ubuntu_11 10_xenkvm&num=1 • Ubuntu 12.04 KVM/Xen Virtualization: Intel vs. AMD http://www.phoronix.com/scan.php?page=article&item=ubuntu_12 04_virt&num=1 • Intel Ivy Bridge Linux Virtualization Performance http://www.phoronix.com/scan.php?page=article&item=intel_iv y_virtualization&num=5 • http://en.wikipedia.org/wiki/Hypervisor 51
    52. 52. Q&A 52
    53. 53. THANK YOU 53
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×