XPDS13 : PVH Technical Deep Dive - George Dunlap, Citrix
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

XPDS13 : PVH Technical Deep Dive - George Dunlap, Citrix

on

  • 1,282 views

PVH is a new guest type that is designed to take the best features of both PV and HVM guests. This talk will be a technical deep-dive: explaining exactly which features of HVM and PV guests are used, ...

PVH is a new guest type that is designed to take the best features of both PV and HVM guests. This talk will be a technical deep-dive: explaining exactly which features of HVM and PV guests are used, and which ones are discarded, and why. The primary goal will be to inform developers about the details of PVH to make it easier for them to interact with the code.

Statistics

Views

Total Views
1,282
Views on SlideShare
1,243
Embed Views
39

Actions

Likes
1
Downloads
56
Comments
0

3 Embeds 39

http://www.xenproject.org 27
http://xenproject.org 6
https://twitter.com 6

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

XPDS13 : PVH Technical Deep Dive - George Dunlap, Citrix Presentation Transcript

  • 1. Intro PV and HVM PVH and Xen PVH and Linux PVH technical deep-dive George Dunlap Edinburg – 21-23 October, 2013 PVH Issues Spare slides
  • 2. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Intro: PVH Edinburg – 21-23 October, 2013 PVH technical deep-dive 2 / 25
  • 3. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Goal Give you a technical overview of PVH, so that... Edinburg – 21-23 October, 2013 PVH technical deep-dive 3 / 25
  • 4. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Goal Give you a technical overview of PVH, so that... you can understand the characteristics, advantages, and disadvantages... Edinburg – 21-23 October, 2013 PVH technical deep-dive 3 / 25
  • 5. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Goal Give you a technical overview of PVH, so that... you can understand the characteristics, advantages, and disadvantages... and approach the code to improve and fix Edinburg – 21-23 October, 2013 PVH technical deep-dive 3 / 25
  • 6. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Overview PV, HVM, and PVH Edinburg – 21-23 October, 2013 PVH technical deep-dive 4 / 25
  • 7. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Overview PV, HVM, and PVH PVH from Xen’s perspective Edinburg – 21-23 October, 2013 PVH technical deep-dive 4 / 25
  • 8. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Overview PV, HVM, and PVH PVH from Xen’s perspective PVH from Linux’s perspective Edinburg – 21-23 October, 2013 PVH technical deep-dive 4 / 25
  • 9. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Overview PV, HVM, and PVH PVH from Xen’s perspective PVH from Linux’s perspective Issues in PVH Edinburg – 21-23 October, 2013 PVH technical deep-dive 4 / 25
  • 10. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PV and HVM Issues with PV Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
  • 11. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PV and HVM Issues with PV PVMMU in Linux Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
  • 12. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PV and HVM Issues with PV PVMMU in Linux 64-bit hypercalls Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
  • 13. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PV and HVM Issues with PV PVMMU in Linux 64-bit hypercalls Issues with HVM (PVHVM) Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
  • 14. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PV and HVM Issues with PV PVMMU in Linux 64-bit hypercalls Issues with HVM (PVHVM) Qemu process Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
  • 15. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PV and HVM Issues with PV PVMMU in Linux 64-bit hypercalls Issues with HVM (PVHVM) Qemu process Legacy boot Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
  • 16. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PV and HVM Issues with PV PVMMU in Linux 64-bit hypercalls Issues with HVM (PVHVM) Qemu process Legacy boot Devices emulated in Xen Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
  • 17. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH “PV in an HVM container” Edinburg – 21-23 October, 2013 PVH technical deep-dive 6 / 25
  • 18. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH “PV in an HVM container” Goal: take the best aspects of PV and HVM Edinburg – 21-23 October, 2013 PVH technical deep-dive 6 / 25
  • 19. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH “PV in an HVM container” Goal: take the best aspects of PV and HVM Written by Mukesh Rathor @ Oracle Edinburg – 21-23 October, 2013 PVH technical deep-dive 6 / 25
  • 20. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH “PV in an HVM container” Goal: take the best aspects of PV and HVM Written by Mukesh Rathor @ Oracle Significant revisions by George Dunlap @ Citrix Edinburg – 21-23 October, 2013 PVH technical deep-dive 6 / 25
  • 21. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH from Xen’s perspective Begin with HVM guest Edinburg – 21-23 October, 2013 PVH technical deep-dive 7 / 25
  • 22. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH from Xen’s perspective Begin with HVM guest Disable HVM-specific things not needed Edinburg – 21-23 October, 2013 PVH technical deep-dive 7 / 25
  • 23. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH from Xen’s perspective Begin with HVM guest Disable HVM-specific things not needed Start & pin in 64-bit paged mode Edinburg – 21-23 October, 2013 PVH technical deep-dive 7 / 25
  • 24. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH from Xen’s perspective Begin with HVM guest Disable HVM-specific things not needed Start & pin in 64-bit paged mode Enable PV path for a handful of things Edinburg – 21-23 October, 2013 PVH technical deep-dive 7 / 25
  • 25. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: Things disabled Device model (qemu) Edinburg – 21-23 October, 2013 PVH technical deep-dive 8 / 25
  • 26. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: Things disabled Device model (qemu) Emulated hardware (APIC, PIT, etc) Edinburg – 21-23 October, 2013 PVH technical deep-dive 8 / 25
  • 27. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: Things disabled Device model (qemu) Emulated hardware (APIC, PIT, etc) Nested HVM Edinburg – 21-23 October, 2013 PVH technical deep-dive 8 / 25
  • 28. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: Things disabled Device model (qemu) Emulated hardware (APIC, PIT, etc) Nested HVM MSIX Edinburg – 21-23 October, 2013 PVH technical deep-dive 8 / 25
  • 29. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: Pinning in 64-bit Set initial values ofr cr0, cr4, EFER Edinburg – 21-23 October, 2013 PVH technical deep-dive 9 / 25
  • 30. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: Pinning in 64-bit Set initial values ofr cr0, cr4, EFER Certain paging things have to happen at start of day Edinburg – 21-23 October, 2013 PVH technical deep-dive 9 / 25
  • 31. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: Pinning in 64-bit Set initial values ofr cr0, cr4, EFER Certain paging things have to happen at start of day Disable writes to EFER Edinburg – 21-23 October, 2013 PVH technical deep-dive 9 / 25
  • 32. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: Pinning in 64-bit Set initial values ofr cr0, cr4, EFER Certain paging things have to happen at start of day Disable writes to EFER Don’t allow guest to change paging-related cr0 bits Edinburg – 21-23 October, 2013 PVH technical deep-dive 9 / 25
  • 33. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: PV paths Enable PV hypercalls Edinburg – 21-23 October, 2013 PVH technical deep-dive 10 / 25
  • 34. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: PV paths Enable PV hypercalls PV e820 map Edinburg – 21-23 October, 2013 PVH technical deep-dive 10 / 25
  • 35. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: PV paths Enable PV hypercalls PV e820 map PVH VCPU boot Edinburg – 21-23 October, 2013 PVH technical deep-dive 10 / 25
  • 36. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: PV paths Enable PV hypercalls PV e820 map PVH VCPU boot PV CPUID Edinburg – 21-23 October, 2013 PVH technical deep-dive 10 / 25
  • 37. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: PV paths Enable PV hypercalls PV e820 map PVH VCPU boot PV CPUID PV PIO Edinburg – 21-23 October, 2013 PVH technical deep-dive 10 / 25
  • 38. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH from Linux’s perspective xen hvm domain() false, xen pv domain() true Edinburg – 21-23 October, 2013 PVH technical deep-dive 11 / 25
  • 39. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH from Linux’s perspective xen hvm domain() false, xen pv domain() true Act natural (get rid of PV special cases) Edinburg – 21-23 October, 2013 PVH technical deep-dive 11 / 25
  • 40. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH from Linux’s perspective xen hvm domain() false, xen pv domain() true Act natural (get rid of PV special cases) Autotranslate side effects Edinburg – 21-23 October, 2013 PVH technical deep-dive 11 / 25
  • 41. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH from Linux’s perspective xen hvm domain() false, xen pv domain() true Act natural (get rid of PV special cases) Autotranslate side effects Use PVHVM callback vector setup Edinburg – 21-23 October, 2013 PVH technical deep-dive 11 / 25
  • 42. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH from Linux’s perspective xen hvm domain() false, xen pv domain() true Act natural (get rid of PV special cases) Autotranslate side effects Use PVHVM callback vector setup VCPU bring-up Edinburg – 21-23 October, 2013 PVH technical deep-dive 11 / 25
  • 43. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled No PV IDT Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
  • 44. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled No PV IDT No PV irq ops Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
  • 45. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled No PV IDT No PV irq ops No PV CPUID Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
  • 46. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled No PV IDT No PV irq ops No PV CPUID Native Syscall/sysenter Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
  • 47. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled No PV IDT No PV irq ops No PV CPUID Native Syscall/sysenter No VM assists Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
  • 48. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled No PV IDT No PV irq ops No PV CPUID Native Syscall/sysenter No VM assists No event / failsafe callbacks Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
  • 49. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled No PV IDT No PV irq ops No PV CPUID Native Syscall/sysenter No VM assists No event / failsafe callbacks No need to set IOPL Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
  • 50. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled MMUOPS: No need to pin pagetables Edinburg – 21-23 October, 2013 PVH technical deep-dive 13 / 25
  • 51. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled MMUOPS: No need to pin pagetables No pfn/mfn conversion Edinburg – 21-23 October, 2013 PVH technical deep-dive 13 / 25
  • 52. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled MMUOPS: No need to pin pagetables No pfn/mfn conversion No need to special-case PT protections Edinburg – 21-23 October, 2013 PVH technical deep-dive 13 / 25
  • 53. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Things disabled MMUOPS: No need to pin pagetables No pfn/mfn conversion No need to special-case PT protections Only pvmmu op: flush tlb others Edinburg – 21-23 October, 2013 PVH technical deep-dive 13 / 25
  • 54. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate PV Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
  • 55. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate PV Pagetables controlled by Xen Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
  • 56. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate PV Pagetables controlled by Xen Real MFN in pagetables Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
  • 57. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate PV Pagetables controlled by Xen Real MFN in pagetables “Map this page here” hypercall Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
  • 58. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate PV Pagetables controlled by Xen Real MFN in pagetables “Map this page here” hypercall PVH Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
  • 59. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate PV Pagetables controlled by Xen Real MFN in pagetables “Map this page here” hypercall PVH Pagetables controlled by guest Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
  • 60. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate PV Pagetables controlled by Xen Real MFN in pagetables “Map this page here” hypercall PVH Pagetables controlled by guest gpfn in pagetables Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
  • 61. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate PV Pagetables controlled by Xen Real MFN in pagetables “Map this page here” hypercall PVH Pagetables controlled by guest gpfn in pagetables Every mapped page must be in the p2m Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
  • 62. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate “Special pages”: grant frame, make a hole in the p2m Edinburg – 21-23 October, 2013 PVH technical deep-dive 15 / 25
  • 63. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate “Special pages”: grant frame, make a hole in the p2m Need to add p2m entries to map foreign pages Edinburg – 21-23 October, 2013 PVH technical deep-dive 15 / 25
  • 64. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: Autotranslate “Special pages”: grant frame, make a hole in the p2m Need to add p2m entries to map foreign pages Grant table: similar to HVM (but not the same) Edinburg – 21-23 October, 2013 PVH technical deep-dive 15 / 25
  • 65. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: VCPU bring-up Brought up via hypercalls (PV) Edinburg – 21-23 October, 2013 PVH technical deep-dive 16 / 25
  • 66. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: VCPU bring-up Brought up via hypercalls (PV) Guest in control of IDT, Xen can’t guarantee loading properly Edinburg – 21-23 October, 2013 PVH technical deep-dive 16 / 25
  • 67. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: VCPU bring-up Brought up via hypercalls (PV) Guest in control of IDT, Xen can’t guarantee loading properly Only set GS by default Edinburg – 21-23 October, 2013 PVH technical deep-dive 16 / 25
  • 68. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Linux: VCPU bring-up Brought up via hypercalls (PV) Guest in control of IDT, Xen can’t guarantee loading properly Only set GS by default All other state must be set in on-cpu boot code Edinburg – 21-23 October, 2013 PVH technical deep-dive 16 / 25
  • 69. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH: Not yet working 32-bit Edinburg – 21-23 October, 2013 PVH technical deep-dive 17 / 25
  • 70. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH: Not yet working 32-bit vtsc Edinburg – 21-23 October, 2013 PVH technical deep-dive 17 / 25
  • 71. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH: Not yet working 32-bit vtsc shadow mode Edinburg – 21-23 October, 2013 PVH technical deep-dive 17 / 25
  • 72. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides PVH: Not yet working 32-bit vtsc shadow mode vcpu hotplug Edinburg – 21-23 October, 2013 PVH technical deep-dive 17 / 25
  • 73. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: HVM paths Original idea: “Lightweight” container Edinburg – 21-23 October, 2013 PVH technical deep-dive 18 / 25
  • 74. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: HVM paths Original idea: “Lightweight” container Reality: Large amount of code necessary for minimum functionality Edinburg – 21-23 October, 2013 PVH technical deep-dive 18 / 25
  • 75. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: HVM paths Original idea: “Lightweight” container Reality: Large amount of code necessary for minimum functionality Very large amount of code duplication Edinburg – 21-23 October, 2013 PVH technical deep-dive 18 / 25
  • 76. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: HVM paths Original idea: “Lightweight” container Reality: Large amount of code necessary for minimum functionality Very large amount of code duplication Current patch: use HVM paths, with special cases for PVH Edinburg – 21-23 October, 2013 PVH technical deep-dive 18 / 25
  • 77. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Direct access permitted Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
  • 78. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Direct access permitted pv PIT Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
  • 79. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Direct access permitted pv PIT cmos idx Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
  • 80. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Direct access permitted pv PIT cmos idx pci config space Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
  • 81. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Direct access permitted pv PIT cmos idx pci config space “Abusing” IO with SMM Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
  • 82. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Direct access permitted pv PIT cmos idx pci config space “Abusing” IO with SMM Execute exact instruction with guest GPRs Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
  • 83. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Problems Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
  • 84. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Problems Ugly changes to allow PV paths to emulate for PVH guests Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
  • 85. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Problems Ugly changes to allow PV paths to emulate for PVH guests Race condition because of double checking Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
  • 86. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Problems Ugly changes to allow PV paths to emulate for PVH guests Race condition because of double checking Possibilities Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
  • 87. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Problems Ugly changes to allow PV paths to emulate for PVH guests Race condition because of double checking Possibilities Need 2 sets of access controls: For user procs, and for guest Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
  • 88. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Problems Ugly changes to allow PV paths to emulate for PVH guests Race condition because of double checking Possibilities Need 2 sets of access controls: For user procs, and for guest PV only has one Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
  • 89. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Issues: PIO instructions Problems Ugly changes to allow PV paths to emulate for PVH guests Race condition because of double checking Possibilities Need 2 sets of access controls: For user procs, and for guest PV only has one PVH: Have two, so may not need to emulate anymore Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
  • 90. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 91. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 92. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Shadow: Trip through Xen Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 93. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Shadow: Trip through Xen TLB effectiveness Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 94. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Shadow: Trip through Xen TLB effectiveness HAP: Superpages allow 2MiB / 1GiB TLB entries Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 95. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Shadow: Trip through Xen TLB effectiveness HAP: Superpages allow 2MiB / 1GiB TLB entries Shadow: Still only 4k TLB entries Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 96. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Shadow: Trip through Xen TLB effectiveness HAP: Superpages allow 2MiB / 1GiB TLB entries Shadow: Still only 4k TLB entries TLB replacement cost Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 97. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Shadow: Trip through Xen TLB effectiveness HAP: Superpages allow 2MiB / 1GiB TLB entries Shadow: Still only 4k TLB entries TLB replacement cost Shadow: Worst case, 4 memory reads Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 98. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Shadow: Trip through Xen TLB effectiveness HAP: Superpages allow 2MiB / 1GiB TLB entries Shadow: Still only 4k TLB entries TLB replacement cost Shadow: Worst case, 4 memory reads HAP: 9-16 memory reads Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 99. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Shadow: Trip through Xen TLB effectiveness HAP: Superpages allow 2MiB / 1GiB TLB entries Shadow: Still only 4k TLB entries TLB replacement cost Shadow: Worst case, 4 memory reads HAP: 9-16 memory reads Does this matter? Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 100. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Shadow: Trip through Xen TLB effectiveness HAP: Superpages allow 2MiB / 1GiB TLB entries Shadow: Still only 4k TLB entries TLB replacement cost Shadow: Worst case, 4 memory reads HAP: 9-16 memory reads Does this matter? Kernel build: Shadow 30% slower than HAP Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 101. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides HAP vs Shadow Page table update HAP: Memory write Shadow: Trip through Xen TLB effectiveness HAP: Superpages allow 2MiB / 1GiB TLB entries Shadow: Still only 4k TLB entries TLB replacement cost Shadow: Worst case, 4 memory reads HAP: 9-16 memory reads Does this matter? Kernel build: Shadow 30% slower than HAP SpecJBB: Shadow 30% faster than HAP Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
  • 102. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Overview PV, HVM, and PVH Edinburg – 21-23 October, 2013 PVH technical deep-dive 22 / 25
  • 103. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Overview PV, HVM, and PVH PVH from Xen’s perspective Edinburg – 21-23 October, 2013 PVH technical deep-dive 22 / 25
  • 104. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Overview PV, HVM, and PVH PVH from Xen’s perspective PVH from Linux’s perspective Edinburg – 21-23 October, 2013 PVH technical deep-dive 22 / 25
  • 105. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Overview PV, HVM, and PVH PVH from Xen’s perspective PVH from Linux’s perspective Issues in PVH Edinburg – 21-23 October, 2013 PVH technical deep-dive 22 / 25
  • 106. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Goal Give you a technical overview of PVH, so that... Edinburg – 21-23 October, 2013 PVH technical deep-dive 23 / 25
  • 107. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Goal Give you a technical overview of PVH, so that... you can understand the characteristics, advantages, and disadvantages... Edinburg – 21-23 October, 2013 PVH technical deep-dive 23 / 25
  • 108. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Goal Give you a technical overview of PVH, so that... you can understand the characteristics, advantages, and disadvantages... and approach the code to improve and fix Edinburg – 21-23 October, 2013 PVH technical deep-dive 23 / 25
  • 109. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Questions Questions? Feedback? george.dunlap@citrix.com Check out our blog: http://blog.xen.org/ Edinburg – 21-23 October, 2013 PVH technical deep-dive 24 / 25
  • 110. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: PVH hypercalls memory op Edinburg – 21-23 October, 2013 PVH technical deep-dive 25 / 25
  • 111. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: PVH hypercalls memory op console io Edinburg – 21-23 October, 2013 PVH technical deep-dive 25 / 25
  • 112. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: PVH hypercalls memory op console io vcpu op Edinburg – 21-23 October, 2013 PVH technical deep-dive 25 / 25
  • 113. Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides Xen: PVH hypercalls memory op console io vcpu op mmuext op Edinburg – 21-23 October, 2013 PVH technical deep-dive 25 / 25