Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

HKG15-400: Next steps in KVM enablement on ARM


Published on

HKG15-400: Next steps in KVM enablement on ARM
Date: February 12, 2015
★ Session Summary ★
Next steps in KVM enablement on ARM
★ Resources ★
★ Event Details ★
Linaro Connect Hong Kong 2015 - #HKG15
February 9-13th, 2015
Regal Airport Hotel Hong Kong Airport

Published in: Software
  • Login to see the comments

HKG15-400: Next steps in KVM enablement on ARM

  1. 1. KVM/arm64 Architectural Evolutions Marc Zyngier <> LCA ’15 1 CONFIDENTIAL
  2. 2. What to Expect in This Presentation During this presentation, we will discuss: I The ARM® architecture Virtualization Extensions... I ... and how KVM/ARM uses them I How the ARM architecture is evolving... I ... and what this means for KVM/ARM 2 CONFIDENTIAL
  3. 3. ARMv8-A Privilege Model I Supports both AArch64 and AArch32 execution state I 32-64bit interworking limited to exception boundaries I AArch64 always has a higher privilege than AArch32 I AArch64 state is inclusive of the lower-privileged 32bit exception level 3 CONFIDENTIAL
  4. 4. ARM Architecture Virtualization Extensions I Introduced with the latest revision of the ARMv7 architecture I New hypervisor execution state (EL2 or HYP) I Non-secure world, higher privilege than EL1 I Second stage translation I Adds an extra level of indirection between guests and physical memory I A form of nested page tables, as implemented by many other architectures I TLBs are tagged by Virtual Machine ID (VMID) I Ability to trap access to most system registers I The hypervisor decides what it wants to trap I Can handle IRQs, FIQs and asynchronous aborts I The guest doesn’t see physical interrupts firing, for example I Guests can call into HYP mode (HVC instruction) I Allows for para-virtualized services I Standard architecture peripherals are virtualization-aware I GIC and timers have specific features to help virtualization 4 CONFIDENTIAL
  5. 5. EL2: Not EL1++ HYP is not a superset of NS-EL1, but rather an orthogonal mode allowing for multiple NS-EL1 guests to be multiplexed on the hardware. I Own translation regime I Separate Stage 1 translation, no Stage 2 translation I Broadly follows the LPAE format I Only one Translation Table Base Register (TTBR0_EL2) I It would be very difficult to run Linux in EL2 I Requires too many changes to be practical I Instead, the EL2 mode can be used as a “world switch” I Between guests (bare metal hypervisor / Type-1) I Between host and guests (hosted hypervisor / Type-2). This makes the host a form of specialized guest 5 CONFIDENTIAL
  6. 6. KVM/ARM: General Architecture KVM/ARM uses HYP mode to switch between host and guest Saving and restoring: I Stage 2 translation table I Trap configuration I GP registers I System control registers I Floating point I GIC configuration I Timers EL1 EL0 EL2 KVM Host Kernel Switching Code Guest Kernel Guest Kernel Host Userspace Userspace Userspace Guest Guest 6 CONFIDENTIAL
  7. 7. The Evolution of ARMv8 The ARM architecture is in constant evolution to better fit the software ecosystem. The upcoming ARMv8.1 revision contains a number of points of interest: I New atomic instructions I New SIMD instructions I Memory management improvements I Performance monitoring improvements I VMIDs expanded to 16 bits I New Virtualization Host extensions, designed for Type-2 hypervisors Unsurprisingly, this last item is very interesting for KVM 7 CONFIDENTIAL
  8. 8. ARMv8.1: an Improved EL2 The Virtualization Host Extensions (VHE) expand the capabilities of EL2: I Designed to improve the support of Type-2 hypervisors I Allows the host OS to be run at EL2 I The host OS requires minimal changes to run at EL2 I User-space still runs at EL0 I Guest OSes run at EL1 I Host has no software running at EL1 I AArch64 specific EL2 becomes a strict superset of EL1 8 CONFIDENTIAL
  9. 9. VHE: Software compatibility Major design goals for VHE: I Make architecture features discoverable I Allow EL1 software to run at EL2 transparently I Put the burden of the change on virtualization software When VHE is in use: I Most EL2 system registers are accessed with their EL0/EL1 counterpart I TCR_EL1 access at EL2 really accesses TCR_EL2 I EL0/EL1 system register access from EL2 requires a specific accessor I To access TCR_EL1, you must use TCR_EL12 As a consequence of the above: I The kernel should run mostly unmodified I Virtualization software has to be modified 9 CONFIDENTIAL
  10. 10. VHE: Impacts on the Linux Kernel With VHE enabled, the Linux kernel can now be run at EL2. Changes required for the kernel itself: I Prevent the kernel from switching to EL1 I New debug configuration I Use HYP timer interrupt instead of the Guest’s I And that’s about it. The KVM changes are more invasive: I New method to enter the switch code I New way to address guest’s system registers I Vectors are switched depending on the role (host kernel or hypervisor) EL1 EL0 EL2 Guest Kernel Guest Kernel Host Userspace Userspace Userspace Guest Guest Host Kernel + KVM 10 CONFIDENTIAL
  11. 11. VHE: Impacts on the KVM/arm64 Architecture I Host system register save/restore is reduced I Almost no system register sharing I Down to four registers (tpidr*, actlr) I Guest system register lazy save/restore I Can be deferred until actually required by the host I Or until scheduling another VM I Could significantly reduce system register churn I Reduced physical interrupt latency I Only a minimal state has to be restored before handling the interrupt I No trap to enter the KVM code I Just a normal function call I No HYP mappings anymore I The whole kernel is there I Heavily re-written switching code 11 CONFIDENTIAL
  12. 12. VHE: One Kernel to Rule Them All I We want one single kernel binary I We want it to support all revisions of ARMv8 I That does include KVM as well I We need to be able to pick the right code at runtime Two complementary approaches: I Using feature bits to make runtime decisions I if (is_kernel_in_hyp_mode()) {...} else {...} I Simple, but only on slow path I Using runtime kernel patching to rewite critical code I ifvhe "mrs x2, TCR_EL1", "mrs x2, TCR_EL12" I Very efficient, good for fast path I Less readable, harder to maintain I Mostly for assembly code 12 CONFIDENTIAL
  13. 13. VHE: News from the Coding Front This is still a work in progress: I Prototype code based on 3.15-rc3 I A nasty ball of hacks I Only used for architecture evaluation I Unmaintainable, and unmaintained I Will hopefully be deleted soon I Proper support in progress I Based on Mainline of the Day I Using code patching extensively I Tested on ARM Fast Models I Hopefully public soon (-ish) 13 CONFIDENTIAL
  14. 14. Thank You The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited (or its subsidiaries) in the EU and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners. 14 CONFIDENTIAL