Successfully reported this slideshow.
Your SlideShare is downloading. ×

Static partitioning virtualization on RISC-V

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
RISC-V Zce Extension
RISC-V Zce Extension
Loading in …3
×

Check these out next

1 of 30 Ad
Advertisement

More Related Content

Slideshows for you (20)

Similar to Static partitioning virtualization on RISC-V (20)

Advertisement

More from RISC-V International (20)

Recently uploaded (20)

Advertisement

Static partitioning virtualization on RISC-V

  1. 1. Static Partitioning Virtualization on RISC-V José Martins and Sandro Pinto RISC-V Summit @ Virtual December 8th, 2020
  2. 2. Agenda Introduction Motivation and Background 01 RISC-V Virtualization RISC-V virtualization in a nutshell 02 RISC-V H-Extension Implementation & enhancements 03 Evaluation Tests, experiments, and results 04 Conclusion Final thoughts and Takeaways 05
  3. 3. Introduction Motivation and Background
  4. 4. Mixed-Criticality 4 ESRGv3 RISC-V Summit 20 • High Complexity • Multiple Subsystems • Heterogeneous Software Stacks (RTOS, GPOS) • Different Criticality Levels • Size, Weight, Power and Cost (SWaP-C) Automotive Industrial Automation
  5. 5. Embedded Virtualization 5 ESRGv3 ESRGv3 RISC-V Summit 20 Consolidation Size Weight Power Cost Low Engineering Cost Full-Virtualization allows direct porting of guest OSs Fault Containment Sandboxed Enviroments Performance Low Virtualization Overhead Close to Native Performance Security Small TCB Side-channel TEE support Open Design Real-time Low latencies Determinism/Predictability Freedom-from-interference
  6. 6. Traditional Hypervisors 6 ESRGv3 ▪ Not designed for embedded/MCS: ▪ Although retrofitted with success ▪ High-overhead IO: ▪ Emulated ▪ Para-virtualization/Backend-drivers ▪ Large Code Base : ▪ Hosted hypervisor ▪ Privileged VMs run large monolithic OSs (typically Linux) part of TCB. ESRGv3 RISC-V Summit 20
  7. 7. Static Partitioning Virtualization 7 ESRGv3 Jailhouse  Static Partitioning:  Thin configuration/partition layer  1:1 vCPU-to-pCPU mapping  Static memory assignment  Device Pass-through  Hardware interrupts  Jailhouse:  Needs “root cell” to boot and manage VMs  Large boot time  Xen Dom0-less*:  DomUs may boot w/o Dom0  Direct Device assignment Dom0-less ESRGv3 RISC-V Summit 20 * https://www.youtube.com/watch?v=OrtV6gyHW74
  8. 8. Bao Overview 8 ESRGv3 01 02 03  Type-1 / Bare-metal  Static Partitioning:  1:1 vCPU-to-pCPU mapping  Static memory assignment  Device Pass-through  Hardware-assisted:  2nd-stage translation  Interrupt virtualization support  IOMMU  Dependencies:  No external libraries / privileged VMs  Small TCB (~7K SLoC)  Real-time & Security:  Predictability / Freedom-from-interference  Side-channels / TEE support ESRGv3 RISC-V Summit 20 José Martins, Adriano Tavares, Marco Solieri, Marko Bertogna, and Sandro Pinto. "Bao: A Lightweight Static Partitioning Hypervisor for Modern Multi-Core Embedded Systems". In NG-RES 2020 https://drops.dagstuhl.de/opus/volltexte/2020/11779/
  9. 9. Bao Hypervisor Support 9 ESRGv3 01 03  Architectures:  Armv8-A  RISC-V (v0.6.1 Hyp Spec.)  Platforms:  Zynq US+ (ZCUx and Ultra96)  HiSilicon96 (Hikey96)  NXP i.MX8  Nvidia Tegra TX2  QEMU  Rocket @ ZCU  Firmware:  Arm Trusted Firmware (PSCI) on Arm  Supervisor Binary Interface (SBI) on RISC-V  U-boot  Guests:  Bare-metal  Linux / Android  RTOSs (FreeRTOS, Erika) ESRGv3 RISC-V Summit 20
  10. 10. RISC-V Virtualization RISC-V virtualization in a nutshell
  11. 11. Hypervisor Extension 11 ESRGv3  Designed for type-1 and type-2 hypervisors  Additional execution modes  HS-mode (hypervisor-extended supervisor)  VS-mode & VU-mode  Additional CSR registers:  HS-mode CSRs for hypervisor capabilities (e.g., hstatus)  HS-mode CSRs for accessing Guest/VM state (e.g., vsstatus)  Two-Stage MMU:  Stage 1: VS-mode page table (GVA -> GPA)  Stage 2: HS-mode guest page table (GPA -> HPA) ESRGv3 RISC-V Summit 20 https://github.com/riscv/riscv-isa-manual/releases/latest
  12. 12. Hypervisor Extension 12 ESRGv3 ESRGv3 RISC-V Summit 20 Hypervisor Firmware (SBI) Decreasing Privilege level M Guest OS Guest User Space Host User Space Virtualised Environment Non-virtualised Environment HS VS Adapted from: Alistair Francis (WD), “Developing the RISC-V Hypervisor Extensions in QEMU”, Embedded Linux Conference Europe, 2019 VU M HS U
  13. 13. Spec Status 13 ESRGv3 ESRGv3 RISC-V Summit 20  Currently version 0.6.1  Feedback from open source projects  Contributions from organizations and individuals  H-Extension spec close to freeze state  Hypervisor “group” (join RISC-V Hypervisor sync-up calls!)  Function completeness (KVM & Xvisor)  RTL implementations (we have contributed with one )  KVM and Xvisor running on FPGA implementation (we have ran Xvisor on an FPGA )  Open source projects support:  QEMU (v0.6.1)  KVM and Xvisor (v0.6.1)
  14. 14. RISC-V Hypervisors 14 ESRGv3  KVM:  Type-2 hypervisor, Linux-based architecture, open-source  Enterprise virtualization setups (datacenters and private clouds)  https://github.com/kvm-riscv/linux  Xvisor:  Type-1 hypervisor, monolithic architecture, open-source  Embedded systems with soft real-time requirements  https://github.com/avpatel/xvisor-next  Bao:  Type-1 hypervisor, static partitioning architecture, open-source  Mixed-criticality systems with strong real-time and security requirements  https://github.com/bao-project/bao-hypervisor ESRGv3 RISC-V Summit 20
  15. 15. RISC-V H-Extension Implementation & enhancements
  16. 16. Spec Checklist 16 ESRGv3 01 02 03  H-Extension, Version 0.6.1  RV64 and sv39 only  Implemented:  h- and vs- csrs and respective functionality  m- registers extended accordingly  hypervisor load/store instructions  new virtual instruction exception  hfence instructions (limited, flushes complete TLB)  guest external interrupts (w/ PLIC virtualization extension)  Not Implemented:  ASID/VMID support  Transformed Instruction or Pseudoinstruction for htinst/mtinst ESRGv3 RISC-V Summit 20
  17. 17. H-Extension Implementation 17 ESRGv3 01 02 03  Available at: https://github.com/josecm/rocket-chip/tree/hyp  Synced with chipyard’s master branch commit of rocket-chip (@1872f5d, 6 months ago)  ~1100 sLOC modifications to rocket-chip. Mainly on:  CSR  PTW  TLB ESRGv3 RISC-V Summit 20
  18. 18. H-Extension Implementation 18 ESRGv3 01 02 03  Next Steps:  Refactoring/Optimizations/Clean-up  ASID/VMID support  Increase hfence granularity  Improve 2nd-stage translation data structures (e.g. dedicated guest physical address TLB)  Software:  Ad hoc testing  Bao Hypervisor  Xvisor ESRGv3 RISC-V Summit 20
  19. 19. PLIC H-Extension 19 ESRGv3 01 02 03  Main requirements:  minimal design  direct injection of physical interrupts for the active guest  no traps on claim/complete  mix of physical and virtual interrupts  Extra GEILEN VS-contexts per physical hart  Hypervisor must trap-and-emulate VS-context priority and enable registers.  Hypervisor gives guests direct access to VS- context’s claim/complete registers.  Virtual interrupts are injected by writing to virtual interrupt injection registers (VIIR) which can be grouped. ESRGv3 RISC-V Summit 20 02 PLIC RISC-V Hart 0 RISC-V Hart 1 RISC-V Hart n M-mode External Interrupt S-mode External Interrupt VS-mode External Interrupt 0 VS-mode GEILEN-1 * The PLIC itself might be deprecated as a new standard interrupt controller is in the workings. https://github.com/josecm/riscv-plic-spec/tree/virt
  20. 20. Cache and Bandwidth Partitioning 20 ESRGv3  Software-based techniques:  Increases TCB  High Overheads  Coarse-grained  E.g.: page coloring, PMU event based throttling  Cache Partitioning:  Per bus-master way-locking  Lightweight modification to eviction circuit  Memory bandwidth throttling:  Bandwidth regulation unit (BRU)  Per partition memory bandwidth budgets ESRGv3 RISC-V Summit 20 Adapted from: "BRU: Bandwidth Regulation Unit for Real- Time Multicore Processors,". F. Farshchi, Q. Huang and H. Yun. RTAS 2020
  21. 21. Evaluation Tests, experiments, and results
  22. 22. Experimental Setup 22 ESRGv3 Hardware ▪ Hexa-core rocket ▪ 3.2GHz ▪ 16 KB iL1$ and dL1$ ▪ 8-way, 512 KB unified L2$ w/ way-locking ▪ Plic Virt extensions ▪ BRU System Configurations ▪ Bare-metal / no hypervisor (bare) ▪ Single guest (solo) ▪ Interference (interf) ▪ Partitioning (solo+part) ▪ Partitioning w/ Interference (interf+part) Metrics ▪ Performance Overhead ▪ Interrupt Latency RISC-V Summit 20 Software Stack ▪ OpenSBI ▪ Bao Hypervisor ▪ Linux OS (VM) ▪ Baremetal App (VMs) Memory Resource Partitioning ▪ 4 L2$ ways ▪ 1800 MiB/s
  23. 23. Performance Overhead 23 ESRGv3 ESRGv3 RISC-V Summit 20 ▪ MiBench Benchmark Suite ▪ Automotive subset ▪ Relative to bare ▪ Bare absolute values ▪ Higher is worse ▪ Negligible variance Hosted execution brings non-negligible overheads (2 to 6%) . We believe this is mainly due to high cost of tick timer handling and non-optimized two-stage translation Memory subsystem interference has performance impacts from 5% to 70% Cache and memory bandwidth partition mitigations significantly reduce interference (1 to 23%) but not completely
  24. 24. Interrupt Latency 24 ESRGv3 ESRGv3 RISC-V Summit 20 ▪ Custom benchmark ▪ Auto-Reload timer at 100 Hz ▪ Absolute time in ns ▪ L1 Instruction cache invalidated at each tick Trap-and-emulate approach introduces an extra 650 ns (≈800%) in guest average interrupt latency Interference increases latency but can be somewhat attenuated by memory resource partitioning Direct interrupt injection keeps native latency and is not very susceptible to the effects of interference
  25. 25. Conclusion Final thoughts and Takeaways
  26. 26. We have presented the first public implementation valuation of the hypervisor extensions in a RISC-V
  27. 27. 27 ESRGv3  No "real" silicon with H-extension yet  probably not for the foreseeable future  Static Partitioning “Virtualization”  M-mode = hypervisor  PMP configuration per hart  PLIC emulation  IOPMP  Bao porting  WiP version @ PolarFire Icicle (Renode)  Virtualization for RISC-V “as of today”  Extended platform support ESRGv3 RISC-V Summit 20 But… do we really need it ?!
  28. 28. TAKEAWAYS 28 • Current state of the RISC-V hypervisor extension specification • First implementation of the hypervisor extensions in a RISC-V core • Static partitioning virtualization and what hardware supports it needs The spec is close to freeze but we encourage the community to contribute 1 There is a need for additional implementations in other RISC-V cores 2 There is a need to define standard extensions that specify memory resource partition interfaces and IOMMUs 3 ESRGv3 RISC-V Summit
  29. 29. THANK YOU! jose.martins@dei.uminho.pt LinkedIn - https://www.linkedin.com/in/josecmar/ Twitter - https://twitter.com/josecarmartins Github - https://github.com/josecm sandro.pinto@dei.uminho.pt LinkedIn - https://www.linkedin.com/in/sandro2pinto Twitter - https://twitter.com/sandro2pinto Github - https://github.com/sandro2pinto/
  30. 30. Q&A https://github.com/bao-project/bao-hypervisor https://github.com/josecm/rocket-chip/tree/hyp

×