Download as PDF, PPTX
![ACRN Physical Interrupt Handler
Enable Intr
VM Exit
Other Exit
Handler
Disable Intr
External Intr Exit
Handler
Enable Intr
Handle pending
req
VM Entry
Dispatch Interrupt
irq_desc[vector_2_irq(vec)]
Spurious
Handler
LAPIC ACK EOI
Action Handler
Edge
Handler
LAPIC ACK EOI
Action Handler
Level
Handler
LAPIC ACK EOI
Action Handler
IOAPIC Mask Intr
• Unregistered Vector • VCPU Notification IPI
• Local Timer
• PTDev MSI
• Non-PTDev INTx
1
1
2
1 Physical interrupt could come from VMX root mode after enabled interrupt
2 Physical interrupt could come from VMX non-root mode which trigger a vm exit
Dev Level
Handler
LAPIC ACK EOI
Action Handler
IOAPIC Mask Intr
• PTDev INTx (IOAPIC
Unmask Intr when
vIOAPIC/vPIC get
ACK EOI)
IOAPIC unMask
Intr
Do Softirq
Note:
action for PTDev leads to
virtual interrupt injection](https://image.slidesharecdn.com/projectacrnhypervisorintroduction-200605185201/75/Project-ACRN-hypervisor-introduction-16-2048.jpg)
Uploaded byProject ACRN
Project ACRN hypervisor introduction
The document discusses ACRN, an open-source lightweight hypervisor intended for consolidating heterogeneous workloads and streamlining IoT edge development. It provides an overview of ACRN's architecture and key modules, including boot process, CPU virtualization, memory management, interrupt handling, pass-through devices, and device model for handling I/O requests. The document also outlines enhancements in ACRN 2.0, such as supporting new operating systems and safety/real-time virtual machines.
More Related Content
![[Container Runtime Meetup] runc & User Namespaces](https://cdn.slidesharecdn.com/ss_thumbnails/publicruncusernamespacescontainerruntimemeetup1-190924085316-thumbnail.jpg?width=640&height=640&fit=bounds)
![[Paris Container Day 2021] nerdctl: yet another Docker & Docker Compose imple...](https://cdn.slidesharecdn.com/ss_thumbnails/paris-210602095121-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Project ACRN hypervisor introduction
Project ACRN hypervisor introduction
- 1. ACRN: A BigLittle Hypervisor for IoT Development CHEN, Jason Jason.cj.chen@intel.com 3/11/2020
- 2. Agenda ❑ Overview - ACRNIntroduction - ACRN Scenario - ACRN Architecture ❑ Modules - ACRN Boot - ACRN CPU - ACRN Memory - ACRN Interrupt - ACRN PTDev - ACRN DM – IO Request
- 3. Introduction ACRN™ is aflexible, open-source, lightweight hypervisor - intended to enable consolidation of heterogeneous workloads, and to streamline IoT edge development. - A Linux Foundation Project - Launched in March 2018 - Version 1.0 released in May 2019
- 4. Ready for Production •Released in May 2019 Key Features • Safety and Security Isolation (Cluster + IVI) • Extensive Sharing Capabilities • Graphics, media, USB, audio, camera etc. • Advanced DMA/graphics buffer sharing • Multiple OS Support • Clear Linux, Yocto, Ubuntu • Android, AGL, AliOS • MISRA-C Compliance ACRN hypervisor Device Model Service VM User VMs Trusty ACRN 1.0 Scenario
- 5. Trusty World ACRN 1.0 Architecture Subtextgoes here Service VM (PIT, PCI, ACPI ..) Hypercalls VT-d EPT ACRN Hypervisor VMX Trusty API vPIC/vLAPIC/ vIOAPIC/vMSI ACRN Device Model (Mediators) VM Manager Linux VM virtio FE Drivers User Kernel User Kernel VM API SOC Platform (Apollo Lake etc.) Firmware (UEFI, SlimBoot etc.) CSE Virtual Firmware IVI App Normal World virtio FE Drivers User Kernel Virtual Firmware VMX non-root operation VMX root operation Android VM Encrypt/ signature Native Device Driver Native Device Driver Kernel Mediators
- 6. Hybrid-Mode • Partition +Shared VM Type • Safety VM • Real-time VM Kata Container More OS Support: • Zephyr, VxWorks, RT-Linux, Windows FUSA Certification ACRN hypervisor Device Model Safety App Service VM User VMsSafety VM (RTVM) ACRN 2.0 Scenario
- 7. ACRN 2.0 Architecture ` ServiceVM ACRN Device Model VMX VT-d EPT ACRN OSPM vPIC/vLAPIC/ vIOAPIC/vMSI ACRN DM (Device Model) Hypercalls VM Manager Native Device Drivers User VM1 (Post-Launched) virtio FE Drivers Virtual Firmware (OVMF) Emulated Device Drivers User VM0 (Pre-Launched) Passthru Device Drivers Passthru Device Drivers vPCI/vBridge/vUART/vRTC/vWDT… RT VM (Post-Launched) Virtual Firmware (OVMF) HV Emulated Device Drivers ACRN Service Driver CPU Scheduler virtio FE Drivers(PMD) Passthru Device Drivers
- 8.
- 9. Agenda ❑ Overview - ACRNIntroduction - ACRN Scenario - ACRN Architecture ❑ Modules - ACRN Boot - ACRN CPU - ACRN Memory - ACRN Interrupt - ACRN PTDev - ACRN DM – IO Request
- 10. ACRN Boot • SupportSBL & UEFI Firmware boot Firmware (SBL or UEFI) Multiboot Interface mmap info acrn.out sos image module ACRN Hypervisor • Multiboot1/2 interface • Hypervisor boot from BSP • Hypervisor launch SOS directly • Hypervisor launch UOS through SOS-DM hypercall
- 11. ACRN CPU Virtualization VM0 Hypervisor CPUScheduler VCPU0 VCPU1 VCPU0-VM0 vCR vMSR vCPUID VCPU1-VM0 vCR vMSR vCPUID VCPU0-VM1 vCR vMSR vCPUID VCPU1-VM1 vCR vMSR vCPUID VM1 VCPU0 VCPU1 Hardware PCPU0 PCPU1 • Dedicated VCPU instances for each VM in hypervisor • Emulation for • CPUID • Control Register • MSR • VM-Exit • CPUID Instruction • Control Register Access • RDMSR/WRMSR Instruction
- 12. ACRN Memory (Concept) •GVA to GPA • MMU + Page Table in Guest • GPA to HPA • EPT + Page Table in Hypervisor (per VM) • HPA to HVA • MMU + Page Table in Hypervisor • VM-Exit – EPT Violation • no page mapping in EPT page tables • for MMIO emulation Hypervisor VM1 VCPU1 VCPU2 MMU GVA GPA Page Table GVA Page Table GVA Page Table GVA Page Table GVA Page Table GVA Page Table VM2 VCPU1 VCPU2 MMU GVA GPA Page Table GVA Page Table GVA Page Table GVA Page Table GVA Page Table GVA Page Table EPT EPT Page Table Page Table HPAHPA HVA MMU Page Table
- 13. ACRN Memory Layout •Identical mapping for HPA <-> HVA • Identical mapping for SOS GPA <-> HPA • SOS DM allocate UOS memory based on Hugetlb
- 14. ACRN Interrupt Controller •In hypervisor(native) layer, ACRN configures physical PIC/IOAPIC/LAPIC to support different interrupt source from local timer/IPI to external INTx/MSI. • In guest layer, ACRN may emulates virtual PIC, virtual IOAPIC and virtual LAPIC, and provides full APIs which allow the virtual interrupt injection from emulated or pass-thru devices. • In guest layer, ACRN may choose pass-thru LAPIC, which pass-thru almost all LAPIC registers except ICR/XAPICID/LDR, and guest will run under x2apic mode w/o vPIC & vIOAPIC. • VM-Exit - PIO Access : VPIC (0x20/0x21/0xA0/0xA1…) - EPT Violation: VIOAPIC/VLAPIC - APIC Access/Write: APICV Hypervisor VM0 VCPU0 VLAPIC0 VCPU1 VLAPIC1 VIOAPIC PCPU0 LAPIC0 PCPU1 LAPIC1 VM1 VCPU2 VLAPIC0 VCPU0 LAPIC-PT PCPU2 LAPIC2 PCPU3 LAPIC3 IOAPIC VPIC PIC Hardware Local Timer INTx Devices MSI DevicesIPI VPIC VIOAPIC Note: under LAPIC PT mode, hypervisor will not handle native interrupt, and only MSI interrupt is supported for this guest and such MSI interrupt will be injected into guest directly
- 15. ACRN Interrupt Injection •Source - Emulated device - Pass-thru device - Others (timer/IPI) – not show here • Type - INTx – through PIC/IOAPIC -> LAPIC - MSI – through LAPIC - IPI/LVT – through LAPIC • Physical Interrupt Handler - do irq for physical interrupt - physical interrupt may happen in hypervisor(vmx root mode) or guest (vmx non-root mode -> vm-exit) • Virtual Interrupt Handler - virtual interrupt injection to guest - request may come from DM(hypercall) or PTDev(physical interrupt handler) Hardware Hypervisor Service VM PCPU0 LAPIC0 PCPU1 LAPIC1 User VM PCPU2 LAPIC2 IOAPIC Pass-thru Dev w/ INTx DM Emulated Device (INTx or MSI) VCPU0 VLAPIC0 VIOAPIC VCPU1 VLAPIC1 Pass-thru Dev w/ MSI Physical Interrupt Handler Virtual Interrupt Handler
- 16. ACRN Physical InterruptHandler Enable Intr VM Exit Other Exit Handler Disable Intr External Intr Exit Handler Enable Intr Handle pending req VM Entry Dispatch Interrupt irq_desc[vector_2_irq(vec)] Spurious Handler LAPIC ACK EOI Action Handler Edge Handler LAPIC ACK EOI Action Handler Level Handler LAPIC ACK EOI Action Handler IOAPIC Mask Intr • Unregistered Vector • VCPU Notification IPI • Local Timer • PTDev MSI • Non-PTDev INTx 1 1 2 1 Physical interrupt could come from VMX root mode after enabled interrupt 2 Physical interrupt could come from VMX non-root mode which trigger a vm exit Dev Level Handler LAPIC ACK EOI Action Handler IOAPIC Mask Intr • PTDev INTx (IOAPIC Unmask Intr when vIOAPIC/vPIC get ACK EOI) IOAPIC unMask Intr Do Softirq Note: action for PTDev leads to virtual interrupt injection
- 17. ACRN Virtual Interrupt/ExceptionInjection ** it’s part of acrn_handle_pending_request High priority exception exist? Inject hi exception through VMX_INT_INFO Interrupt Window Check Interrupt Pending from vPIC or vLAPIC (basic APICv)? NMI exist? Previous Pending IDT Intr/ Excep? Guest Irq Enabled? Inject NMI through VMX_INT_INFO Inject VMX IDT Pending Intr/Excep through VMX_INT_INFO Low priority Exception exist? Y Enable Interrupt Window Y VM Exit N N Y N Y N Y N Inject lo exception through VMX_INT_INFO Y N Pending extint from vPIC? Inject extint through VMX_INT_INFO Y Pending interrupt from vLAPIC? N N Inject lapic pending interrupt through VMX_INT_INFO Guest irq enabled && !irq_injected Basic APICv Y N Inject lapic pending interrupt through VID PIR Advanced APICv Inject virtual interrupt from vPIC/vIOAPIC/vLAPIC - acrn_inject_pending_intr VM Entry • Request for virtual interrupt injection come from virtual interrupt controller - vPIC, vIOAPIC, vLAPIC • Request for virtual exception injection come from different modules who need emulate an exception – instruction emulation, CR/MSR emulation … • The injection be done based on priority
- 18. ACRN Pass-thru Device •ACRN support PCI-based pass-thru devices • With the help of EPT, no VM- Exit for MMIO access • Necessary VM-Exit for PCI config space access and interrupt injection • ACRN support PT IRQ mapping mechanism for pass-thru devices Hardware Hypervisor VM Pass-thru Dev vPCI EPT PCI Config Space MMIO Interrupt PT IRQ Pass-thru Dev PCI Config Space MMIO Interrupt Main Purpose – try best to avoid VM-Exit!!!
- 19. ACRN Pass-thru IRQ •PTDev - a PTDev may trigger interrupt from INTx or MSI/MSIX - exclusively assigned to different VMs • PTIRQ Entry: - maintain PTDev interrupt source (INTx & MSI) mapping between native and virtual - exclusively assigned to different VMs associated with PTDev • Physical Interrupt Management: - allocate physical vector in hypervisor when PTDev assign PTIRQ entry - physical IOAPIC RTE & MSI fields can be update according to virtual configuration (except vector field) - physical interrupt handler will look up PTIRQ entries to find virtual dest then do virtual interrupt injection after receive external interrupt from PTDev Hypervisor(Native) VM0 IOAPIC vPIC PT PCI Dev vBDF = (a, b, c) PT PCI Dev BDF = (A, B, C) 1 MSI, 1 INTx VM1 vIOAPIC PT PCI Dev vBDF = (d, e, f) PT PCI Dev BDF = (D, E, F) 2 MSIX, 1 INTx ptirq entry = { type = intx, vm = vm0, int_src = pic, virt_pin = a, phy_pin = A, } ptirq entry = { type = intx, vm = vm1, int_src = iopic, virt_pin = c, phy_pin = C, } ptirq entry = { type = msix, vm = vm1, virt_bdf = def, virt_idx = x, phy_bdf = DEF, phy_idx = X,} INTx pins (PIC/IOAPIC) MSI/MSIX entry vecPhy RTE: Vector is fixed after allocated Others fields could be changed according to vRTE changes Physical Interrupt Management Phy MSI addr: Vector is fixed after allocated Others fields could be changed according to vMSI changes vecPhy MSI data: Not exist for vPT- LAPIC case External interrupt from PTDev A C a c x X
- 20. SOS Device Model ACRN HV UOS Virtual Dev HVDevice Model ACRN Device Model - IO Request • IO Trap - hypervisor trap out (VM-Exit) virtual device’s PIO/MMIO access from a guest (UOS) • IO Request: - hypervisor figure out the target address and target size for the PIO/MMIO access , to form a IO request - instruction decode needed for a MMIO access - the IO request will dispatch to specific device model based on target address • IO Emulation - a virtual device provide IO emulation for its PIO or MMIO based on the IO request - the virtual device may be emulated by different device models from HV DM, SOS kernel BE or SOS user space DM
- 21.