In cloud computing environments, VMs require fast access to resources like storage and networking. The hardware that the VMs access is implemented in software and/or by passing through a dedicated hardware device. Software-based solutions consume extra CPU cycles, thus resulting in poor performance. Also, these require to expose a device-model to the guest, thus increasing the attack surface. Conversely, hardware passthrough provides better performance and security but can be expensive in terms of the number of physical resources, since each device is dedicated to a single VM. This talk focuses on how Vates is working on sharing hardware resources among VMs by relying on dedicated processors named Data Processing Units (DPU). More precisely, Vates work on offloading Xen hypervisor of storage emulation by relying on Kalray K200 DPU PCIe controllers, a hardware accelerator based on MPPA architecture. ----------------------------------------- The CloudStack Collaboration Conference 2023 took place on 23-24th November. The conference, arranged by a group of volunteers from the Apache CloudStack Community, took place in the voco hotel, in Porte de Clichy, Paris. It hosted over 350 attendees, with 47 speakers holding technical talks, user stories, new features and integrations presentations and more.

1. Enabling DPU HW in XCP-ng
cloud platform environment

2. Agenda
I. XCP-ng cloud platform
II. I/O devices virtualization
III. Kalray K-200 DPU accelerator
IV. Missing bits K-200  XCP-ng
V. Next
2

3. XCP-ng cloud platform
➢ XEN Hypervisor based solution
➢ Type I (bare metal) Hypervisor
➢ Pionners of ParaVirtualization
➢ CPU/Memory Efficient Virtualization
➢ Enhanced Security (Reduced TCB,…)
➢ Dom0 – (resources dispatch, VM
mangement
➢ Many ways of I/O Virtualization
3
Hardware
Xen Hypervisor
Dom0
VM
DomU
VM
DomU
VM
DomU
VM
DomU
VM
DomU
VM

4. I/O devices virtualization
I/O Device Software Emulation :
Trapping accesses to and emulating HW
controller registers (MMIO or IO ports)
Pros & Cons:
+ use of native driver in VM
- poor performances
- complex (i.e. error prone)
- difficult to implement in Hypervisor
- bottleneck effect
4
Ex: ATA interface (PIO mode)
~200 ATA commands

5. I/O devices virtualization
I/O Device Sharing :
The HW controller context is saved and
restored on some event (scheduling the
VM to run,…)
Pros & Cons:
+ use of native driver in VM
- performances issues
- not appliable to complex devices
- security issues (side-channels,….)
5
DomU#1
VM
DomU#2
VM
DomU#3
VM
framebuffer

6. I/O devices virtualization
I/O Device Paravirtualizing :
The higher abstraction of device is « offered »
to the VM (client/server model)
Pros & Cons:
+ simple implementation
+ correct performances
- need specific driver in the VM
- bottleneck effect
6
DomU VM
Dom0 VM
Back-end
Driver
Front-end
Driver
Xen Hypervisor
HW
ctrl
r/w block
Ex: Virtual Block Device

7. I/O devices virtualization
I/O Device Passthrough :
Granting the device access to the VM
Pros & Cons:
+ using of native drivers
+ excellent performances
+ full fonctoinnalities
+ no additional code (no that much)
- costly solution
- difficult to scale
7
DomU VM
NVMe
driver
Xen Hypervisor
Ex: NVMe Device
NVMe Ctrl

8. 8
IDEAL WORLD
➢ provide to the VM an access to HW device (passthrough)
➢ the SW flexibility to specify the device capacities
➢ the solutions that scales in Cloud Environnement
Flexibles HW accelerators: SR-IOV, DPU

9. Kalray K-200 DPU
K200 :
➢ virtual NVMe controllers based on the top of real
NVMe resources
➢ MPPA VLIW proprietary architecture
➢ PCIe P2P transaction (totally off- loading CPU)
➢ IOMMU to « accomodate » 2 distinct address spaces
➢ higly configurable vNVMe devices (size, raid, …)
➢ user mode driver (VFIO based)
➢ SPDK based toostack to manage virtual NVMe
9
NVMe#1 NVMe#2 NVMe#2
vNVMe#1 …. vNVMe#n

10. Missing bits to enable K200 integration
10
K200
Driver
NVMe
driver
vNVMe
K200
DomU
Dom0
VFIO
framework
IOMMU
framework
PV IOMMU
driver
Xen Hyp
IOMMU framework
XAPI
toolstack
Major steps & developpements:
➢ XEN IOMMU framework modifications
to export IOMMU acces to guest
➢ Linux kernel PV IOMMU driver
➢ XAPI toolstack to issuing SPDK
commands
➢ Tweaking in DomU
IOMMU

11. 11
What is Next?
NVMe oF

12. 12
THANKS!
Any questions?
You can find me at andrei.semenov@vates.tech