XPDDS17: Keynote: Towards a Configurable and Slimmer x86 Hypervisor - Wei Liu, Citrix

•

0 likes•726 views

The classic PV interface has given Xen a head start in the cloud computing era. However, with the advance of hardware technology, its usage has be declining significantly. The complexity of PV ABI has been a source of security bugs and friction between Xen and other communities. This talk tries to lay out a plan to make x86 Xen configurable to support different guests types, and then move the PV ABI to a PVH container so that it is possible to support PV workload in a more secure manner.

Technology

Towards a conﬁgurable and slimmer x86
hypervisor
Liu Wei
Budapest – July 11-13, 2017

Current state of aﬀairs
PV mode: no hardware extension needed, used in legacy
systems, useful in certain cases like running unikernel and
nested-virt without vVMX or vSVM
HVM mode: needs hardware support and QEMU for
emulation, has become the mainstream Xen VM mode
PVH mode: essentially HVM without QEMU, under
development
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 2 / 19

Two (big) projects
Splitting PV and HVM code: Refactor x86 hypervisor code.
Make guest supporting code conﬁgurable via Kconﬁg
PV ABI in PVH container: Implement a PV ABI shim. Use it
to translate PV hypercalls into PVH ones when necessary
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 3 / 19

Why splitting PV and HVM code?
Users can pick and choose the guest interfaces
Smaller binary, smaller attack surface
Reclaim precious address space if PV is disabled, to let Xen
support >16TB host memory more easily
Improve x86 hypervisor code base
*NOT* intending to kill PV in the hypervisor
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 4 / 19

Xen x86 PV memory layout
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 5 / 19

The conceptual map of code
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 6 / 19

The reality
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 7 / 19

Current code
do foo ( . . . )
{
/∗ . . . ∗/
i f (hvm) {
do foo hvm ( ) ;
return ;
}
/∗ l o t s of code to do foo f o r pv ∗/
return ;
}
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 8 / 19

Current code
do bar ( . . . )
{
/∗ . . . ∗/
i f (hvm) { /∗ Some code ∗/ };
i f ( pv ) { /∗ Some code ∗/ };
/∗ l o t s of code f o r common case ∗/
i f (hvm) { /∗ Some code ∗/ };
i f ( pv ) { /∗ Some code ∗/ };
return ;
}
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 9 / 19

Future code
do baz ( . . . )
{
/∗ code f o r common case ∗/
i f (hvm)
do baz hvm ( ) ;
i f ( pv )
do baz pv ( ) ;
return ;
}
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 10 / 19

Game plan for splitting PV and HVM code
Identify all the components that need refactoring
Dom0 builder
Domain handling code
Trap handling code
Memory management code
Guest memory accessor
...
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 11 / 19

Game plan for splitting PV and HVM code
Coarse-grained refactoring – mostly for PV code
Move code around
Split code into manageable trunks
Do some basic cleanups:
Use better function names
Use better coding style
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 12 / 19

Game plan for splitting PV and HVM code
Fine-grained refactoring – for both PV and HVM code
Abstract out a set of guest interfaces
Adjust internal interfaces between components if necessary
Fix x86 common code
Make PV and HVM conﬁgurable
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 13 / 19

Why PV ABI in PVH container?
Continue to support PV in a more secure manner
Have more than 128GB worth of 32bit PV guests
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 14 / 19

PV ABI in PVH container
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 15 / 19

Game plan for PV ABI in PVH container
Build the PV shim – essentially a stripped-down Xen
hypervisor
Go through all PV hypercall handlers, categorize them into the
aforementeioned groups
Further refactor PV guest supporting code: provide the ”real
PV” handlers and ”PV shim” handlers while sharing as much
code as possible
Change the build system to pull in the right objects
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 16 / 19

Game plan for PV ABI in PVH container
Adjust Xen toolstack
Construct a PVH guest while using the PV shim as ”ﬁrmware”
Further improvements (open questions at the moment)
Provide mechanism to parse guest kernel inside the container,
something like pvgrub
Provide mechanism to pass-through PCI devices (if that’s still
relevant)
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 17 / 19

Current status
Doing coarse-grained refactoring:
Dom0 builder (done)
Domain handling code (done)
Trap handling code (done)
Memory management code (doing)
Guest memory accessor
...
ETA: Some point in the future :)
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 18 / 19

Q&A
Budapest – July 11-13, 2017 Towards a conﬁgurable and slimmer x86 hypervisor 19 / 19

This talk will start with an overview of the x86 PVH Dom0 architecture together with some basic information about it, in order for the audience to understand why does Xen need PVH, and even more, why does Xen need a PVH Dom0 at all. Then it will dive into the hypervisor side implementation, in order to understand how is PVH Dom0 implemented, and in which way it interacts with the existing Xen internal interfaces and subsystems. Details about the current implementation status will also be provided, together with a roadmap of the missing bits. Hopefully after the talk the audience should have a good understanding about what's this new PVH mode, and how is it implemented inside of Xen.

XPDDS17: Xen-lite for ARM: Adapting Xen for a Samsung Exynos MicroServer with...

The Linux Foundation

This document summarizes a presentation on adapting Xen for multi-tenant virtualization on ARM-based embedded devices with FPGA acceleration. It discusses moving PV drivers and I/O handling into the hypervisor to reduce overhead. Performance tests show MicroVisor reduces boot times and I/O latency compared to stock Xen. Integrating FPGA acceleration for networking and storage aims to improve performance for network function virtualization workloads on low-power edge devices.

XPDDS17: Bring up PCI Passthrough on ARM - Julien Grall, ARM

The Linux Foundation

Device passthrough allows the user to give control of physical devices (NIC, graphic card, etc) to a virtual machine, giving it full and direct access to the device. This has several potential uses, including avoiding overhead of the device para-virtualization and being able to run unmodified guest on Xen on ARM. Whilst PCI passthrough is already something well established on Xen, ARM support will require some fundamental changes due to architectural differences. For instance, the MSI doorbell will be translated by the IOMMU. During this session, we will look at the state of PCI passthrough on x86. Then, we will describe the interactions of the components (PCI Root controller, interrupt controller and IOMMU) required for the use of PCI on an ARM system. With this background, we will round out the discussion with the proposed approach for implementing PCI passthrough on ARM.

XPDDS17: PL011 UART Emulation in Xen on ARM - Bhupinder Thakur, Qualcomm Data...

The Linux Foundation

Linaro has published VM System specification for ARM Processors, which provides a set of guidelines for both guest OS and hypervisor implementations, such that building OS images according to these guidelines guarantees that those images can also run on hypervisors compliant with this specification. One of the spec requirements is that the hypervisor must provide an emulated PL011 UART as a serial console which meets the minimum requirements in SBSA UART as defined in ARM Server Base Architecture Document. In this presentation, Bhupinder Thakur will cover the motivation behind VM System spec requirements and the work done for adding support of PL011 emulation in Xen.

XPDDS17: Xen Test Lab: The Installation and Our Plans - Ian Jackson, Citrix

The Linux Foundation

The Xen Project uses a bespoke Continuous Integration system, osstest. This system has a number of unique architectural features that make it flexible and powerful. In this talk, I'll take you through some of these, such as: the distributed job scheduler; the standalone vs. infrastructure abstraction; and some of the more advanced command-line interfaces useful in infrastructure installations. Transcript: See https://schd.ws/hosted_files/xendeveloperanddesignsummit2017/2a/talk.txt

XPDDS17: How to Abstract Hardware Acceleration Device in Cloud Environment - ...

The Linux Foundation

Intel® QuickAssist Technology (QAT) offers acceleration for the compute-intensive workloads of cryptography and compression. It supports Single Root I/O Virtualization (SR-IOV), which allows a single physical device to be shared by multiple guests. To better support fair sharing of capacity in a multi-tenant environment, Intel supports the concept of service level agreements. The service level is expressed using the abstraction of “acceleration units”. In this talk we will explain why we chose to define such an abstraction, and why specifying the capacity using raw throughput or operation rate alone is insufficient for accelerators – in brief, because the capacity is so heavily dependent on factors such as algorithm, direction (encrypt/sign/compress vs. decrypt/verify/decompress), key size, request size, compression level, etc. We go on to describe how such SLAs can be used to ensure that guests can be guaranteed some minimum acceleration capacity, and/or limited to some maximum. Finally, we describe use cases where this might be useful, such as when offering “acceleration as a service” in a cloud or Network Functions Virtualization (NFV) environment.

Storage based snapshots for KVM VMs in CloudStack

ShapeBlue

This document discusses storage-based snapshots for KVM VMs in CloudStack. It describes how CloudStack can leverage underlying storage providers' capabilities to take consistent snapshots of VMs and their disks while the VMs are running, regardless of disk format. This is done by freezing the VM, taking asynchronous snapshots of its disks, then unfreezing the VM. This provides faster, more space-efficient snapshots compared to CloudStack's default VM snapshot implementation for qcow disks. The technique works with various storage providers like NFS, Ceph, and StorPool.

Quickly Debug VM Failures in OpenStack

LinuxCon ContainerCon CloudOpen China

This document discusses how to quickly debug VM failures in OpenStack. It begins by explaining where VMs run in OpenStack and how VM startup is implemented. It then describes how to use various logs from OpenStack components like Nova, Neutron, and libvirt/Qemu to troubleshoot issues. The document also provides examples of common VM failure types and outlines steps for debugging, including checking configuration settings, log files, and using troubleshooting tools like Virsh and ABRT. Finally, it calls for collecting VM failure scenarios and solutions to integrate into monitoring tools like StackTach.

Ceph, an open source distributed storage system, has been ported to run natively on Windows Server to provide improved performance over the iSCSI gateway and better integration into the Windows ecosystem. Key components like librbd and librados have been ported to Windows and a new WNBD kernel driver implements fast communication with Linux-based Ceph OSDs. This allows access to RBD block devices and CephFS file systems from Windows with comparable performance to Linux.

XPDS16: Keeping coherency on ARM - Julien Grall, ARM

The Linux Foundation

The ARM architecture strongly recommends to use a break-before-make when changing translation table entries whenever certain conditions are met. Failing to do so may result in getting TLB conflicts or breaking the coherency. During this session, we will introduce break-before-make and when the code handling page tables should use it. We will also discuss the modifications required in Xen to avoid breaking the coherency.

XPDDS17: Recent and Ongoing Xen Related Work in the Linux Kernel - Jürgen Gr...

The Linux Foundation

XPDS14 - OSv - A Modern Semi-POSIX LibraryOS - Glauber Costa, Cloudius Systems

The Linux Foundation

During last year, we have seen the rise of LibraryOSes in the hypervisor world. Fast, scalable and light, they bring to hypervisors the resource efficiency of Containers while maintaining many of its well known isolation and manageability advantages. While most LibraryOSes are niche in what they do, OSv is designed to run almost any POSIX compliant application and just have a slight focus in the Java application. Which applications can we run? And why? What's the story with Java and what can users gain from it? And more importantly: How can Xen users and the Xen community at large benefit from OSv? Those are some of the questions that I intend to cover in this presentation.

Ceph RBD Update - June 2021

Ceph Community

The document summarizes new features and updates in Ceph's RBD block storage component. Key points include: improved live migration support using external data sources; built-in LUKS encryption; up to 3x better small I/O performance; a new persistent write-back cache; snapshot quiesce hooks; kernel messenger v2 and replica read support; and initial RBD support on Windows. Future work planned for Quincy includes encryption-formatted clones, cache improvements, usability enhancements, and expanded ecosystem integration.

Integration of Glusterfs in to commvault simpana

Gluster.org

Integration of GlusterFS into Commvault's data platform provides a software defined storage solution that can horizontally scale out storage and compute. It uses GlusterFS to create a storage pool of commodity servers that runs Commvault's data management software. New nodes can be added easily by automating the installation of the necessary software on bare metal servers or by manually configuring existing servers. This allows the storage pool to dynamically expand its capacity and maintain data availability even if nodes fail.

2021.06. Ceph Project Update

Ceph Community

The document provides an agenda and overview of the Ceph Project Update and Ceph Month event in June 2021. Some key points: - Ceph is open source software that provides scalable, reliable distributed storage across commodity hardware. - Ceph Month will include weekly sessions on topics like RADOS, RGW, RBD, and CephFS to promote interactive discussion. - The Ceph Foundation is working on projects to improve documentation, training materials, and lab infrastructure for testing and development.

CephFS Update

Ceph Community

The document summarizes updates to CephFS in the Pacific release, including improvements to usability, performance, ecosystem integration, multi-site capabilities, and quality. Key updates include MultiFS now being stable, MDS autoscaling, cephfs-top for performance monitoring, scheduled snapshots, NFS gateway support, feature bits for compatibility checking, and improved testing coverage. Performance improvements include ephemeral pinning, capability management optimizations, and asynchronous operations. Multi-site replication between clusters is now possible with snapshot-based mirroring.

ops300 Week5 storage (1)

trayyoo

This document provides instructions and specifications for setting up a storage environment for practical testing in Proxmox. It describes setting up various virtual machines and containers with Linux and Windows operating systems along with networking tools. It also covers configuring block storage, file storage, and object storage using local disks, RBD/Ceph distributed storage, and examples of container and virtual machine networking configurations for testing interconnection goals.

Deploying CloudStack and Ceph with flexible VXLAN and BGP networking

ShapeBlue

1) The document discusses using VXLAN, BGP and EVPN to implement a layer 3 network for a cloud deployment using Ceph and CloudStack. This allows scaling beyond the limits of layer 2 networks and VLANs. 2) Key infrastructure components discussed include Dell S5232F-ON switches running Cumulus Linux, SuperMicro hypervisors and Ceph storage servers using NVMe SSDs. 3) The deployment provides high performance private and public cloud infrastructure with scalable networking and over 650TB of reliable Ceph storage per rack.

Bsdtw17: lightning talks/wip sessions

Scott Tsai

This summary covers two lightning talks from BSDTW17: 1) The first talk proposed allowing multiple encrypted GEOM providers to attach using the same passphrase, improving on having to enter the passphrase multiple times. A new rc.conf syntax was suggested. 2) The second talk discussed ongoing work to improve MP-safe networking in NetBSD, including improvements to layers 2-3, drivers, and other components like pfill and npf. Remaining work and questions were also outlined.

Proxmox ve-datasheet

Miguel Angel

Proxmox VE is a complete virtualization management solution that allows virtualization of Linux and Windows workloads. It combines KVM hypervisor and LXC containers on a single management platform with strong high-availability support through a multi-master cluster design with no single point of failure. It includes a web-based management interface to easily control virtualization functions and provides access to logs across all cluster nodes. Proxmox VE is open source software that ensures flexibility and security for enterprise virtualization deployments.

Building For Mer

David Greaves

OBS (Open Build System) is a system for managing source, binary, and package repositories. It allows developers to build packages remotely on dedicated build servers, or locally via command line tools. The Mer project uses OBS to build packages for various target architectures and distributions in a structured repository layout. Developers can submit new packages to OBS projects for autobuilding and inclusion in the Mer release cycles.

Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013

Gluster.org

This document discusses integrating the open-source GlusterFS distributed file system with QEMU virtualization and the oVirt virtualization management platform. It provides an overview of GlusterFS and how it can be used as virtual machine image storage. It then describes integrating GlusterFS with QEMU either directly or through NFS, as well as new features in oVirt 3.1 for managing GlusterFS volumes from within oVirt. Upcoming integrations between oVirt and GlusterFS are also outlined.

Docker + GCE + etcd + ray tracing

Syoyo Fujita

This document discusses using Docker, Google Compute Engine (GCE), and etcd to distribute a ray tracing workload across thousands of nodes. Docker would allow controlling resources for each renderer instance and managing versions. Etcd would enable scalably storing cluster information like node addresses. GCE is suitable due to its CPU-intensive instances and ability to quickly provision them. The proposed architecture involves render nodes pulling tasks from a Redis queue via etcd and processing scenes in parallel. Future work includes leveraging accelerators and automating the process.

Docker infiniband

Syoyo Fujita

The document discusses using Docker containers with InfiniBand networking. It shows that Docker containers can access InfiniBand directly with near-native performance by mounting the InfiniBand device into the container. This allows high-speed networking between containers without virtualization overhead. Future work proposed includes adding access controls, supporting other direct-access devices like GPUs, and building a virtualized cloud platform for high-performance rendering tasks using these techniques.

Linux based Stubdomains

The Linux Foundation

As the current stubdomain based on minios is difficult to maintain, we have worked on a stubdomain based on Linux. This helps to use QEMU upsteam in the stubdom with little change. So first I will present how a Linux based stubdomain is built and lauched, and the difficulties around it. Then, to see if this is a viable option, I will show disk and network benchmarks to compare it with a traditional QEMU in dom0 configuration. To finish, I will present the current limitations of this type of stubdomains.

Accessing gluster ufo_-_eco_willson

Gluster.org

This document provides an introduction to accessing Gluster volumes through various methods like traditional filesystem mounts, NFS/CIFS, and the Unified File and Object (UFO) feature. It discusses the benefits of using UFO to access data simultaneously via both REST API and filesystem. It also includes examples of common commands for authenticating and interacting with objects and containers when using UFO with Gluster.

2021.02 new in Ceph Pacific Dashboard

Ceph Community

Ceph Pacific is a major release of the Ceph distributed storage system scheduled for March 2021. It focuses on five key themes: usability, performance, ecosystem integration, multi-site capabilities, and quality. New features in Pacific include automated upgrades, improved dashboard functionality, snapshot-based CephFS mirroring, per-bucket replication in RGW, and expanded telemetry collection. Looking ahead, the Quincy release will focus on continued improvements in these areas such as resource-aware scheduling in cephadm and multi-site monitoring capabilities.

Fosdem 17 - Towards a HVM-like Dom0 for Xen

The Linux Foundation

Subtitle: Reducing the OS burden while taking advantage of new hardware features Xen is a hypervisor using a microkernel design that allows running multiple concurrent operating systems on the same hardware. One of the key features of Xen is that it is OS agnostic, meaning that any OS (with proper support) can be used as a host. Xen has a long history going back to the 90s when it was designed and the early 2000s when it was released. As a consequence of this, many of the assumptions and virtualization techniques backed into it are now superseeded by new hardware features, that make virtualization more transparent from an OS point of view. This talk provides an overview on the different kind of guests supported by Xen and how these new hardware features are used in order to improve and evolve them. It also describes the design and implementation of a new guest type, called PVHv2, and how it can be used as a control domain (Dom0). Also see: https://fosdem.org/2017/schedule/event/iaas_towahvm/

BSDCan 2015: How to Port BSD as a Xen on ARM Guest

The Linux Foundation

This document discusses how to port a BSD operating system to run as a guest on Xen hypervisor for ARM architectures. It covers the requirements for the DOM0 kernel, guest boot ABI specifications, need for device tree support, PV drivers, and debugging techniques. It also provides details on FreeBSD's ongoing work to support Xen as a paravirtualized ARM guest, including updated PV drivers and kernel configurations for ARM 32-bit and 64-bit architectures.

What's hot

Make room! Make room!

Denis Chapligin

Ceph on Windows

Ceph Community

XPDS16: Keeping coherency on ARM - Julien Grall, ARM

The Linux Foundation

XPDDS17: Recent and Ongoing Xen Related Work in the Linux Kernel - Jürgen Gr...

The Linux Foundation

XPDS14 - OSv - A Modern Semi-POSIX LibraryOS - Glauber Costa, Cloudius Systems

The Linux Foundation

Ceph RBD Update - June 2021

Ceph Community

Integration of Glusterfs in to commvault simpana

Gluster.org

2021.06. Ceph Project Update

Ceph Community

CephFS Update

Ceph Community

ops300 Week5 storage (1)

trayyoo

Deploying CloudStack and Ceph with flexible VXLAN and BGP networking

ShapeBlue

Bsdtw17: lightning talks/wip sessions

Scott Tsai

Proxmox ve-datasheet

Miguel Angel

Building For Mer

David Greaves

Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013

Gluster.org

Docker + GCE + etcd + ray tracing

Syoyo Fujita

Docker infiniband

Syoyo Fujita

Linux based Stubdomains

The Linux Foundation

Accessing gluster ufo_-_eco_willson

Gluster.org

2021.02 new in Ceph Pacific Dashboard

Ceph Community

What's hot (20)

Make room! Make room!

Ceph on Windows

XPDS16: Keeping coherency on ARM - Julien Grall, ARM

XPDDS17: Recent and Ongoing Xen Related Work in the Linux Kernel - Jürgen Gr...

XPDS14 - OSv - A Modern Semi-POSIX LibraryOS - Glauber Costa, Cloudius Systems

Ceph RBD Update - June 2021

Integration of Glusterfs in to commvault simpana

2021.06. Ceph Project Update

CephFS Update

ops300 Week5 storage (1)

Deploying CloudStack and Ceph with flexible VXLAN and BGP networking

Bsdtw17: lightning talks/wip sessions

Proxmox ve-datasheet

Building For Mer

Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013

Docker + GCE + etcd + ray tracing

Docker infiniband

Linux based Stubdomains

Accessing gluster ufo_-_eco_willson

2021.02 new in Ceph Pacific Dashboard

Similar to XPDDS17: Keynote: Towards a Configurable and Slimmer x86 Hypervisor - Wei Liu, Citrix

Fosdem 17 - Towards a HVM-like Dom0 for Xen

The Linux Foundation

BSDCan 2015: How to Port BSD as a Xen on ARM Guest

The Linux Foundation

Scaleable PHP Applications in Kubernetes

Robert Lemke

Kubernetes is also called the "distributed Linux of the cloud" – which implies that it provides fundamental infrastructure, which can solve a lot of challenges. Let’s see how PHP applications fit into this picture. In this presentation, we are going to explore when Kubernetes is a good fit for operating your PHP application and how it can be done in practice. We’ll look at the whole lifecycle: how to build your application, create or choose the right Docker images, deploy and scale, and how to deal with performance and monitoring. At the end you will have a good understanding about all the different stages and building blocks for running a PHP application with Kubernetes in production.

Hyper-v Storage

Paulo Freitas

The document summarizes new storage technologies in Windows Server 2012 for Hyper-V including support for virtual Fibre Channel, NPIV, live migration with Fibre Channel connectivity, MPIO, the new VHDX virtual hard disk format, and use of SMB file shares for virtual machine storage. It also discusses features like virtual Fibre Channel adapters, the benefits of MPIO, and support for virtual hard disks on 4KB native disks.

PBX on a non-specialized distro

StefanoFancello

2014 - 6 - Paving the way to a virtualized base station

Eran Bello

Network Functions Virtualization (NFV), Software Defined Networking (SDN) and Software Defined Access (SDA) are all poised to transform the traditional complex appliance based networks into software running on standard COTS infrastructure. One of the last bastions of this transformation is the wireless access, the base station. This presentation will review the opportunities and challenges related to the base station, use cases and proposed ways to overcome it, with a sneak preview to ASOCS's disruptive Virtual Base station design and approach.

CloudStack IPv6 in production

ShapeBlue

BDE_SC4_WS3_6_Luigi Selmi - Pilot SC4

BigData_Europe

The document describes a pilot project to build a scalable and fault-tolerant platform to process unlimited data sets using open source frameworks. It will use Apache Kafka as a messaging system, Apache Flink for stream and batch processing, PostgreSQL with PostGIS for road network storage and Elasticsearch for result storage. The platform will include a Docker swarm cluster to ingest and map-match real-time floating car data and classify roads by traffic level. It will also develop a short-term traffic forecasting algorithm using a feedforward neural network. The SANSA stack will enable additional use cases by linking semantic technologies and linked data.

Ceph Day Bring Ceph To Enterprise

Alex Lau

Suse Enterprise Storage 3 provides iSCSI access to connect to ceph storage remotely over TCP/IP, allowing clients to access ceph storage using the iSCSI protocol. The iSCSI target driver in SES3 provides access to RADOS block devices. This allows any iSCSI initiator to connect to SES3 over the network. SES3 also includes optimizations for iSCSI gateways like offloading operations to object storage devices to reduce locking on gateway nodes.

XPDDS19: Status of PCI Emulation in Xen - Roger Pau Monné, Citrix Systems R&D

The Linux Foundation

PCI is a local computer bus for attaching hardware devices in a computer, and is the main peripheral bus on modern x86 systems. As such, having a proper way to emulate it is crucial for Xen to be able to expose both fully emulated devices or passthrough devices to guests. This talk will focus on the current status of PCI emulation in Xen, how and where it is used, what are its main limitations and future plans to improve it in order to be more robust and modular.

Arm versus x86

JochenPolster

Ceph Day Tokyo - Bring Ceph to Enterprise

Ceph Community

iSCSI provides a standard way to access Ceph block storage remotely over TCP/IP. SUSE Enterprise Storage 3 includes an iSCSI target driver that allows any iSCSI initiator to connect to Ceph storage. This provides multiple platforms with standardized access to Ceph without needing to join the cluster. Optimizations are made in iSCSI to efficiently handle SCER operations by offloading work to OSDs. openATTIC provides a web-based interface for managing Ceph and other storage. It currently allows pool, OSD, and RBD management along with cluster monitoring. Future plans include extended pool and OSD management, CephFS and RGW integration, and deployment/configuration of Ceph nodes via Salt.

Ceph Day Taipei - Bring Ceph to Enterprise

Ceph Community

i. SUSE Enterprise Storage 3 provides iSCSI access to connect remotely to ceph storage over TCP/IP, allowing any iSCSI initiator to access the storage over a network. The iSCSI target driver sits on top of RBD (RADOS block device) to enable this access. ii. Configuring the lrbd package simplifies setting up an iSCSI gateway to Ceph. Multiple gateways can be configured for high availability using targetcli utility. iii. Optimizations have been made to the iSCSI gateway to efficiently handle certain SCSI operations like atomic compare and write by offloading work to OSDs to avoid locking on gateway nodes.

Storage based on_openstack_mariocho

Mario Cho

The document discusses software defined storage based on OpenStack. It provides background on the author's experience including medical image processing and OpenStack development. It then describes key OpenStack storage components including Cinder for block storage, Swift for object storage, and Manila for shared file systems. Cinder uses plugins to support different backend storage types and utilizes a scheduler to determine which host to provision volumes. Swift uses a ring hashing algorithm to partition and replicate data across multiple storage nodes for high scalability and availability.

Using containers and Continuous Packaging to Build native FOSSology packages

Bruno Cornec

During last LinuxCon, Bruno presented the continuous packaging approach used with a tool like project-builder.org to package upstream projects for hundreds of Linux distributions tuples in an automatic manner. Discussions happened there with the FOSSology project which wanted to benefit from this approach to produce Linux packages for their users. Both projects have since that worked jointly to make it a reality, and want to share their return of experience on this journey, benefits obtained, issues encountered and how they were fixed. After a reminder of the basics on continuous packing, the presentation will give a concrete example of what was setup using the infrastructure of the LinuxFoudation to enable the automatic creation of rpm and deb packages for FOSSology, launched during the continuous integration process already in place. A demo of the build process will also be made.

[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.js

Iñaki Baz Castillo

This document discusses mediasoup, a WebRTC SFU (selective forwarding unit) module for Node.js. It allows building multi-party video applications and exposes a JavaScript API. Mediasoup handles the media layer and routing of streams between participants without mixing or decoding/encoding streams. The document outlines different topology options like full mesh, MCU, and SFU and how mediasoup implements the SFU topology. It emphasizes that mediasoup is a minimalist Node.js module that exposes APIs for low-level and high-level WebRTC control and just handles media forwarding, not signaling. An example demo application called Firstsight is shown that uses mediasoup on the backend with React on the frontend.

Installing Component Pack 6.0.0.6

LetsConnect

This document provides instructions for installing IBM Connections Component Pack 6.0.0.6 on a Kubernetes cluster. It describes how to set up the prerequisites like Docker and Kubernetes, initialize the Kubernetes master, join worker nodes, install Helm, set up a Docker registry, create persistent volumes, and install each Component Pack microservice using Helm charts. It also covers labeling nodes, pushing images, bootstrapping, and installing monitoring dashboards.

Hyper-V 2008 R2 Best Practices

Aidan Finn

This document provides a summary of a presentation on Windows Server 2008 R2 Hyper-V best practices. It covers topics such as assessing existing infrastructure, designing virtual machines and hosts, deploying Hyper-V and VMM, integrating with OpsMgr, and backing up with DPM 2010. The presentation emphasizes the importance of requirements gathering, server sizing, storage design, and following application vendor guidance. It also promotes clustered hosting and the use of tools like MAP, VMM, and OpsMgr for management.

OpenNebulaConf2017EU: Transforming an Old Supercomputer into a Cloud Platform...

OpenNebula Project

Currently, typical supercomputers have an expected useful life of 3 or 4 years. One way of another, after this time period, infrastructure is typically replaced or upgraded to face the increasing resource demand by users and companies. This always gives rise to the same question, namely, what should be done with the old hardware if it was replaced? Possible solutions come in the form of decommissioning, splitting up and using it for spare parts, or donating it, but in several cases, the hardware can still provide value when used for different tasks. In this talk we will describe how we have converted the old Tier1 Flemish supercomputer https://www.ugent.be/hpc/en/infrastructure/tier1 into a cloud platform using OpenNebula. During this conversion process, we faced several technical challenges. The first and foremost of these was how to recycle hardware that was designed for use in a classical HPC environment to use in a private cloud. We will describe which steps were taken to address isolating VM traffic through the existing InfiniBand interconnect using the VXLAN network technology. We will also address how we managed mapping our internal (university) and external (industry) users using the OpenNebula “remote” authentication plugin. Finally, we will discuss how we used the InfiniBand interconnect to share the Ceph storage backend and VM traffic in a secure manner. After a testbed phase, in which only pilot users are given access and provide feedback, the new UGent HPC Cloud platform called “Grimer” will be available in production. YouTube: https://youtu.be/jHchktxIZnM

Ceph Day Berlin: Ceph and iSCSI in a high availability setup

Ceph Community

This document discusses setting up highly available iSCSI storage using Ceph. It describes using Linux Target (tgt) iSCSI targets connected to a Ceph cluster to provide block storage. Multiple tgt instances are run on separate hosts with a shared Ceph backend to provide high availability. Configuration of Ceph, tgt, multipathing software and iSCSI initiators is covered to access the storage over iSCSI.

Similar to XPDDS17: Keynote: Towards a Configurable and Slimmer x86 Hypervisor - Wei Liu, Citrix (20)

Fosdem 17 - Towards a HVM-like Dom0 for Xen

BSDCan 2015: How to Port BSD as a Xen on ARM Guest

Scaleable PHP Applications in Kubernetes

Hyper-v Storage

PBX on a non-specialized distro

2014 - 6 - Paving the way to a virtualized base station

CloudStack IPv6 in production

BDE_SC4_WS3_6_Luigi Selmi - Pilot SC4

Ceph Day Bring Ceph To Enterprise

XPDDS19: Status of PCI Emulation in Xen - Roger Pau Monné, Citrix Systems R&D

Arm versus x86

Ceph Day Tokyo - Bring Ceph to Enterprise

Ceph Day Taipei - Bring Ceph to Enterprise

Storage based on_openstack_mariocho

Using containers and Continuous Packaging to Build native FOSSology packages

[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.js

Installing Component Pack 6.0.0.6

Hyper-V 2008 R2 Best Practices

OpenNebulaConf2017EU: Transforming an Old Supercomputer into a Cloud Platform...

Ceph Day Berlin: Ceph and iSCSI in a high availability setup

More from The Linux Foundation

ELC2019: Static Partitioning Made Simple

The Linux Foundation

Static partitioning is used to split an embedded system into multiple domains, each of them having access only to a portion of the hardware on the SoC. It is key to enable mixed-criticality scenarios, where a critical application, often based on a small RTOS, runs alongside a larger non-critical app, typically based on Linux. The two domains cannot interfere with each other. This talk will explain how to use Xen for static partitioning. It will introduce dom0-less, a new Xen feature written for the purpose. Dom0-less allows multiple VMs to start at boot time directly from the Xen hypervisor, decreasing boot times drastically. It makes it very easy to partition the system without virtualization overhead. Dom0 becomes unnecessary. This presentation will go into details on how to setup a Xen dom0-less system. It will show configuration examples and explain device assignment. The talk will discuss its implications for latency-sensitive and safety-critical environments.

XPDDS19: How TrenchBoot is Enabling Measured Launch for Open-Source Platform ...

The Linux Foundation

Daniel Smith discusses TrenchBoot, a project aiming to establish a unified approach to harnessing boot integrity technologies across open source platforms. TrenchBoot will enable establishing hardware-rooted integrity during platform boot (first launch inspection), runtime (runtime inspection), and other states. For runtime inspection, TrenchBoot will develop a way to securely re-establish the integrity of Xen at any time without rebooting by dynamically launching an integrity kernel to inspect and verify Xen. The talk outlines the initial and future work of TrenchBoot to integrate these capabilities with Linux, Xen and other open source projects.

XPDDS19 Keynote: Xen in Automotive - Artem Mygaiev, Director, Technology Solu...

The Linux Foundation

Artem will briefly cover what has been done since the first talk on Xen in Automotive domain back in 2013, what is going on now and what is still missing for broad adaptation of Xen in vehicles. The following topics will be covered: Embedded/automotive features of Xen Collaboration with AGL and GENIVI organizations for standardization Efforts on Functional Safety compliance Artem will also go over typical automotive use scenarios for Xen which may not be the same as generic computing use of hypervisor.

XPDDS19 Keynote: Xen Project Weather Report 2019 - Lars Kurth, Director of Op...

The Linux Foundation

XPDDS19 Keynote: Unikraft Weather Report

The Linux Foundation

In recent years unikernels have shown immense performance potential (e.g., boot times of only a few ms, image sizes of only hundreds of KBs).The fundamental drawback of unikernels is that they require that applications be manually ported to the underlying minimalistic OS, needing both expert work and often considerable amount of time. The Unikraft project provides a unikernel code base and build system that significantly simplifies the building of unikernels. In addition to support for a number CPU architectures, languages and frameworks, Unikraft provides debugging and tracing features that are generally sorely missing from unikernel projects. In this talk we will talk about these features, show a set of preliminary performance numbers, and provide a roadmap for the project's future.

XPDDS19 Keynote: Secret-free Hypervisor: Now and Future - Wei Liu, Software E...

The Linux Foundation

XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx

The Linux Foundation

This talk will introduce Dom0-less: a new way of using Xen to build mixed-criticality solutions. Dom0-less is a Xen feature that adds a novel approach to static partitioning based on virtualization. It allows multiple domains to start at boot time directly from the Xen hypervisor, decreasing boot times dramatically. Xen userspace tools, such as xl and libvirt, become optional. Dom0-less extends the existing device tree based Xen boot protocol to cover information required by additional domains. Binaries, such as kernels and ramdisks, are loaded by the bootloader (u-boot) and advertised to Xen via new device tree bindings. The audience will learn how to use Dom0-less to partition the system. Uboot and device tree configuration details will be explained to enable the audience to get the most out of this feature. The talk will include a status update and details on future plans.

XPDDS19 Keynote: Patch Review for Non-maintainers - George Dunlap, Citrix Sys...

The Linux Foundation

As the number of contributions grow, reviewer bandwidth becomes a bottleneck; and maintainers are always asking for more help. However, ultimately maintainers must at least Ack every patch that goes in; so if you're not a maintainer, how can you contribute? Why should anyone care about your opinion? This talk will try to lay out some advice and guidelines for non-maintainers, for how they can do code review in a way which will effectively reduce the load on maintainers when they do come to review a patch.

XPDDS19: Memories of a VM Funk - Mihai Donțu, Bitdefender

The Linux Foundation

This talk is a follow-up to our Summit 2017 presentation in which we covered our plans for Intel VMFUNC and #VE, as well as related use-cases. This year, we will provide a report on what we have accomplished in Xen 4.12, and what remains to be addressed. We will also give a brief status update of VMI on AMD hardware. The session will end with some real-world numbers of the Hypervisor Introspection solution running on Citrix Hypervisor 8.0 with #VE enabled.

OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...

The Linux Foundation

Safety certification is one of the essential requirements for software to be used in highly regulated industries. Besides technical and compliance issues (such as ISO 26262 vs IEC 611508) transitioning an existing project to become more easily safety certifiable requires significant changes to development practices within an open source project. In this session, we will lay out some challenges of making safety certification achievable in open source and the Xen Project. We will outline the process the Xen Project has followed thus far and highlight lessons learned along the way. The talk will primarily focus on necessary process, tooling changes and community challenges that can prevent progress. We will be offering an in-depth review of how Xen Project is approaching this challenging goal and try to derive lessons for other projects and contributors.

OSSJP/ALS19: The Road to Safety Certification: How the Xen Project is Making...

The Linux Foundation

This document summarizes a discussion around enabling functional safety certification for the Xen open source hypervisor project. Key points discussed include: - Establishing a split development model with open and closed parts to balance community needs and safety requirements. - Developing reference implementations and stacks supported by multiple vendors to demonstrate safety certification feasibility. - Creating plans and processes around requirements, documentation, verification testing, and tooling integration to begin filling gaps for certification. - Addressing challenges around funding, resources, expertise, and maintaining contributions to ensure any initial work is sustainable long-term. - Taking an iterative, agile approach to make early progress while further securing necessary funding and support from interested parties.

XPDDS19: Speculative Sidechannels and Mitigations - Andrew Cooper, Citrix

The Linux Foundation

2018 saw fundamental shifts in security boundaries which were previously taken for granted. A lot of work has been done in the past 2 years, and largely in secret under embargo, but there is plenty more work to be done to strengthen the existing mitigations and to try to recover some performance without reopening security holes. This talk will look at speculative execution sidechannels, the work which has already been done to mitigate the security holes, and future work which hopes to bring some improvements.

XPDDS19: Keeping Coherency on Arm: Reborn - Julien Grall, Arm ltd

The Linux Foundation

The Arm architecture provides a set of guidelines that any software should abide by when accessing the memory with MMU off and update page-tables. Failing to do so may result in getting TLB conflicts or breaking coherency. In a previous talk ("Keeping coherency on Arm"), we focused on updating safely the stage-2 (aka P2M) page-tables. This talk will focus on the boot code and Xen memory management. During this session, we will introduce some of the guidelines and when they should be used. We will also discuss how Xen boot sequence needs to be reworked to avoid breaking the guidelines.

XPDDS19: QEMU PV Backend 'qdevification'... What Does it Mean? - Paul Durrant...

The Linux Foundation

For many years the QEMU codebase has contained PV backends for Xen guests, giving them paravirtual access to storage, network, keyboard, mouse, etc. however these backends have not been configurable as QEMU devices as their implementation did not fully adhere to the QEMU Object Model (QOM). Particularly the PV storage backend not using proper QOM devices, or qdevs, meant that the QEMU block layer needed to maintain legacy code that was cluttering up the source. This was causing push-back from the maintainers who did not want to accept any patches relating to that Xen backend until it was 'qdevified'. In this talk, I'll explain the modifications I made to QEMU to achieve 'qdevification' of the PV storage backend, how compatibility with the libxl toolstack was maintained, and what the next steps in both QEMU and libxl development should be.

XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems

The Linux Foundation

XPDDS19: Bringing Xen to the Masses: The Story of Building a Community-driven...

The Linux Foundation

Xen is a very powerful hypervisor with a talented and diverse developers community. Despite the fact it's almost everywhere (from the Cloud to the embedded world), it can be difficult to set up and manage as a system administrator. General purpose distros have Xen packages, but that's just a start in your Xen journey: you need some tooling and knowledge to have a working and scalable platform. XCP-ng was built to overcome those issues: by bringing Xen to the masses with a fully turnkey distro with Xen as its core. It's the logical sequel to the XCP project, with a community focus from the start. We'll see how it happened, what we did, and what's next. Finally, we'll see the impact of XCP-ng on the Xen Project.

XPDDS19: Will Robots Automate Your Job Away? Streamlining Xen Project Contrib...

The Linux Foundation

XPDDS19: Client Virtualization Toolstack in Go - Nick Rosbrook & Brendan Kerr...

The Linux Foundation

High level toolstacks for server and cloud virtualization are very mature with large communities using and supporting them. Client virtualization is a much more niche community with unique requirements when compared to those found in the server space. In this talk, we’ll introduce a client virtualization toolstack for Xen (redctl) that we are using in Redfield, a new open-source client virtualization distribution that builds upon the work done by the greater virtualization and Linux communities. We will present a case for maturing libxl’s Go bindings and discuss what advantages Go has to offer for high level toolstacks, including in the server space.

XPDDS19: Core Scheduling in Xen - Jürgen Groß, SUSE

The Linux Foundation

Today Xen is scheduling guest virtual cpus on all available physical cpus independently from each other. Recent security issues on modern processors (e.g. L1TF) require to turn off hyperthreading for best security in order to avoid leaking information from one hyperthread to the other. One way to avoid having to turn off hyperthreading is to only ever schedule virtual cpus of the same guest on one physical core at the same time. This is called core scheduling. This presentation shows results from the effort to implement core scheduling in the Xen hypervisor. The basic modifications in Xen are presented and performance numbers with core scheduling active are shown.

XPDDS19: Implementing AMD MxGPU - Jonathan Farrell, Assured Information Security

The Linux Foundation

Jon Farrell summarizes his experience implementing AMD MxGPU virtualization. He began by learning about MxGPU on GitHub and setting up an Ubuntu server with Xen. Initial setup involved building a custom kernel, blacklisting amdgpu, and enabling IOMMU. Guest preparation required installing a VNC server and driver. Difficulties included choosing the right Xen version, starting modules, kernel building, and MxGPU patches. Performance metrics were estimated based on similar benchmarks. Overall, Farrell learned about building Xen, custom kernels, guest VMs, and device passthrough with XL. He sees potential for this technology in cloud gaming platforms and Xen virtualization.

More from The Linux Foundation (20)