XPDS13: Gnome outreach - Destination Xen - Elena Ufimtseva

•

0 likes•1,826 views

In this talk I plan to describe my first impression from working on open source project, how the community is organized and what can be gained by contributing to open source community. As a supporting fact of positive outcome from participating in similar projects, I will provide an example from my personal experience. It demonstrates how anyone can dive into open source development and work on project of choice. I introduce my work on virtual NUMA within XenProject and OPW program. It will demonstrate the challenges that newbie will face diving into mature development product and process, the necessity of discipline, commitment and self-organization. I will show what I had to learn in order to move forward, what I had to focus on and how to progress. The work what was done at this moment would not be possible without help of project mentors, source code knowledge, learning how to extract information from patches and ask right questions. I hope this example will bring more passionate developers into the community of open source software and hacking.

Technology

OPW - Xen vNUMA
OPW - outreach program for women
Me – Elena - Linux kernel XenProject Intern
vNUMA – virtual NUMA
Xen – you know...
was working on OPW all summer 2013, had fun and really enjoyed coding, best mentors in town

+

+

Mentors
OPW
Mentors

Xen and kernel
mentors

NUMA and Linux

NUMA topology parsing

NUMA aware scheduling

Automatic NUMA balancing

libnuma

Xen vNUMA
●

'enlighten' guest with vNUMA topology

●

performance in mind

●

let user decide on virtual topology

pNUMA + vNUMA
virtual NUMA nodes
VM 1

A

VM 2

B

VM 3

A

B

C

A

B

node1
A

node2
B

A

A

Physical NUMA nodes

B

C

B

OPW internship
●

OPW, Linux Foundation, XenProject

●

Meet my mentors

●

development community

●

Learn source code

●

Xen and kernel

●

vNUMA

What was done
●

Xen and Linux patches

●

Understand code

●

Comments from community

●

vNUMA in Xen rc 4.4

PV guest with vNUMA
Configure vNUMA topology

Or leave it automatic

Boot vNUMA aware guest

name = "vm1"
memory = 4096
vcpus = 4
vnodes = 2
vnumamem = [2048, 2048]
vdistance = [10, 20]
vnuma_vcpumap =[1, 0, 1, 0]
vnuma_vnodemap = [0, 1, 1, 0, 1, 1]
name = "vm2"
memory = 4096
vnodes = 2

PV guest with vNUMA
Configure vNUMA topology
name = "vm1"
memory = 4096
vcpus = 4
vnodes = 2
vnumamem = [2048, 2048]
vdistance = [10, 20]
vnuma_vcpumap =[1, 0, 1, 0]
vnuma_vnodemap = [0, 1, 1, 0, 1, 1]

Boot vNUMA aware guest

What has to be done
●

A lot! :)

●

Dom0/HVM NUMA awareness

●

Automatic NUMA balancing support

●

Performance evaluations

●

Heuristics/Statistics

What I have learned
●

Source code

●

Patches

●

Coding styles

●

Xen and para-virtual guests

●

Linux, booting, memory management

Thank you
●

Linux Foundation and GNOME

●

XenProject, Citrix, Oracle

●

Xen project mentors and Linux Kernel Mentors
Dario Steffano George Konrad Sarah Greg

More from The Linux Foundation

Artem will briefly cover what has been done since the first talk on Xen in Automotive domain back in 2013, what is going on now and what is still missing for broad adaptation of Xen in vehicles. The following topics will be covered: Embedded/automotive features of Xen Collaboration with AGL and GENIVI organizations for standardization Efforts on Functional Safety compliance Artem will also go over typical automotive use scenarios for Xen which may not be the same as generic computing use of hypervisor.

XPDDS19 Keynote: Xen in Automotive - Artem Mygaiev, Director, Technology Solu...

The Linux Foundation

XPDDS19 Keynote: Xen Project Weather Report 2019 - Lars Kurth, Director of Op...

The Linux Foundation

In recent years unikernels have shown immense performance potential (e.g., boot times of only a few ms, image sizes of only hundreds of KBs).The fundamental drawback of unikernels is that they require that applications be manually ported to the underlying minimalistic OS, needing both expert work and often considerable amount of time. The Unikraft project provides a unikernel code base and build system that significantly simplifies the building of unikernels. In addition to support for a number CPU architectures, languages and frameworks, Unikraft provides debugging and tracing features that are generally sorely missing from unikernel projects. In this talk we will talk about these features, show a set of preliminary performance numbers, and provide a roadmap for the project's future.

XPDDS19 Keynote: Unikraft Weather Report

The Linux Foundation

XPDDS19 Keynote: Secret-free Hypervisor: Now and Future - Wei Liu, Software E...

The Linux Foundation

This talk will introduce Dom0-less: a new way of using Xen to build mixed-criticality solutions. Dom0-less is a Xen feature that adds a novel approach to static partitioning based on virtualization. It allows multiple domains to start at boot time directly from the Xen hypervisor, decreasing boot times dramatically. Xen userspace tools, such as xl and libvirt, become optional. Dom0-less extends the existing device tree based Xen boot protocol to cover information required by additional domains. Binaries, such as kernels and ramdisks, are loaded by the bootloader (u-boot) and advertised to Xen via new device tree bindings. The audience will learn how to use Dom0-less to partition the system. Uboot and device tree configuration details will be explained to enable the audience to get the most out of this feature. The talk will include a status update and details on future plans.

XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx

The Linux Foundation

As the number of contributions grow, reviewer bandwidth becomes a bottleneck; and maintainers are always asking for more help. However, ultimately maintainers must at least Ack every patch that goes in; so if you're not a maintainer, how can you contribute? Why should anyone care about your opinion? This talk will try to lay out some advice and guidelines for non-maintainers, for how they can do code review in a way which will effectively reduce the load on maintainers when they do come to review a patch.

XPDDS19 Keynote: Patch Review for Non-maintainers - George Dunlap, Citrix Sys...

The Linux Foundation

This talk is a follow-up to our Summit 2017 presentation in which we covered our plans for Intel VMFUNC and #VE, as well as related use-cases. This year, we will provide a report on what we have accomplished in Xen 4.12, and what remains to be addressed. We will also give a brief status update of VMI on AMD hardware. The session will end with some real-world numbers of the Hypervisor Introspection solution running on Citrix Hypervisor 8.0 with #VE enabled.

XPDDS19: Memories of a VM Funk - Mihai Donțu, Bitdefender

The Linux Foundation

Safety certification is one of the essential requirements for software to be used in highly regulated industries. Besides technical and compliance issues (such as ISO 26262 vs IEC 611508) transitioning an existing project to become more easily safety certifiable requires significant changes to development practices within an open source project. In this session, we will lay out some challenges of making safety certification achievable in open source and the Xen Project. We will outline the process the Xen Project has followed thus far and highlight lessons learned along the way. The talk will primarily focus on necessary process, tooling changes and community challenges that can prevent progress. We will be offering an in-depth review of how Xen Project is approaching this challenging goal and try to derive lessons for other projects and contributors.

OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...

The Linux Foundation

Safety certification is one of the essential requirements for software to be used in highly regulated industries. The Xen Project, a secure and stable hypervisor that is used in many different markets, has been exploring the feasibility of building safety certified products on top of Xen for a year, looking at key aspects of its code base and development practices. In this session, we will lay out the motivation and challenges of making safety certification achievable in open source and the Xen Project. We will outline the process the project has followed thus far and highlight lessons learned along the way. The talk will cover technical enablers, necessary process and tooling changes and community challenges offering an in-depth review of how Xen Project is approaching this exciting and and challenging goal.

OSSJP/ALS19: The Road to Safety Certification: How the Xen Project is Making...

The Linux Foundation

2018 saw fundamental shifts in security boundaries which were previously taken for granted. A lot of work has been done in the past 2 years, and largely in secret under embargo, but there is plenty more work to be done to strengthen the existing mitigations and to try to recover some performance without reopening security holes. This talk will look at speculative execution sidechannels, the work which has already been done to mitigate the security holes, and future work which hopes to bring some improvements.

XPDDS19: Speculative Sidechannels and Mitigations - Andrew Cooper, Citrix

The Linux Foundation

The Arm architecture provides a set of guidelines that any software should abide by when accessing the memory with MMU off and update page-tables. Failing to do so may result in getting TLB conflicts or breaking coherency. In a previous talk ("Keeping coherency on Arm"), we focused on updating safely the stage-2 (aka P2M) page-tables. This talk will focus on the boot code and Xen memory management. During this session, we will introduce some of the guidelines and when they should be used. We will also discuss how Xen boot sequence needs to be reworked to avoid breaking the guidelines.

XPDDS19: Keeping Coherency on Arm: Reborn - Julien Grall, Arm ltd

The Linux Foundation

For many years the QEMU codebase has contained PV backends for Xen guests, giving them paravirtual access to storage, network, keyboard, mouse, etc. however these backends have not been configurable as QEMU devices as their implementation did not fully adhere to the QEMU Object Model (QOM). Particularly the PV storage backend not using proper QOM devices, or qdevs, meant that the QEMU block layer needed to maintain legacy code that was cluttering up the source. This was causing push-back from the maintainers who did not want to accept any patches relating to that Xen backend until it was 'qdevified'. In this talk, I'll explain the modifications I made to QEMU to achieve 'qdevification' of the PV storage backend, how compatibility with the libxl toolstack was maintained, and what the next steps in both QEMU and libxl development should be.

XPDDS19: QEMU PV Backend 'qdevification'... What Does it Mean? - Paul Durrant...

The Linux Foundation

PCI is a local computer bus for attaching hardware devices in a computer, and is the main peripheral bus on modern x86 systems. As such, having a proper way to emulate it is crucial for Xen to be able to expose both fully emulated devices or passthrough devices to guests. This talk will focus on the current status of PCI emulation in Xen, how and where it is used, what are its main limitations and future plans to improve it in order to be more robust and modular.

XPDDS19: Status of PCI Emulation in Xen - Roger Pau Monné, Citrix Systems R&D

The Linux Foundation

XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems

The Linux Foundation

Xen is a very powerful hypervisor with a talented and diverse developers community. Despite the fact it's almost everywhere (from the Cloud to the embedded world), it can be difficult to set up and manage as a system administrator. General purpose distros have Xen packages, but that's just a start in your Xen journey: you need some tooling and knowledge to have a working and scalable platform. XCP-ng was built to overcome those issues: by bringing Xen to the masses with a fully turnkey distro with Xen as its core. It's the logical sequel to the XCP project, with a community focus from the start. We'll see how it happened, what we did, and what's next. Finally, we'll see the impact of XCP-ng on the Xen Project.

XPDDS19: Bringing Xen to the Masses: The Story of Building a Community-driven...

The Linux Foundation

XPDDS19: Will Robots Automate Your Job Away? Streamlining Xen Project Contrib...

The Linux Foundation

High level toolstacks for server and cloud virtualization are very mature with large communities using and supporting them. Client virtualization is a much more niche community with unique requirements when compared to those found in the server space. In this talk, we’ll introduce a client virtualization toolstack for Xen (redctl) that we are using in Redfield, a new open-source client virtualization distribution that builds upon the work done by the greater virtualization and Linux communities. We will present a case for maturing libxl’s Go bindings and discuss what advantages Go has to offer for high level toolstacks, including in the server space.

XPDDS19: Client Virtualization Toolstack in Go - Nick Rosbrook & Brendan Kerr...

The Linux Foundation

Today Xen is scheduling guest virtual cpus on all available physical cpus independently from each other. Recent security issues on modern processors (e.g. L1TF) require to turn off hyperthreading for best security in order to avoid leaking information from one hyperthread to the other. One way to avoid having to turn off hyperthreading is to only ever schedule virtual cpus of the same guest on one physical core at the same time. This is called core scheduling. This presentation shows results from the effort to implement core scheduling in the Xen hypervisor. The basic modifications in Xen are presented and performance numbers with core scheduling active are shown.

XPDDS19: Core Scheduling in Xen - Jürgen Groß, SUSE

The Linux Foundation

The use of Virtual GPUs (vGPUs) has widely grown in server farms to give Virtual Machines (VMs) dedicated graphics. Software rendering with virtual CPUs can only take us so far and even with Intel-GVT, which uses integrated graphics, there isn't enough power to do the fun stuff. In this presentation, Jon Farrell will be talking about the process of implementing AMD MxGPU on Xen, challenges that he encountered while doing it, and discussing performance metrics of bare metal and vGPU VM on popular benchmarks like 3D Mark* and The Witcher 3. To wrap up his presentation, Jon will share his thoughts about future research and where this technology can take us.

XPDDS19: Implementing AMD MxGPU - Jonathan Farrell, Assured Information Security

The Linux Foundation

Current support of nested virtualization with Xen is limited to fully emulated devices for the L1 hypervisor (L0 hypervisor being the one running on the physical machine). For being able to let L2 dom0 make use of L1 PV devices several new interfaces are needed. In this design session I'll present my ideas how to add support of PV devices for L2 dom0. There are several possibilities how to do the work which I'd like to discuss.

XPDDS19: Support of PV Devices in Nested Xen - Jürgen Groß, SUSE

The Linux Foundation

More from The Linux Foundation (20)