Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Lars Kurth
Community Manger, Xen Project
Chairman, Xen Project Advisory Board
Lead CentOS Virtualization SIG
Director, Ope...
Released on January 15, 2015 (10 months of development)
Resources:
Blog: bit.do/xen-4-5-blog
Docs: bit.do/xen-4-5-docs
Dow...
0
5
10
15
20
25
30
35
40
45
50
2010 2011 2012 2013 2014 2015
4.0 4.1 4.2 4.3 4.4 4.5
Xen 4.x Number of New Major Features
0
50
100
150
200
250
2010 2011 2012 2013 2014
Developers
Employers
Using GitDM over Git logs using our database of develop...
with a twist
HWCPUsMemoryI/O
VM1
Guest OS
Applications
VM0 (or Dom0)
Dom0 Kernel
Drivers
VM2 VMn
Applications
Guest OS
Applications
Gue...
Xen 4.5: XEND / XM has been removed
XL now the default interface into Xen
Resources:
Docs: bit.do/xen-xl
Comparison: bit.d...
Xen via Libvirt in Openstack:
Great Platform for Production Deployments
Get into Quality Group A in 2015
Great Platform fo...
Number 1 priority for the project
Vendor funded Test Infrastructure
More capacity & coverage
Automated performance testing...
Overview
Xen 4.5: Real-Time Deferrable Server Scheduler
What is next?
Resources:
Docs: bit.do/xen-schedulers
HWCPUsMemoryI/O
Dom0
Dom0 Kernel
Drivers
The Xen Project Hypervisor supports several
different schedulers with different p...
HWCPUsMemoryI/O
Dom0
Dom0 Kernel
Drivers
The Xen Project Hypervisor supports several
different schedulers with different p...
HWCPUsMemoryI/O
Dom0
Dom0 Kernel
Drivers
VM1
Guest OS
RT App
VMn
Guest OS
RT App
e.g. RTDS Scheduler
VMn+1
Apps
Guest OS
V...
Soft Real-time CPU scheduler (experimental)
Guarantees CPU capacity to guest VMs on SMP hosts
Budget: Amount of time assig...
Embedded & automotive
Latency sensitive workloads
Guaranteed QoS
Cloud based gaming, video,
TV delivery, …
Guaranteed QoS
...
Scheduler Use-cases Xen 4.5 Plans for 4.6+
Credit General Purpose Supported
Default
Supported
Optional
Credit 2 General Pu...
Overview
Jan 2015: Intel GVT-g (XenGT) Updates
What is next?
Resources:
News: bit.do/xengt-jan15
Docs: bit.do/xengt-jan15-...
Watch the demo at
https://www.youtube.com/
watch?v=V2i8HCcAnY8
Virtual GPU per VM
Performance critical resources
directly ...
XenGT support is currently out-of-tree
Q4-2014 refresh by Intel: In use by XenClient 5.5
First patches have been posted fo...
Virtualization Modes
PVH Dom 0
Other Performance improvements
What is next?
Shortcut Mode With
HVM / Fully Virtualized HVM
HVM + PV drivers HVM PV Drivers
PVHVM HVM PVHVM Drivers
PVH PV pvh=1
PV PV
...
Shortcut Mode With
HVM / Fully Virtualized HVM
HVM + PV drivers HVM PV Drivers
PVHVM HVM PVHVM Drivers
PVH PV pvh=1
PV PV
...
PVH PV P P P VH
PV PV P P P P
ARM PV P P P VH
Simplicity: Less code & fewer Interfaces in Linux/FreeBSD
– Security : small...
Feature Complete
Hardware support for AMD x86 chips
Add support for PCI passthrough
Migration of PVH Dom U’s (including sy...
x86
HPET: Better and faster resolution values
Parallel memory scrubbing on boot (large machines)
Lower interrupt latency f...
Vulnerabilities published in 2014
Evolution of Xen Security Features
Xen 4.5 : Virtual Machine Introspection
A new Model f...
Escalation Linux Container KVM + QEMU Xen (PV)
Xen (HVM+Stub)
Privilege
Escalation
(guest to host)
7 – 9 3 – 5 0
Denial of...
bugs.xenproject.org/xen/bug/44
(agreed, but not yet published)
2007 2008 2009 2010 201520142011 2012 2013
Stub Domains : QEMU in separate domains
Flask / Xen Security Modules (Xen’s ver...
2007 2008 2009 2010 201520142011 2012 2013
XenAccess / XenProbes VM Introspection (via LibVMI)
Exposed lots of existing Xe...
Watch the demo at
https://www.youtube.com/wa
h?v=ZJPHfpDiN4o
Credit: Tamas K Lengyel
VM3
Guest OS
App
VMn
Guest OS
App
VM2
Guest OS
App
Dom0
Dom0 Kernel
Drivers Agent(s) Agent(s) Agent(s)
Installed in-guest ...
Several
VM3 VMnVM2Dom0
Dom0 Kernel
Drivers
VM3
Guest OS
App
VMn
Guest OS
App
VM2
Guest OS
App
Security
Appliance
VM1
Intro...
Major re-work of Virtual Machine Introspection
Optimization, Code cleanup/future-proofing
Support for ARM CPUs
Intel #VE s...
Reduce TCB
QEMU secure mode for HVM without stub domains
Move the instruction emulator into non-privilege mode
Move the Xe...
Introduction
Xen 4.5: Remus & COLO
What’s Next?
Resources:
Remus: bit.do/xen-remus
COLO: bit.do/xen-colo
HA: bit.do/xen-ha
Remus: Non-stop Service Replication
Continually live migrates a copy of a running VM to a backup server
Automatically acti...
Remus
Some “loose ends”, e.g. one fix for PV
guests not in upstream kernel
Better tools integration and control
(“xl remus...
ARM Additions
What is next?
X86
Larger VMs
Up to 1TB of guest RAM
Lower virtualization overhead
Super page mappings and faster interrupt EOIs (no maintena...
Hardening
Inclusion of 64 Bit Hardware into test infrastructure
VM Save/Restore and Live Migration
Note: Remus and COLO ar...
Determine the usage of cache by VMs running
Monitors the L3 cache (LLC in most server platforms)
$ xl psr-cmt-attach vm-id...
Release and Roadmap Process
Vinovyn @ Flickr
Release Manager: Wei Liu
Proposal: Tweaked Release Process for Xen 4.6
lists.xenproject.org/archives/html/xen-devel/2015-0...
Master branch on xen.git
Feature Development
Feature
Freeze
point
Wait period
to clear test pushgate
RC’s
Release
Announce...
Master branch on xen.git
Feature Development RC’s
This is when patches for the ongoing release
need to be submitted for re...
Release Manager:
Sends first
Xen x.y Development Update
email on xen-devel@
Deferred features from previous
release, Timet...
Other interesting planned Features
Other sub-projects
Embedded & Automotive
Sound, graphics, and other drivers for Linux and other OS’es
Lots of other enablers: e.g. security f...
Mirage OS
Safer and cleaner TLS stackopenmirage.org/blog/announcing-bitcoin-
pinata
Irmin: Git-like distributed, branchabl...
Slides on www.slideshare.net/xen_com_mgr/
Upcoming SlideShare
Loading in …5
×

LFCOLLAB15: Xen 4.5 and Beyond

42,908 views

Published on

The 4.5 release no a minor "point" update: it is one of the most feature-rich releases in the project's history. It contains several important additions. Most notably, new Xen PVH virtualization mode now supports running as dom0, enhanced support for Remus, significant ARM architecture updates, security improvements, real-time scheduling, support for Intel Cache Monitoring Technology (CMT), as well as improvements for automotive and embedded use-cases. Other enhancements include additional support for FreeBSD, systemd support, additional libvirt support, the release of Mirage OS 2.0, and more.

Besides giving an overview of Xen 4.5, we will explain the project's roadmap process and share what's ahead for 2015: such as improved OpenStack integration and hotpatching (applying security fixes without the need to reboot).

Published in: Technology
  • Follow the link, new dating source: ♥♥♥ http://bit.ly/39pMlLF ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating for everyone is here: ❶❶❶ http://bit.ly/39pMlLF ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

LFCOLLAB15: Xen 4.5 and Beyond

  1. 1. Lars Kurth Community Manger, Xen Project Chairman, Xen Project Advisory Board Lead CentOS Virtualization SIG Director, Open Source Business Office, Citrix lars_kurth
  2. 2. Released on January 15, 2015 (10 months of development) Resources: Blog: bit.do/xen-4-5-blog Docs: bit.do/xen-4-5-docs Download: bit.do/xen-4-5-download Stats: Changesets: 1812 KLOC Added: 81 KLOC Removed: 141 (mostly removal of XM) Contributors:102 individuals Employers: 39 (93 individuals working for them)
  3. 3. 0 5 10 15 20 25 30 35 40 45 50 2010 2011 2012 2013 2014 2015 4.0 4.1 4.2 4.3 4.4 4.5 Xen 4.x Number of New Major Features
  4. 4. 0 50 100 150 200 250 2010 2011 2012 2013 2014 Developers Employers Using GitDM over Git logs using our database of developers and organizations to remove duplicates across all sub-projects Reasons for faster Innovation: More developers and orgs Fewer forked up-streams (e.g. Linux, BSDs, QEMU, …) Architecture clean-up (e.g. XM – XL) Better Development Process
  5. 5. with a twist
  6. 6. HWCPUsMemoryI/O VM1 Guest OS Applications VM0 (or Dom0) Dom0 Kernel Drivers VM2 VMn Applications Guest OS Applications Guest OS Toolstack Scheduler MMU Timers InterruptsConfig back PV front
  7. 7. Xen 4.5: XEND / XM has been removed XL now the default interface into Xen Resources: Docs: bit.do/xen-xl Comparison: bit.do/xen-4-5-xm-2-xl-compare Migration Guide: bit.do/xen-4-5-xm-2-xl Libvirt integration has been vastly improved Resources: Docs: bit.do/xen-libvirt Complete List: bit.do/xen-4-5-blog Dom0 Dom0 Kernel Drivers Toolstack(s) LIBXENLIGHT XL LIBVIRT XEND XM
  8. 8. Xen via Libvirt in Openstack: Great Platform for Production Deployments Get into Quality Group A in 2015 Great Platform for Development Great DevStack support Libvirt: Better Quality, Stability & Usability Drivers: OpenStack, CentOS Virt SIG – learning what a distro needs Resources: Docs: bit.do/xen-openstack Plans: bit.do/xen-openstack-fosdem15 Install Video: https://vimeo.com/119572029 XenServer (XAPI) ESX Hyper-V Group B NOVA LIBVIRT KVM Group A Xen Group C Group A
  9. 9. Number 1 priority for the project Vendor funded Test Infrastructure More capacity & coverage Automated performance testing Vendor funded OpenStack CI loop Xen Project Rack
  10. 10. Overview Xen 4.5: Real-Time Deferrable Server Scheduler What is next? Resources: Docs: bit.do/xen-schedulers
  11. 11. HWCPUsMemoryI/O Dom0 Dom0 Kernel Drivers The Xen Project Hypervisor supports several different schedulers with different properties. Different schedulers can be assigned to… … an entire host e.g. Credit2 Scheduler
  12. 12. HWCPUsMemoryI/O Dom0 Dom0 Kernel Drivers The Xen Project Hypervisor supports several different schedulers with different properties. Different schedulers can be assigned to… … an entire host … a pool of physical CPU’s (=CPU Pool) on a host (VMs need to be assigned to a pool or pinned to a CPU) e.g. RTDS Scheduler e.g. Credit Scheduler
  13. 13. HWCPUsMemoryI/O Dom0 Dom0 Kernel Drivers VM1 Guest OS RT App VMn Guest OS RT App e.g. RTDS Scheduler VMn+1 Apps Guest OS VMm Apps Guest OS e.g. Credit Scheduler Scheduler parameters can be modified per … Host CPU Pool VM
  14. 14. Soft Real-time CPU scheduler (experimental) Guarantees CPU capacity to guest VMs on SMP hosts Budget: Amount of time assigned to a VM Period: Time period in which depleted budgets are replenished Global: Allow VCPU Migration across CPUs Partitioned: Pin VCPU to a physical CPU Schedule VMs per CPU More flexibility & best utilization Migration Overhead & Cache Penalty May underutilize CPU Lower overheads & lower latency
  15. 15. Embedded & automotive Latency sensitive workloads Guaranteed QoS Cloud based gaming, video, TV delivery, … Guaranteed QoS (Price  SLAs  QoS)
  16. 16. Scheduler Use-cases Xen 4.5 Plans for 4.6+ Credit General Purpose Supported Default Supported Optional Credit 2 General Purpose Optimized for lower latency, higher VM density Experimental Supported Default RTDS Soft & Firm Real-time Multicore Embedded, Automotive, Graphics & Gaming in the Cloud, Low Latency Workloads Experimental Hardening Optimization Better XL support <1μs granularity Supported ARINC 653 Hard Real-time Single core Avionics, Drones, Medical Supported Compile time No change Legend: likely in 4.6 possible in 4.6
  17. 17. Overview Jan 2015: Intel GVT-g (XenGT) Updates What is next? Resources: News: bit.do/xengt-jan15 Docs: bit.do/xengt-jan15-docs
  18. 18. Watch the demo at https://www.youtube.com/ watch?v=V2i8HCcAnY8 Virtual GPU per VM Performance critical resources directly assigned to VM
  19. 19. XenGT support is currently out-of-tree Q4-2014 refresh by Intel: In use by XenClient 5.5 First patches have been posted for review on xen-devel Requires some Linux and QEMU patches also Motivation: create a common code base for Xen & KVM Likely complete for Xen 4.6 (or shortly afterwards) Will initially be experimental
  20. 20. Virtualization Modes PVH Dom 0 Other Performance improvements What is next?
  21. 21. Shortcut Mode With HVM / Fully Virtualized HVM HVM + PV drivers HVM PV Drivers PVHVM HVM PVHVM Drivers PVH PV pvh=1 PV PV Old New WindowsLinux,BSDs,… 3.0 4.0 4.4/4.5 Xen
  22. 22. Shortcut Mode With HVM / Fully Virtualized HVM HVM + PV drivers HVM PV Drivers PVHVM HVM PVHVM Drivers PVH PV pvh=1 PV PV Poor Performance Scope for Improvement Optimal Performance VS VS VS VH P VS VS VH P P VS VH P P P VH P P P P P = Paravirtualized VS = Software Virtualized (QEMU) VH = Hardware Virtualized WindowsLinux,BSDs,…
  23. 23. PVH PV P P P VH PV PV P P P P ARM PV P P P VH Simplicity: Less code & fewer Interfaces in Linux/FreeBSD – Security : smaller TCB and attack surface, fewer possible exploits – Clean-up : possibility to simplify Linux kernel and reduce maintenance burden Better Performance & Lower Latency – Dom0 must be a PV guest – 64 bit: VM’s run in ring 0 instead of ring 3 (fewer expensive TLB flushes) This is the most complex part of Xen today!
  24. 24. Feature Complete Hardware support for AMD x86 chips Add support for PCI passthrough Migration of PVH Dom U’s (including systems with PVH Dom 0) Hardening & Tuning Add PVH to test suite and make test failures blocking Benchmarking and performance tests Code clean-up
  25. 25. x86 HPET: Better and faster resolution values Parallel memory scrubbing on boot (large machines) Lower interrupt latency for PCI passthrough (machines > 2 sockets) Soft affinity for non-NUMA machines Multiple IO-REQ services for guests (remove bottlenecks for HVM guests by allowing multiple QEMU back-ends) Intel SandyBridge: VT-d posted interrupts for PVHVM (I/O intensive workloads)
  26. 26. Vulnerabilities published in 2014 Evolution of Xen Security Features Xen 4.5 : Virtual Machine Introspection A new Model for Cloud Security What is next?
  27. 27. Escalation Linux Container KVM + QEMU Xen (PV) Xen (HVM+Stub) Privilege Escalation (guest to host) 7 – 9 3 – 5 0 Denial of Service (by guest of host) 12 5 – 7 3 Information Leak (from host to guest) 1 0 1 Assumptions x86 vulnerabilities from guest to host that hosting/cloud providers worry about Xen (HVM) without stub domains has slightly more than Xen (PV) due to use of QEMU, less than KVM + QEMU Have the underlying analysis (but won’t cover it in the talk)
  28. 28. bugs.xenproject.org/xen/bug/44 (agreed, but not yet published)
  29. 29. 2007 2008 2009 2010 201520142011 2012 2013 Stub Domains : QEMU in separate domains Flask / Xen Security Modules (Xen’s version of SE Linux) vTPM (Virtual Trusted Module) Driver Domains (Network, Disk, … drivers in a separate VM) TODAY: Mainly used by security apps (XenClient, Qubes OS, …), Forensic, Military & Embedded TODAY: In general use (but has trade-offs at cloud scale) XenAccess / XenProbes VM Introspection (via LibVMI) Major Upgrades
  30. 30. 2007 2008 2009 2010 201520142011 2012 2013 XenAccess / XenProbes VM Introspection (via LibVMI) Exposed lots of existing Xen functionality in LibVMI Hypervisor can bring paged out guest memory Mem_access-emulate(-with-no-write) Many more patches currently under review for Xen 4.6
  31. 31. Watch the demo at https://www.youtube.com/wa h?v=ZJPHfpDiN4o Credit: Tamas K Lengyel
  32. 32. VM3 Guest OS App VMn Guest OS App VM2 Guest OS App Dom0 Dom0 Kernel Drivers Agent(s) Agent(s) Agent(s) Installed in-guest agents, e.g. anti-virus software, VM disk & memory scanner, network monitor, etc. Anti virus storm, deployment/maintenance, …
  33. 33. Several VM3 VMnVM2Dom0 Dom0 Kernel Drivers VM3 Guest OS App VMn Guest OS App VM2 Guest OS App Security Appliance VM1 Introspection Engine Protected area Agent Agent Agent Hybrid approach: no need to move everything outside (chose best trade-off) XSM/Flask
  34. 34. Major re-work of Virtual Machine Introspection Optimization, Code cleanup/future-proofing Support for ARM CPUs Intel #VE support Turn on Xen Security Modules on by default and include in test suite Disabled today and not automatically tested Specialist Use General Use!
  35. 35. Reduce TCB QEMU secure mode for HVM without stub domains Move the instruction emulator into non-privilege mode Move the Xen compatibility layer into a lower privilege ring Binary Live Patching for the Xen Hypervisor Depends on which solution the kernel will standardize on (kpatch / kGraft / ftrace-based) We want to share tooling
  36. 36. Introduction Xen 4.5: Remus & COLO What’s Next? Resources: Remus: bit.do/xen-remus COLO: bit.do/xen-colo HA: bit.do/xen-ha
  37. 37. Remus: Non-stop Service Replication Continually live migrates a copy of a running VM to a backup server Automatically activates if the primary server fails Expensive in terms of overheads and hardware requirements COLO: A different approach (building on top of Remus) Relaxes requirement of backup server/VM being an exact replica If backup server generates the same response to input we are able to fail over without service stop Eliminates overheads, reduces hardware requirements
  38. 38. Remus Some “loose ends”, e.g. one fix for PV guests not in upstream kernel Better tools integration and control (“xl remus” instead of “remus”) Optimizations for COLO COLO Out-of-tree Integrates with Remus via “xl remus” – works with Xen 4.5 Some known issues Fix “loose ends” Include into Xen Hypervisor code base Switch block replication from blktap2 to qdisk (motivation: performance & alignment) Hardening
  39. 39. ARM Additions What is next? X86
  40. 40. Larger VMs Up to 1TB of guest RAM Lower virtualization overhead Super page mappings and faster interrupt EOIs (no maintenance interrupts) Improved Interrupt handling Support for priorities and irq migration (virtual and physical) Near feature parity with x86 Boot via UEFI firmware QEMU PV backends (disk, console, keyboard, mouse, framebuffer) Many new IP blocks, firmware interfaces and platforms are supported E.g. AMD Seattle 64-bit server SoC – see bit.do/xen-4-5-docs
  41. 41. Hardening Inclusion of 64 Bit Hardware into test infrastructure VM Save/Restore and Live Migration Note: Remus and COLO are architecture independent PCI Passthrough Note: passthrough of MMIO regions works in 4.6 ACPI and UEFI support for guests More IP blocks, … Support for more Hardware
  42. 42. Determine the usage of cache by VMs running Monitors the L3 cache (LLC in most server platforms) $ xl psr-cmt-attach vm-id $ xl psr-cmt-show cache_occupancy Identify noisy neighbor VMs and take corrective action E.g. Migrate VM to a different host E.g. CPU pinning, CPU pools, schedulers What’s Next? Intel Cache Allocation Technology Longer term: schedulers can use HW utilization information
  43. 43. Release and Roadmap Process Vinovyn @ Flickr
  44. 44. Release Manager: Wei Liu Proposal: Tweaked Release Process for Xen 4.6 lists.xenproject.org/archives/html/xen-devel/2015-02/msg01214.html Development start: 6 Jan 2015 Feature freeze: 10 Jul 2015 Release date: 9 Oct 2015 (could release earlier)
  45. 45. Master branch on xen.git Feature Development Feature Freeze point Wait period to clear test pushgate RC’s Release Announcement RELEASE-4.5.0 branch on xen.git
  46. 46. Master branch on xen.git Feature Development RC’s This is when patches for the ongoing release need to be submitted for review Wait period to clear test pushgate No new features will be accepted, unless there is a Freeze Exception Bug fixes are allowed, with approval by Maintainers/Release Manager Release Manager declares that only bug fixes deemed blockers can be accepted
  47. 47. Release Manager: Sends first Xen x.y Development Update email on xen-devel@ Deferred features from previous release, Timetable, etc. Release Manager: Sends Monthly Xen x.y Development Update email on xen-devel@ Release Manager: RC Announcements, Test Days Release Manager: RC Announcement Contributors: Expected to reply if they are working on a feature that is not on the list of tracked features Expected to provide Status updates on features & bugs on the list Not engaging with the process may lead to removal or downgrading Contributors: Expected to reply if they are working on a feature that is not on the list of tracked features and tracked bugs Same as above: can also ask for Freeze Exceptions Contributors: Expected to provide Status updates on tracked bugs on the list
  48. 48. Other interesting planned Features Other sub-projects
  49. 49. Embedded & Automotive Sound, graphics, and other drivers for Linux and other OS’es Lots of other enablers: e.g. security features Certification VMWare Tools support Run VMWare images unmodified in Xen More: First 4.6 Development Update lists.xenproject.org/archives/html/xen-devel/2015-02/msg01816.html
  50. 50. Mirage OS Safer and cleaner TLS stackopenmirage.org/blog/announcing-bitcoin- pinata Irmin: Git-like distributed, branchable storage Jitsu: a DNS server that spawns unikernels in response to DNS requests IPv6, Tooling, etc. VMn Language run-time Application Cubieboard2 serving 2048 game @ FOSDEM’15
  51. 51. Slides on www.slideshare.net/xen_com_mgr/

×