Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

XPDDS17: Keynote: Xen 4.8 at Gandi - Vincent Legout, Gandi


Published on is a cloud provider running about 10000 VMs since 2008. We recently updated our infrastructure from Xen 4.1 to Xen 4.8 and decided to move all of our platform to Xen (from a mix of Xen and KVM). This plaform uses home-made code based on Xen python bindings and xl to orchestrate VMs. This talk will present our use cases and the experience we had with Xen, the shortcomings or issues we had while upgrading our platform, what features we use, and present some new features we would like to have in Xen. For example, it will discuss how we use live patching and live migration. The talk will consider both the Xen hypervisor and its associated userspace utilities.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

XPDDS17: Keynote: Xen 4.8 at Gandi - Vincent Legout, Gandi

  1. 1. Xen 4.8 at Gandi Vincent Legout Xen Developer and Design Summit 2017 July 12, 2017
  2. 2. Gandi I have been working for Gandi for ≈ 1 year on Simple Hosting (PaaS, i.e. LXC) & Servers (IaaS) 2 / 17
  3. 3. Outline Xen history at Gandi How we moved from Xen 4.1 to Xen 4.8 How we use 4.8 features (especially live patching) 3 / 17
  4. 4. Virtualization at Gandi ≈ 10000 VMs today (customers & internal) x86 only (32 & 64 bits) Xen 4.8 (almost) everywhere mostly PV, some PVHVM (e.g. for VF, FreeBSD) Home made code to orchestrate VMs daemon running on each host makes calls to xl 4 / 17
  5. 5. Xen features usage Default scheduler & cpupool (no NUMA) Cap of domU according to its number of vCPUs similar weight for all vCPUs, no pinning VMs: only net, block (iSCSI), & console (e.g. no USB, video) CPU hotplug: attach & detach no longer any memory ballooning PCI passthrough for internal VMs PV & PVHVM with direct kernel boot (kernel or grub) 5 / 17
  6. 6. Xen versions history 2008 Aug. 2011 Oct. 2014 Dec. 2016 started with Xen 3.1 regular upgrades up to Xen 4.1 no Xen upgrade switch VM creation to KVM move everything back to Xen 6 / 17
  7. 7. Upgrade from 4.1 to 4.8 Why? Reduce impact for customers, avoid reboot Live patching Security support 7 / 17
  8. 8. Tooling & Build process Upgrade to use xl and python bindings (xen.lowlevel) Was previously using xend Few bugs in xl, patches sent and accepted very quickly very much appreciated! allows us to carry almost no patch Hypervisor: built from the git stable branch Userspace: based on the package in Debian Stretch 8 / 17
  9. 9. Live migration from 4.1 to 4.8 Not officially supported special tool to convert migration stream Test migrations went well but issues after a few days/weeks disk corruptions, crashes might be due to domU kernel (Linux suspend & resume?) Reboot was needed XSAs arrived just in time, a good excuse to reboot all VMs! 9 / 17
  10. 10. Refresh VMs & disks Disks: from no partition or MBR/DOS to GPT plan transition to OVMF & UEFI Switch to PvGrub by default avoid maintaining custom kernels will it still work with PVH2? Still need to support all existing VMs and disks 10 years old VMs still running! 10 / 17
  11. 11. domU kernels Used to carry patches, lead to a few issues Issue with detaching disks no automatic unmount No daemon inside domU communication only through system events e.g. custom udev rules 11 / 17
  12. 12. Live Patching Used since upgrading to Xen 4.8 (late 2016) Easy to use livepatch-build-tools.git & xen-livepatch Did not generate any crash/bug 12 / 17
  13. 13. Live Migration Especially useful for XSA that cannot be live patched avoid having to reboot all VMs Custom tool to migrate VMs according to free nodes, their CPUs, ... Rolling updates, no VM downtime 13 / 17
  14. 14. Live Migration Issues A few, only < 0.5% of all the VMs crashed only VMs in bad states, no suspend & resume Issues with a custom domU Linux kernel (3.10-i386) 14 / 17
  15. 15. Conclusion Overall, really satisfied with Xen especially Xen 4.8 new features XSA management, embargoes Some downsides: several XSAs lately clearer roadmap (e.g. PV/PVHVM, PVH2, OVMF) 15 / 17
  16. 16. Future - Plans More testing push tests on features we rely on Staying as close as possible from upstream avoid as much as possible carrying patches try/test new releases & features Switch to PVH2? 16 / 17
  17. 17. Thanks! Questions? 17 / 17