XCP combines the Xen hypervisor with enhanced security, storage, and network virtualization technologies to offer a rich set of virtual infrastructure cloud services. These XCP cloud services can beleveraged by cloud providers to enable isolation and multi-tenancy capabilities in their environments. XCP also provides the user requirements of security, availability, performance, and isolation for private and public cloud deployments. [Presented as part of the Open Source Build a Cloud program on 2/28/2012 - http://cloudstack.org/about-cloudstack/cloudstack-events.html?categoryid=6]

1. X E N C LO U D P L AT FO R M
Todd Deshane
Technology Evangelist, Xen.org
todd.deshane@xen.org

2. X E N I N T H E C L O U D H I S TO R Y
'99
XenoServers Project Global Public Computing
(Cambridge University)
"Xenoserver are machines that can “This dissertation proposes a
safely and securely perform useful work new distributed computing
on behalf of any user who is prepared to paradigm, termed global public
pay for the resources consumed" computing, which allows any user
to run any code anywhere. Such
Reed et al., 7th Workshop on Hot Topics platforms price computing
in Operating Systems, 1999 resources, and ultimately charge
users for resources consumed.“
Evangelos Kotsovinos, PhD
dissertation, 2004

3. X E N I N T H E C LO U D H I STO RY
XCP 1.0 Kronos
'99 Nov ‘02 Oct ‘03 ‘06 ‘08 ‘09 ‘11 ‘12
XenoServers Amazon
Project EC2
Dom0
and
in Linux
Slicehost
Xen launched Rackspace
Repository Cloud
Published
Xen XCP
Presented Announced XCP 1.5
at SOSP

4. XEN WAS DESIGNED FOR THE
CLOUD AND THE CLOUDS ARE
BUILT ON XEN!

5. BA S I C X E N CO N C E P T S
Control domain One or more service VMn VM in Xen
(Dom0) domains termionlogy
VM1 is DomU
VM0
Guest OS
and Apps
Dom0 Kernel
Scheduler, MMU Xen Hypervisor
Host HW
I/O Memory CPUs

6. X E N G U EST V I RT UA L I ZAT I O N T Y P ES
• Paravirtualization (PV)
o Guest kernel made Xen-aware (open source kernels)
o Avoid or replace non-virtualizable operations
o Very fast and legacy hardware is supported
• Hardware-assisted virtualization (HVM)
o Unmodified guest (full virtualization of proprietary OSes)
o Requires Intel VT-x or AMD-V (virtualization in hardware)
• PV on HVM
o Emulate when necessary/beneficial (e.g. boot/BIOS)
o Performs better or near PV for a variety of workloads

7. XEN AND THE LINUX KERNEL
Xen was initially a University research project
Invasive changes to the kernel to run Linux as a PV
guest
Even more changes to run Linux as dom0

8. XEN AND THE LINUX KERNEL
DomU/Dom0 Xen guest support was not in
upstream Linux kernel
Great maintenance effort on distributions
Fedora/Ubuntu temporarily dropped Dom0 support
Xen was harder to install

9. XEN AND THE LINUX KERNEL
PVOPS Project
Xen DomU since Linux 2.6.23
Xen Dom0 since Linux 3.0
On-going work on feature set and performance
optimization. Dom0 support in distros much better!

10. X E N C LO U D P L AT FO R M ( XC P )

11. XC P OV E RV I E W
• Open source version of Citrix XenServer
o wiki.xen.org/wiki/XCP/XenServer_Feature_Matrix
• Enterprise-ready server virt. and cloud platform
o Extends Xen beyond a single physical machine (host pools)
o Built-in templates for Windows and Linux guests
o Open vSwitch built-in and is default networking stack
• Datacenter and cloud-ready management API
o XenAPI (XAPI) is fully open source
o CloudStack and OpenStack integration

12. XC P TO O L S TA C K D I S T R O I N D E P E N D E N C E
 Extend the delivery model beyond XCP Appliance
• apt-get install xcp-xapi or yum install xcp-xapi
 Debian Wheezy, Ubuntu 12.04 LTS working
(Project Kronos )
 Fedora and CentOS in progress (Project Zeus)
 Volunteer to help and/or add support for your
favorite distro

13. X E N VS . XC P VS . P ROJ EC T K RO N O S
Xen XCP Project Kronos
Hypervisor: latest stable via Debian/Ubuntu package
Dom0 OS: CentOS, Debian, Fedora, NetBSD, CentOS 5.5 Debian and Ubuntu
OpenSuse, RHEL 5.x, Solaris 11, …
Dom 0: 32 and 64 bits 64 bits 32 and 64 bits
Linux 3 Dom0: Yes As of XCP 1.5, not yet Yes
Toolstack: Libxl or Libvirt stable XAPI latest XAPI
Advanced functionality: build it yourself Open vSwitch , storage build it yourself
repositories, and signed PV drivers
built-in
Configurations: everything directed by stable XAPI directed by latest XAPI
Usage Model: distro support or do it yourself Shrink wrapped and tested distro support or do it yourself
Distribution: Source or via Linux/Unix ISO (source available) source or via Debian/Ubuntu
distribution packages

14. XC P/ X A P I F U T U R E
• XCP & XAPI configuration of choice for clouds
o Optimized for usage patterns in cloud orchestration (e.g.
OpenStack, CloudStack OpenNebula)
• Exploit advanced Xen security features
o Driver domains, stub domains
o Dom0 dissagregation
• Xen on ARM (XAPI on ARM)

15. XC P 1 . 5
• Architectural Improvements: Xen 4.1, GPT, smaller Dom0
• GPU pass through: for VMs serving high end graphics
• Performance and Scalability: 1 TB mem/host 16 VCPUs/VM, 128
GB/VM. (Note: limits are due to XAPI restriction not Xen hypervisor)
• Enhanced Guest OS Support: Support for Ubuntu 10.04 (32/64-bit).
Updated support for Debian Squeeze 6.0 64-bit, Oracle Enterprise Linux 6.0
(32/64-bit) and SLES 10 SP4 (32/64-bit). Experimental VM templates for CentOS
6.0 (32/64-bit), Ubuntu 10.10 (32/64-bit) and Solaris 10.

16. X A P I : W H AT I S I T ?
• XAPI (or XenAPI) is the backbone of XCP
o Provides the glue between all components
• It's a XML-RPC style API, served via HTTPS
o Provided by a service on every XCP Dom0 host
o Designed to by highly programmable
o API bindings for many languages:
.NET, Java, C, Powershell, Python
• XAPI is Extensible via plugins (e.g. used by OpenStack)

17. X A P I F RO M 3 0 0 0 0 F E E T ( STO R AG E )
VM VM
Virtual Block Device Virtual Block Device Virtual Block Device
Virtual Disk Image Virtual Disk Image
Storage Repository
Physical Block Device(s) LVM, iSCSI, NFS. etc.

18. X A P I F RO M 3 0 0 0 0 F E E T ( N E T WO R K )
VM VM
Virtual Interface Virtual Interface Virtual Interface
(VIF) (VIF) (VIF)
Network Network
Physical Interface(s) Physical Interface(s) Bridge, Open
(PIF(s)) (PIF(s)) vSwitch, Bond, VLAN, etc.

19. X A P I F RO M 3 0 0 0 0 ( M E T R I C S )
host_metrics VM_metrics PIF_metrics VBD_Metrics
Host VM PIF VBD
xen.org/files/XenCloud/ocamldoc/apidoc

20. X A P I OV E RV I E W
• VM lifecycle management: live
snapshots, checkpoint, migration
• Resource pools: live migration, auto configuration, disaster
recovery
• Flexible storage (NFS, iSCSI, LVM) and networking (Open
vSwitch)
• Event tracking: progress (VM status), notification (disk full)
• Upgrade and patching capabilities (of XCP)
• Real-time performance monitoring and alerting (metrics)

21. O P E N VSW I TC H
• Software switch, similar to:
o VMware vNetwork Distributed Switch
o Cisco Nexus 1000V
• Distribution agnostic. Plugs right into Linux kernel.
• Reuses existing Linux kernel networking subsystems.
• Backwards-compatible with traditional userspace tools.

22. W H Y U S E O P E N VSW I TC H W I T H C LO U D ?
See also: XenServer Distributed
Virtual Switch Controller
• Automated control: OpenFlow
• Multi-tenancy
• Monitoring and QoS

23. X A P I M A N AG E M E N T O P T I O N S
• XAPI frontend command line tool: XE (tab-completable)
• Desktop GUIs
o Citrix XenCenter (Windows-only)
o OpenXenManager (open source cross-platform XenCenter clone)
• Web interfaces
o Xen VNC Proxy (XVP)
 lightweight VM console only
 user access control to VMs (multi-tenancy)
o XenWebManager (web-based clone of OpenXenManager)
• XCP Ecosystem: ProjectPage, ProductsPage

24. O P E N X E N M A N AG E R

25. X E N V N C P ROX Y ( X V P )

26. XC P A N D C LO U D O RC H EST R AT I O N STAC KS

27. S EC U R I T Y A N D T H E N E X T WAV E O F X E N
• Security is key requirement for Cloud
• Security is the primary goal of client virtualization
o Desktop, Laptops, Tablets & Smart Phones
• Maintaining isolation between VMs is critical
o Spatial and temporal isolation
o Run multiple VMs with policy controlled information flow
(e.g. Personal VM; Corporate VM; VM for web browsing;
VM for banking)

28. P L AC E M E N T O F C LO U D M A N AG E M E N T TO O L
Installed in a VM (DomU) Directly installed on Dom0
 Pros  Pros
 Isolation of cloud VM  Simple install
 Security properties  Flexibility
 Pre-packaged appliance  Cons
 Cons  Less isolation
 More complex  Cloud service is potential entry
 Less flexible point to compromise Dom0

29. A RC H I T EC T U R E CO N S I D E R AT I O N S
Type 1: Bare metal Hypervisor Type 2: OS ‘Hosted’
User-level VMM
VMn VMn
VM1 User Device VM1
Apps
Virtual VM Models
VM
Machine Guest OS Guest OS
Control and Apps and Apps
Host OS
Scheduler Hypervisor
Device Device Ring-0 VM Monitor
Drivers/Models MMU Drivers “Kernel “
I/O Memory CPUs Host HW Host HW I/
Memory
CPU
O s
Provides partition isolation + Low cost, no additional drivers
reliability,higher security Ease of use & installation

30. X E N : T Y P E 1 W I T H A T W I ST
• Thin hypervisor
Control domain
(dom0) o Key functionality moved
to Dom0
Device
Models VMn • Using Linux PVOPS
VM1
o Take advantage of PV
Drivers VM0
Guest OS o PV on HVM
Linux, BSD, etc. and Apps
o No additional device
Sched
MMU XSM Hypervisor drivers (Linux 3.x dom0)
•
uler
Host HW
In other words
I/O Memory CPUs
o Low cost (drivers)
o Isolation & security

31. X E N S EC U R I T Y A DVA N TAG ES
• Even without advanced security features
o Well-defined trusted computing base
o (much cleaner than a type-2 hypervisor)
o No extra services in hypervisor layer
• More Robustness: Mature, tried & tested architecture
• Xen Security Modules (or XSM)
o Developed and contributed to Xen by NSA
o Generalized security framework for Xen
o The Xen equivalent of SELinux

32. X E N D O M 0 D I SAG G R EG AT I O N
• Split control domain (Dom0) into driver, stub and service
domains
o Each contains a specific set of control logic
o See: ”Breaking up is hard to do” @ Xen Papers
• Unique benefit of the Xen architecture
o Security: minimum privilege; narrow interfaces
o Performance: lightweight, e.g. Mini-OS service domains
o Robustness: ability to safely restart parts of the system
o Scalability: more distributed system (less reliant on single
control domain)

33. Q U B ES O S / X E N C L I E N T X T
• First products configured to take advantage of the security
benefits of Xen’s architecture
• Isolated driver domains
• Virtual hardware emulation domains
• Service VMs (global and per-guest)
• Xen Security Modules (XSM)

34. X E N C L I E N T A RC H I T EC T U R E
Per host/device Per guest
Service VMs Service VMs
Managemen
User VM
Emulation
User VM
Isolation
Emulate
t Domain
Isolation
Isolation
Device
Network
Domain
Control
Device
VPN
VPN
Policy Granularity Policy Granularity
Xen Hypervisor
Xen Security Modules
VT-d TXT
Intel vPro Hardware AES-
VT-x
NI

35. X E N S EC U R I T Y I N T H E C LO U D
• Xen-based server products (such as XCP) will start
making use of advanced security features
o Driver domains, stub domains, service domains
o Xen Security Modules (XSM)
o Dom0 Disaggregation
• Driver domains, stub domains, services domains, and
XSM already in upstream Xen
• Dom0 disaggregation code coming to xen-unstable

36. S U M M A RY: W H Y X E N ?
• Designed for the Cloud : many advantages for cloud use!
o Resilience, robustness & scalability
o Security and architecture: small, clean attack surface, isolation
properties, and advanced security features
• Widely used in production by public cloud providers
• XCP & XAPI
o Ready for use with cloud orchestration stacks
o XCP and Project Kronos: flexibility and choice
o Lots of additional improvements for cloud still to come
• Flexibility and choice of usage models
• Open Source with a large community and eco-system

37. THANK YOU!

38. X E N R ES O U RC ES
• IRC: ##xen @ FREENODE
• Mailing List: xen-users & xen-api
• Wiki: wiki.xen.org
o Beginners & User Categories
• Excellent XCP Tutorials
o A day worth of material
@xen.org/community/xenday11

39. R E F E R E N C ES
 Xen and the Art of Repeated Research, Clarkson
University
 www.clarkson.edu/class/cs644/xen/files/repeatedxen-
usenix04.pdf
 XenAPI (XAPI) Classes
 xen.org/files/XenCloud/ocamldoc/apidoc

40. H OW TO CO N T R I B U T E
• Same process as for Linux Kernel
o Same license: GPLv2
o Same roles:
Developers, Maintainers, Committers
o Contributions by patches + sign-off
(Developer Certificate of Origin)
o Details @
xen.org/projects/governance.html

41. CO M M U N I T Y & ECO SYST E M M A P
xen.org/community/projects
Resear
A
Hostin ch
Xen
g
D
Vendor
Project
s
s D
XCP XCP
Produc # Project
s
ts
s
Xen Consul
Produc ting
ts Consul People
ting
Firms