This document discusses hypervisor choices for CloudStack and their pros and cons. It analyzes KVM, Linux containers, Hyper-V, vSphere, and XenServer. KVM is highlighted as low cost but lacks advanced storage options and features. Linux containers are also low cost but require KVM. Hyper-V has unlimited Windows licenses but limited advanced features. vSphere supports many apps and OSes but has high licensing fees. XenServer provides security groups, large VLAN support and integrates well with CloudStack. The best choice depends on the workload and use case.

1. Tim Mackey – XenServer Community Manager and Evangelist
刘宇超 ——CloudStack中国社区
CloudStack 4.3 Hypervisor 的选型
Understanding the choices available
Build a Cloud Day – SCALE 12x

2. 建立一个成功的云
我们试图实现哪些关键点？

3. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
提供的服务
• 清晰定义——你想要提供的：
ᵒ 应用的类型
ᵒ 谁访问，谁拥有
ᵒ 访问类型
• 定义——需要管理的模板：
ᵒ 操作系统
ᵒ Patching 需求
• 定义——合规性和可用性的预期范围
ᵒ 监控和备份的

4. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
定义租户的需求
• 数据贴近用户（Department data local to department）
ᵒ 应用数据储存的位置
• 数据和服务的隔离
ᵒ VM 迁移和host HA
ᵒ Network 服务
• PII/PCI加密
ᵒ 数据存储位置的关键key的保护
ᵒ 针对云的特点进行加密
• 提供Showback
ᵒ 不仅是计量, 合规和审计

5. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
虚拟化架构
• 由提供的服务来决定Hypervisor
ᵒ 不要基于某个“标准”选择hypervisor
ᵒ 了解虚拟化的实际cost
ᵒ 多hypervisor混合 “OK”
ᵒ Bare metal 也是一种hypervisor
• 是否应该“池化”资源
ᵒ 是否真的需要池化资源？
ᵒ 云管理方案是否更好？
ᵒ 共享存储的实际cost
• 由hypervisor决定主存储（Primary storage）
• 由方案决定辅助存储（Template storage）
ᵒ NFS是个典型的低成本的选择

6. 主要的选择….

7. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Networking StorageCompute
XenServer
Xen Project Hypervisor
Standard Linux Distribution (dom0)
qemudrivers
xapi
Guest
Driver front
Driver back
Guest
Driver front
patches

8. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Microsoft Hyper-V
Networking StorageCompute
Hyper-V Hypervisor
Standard Windows Server (parent partition)
drivers
WMI
Guest
VMBus
Guest
Virtual SP
VMBus VMBus
Devices Devices

9. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
vSphere 5.1 – Managed by vCenter
vmkernel
Guest
Virtual driver
vSCSI
Guest
Virtual driver
Task
Scheduler
Service
Console
vmklinux
vCenter
drivers
vNIC
Networking StorageCompute

10. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
KVM (Linux + KVM only)
Standard Linux Distribution
qemudrivers
Guest
Virtual driver
virtio
Guest
Virtual driver
KVM Module
libvirt
agent
Networking StorageCompute

11. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Linux Containers
Standard Linux Distribution
Namespace
Container
Namespace
Container
KVM Module
libvirt
agentCgroups
Cgroup Cgroup
Namesspaces
Networking StorageCompute

12. 网络的定义

13. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Flat Network – Basic Layer 3 Network
Option XenServer vSphere KVM LXC Hyper-V
Security Groups Yes- bridge - Yes Yes Yes
IPv6 - - Yes Yes -
Multiple IPs per NIC Yes Yes Yes Yes Yes
Nicira NVP Yes - Yes - -
BigSwitch VNS Yes - Yes - -
65.11.1.2
65.11.1.3
65.11.1.4
65.11.1.5
Public Network
65.11.0.0/16
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
DHCP,
DNS
CloudStack
Virtual Router
Security Group 1
Security Group 2

14. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
VLANs for Private Cloud
Option XenServer vSphere KVM LXC Hyper-V
Max VLANs 800 254 1024 1024 4094
IPv6 - - Yes Yes -
Multiple IPs
per NIC
Yes Yes Yes Yes Yes
VPC Yes Yes Yes - Yes
Nicira NVP Yes - Yes - -
BigSwitch VNS Yes - Yes - -
MidoKura - - Yes - -
NetScaler Yes Yes Yes - Yes
F5 BigIP Yes Yes Yes - Yes
Juniper SRX - Yes Yes - Yes
Cisco VNMC - Yes - - -
10.1.1.1
10.1.1.3
10.1.1.4
10.1.1.5
Public
Network/Internet
Guest Virtual Network 10.0.0.0/8
VLAN 100
DHCP, DNS
NAT
Load Balancing
VPN
Public IP
65.37.14.1
Gateway
10.1.1.1
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
CloudStack
Virtual
Router

15. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Beyond the VLAN – Network Virtualization
Option XenServer vSphere KVM LXC Hyper-V
OVS GRE tunnels Yes - - - -
Nicira STT tunnel Yes Yes Yes - -
MidoNet - - Yes - -
VXLAN - Yes Yes - -
NVGRE - - - - -
Nexus 1000v - Yes - - -
Juniper Contrail Yes - - - -
Palo Alto Yes Yes Yes - -

16. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Virtual Private Cloud and nTier Applications
Feature XenServer vSphere KVM LXC Hyper-V
PVLAN Yes - ovs Yes ovs - -
Web
App
DB
Router
DC1
DC2
DC3
DC4
DC5
DC6
VLAN 1
VLAN 2
VLAN 3
S2S VPN
Private
GW

17. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Delivering specific network services
• KVM
ᵒ IPv6
ᵒ Security groups
ᵒ Large quantity of VLANs
• vSphere
ᵒ VXLAN required vSphere Enterprise Plus
ᵒ Cisco Nexus 1000v 和 ASA 1000v required vSphere Enterprise Plus
• XenServer
ᵒ Security groups
ᵒ Large quantity of VLANs
ᵒ Juniper Contrail

18. Instances 需要一个家…
存储，存储，还是存储

19. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Primary Storage 选项
Feature XenServer vSphere KVM LXC Hyper-V
Local storage Yes Yes Yes Yes Yes
NFS Yes Yes Yes Yes -
SMB - - - - SMB3
Single path iSCSI Yes Yes Yes - -
Multipath iSCSI PreSetup - - - -
Direct array - VAAI - - -
Shared Mount - - Yes Yes -
Template format VHD OVA QCOW2 TAR VHD
SolidFire Plugin Yes Yes Yes - -
NetApp Plugin Yes Yes Yes - -
Zone wide - Yes Yes - -
Ceph RBD - - Yes - -
Clustered LVM - - Yes - -
Cluster
Host
Host
Primary Storage

20. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Secondary Storage 选项
Option XenServer vSphere KVM LXC Hyper-V
NFS Yes Yes Yes Yes -
Swift(1) Yes Yes Yes Yes -
S3 compatible (2) Yes Yes Yes Yes -
SMB - - - - Yes
(1) Requires NFS staging area
(2) Can be region wide, but must not have NFS secondary storage in zone
Zone
Secondary Storage
Pod
Cluster
Host
Host
Primary Storage

21. 虚拟化核心能力
重要的限制和功能

22. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
CloudStack Features
Feature XenServer vSphere KVM LXC Hyper-V
Disk IO Statistics Yes - Yes - Yes
Memory Overcommit Yes (4x) Yes - - -
Dedicated resources Yes Not with HA/DRS Yes - Yes
Disk IO throttling - - Yes Yes -
Disk snapshot (running) Yes Yes - - -
Disk snapshot (pluggable) Partial Partial - - -
Disk snapshot (Stopped) Yes Yes Yes - Yes
Memory snapshot Yes Yes Yes - -
Zone wide primary storage - Yes Yes Yes -
Resize disk Offline Online Grow Online - -
High availability CloudStack Native CloudStack - CloudStack
CPU sockets 6.2 and higher Yes Yes Host count Yes
Affinity groups Yes Yes Yes - Yes

23. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
XenServer 6.2
Feature
Source code model Open Source (GPLv2)
Maximum VM Density 650 (Linux)
CloudStack VM Density 500
CloudStack integration Direct XAPI calls
Maximum native cluster size 16
Maximum pRAM 1 TB
Largest VM 16vCPU/128GB
Windows Operating System All Windows supported by Microsoft
Linux Operating Systems RHEL, CentOS, Debian, Ubuntu, SLES, OEL
Advanced features supported ovs, Storage XenMotion, DMC

24. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
vSphere 5.5 ( 5.5 support added in ACS 4.3)
Feature
Source code model Proprietary
Maximum VM Density 512
CloudStack VM Density 128
CloudStack integration vCenter
Maximum native cluster size 32
Maximum pRAM 4 TB
Largest VM 64 vCPU/1TB
Windows Operating Systems DOS, All Windows Server/Client
Linux Operating Systems Most
Advanced features supported HA, DRS, vDS, Storage vMotion

25. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
KVM (RHEL/CentOS 6.5 and Ubuntu 12.04)
Feature
Source code model Open Source (GPLv2)
Maximum VM Density 10 times the number of pCores
CloudStack VM Density 50
CloudStack integration CloudStack Agent (libvirt)
Maximum native cluster size No native cluster support
Maximum pRAM 2 TB
Largest VM 160 vCPU/2TB
Windows Operating Systems Windows XP and higher
Linux Operating Systems Varies
Advanced features supported None

26. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Linux Containers
Feature
Source code model Open Source (GPLv2)
Maximum container Density 6000 (theoretical)
CloudStack container Density 50
CloudStack integration CloudStack Agent (libvirt), requires KVM for SVMs
Maximum native cluster size N/A
Maximum pRAM 2 TB
Largest container 2 TB
Windows Operating Systems N/A
Linux Operating Systems Kernel compatible distros

27. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Microsoft Hyper-V
Feature
Source code model Proprietary
Maximum VM Density 1024
CloudStack VM Density 1024
CloudStack integration CloudStack Agent (C# calling WMI)
Maximum native cluster Size 64
Maximum pRAM 4 TB
Largest VM 64 vCPU/1TB
Windows Operating Systems All Windows supported by Microsoft
Linux Operating Systems RHEL, CentOS, Debian, Ubuntu, SLES, OEL
Advanced features supported None

28. 选择 “best one”
从hypervisor的视角来看

29. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
KVM
• 主要优势：
ᵒ 低成本
ᵒ 相似的管理模式
ᵒ 社区开发积极，可支持广泛的CloudStack 功能
• Cloud中的使用案例：
ᵒ 以Linux 为中心的工作负载
ᵒ Dev/test clouds
ᵒ Web类型的托管
ᵒ 基于SDN的提高云租户的密度
• 弱点：
ᵒ 需要安装CloudStack libvir t agent
ᵒ 有限的原生存储可选方案
ᵒ 高级的原生功能并未支持

30. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Linux Containers
• 主要优势：
ᵒ 低成本
ᵒ 相似的管理模式
• Cloud中的使用案例：
ᵒ Dev/test clouds
ᵒ Web 应用的托管
• 弱点：
ᵒ 需要安装CloudStack libvir t agent
ᵒ 需要KVM 运行system VMs
ᵒ 高级的原生功能并未支持
ᵒ 在CloudStack 4.2首次加入

31. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Microsoft Hyper-V
• 主要优势：
ᵒ 无限的Windows Ser ver VM 许可证
ᵒ Windows 相似的管理方式
• Cloud中的使用案例：
ᵒ Windows 和Linux 工作负载
ᵒ Dev/test clouds
ᵒ .Net web 应用的托管
ᵒ Desktop as a Ser vice clouds
• 弱点：
ᵒ 高级的原生功能支持较少
ᵒ 在CloudStack 4.3首次加入

32. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
vSphere
• 主要优势：
ᵒ 支持广泛的应用和操作系统
ᵒ vSphere 管理员容易的进行池化管理
ᵒ 有大量的合作伙伴形成生态圈
ᵒ 不少CloudStack 功能已经直接实现了
ᵒ 通过vCenter直接进行管理整合
• Cloud中的使用案例：
ᵒ 私有企业云
ᵒ Dev/test clouds
• 弱点：
ᵒ vSphere 前期许可证和后续支持费用大
ᵒ vCenter 集成需要冗余设计
ᵒ 每个Zone是单独vSphere中的数据中心的模式

33. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
XenServer
• 主要优势：
ᵒ 低成本
ᵒ 社区开发积极，可支持广泛的CloudStack功能
ᵒ 大量CloudStack 安装使用
ᵒ 直接通过XAPI toolstack 整合
• Cloud中的使用案例：
ᵒ Linux 为中心的工作负载
ᵒ Dev/test clouds
ᵒ Web 托管
ᵒ Desktop as a Ser vice clouds
ᵒ 高密度VM和安全的租户隔离
• 弱点：
ᵒ 高级的原生功能支持较少

34. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
多Hypervisor的支持又是怎样的？
• Networking
ᵒ 确保匹配好网络标签
ᵒ 选择多个hyper visors 之间交集的部分作为拓扑
• Storage
ᵒ system VMs 都有特定的hyper visor类型
ᵒ 主存储都要受zone的限制
• Operations
ᵒ vSphere Datacenter 不能跨 zones
ᵒ Hyper-V 暂时还不可以和其他hyper visors 混合在一个zone内
ᵒ HA不能实现在 hyper visor 间的迁移
ᵒ 基于cluster/pod 层级进行Capacity 规划更难一些

35. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Tying it all Together
1. Define success criteria
2. Select a topology which works
3. Decide on storage options
4. Define supported configurations
5. Select preferred hypervisor(s)
6. Validate matrix
7. Build your Cloud

36. © Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy