Technical overview on deployment scenarios and architecture. Given at Bangalore and Hyderabad CloudStack meet ups in October.

1. Build your own Infrastructure Cloud
with Apache CloudStack
Kevin Kluge
Vice President, Cloud Platforms Group, Citrix Systems Inc.
 Kevin is an expert in Large Scale Systems and Infrastructure Clouds
and manages the Cloud Platforms product group at Citrix.
 Previously Vice President, Engineering at Cloud.com, acquired by
Citrix in 2011.
 Held engineering leadership positions at
Yahoo!, Zimbra, Corvigo, Openwave Systems, and Onebox.com.
 Kevin has a MS and BS in Computer Science from Stanford
University.

2. Apache CloudStack (Incubating)
An Introduction
Kevin Kluge
Apache CloudStack Committer

3. Use CloudStack to build IaaS clouds (like EC2)
• Create VMs, disks • Java based
networks, network services • Scalable
• Self service • Many vendor integrations
• Meter usage • Native and EC2 API

4. How did Amazon build EC2?
Amazon eCommerce Platform
AWS API (EC2, S3, …)
Amazon Orchestration Software
Open Source Xen Hypervisor
Commodity Commodity
Networking
Servers Storage

5. How can you build your cloud?
Amazon eCommerce Platform
Your Portal (Optional)
AWS API (EC2, S3, …)
CloudStack or AWS API
CloudStack Orchestration Software
Amazon Orchestration Software
ESXi, KVM, XenServer/XCP, OVM
Open Source Xen Hypervisor
Networking Servers Storage

6. Project history
• 2008/2009: closed-source development
• First deployments in late 2009
• May 2010: ~98% open source as GPLv3 (open core)
• August 2011: 100% open source GPLv3
• April 2012: Switch to Apache License v2
• Submit code to Apache Software Foundation

7. Project current state
• In incubation within Apache Software Foundation
• Imminent first release!
• Bugs and wiki mostly moved to ASF infra
• Mailing list traffic moved to ASF infra
• Many non-Citrix contributors, committers, and PPMC
members

8. Yes, the ASF is great
Enter ASF

9. IaaS Cloud Concepts

10. Virtualization alone does not make a cloud
Server Virtualization Cloud
Built for traditional enterprise Designed around big data,
apps & client-server compute massive scale & next-gen apps
• Scale-up (pool-based resourcing) • Scale-out (horizontal resourcing)
• IT management-centric • Autonomic management
• 1 administrator for 100’s of servers • 1 administrator for 1,000’s of servers
• Proprietary vendor stack • Open, value-added stack

11. Clouds must reliably run all types of workloads
Traditional Workload Cloud Workload
Expect reliability Design for failure
Back-up everything Ephemeral resources
HA, Fault tolerance Multi-site redundancy
Admin control recovery Self-service recovery
Think Server Virtualization Think Amazon Web Services

12. Embrace traditional and extend to Cloud-era
Cloud-era Workloads Traditional Workloads
CloudStack Mgmt
Server Traditional Zone
vSphere
Enterprise Networking (e.g., VLAN)
Cloud-era Cloud-era Cloud-era
Availability Availability Availability
Zone Zone Zone
ESXi ESXi ESXi
Cluster Cluster Cluster
Object Storage Enterprise Storage (e.g., SAN)

13. Apache CloudStack
Management Server
Traditional Traditional Cloud-era Traditional Cloud-era
Availability Availability Availability Availability Availability
Zone Zone Zone Zone Zone

14. Object store is critical for Cloud-era workloads
Amazon-Style Cloud
CloudStack
Mgmt. Server • Workloads are distributed across
availability zones
• No guarantee on zone reliability
• DBs and Templates snapped to
Availability Availability Availability object store.
Zone Zone Zone
• For small failures, recreate instance
in same zone
• For DR, recreate instance in different
Object Store zone
• Dramatically less expensive

15. Deployment and Software
Architecture

16. • Single Management Server can
Data Center 1 Data Center 2 manage multiple zones
Data Center 2
Management Data Center 3
Servers
Zone 2
• Zones can be geographically
Zone 2 distributed but low latency links
Zone 3 are expected for better
Zone1
Zone 4 3
Zone
performance
• Single MS node can manage up to
Data Center 2
Data Center 2 10K hosts.
Data Center 2
Zone 2
Zone 2
• Multiple MS nodes can be
Zone 2 Zone 3 deployed as cluster for scale or
Zone 3 redundancy
Zone 3

17. Standby Mgmt
Cloud-era zone deployment Server Cluster
Admin Internet
Availability Zone 2
Primary Mgmt
Server Cluster
Primary
Router
MySQL
Backup Load Balancer
MySQL
L3 Core Switch
Top of Rack
Switch
Object Store
Servers
… … … … …
Availability Zone 1
Pod 1 Pod 2 Pod 3 Pod N

18. Internet
Traditional zone
deployment Object
Load Balancer
Store Core Switch
… Aggregation
Switch
TOR Switch
Compute
Nodes
NFS
Primary
10Gbps 1Gbps 10Gbps 1Gbps 10Gbps 1Gbps Storage
Storage Guest Storage Guest Storage Guest
& Mgmt & Mgmt & Mgmt
Pod 1 Pod 2 Pod 200

20. Management Server
XAPI HTTP
vCenter Agent Agent
XenServer
KVM OVM
XCP ESX
• XS 5.6, 5.6FP1, 5.6 • ESX 4.1, 5.0 • RHEL • OVM 2.2
SP2, 6.0.2, XCP 1.1 • Full Snapshots 6.0, 6.1, 6.2, Ubuntu • No Snapshots
• Incremental Snapshots • VMDK 12.04 • RAW
• VHD • NFS, iSCSI, FC & Local disk • Full Snapshots (not live) • NFS & iSCSi
• NFS, iSCSI, FC & Local disk • Storage over-provisioning: • QCOW2 • No storage over-
• Storage over- NFS, iSCSI • NFS, iSCSI & FC provisioning
provisioning: NFS • Storage over-
provisioning: NFS

21. Mgmt Server CPU Util.
Seconds to deploy
25,000 …. to …. 30,000 VMs 0 …. to …. 30,000 VMs
• Simulator developed to test massive scale
• Four Management Servers can manage 30,000 hosts
• Scale to hundreds of thousands of hosts possible with
multiple management server clusters (regions)

22. Features

23. Compute Hypervisor
XCP/XS VMware Oracle VM KVM Bare metal
Storage Block & Object
Fiber Object
Local Disk iSCSI NFS
Channel Stores
Network Network & Network Services
Network Load
Isolation Firewall VPN
Type balancer

24. Users
Change
VM Operations Console Access VM Status
Service Offering
Start
• CPU Utilized 2 CPUs 4 CPUs
Stop
1 GB RAM 4 GB RAM
Restart • Network Read 20 GB 200 GB
Destroy 20 Mbps 100 Mbps
• Network Writes

25. Add / Delete VM 1
Volumes Volume
Create Templates Volume Template
from Volumes
Schedule Now
Hourly Weekly
Snapshots Daily Monthly
View Snapshot ….
History 12/2/2012 7.30 am 2/2/2012 7.30 am

26. Specify Resource Levels Configure Properties Define Scope
Compute Disk Network
Name Name Name
CPU Cores Custom Disk Size Network Rate
CPU (MHz) Disk Size (GB) Redundant VR
Memory (MB) Storage Tag Firewall
Host Tag Storage Tag Load balancer
Enable HA Public Public
CPU Cap
Public

27. Resources
Domain VMs, IPs, Snapshots…
• Domain is a unit of isolation that
Org A represents a customer org, business
Admin unit or a reseller
Domain
• Domain can have arbitrary levels of
Reseller A
sub-domains
Admin
Resources
Sub-Domain
Org C
VMs, IPs, Snapshots… • A Domain can have one or more
Admin accounts
Account
• An Account represents one or more
Group A
users and is the basic unit of
Account
isolation
Group B
• Admin can limit resources at the
User 1
Account or Domain levels
User 2

28. • Create Networks and attach
VMs
• Acquire public IP address for
NAT & load balancing
• Control traffic to VM using
ingress and egress firewall
rules
• Set up rules to load balance
traffic between VMs

29. Network offering
• Provides cloud operator
defined service features
• Isolation
• Load Balancing
• VPN
• Firewall
• Supports Physical Devices
• NetScaler
• F5 BIG-IP Pod 1 Pod 1 Pod N
• Juniper SRX Zone 1 Zone N

30. Network Services Managed Externally Network Services Managed by CS
Public Network
65.11.0.0/16Security Security
Group 1 Public Group 1
65.11.1.2 Guest Network/Internet 65.11.1.2 Guest
VM 1 VM 1
65.11.1.3 Physical 65.11.1.3 Guest
Guest
Load VM 2
VM 2 Balancer
65.11.1.4
EIP, 65.11.1.4 Guest
Guest
VM 3 ELB VM 3
65.11.1.5 Guest
65.11.1.5 Guest
VM 4 VM 4
CS CS
Security DHCP, Virtual Security
DHCP, Virtual
Group 2 DNS Router Group 2
DNS Router

31. CS Virtual Router provides Network Services External Devices provide Network Services
Guest Virtual Network 10.0.0.0/8 Guest Virtual Network 10.0.0.0/8
Public VLAN 100 Public VLAN 100
Network/Internet Network/Internet
Guest Public IP Private IP Guest
10.1. VM 1 6.37.1.12 Juniper 10.1.1.111 10.1. VM 1
CS Gateway 1.1 SRX 1.1
6.37..1.11 Firewall
Virtual address Guest Guest
Router 10.1.1.1 10.1. VM 2 10.1. VM 2
Private IP
DHCP, DNS 1.3 Physical 10.1.1.112 1.3
NAT Guest Load Guest
Load Balancing 10.1. VM 3 Public IP Balancer 10.1. VM 3
VPN 1.4 6.37.1.11 1.4
Guest Guest
10.1. VM 4 10.1. VM 4
1.5 1.5
CS
DHCP, Virtual
DNS Router

32. Layer-2 Layer-3
Isolation VLAN/SDN Security Groups
Performance Better Better
Network setup Moderate Easy
Support broadcast Yes No
Scalability Good Best
Interoperability with Good Poor
physical servers

33. CloudStack storage
Primary Storage
• Configured at Cluster-level. Close to hosts
for better performance
• Stores all disk volumes for VMs in a cluster L3 switch
• Cluster can have one or more primary
storages Pod 1 L2 switch
• Local disk, iSCSI, FC or NFS Secondary
Cluster 1 Storage
Host 1
Secondary Storage Primary
Storage
• Configured at Zone-level Host 2
• Stores all Templates, ISOs and Snapshots
• Zone can have one or more secondary
storages
• NFS, OpenStack Swift, others coming

34. Futures

35. Apache CloudStack CloudStack API API
Apache CloudStack CloudStack
Apache API API
Apache
Firewall
Hypervisor Baremetal Switches Security Storage
Load Bal

36. Futures
• Object storage and SDN short term
• Blade orchestration
• Region support
• Additional hypervisors (need some container support)
• Code modularity improvements (OSGI?)
• App-specific integration (Hadoop?)
• Improved CLI
• Additional API support (Google, evolving standards)

37. The future needs you!
Project web site: http://incubator.apache.org/projects/cloudstack.html
Mailing lists:
cloudstack-dev-subscribe@incubator.apache.org
cloudstack-users-subscribe@incubator.apache.org
IRC: #CloudStack on irc.freenode.net
Join your local CloudStack group!

38. Thank You