• Save
Evolution of CloudStack Architecture (Collab 2012)
Upcoming SlideShare
Loading in...5
×
 

Evolution of CloudStack Architecture (Collab 2012)

on

  • 3,658 views

 

Statistics

Views

Total Views
3,658
Views on SlideShare
2,796
Embed Views
862

Actions

Likes
10
Downloads
0
Comments
0

5 Embeds 862

http://cloudierthanthou.wordpress.com 823
https://twitter.com 20
http://www.techgig.com 16
http://moderation.local 2
http://digg.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Evolution of CloudStack Architecture (Collab 2012) Evolution of CloudStack Architecture (Collab 2012) Presentation Transcript

  • The Evolution of CloudStack The roads taken and not taken @chiradeep Collab 2012Saturday, December 1, 2012 1
  • We remember the most painful and pleasurable parts of an experience, not the durationSaturday, December 1, 2012 2
  • In the BeginningSaturday, December 1, 2012 3
  • Circa 2008Saturday, December 1, 2012 4
  • Circa 2008 !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 Open Source Xen (custom) !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 Open Source Xen Security Groups (custom) !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 Open Source Xen (custom) Security Groups EBS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 Open Source Xen (custom) Security Groups EBS S3 !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 Proprietary Orchestration Services Open Source Xen (custom) Security Groups EBS S3 !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 EC2 API Proprietary Orchestration Services Open Source Xen (custom) Security Groups EBS S3 !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 EC2 EBS API API Proprietary Orchestration Services Open Source Xen (custom) Security Groups EBS S3 !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 EC2 EBS S3 API API API Proprietary Orchestration Services Open Source Xen (custom) Security Groups EBS S3 !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • Circa 2008 EC2 EBS S3 API API API Proprietary Orchestration Services Secret Sauce Open Source Xen (custom) Security Groups EBS S3 !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 4
  • The  Virtual  Datacenter  OS  allows  businesses  to   efficiently  pool  all  types  of  hardware  resources  -­‐   servers,  storage  and  network  –  into  an  aggregated   on-­‐premise  cloud -­‐  VMWare  Press  Release  9/2008Saturday, December 1, 2012 5
  • The  Virtual  Datacenter  OS  allows  businesses  to   efficiently  pool  all  types  of  hardware  resources  -­‐   servers,  storage  and  network  –  into  an  aggregated   on-­‐premise  cloud -­‐  VMWare  Press  Release  9/2008 Eucalyptus  is  the  only  cloud  architecture  to  support   the  same  applicaKon  programming  interfaces  (APIs)   as  public  clouds,  and  today  Eucalyptus  is  fully   compaKble  with  the  Amazon  AWS  public  cloud   infrastructure.   Eucalyptus  Systems  Press  Release  4/2009Saturday, December 1, 2012 5
  • 2.0 AWS 2.2 3.0 Quality 4.X Prototype 1.0 GA Refactor Compatibility Refactor Improvements Refactor 2008 2009 2010 2011 2012 Sept 2008: Nov 2009: May 2010: July 2011: April 2012: VMOps CloudStack Cloud.com Citrix Apache Founded 1.0 GA Launch & Acquires CloudStack CloudStack Cloud.com 2.0 GASaturday, December 1, 2012 6
  • Circa 3/2009Saturday, December 1, 2012 7
  • Circa 3/2009 !"#$%&(Saturday, December 1, 2012 7
  • Circa 3/2009 !"#$%&( !"#$%&(Saturday, December 1, 2012 7
  • Circa 3/2009 !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 7
  • Circa 3/2009 Open Source Xen (custom) !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 7
  • Circa 3/2009 Open Source Xen Virtual Networks (custom) !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 7
  • Circa 3/2009 Open Source Xen (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 7
  • Circa 3/2009 Proprietary Control Plane Open Source Xen (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 7
  • Circa 3/2009 Proprietary Orchestration Services Proprietary Control Plane Open Source Xen (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 7
  • Circa 3/2009 GUI Proprietary Orchestration Services Proprietary Control Plane Open Source Xen (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 7
  • Circa 3/2009 GUI Proprietary Orchestration Services Proprietary Control Plane Open Source Xen (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 8
  • Circa 3/2009 GWT GUI Proprietary Orchestration Services Proprietary Control Plane Open Source Xen (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 8
  • Circa 3/2009 GWT GUI Java App Server Proprietary Orchestration Services Proprietary Control Plane Open Source Xen (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 8
  • Circa 3/2009 GWT GUI Java App Server Proprietary Orchestration Services JSON over TCP Proprietary Control Plane Open Source Xen (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 8
  • Circa 3/2009 GWT GUI Java App Server Proprietary Orchestration Services JSON over TCP Proprietary Control Plane NFS (ZFS on Open Source Xen OpenSolaris) (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 8
  • Circa 3/2009 GWT GUI Java App Server Proprietary Orchestration Services JSON over TCP Proprietary Control Plane UDP Tunnels NFS (ZFS on Open Source Xen OpenSolaris) (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 8
  • Circa 3/2009 GWT GUI Java App Server Proprietary Orchestration Services JSON over TCP Proprietary Control Plane UDP Tunnels NFS (ZFS on Open Source Xen OpenSolaris) Agent (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 8
  • Circa 3/2009 GWT GUI Java App Server Proprietary Orchestration Services JSON over TCP Proprietary Control Plane UDP Tunnels NFS (ZFS on Open Source Xen Agent OpenSolaris) Agent (custom) Virtual Networks NFS !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 8
  • Availability Zone ArchitectureSaturday, December 1, 2012 9
  • Availability Zone Architecture L3 coreSaturday, December 1, 2012 9
  • Availability Zone Architecture End users DC Edge L3 coreSaturday, December 1, 2012 9
  • Availability Zone Architecture End users DC Edge L3 core Access Sw Hypervisor (Xen) Primary Storage ZFS PodSaturday, December 1, 2012 9
  • Availability Zone Architecture End users DC Edge L3 core Access Sw Hypervisor (Xen) Primary Storage ZFS Pod PodSaturday, December 1, 2012 9
  • Availability Zone Architecture End users DC Edge L3 core Access Sw Hypervisor (Xen) Primary Storage ZFS Pod Pod PodSaturday, December 1, 2012 9
  • Availability Zone Architecture End users DC Edge L3 core Access Sw Hypervisor (Xen) Primary Storage ZFS Pod Pod Pod PodSaturday, December 1, 2012 9
  • Availability Zone Architecture End users DC Edge L3 core Access Sw Hypervisor (Xen) Primary Storage ZFS Pod Pod Pod PodSaturday, December 1, 2012 9
  • Availability Zone Architecture End users DC Edge L3 core Access Sw Hypervisor (Xen) Primary Storage ZFS Pod Pod Pod Pod PodSaturday, December 1, 2012 9
  • Availability Zone Architecture End users Admin/User API CloudStack DC Edge MySQL L3 core Access Sw Hypervisor (Xen) Primary Storage ZFS Pod Pod Pod Pod PodSaturday, December 1, 2012 9
  • Availability Zone Architecture End users Admin/User API CloudStack DC Edge MySQL ECMP/BGP L3 core Access Sw Hypervisor (Xen) Primary Storage ZFS Pod Pod Pod Pod PodSaturday, December 1, 2012 9
  • Multiple zones CloudStack MySQL Zone1Saturday, December 1, 2012 10
  • Multiple zones CloudStack MySQL Internet Zone1 Zone2 Zone3Saturday, December 1, 2012 10
  • Network VirtualizationSaturday, December 1, 2012 11
  • Network Virtualization Tenant    1  Virtual  Network  10.1.1.0/24 Tenant  1    10.1.1.2 Gateway  address   VM  1 10.1.1.1 Tenant  1    10.1.1.3 VM  2 Tenant  1    10.1.1.4 VM  3 Tenant  1    10.1.1.5 VM  4Saturday, December 1, 2012 11
  • Network Virtualization Tenant    1  Virtual  Network  10.1.1.0/24 Tenant  1    10.1.1.2 Gateway  address   VM  1 10.1.1.1 Tenant  1   Tenant  1    10.1.1.3 Virtual  Router VM  2 NAT Internet DHCP Tenant  1    10.1.1.4 FW VM  3 Load  Balancing Tenant  1    10.1.1.5 VM  4Saturday, December 1, 2012 11
  • Network Virtualization Tenant    1  Virtual  Network  10.1.1.0/24 Public  IP  address   Tenant  1    10.1.1.2 65.37.141.11 Gateway  address   VM  1 65.37.141.36 10.1.1.1 Public  Network Tenant  1   Tenant  1    10.1.1.3 Virtual  Router VM  2 NAT Internet DHCP Tenant  1    10.1.1.4 FW VM  3 Load  Balancing Tenant  1    10.1.1.5 VM  4Saturday, December 1, 2012 11
  • Network Virtualization Tenant    1  Virtual  Network  10.1.1.0/24 Public  IP  address   Tenant  1    10.1.1.2 65.37.141.11 Gateway  address   VM  1 65.37.141.36 10.1.1.1 Public  Network Tenant  1   Tenant  1    10.1.1.3 Virtual  Router VM  2 NAT Internet DHCP Tenant  1    10.1.1.4 FW VM  3 Load  Balancing Tenant  1    10.1.1.5 VM  4 Tenant  2  Virtual  Network  10.1.1.0/24 Public  IP  address   65.37.141.24 Gateway  address   Tenant  2    10.1.1.2 10.1.1.1 VM  1 65.37.141.80 Tenant  2   Tenant  2    10.1.1.3 Virtual  Router   VM  2 Appliance FW NAT Tenant  2    10.1.1.4 DHCP VM  3Saturday, December 1, 2012 11
  • Virtual Machine Placement End users CloudStack DC Edge MySQL L3/L2 core Hypervisor (Xen) Primary Storage ZFS Pod Pod Pod Pod PodSaturday, December 1, 2012 12
  • Network Virtualization - why? AWS-style security groups “unfamiliar” Adopt traditional L2 model for end-users VLANs do not scale use network virtualization to realize thisSaturday, December 1, 2012 13
  • Network Virtualization UDP encapsulation of Ethernet Frames similar to VXLAN (128-bit address vs 24-bit for VXLAN) Driver in dom0 Virtual Router to provide edge services including Load Balancing Hand-rolled Fedora 8 para-virtualizedSaturday, December 1, 2012 14
  • ZFS (Volume Service) ZFS for highly scalable, reliable storage RAID-Z SSD cache NFS for shared storage Hand-rolled version of OpenSolaris 2008.11Saturday, December 1, 2012 15
  • OrchestrationSaturday, December 1, 2012 16
  • Orchestration JVMSaturday, December 1, 2012 16
  • Orchestration JVM MySQLSaturday, December 1, 2012 16
  • Orchestration JVM Message Bus MySQLSaturday, December 1, 2012 16
  • Orchestration JVM Front- end Message Bus MySQLSaturday, December 1, 2012 16
  • Orchestration JVM VM Front- Orchest end rator Message Bus MySQLSaturday, December 1, 2012 16
  • Orchestration JVM VM Network Front- Orchest Orchestr end rator ator Message Bus MySQLSaturday, December 1, 2012 16
  • Orchestration JVM VM Network Storage Front- Orchest Orchestr Orchest end rator ator rator Message Bus MySQLSaturday, December 1, 2012 16
  • Orchestration JVM VM Network Storage Host/ Front- Orchest Orchestr Orchest Resource end rator ator rator Manager Message Bus MySQLSaturday, December 1, 2012 16
  • Orchestration JVM VM Network Storage Host/ Front- Orchest Orchestr Orchest Resource end rator ator rator Manager Message Bus Hypervi Hypervi Hypervisor sor Agent sor (JVM) MySQLSaturday, December 1, 2012 16
  • Orchestration JVM VM Network Storage Host/ Front- Orchest Orchestr Orchest Resource end rator ator rator Manager Message Bus Hypervi Hypervi Hypervisor Storage Storage Storage sor Agent sor Agent Agent Agent (JVM) (JVM) MySQLSaturday, December 1, 2012 16
  • Orchestration JVM VM Network Storage Host/ Front- Orchest Orchestr Orchest Resource end rator ator rator Manager Message Bus Hypervi Hypervi Hypervisor Storage Storage Storage Console Console sor Agent sor Agent Agent Agent Agent Agent (JVM) (JVM) MySQLSaturday, December 1, 2012 16
  • Orchestration JVM VM Network Storage Host/ GWT-RPC Front- Browser Orchest Orchestr Orchest Resource end rator ator rator Manager Message Bus Hypervi Hypervi Hypervisor Storage Storage Storage Console Console sor Agent sor Agent Agent Agent Agent Agent (JVM) (JVM) MySQLSaturday, December 1, 2012 16
  • Reject Pile (2009) Local disks of hypervisors presented as network block storage with network RAID Too complex (sheepdog-like) Split-brain issues DRBD Too complex to automate recovery Commercial Arrays Limited volumes, limited snapshotsSaturday, December 1, 2012 17
  • Reject Pile (2009) Security Groups Not well understood VLANs 4K limit Physical Network Appliances No APIs (Cisco at least) Can’t decapsulate UDP tunnelSaturday, December 1, 2012 18
  • Reject Pile (early 2009) KVM Unstable Windows support No CPU allocationSaturday, December 1, 2012 19
  • April 2009Saturday, December 1, 2012 20
  • April 2009 “5 guys in a garage”Saturday, December 1, 2012 20
  • Ship It! ( Beta 6/2009) ISO/CD with Xen 3.3 / CentOs customized ISO/CD with OSOL 2008.11 + patches + drivers Java binaries System VM: Fedora Core 8 VM bits including dnsmasq, iptables, HAProxy Installers, documentationSaturday, December 1, 2012 21
  • Road to 1.0 ISCSI - ZFS zvols -> ∞ snapshots Performance tuning (UDP tunnels, zvol) Usage, metering, Formal user / admin Query-based API JQuery-based GUI - skinnable Windows PV driversSaturday, December 1, 2012 22
  • More rejects (2009) Network RAID of zvols over ISCSI/md Performance Re-mirror time / penalty Consistent snapshots Split-brainSaturday, December 1, 2012 23
  • 1.0 - Ship It (Nov 2009) 6 engineers + 3 QA + 1 sales 3 beta sites Traction with SPs interested in competing with AWSSaturday, December 1, 2012 24
  • Orchestration + Usage server JVM Front- endSaturday, December 1, 2012 25
  • Orchestration + Usage server JVM MySQL Front- endSaturday, December 1, 2012 25
  • Orchestration + Usage server Browser LoadBalan JVM MySQL ced HTTP Query Front- API endSaturday, December 1, 2012 25
  • Orchestration + Usage server Browser LoadBalan JVM MySQL ced HTTP VM Network Storage Host/ Query Front- API Orchest Orchestr Orchest Resource end rator ator rator ManagerSaturday, December 1, 2012 25
  • Orchestration + Usage server Browser LoadBalan JVM MySQL ced HTTP VM Network Storage Host/ Query Front- API Orchest Orchestr Orchest Resource end rator ator rator Manager Message Bus Hypervi Hypervi Hypervisor Storage Storage Storage Console Console sor Agent sor Agent Agent Agent Agent Agent (JVM) (JVM)Saturday, December 1, 2012 25
  • Orchestration + Usage server Browser LoadBalan JVM MySQL ced Storage VM VM Network Network Storage HTTP Front- Front- Orchest Network Storage Host/ VM Orchestr Orchest Host/ Query Front- Orchest Orchestr Orchest Resource end API end Orchest Orchestr rator rator ator Orchest Resource end rator ator rator rator ator rator Manager Manager Clustered Message Bus Orchestration Message Bus Message Bus Server Hypervi Hypervi Hypervisor Storage Storage Storage Console Console sor Agent sor Agent Agent Agent Agent Agent (JVM) (JVM)Saturday, December 1, 2012 25
  • Orchestration + Usage server Browser LoadBalan JVM MySQL ced Storage VM VM Network Network Storage HTTP Front- Front- Orchest Network Storage Host/ VM Orchestr Orchest Host/ Query Front- Orchest Orchestr Orchest Resource end API end Orchest Orchestr rator rator ator Orchest Resource end rator ator rator rator ator rator Manager Manager Usage Clustered Message Bus Orchestration Message Bus Message Bus Server Hypervi Hypervi Hypervisor Storage Storage Storage Console Console sor Agent sor Agent Agent Agent Agent Agent (JVM) (JVM)Saturday, December 1, 2012 25
  • 1.0 Issues Multicast scaling Oracle acquisition of OpenSolaris Hiring “full-stack” developers 5 startups - Hypervisor + Storage + Network Virtualization + Network appliance + OrchestrationSaturday, December 1, 2012 26
  • Multicast Scaling Broadcast (ARP) / Multicast from VM translates to UDP multicast Switches snoop on IGMP to prevent flooding all ports Most switches fall-over after 64 Multicast groups each tenant -> 1 multicast groupSaturday, December 1, 2012 27
  • Road Ahead Multi-hypervisor, SAN, and VLAN support Use commercial XenServer Commercial filers / ISCSI arrays Support KVM + NFS Product first, architecture second From web hosting to enterprise workloadSaturday, December 1, 2012 28
  • Road Ahead Throw out custom hypervisor and OpenSolaris VLANs Secondary Storage to store snapshots and imagesSaturday, December 1, 2012 29
  • Circa 3/2010 GUI Proprietary Orchestration Services Proprietary Control Plane Open Source Xen Storage Agent Virtual Networks Agent (custom) Arrays !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 30
  • Circa 3/2010 JQuery GUI Proprietary Orchestration Services Proprietary Control Plane Open Source Xen Storage Agent Virtual Networks Agent (custom) Arrays !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 30
  • Circa 3/2010 JQuery GUI Proprietary Orchestration Services Proprietary Control Plane VLANs Open Source Xen Storage Agent Virtual Networks Agent (custom) Arrays !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 30
  • Circa 3/2010 JQuery GUI Proprietary Orchestration Services Proprietary Control Plane VLANs Open Source Xen Storage Virtual Networks Agent (custom) Arrays !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 30
  • Circa 3/2010 JQuery GUI Proprietary Orchestration Services Proprietary Control Plane VLANs Open Source Xen Storage Virtual Networks (custom) Arrays !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 30
  • Circa 3/2010 JQuery GUI Proprietary Orchestration Services Proprietary Control Plane VLANs Storage Virtual Networks Arrays !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 30
  • Circa 3/2010 JQuery GUI Proprietary Orchestration Services Proprietary Control Plane VLANs Commercial Storage Virtual Networks XenServer Arrays !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 30
  • Circa 3/2010 JQuery GUI Proprietary Orchestration Services Proprietary Control Plane ISCSI/ VLANs NFS Commercial Storage Virtual Networks XenServer Arrays !"#$%&( !"#$%&( !"#$%&(Saturday, December 1, 2012 30
  • Orchestration + Usage server Browser LoadBalan JVM MySQL ced Front- HTTP end VM Network Storage Host/ Query API Orchest Orchestr Orchest Resource rator ator rator Manager Job Queue Usage Clustered Message Bus Orchestration Server Hypervi XS sor Agent XAPI KVM Console Console Agent Secondary Agent Agent Storage Agent XenServers (JVM) (JVM) (JVM)Saturday, December 1, 2012 31
  • Availability Zone Architecture End users Admin/User API CloudStack DC Edge MySQL Access Sw Hypervisor (Xen /VMWare/KVM) Pod Pod Pod Pod PodSaturday, December 1, 2012 32
  • Availability Zone Architecture End users Admin/User API CloudStack DC Edge MySQL Access Sw Hypervisor (Xen /VMWare/KVM) Secondary Storage Pod Pod Pod Pod PodSaturday, December 1, 2012 32
  • Availability Zone Architecture End users Admin/User API CloudStack DC Edge MySQL L2/L3 core Access Sw Hypervisor (Xen /VMWare/KVM) Secondary Storage Pod Pod Pod Pod PodSaturday, December 1, 2012 32
  • Availability Zone Architecture End users Admin/User API CloudStack DC Edge MySQL L2/L3 core Access Sw Hypervisor (Xen /VMWare/KVM) Secondary Storage Primary Storage NFS/ISCSI/FC Pod Pod Pod Pod PodSaturday, December 1, 2012 32
  • System VM Data Path problem Need to move bits between storage types Present consoles to end users Scale up / down these data path servicesSaturday, December 1, 2012 33
  • System VMs Virtual Routers, Console Proxy VMs, Secondary Storage VMs based on the same vm image Orchestration and scaling baked inSaturday, December 1, 2012 34
  • 2.0 6/2010 Shipping just java binaries + system vm image Open sourced (GPL) Company name changed to Cloud.com New product - Cloud PortalSaturday, December 1, 2012 35
  • Citrix Cloud Portal Business and operations support platform for Service Providers Account management Self-service portal Billing and metering CRM and support ticketing Not OSSSaturday, December 1, 2012 36
  • Reject Pile (2010) “Zone” and “Pod” VLANs to go beyond 4k limit Pod VLANs bridged using zone VLANs Another SDN attempt using GRE on Open vSwitchSaturday, December 1, 2012 37
  • Flexibility and Scale (2010-2011) Network, hypervisor and vm placement flexibility Tens of thousands of hypervisors AWS APISaturday, December 1, 2012 38
  • Network Flexibility (2.2 refactor) Hardware appliances instead of virtual router Upgrade / downgrade to/from hardware to virtual Pick and choose services (L2-L7) Security GroupsSaturday, December 1, 2012 39
  • Network Flexibility Scale  out  edge  services  using  virtual  appliances 10.1.1.0/24 VLAN  100 VM  1 10.1.1.2 65.37.141.111 10.1.1.1 65.37.141.112 CS Virtual   10.1.1.3 VM  2 Router DHCP,  DNS NAT 10.1.1.4 VM  3 Load  Balancing VM  4 10.1.1.5Saturday, December 1, 2012 40
  • Network Flexibility Scale  out  edge  services  using  virtual  appliances Scale  up  using  hardware  devices 10.1.1.0/24 10.1.1.0/24 VLAN  100 VLAN  100 65.37.141.111 10.1.1.1 10.1.1.2 VM  1 VM  1 10.1.1.2 Juniper  SRX 65.37.141.111 10.1.1.1 Firewall NAT,   65.37.141.112 VPN CS VM  2 Virtual   10.1.1.3 VM  2 10.1.1.3 Router 65.37.141.112 10.1.1.112 DHCP,  DNS Netscaler NAT Load   10.1.1.4 VM  3 VM  3 Load  Balancing Balancer 10.1.1.4 VM  4 VM  4 10.1.1.5 10.1.1.5 CS DHCP,   Virtual   DNS RouterSaturday, December 1, 2012 40
  • VM Placement Customize vm placement Pick hosts and storage according to rules Affinity / anti-affinitySaturday, December 1, 2012 41
  • Multi-hypervisor XenServer, KVM and VMWare in the same zone Differences in network and storage behavior Different image formats Same system vm imageSaturday, December 1, 2012 42
  • Plugin Architecture (2.2) Orchestration EngineSaturday, December 1, 2012 43
  • Plugin Architecture (2.2) Plugin Framework Orchestration EngineSaturday, December 1, 2012 43
  • Plugin Architecture (2.2) Hypervisor Hypervisor Plugins Plugins Plugin Framework Orchestration EngineSaturday, December 1, 2012 43
  • Plugin Architecture (2.2) Hypervisor Hypervisor Plugins Plugins Plugin Framework Orchestration Network Network Plugins Plugins EngineSaturday, December 1, 2012 43
  • Plugin Architecture (2.2) Hypervisor Hypervisor Plugins Plugins Plugin Framework Orchestration Network Network Plugins Plugins Engine Allocator Allocator Plugins PluginsSaturday, December 1, 2012 43
  • Plugin Architecture (2.2) Hypervisor Hypervisor Plugins Plugins Plugin Framework Orchestration Network Network Plugins Plugins Engine Allocator Allocator Plugins Plugins Storage PluginsSaturday, December 1, 2012 43
  • Plugin Architecture Hypervisor Hypervisor Plugins Plugins Plugin Framework Orchestration Network Network Plugins Plugins Engine Allocator Allocator Plugins PluginsSaturday, December 1, 2012 44
  • Plugin Architecture •XenServer •VMWare •KVM •OracleVM Hypervisor Hypervisor Plugins Plugins Plugin Framework Orchestration Network Network Plugins Plugins Engine Allocator Allocator Plugins PluginsSaturday, December 1, 2012 44
  • Plugin Architecture •XenServer •VMWare •KVM •OracleVM Hypervisor Hypervisor Plugins Plugins Plugin •Nicira Framework •Netscaler Orchestration Network Network Plugins •Brocade •MidoNet Plugins Engine Allocator Allocator Plugins PluginsSaturday, December 1, 2012 44
  • Plugin Architecture •XenServer •VMWare •KVM •OracleVM Hypervisor Hypervisor Plugins Plugins Plugin •Nicira Framework •Netscaler Orchestration Network Network Plugins •Brocade •MidoNet Plugins Engine Random • Allocator Allocator •User- Plugins Plugins concentrated •Intel TXT •AffinitySaturday, December 1, 2012 44
  • Security Groups Web   DB   Web   VM VM VM Web   Web   DB   VM VM VM … … … Web   Web   VM VM Ingress  Rule:  Allow  VMs  in  Web  Security  Group  access  to  VMs  in  DB  Security  Group  on  Port  3306Saturday, December 1, 2012 45
  • Security Groups Web   DB   Web   VM VM VM Web Security   Group Web   Web   DB   VM VM VM … … … Web   Web   VM VM Ingress  Rule:  Allow  VMs  in  Web  Security  Group  access  to  VMs  in  DB  Security  Group  on  Port  3306Saturday, December 1, 2012 45
  • Security Groups Web   DB   Web   VM VM VM Web DB   Security   Security   Group Group Web   Web   DB   VM VM VM … … … Web   Web   VM VM Ingress  Rule:  Allow  VMs  in  Web  Security  Group  access  to  VMs  in  DB  Security  Group  on  Port  3306Saturday, December 1, 2012 45
  • Security Groups Web   DB   Web   VM VM VM Web DB   Security   Security   Group Group Web   Web   DB   VM VM VM … … … Web   Web   VM VM Ingress  Rule:  Allow  VMs  in  Web  Security  Group  access  to  VMs  in  DB  Security  Group  on  Port  3306Saturday, December 1, 2012 45
  • A million firewalls? Manage the config state of 10^6 firewalls? “Eventual consistency” Iptable Rule explosion on hypervisor use ‘ipset’ to optimize lookupSaturday, December 1, 2012 46
  • 2.1 -> 3.02 Features, features features VMWare, Oracle VM, Baremetal Multi-tier networking, VPC, VPN, more vendors EC2 API, S3 API SDN Focus on qualitySaturday, December 1, 2012 47
  • GRE-based SDN Ø Create  Full  Mesh  of  GRE  tunnels   (if  they  dont  already  exist)   between  hosts  on  which  VMs   are  deployed Host  1 Host  3 Ø CloudStack  SDN  controller   VM  1 programs  the  Open  vSwitch   (OVS)  on  XenServer  to  configure   GRE  Tunnel GRE  tunnels Host  2 Host  4 VM VM 2 3 VR GRE  Tunnel GRE  TunnelSaturday, December 1, 2012 48
  • GRE-based SDN Ø Create  Full  Mesh  of  GRE  tunnels   (if  they  dont  already  exist)   Tenant1 between  hosts  on  which  VMs   are  deployed Host  1 Host  3 Ø CloudStack  SDN  controller   VM  1 programs  the  Open  vSwitch   (OVS)  on  XenServer  to  configure   GRE  Tunnel GRE  tunnels Ø Assign  Tenant  key  to  the   customer  that  allows  traffic   Host  2 Host  4 isolaMon  from  other  tenants VM VM 2 3 VR GRE  Tunnel GRE  TunnelSaturday, December 1, 2012 48
  • GRE-based SDN Ø Create  Full  Mesh  of  GRE  tunnels   (if  they  dont  already  exist)   Tenant1 between  hosts  on  which  VMs   Tenant2 are  deployed Host  1 Host  3 Ø CloudStack  SDN  controller   VM  1 programs  the  Open  vSwitch   (OVS)  on  XenServer  to  configure   GRE  Tunnel GRE  tunnels Ø Assign  Tenant  key  to  the   customer  that  allows  traffic   Host  2 Host  4 isolaMon  from  other  tenants VM VM VR 2 3 Ø New  customers  can  share  the   established  GRE  tunnels  with   GRE  Tunnel GRE  Tunnel separate  tenant  keysSaturday, December 1, 2012 48
  • GRE-based SDN Ø Create  Full  Mesh  of  GRE  tunnels   (if  they  dont  already  exist)   Tenant1 between  hosts  on  which  VMs   Tenant2 are  deployed Host  1 Host  3 VM Ø CloudStack  SDN  controller   VM  1 VM  1 3 VR programs  the  Open  vSwitch   (OVS)  on  XenServer  to  configure   GRE  Tunnel GRE  tunnels Ø Assign  Tenant  key  to  the   customer  that  allows  traffic   Host  2 Host  4 isolaMon  from  other  tenants VM VM VR 2 VM  2 3 Ø New  customers  can  share  the   established  GRE  tunnels  with   GRE  Tunnel GRE  Tunnel separate  tenant  keysSaturday, December 1, 2012 48
  • GRE-based SDN Ø Create  Full  Mesh  of  GRE  tunnels   (if  they  dont  already  exist)   Tenant1 between  hosts  on  which  VMs   Tenant2 are  deployed Host  1 Host  3 VM Ø CloudStack  SDN  controller   VM  1 VM  1 3 VR programs  the  Open  vSwitch   (OVS)  on  XenServer  to  configure   GRE  Tunnel GRE  tunnels Ø Assign  Tenant  key  to  the   customer  that  allows  traffic   Host  2 Host  4 isolaMon  from  other  tenants VM VM VR 2 VM  2 3 Ø New  customers  can  share  the   established  GRE  tunnels  with   GRE  Tunnel GRE  Tunnel separate  tenant  keysSaturday, December 1, 2012 48
  • 2012 SDN is hot and emerging Network vendors embrace virtual appliances Several almost-good-enough EBS solutions exist Network vendors and storage vendors eager to “plug-in” to Cloud OrchestratorsSaturday, December 1, 2012 49
  • Rewind Started off with EC2-like Clouds Customers led us to support more “traditional” technologies (VLAN, SAN) Mainly because they wanted to support traditional workloads EC2-style CloudStack deployment adopted by large Web companies (sec. groups, local disk)Saturday, December 1, 2012 50
  • Fast Forward AWS has an SDN solution (“VPC”) VXLAN and NVGRE are being proposed as SDN solutions Web-scale companies are moving workloads off of AWS, want to use same workflows Enterprises are adopting “private clouds”Saturday, December 1, 2012 51
  • “Cloud” for traditional workload Can achieve significant reliability for applications running in one zone. vCenter/XenCenter Reliability of individual nodes is Enterprise  Networking  (e.g.,  VLAN) very high. Server   Server   Server   All zone storage is replicated to a Cluster Cluster Cluster second storage platform (synchronous or asynchronous) Enterprise  Storage  (e.g.,  SAN) In event of failure, images are recovered from second storage array. Existing workloads will run reliably.Saturday, December 1, 2012 52
  • AWS-style Zone Hypervisor Xenserver/KVMSaturday, December 1, 2012 53
  • AWS-style Zone Hypervisor Xenserver/KVM Storage Local EBSSaturday, December 1, 2012 53
  • AWS-style Zone Hypervisor Xenserver/KVM Storage Local EBS Networking L3 SDN L2 Elastic IpSaturday, December 1, 2012 53
  • AWS-style Zone Hypervisor Xenserver/KVM Storage Local EBS Networking L3 SDN L2 Elastic Ip Networking Services FW LB GSLBSaturday, December 1, 2012 53
  • AWS-style Zone Hypervisor Xenserver/KVM Storage Local EBS Networking L3 SDN L2 Elastic Ip Networking Services FW LB GSLB Multi-tier Apps L3 VPC CloudFormationSaturday, December 1, 2012 53
  • AWS-style Zone Hypervisor Xenserver/KVM Storage Local EBS S3 Networking L3 SDN L2 Elastic Ip Networking Services FW LB GSLB Multi-tier Apps L3 VPC CloudFormationSaturday, December 1, 2012 53
  • AWS-style Zone Hypervisor Xenserver/KVM CloudStack can Storage Local EBS S3 support both styles of zones Networking L3 SDN L2 Elastic Ip Networking Services FW LB GSLB Multi-tier Apps L3 VPC CloudFormationSaturday, December 1, 2012 53
  • 4.x Refactor 2.0 AWS 2.2 3.0 Quality 4.X Prototype 1.0 GA Refactor Compatibility Refactor Improvements Refactor 2008 2009 2010 2011 2012 Sept 2008: Nov 2009: May 2010: July 2011: April 2012: VMOps CloudStack Cloud.com Citrix Apache Founded 1.0 GA Launch & Acquires CloudStack CloudStack Cloud.com 2.0 GA 3rd major refactoring of CloudStack code Apache contribution drive rapid growth of CloudStack developer base Apache license compliance Services framework Hadoop integrationSaturday, December 1, 2012 54