CloudStack Hyderabad Meetup: Using CloudStack to build IaaS clouds

6,079 views

Published on

Keynote by Kevin Kluge,
1 November 2012,
CloudStack Hyderabad Meetup,
Lemon Tree, Hyderabad.

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,079
On SlideShare
0
From Embeds
0
Number of Embeds
4,065
Actions
Shares
0
Downloads
121
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

CloudStack Hyderabad Meetup: Using CloudStack to build IaaS clouds

  1. 1. Apache CloudStack (Incubating)An IntroductionKevin KlugeApache CloudStack Committer
  2. 2. Use CloudStack to build IaaS clouds (like EC2)•  Create VMs, disks •  Java based networks, network services •  Scalable•  Self service •  Many vendor integrations•  Meter usage •  Native and EC2 API
  3. 3. How did Amazon build EC2? Amazon eCommerce Platform AWS API (EC2, S3, …) Amazon Orchestration Software Open Source Xen Hypervisor Commodity Commodity Networking Servers Storage
  4. 4. How can you build your cloud? Amazon eCommerce Platform Your Portal (Optional) AWS API (EC2, S3, …) CloudStack or AWS API CloudStack Orchestration Software Amazon Orchestration Software ESXi, KVM, XenServer/XCP, OVM Open Source Xen Hypervisor Networking Servers Storage
  5. 5. Project history •  2008/2009: closed-source development •  First deployments in late 2009 •  May 2010: ~98% open source as GPLv3 (open core) •  August 2011: 100% open source GPLv3 •  April 2012: Switch to Apache License v2 •  Submit code to Apache Software Foundation
  6. 6. IaaS Cloud Concepts
  7. 7. Virtualization alone does not make a cloud Server  Virtualiza0on   Cloud   Built for traditional enterprise Designed around big data, apps & client-server compute massive scale & next-gen apps •  Scale-up (pool-based resourcing) •  Scale-out (horizontal resourcing) •  IT management-centric •  Autonomic management •  1 administrator for 100’s of servers •  1 administrator for 1,000’s of servers •  Proprietary vendor stack •  Open, value-added stack
  8. 8. Clouds must reliably run all types of workloads Traditional Workload Cloud Workload Expect reliability Design for failure Back-up everything Ephemeral resources HA, Fault tolerance Multi-site redundancy Admin control recovery Self-service recovery Think Server Virtualization Think Amazon Web Services
  9. 9. Embrace traditional and extend to Cloud-era Cloud-era Workloads Traditional Workloads CloudStack  Mgmt   Server   TradiGonal  Zone   vSphere   Enterprise  Networking  (e.g.,  VLAN)  Cloud-­‐era   Cloud-­‐era   Cloud-­‐era  Availability   Availability   Availability   Zone   Zone   Zone   ESXi   ESXi   ESXi   Cluster   Cluster   Cluster   Object  Storage   Enterprise  Storage  (e.g.,  SAN)  
  10. 10. Apache  CloudStack   Management  Server  TradiGonal   TradiGonal   Cloud-­‐era   TradiGonal   Cloud-­‐era  Availability   Availability   Availability   Availability   Availability   Zone   Zone   Zone   Zone   Zone  
  11. 11. Object store is critical for Cloud-era workloads Amazon-Style Cloud CloudStack     Mgmt.  Server   •  Workloads are distributed across availability zones •  No guarantee on zone reliability •  DBs and Templates snapped toCloud-­‐era   Cloud-­‐era   Cloud-­‐era   object store.Availability   Availability   Availability   Zone   Zone   Zone   •  For small failures, recreate instance in same zone •  For DR, recreate instance in different Object Store zone •  Dramatically less expensive
  12. 12. Deployment Architecture
  13. 13. •  Single  Management  Server  can  Data  Center  1   Data  Center  2   manage  mulGple  zones   Data  Center  2   Management   Data  Center  3   Servers   Zone  2   •  Zones  can  be  geographically   Zone  2   distributed  but  low  latency  links   Zone  3   Zone1   are  expected  for  beRer   Zone  4   3   Zone   performance   •  Single  MS  node  can  manage  up  to   Data  Center  2   Data  Center  2   10K  hosts.     Data  Center  2   Zone  2   •  MulGple  MS  nodes  can  be   Zone  2   Zone  2  Zone  3   deployed  as  cluster  for  scale  or   Zone  3   redundancy   Zone  3  
  14. 14. Standby  Mgmt   Cloud-era zone deployment Server  Cluster   Admin   Internet   Availability  Zone  2   Primary  Mgmt   Server  Cluster   Primary   Router   MySQL   Backup   Load  Balancer   MySQL   L3  Core  Switch  Top  of  Rack   Switch   …   Object  Store   Servers   …   …   …   …   Availability  Zone  1   Pod  1   Pod  2   Pod  3   Pod  N  
  15. 15. InternetTraditional zonedeployment Object   Load  Balancer Store Core  Switch … Aggrega0on   Switch TOR  Switch Compute   Nodes     NFS   Primary   10Gbps   1Gbps   10Gbps   1Gbps   10Gbps   1Gbps   Storage Storage   Guest Storage   Guest Storage   Guest &  Mgmt   &  Mgmt &  Mgmt Pod  1 Pod  2 Pod  200
  16. 16. Management  Server   XAPI   HTTP   vCenter Agent Agent XenServer KVM OVM XCP ESX•  XS  5.6,  5.6FP1,  5.6  SP2,   •  ESX  4.1,  5.0     •  RHEL  6.0,  6.1,  6.2,   •  OVM  2.2   6.0.2,  XCP  1.1   •  Full  Snapshots   Ubuntu  12.04     •  No  Snapshots  •  Incremental  Snapshots   •  VMDK   •  Full  Snapshots  (not  live)   •  RAW  •  VHD   •  NFS,  iSCSI,  FC  &  Local  disk   •  QCOW2   •  NFS  &  iSCSi  •  NFS,  iSCSI,  FC  &  Local  disk   •  Storage  over-­‐provisioning:   •  NFS,  iSCSI  &  FC   •  No  storage  over-­‐•  Storage  over-­‐ NFS,  iSCSI   •  Storage  over-­‐ provisioning   provisioning:  NFS   provisioning:  NFS  
  17. 17. Mgmt Server CPU Util.Seconds to deploy 25,000 …. to …. 30,000 VMs 0 …. to …. 30,000 VMs •  Simulator developed to test massive scale •  Four Management Servers can manage 30,000 hosts •  Scale to hundreds of thousands of hosts possible with multiple management server clusters (regions)
  18. 18. Features
  19. 19. Compute   Hypervisor       XCP/XS   VMware   Oracle  VM   KVM   Bare  metal       Storage   Block  &  Object   Fiber   Object       Local  Disk   iSCSI   Channel   NFS   Stores       Network   Network  &  Network  Services   Network   Load       IsolaGon   Firewall   VPN   Type   balancer      
  20. 20. Users ChangeVM Operations Console Access VM Status Service Offering Start   •  CPU  UGlized   2  CPUs     4  CPUs     Stop   1  GB  RAM   4  GB  RAM       Restart   •  Network  Read   20  GB   200  GB       Destroy   20  Mbps   100  Mbps   •  Network  Writes  
  21. 21. Add / Delete VM  1     Volumes Volume  Create Templates   Volume     Template   from Volumes Schedule Now   Hourly     Weekly     Snapshots Daily   Monthly  View Snapshot ….   History 12/2/2012  7.30  am   2/2/2012  7.30  am  
  22. 22. Resources   Domain VMs,  IPs,  Snapshots…   •  Domain  is  a  unit  of  isolaGon  that   Org A   represents  a  customer  org,  business     Admin unit  or  a  reseller     Domain Reseller A •  Domain  can  have  arbitrary  levels  of   sub-­‐domains   Admin Sub-Domain Resources   Org C VMs,  IPs,  Snapshots…     •  A  Domain  can  have  one  or  more   Admin accounts     Account •  An  Account  represents  one  or  more   Group A users  and  is  the  basic  unit  of   Account isolaGon   Group B •  Admin  can  limit  resources  at  the   User 1 Account  or  Domain  levels   User 2
  23. 23. Specify  Resource  Levels   Configure  ProperGes                   Define  Scope   Compute   Disk   Network   Name     Name   Name           CPU  Cores    Custom  Disk  Size     Network  Rate   CPU  (MHz)   Disk  Size  (GB)   Redundant  VR   Memory  (MB)   Storage  Tag   Firewall   Host  Tag   Storage  Tag   Load  balancer   Enable  HA   Public   Public   CPU  Cap   Public  
  24. 24. •  Create Networks and attach VMs•  Acquire public IP address for NAT & load balancing•  Control traffic to VM using ingress and egress firewall rules•  Set up rules to load balance traffic between VMs
  25. 25. Network  Services  Managed  Externally   Network  Services  Managed  by  CS   Public  Network   65.11.0.0/16Security   Security   Group  1 Public  Network/ Group  1 65.11.1.2 Guest  VM   Internet 65.11.1.2 Guest  VM   1   1   65.11.1.3 Guest  VM   Physical   65.11.1.3 Guest  VM   Load   2   2   Balancer   65.11.1.4 Guest  VM   EIP,   65.11.1.4 Guest  VM   3   ELB   3   65.11.1.5 Guest  VM   65.11.1.5 Guest  VM   4   4   CS   CS   Security   DHCP,   Virtual   Security   DHCP,   Virtual   Group  2 DNS   Router   Group  2 DNS   Router  
  26. 26. CS  Virtual  Router  provides  Network  Services   External  Devices  provide  Network  Services   Guest  Virtual  Network  10.0.0.0/8   Guest  Virtual  Network  10.0.0.0/8  Public  Network/ VLAN  100 Public  Network/ VLAN  100Internet Internet Guest   Public  IP   Private  IP   Guest   10.1. VM  1   6.37.1.12 Juniper   10.1.1.111 10.1. VM  1   CS   Gateway   1.1 SRX   1.1 6.37..1.11 Firewall   Virtual   address   Guest   Guest   Router   10.1.1.1 10.1. VM  2   10.1. VM  2   Private  IP   DHCP,  DNS   1.3 Physical   10.1.1.112 1.3 NAT   Guest   Load   Guest   Load  Balancing   10.1. VM  3   Public  IP   Balancer   10.1. VM  3   VPN 1.4 6.37.1.11 1.4 Guest   Guest   10.1. VM  4   10.1. VM  4   1.5 1.5 CS   DHCP,   Virtual   DNS   Router  
  27. 27. Layer-­‐2   Layer-­‐3  IsolaGon   VLAN/SDN   Security  Groups  Performance   BeRer   BeRer  Network  setup   Moderate   Easy  Support  broadcast   Yes   No  Scalability   Good   Best  Interoperability  with   Good   Poor  physical  servers  
  28. 28. CloudStack storage   Primary  Storage  •  Configured at Cluster-level. Close to hosts for better performance•  Stores all disk volumes for VMs in a cluster L3 switch•  Cluster can have one or more primary storages Pod  1   L2 switch•  Local disk, iSCSI, FC or NFS   Secondary                                                                                                             Cluster  1   Storage   Host 1   Secondary  Storage   Primary   Storage  •  Configured at Zone-level Host 2•  Stores all Templates, ISOs and Snapshots•  Zone can have one or more secondary storages•  NFS, OpenStack Swift, others coming  
  29. 29. Futures
  30. 30. Apache CloudStack CloudStack API API Apache CloudStack CloudStack Apache API API Apache FirewallHypervisor Baremetal Switches Security Storage Load Bal
  31. 31. Futures•  Object storage and SDN short term•  Blade orchestration•  Region support•  Additional hypervisors (need some container support)•  Code modularity improvements (OSGI?)•  App-specific integration (Hadoop?)•  Improved CLI•  Additional API support (Google, evolving standards)
  32. 32. Thank You
  33. 33. Project current state •  In incubation within Apache Software Foundation •  Imminent first release! •  Bugs and wiki mostly moved to ASF infra •  Mailing list traffic moved to ASF infra •  Many non-Citrix contributors, committers, and PPMC members
  34. 34. Yes, the ASF is great Enter ASF
  35. 35. The future needs you!Project web site: http://incubator.apache.org/projects/cloudstack.htmlMailing lists:cloudstack-dev-subscribe@incubator.apache.orgcloudstack-users-subscribe@incubator.apache.orgIRC: #CloudStack on irc.freenode.netJoin your local CloudStack group!

×