Hacking apache cloud stack


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Hacking apache cloud stack

  1. 1. Hacking on Apache CloudStack
  2. 2. Agenda •  Introductions •  Session 1: Introduction to CloudStack Nitin Mehta: Committer Apache CloudStack Venkata SwamyBabu Budumuru: Commiter Apache Cloudstack •  Session 2: Hands on with DevCloud Kishan Kavala: Committer Apache CloudStack •  Chit Chatting with snacks J
  3. 3. Session 1 Introduction to CloudStack
  4. 4. Cloud Built for traditional enterprise apps & client-server compute •  Enterprise arch for 100s of hosts •  Scale-up (pool-based resourcing) •  IT management-centric •  1 administrator for Dozens of servers •  Apps assume reliability •  Proprietary vendor stack Designed around big data, massive scale & next-gen apps •  Cloud arch for 1000s of hosts •  Scale-out (horizontal resourcing) •  Autonomic management •  1 administrator for 1,000’s of servers •  Apps assume failure •  Open, value-added stack Server Virtualization++ Cloud Computing Virtualization is not Cloud computing
  5. 5. • Tenets of Cloud o  Shared infrastructure and Multi-tenancy o  Self Service o  Elasticity o  Built for massive Scale o  Service agility o  Pay-as-you-go o  APIs and Extreme Automation • IAAS/PAAS/SAAS • Public/Private/Hybrid clouds Cloud Computing (contd..)
  6. 6. •  Turnkey orchestration platform for delivering IAAS clouds o  Secure, multi-tenant o  Self-service o  Service agility and elasticity o  Built for large scale o  Pay-as-you-go •  Deploys on premise (private) or as a hosted (public) cloud •  Can be used for hybrid clouds •  built in java, provides native REST API’s and EC2 API •  Has python, Ruby clients and CLI as well What is Apache CloudStack
  7. 7. A  bit  of  History   • Original  company  Cloud.com  (2008)   • Open  source  (GPLv3)  as  CloudStack  (2010)   • Acquired  by  Citrix  (July  2011)   • Relicensed  under  ASL  v2  April  3,  2012   • Accepted  as  Apache  IncubaKng  Project  April  16,   2012   • Many  non-­‐Citrix  contributors,  commiPers,  PPMC   members   • Top  Level  Poject  (April  2013)  
  8. 8. Who is contributing • Sungard: Unit test cases • Carnigo: Object store plug-in • Ceph/Rbd support by Wido • CLVM/KVM by Marcus • Nicira NVP: Schuberg Philis • Basho: Object Store • Brocade ADX ADC support • Midokura midonet SDN controller integration
  9. 9. How to contribute • Its not just about code! As community member you can engage in o Discussions: Design, Use Case, deployment issues o Bug reporting, feature requests o Code reviews o Build, tools, infrastructure o Helping out on the IRC o Documentation o Submit bug fixes, features
  10. 10. How to contribute (contd..) • Git repo, bug tracker, wiki are on ASF infra • Project website o  http://cloudstack.apache.org/ o  http://www.cloudstack.org • Mailing lists (cloudstack.org/discuss/mailing- lists.html) o  dev-subscribe@cloudstack.apache.org o  users-subscribe@cloudstack.apache.org • Cloudstack -101
  11. 11. CloudStack managed cloud Compute StorageNetwork Admin Users Org B End User Cloud Admin On-demand infrastructure as a service CloudStack Management Server REST API UI Cli EC2 Admin Users Org A Consume resources Provision resources manage resources
  12. 12. •  Hosts •  Servers onto which services will be provisioned •  Primary Storage •  VM storage •  Cluster •  A grouping of hosts and their associated storage •  Pod •  Collection of clusters •  Network •  Logical network associated with service offerings •  Secondary Storage •  Template, snapshot and ISO storage •  Zone •  Collection of pods, network offerings and secondary storage •  Management Server Farm •  Responsible for all management and provisioning tasks Core CloudStack Components Zone CloudStack Pod Cluster Host Host Network Primary Storage VM VM CloudStack Pod Cluster Secondary Storage
  13. 13. Pod 1 …. Cluster N Access Layer Host 2 Cluster 1 CloudStack Deployment Architecture Host 1 Ø  Hypervisor is the basic unit of scale. Ø  Cluster consists of one ore more hosts of same hypervisor Ø  All hosts in cluster have access to shared (primary) storage Ø  Pod is one or more clusters, usually with L2 switches. Ø  Availability Zone has one or more pods, has access to secondary storage. Ø  One or more zones represent cloudPrimary Storage Zone 1 …. L3 core Secondary Storage Pod N CloudStack Management Server Internet
  14. 14. Zone1 Data Center 1 Data Center 2 Zone 3 Zone 2 Data Center 2 Zone 3 Zone 2 Data Center 2 Zone 3 Zone 2 Data Center 2 Zone 3 Zone 2 Data Center 2 Zone 3 Zone 2 Data Center 3 Zone 4 Management Server Ø  Single Management Server can manage multiple zones Ø  Zones can be geographically distributed but low latency links are expected for better performance Ø  Single MS node can manage up to 5K hosts. Ø  Multiple MS nodes can be deployed as cluster for scale or redundancy CloudStack Managing Multiple Zones
  15. 15. Infrastructure provisioning
  16. 16. Compute/Disk/Network Offering
  17. 17. Select Operating System •  Windows, Linux Select Compute Offering •  CPU & RAM Select Disk Offering •  Volume Size Select Network Offering •  Network & Services Create VM Create Virtual Machines via Offerings
  18. 18. Virtual Machine Management Users Start Stop Restart Destroy VM Operations Console Access •  CPU Utilized •  Network Read •  Network Writes VM Status Change Service Offering 2 CPUs 1 GB RAM 20 GB 20 Mbps 4 CPUs 4 GB RAM 200 GB 100 Mbps
  19. 19. Volume & Snapshot Management Volume VM 1 Add / Delete Volumes Schedule Snapshots Hourly Daily Weekly Monthly Now Create Templates from Volumes Volume Template View Snapshot History ….
  20. 20. A  Very  Flexible  IaaS  Pla5orm   Compute XenServer VMware KVMOracle VM Bare metal Hypervisor Storage Local Disk iSCSI NFS Fiber Channel Swift Block & Object Network Network Type Isolation Load balancer Firewall VPN Network & Network Services Primary  Storage   Secondary  Storage   Ceph Riak
  21. 21. Pod 1 Host 2 Cluster 1 Host 1 Primary Storage L3 switch Secondary Storage L2 switch CloudStack Storage •  Configured at Cluster-level. Close to hosts for better performance •  Stores all disk volumes for VMs in a cluster •  Cluster can have one or more primary storages •  Local disk, iSCSI, FC or NFS Primary Storage •  Configured at Zone-level •  Stores all Templates, ISOs and Snapshots •  Zone can have one or more secondary storages •  NFS, OpenStack Swift Secondary Storage •  Storage available on hypervisor hist Local Storage Local storage Availability zone
  22. 22. •  Primary Storage •  Cluster level storage for VMs •  Connected directly to hosts •  NFS, iSCSI, FC and Local •  Secondary Storage •  Zone level storage for template, ISOs and snapshots •  NFS or OpenStack Swift via CloudStack System VM •  Templates and ISOs •  Imported into CloudStack •  Can be private or public Role of Storage and Templates Zone Secondary Storage Pod Cluster Host Host Primary Storage Template
  23. 23. 1.  User Requests Instance 2.  Provision Optional Network Services 3.  Copy instance template from secondary storage to primary storage on appropriate cluster 4.  Create any requested data volumes on primary storage for the cluster 5.  Create instance 6.  Start instance Provisioning Process Zone Secondary Storage Pod Cluster Host Host Primary Storage VM Template
  24. 24. Domain is a unit of isolation that represents a customer org, business unit or a reseller Domain can have arbitrary levels of sub-domains A Domain can have one or more accounts An Account represents one or more users and is the basic unit of isolation Admin can limit resources at the Account or Domain levels Admin Org A Admin Reseller A Domain Domain Admin Org C Sub-Domain User 1 User 2 Group B Account Group A Account VMs, IPs, Snapshots… VMs, IPs, Snapshots… Resources Resources Multi-tenancy & Account Management
  25. 25. User Dashboard: Consumed Resources •  Running, Stopped & Total VMs •  Public IPs •  Private networks •  Latest Events
  26. 26. Admin Dashboard: Consumed Resources •  Provides zone wide resource consumption •  Also provides latest alerts and events
  27. 27. Edge services with System VMs •  System VMs optimize and scale the datapath on behalf of CloudStack o  Stateless, can be destroyed and recreated from database state o  Highly Available o  Communicates with Management Server over management network o  Usually have 3 interfaces: control, guest and public •  Console Proxy VM o  Provides AJAX-style HTTP-only console viewer o  Grabs VNC output from hypervisor o  Scales out (more spawned) as load increases o  Java-based server Communicates with MS over message bus •  Secondary Storage VM o  Provides image (template) management services o  Download from HTTP file share or Swift o  Copy between zones o  Scale out to handle multiple NFS mounts o  Java-based server communicates with MS over message bus
  28. 28. •  Virtual Router VM o  Provides multiple network services o  IPAM (DHCP), DNS, NAT, Source NAT, Firewall, PF, VPN o  User-data, Meta-data, SSH keys and password change server o  Redundancy via VRRP o  MS configures VR over SSH §  Proxied via the hypervisor on XS and KVM Edge services with System VMs (contd.)
  29. 29. Network & Network Services •  Create Networks and attach VMs •  Acquire public IP address for NAT & load balancing •  Control traffic to VM using ingress and egress firewall rules •  Set up rules to load balance traffic between VMs
  30. 30. Networking feature overview •  Orchestration of L2 – L7 network services o  IPAM, DNS, Gateway, Firewall, NAT, LB, VPN, etc •  Mix-and-match services and providers •  Out-of-the-box integration with automated deployment of virtual routers o  Highly available network services using CloudStack HA and VRRP •  Orchestrate external providers such as hardware firewalls and load balancers o  Devices can provide multiple services o  Admin API to configure external devices o  Plugin-based extensions for network behavior and admin API extensions •  Multiple multi-tenancy [network isolation] options •  Integrated traffic accounting •  Access control •  Software Defined Networking (Nicira NVP)
  31. 31. L2 Features •  Choice of network isolation o  Physical, VLAN, L3 (anti-spoof), Overlay[GRE] o  Physical isolation through network labels [limited to # of nics or bonds] •  Multi-nic o  Deploy instance in multiple networks o  Control default route •  Access control o  Shared networks, project networks •  QoS [max rate] •  Traffic monitoring •  Hot-plug / detach of nics
  32. 32. L3 Features •  IPAM [DHCP], Public IP address management o  VR acts as DHCP server o  Can request multiple public IPs per tenant •  Gateway (default gateway) o  Redundant VR (using VRRP) o  Inter-subnet routing o  Static routing control •  Remote Access VPN o  L2TP over IPSec using PSK o  Virtual Router only •  Firewall based on source cidr •  Static NAT [1:1] o  Including “Elastic IP” in Basic Zone •  Source NAT o  Per-network, or interface NAT •  Public Traffic usage o  Monitoring on the Virtual Router / External network device o  Integration with sFlow collectors •  Site-to-Site VPN o  IPSec VPN based on VR •  L3 ACLs
  33. 33. L4 Features • Security groups for L3-isolation o  “Basic Zone” in docs o  Default AWS-style networking o  Scales much better than VLANs • Stateful firewall for TCP, UDP and ICMP • Port forwarding [“Advanced Zone”] o  Conserve public Ips
  34. 34. L7 features • Loadbalancer o  VR has HAProxy built in o  External Loadbalancer support §  Netscaler (MPX/SDX/VPX) §  F5 BigIP §  Can dedicate an LB appliance to an account or share it among tenants o  Loadbalancer supported with L3-isolation as well o  Stickiness support o  SSL support [future] o  Health Checks [future] • User-data & meta-data o  Fetched from virtual router • Password change server
  35. 35. CloudStack Terminology •  Guest network o  The tenant network to which instances are attached •  Storage network o  The physical network which connects the hypervisor to primary storage •  Management network o  Control Plane traffic between CloudStack management server and hypervisor clusters •  Public network o  “Outside” the cloud [usually Internet] o  Shared public VLANs trunked down to all hypervisors •  All traffic can be multiplexed on to the same underlying physical network using VLANs o  Usually Management network is untagged o  Storage network usually on separate nic (or bond) •  Admin informs CloudStack how to map these network types to the underlying physical network o  Configure traffic labels on the hypervisor o  Configure traffic labels on Admin UI
  36. 36. CloudStack Network Service Providers •  A Network Service Provider is hardware or virtual appliance that makes a network service possible in CloudStack ; for example, a Citrix NetScaler appliance can be installed in the cloud to provide Load-Balancing services. •  Administrators can have multiple instances of the same service provider in a network; for example, more than one Citrix NetScaler or Juniper SRX device can be added to CloudStack •  CloudStack supports the following Network Providers: o  CloudStack Virtual Router (default) o  Citrix NetScaler SDX, VPX and MPX models o  Juniper SRX o  F5 BigIP
  37. 37. Network Service Providers Matrix Feature Virtual Router Citrix NetScaler Juniper SRX F5 BigIP Remote Access VPN YES N/A N/A N/A Firewall YES N/A YES N/A Source NAT YES N/A YES N/A Static NAT YES YES YES N/A Load Balancing YES YES N/A YES Port Forwarding YES N/A YES N/A Elastic IP N/A YES N/A N/A Elastic LB N/A YES N/A N/A DHCP/DNS/User Data YES N/A N/A N/A •  Network offerings is basically a definition of what Network Services are available when this offering is used. The available Network Services are: VPN, DHCP, DNS, Firewall, Load Balancer, User Data, Source NAT, Static NAT, Port Forwarding and Security Groups*
  38. 38. •  Cloud provider defines the feature set for guest networks •  Toggle features or service levels o  Security groups on/off o  Load balancer on/off o  Load balancer software/hardware o  VPN, firewall, port forwarding •  User chooses network offering when creating network •  Enables upgrade between network offerings •  Default offerings built-in o  For classic CloudStack networking Network Offerings
  39. 39. Add Guest Networks •  Choice to choose L3 subnet, default gateway •  Choice of network offerings
  40. 40. Editing Guest Networks When editing a guest network users can change the network offering. They can either upgrade to a “premium” network offering (for example offering that uses hardware Load-balancer) or downgrade to a “cheaper” network.
  41. 41. •  Restarting the network will simply resend all the LB, Firewall and Port-Forwarding rules to the network provider •  Restarting the Network with “Clean up”: •  restarKng  network  elements  -­‐  virtual  routers,  DHCP   servers   •  If  virtual  router  is  used,  it  will  be  destroyed  and   recreated     •  Reapplying  all  public  IPs  to  the  network  provider   •  Reapplying  load-­‐Balancing/Port-­‐Forwarding/Firewall   rules   Restarting/Cleaning Up a Guest Network
  42. 42. •  An Isolated Guest Network can only be deleted if no VMs are using these network (e.g. Completely destroyed and expunged) •  Deleting a Network will Destroy the Virtual Router (if used) and will release the Public IPs back to the IP Pool Deleting a Guest Network
  43. 43. Basic vs Advanced Networking • Segmentation based on feature set and ease-of- deployment • Both are feature-rich • Basic implements true AWS-style L3-isolation o  Tenants do not get contiguous IP addresses or subnets o  Network segmentation based on Security Groups o  Tremendous scale (tens of thousands) • Advanced Zone offers full L3 subnets and L2 isolation o  VLANs are default implementation (4K limit) o  More features (source NAT, PF, LB, VPN)
  44. 44. Storage 1 Hypervisor  1 Hypervisor  N Hypervisor  8 Access  Switch(es)Cloudstack   Server   VM Traffic Control Plane Traffic Storage Traffic Cloudstack   Servers Storage k …   Pod 1 CLUSTER 1 …   CLUSTER 4 Core (L3) Network …   Pod 2 Pod N Physical Network in Zone Storage 2 Hypervisor  N+1 Public Traffic
  45. 45. … DB Security Group Web Security Group Layer 3 cloud networking … … Web VM Web VM Web VM Web VM DB VM Web VM DB VM Web VM
  46. 46. Guest Networks with L3 isolation Guest   1  VM  1 Guest   2  VM  1 Guest   1  VM  2 Guest   2  VM  2 Public   Internet Public  IP   address     Guest   address Guest   address Guest   address Guest   address Load   Balancer Guest   2  VM  3 Guest   1  VM  3 Guest   1  VM  4 Guest   address Guest   address Guest   address L3  Core   Switch Pod  1  L2   Switch Pod  3  L2   Switch 10.1.16. 1 … …  2  L2   Switch
  47. 47. Hypervisor  1 Hypervisor  N Hypervisor  8 Access  Switch(es) VM Traffic …   Pod K CLUSTER 1 …   CLUSTER 4 Core (L3) Network …   Pod M Pod N Guest Networks with L2 isolation Hypervisor  N+1 Public Traffic Hypervisor R R V VV V Hypervisor V V V R Tenant VM Tenant Virtual Router
  48. 48. … L2 isolation: VLAN networking … … User 2 User 2 User 1 User 1 User 1 User 1 User 1 User 2 User 1
  49. 49. SDN at Work Host 1 Host 2 Host 3 Host 4 GRE Tunnel GRE Tunnel GRE Tunnel VM 1 VM 2 VM 3 V R OVS OVS OVS CloudStack Mgmt Server SDN Controlle r VM 1 VM 2 VM 3 V R OVS GRE Tunnel
  50. 50. Guest virtual layer-2 network Guest   1  VM  1 Guest   1  VM  2 Guest   1  VM  3 Guest   1  VM  4 Public   Internet Public   Network Guest  Virtual  Network Gateway   address NAT   DHCP   Load   Balancing   VPN Public  IP   address Guest   address Guest   address Guest   address Guest   address Guest  1   Virtual   Router Guest   2  VM  1 Guest   2  VM  2 Guest   2  VM  3 Guest  Virtual  Network Gateway   address NAT   DHCP   Load   Balancing   VPN Guest   address Guest   address Guest   address Guest  2   Virtual   Router Public  IP   address
  51. 51. Layer-2 Guest Virtual Network Public   Network/ Internet Guest  Virtual  Network   VLAN  100 Gateway   address DHCP,  DNS   NAT   Load  Balancing   VPN Public  IP Guest VM 1 Guest VM 2 Guest VM 3 Guest VM 4 CS Virtual Router Public   Network/ Internet Guest  Virtual  Network   VLAN  100 Private  IP DHCP,   DNS   Public  IP   65.37.141. 112 Guest VM 1 Guest VM 2 Guest VM 3 Guest VM 4 NetScaler Load Blancer Private  IP Public  IP Juniper SRX Firewall CS Virtual Router provides Network Services External Devices provide Network Services CS Virtual Router
  52. 52. Layer-3 Guest Network Public  Network Guest VM 1 Guest VM 2 Guest VM 3 Guest VM 4 Public  Network/ Internet NetScaler Load Blancer Network Services Managed Externally Network Services Managed by CS DHCP,   DNS   CS Virtual Route r Security  Group   1 Security  Group   2 Guest VM 1 Guest VM 2 Guest VM 3 Guest VM 4 DHCP,   DNS   CS Virtual Router Security  Group   1 Security  Group   2 EIP,   ELB L3 switch
  53. 53. Multi-tier network Web VM 1 Web VM 2 Web VM 3 Web VM 4 Virtual  Network   VLAN  100 Virtual  Network   VLAN  1001 App VM 1 Virtual  Network   VLAN  141 App VM 2 DB VM 1 CS Virtual Router Customer Premises IPSec or SSL site-to-site VPN Internet Monitoring VLAN Virtual Router Services •  IPAM •  DNS •  LB [intra] •  S-2-S VPN •  Static Routes •  ACLs •  NAT, PF •  FW [ingress & egress] •  BGP Loadbalancer
  54. 54. Session 2 Developing with DevCloud
  55. 55. DevCloud • CloudStack requires o  Hypervisor o  Network o  Storage
  56. 56. DevCloud • self-contained CloudStack runs in the appliance
  57. 57. DevCloud • Several use cases o  Try CloudStack in an isolated sandbox. Runs within the appliance o  Develop CloudStack on own machine, build locally and deploy new version in DevCloud (Build and test) o  Develop and Run locally, use DevCloud as Xen hosts
  58. 58. Thanks