Hacking             onApache (Incubating) CloudStack
Tutorial Outline•  Session 1: Introduction to CloudStack     Murali Reddy: Committer Apache CloudStack•  Session 2: Archit...
Session 1Introduction to CloudStack
Cloud ComputingVirtualization is not Cloud computing         Server Virtualization++                                 Cloud...
Cloud Computing (contd..)•  Tenets of Cloud  o  Shared infrastructure and Multi-tenancy  o  Self Service  o  Elasticity  o...
What is Apache CloudStack•  Turnkey orchestration platform for delivering IAAS clouds   o  Secure, multi-tenant   o  Self-...
A	  bit	  of	  History	  •  Original	  company	  Cloud.com	  (2008)	  •  Open	  source	  (GPLv3)	  as	  CloudStack	  (2010...
Who is contributing•  Sungard: Unit test cases•  Carnigo: Object store plug-in•  Ceph/Rbd support by Wido•  CLVM/KVM by Ma...
How to contribute•  Its not just about code! As community member  you can engage in   o Discussions: Design, Use Case, dep...
How to contribute (contd..)•  Git repo, bug tracker, wiki are on ASF infra•  Project website   o  http://incubator.apache....
On-demand infrastructure as a service                                          Org A	                         Org B	      ...
Core CloudStack Components                                                                                      VM•     Ho...
CloudStack Deployment ArchitectureCloudStackManagement                                      Internet           Ø  Hypervi...
CloudStack Managing Multiple ZonesData Center 1           Data Center 2       Ø    Single Management Server can          ...
Infrastructure provisioning
Infrastructure provisioning (contd.)
Compute/Disk/Network Offering
Create Virtual Machines via Offerings                             Select Operating System                               • ...
Virtual Machine Management                                  Users	                                                        ...
Volume & Snapshot Management                   VM 1  Add / Delete    Volumes                                     VolumeCre...
A	  Very	  Flexible	  IaaS	  Pla5orm	  Compute   Hypervisor           XenServer             VMware                 Oracle ...
CloudStack Storage     Primary Storage •      Configured at Cluster-level. Close to hosts for        better performance • ...
Role of Storage and Templates•  Primary Storage •  Cluster level storage for VMs                            Host •  Connec...
Provisioning Process1.    User Requests Instance                                    VM2.    Provision Optional Network    ...
Object Store                CloudStack Mgmt                  •  Object store used to store                     Server     ...
Multi-tenancy & Account Management                      Resources  Domain	                       VMs, IPs, Snapshots…     ...
User Dashboard: Consumed Resources•    Running, Stopped &     Total VMs•    Public IPs•    Private networks•    Latest Eve...
Admin Dashboard:                      Consumed Resources•    Provides zone wide     resource     consumption•    Also prov...
Edge services with System VMs•    System VMs optimize and scale the datapath on behalf of CloudStack      o    Stateless, ...
Edge services with System VMs (contd.)•    Virtual Router VM      o  Provides multiple network services      o  IPAM (DHCP...
Network & Network Services•    Create Networks and attach VMs•    Acquire public IP address for     NAT & load balancing• ...
Networking feature overview•  Orchestration of L2 – L7 network services    o  IPAM, DNS, Gateway, Firewall, NAT, LB, VPN, ...
L2 Features•  Choice of network isolation      o  Physical, VLAN, L3 (anti-spoof), Overlay[GRE]      o  Physical isolation...
L3 Features•    IPAM [DHCP], Public IP address management      o    VR acts as DHCP server      o    Can request multiple ...
L4 Features•  Security groups for L3-isolation  o  “Basic Zone” in docs  o  Default AWS-style networking  o  Scales much b...
L7 features•  Loadbalancer   o  VR has HAProxy built in   o  External Loadbalancer support       §  Netscaler (MPX/SDX/VP...
CloudStack Terminology•    Guest network      o  The tenant network to which instances are attached•    Storage network   ...
CloudStack Network Service Providers•    A Network Service Provider is hardware or virtual     appliance that makes a netw...
Network Service Providers Matrix•  Network offerings is basically a definition of what Network Services are   available wh...
Network Offerings•  Cloud provider defines the   feature set for guest networks•  Toggle features or service  levels   o  ...
Add Guest Networks•    Choice to choose L3     subnet, default gateway•    Choice of network     offerings
Editing Guest NetworksWhen editing a guest network users canchange the network offering. They caneither upgrade to a “prem...
Restarting/Cleaning Up a Guest Network•  Restarting the network will simply   resend all the LB, Firewall and   Port-Forwa...
Deleting a Guest Network•  An Isolated Guest Network can only be deleted if no VMs   are using these network (e.g. Complet...
Basic vs Advanced Networking•  Segmentation based on feature set and ease-of-     deployment•    Both are feature-rich•   ...
Physical Network in Zone                                                         Core (L3) Network                        ...
Layer 3 cloud networking    Web                   DB              Web    VM                    VM              VM         ...
Guest Networks with L3 isolationPublic	     Public	  IP	                                                      Guest	      ...
Guest Networks with L2 isolation                                            Core (L3) Network                             ...
L2 isolation: VLAN networking                     User                      1    User     1                              U...
SDN at Work                     CloudStack Mgmt Server                            SDN                           Controlle ...
Guest virtual layer-2 network                                                            Guest	  Virtual	  Network	       ...
Layer-2 Guest Virtual Network   CS Virtual Router provides Network Services                                               ...
Layer-3 Guest NetworkNetwork Services Managed Externally                                       Network Services Managed by...
Multi-tier network                                           Internet                                                     ...
Session 2Architecture of CloudStack
Problem Definition•  Offer a scalable, flexible, manageable IAAS platform  that orchestrate physical and virtual resources...
Problem Definition (contd..)•  Manageable   o  Hide complexity of underlying resources   o  Rich functional end-user and a...
Problem Definition (contd..)•    Resource Allocation     o  Hypervisor CPU, Memory     o  Storage space     o  Avoid set o...
Scaling: Horizontal Scaling                Single-node                                             Multi-node             ...
Resource Load Balancing•  As management server is added into the cluster, resources are     rebalanced seamlessly.     o  ...
Cloud                                                Other                                           CLI           UI     ...
Interactions                                                                                                          OVM ...
Management Server Layering
Balancing Incoming Requests•  Each management server has two worker thread pools for     incoming requests: effectively tw...
Inside a Management ServerCloudStack   API                API               Servlet                                       ...
CloudStack API Sync/Async commands•    Package and Location     cloudstack-oss/api/src/com/cloud/api/…•  BaseCmd (base cla...
CloudStack APIConfigurationCommands are configured in cloudstack-oss/client/command.properties.inFormat:        <command n...
CloudStack API: adding APIAdding a new command  Determine type of command       Synchronous       Synchronous List Based  ...
Management Layer•  Management layer is collection of Managers   o  Managers are responsible for directing a specific area ...
Management Layer: Adapters•  Modularization and customization within the CloudStack  management server is achieved through...
Management Layer: Adapters•     Adapters are executed as a chain in the order that they are      configured•     Defined i...
Adapter Interfaces Available•  Discoverer              •  VirtualMachineGuru•  StoragePoolDiscoverer   •  HypervisorGuru• ...
Adapters: VM orchestration•  Deployment Planner   o  First Fit planner•  Host Allocator  o  First Fit  o  Random•  Storage...
Adapters: Network Orchestration•    Network Guru (Responsible for L2-L3)      o  Design      o  Implement      o  Allocate...
Extending CloudStack Networking                                 2. prepare (Network, Nic, DeployDestination, VmInfo) 1. pr...
Sequence Flow for VM Creation                                                                                             ...
Management Layer: Adapters flow
Server Resources                    Agent             •  Resources are carried in                                         ...
Resource Layer
Working toward 4.1 release•  4.1 is next major release  o  Moving away from monolithic architecture to loosely     coupled...
Session 3Developing with DevCloud
DevCloud• CloudStack requires  o  Hypervisor  o  Network  o  Storage
DevCloud• self-contained CloudStack runs in the appliance
DevCloud• Several use cases  o  Try CloudStack in an isolated sandbox. Runs within     the appliance  o  Develop CloudStac...
Thanks
Upcoming SlideShare
Loading in...5
×

Hacking apache cloud stack

24,155

Published on

0 Comments
13 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
24,155
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
333
Comments
0
Likes
13
Embeds 0
No embeds

No notes for slide

Hacking apache cloud stack

  1. 1. Hacking onApache (Incubating) CloudStack
  2. 2. Tutorial Outline•  Session 1: Introduction to CloudStack Murali Reddy: Committer Apache CloudStack•  Session 2: Architecture of CloudStack Murali Reddy: Committer Apache CloudStack•  Session 3: Hands on with DevCloud Kishan Kavala: Committer Apache CloudStack Rajesh Battala: Contributor Apache CloudStack
  3. 3. Session 1Introduction to CloudStack
  4. 4. Cloud ComputingVirtualization is not Cloud computing Server Virtualization++ Cloud Built for traditional enterprise Designed around big data, apps & client-server compute massive scale & next-gen apps •  Enterprise arch for 100s of hosts •  Cloud arch for 1000s of hosts •  Scale-up (pool-based resourcing) •  Scale-out (horizontal resourcing) •  IT management-centric •  Autonomic management •  1 administrator for Dozens of servers •  1 administrator for 1,000’s of servers •  Apps assume reliability •  Apps assume failure •  Proprietary vendor stack •  Open, value-added stack
  5. 5. Cloud Computing (contd..)•  Tenets of Cloud o  Shared infrastructure and Multi-tenancy o  Self Service o  Elasticity o  Built for massive Scale o  Service agility o  Pay-as-you-go o  APIs and Extreme Automation•  IAAS/PAAS/SAAS•  Public/Private/Hybrid clouds
  6. 6. What is Apache CloudStack•  Turnkey orchestration platform for delivering IAAS clouds o  Secure, multi-tenant o  Self-service o  Service agility and elasticity o  Built for large scale o  Pay-as-you-go•  Deploys on premise (private) or as a hosted (public) cloud•  Can be used for hybrid clouds•  built in java, provides native REST API’s and EC2 API•  Has python, Ruby clients and CLI as well
  7. 7. A  bit  of  History  •  Original  company  Cloud.com  (2008)  •  Open  source  (GPLv3)  as  CloudStack  (2010)  •  Acquired  by  Citrix  (July  2011)  •  Relicensed  under  ASL  v2  April  3,  2012  •  Accepted  as  Apache  IncubaKng  Project  April  16,   2012  •  First  Apache  (ACS  4.0)  released  •  Many  non-­‐Citrix  contributors,  commiRers,  PPMC   members  
  8. 8. Who is contributing•  Sungard: Unit test cases•  Carnigo: Object store plug-in•  Ceph/Rbd support by Wido•  CLVM/KVM by Marcus•  Nicira NVP: Schuberg Philis•  Basho: Object Store•  Brocade ADX ADC support•  Midokura midonet SDN controller integration
  9. 9. How to contribute•  Its not just about code! As community member you can engage in o Discussions: Design, Use Case, deployment issues o Bug reporting, feature requests o Code reviews o Build, tools, infrastructure o Helping out on the IRC o Documentation o Submit bug fixes, features
  10. 10. How to contribute (contd..)•  Git repo, bug tracker, wiki are on ASF infra•  Project website o  http://incubator.apache.org/cloudstack/ o  http://www.cloudstack.org•  IRC o  #cloudstack on irc.freenode.net o  Wednesday - 10:30 PM IST, 5:00 UTC•  Mailing lists (cloudstack.org/discuss/mailing- lists.html) o  cloudstack-dev-subscribe@incubator.apache.org o  cloudstack-users-subscribe@incubator.apache.org•  http://www.slideshare.net/cloudstack
  11. 11. On-demand infrastructure as a service Org A Org B Admin Admin Users Users Cloud Admin End User Provision Consumeresources resources UI Cli EC2 CloudStack managed cloud REST API Compute Network Storage CloudStack Management Server manage resources
  12. 12. Core CloudStack Components VM•  Hosts •  Servers onto which services will be provisioned Host VM Network•  Primary Storage Host •  VM storage•  Cluster Primary •  A grouping of hosts and their associated storage Storage•  Pod •  Collection of clusters Cluster•  Network Secondary Storage Cluster •  Logical network associated with service offerings•  Secondary Storage •  Template, snapshot and ISO storage CloudStack Pod•  Zone •  Collection of pods, network offerings and secondary CloudStack Pod storage•  Management Server Farm Zone •  Responsible for all management and provisioning tasks
  13. 13. CloudStack Deployment ArchitectureCloudStackManagement Internet Ø  Hypervisor is the basic unit Server of scale.Zone 1 Ø  Cluster consists of one ore more hosts of same L3 core hypervisor Ø  All hosts in cluster havePod 1 Access Layer Pod N access to shared (primary) Secondary storage …. Storage Cluster N Ø  Pod is one or more clusters, usually with L2 switches. …. Ø  Availability Zone has one or more pods, has access to Cluster 1 secondary storage. Host 1 Ø  One or more zones Primary represent cloud Storage Host 2
  14. 14. CloudStack Managing Multiple ZonesData Center 1 Data Center 2 Ø  Single Management Server can Data Center 2 Management Data Center 3 manage multiple zones Server Zone Zone Ø  Zones can be geographically 2 distributed but low latency links 2 Zone Zone are expected for betterZone1 Zone 3 performance 4 3 Ø  Single MS node can manage up to 5K hosts. Data Center 2 Ø  Multiple MS nodes can be Data Center 2 deployed as cluster for scale or Data Center 2 redundancy Zone Zone 2ZoneZone 2 2 Zone 3 Zone 3 3
  15. 15. Infrastructure provisioning
  16. 16. Infrastructure provisioning (contd.)
  17. 17. Compute/Disk/Network Offering
  18. 18. Create Virtual Machines via Offerings Select Operating System •  Windows, Linux Select Compute Offering •  CPU & RAM Select Disk Offering •  Volume Size Select Network Offering •  Network & Services Create VM
  19. 19. Virtual Machine Management Users ChangeVM Operations Console Access VM Status Service Offering Start •  CPU Utilized 2 CPUs 4 CPUs Stop 1 GB 4 GB •  Network Read RAM RAM Restart •  Network Writes 20 GB 200 GB Destroy 20 100 Mbps Mbps
  20. 20. Volume & Snapshot Management VM 1 Add / Delete Volumes VolumeCreate Templates Volume Template from Volumes Hourly Weekly Schedule Now Snapshots Daily Monthly …. View Snapshot History
  21. 21. A  Very  Flexible  IaaS  Pla5orm  Compute Hypervisor XenServer VMware Oracle VM KVM Bare metalStorage Block & Object Fiber Local Disk iSCSI NFS Swift Ceph Riak Channel Primary  Storage   Secondary  Storage  Network Network & Network Services Load Network Type Isolation Firewall VPN balancer
  22. 22. CloudStack Storage Primary Storage •  Configured at Cluster-level. Close to hosts for better performance •  Stores all disk volumes for VMs in a cluster L3 switch •  Cluster can have one or more primary storages L2 switch Pod 1 •  Local disk, iSCSI, FC or NFS Cluster 1 Secondary Storage Secondary Storage Host 1 Local •  Configured at Zone-level storage Primary Storage •  Stores all Templates, ISOs and Snapshots Host 2 •  Zone can have one or more secondary storages •  NFS, OpenStack Swift Availability zone Local Storage•  Storage available on hypervisor hist
  23. 23. Role of Storage and Templates•  Primary Storage •  Cluster level storage for VMs Host •  Connected directly to hosts •  NFS, iSCSI, FC and Local Host•  Secondary Storage Primary Storage •  Zone level storage for template, ISOs and Cluster snapshots •  NFS or OpenStack Swift via CloudStack Pod System VM•  Templates and ISOs •  Imported into CloudStack •  Can be private or public Secondary Storage Zone Template
  24. 24. Provisioning Process1.  User Requests Instance VM2.  Provision Optional Network Host Services Host3.  Copy instance template from Primary Storage secondary storage to primary Cluster storage on appropriate cluster Pod4.  Create any requested data volumes on primary storage for the Template cluster5.  Create instance Secondary Storage6.  Start instance Zone
  25. 25. Object Store CloudStack Mgmt •  Object store used to store Server templates and snapshots •  VM’s can be distributed across the availability zones •  For DR create instancesAvailability Zone Availability Zone Availability Zone in different zones Object Storage
  26. 26. Multi-tenancy & Account Management Resources Domain VMs, IPs, Snapshots… Domain is a unit of isolationOrg A that represents a customer Admin org, business unit or a reseller Domain Reseller A Domain can have arbitrary Admin Sub-Domain Resources levels of sub-domains VMs, IPs, Snapshots… Org C A Domain can have one or Admin more accounts Account Group A An Account represents one or more users and is the Account basic unit of isolation Group B Admin can limit resources at User 1 the Account or Domain levels User 2
  27. 27. User Dashboard: Consumed Resources•  Running, Stopped & Total VMs•  Public IPs•  Private networks•  Latest Events
  28. 28. Admin Dashboard: Consumed Resources•  Provides zone wide resource consumption•  Also provides latest alerts and events
  29. 29. Edge services with System VMs•  System VMs optimize and scale the datapath on behalf of CloudStack o  Stateless, can be destroyed and recreated from database state o  Highly Available o  Communicates with Management Server over management network o  Usually have 3 interfaces: control, guest and public•  Console Proxy VM o  Provides AJAX-style HTTP-only console viewer o  Grabs VNC output from hypervisor o  Scales out (more spawned) as load increases o  Java-based server Communicates with MS over message bus•  Secondary Storage VM o  Provides image (template) management services o  Download from HTTP file share or Swift o  Copy between zones o  Scale out to handle multiple NFS mounts o  Java-based server communicates with MS over message bus
  30. 30. Edge services with System VMs (contd.)•  Virtual Router VM o  Provides multiple network services o  IPAM (DHCP), DNS, NAT, Source NAT, Firewall, PF, VPN o  User-data, Meta-data, SSH keys and password change server o  Redundancy via VRRP o  MS configures VR over SSH §  Proxied via the hypervisor on XS and KVM
  31. 31. Network & Network Services•  Create Networks and attach VMs•  Acquire public IP address for NAT & load balancing•  Control traffic to VM using ingress and egress firewall rules•  Set up rules to load balance traffic between VMs
  32. 32. Networking feature overview•  Orchestration of L2 – L7 network services o  IPAM, DNS, Gateway, Firewall, NAT, LB, VPN, etc•  Mix-and-match services and providers•  Out-of-the-box integration with automated deployment of virtual routers o  Highly available network services using CloudStack HA and VRRP•  Orchestrate external providers such as hardware firewalls and load balancers o  Devices can provide multiple services o  Admin API to configure external devices o  Plugin-based extensions for network behavior and admin API extensions•  Multiple multi-tenancy [network isolation] options•  Integrated traffic accounting•  Access control•  Software Defined Networking (Nicira NVP)
  33. 33. L2 Features•  Choice of network isolation o  Physical, VLAN, L3 (anti-spoof), Overlay[GRE] o  Physical isolation through network labels [limited to # of nics or bonds]•  Multi-nic o  Deploy instance in multiple networks o  Control default route•  Access control o  Shared networks, project networks•  QoS [max rate]•  Traffic monitoring•  Hot-plug / detach of nics
  34. 34. L3 Features•  IPAM [DHCP], Public IP address management o  VR acts as DHCP server o  Can request multiple public IPs per tenant•  Gateway (default gateway) o  Redundant VR (using VRRP) o  Inter-subnet routing o  Static routing control•  Remote Access VPN o  L2TP over IPSec using PSK o  Virtual Router only•  Firewall based on source cidr•  Static NAT [1:1] o  Including “Elastic IP” in Basic Zone•  Source NAT o  Per-network, or interface NAT•  Public Traffic usage o  Monitoring on the Virtual Router / External network device o  Integration with sFlow collectors•  Site-to-Site VPN o  IPSec VPN based on VR•  L3 ACLs
  35. 35. L4 Features•  Security groups for L3-isolation o  “Basic Zone” in docs o  Default AWS-style networking o  Scales much better than VLANs•  Stateful firewall for TCP, UDP and ICMP•  Port forwarding [“Advanced Zone”] o  Conserve public Ips
  36. 36. L7 features•  Loadbalancer o  VR has HAProxy built in o  External Loadbalancer support §  Netscaler (MPX/SDX/VPX) §  F5 BigIP §  Can dedicate an LB appliance to an account or share it among tenants o  Loadbalancer supported with L3-isolation as well o  Stickiness support o  SSL support [future] o  Health Checks [future]•  User-data & meta-data o  Fetched from virtual router•  Password change server
  37. 37. CloudStack Terminology•  Guest network o  The tenant network to which instances are attached•  Storage network o  The physical network which connects the hypervisor to primary storage•  Management network o  Control Plane traffic between CloudStack management server and hypervisor clusters•  Public network o  “Outside” the cloud [usually Internet] o  Shared public VLANs trunked down to all hypervisors•  All traffic can be multiplexed on to the same underlying physical network using VLANs o  Usually Management network is untagged o  Storage network usually on separate nic (or bond)•  Admin informs CloudStack how to map these network types to the underlying physical network o  Configure traffic labels on the hypervisor o  Configure traffic labels on Admin UI
  38. 38. CloudStack Network Service Providers•  A Network Service Provider is hardware or virtual appliance that makes a network service possible in CloudStack ; for example, a Citrix NetScaler appliance can be installed in the cloud to provide Load-Balancing services.•  Administrators can have multiple instances of the same service provider in a network; for example, more than one Citrix NetScaler or Juniper SRX device can be added to CloudStack•  CloudStack supports the following Network Providers: o  CloudStack Virtual Router (default) o  Citrix NetScaler SDX, VPX and MPX models o  Juniper SRX o  F5 BigIP
  39. 39. Network Service Providers Matrix•  Network offerings is basically a definition of what Network Services are available when this offering is used. The available Network Services are: VPN, DHCP, DNS, Firewall, Load Balancer, User Data, Source NAT, Static NAT, Port Forwarding and Security Groups*Feature Virtual Citrix Juniper F5 BigIP Router NetScaler SRXRemote Access VPN YES N/A N/A N/AFirewall YES N/A YES N/ASource NAT YES N/A YES N/AStatic NAT YES YES YES N/ALoad Balancing YES YES N/A YESPort Forwarding YES N/A YES N/AElastic IP N/A YES N/A N/AElastic LB N/A YES N/A N/ADHCP/DNS/User Data YES N/A N/A N/A
  40. 40. Network Offerings•  Cloud provider defines the feature set for guest networks•  Toggle features or service levels o  Security groups on/off o  Load balancer on/off o  Load balancer software/hardware o  VPN, firewall, port forwarding•  User chooses network offering when creating network•  Enables upgrade between network offerings•  Default offerings built-in o  For classic CloudStack networking
  41. 41. Add Guest Networks•  Choice to choose L3 subnet, default gateway•  Choice of network offerings
  42. 42. Editing Guest NetworksWhen editing a guest network users canchange the network offering. They caneither upgrade to a “premium” networkoffering (for example offering that useshardware Load-balancer) or downgrade to a“cheaper” network.
  43. 43. Restarting/Cleaning Up a Guest Network•  Restarting the network will simply resend all the LB, Firewall and Port-Forwarding rules to the network provider•  Restarting the Network with “Clean up”: •  restarKng  network  elements  -­‐  virtual  routers,  DHCP   servers   •  If  virtual  router  is  used,  it  will  be  destroyed  and   recreated     •  Reapplying  all  public  IPs  to  the  network  provider   •  Reapplying  load-­‐Balancing/Port-­‐Forwarding/Firewall   rules  
  44. 44. Deleting a Guest Network•  An Isolated Guest Network can only be deleted if no VMs are using these network (e.g. Completely destroyed and expunged)•  Deleting a Network will Destroy the Virtual Router (if used) and will release the Public IPs back to the IP Pool
  45. 45. Basic vs Advanced Networking•  Segmentation based on feature set and ease-of- deployment•  Both are feature-rich•  Basic implements true AWS-style L3-isolation o  Tenants do not get contiguous IP addresses or subnets o  Network segmentation based on Security Groups o  Tremendous scale (tens of thousands)•  Advanced Zone offers full L3 subnets and L2 isolation o  VLANs are default implementation (4K limit) o  More features (source NAT, PF, LB, VPN)
  46. 46. Physical Network in Zone Core (L3) Network Pod 1 Pod 2 Pod N Cloudstack   Access  Switch(es) Server   Cloudstack   Servers CLUSTER 1 …   Hypervisor  1VM Traffic …   Hypervisor  8Control Plane TrafficStorage Traffic Storage 2 Storage 1Public Traffic …   CLUSTER 4 Hypervisor  N Hypervisor  N+1 Storage k
  47. 47. Layer 3 cloud networking Web DB Web VM VM VM Web DB Security Security Group Group Web Web DB VM VM VM… … … Web Web VM VM
  48. 48. Guest Networks with L3 isolationPublic   Public  IP   Guest   Guest  Internet address   1  VM  1 address   65.37.141.11   10.1.0.2 10.1.0.1 Guest   65.37.141.24   Pod  1  L2   Guest   65.37.141.36   Switch 2  VM  1 address   65.37.141.80   10.1.0.3   Guest   Guest   1  VM  2 address   L3  Core   Switch Pod  2  L2   Switch 10.1.8.1 … 10.1.0.4 Guest   Guest   10.1.16. 2  VM  2 address   Load   Pod  3  L2   Balancer 1 10.1.16.12 Switch Guest   2  VM  3 Guest   address   10.1.16.21 … Guest   1  VM  3 Guest   address   10.1.16.47 Guest   Guest   1  VM  4 address   10.1.16.85
  49. 49. Guest Networks with L2 isolation Core (L3) Network Pod K Pod M Pod N Access  Switch(es) V V Hypervisor R V Hypervisor CLUSTER 1 …   Hypervisor  1 RVM Traffic …   Hypervisor  8Public Traffic …   CLUSTER 4 V V Hypervisor  N V Tenant VM Hypervisor  N+1 V R Tenant Virtual Router
  50. 50. L2 isolation: VLAN networking User 1 User 1 User User 1 2 User 1 User User 2 1 User 2… … … User 1
  51. 51. SDN at Work CloudStack Mgmt Server SDN Controlle r Host 1 OVS Host 3 OVS VM VM VM V 1 1 3 RGRE Tunnel GRE Tunnel Host 2 OVS Host 4 OVS VM VM VM V 2 2 3 RGRE Tunnel GRE Tunnel
  52. 52. Guest virtual layer-2 network Guest  Virtual  Network   10.1.1.0/24 Public   Public  IP   Guest   Gateway   Guest   Network address   1  VM  1 address   address   65.37.141.11   10.1.1.1 10.1.1.2 65.37.141.36 Guest  1   Guest   Guest  Public   Virtual   1  VM  2 address  Internet Router 10.1.1.3 NAT   Guest   Guest   DHCP   1  VM  3 address   Load   10.1.1.4 Balancing   Guest   Guest   VPN 1  VM  4 address   10.1.1.5 Guest  Virtual  Network   Public  IP   10.1.1.0/24 address   Gateway   Guest   Guest   65.37.141.24   address   2  VM  1 address   65.37.141.80 10.1.1.1 10.1.1.2 Guest  2   Guest   Guest   Virtual   2  VM  2 address   Router 10.1.1.3 NAT   Guest   Guest   DHCP   2  VM  3 address   Load   10.1.1.4 Balancing   VPN
  53. 53. Layer-2 Guest Virtual Network CS Virtual Router provides Network Services External Devices provide Network Services Guest  Virtual  Network  10.1.1.1/8   Guest  Virtual  Network  10.1.1.1/8   VLAN  100 VLAN  100Public   Public  Network/ Network/Internet Guest Internet Guest Public  IP   Private  IP   10.1.1.1 10.1.1.1 VM 1 10.1.1.111 VM 1 Gateway   65.37.141.111 JuniperPublic  IP   SRX address  65.37.141.11 CS Firewall 10.1.1.1 Guest Guest Virtual 10.1.1.3 VM 2 10.1.1.3 VM 2 Router Public  IP   Private  IP   DHCP,  DNS   65.37.141. NetScaler 10.1.1.112 NAT   Guest 112 Load Guest Load  Balancing   10.1.1.4 VM 3 Blancer VM 3 10.1.1.4 VPN Guest Guest 10.1.1.5 VM 4 10.1.1.5 VM 4 CS DHCP,   Virtual Router DNS  
  54. 54. Layer-3 Guest NetworkNetwork Services Managed Externally Network Services Managed by CS Public  Network   65.11.0.0/16 Security  Group   Security  Group   Public  Network/ 1 1 Internet 10.1.2.3 65.11.1.2 Guest Guest VM 1 VM 1 10.2.12.4 65.11.1.3 65.11.1.2 NetScaler L3 Guest Guest 65.11.1.3 Load switch VM 2 VM 2 Blancer 65.11.1.4 EIP,   ELB   10.5.2.99 65.11.1.4 Guest Guest VM 3 VM 3 10.1.2.18 65.11.1.5 Guest Guest VM 4 VM 4 CS CS Virtual DHCP,   Virtual Security  Group   DHCP,   Route Security  Group   Router DNS   2 DNS   r 2
  55. 55. Multi-tier network Internet IPSec or SSL site-to-site VPN CS Customer Virtual Router Loadbalancer Premises Monitoring VLANVirtual Router Services App VM•  IPAM 10.1.2.31 1•  DNS 10.1.1.1 Web VM 1•  LB [intra]•  S-2-S VPN App VM 10.1.2.24•  Static Routes Web VM 2•  ACLs 10.1.1.3 2•  NAT, PF•  FW [ingress & egress] Web VM DB VM•  BGP 10.1.1.4 3 10.1.3.24 1 Web VM 10.1.1.5 4 Virtual  Network     Virtual  Network     Virtual  Network     10.1.1.0/24   10.1.2.0/24   10.1.3.0/24   VLAN  100 VLAN  1001 VLAN  141
  56. 56. Session 2Architecture of CloudStack
  57. 57. Problem Definition•  Offer a scalable, flexible, manageable IAAS platform that orchestrate physical and virtual resources to offer self-service infrastructure provisioning and monitoring•  Flexible o  Handle new physical resource types § Hypervisors, storage, networking o  Add new APIs o  Add new services o  Add new networking models
  58. 58. Problem Definition (contd..)•  Manageable o  Hide complexity of underlying resources o  Rich functional end-user and admin UI o  Admin API to automate operations o  Easy install, upgrade for small -> large clouds o  Simple scaling, automated resilience•  Scalable architecture o  1 -> N hypervisors / VMs / virtual resources o  1 -> N end users
  59. 59. Problem Definition (contd..)•  Resource Allocation o  Hypervisor CPU, Memory o  Storage space o  Avoid set of pods, clusters, hosts•  Capacity scanning o  Snapshot of resources consumed o  Trigger capacity threshold violations•  Garbage collection o  Network resources (IP, VLAN, CIDR etc) o  Compute (VM, CPU, memory) o  Storage (volumes)•  Synchronizing the resource states•  Infrastructure resource failures•  Fencing
  60. 60. Scaling: Horizontal Scaling Single-node Multi-node Deployment Deployment Manage ment ServerUser API User API Manage Manage ment MySQL Load ment Server DB Balancer ServerAdmin API Admin API Manage MySQL ment DB Server Back Up DB Replication Ø  MS is stateless. MS can be deployed as physical server or VM Infrastructure Infrastructure Ø  Single MS node can manage up Resources Resources to 10K hosts. Multiple nodes can be deployed for scale or redundancy
  61. 61. Resource Load Balancing•  As management server is added into the cluster, resources are rebalanced seamlessly. o  MS2 signals to MS1 to hand over a resource o  MS1 wait for the commands on the resources to finish o  MS1 holds further commands in a queue o  MS1 signals to MS2 to take over o  MS2 connects o  MS2 signals to MS1 to complete transfer o  MS1 discards its resource and flows the commands being held to MS2•  Listeners are provided to business logic to listen on connection status and adjusts work based on who’s connected.•  By only working on resources that are connected to the management server the process is on, work is auto-balanced between management servers.•  Also reduces the message routing between the management servers.
  62. 62. Cloud Other CLI UI Clients Portal Management Server REST API End User Other Pluggable Service API OAM&P API EC2 API API APIs EngineConsole Proxy ACL & Authentication Security AdaptersManagement -  Accounts, Domains, and Projects -  ACL, limits checking Account Management Connectors Template Services API Access Plugin API Deployment Planning HA Kernel Job Services API -  Drives long running VM Network Configurations Queue Usage operations Calculations -  Syncs between resources managed and DB Network Elements Additional -  Generates events Services Hypervisor Gurus Cluster Resource Job Alert & Event Database Managemen Managemen Management Management Access DB t t Event Bus Message Bus Hypervisor Network Storage Image Snapshot Resources Resources Resources Resources Resources
  63. 63. Interactions OVM Cluster Primary Storage vcenter Monitoring Primary CS API vSphere Cluster Storage End User UI Primary XS Cluster Storage Admin UI Clustered CloudStack XAPI Domai CS Admin & CloudStack CloudStack n End-user API Primary Admin Management JSON KVM Cluster Storage UI Server NetConf Juniper SRXCloud user Nitro API{API client (Fog/etc)} VNC JSON ec2 API JSON Netscaler Cloud user Console Console {ec2 API client } Proxy VM Proxy VM NFS MySQL Server Sec. {Proxied} SSH Sec. Storage NFS NFS Storage VM Ajax HTTPS VM Console Router VM HTTP (Template Download) Router VM HTTP (Template Copy) Router VM Cloud user HTTP (Swift)
  64. 64. Management Server Layering
  65. 65. Balancing Incoming Requests•  Each management server has two worker thread pools for incoming requests: effectively two servers in one. o  Executor threads provided by tomcat o  Job threads waiting on job queue•  All incoming requests that requires mostly DB operations are short in duration and are executed by executor threads because incoming requests are already load balanced by the load balancer•  All incoming requests needing resources, which often have long running durations, are checked against ACL by the executor threads and then queued and picked up by job threads.•  # of job threads are scaled to the # of DB connections available to the management server•  Requests may take a long time depending on the constraint of the resources but they don’t fail.
  66. 66. Inside a Management ServerCloudStack API API Servlet Plugins Commands cmd.execute() Plugins Plugins Async Job Queue Services Agent API Kernel (Cmds) Mgr API Responses Mess age Resources Bus Local Or Remote Agent Manager Hypervisor Network Native Device APIs API MySQL
  67. 67. CloudStack API Sync/Async commands•  Package and Location cloudstack-oss/api/src/com/cloud/api/…•  BaseCmd (base class)All commands descend from the BaseCmd base class
  68. 68. CloudStack APIConfigurationCommands are configured in cloudstack-oss/client/command.properties.inFormat: <command name>=<java classname>;<ACL> *note* ACL is calculated as a bitmap with the following, 1 = ADMIN, 2 = RESOURCE_DOMAIN_ADMIN, 4 = DOMAIN_ADMIN, 8 = USERExample: ### snapshot commands! createSnapshot=com.cloud.api.commands.CreateSnapshotCmd;15! listSnapshots=com.cloud.api.commands.ListSnapshotsCmd;15! deleteSnapshot=com.cloud.api.commands.DeleteSnapshotCmd;15! createSnapshotPolicy=com.cloud.api.commands.CreateSnapshotPoli cyCmd;15! deleteSnapshotPolicies=com.cloud.api.commands.DeleteSnapshotPo liciesCmd;15! listSnapshotPolicies=com.cloud.api.commands.ListSnapshotPolici esCmd;15!
  69. 69. CloudStack API: adding APIAdding a new command Determine type of command Synchronous Synchronous List Based Asynchronous Asynchronous Create based Create your command Define request parameters Implement the execute() method Implement an appropriate ResponseObject Add new command to command.properties.in
  70. 70. Management Layer•  Management layer is collection of Managers o  Managers are responsible for directing a specific area of the cloud §  Storage Manager •  Manages primary storage server (allocation, life-cycle, attach, detach, user volumes, life-cycle of the primary storage server itself) §  Network Manager •  Manages network configurations, IP Allocations, Port Forwarding, Load Balancers etc. §  User Vm Manager •  Manages life-cycle of VMs created in the cloud §  And many more!!!•  Managers coordinate with each other to achieve a task
  71. 71. Management Layer: Adapters•  Modularization and customization within the CloudStack management server is achieved through the use of the Adapter framework.•  Each Adapter is uniquely identified by the interface it exposes and represents the boundary between CloudStack and the individual component and/or processes that can be configured into the system•  Adapters provide extensibility and in many cases device specific implementation details while maintaining a simple and consistent interface.
  72. 72. Management Layer: Adapters•  Adapters are executed as a chain in the order that they are configured•  Defined in cloudstack-oss/client/tomcatconf/components.xml.in<adapters key="com.cloud.network.guru.NetworkGuru”> <adapter name="StorageNetworkGuru” class="com.cloud.network.guru.StorageNetworkGuru"/> <adapter name="ExternalGuestNetworkGuru" class="com.cloud.network.guru.ExternalGuestNetworkGuru"/> <adapter name="PublicNetworkGuru" class="com.cloud.network.guru.PublicNetworkGuru"/> <adapter name="PodBasedNetworkGuru" class="com.cloud.network.guru.PodBasedNetworkGuru"/> <adapter name="ControlNetworkGuru" class="com.cloud.network.guru.ControlNetworkGuru"/> <adapter name="DirectNetworkGuru" class="com.cloud.network.guru.DirectNetworkGuru"/> <adapter name="DirectPodBasedNetworkGuru" class="com.cloud.network.guru.DirectPodBasedNetworkGuru"/> <adapter name="OvsGuestNetworkGuru" class="com.cloud.network.guru.OvsGuestNetworkGuru"/></adapters>
  73. 73. Adapter Interfaces Available•  Discoverer •  VirtualMachineGuru•  StoragePoolDiscoverer •  HypervisorGuru•  StoragePoolAllocator •  Listener•  ConsoleProxyAllocator •  UserAuthenticator•  Investigator •  SecurityChecker•  FenceBuilder•  DeploymentPlanner•  NetworkGuru•  NetworkElement•  And more…
  74. 74. Adapters: VM orchestration•  Deployment Planner o  First Fit planner•  Host Allocator o  First Fit o  Random•  Storage Allocator o  First Fit o  Random
  75. 75. Adapters: Network Orchestration•  Network Guru (Responsible for L2-L3) o  Design o  Implement o  Allocate o  Release o  Shutdown e.g. guest network guru, OVS network guru etc•  Network Element (Responsible for L4-L7) o  Implement o  Shutdown e.g. F5, SRX, NetScaler, Virtual Router
  76. 76. Extending CloudStack Networking 2. prepare (Network, Nic, DeployDestination, VmInfo) 1. prepare (part of start vm) Network PluggableServi Network Element ce Manager Device Configuration MyDnsDeviceS Admin API (CRUD) DnsService ervice 3. addDnsRecord(ip, fqdn)Demonstrates one way to MyDnsDeviceM MySQL MyDnsElementinform an external DNS anagerserver when an instancestarts. AgentMana 4.Enqueue AddDnsRecord ger QueueClasses shaded blue forma plugin / service bundleto integrate an external MyDnsDeviceRDNS server. Clients of the esourceinstance can then useDNS names to access the 5.API call to Dns Deviceinstance.
  77. 77. Sequence Flow for VM Creation Deployme Server Job Services User VM VirtualMac Network Storage Network Network Templat nt ResourceThreads API Mgr hine Mgr Mgr Mgr Guru Element e Mgr Planner s Start VM Start User VM Start VM Get a Deployment Plan (Host and StoragePool) Prepare Nics Reserve resources for Nic Notify that Nic is about to be started in network Agent Calls Prepare Volumes Prepare template on Primary Storage Agent Calls Agent Start VM Call Stores job result
  78. 78. Management Layer: Adapters flow
  79. 79. Server Resources Agent •  Resources are carried in service VMs to be in close Hypervisor Resources network proximity to the physical resources it managesResource API Network Resources •  Easily scales to utilize the most abundant resource in data Storage Resources center (CPU & RAM) Image & Template •  Communicates with Resources Orchestration Server over message bus (JSON) Snapshot Resources •  Can be replicated for fault tolerance •  Control gateway to resources within data center
  80. 80. Resource Layer
  81. 81. Working toward 4.1 release•  4.1 is next major release o  Moving away from monolithic architecture to loosely coupled subsystems o  Spring for IOC container and AOP o  Storage subsystem refactoring o  Network subsystem refactoring o  New orchestration engine o  Regions support
  82. 82. Session 3Developing with DevCloud
  83. 83. DevCloud• CloudStack requires o  Hypervisor o  Network o  Storage
  84. 84. DevCloud• self-contained CloudStack runs in the appliance
  85. 85. DevCloud• Several use cases o  Try CloudStack in an isolated sandbox. Runs within the appliance o  Develop CloudStack on own machine, build locally and deploy new version in DevCloud (Build and test) o  Develop and Run locally, use DevCloud as Xen hosts
  86. 86. Thanks
  1. ¿Le ha llamado la atención una diapositiva en particular?

    Recortar diapositivas es una manera útil de recopilar información importante para consultarla más tarde.

×