Hacking apache cloud stack

  • 18,709 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
18,709
On Slideshare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
307
Comments
0
Likes
12

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Hacking onApache (Incubating) CloudStack
  • 2. Tutorial Outline•  Session 1: Introduction to CloudStack Murali Reddy: Committer Apache CloudStack•  Session 2: Architecture of CloudStack Murali Reddy: Committer Apache CloudStack•  Session 3: Hands on with DevCloud Kishan Kavala: Committer Apache CloudStack Rajesh Battala: Contributor Apache CloudStack
  • 3. Session 1Introduction to CloudStack
  • 4. Cloud ComputingVirtualization is not Cloud computing Server Virtualization++ Cloud Built for traditional enterprise Designed around big data, apps & client-server compute massive scale & next-gen apps •  Enterprise arch for 100s of hosts •  Cloud arch for 1000s of hosts •  Scale-up (pool-based resourcing) •  Scale-out (horizontal resourcing) •  IT management-centric •  Autonomic management •  1 administrator for Dozens of servers •  1 administrator for 1,000’s of servers •  Apps assume reliability •  Apps assume failure •  Proprietary vendor stack •  Open, value-added stack
  • 5. Cloud Computing (contd..)•  Tenets of Cloud o  Shared infrastructure and Multi-tenancy o  Self Service o  Elasticity o  Built for massive Scale o  Service agility o  Pay-as-you-go o  APIs and Extreme Automation•  IAAS/PAAS/SAAS•  Public/Private/Hybrid clouds
  • 6. What is Apache CloudStack•  Turnkey orchestration platform for delivering IAAS clouds o  Secure, multi-tenant o  Self-service o  Service agility and elasticity o  Built for large scale o  Pay-as-you-go•  Deploys on premise (private) or as a hosted (public) cloud•  Can be used for hybrid clouds•  built in java, provides native REST API’s and EC2 API•  Has python, Ruby clients and CLI as well
  • 7. A  bit  of  History  •  Original  company  Cloud.com  (2008)  •  Open  source  (GPLv3)  as  CloudStack  (2010)  •  Acquired  by  Citrix  (July  2011)  •  Relicensed  under  ASL  v2  April  3,  2012  •  Accepted  as  Apache  IncubaKng  Project  April  16,   2012  •  First  Apache  (ACS  4.0)  released  •  Many  non-­‐Citrix  contributors,  commiRers,  PPMC   members  
  • 8. Who is contributing•  Sungard: Unit test cases•  Carnigo: Object store plug-in•  Ceph/Rbd support by Wido•  CLVM/KVM by Marcus•  Nicira NVP: Schuberg Philis•  Basho: Object Store•  Brocade ADX ADC support•  Midokura midonet SDN controller integration
  • 9. How to contribute•  Its not just about code! As community member you can engage in o Discussions: Design, Use Case, deployment issues o Bug reporting, feature requests o Code reviews o Build, tools, infrastructure o Helping out on the IRC o Documentation o Submit bug fixes, features
  • 10. How to contribute (contd..)•  Git repo, bug tracker, wiki are on ASF infra•  Project website o  http://incubator.apache.org/cloudstack/ o  http://www.cloudstack.org•  IRC o  #cloudstack on irc.freenode.net o  Wednesday - 10:30 PM IST, 5:00 UTC•  Mailing lists (cloudstack.org/discuss/mailing- lists.html) o  cloudstack-dev-subscribe@incubator.apache.org o  cloudstack-users-subscribe@incubator.apache.org•  http://www.slideshare.net/cloudstack
  • 11. On-demand infrastructure as a service Org A Org B Admin Admin Users Users Cloud Admin End User Provision Consumeresources resources UI Cli EC2 CloudStack managed cloud REST API Compute Network Storage CloudStack Management Server manage resources
  • 12. Core CloudStack Components VM•  Hosts •  Servers onto which services will be provisioned Host VM Network•  Primary Storage Host •  VM storage•  Cluster Primary •  A grouping of hosts and their associated storage Storage•  Pod •  Collection of clusters Cluster•  Network Secondary Storage Cluster •  Logical network associated with service offerings•  Secondary Storage •  Template, snapshot and ISO storage CloudStack Pod•  Zone •  Collection of pods, network offerings and secondary CloudStack Pod storage•  Management Server Farm Zone •  Responsible for all management and provisioning tasks
  • 13. CloudStack Deployment ArchitectureCloudStackManagement Internet Ø  Hypervisor is the basic unit Server of scale.Zone 1 Ø  Cluster consists of one ore more hosts of same L3 core hypervisor Ø  All hosts in cluster havePod 1 Access Layer Pod N access to shared (primary) Secondary storage …. Storage Cluster N Ø  Pod is one or more clusters, usually with L2 switches. …. Ø  Availability Zone has one or more pods, has access to Cluster 1 secondary storage. Host 1 Ø  One or more zones Primary represent cloud Storage Host 2
  • 14. CloudStack Managing Multiple ZonesData Center 1 Data Center 2 Ø  Single Management Server can Data Center 2 Management Data Center 3 manage multiple zones Server Zone Zone Ø  Zones can be geographically 2 distributed but low latency links 2 Zone Zone are expected for betterZone1 Zone 3 performance 4 3 Ø  Single MS node can manage up to 5K hosts. Data Center 2 Ø  Multiple MS nodes can be Data Center 2 deployed as cluster for scale or Data Center 2 redundancy Zone Zone 2ZoneZone 2 2 Zone 3 Zone 3 3
  • 15. Infrastructure provisioning
  • 16. Infrastructure provisioning (contd.)
  • 17. Compute/Disk/Network Offering
  • 18. Create Virtual Machines via Offerings Select Operating System •  Windows, Linux Select Compute Offering •  CPU & RAM Select Disk Offering •  Volume Size Select Network Offering •  Network & Services Create VM
  • 19. Virtual Machine Management Users ChangeVM Operations Console Access VM Status Service Offering Start •  CPU Utilized 2 CPUs 4 CPUs Stop 1 GB 4 GB •  Network Read RAM RAM Restart •  Network Writes 20 GB 200 GB Destroy 20 100 Mbps Mbps
  • 20. Volume & Snapshot Management VM 1 Add / Delete Volumes VolumeCreate Templates Volume Template from Volumes Hourly Weekly Schedule Now Snapshots Daily Monthly …. View Snapshot History
  • 21. A  Very  Flexible  IaaS  Pla5orm  Compute Hypervisor XenServer VMware Oracle VM KVM Bare metalStorage Block & Object Fiber Local Disk iSCSI NFS Swift Ceph Riak Channel Primary  Storage   Secondary  Storage  Network Network & Network Services Load Network Type Isolation Firewall VPN balancer
  • 22. CloudStack Storage Primary Storage •  Configured at Cluster-level. Close to hosts for better performance •  Stores all disk volumes for VMs in a cluster L3 switch •  Cluster can have one or more primary storages L2 switch Pod 1 •  Local disk, iSCSI, FC or NFS Cluster 1 Secondary Storage Secondary Storage Host 1 Local •  Configured at Zone-level storage Primary Storage •  Stores all Templates, ISOs and Snapshots Host 2 •  Zone can have one or more secondary storages •  NFS, OpenStack Swift Availability zone Local Storage•  Storage available on hypervisor hist
  • 23. Role of Storage and Templates•  Primary Storage •  Cluster level storage for VMs Host •  Connected directly to hosts •  NFS, iSCSI, FC and Local Host•  Secondary Storage Primary Storage •  Zone level storage for template, ISOs and Cluster snapshots •  NFS or OpenStack Swift via CloudStack Pod System VM•  Templates and ISOs •  Imported into CloudStack •  Can be private or public Secondary Storage Zone Template
  • 24. Provisioning Process1.  User Requests Instance VM2.  Provision Optional Network Host Services Host3.  Copy instance template from Primary Storage secondary storage to primary Cluster storage on appropriate cluster Pod4.  Create any requested data volumes on primary storage for the Template cluster5.  Create instance Secondary Storage6.  Start instance Zone
  • 25. Object Store CloudStack Mgmt •  Object store used to store Server templates and snapshots •  VM’s can be distributed across the availability zones •  For DR create instancesAvailability Zone Availability Zone Availability Zone in different zones Object Storage
  • 26. Multi-tenancy & Account Management Resources Domain VMs, IPs, Snapshots… Domain is a unit of isolationOrg A that represents a customer Admin org, business unit or a reseller Domain Reseller A Domain can have arbitrary Admin Sub-Domain Resources levels of sub-domains VMs, IPs, Snapshots… Org C A Domain can have one or Admin more accounts Account Group A An Account represents one or more users and is the Account basic unit of isolation Group B Admin can limit resources at User 1 the Account or Domain levels User 2
  • 27. User Dashboard: Consumed Resources•  Running, Stopped & Total VMs•  Public IPs•  Private networks•  Latest Events
  • 28. Admin Dashboard: Consumed Resources•  Provides zone wide resource consumption•  Also provides latest alerts and events
  • 29. Edge services with System VMs•  System VMs optimize and scale the datapath on behalf of CloudStack o  Stateless, can be destroyed and recreated from database state o  Highly Available o  Communicates with Management Server over management network o  Usually have 3 interfaces: control, guest and public•  Console Proxy VM o  Provides AJAX-style HTTP-only console viewer o  Grabs VNC output from hypervisor o  Scales out (more spawned) as load increases o  Java-based server Communicates with MS over message bus•  Secondary Storage VM o  Provides image (template) management services o  Download from HTTP file share or Swift o  Copy between zones o  Scale out to handle multiple NFS mounts o  Java-based server communicates with MS over message bus
  • 30. Edge services with System VMs (contd.)•  Virtual Router VM o  Provides multiple network services o  IPAM (DHCP), DNS, NAT, Source NAT, Firewall, PF, VPN o  User-data, Meta-data, SSH keys and password change server o  Redundancy via VRRP o  MS configures VR over SSH §  Proxied via the hypervisor on XS and KVM
  • 31. Network & Network Services•  Create Networks and attach VMs•  Acquire public IP address for NAT & load balancing•  Control traffic to VM using ingress and egress firewall rules•  Set up rules to load balance traffic between VMs
  • 32. Networking feature overview•  Orchestration of L2 – L7 network services o  IPAM, DNS, Gateway, Firewall, NAT, LB, VPN, etc•  Mix-and-match services and providers•  Out-of-the-box integration with automated deployment of virtual routers o  Highly available network services using CloudStack HA and VRRP•  Orchestrate external providers such as hardware firewalls and load balancers o  Devices can provide multiple services o  Admin API to configure external devices o  Plugin-based extensions for network behavior and admin API extensions•  Multiple multi-tenancy [network isolation] options•  Integrated traffic accounting•  Access control•  Software Defined Networking (Nicira NVP)
  • 33. L2 Features•  Choice of network isolation o  Physical, VLAN, L3 (anti-spoof), Overlay[GRE] o  Physical isolation through network labels [limited to # of nics or bonds]•  Multi-nic o  Deploy instance in multiple networks o  Control default route•  Access control o  Shared networks, project networks•  QoS [max rate]•  Traffic monitoring•  Hot-plug / detach of nics
  • 34. L3 Features•  IPAM [DHCP], Public IP address management o  VR acts as DHCP server o  Can request multiple public IPs per tenant•  Gateway (default gateway) o  Redundant VR (using VRRP) o  Inter-subnet routing o  Static routing control•  Remote Access VPN o  L2TP over IPSec using PSK o  Virtual Router only•  Firewall based on source cidr•  Static NAT [1:1] o  Including “Elastic IP” in Basic Zone•  Source NAT o  Per-network, or interface NAT•  Public Traffic usage o  Monitoring on the Virtual Router / External network device o  Integration with sFlow collectors•  Site-to-Site VPN o  IPSec VPN based on VR•  L3 ACLs
  • 35. L4 Features•  Security groups for L3-isolation o  “Basic Zone” in docs o  Default AWS-style networking o  Scales much better than VLANs•  Stateful firewall for TCP, UDP and ICMP•  Port forwarding [“Advanced Zone”] o  Conserve public Ips
  • 36. L7 features•  Loadbalancer o  VR has HAProxy built in o  External Loadbalancer support §  Netscaler (MPX/SDX/VPX) §  F5 BigIP §  Can dedicate an LB appliance to an account or share it among tenants o  Loadbalancer supported with L3-isolation as well o  Stickiness support o  SSL support [future] o  Health Checks [future]•  User-data & meta-data o  Fetched from virtual router•  Password change server
  • 37. CloudStack Terminology•  Guest network o  The tenant network to which instances are attached•  Storage network o  The physical network which connects the hypervisor to primary storage•  Management network o  Control Plane traffic between CloudStack management server and hypervisor clusters•  Public network o  “Outside” the cloud [usually Internet] o  Shared public VLANs trunked down to all hypervisors•  All traffic can be multiplexed on to the same underlying physical network using VLANs o  Usually Management network is untagged o  Storage network usually on separate nic (or bond)•  Admin informs CloudStack how to map these network types to the underlying physical network o  Configure traffic labels on the hypervisor o  Configure traffic labels on Admin UI
  • 38. CloudStack Network Service Providers•  A Network Service Provider is hardware or virtual appliance that makes a network service possible in CloudStack ; for example, a Citrix NetScaler appliance can be installed in the cloud to provide Load-Balancing services.•  Administrators can have multiple instances of the same service provider in a network; for example, more than one Citrix NetScaler or Juniper SRX device can be added to CloudStack•  CloudStack supports the following Network Providers: o  CloudStack Virtual Router (default) o  Citrix NetScaler SDX, VPX and MPX models o  Juniper SRX o  F5 BigIP
  • 39. Network Service Providers Matrix•  Network offerings is basically a definition of what Network Services are available when this offering is used. The available Network Services are: VPN, DHCP, DNS, Firewall, Load Balancer, User Data, Source NAT, Static NAT, Port Forwarding and Security Groups*Feature Virtual Citrix Juniper F5 BigIP Router NetScaler SRXRemote Access VPN YES N/A N/A N/AFirewall YES N/A YES N/ASource NAT YES N/A YES N/AStatic NAT YES YES YES N/ALoad Balancing YES YES N/A YESPort Forwarding YES N/A YES N/AElastic IP N/A YES N/A N/AElastic LB N/A YES N/A N/ADHCP/DNS/User Data YES N/A N/A N/A
  • 40. Network Offerings•  Cloud provider defines the feature set for guest networks•  Toggle features or service levels o  Security groups on/off o  Load balancer on/off o  Load balancer software/hardware o  VPN, firewall, port forwarding•  User chooses network offering when creating network•  Enables upgrade between network offerings•  Default offerings built-in o  For classic CloudStack networking
  • 41. Add Guest Networks•  Choice to choose L3 subnet, default gateway•  Choice of network offerings
  • 42. Editing Guest NetworksWhen editing a guest network users canchange the network offering. They caneither upgrade to a “premium” networkoffering (for example offering that useshardware Load-balancer) or downgrade to a“cheaper” network.
  • 43. Restarting/Cleaning Up a Guest Network•  Restarting the network will simply resend all the LB, Firewall and Port-Forwarding rules to the network provider•  Restarting the Network with “Clean up”: •  restarKng  network  elements  -­‐  virtual  routers,  DHCP   servers   •  If  virtual  router  is  used,  it  will  be  destroyed  and   recreated     •  Reapplying  all  public  IPs  to  the  network  provider   •  Reapplying  load-­‐Balancing/Port-­‐Forwarding/Firewall   rules  
  • 44. Deleting a Guest Network•  An Isolated Guest Network can only be deleted if no VMs are using these network (e.g. Completely destroyed and expunged)•  Deleting a Network will Destroy the Virtual Router (if used) and will release the Public IPs back to the IP Pool
  • 45. Basic vs Advanced Networking•  Segmentation based on feature set and ease-of- deployment•  Both are feature-rich•  Basic implements true AWS-style L3-isolation o  Tenants do not get contiguous IP addresses or subnets o  Network segmentation based on Security Groups o  Tremendous scale (tens of thousands)•  Advanced Zone offers full L3 subnets and L2 isolation o  VLANs are default implementation (4K limit) o  More features (source NAT, PF, LB, VPN)
  • 46. Physical Network in Zone Core (L3) Network Pod 1 Pod 2 Pod N Cloudstack   Access  Switch(es) Server   Cloudstack   Servers CLUSTER 1 …   Hypervisor  1VM Traffic …   Hypervisor  8Control Plane TrafficStorage Traffic Storage 2 Storage 1Public Traffic …   CLUSTER 4 Hypervisor  N Hypervisor  N+1 Storage k
  • 47. Layer 3 cloud networking Web DB Web VM VM VM Web DB Security Security Group Group Web Web DB VM VM VM… … … Web Web VM VM
  • 48. Guest Networks with L3 isolationPublic   Public  IP   Guest   Guest  Internet address   1  VM  1 address   65.37.141.11   10.1.0.2 10.1.0.1 Guest   65.37.141.24   Pod  1  L2   Guest   65.37.141.36   Switch 2  VM  1 address   65.37.141.80   10.1.0.3   Guest   Guest   1  VM  2 address   L3  Core   Switch Pod  2  L2   Switch 10.1.8.1 … 10.1.0.4 Guest   Guest   10.1.16. 2  VM  2 address   Load   Pod  3  L2   Balancer 1 10.1.16.12 Switch Guest   2  VM  3 Guest   address   10.1.16.21 … Guest   1  VM  3 Guest   address   10.1.16.47 Guest   Guest   1  VM  4 address   10.1.16.85
  • 49. Guest Networks with L2 isolation Core (L3) Network Pod K Pod M Pod N Access  Switch(es) V V Hypervisor R V Hypervisor CLUSTER 1 …   Hypervisor  1 RVM Traffic …   Hypervisor  8Public Traffic …   CLUSTER 4 V V Hypervisor  N V Tenant VM Hypervisor  N+1 V R Tenant Virtual Router
  • 50. L2 isolation: VLAN networking User 1 User 1 User User 1 2 User 1 User User 2 1 User 2… … … User 1
  • 51. SDN at Work CloudStack Mgmt Server SDN Controlle r Host 1 OVS Host 3 OVS VM VM VM V 1 1 3 RGRE Tunnel GRE Tunnel Host 2 OVS Host 4 OVS VM VM VM V 2 2 3 RGRE Tunnel GRE Tunnel
  • 52. Guest virtual layer-2 network Guest  Virtual  Network   10.1.1.0/24 Public   Public  IP   Guest   Gateway   Guest   Network address   1  VM  1 address   address   65.37.141.11   10.1.1.1 10.1.1.2 65.37.141.36 Guest  1   Guest   Guest  Public   Virtual   1  VM  2 address  Internet Router 10.1.1.3 NAT   Guest   Guest   DHCP   1  VM  3 address   Load   10.1.1.4 Balancing   Guest   Guest   VPN 1  VM  4 address   10.1.1.5 Guest  Virtual  Network   Public  IP   10.1.1.0/24 address   Gateway   Guest   Guest   65.37.141.24   address   2  VM  1 address   65.37.141.80 10.1.1.1 10.1.1.2 Guest  2   Guest   Guest   Virtual   2  VM  2 address   Router 10.1.1.3 NAT   Guest   Guest   DHCP   2  VM  3 address   Load   10.1.1.4 Balancing   VPN
  • 53. Layer-2 Guest Virtual Network CS Virtual Router provides Network Services External Devices provide Network Services Guest  Virtual  Network  10.1.1.1/8   Guest  Virtual  Network  10.1.1.1/8   VLAN  100 VLAN  100Public   Public  Network/ Network/Internet Guest Internet Guest Public  IP   Private  IP   10.1.1.1 10.1.1.1 VM 1 10.1.1.111 VM 1 Gateway   65.37.141.111 JuniperPublic  IP   SRX address  65.37.141.11 CS Firewall 10.1.1.1 Guest Guest Virtual 10.1.1.3 VM 2 10.1.1.3 VM 2 Router Public  IP   Private  IP   DHCP,  DNS   65.37.141. NetScaler 10.1.1.112 NAT   Guest 112 Load Guest Load  Balancing   10.1.1.4 VM 3 Blancer VM 3 10.1.1.4 VPN Guest Guest 10.1.1.5 VM 4 10.1.1.5 VM 4 CS DHCP,   Virtual Router DNS  
  • 54. Layer-3 Guest NetworkNetwork Services Managed Externally Network Services Managed by CS Public  Network   65.11.0.0/16 Security  Group   Security  Group   Public  Network/ 1 1 Internet 10.1.2.3 65.11.1.2 Guest Guest VM 1 VM 1 10.2.12.4 65.11.1.3 65.11.1.2 NetScaler L3 Guest Guest 65.11.1.3 Load switch VM 2 VM 2 Blancer 65.11.1.4 EIP,   ELB   10.5.2.99 65.11.1.4 Guest Guest VM 3 VM 3 10.1.2.18 65.11.1.5 Guest Guest VM 4 VM 4 CS CS Virtual DHCP,   Virtual Security  Group   DHCP,   Route Security  Group   Router DNS   2 DNS   r 2
  • 55. Multi-tier network Internet IPSec or SSL site-to-site VPN CS Customer Virtual Router Loadbalancer Premises Monitoring VLANVirtual Router Services App VM•  IPAM 10.1.2.31 1•  DNS 10.1.1.1 Web VM 1•  LB [intra]•  S-2-S VPN App VM 10.1.2.24•  Static Routes Web VM 2•  ACLs 10.1.1.3 2•  NAT, PF•  FW [ingress & egress] Web VM DB VM•  BGP 10.1.1.4 3 10.1.3.24 1 Web VM 10.1.1.5 4 Virtual  Network     Virtual  Network     Virtual  Network     10.1.1.0/24   10.1.2.0/24   10.1.3.0/24   VLAN  100 VLAN  1001 VLAN  141
  • 56. Session 2Architecture of CloudStack
  • 57. Problem Definition•  Offer a scalable, flexible, manageable IAAS platform that orchestrate physical and virtual resources to offer self-service infrastructure provisioning and monitoring•  Flexible o  Handle new physical resource types § Hypervisors, storage, networking o  Add new APIs o  Add new services o  Add new networking models
  • 58. Problem Definition (contd..)•  Manageable o  Hide complexity of underlying resources o  Rich functional end-user and admin UI o  Admin API to automate operations o  Easy install, upgrade for small -> large clouds o  Simple scaling, automated resilience•  Scalable architecture o  1 -> N hypervisors / VMs / virtual resources o  1 -> N end users
  • 59. Problem Definition (contd..)•  Resource Allocation o  Hypervisor CPU, Memory o  Storage space o  Avoid set of pods, clusters, hosts•  Capacity scanning o  Snapshot of resources consumed o  Trigger capacity threshold violations•  Garbage collection o  Network resources (IP, VLAN, CIDR etc) o  Compute (VM, CPU, memory) o  Storage (volumes)•  Synchronizing the resource states•  Infrastructure resource failures•  Fencing
  • 60. Scaling: Horizontal Scaling Single-node Multi-node Deployment Deployment Manage ment ServerUser API User API Manage Manage ment MySQL Load ment Server DB Balancer ServerAdmin API Admin API Manage MySQL ment DB Server Back Up DB Replication Ø  MS is stateless. MS can be deployed as physical server or VM Infrastructure Infrastructure Ø  Single MS node can manage up Resources Resources to 10K hosts. Multiple nodes can be deployed for scale or redundancy
  • 61. Resource Load Balancing•  As management server is added into the cluster, resources are rebalanced seamlessly. o  MS2 signals to MS1 to hand over a resource o  MS1 wait for the commands on the resources to finish o  MS1 holds further commands in a queue o  MS1 signals to MS2 to take over o  MS2 connects o  MS2 signals to MS1 to complete transfer o  MS1 discards its resource and flows the commands being held to MS2•  Listeners are provided to business logic to listen on connection status and adjusts work based on who’s connected.•  By only working on resources that are connected to the management server the process is on, work is auto-balanced between management servers.•  Also reduces the message routing between the management servers.
  • 62. Cloud Other CLI UI Clients Portal Management Server REST API End User Other Pluggable Service API OAM&P API EC2 API API APIs EngineConsole Proxy ACL & Authentication Security AdaptersManagement -  Accounts, Domains, and Projects -  ACL, limits checking Account Management Connectors Template Services API Access Plugin API Deployment Planning HA Kernel Job Services API -  Drives long running VM Network Configurations Queue Usage operations Calculations -  Syncs between resources managed and DB Network Elements Additional -  Generates events Services Hypervisor Gurus Cluster Resource Job Alert & Event Database Managemen Managemen Management Management Access DB t t Event Bus Message Bus Hypervisor Network Storage Image Snapshot Resources Resources Resources Resources Resources
  • 63. Interactions OVM Cluster Primary Storage vcenter Monitoring Primary CS API vSphere Cluster Storage End User UI Primary XS Cluster Storage Admin UI Clustered CloudStack XAPI Domai CS Admin & CloudStack CloudStack n End-user API Primary Admin Management JSON KVM Cluster Storage UI Server NetConf Juniper SRXCloud user Nitro API{API client (Fog/etc)} VNC JSON ec2 API JSON Netscaler Cloud user Console Console {ec2 API client } Proxy VM Proxy VM NFS MySQL Server Sec. {Proxied} SSH Sec. Storage NFS NFS Storage VM Ajax HTTPS VM Console Router VM HTTP (Template Download) Router VM HTTP (Template Copy) Router VM Cloud user HTTP (Swift)
  • 64. Management Server Layering
  • 65. Balancing Incoming Requests•  Each management server has two worker thread pools for incoming requests: effectively two servers in one. o  Executor threads provided by tomcat o  Job threads waiting on job queue•  All incoming requests that requires mostly DB operations are short in duration and are executed by executor threads because incoming requests are already load balanced by the load balancer•  All incoming requests needing resources, which often have long running durations, are checked against ACL by the executor threads and then queued and picked up by job threads.•  # of job threads are scaled to the # of DB connections available to the management server•  Requests may take a long time depending on the constraint of the resources but they don’t fail.
  • 66. Inside a Management ServerCloudStack API API Servlet Plugins Commands cmd.execute() Plugins Plugins Async Job Queue Services Agent API Kernel (Cmds) Mgr API Responses Mess age Resources Bus Local Or Remote Agent Manager Hypervisor Network Native Device APIs API MySQL
  • 67. CloudStack API Sync/Async commands•  Package and Location cloudstack-oss/api/src/com/cloud/api/…•  BaseCmd (base class)All commands descend from the BaseCmd base class
  • 68. CloudStack APIConfigurationCommands are configured in cloudstack-oss/client/command.properties.inFormat: <command name>=<java classname>;<ACL> *note* ACL is calculated as a bitmap with the following, 1 = ADMIN, 2 = RESOURCE_DOMAIN_ADMIN, 4 = DOMAIN_ADMIN, 8 = USERExample: ### snapshot commands! createSnapshot=com.cloud.api.commands.CreateSnapshotCmd;15! listSnapshots=com.cloud.api.commands.ListSnapshotsCmd;15! deleteSnapshot=com.cloud.api.commands.DeleteSnapshotCmd;15! createSnapshotPolicy=com.cloud.api.commands.CreateSnapshotPoli cyCmd;15! deleteSnapshotPolicies=com.cloud.api.commands.DeleteSnapshotPo liciesCmd;15! listSnapshotPolicies=com.cloud.api.commands.ListSnapshotPolici esCmd;15!
  • 69. CloudStack API: adding APIAdding a new command Determine type of command Synchronous Synchronous List Based Asynchronous Asynchronous Create based Create your command Define request parameters Implement the execute() method Implement an appropriate ResponseObject Add new command to command.properties.in
  • 70. Management Layer•  Management layer is collection of Managers o  Managers are responsible for directing a specific area of the cloud §  Storage Manager •  Manages primary storage server (allocation, life-cycle, attach, detach, user volumes, life-cycle of the primary storage server itself) §  Network Manager •  Manages network configurations, IP Allocations, Port Forwarding, Load Balancers etc. §  User Vm Manager •  Manages life-cycle of VMs created in the cloud §  And many more!!!•  Managers coordinate with each other to achieve a task
  • 71. Management Layer: Adapters•  Modularization and customization within the CloudStack management server is achieved through the use of the Adapter framework.•  Each Adapter is uniquely identified by the interface it exposes and represents the boundary between CloudStack and the individual component and/or processes that can be configured into the system•  Adapters provide extensibility and in many cases device specific implementation details while maintaining a simple and consistent interface.
  • 72. Management Layer: Adapters•  Adapters are executed as a chain in the order that they are configured•  Defined in cloudstack-oss/client/tomcatconf/components.xml.in<adapters key="com.cloud.network.guru.NetworkGuru”> <adapter name="StorageNetworkGuru” class="com.cloud.network.guru.StorageNetworkGuru"/> <adapter name="ExternalGuestNetworkGuru" class="com.cloud.network.guru.ExternalGuestNetworkGuru"/> <adapter name="PublicNetworkGuru" class="com.cloud.network.guru.PublicNetworkGuru"/> <adapter name="PodBasedNetworkGuru" class="com.cloud.network.guru.PodBasedNetworkGuru"/> <adapter name="ControlNetworkGuru" class="com.cloud.network.guru.ControlNetworkGuru"/> <adapter name="DirectNetworkGuru" class="com.cloud.network.guru.DirectNetworkGuru"/> <adapter name="DirectPodBasedNetworkGuru" class="com.cloud.network.guru.DirectPodBasedNetworkGuru"/> <adapter name="OvsGuestNetworkGuru" class="com.cloud.network.guru.OvsGuestNetworkGuru"/></adapters>
  • 73. Adapter Interfaces Available•  Discoverer •  VirtualMachineGuru•  StoragePoolDiscoverer •  HypervisorGuru•  StoragePoolAllocator •  Listener•  ConsoleProxyAllocator •  UserAuthenticator•  Investigator •  SecurityChecker•  FenceBuilder•  DeploymentPlanner•  NetworkGuru•  NetworkElement•  And more…
  • 74. Adapters: VM orchestration•  Deployment Planner o  First Fit planner•  Host Allocator o  First Fit o  Random•  Storage Allocator o  First Fit o  Random
  • 75. Adapters: Network Orchestration•  Network Guru (Responsible for L2-L3) o  Design o  Implement o  Allocate o  Release o  Shutdown e.g. guest network guru, OVS network guru etc•  Network Element (Responsible for L4-L7) o  Implement o  Shutdown e.g. F5, SRX, NetScaler, Virtual Router
  • 76. Extending CloudStack Networking 2. prepare (Network, Nic, DeployDestination, VmInfo) 1. prepare (part of start vm) Network PluggableServi Network Element ce Manager Device Configuration MyDnsDeviceS Admin API (CRUD) DnsService ervice 3. addDnsRecord(ip, fqdn)Demonstrates one way to MyDnsDeviceM MySQL MyDnsElementinform an external DNS anagerserver when an instancestarts. AgentMana 4.Enqueue AddDnsRecord ger QueueClasses shaded blue forma plugin / service bundleto integrate an external MyDnsDeviceRDNS server. Clients of the esourceinstance can then useDNS names to access the 5.API call to Dns Deviceinstance.
  • 77. Sequence Flow for VM Creation Deployme Server Job Services User VM VirtualMac Network Storage Network Network Templat nt ResourceThreads API Mgr hine Mgr Mgr Mgr Guru Element e Mgr Planner s Start VM Start User VM Start VM Get a Deployment Plan (Host and StoragePool) Prepare Nics Reserve resources for Nic Notify that Nic is about to be started in network Agent Calls Prepare Volumes Prepare template on Primary Storage Agent Calls Agent Start VM Call Stores job result
  • 78. Management Layer: Adapters flow
  • 79. Server Resources Agent •  Resources are carried in service VMs to be in close Hypervisor Resources network proximity to the physical resources it managesResource API Network Resources •  Easily scales to utilize the most abundant resource in data Storage Resources center (CPU & RAM) Image & Template •  Communicates with Resources Orchestration Server over message bus (JSON) Snapshot Resources •  Can be replicated for fault tolerance •  Control gateway to resources within data center
  • 80. Resource Layer
  • 81. Working toward 4.1 release•  4.1 is next major release o  Moving away from monolithic architecture to loosely coupled subsystems o  Spring for IOC container and AOP o  Storage subsystem refactoring o  Network subsystem refactoring o  New orchestration engine o  Regions support
  • 82. Session 3Developing with DevCloud
  • 83. DevCloud• CloudStack requires o  Hypervisor o  Network o  Storage
  • 84. DevCloud• self-contained CloudStack runs in the appliance
  • 85. DevCloud• Several use cases o  Try CloudStack in an isolated sandbox. Runs within the appliance o  Develop CloudStack on own machine, build locally and deploy new version in DevCloud (Build and test) o  Develop and Run locally, use DevCloud as Xen hosts
  • 86. Thanks