Apache CloudStack Architecture by Alex Huang


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The core components of a CloudStack implementation are:Hosts – Hosts are servers from at least one of the supported virtualization providers. CloudStack fully supports hosts from multiple providers, but does not convert VM images from one hypervisor type to another. Depending on the hypervisor, a “host” may be a higher level concept. For example, in XenServer a CloudStack “host” is equivalent to a XenServer resource pool and the “host” entry is the pool master.Primary Storage – Primary storage is the hypervisor level storage containing the deployed VM storage. Primary storage options will vary by hypervisor, and depending upon the hypervisor selected, CloudStack may impose requirements upon it.Cluster – Host groups are combined into Clusters which contain the primary storage options for the Cluster. Primary storage isn’t shared outside of a Cluster. In the case of CloudStack, a Cluster in of itself does not imply modification of any clustering concept within the hypervisor. For example, in XenServer a resource pool is a host to CloudStack, and CloudStack does not create a super set of Cluster functionality for XenServer. Pod -- Host groups are combined first into Clusters and then into Pods. For many customers, a pod represents a high level physical concept like a server rackNetwork – Network is the logical and physical network associated with service offerings. Multiple concurrent network service offerings and topologies can be supported within CloudStackSecondary Storage – Secondary storage is the storage system used for template and ISO management. It also is where snapshot events occur.Zone – A zone is a collection pods to form some level of service availability. While Amazon EC2 defines an availability zone as a data center, CloudStack keeps the concept more abstract allowing cloud operators to have multiple availability zones within a given data center.Management Server Farm – The CloudStack management server farm is a grouping of CentOS/RHEL CloudStack servers forming a web farm, with an underlying MySQL cluster database. The management server farm can manage multiple Zones, and can be virtualized.
  • Slide 2 (control), Slide 3 network topology (AWS), Slide 4 network topology (multi-tier), Slide 5 Network topology (management network), Slide 5 network topology (MPLS)However, this problem gets a little more complicated when we consider what a cloud operator wants: Control. On the hardware resource side, the cloud operator wants to control which hardware resource is used to provide the cloud infrastructure. They want to say the hypervisor is XenServer, KVM, or VmWare and the load balancer is a Netscaler or F5 and so on. On the end user side, they want to control the who can do what to which virtual entities, place governace rules, and limit the amount of resources a user can consume. This is still fairly straight-forward. We place two sets of APIs: Services API and Resource API. Above the Service API, CloudStack performs Auth and ACL checks, single sign-ons, integration with accounts databases such as ADC and LDAP, and provides checks against limits and governnes rules. Under the Service API is the CloudStack Orchestration Engine. When Orchestration Engine needs to contact a certain hardware, it speaks the Resource API with the abstraction layer implemented by ServerResource. And to ensure that the management server observes this abstraction layer, it can only communicate with ServerResource over a JSON based message bus.
  • Apache CloudStack Architecture by Alex Huang

    1. 1. Apache CloudStackArchitecture Alex Huang Software Architect, Citrix Systems
    2. 2. Deployment Architecture
    3. 3. Components• Hosts • Servers onto which services will be provisioned VM• Primary Storage Host • VM disk storage Network VM• Cluster Host • A grouping of hosts and their associated storage Primary• Pod Storage • Collection of clusters in the same failure boundary• Network Cluster • Logical network associated with service offerings Secondary Cluster• Secondary Storage Storage • Template, snapshot and ISO storage CloudStack Pod• Zone • Collection of pods, network offerings and secondary storage CloudStack Pod• Management Server Farm • Management and provisioning tasks Zone
    4. 4. Two Types of StoragePrimary Storage• Stores disk volumes for VMs in a cluster• Configured at Cluster-level.• Close to hosts for better performance L3 switch• Cluster have at least one primary storage• Requires high IOPs (can be expensive) Pod 1 L2 switch Secondary Cluster 1 Storage Host 1 PrimarySecondary Storage Host 2 Storage• Stores all Templates, ISOs and Snapshots• Configured at Zone-level• Zone can have one or more secondary storages• High capacity, low cost commodity storage
    5. 5. Deployment Architecture Internet  Hypervisor is the basic unitManagementServer Cluster of scale. Zone 1  Cluster consists of one ore more hosts of same L3 hypervisor Pod 1 Pod N  All hosts in cluster have L2 Secondary access to shared (primary) …. Storage storage Cluster N  Pod is one or more clusters, usually with L2 switches. ….  Availability Zone has one or Cluster 1 more pods, has access to secondary storage. Host 1 Primary  One or more zones Host 2 Storage represent cloud
    6. 6. Management Server Cluster  MS is stateless. MS can be deployed as physical server or VM MySQLUser API Management  Single MS node can Server manage up to 10K hosts. Load Balancer Replication Multiple nodes can beAdmin API Management deployed for scale or Server redundancy Replica  RHEL 5.4+, Ubuntu 10.0.4, Fedora 16 Infrastructure Resources
    7. 7. Managing Complexity
    8. 8. The Three C’s of Complexity• Control• Choice• Compliance
    9. 9. Giving Control Brings Complexity Org A Org B • ACL Users • Limits Admin Admin • Governance End User Users UsersAdmin Compute Network Storage VM Net Local Hyper-V Cisco ASA NFS Ware Scaler Disk Xen Oracle F5 iSCSI Swift Server VM Bare Jun. SRX FC HDFS KVM Metal
    10. 10. Guest Virtual Layer-2 Network Guest 1 Virtual Network Guest 1 Guest VM 1 Public IP Guest 1 Gateway Guest 1 Guest Virtual VM 2 Router Guest 1 Guest VM 3 Guest 2 Guest VM 1 Public IP Guest 2 Gateway Guest 2 Guest Virtual VM 2 Router Guest 2 Guest VM 3 Guest 2 Virtual Network
    11. 11. Multi-tier Network Virtual Network Virtual Network Virtual Network VLAN 1001 VLAN 141 VLAN 100 App VM 1 Private IP Web VMPublic IP 1 Juniper SRX App VM111 Firewall 2 Web VM 2 Public IP Private IP Netscaler 65.37.1 Load Web VM 41.112 Balancer 3 DB VM 1 Web VM 4 DHCP, Virtual DHCP, Virtual DNS Virtual DHCP, DNS, DNS Router User- Router User- Router data data, User- Source data -NAT, VPN Public IP
    12. 12. Unified Multi-tier Network Internet IPSec or SSL site-to-site VPN Virtual Router Customer Load Balancer Premises Monitoring VLANVirtual Router Services App• IPAM Web VM 1• DNS• LB [intra] VM 1• S-2-S VPN App• Static Routes Web VM 2• ACLs VM 2• NAT, PF• FW [ingress & egress] Web DB VM• BGP VM 3 1 Web VM 4 Virtual Network Virtual Network Virtual Network VLAN 100 VLAN 1001 VLAN 141
    13. 13. Other Topologies Dedicated VLAN with DHCP and DNSNo services [Static IPs] User can request specific IP[s] for NIC Guest Virtual Network Guest Virtual Network VLAN 100 VLAN 100 Guest Guest VM 1 VM 1 Gateway address Guest Guest VM 2 Gateway VM 2 address Guest Guest Core switch VM 3 VM 3 Guest Core switch Guest VM 4 VM 4 Virtual DHCP, DNS Router User-data
    14. 14. Other TopologiesMPLS Use Case Shared VLAN with DHCP and DNS Guest Virtual Network Guest Virtual Network VLAN 100 VLAN 100MPLS VLAN 100 Guest Guest VM 1 VM 1 Gateway address Guest Guest VM 2 Gateway VM 2 address Guest Guest Core switch VM 3 VM 3 Guest Core switch Guest VM 4 VM 4 5 CS CS DHCP, Virtual DHCP, Virtual Router Router DNS DNS User-data User-data
    15. 15. Layer 3 Networking (Amazon Style) Web DB Web VM VM VM Web DB Security Security Group Group Web Web DB VM VM VM … … … Web Web VM VM
    16. 16. Software Architecture
    17. 17. Cloud Other UI CLI Clients Portal Management Server REST API OAM&P API End User API EC2 API Other APIs Pluggable Service API EngineConsole Proxy ACL & Authentication Security AdaptersManagement - Accounts, Domains, and Projects - ACL, limits checking Account Management Connectors Template Access Services API DB Plugin API Deployment Planning HA Orchestration Engine - Drives long running VM Services API Network Gurus Usage operations Calculations - Syncs between resources managed and DB Network Elements Additional - Generates events Services Hypervisor Gurus Cluster Resource Job Alert & Event Database Management Management Management Management Access Message Bus Event Bus Usage Server Resource API Hypervisor Network Storage Image Snapshot Resources Resources Resources Resources Resources
    18. 18. Orchestration Engine• Understands how to orchestrate long running processes (i.e. VM starts, Snapshot copies, Template propagation)• Well defined process steps• Calls Plugin API to execute functionalities that it needs
    19. 19. Plugins• Various ways to add more capability to CloudStack• Implements clearly defined interfaces• All operations must be idempotent• All calls are at transaction boundaries• Compiles only against the Plugin API module
    20. 20. Anatomy of a Plugin • Can be two jars: server component to be deployed on management server and an optional ServerResource Rest API component to be deployed co- - Optional. Required only if needs to expose located with the resource configuration API to admin. • Server component can implement multiple Plugin APIs to add its featurePlugin API Implementation • Can expose its own API through Pluggable Service so administrators Data Access Layer can configure the plugin • As an example, OVS plugin actually implements both NetworkGuru and ServerResource NetworkElement - Optional. Required if Plugin needs to be co- located with the resource - Implements translation layer to talk to resource - Communicates with server component via JSON
    21. 21. Plugin Interfaces Available• NetworkGuru – Implements various network isolation and ip address technologies• NetworkElement – Facilitate network services on network elements to support a VM (i.e. DNS, DHCP, LB, VPN, Port Forwarding, etc)• DeploymentPlanner – Different algorithms to place a VM and volumes.• Investigator – Ways to find out if a host is down or VM is down.• Fencer – Ways to fence off a VM if the state is unknown• UserAuthenticator – Methods of authenticating a user• SecurityChecker – ACL access• HostAllocator – Provides different ways to allocate host• StoragePoolAllocator – Provides different ways to allocate volumes
    22. 22. Separating Data and Control Cloud  Management Servers control allData Center 1 Data Center 3 resources, both virtual Managem and physical ent VR VR Server  SSVMs deployed to CPVM SSVM SSVM CPVM transfer data between Transfer of zones Templates, ISOs, Snapshots  CPVMs deployed to Internet transfer VNC console Data Center 2 traffic VR SSVM  VR deployed for traffic into public internet CPVM  Management Server is never in the data path
    23. 23. Sequence Flow for VM Creation Kernel End User Security User VM VirtualMac Network Storage Network Job Rest API Checkers Mgr hine Mgr Mgr Mgr Guru Scheduling Deploy VM ACL Checks Allocate Entity in CS Allocate VM Allocate NIC Allocate IP Allocate Volume Schedules Deploy Job Returns with job id, VM id Query Job ResultReturns with job status
    24. 24. Sequence Flow for VM Creation Deploymen Server User VM VirtualMac Network Storage Network Network Template tJob Threads Services API Resources Mgr hine Mgr Mgr Mgr Guru Element Mgr Planner Start VM Start User VM Start VM Get a Deployment Plan (Host and StoragePool) Prepare Nics Reserve resources for Nic Notify that Nic is about to be started in network Agent Calls Prepare Volumes Prepare template on Primary Storage Agent Calls Agent Start VM Call Stores job result
    25. 25. Conclusion
    26. 26. Design Goals for CloudStack• Design for complexity – Clear interfaces• Design for scalability – Separate out data path and control paths – Design to maximize the use of database connections• Design against failure – Provide clear boundaries (process and compilation) – Utilize cloud administrator to give guidance
    27. 27. More Information• http://cloudstack.org• Apache mailing lists – cloudstack-users-subscribe@apache.incubator.org – cloudstack-dev-subscribe@apache.incubator.org• Thank you 27