A little bit about me• Cloud.Com Founding Engineer• Software architect for CloudPlatform• Responsible for overall architecture, performance, and scalability• Committer and PPMC member• BS from UC Berkeley and MS from Stanford
Outline• Difference between automation, orchestration, and provisioning• Networking Example• Anatomy of a plugin• Myth busting
What is the difference?• The key is the data center abstraction layer – Virtual Machine, Template, Nic, IP Address, Volume, Network, Rules, Snapshot• Orchestration orchestrates within this abstraction layer• Provisioning manifests concepts in abstraction layer on physical resources• Automation orchestrates above the data center abstraction layer to bring greater functionality
Examples• Orchestration – VM deploy, Volume creation, Network creation, Network rules propagation• Provisioning – Starting a VM on a hypervisor – The actual movement of a volume from one storage pool to another• Automation – HA – Putting a physical resource into maintenance mode
Enabling Innovation• CloudStack Orchestration must not define the innovation. – Partners define their own APIs. – Partners and CloudStack can work together on unified APIs through design process on Apache.• Differentiate between orchestration and provisioning. – CloudStack only orchestrates. – Provisioning is always pushed to the partner.• Clearly defined data center abstraction layer. – Changes in this layer are broadcasted to partners.• Utilize CloudStack’s orchestration to deploy and auto- scale partners’ technologies.
CloudStack Terminology (End User)• Network – A single concept to encapsulate multiple network technologies to simplify representation to the end user. – Each Network always carries its Network Traffic Type. – CloudStack DOESN’T understand how to provision this conceptual network on to the physical network.• NetworkService – L2-L7 network services that partners have written to operate within a Network. – Currently defined: Load Balancing, Port Forwarding, Firewall, Gateway, DNS, DHCP, Static NAT, VPN, Source NAT, User Data.• NetworkOffering – A packaging of the NetworkServices provided to the end user on a particular Network. – NetworkOfferings are put together by cloud service provider.
CloudStack Terminology (Provider)• Network Traffic Type – Traffic types are mapped to the underlying physical network by the cloud service provider. – Traffic type is not the same as network (Guest traffic type can actually be carried on multiple networks) – Currently defined: Public, Guest, Storage (Backup really), Management• NetworkServiceProvider – Plugin that understands how to provide one or more NetworkServices by using VPX or physical resource.• PhysicalNetwork – Actual wiring of the data center.
CloudStack Terminology (Partner)• NetworkGuru – Plugin that understands the network isolation technology, mac addressing scheme, and IP addressing scheme deployed and how to map Network Traffic Types to the underlying physical network. – CloudStack passes Network to NetworkGuru to “implement” before the network is needed by a virtual machine. – CloudStack asks the NetworkGuru to issue ip, mac, and isolation to a virtual machine before it starts. – CloudStack informs the NetworkGuru when a virtual machine stops so it can collect resources. – When all virtual machines in a Network are stopped, CloudStack garbage-collects the Network by asking the NetworkGuru to shutdown the network. – CloudStack provides a default implementation for VLAN based isolation technology.• NetworkElement – Interface that specifies the events CloudStack signals to the NetworkServiceProviders when a Network needs to be “implemented” and shutdown and when a virtual machine joins and leaves a Network.
“Architect” Model• The builder offers multiple blueprints for the owner to build the house.• Owner chooses on a blueprint and then adds on with additional enhancements such as hardwood floors, granite counter tops, etc.• General contractor builds to the blueprint by orchestrating between different sub- contractors to build different parts of the blueprint.• There are two general category of contractors. – Rough-in sub-contractors who take care of plumbing, electricity, framing, foundation. – Finish sub-contractors who put in flooring, kitchen cabinets etc.• Each sub-contractor is responsible for only their work but looks over the entire blueprint to make sure their work can actually be done. – E.g. A lighting plan may conflict or needs to change depending on the framing plan.• General contractor is responsible for sequencing the sub-contractors to make sure everything the sub-contractor is dependent on is ready when the sub-contractor arrives to do his work.• Every change requires a the blueprint to be republished so every sub-contractor can make their appropriate changes.
ComparisonBuilding a house Building a network• Owner • End user• Builder • Cloud Service Provider• General Contractor • CloudStack• Rough-in Sub-Contractors • NetworkGurus• Finish Sub-Contractors • NetworkServiceProviders• Blueprint • Abstraction Layer• Cabinets, Flooring, Counter • NetworkServices Tops, etc
Architectural Principles• CloudStack clearly defines the difference between orchestration and provisioning. – Orchestration the ordering of what needs to happen in CloudStack’s abstraction layer. – Provisioning is the actual work performed at the resource.• CloudStack clearly defines the difference between network definition and network services. – Network definition is handled by NetworkGuru. – Network services is handled by NetworkServiceProvider.• CloudStack broadcasts changes in the network every time NetworkServices and virtual machines changes in the Network.• CloudStack allows the Cloud Service Provider to setup the appropriate mappings between virtual concepts such as Network and Network Traffic Type to the underlying physical network.
Sequence Flow for VM Creation Engine End User Security User VM VirtualMac Network Storage Network Job Rest API Checkers Mgr hine Mgr Mgr Mgr Guru Scheduling Deploy VM ACL Checks Allocate Entity in CS Allocate VM Allocate NIC Allocate IP Allocate Volume Schedules Deploy Job Returns with job id, VM id Query Job ResultReturns with job status
Sequence Flow for VM Deployment Deploymen Server User VM VirtualMac Network Storage Network Network Template tJob Threads Services API Resources Mgr hine Mgr Mgr Mgr Guru Element Mgr Planner Start VM Start User VM Start VM Get a Deployment Plan (Host and StoragePool) Prepare Nics Reserve resources for Nic Notify that Nic is about to be started in network Agent Calls Prepare Volumes Prepare template on Primary Storage Agent Calls Agent Start VM Call Stores job result
CloudStack WebServices API End User OAM&P API AWS API Pluggable Service API Engine API Virtual Router Netwrok Service ProviderAccounts Business Logic User Dispersing Deployment Planner CloudStack Plugins NetScaler Netwrok Service ProviderSecurity First Fit Deployment Planner Resource Manager Manager Manager Manager Manager Capacity UpdateManager Rules HA Events Manager UsageManager CloudStack Orchestration Adapters Network Guru Domain Network Manager Storage Manager Virtual Machine Manager Network Snapshot Template Manager Manager Manager ElementAccount DeploymentManager Planner Hypervisor Limits Guru Manager Framework Agent Manager Cluster Manager Data Access Layer
Anatomy of a Plugin • Server Component: – Deployed on management server Rest API – Can implement multiple Plugin APIs to affect its feature Implementation – Can expose its own API throughPlugin API Pluggable Service so administrators can configure the Data Access Layer plugin – Can access the database • ServerResource: ServerResource – Deployed co-located with the- Optional. Required if Plugin needs to be co- located with the resource physical resource- Implements translation layer to talk to resource- Communicates with server component via JSON – Cannot access the database
Anatomy of a Plugin• Can be two jars: server component to be deployed on management server and an optional ServerResource component to be deployed co- located with the resource• Server component can implement multiple Plugin APIs to affect its feature• Can expose its own API through Pluggable Service so administrators can configure the plugin• As an example, OVS plugin actually implements both NetworkGuru and NetworkElement
Plugin Interfaces Available• NetworkGuru – Implements various physical network technologies and ip address• NetworkElement – Facilitate network services on network elements to support a VM (i.e. DNS, DHCP, LB, VPN, Port Forwarding, etc)• DeploymentPlanner – Different algorithms to place a VM and volumes.• Investigator – Ways to find out if a host is down or VM is down.• Fencer – Ways to fence off a VM if the state is unknown• UserAuthenticator – Methods of authenticating a user• SecurityChecker – ACL access• HostAllocator – Provides different ways to allocate host• StoragePoolAllocator – Provides different ways to allocate volumes
Adding a Plugin to CloudStack• Components are configured though components.xml• Supports DAO, Manager, and Adapter patterns
Extending CloudStack Networking 2. prepare (Network, Nic, DeployDestination, VmInfo) 1. prepare (part of start vm) Network Network Element PluggableService Manager Device Configuration MyDnsDeviceSer Admin API (CRUD) vice DnsService 3. addDnsRecord(ip, fqdn)Demonstrates one way to MyDnsDeviceMa MySQL MyDnsElementinform an external DNS nagerserver when an instancestarts. AgentManag 4.Enqueue AddDnsRecord er QueueClasses shaded blue form aplugin / service bundle tointegrate an external DNS MyDnsDeviceResserver. Clients of the ServerResource ourceinstance can then use DNSnames to access the 5.API call to Dns Deviceinstance.
Myth 1• Network is Layer 2• Network is Layer 3• Network confines how I implement my physical network.
Busted• No. Network is just a concept to represent where virtual machines can access each other. – One Network to rule them all, One Network to define them, One Network to bring them all and in the cloud bind them.• What defines a network to be layer 2 or layer 3 is the NetworkGuru.
Myth 2• I must convert my service to run inside CloudStack’s plugin.• CloudStack controls the deployment of my service.
Busted• CloudStack’s plugin interfaces specify the provisioning events and data associated with the events.• Plugin implementations can be client stubs that call out to your actual service upon receiving these events.• If you so desired, plugin implementations can also expose APIs to configure the plugins.
Myth 3• My service cannot scale independently from CloudStack.• My service’s availability is tied together with CloudStack’s availability.
Busted• CloudStack does not need to know how a service scales. – CloudStack does not manage instances of a service. – CloudStack does not manage instances of the same plugin. Plugin code is always a singleton.
The future needs you!Project web site: http://incubator.apache.org/projects/cloudstack.htmlMailing lists:email@example.com@incubator.apache.orgIRC: #CloudStack on irc.freenode.net
Thank You! Alex Huang Email: firstname.lastname@example.orgBlog: http://xueyuan.github.com/