• Save
StackMate - CloudFormation for CloudStack
Upcoming SlideShare
Loading in...5

StackMate - CloudFormation for CloudStack



Stackmate and Stacktician are orchestrators that allow CloudFormation templates to be deployed on CloudStack.

Stackmate and Stacktician are orchestrators that allow CloudFormation templates to be deployed on CloudStack.



Total Views
Slideshare-icon Views on SlideShare
Embed Views



3 Embeds 45

http://play.daumcorp.com 39
https://twitter.com 4
http://moderation.local 2



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    StackMate - CloudFormation for CloudStack StackMate - CloudFormation for CloudStack Presentation Transcript

    • StackMateYour friend in the Cloud Business@chiradeep
    • Overview• Introduction to CloudFormation• What it takes• Progress• TODO• FAQ• Speculation
    • CloudFormation• AWS service introduced in 2011• Application Management using blueprints• Integrates with provisioning tools within theinstance• “Infrastructure as code”– Readable (json)– Domain specific (cloud resources jargon)– Reviewable (text document)– Reusable
    • Example: XenApp Farm in the Cloud
    • Example: XenApp farm in VPC61 resources23 resource types53 orderingdependencies
    • ComplexOrdering ofResourcecreation
    • CloudFormation TemplateDeclarative specification of an application ‘stack’– Cloud resources and services• Compute, Network, Storage, Object Storage, etc– Parameters and properties to customize theresources– Outputs (metadata) generated by the creation ofthe stack– Implicit or explicit ordering of resource creationText file in JSON format.
    • Typed and Validated ParametersDBName": {"Default": "MyDatabase","Description" : "MySQL database name","Type": "String","MinLength": "1","MaxLength": "64","AllowedPattern" : "[a-zA-Z][a-zA-Z0-9]*","ConstraintDescription" : "must begin with a letter and contain only alphanumericcharacters."},InstanceType" : {"Description" : "WebServer EC2 instance type","Type" : "String","Default" : "m1.small","AllowedValues" : ["t1.micro","m1.small","m1.medium","m1.large","m1.xlarge","m2.xlarge","m2.2xlarge","m2.4xlarge","m3.xlarge","m3.2xlarge","c1.medium","c1.xlarge","cc1.4xlarge","cc2.8xlarge","cg1.4xlarge"],"ConstraintDescription" : "must be a valid EC2 instance type."},Default ValueTypeConstraints
    • ResourcesWebServer: {"Type": "AWS::EC2::Instance","Metadata" : {#bootstrap script fetched by cfn-init},"Properties": {"ImageId" : { “Ref” : “ImageId”},"InstanceType" : { "Ref" : "InstanceType" },"SecurityGroups" : [ {"Ref" : "WebServerSecurityGroup"} ],"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", ["#!/bin/bash -vn","yum update -y aws-cfn-bootstrapn”,"# Install LAMP packagesn",”/opt/aws/bin/cfn-init -s ", { "Ref" : "AWS::StackId" }, " -r WebServer ”,"# Setup MySQL, create a user and a databasen","mysqladmin -u root password ", { "Ref" : "DBRootPassword" }, " || error_exit Failedto initialize root passwordn","# Configure the PHP application - in this case, fixup the page with the rightreferences to the databasen","sed -i "s/REPLACE_WITH_DATABASE/localhost/g" /var/www/html/index.phpn","# All is well so signal successn","/opt/aws/bin/cfn-signal -e 0 -r "LAMP setup complete" ", { "Ref" : "WaitHandle"}, "n"]]}}}Reference to ParametersReference to Other Resourcesec2-initbootstrapFetchfrommetadataserver
    • Outputs"Outputs" : {"WebsiteURL" : {"Value" : { "Fn::Join" : ["", ["http://", { "Fn::GetAtt" :[ "WebServer", "PublicDnsName" ]}]] },"Description" : "URL for newly created LAMP stack"}Reference to Resources created by the stack
    • Wait Conditions• Special resource for temporal ordering ofapplication provisioning– E.g., ensure DB service is up in DB instance beforeinstantiating App Server instance• Wait Condition Handle– URL for application to signal that wait conditionhas been satisfied– Injected into instance via user data
    • Stack Operations• Create– cfn-create-stack --stack-name -f, --template-file | -u, --template-url -d, --disable-rollback -p, --parameters -t, -timeout– Atomic: all resources are created or none are created– Unless rollback is disabled• Delete– Destroy in proper order• Update– Delta from old template: delete removedresources, create new ones.
    • StackMateCloudFormation Template Parser / Validator+Resource creation / deletion orchestrator+Wait condition serverForCloudStack
    • StackMate ArchitectureApplicationTemplate Stacker• Parse• Validate• Resolvedependencies• Execution planStackExecutor• Workflow engine• Execution of plan• CS API clientCloudStackMgmtServerOutputTemplateParameters,CS API key,MappingsRuote is a Workflow engine written in Ruby
    • Stacker• Parsing– Simple: JSON.parse(File.read(template))• Resolve dependencies– Recurse through json data structure looking for ‘Ref:’hashes.– Fill in parameters if possible– Build dependency graph for each resource• Execution plan– Topological sort of resource dependency graph– Rollback plan: reverse sort
    • Directed Acyclic Graph of dependencies Ruote Process DefinitionSequence of CloudStack API calls
    • Q: Why Ruote (or why use workflow)• A: it is a (long-running, distributed) workflow– Persistence– Recovery– Rollback– Possible parallelism• Topological sort does not need to produce linear plan• Ruote is mature– Used by Rightscale, EngineYard
    • • Stacktician is a web application thatembeds StackMate• Graphical UI• Wait condition server• Database persistence enables• Query of stack execution status• Single sign-on with CloudStackcredentials• Stack execution history
    • Stacktician ArchitectureStackMateStacker StackExecutorPersistenceExtensionsThe Rails logo is a registered trademarks of David Heinemeier HanssonStackModelCloudStackMgmtServerRest API via browserDBStackCtrller
    • Stacktician Architecture• Ruby on Rails application• StackMate gem is used for parsing, validation, etc• Bootstrap-based UI• Ruote worker threads– One worker thread for persistence– One thread per resource per stack• Thread is mostly idle (sleep, waiting for CS API call to finish)• Can run Ruote workers outside of Rails for scale• Cloudstack Ruby client• Session is persisted in db
    • Native CloudStack Resources• Resources that do not have an AWS relative– E.g., firewall rule, port forwarding rule• Resources that are modeled differently– LB, Autoscale, IAM• E.g.,:– CS::Compute::FirewallRule– CS::Compute::PortForwardingRule– CS:Compute::IsolatedNetwork• Needs work
    • StackMate next steps• Support delete stack– Add tags to resources created by StackMate– Delete resources in reverse order• Support more AWS resources– Only Instance, SecurityGroup and WaitConditiontoday– Need VPC support• Support CS resource types in template
    • Stacktician next steps• Support delete stack, atomic operation• Support more resources (AWS and CS-specific)• Integrate with CloudStack authentication• Email /AMQP notification of stack events• Support CF Query API in addition to Rails-based REST API
    • Stacktician next steps• Allow user to specify URL for template content• Scaling using Resque to send API jobs tobackend workers.• Full featured admin interface
    • Stacktician future• Metadata server– Instance not limited to 32k of userdata at boot• Update template semantics– Tricky with corner cases• First create new resources in new template• Delete resources not found in new template
    • FAQ• Does it work with Chef/Puppet– Yes, use userdata or metadata (future) to configure chef solo orchef client or puppet client• Why not use Chef/Puppet– Workflow– Atomic operation– Support for more CS resource types• Did you look at <xyz> project that does something similar?– Nope• What about TOSCA ?– Similar, but at a meta-level. That is, TOSCA does not specify theresources, but a way to specify resources and dependencies.Still early
    • FAQ• License– MIT• Where– https://github.com/chiradeep/stackmate– https://github.com/chiradeep/stacktician• Contribution– Pull requests accepted happily
    • Stacktician FuturesStackMateStacker StackExecutorPersistenceExtensionsThe Rails logo is a registered trademarks of David Heinemeier HanssonStackModelCloudStackMgmtServerRest API via browserStackCtrllerGoogleComputeEngineS3(e.g., RiakCS, Ceph,Cloudian)XYZ PAASonCloudStackxAAS onCloudStackxAAS onxyzCloud
    • Stacktician Futures• Complex multi-service orchestration– E.g., deploy my app on CS in zone 1 and thenregister the public IP in a availability monitoringservice– Deploy my app in CS, then call a load testingservice in the Google cloud– Deploy my app in CS, using storage fromS3, Google Storage and Azure Storage– etcThe Rails logo is a registered trademarks of David Heinemeier Hansson