2013 05-openstack-israel-heat

1,606 views

Published on

slides on Heat and influences presented at OpenStack Israel May 2013

Published in: Technology
  • Be the first to comment

  • Be the first to like this

2013 05-openstack-israel-heat

  1. 1. What’s Up Next for Heat?Alex Heneveld / @ahtweetinOpenStack Israel27 May 2013
  2. 2. CloudIt’s not just about VM’s
  3. 3. It’s about storage, networking, maas, ...It’s not just about VM’sCloud
  4. 4. Ultimately it’s about applicationsCloudIt’s not just about VM’s
  5. 5. It makes the clouds rise
  6. 6. Declarative — OpenStack Heat (Grizzly) and AWS CloudFormationProcedural — Apache Whirr, Gigaspaces Cloudify, Cloudsoft BrooklynOpenStack Summit — Red Hat, RAX, IBM, HP, Cloudsoft,OASIS TOSCA and CAMPDSLs in Progress — CAMP PDP and OpenStack Heat (Havana)Approaches to Applications in the Cloud
  7. 7. AWS CloudFormation{"AWSTemplateFormatVersion" : "2010-09-09","Description" : "WordPress is web software …","Parameters" : {"KeyName" : {"Description" : "Name of an existing EC2 KeyPair for SSH access","Type" : "String" },... },"Mappings" : {"AWSInstanceType2Arch" : {"m1.small" : { "Arch" : "64" },"m1.large" : { "Arch" : "64" },... },"AWSRegionArch2AMI" : {"us-east-1" : { "32" : "ami-31814f58", "64" : "ami-1b814f72" }, ... } },"Resources" : {"WebServer": {"Type": "AWS::EC2::Instance","Metadata" : {"AWS::CloudFormation::Init" : {"config" : {"packages" : {"yum" : {"gcc-c++" : [],"make" : [],...
  8. 8. {"AWSTemplateFormatVersion" : "2010-09-09","Description" : "WordPress is web software …","Parameters" : {"KeyName" : {"Description" : "Name of an existing EC2 KeyPair for SSH access","Type" : "String" },... },"Mappings" : {"AWSInstanceType2Arch" : {"m1.small" : { "Arch" : "64" },"m1.large" : { "Arch" : "64" },... },"AWSRegionArch2AMI" : {"us-east-1" : { "32" : "ami-31814f58", "64" : "ami-1b814f72" }, ... } },"Resources" : {"WebServer": {"Type": "AWS::EC2::Instance","Metadata" : {"AWS::CloudFormation::Init" : {"config" : {"packages" : {"yum" : {"gcc-c++" : [],"make" : [],..."Resources" : {"WebServer": {"Type": "AWS::EC2::Instance"AWS CloudFormation
  9. 9. "Resources" : {"WebServer": {"Type": "AWS::EC2::Instance"ParametersMappings(config metadata)WaitConditions &WaitHandles(orchestration)OutputsFunctionsAWS CloudFormation
  10. 10. "Resources" : {"WebServer": {"Type": "AWS::EC2::Instance"AWS CloudFormation
  11. 11. "Resources" : {"WebServer": {"Type": "AWS::EC2::Instance"AWS CloudFormation
  12. 12. "Type": "AWS::AutoScaling::AutoScalingGroup""Type": "AWS::EC2::EIP""Resources" : {"WebServer": {"Type": "AWS::EC2::Instance""Type": "AWS::SQS::Queue""Type": "AWS::CloudFormation::Stack""Type": "AWS::S3::Bucket"AWS CloudFormation
  13. 13. "Type": "AWS::AutoScaling::AutoScalingGroup""Type": "AWS::EC2::EIP""Resources" : {"WebServer": {"Type": "AWS::EC2::Instance""Type": "AWS::SQS::Queue""Type": "AWS::CloudFormation::Stack""Type": "AWS::S3::Bucket"AWS CloudFormationComposition!
  14. 14. — Limited to AWS cloud— Limited to AWS concepts— Cumbersome to write— Limited dependency injectionNice, but:AWS CloudFormation
  15. 15. — CloudFormation clone— Targetting OpenStack resources and AWS compatibility— Adds YAML support (comments and easier)— Multi-cloud support using Apache DeltacloudOpenStack Heat
  16. 16. (AWS CloudFormation){"AWSTemplateFormatVersion" : "2010-09-09","Description" : "WordPress is web software …","Parameters" : {"KeyName" : {"Description" : "Name of an existing EC2 KeyPair for SSH access","Type" : "String" },... },"Mappings" : {"AWSInstanceType2Arch" : {"m1.small" : { "Arch" : "64" },"m1.large" : { "Arch" : "64" },... },"AWSRegionArch2AMI" : {"us-east-1" : { "32" : "ami-31814f58", "64" : "ami-1b814f72" }, ... } },"Resources" : {"WebServer": {"Type": "AWS::EC2::Instance","Metadata" : {"AWS::CloudFormation::Init" : {"config" : {"packages" : {"yum" : {"gcc-c++" : [],"make" : [],...OpenStack Heat
  17. 17. OpenStack HeatHeatTemplateFormatVersion: 2012-12-12Description: WordPress is web software ...Parameters:  KeyName:Description: Name of an existing EC2 KeyPair for SSH accessType: String...Mappings:  AWSInstanceType2Arch:     m1.small: {Arch: 32}    m1.large: {Arch: 64}...  DistroArch2AMI:    F18: {32: F18-i386-cfntools, 64: F18-x86_64-cfntools}, ...Resources:  WebServer:     Type: AWS::EC2::Instance    Metadata:       AWS::CloudFormation::Init:         config:           packages:             yum:              gcc-c++: []               make: []...
  18. 18. — CloudFormation clone— Targetting OpenStack resources and AWS compatibility— Adds YAML support (comments and easier)— Multi-cloud support using Apache DeltacloudOpenStack HeatAvailable NOW in Grizzly!
  19. 19. OpenStack Heat— Still limited to selected clouds— Still limited in concepts— Only slightly less cumbersome to write— Very limited dependency injectionBut...
  20. 20. Procedural Approach10 INPUT "What app do you like? ", A$20 INPUT "What cloud do you like? ", C$30 GOSUB 10040 END100 REM TODO deploy C$ to A$110 RETURN
  21. 21. Procedural Approach— Millions of lines of bash, chef, python, ruby, java, ... or ...— Apache Whirr— Gigaspaces Cloudify— Brooklyn and Cloudsoft AMP
  22. 22. Apache Whirr— Apache top level project— Deploy multi-node applications— Hadoop, Hama, Mahout, web,Cassandra & many more— Define topologies declaratively— Add new roles, configuration,constraints and wiring procedurallyprovisioninstallconfiguremanage
  23. 23. Apache Whirr— Apache top level project— Deploy multi-node applications— Hadoop, Hama, Mahout, web,Cassandra & many more— Define topologies declaratively— Add new roles, configuration,constraints and wiring procedurally$ whirr launch-cluster --cluster-name=hbase --instance-templates=1 hbase-master,6 hbase-regionserver
  24. 24. Apache Whirr— Apache top level project— Deploy multi-node applications— Hadoop, Hama, Mahout, web,Cassandra & many more— Define topologies declaratively— Add new roles, configuration,constraints and wiring procedurallyspec = new ClusterSpec();spec.setProvider("cloudservers-uk");spec.setIdentity(apikey);spec.setCredential(secret);spec.setClusterName("hbase");spec.setInstanceTemplates(ImmutableList.of(new InstanceTemplate(1, "hbase-master"),new InstanceTemplate(6, "hbase-regionserver")));cluster = new ClusterController().launchCluster(spec);
  25. 25. — Limited parameterisation— Defining new roles requires coding— Strict phases limit applicabilityBut...— Composable to one dimension— Portable locations support— Pluggable with bash, Chef, Puppet, and moreApache Whirr
  26. 26. Gigaspaces Cloudify— Apache-licensed Open PaaS Stack— Deployment and Management— Build applications declaratively— Define services procedurally— Configure auto-scaling declaratively
  27. 27. Gigaspaces Cloudify— Apache-licensed Open PaaS Stack— Deployment and Management— Build applications declaratively— Define services procedurally— Configure auto-scaling declarativelyapplication {  name="petclinic"  service {    name = "mysql"  }  service {    name = "tomcat"    dependsOn = ["mysql"]  }}
  28. 28. Gigaspaces Cloudify— Apache-licensed Open PaaS Stack— Deployment and Management— Build applications declaratively— Define services procedurally— Configure auto-scaling declarativelyservice {  name "mysql"  icon "mysql.jpg"  type "DATABASE"  lifecycle{    install "mysql_install.groovy"    start "mysql_start.groovy"    ...  }}
  29. 29. Gigaspaces Cloudify— Apache-licensed Open PaaS Stack— Deployment and Management— Build applications declaratively— Define services procedurally— Configure auto-scaling declarativelydef installLinuxMysql(config,context,builder,currVendor,installScript,myCnfObject) {" builder.sequential {" " chmod(dir:"${context.serviceDirectory}", perm:"+x", includes:"*.sh")" " exec(executable: "${context.serviceDirectory}/${installScript}",failonerror: "true")}...switch (currVendor) {case ["Ubuntu", "Debian", "Mint"]:context.attributes.thisInstance["binFolder"]="/usr/bin"installLinuxMysql(config,context,builder,currVendor,"installOnUbuntu.sh",myCnfObject)...
  30. 30. Gigaspaces Cloudify— Apache-licensed Open PaaS Stack— Deployment and Management— Build applications declaratively— Define services procedurally— Configure auto-scaling declarativelyif sudo grep -q -E [^!]requiretty /etc/sudoers; thenecho "Defaults:`whoami` !requiretty" | sudo tee /etc/sudoers.d/`whoami` >/dev/null    sudo chmod 0440 /etc/sudoers.d/`whoami`fisudo apt-get -y -q update || error_exit $? "Failed on: sudo apt-get -y update"killMySqlProcesssudo apt-get -y -q purge mysql-client* mysql-server* mysql-common*sudo rm -rf /etc/mysql || error_exit $? "Failed on: sudo rm -rf /etc/mysql"sudo DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::=--force-confnew -q -y install mysql-server-core mysql-server mysql-client mysql-common || error_exit $? "Failed ... ”
  31. 31. Gigaspaces Cloudify— Apache-licensed Open PaaS Stack— Deployment and Management— Build applications declaratively— Define services procedurally— Configure auto-scaling declarativelyscalingRule {  serviceStatistics {    metric "Requests per second"    movingTimeRangeInSeconds 20  }  highThreshold {    value 100    instancesIncrease 1  }}
  32. 32. — Defining new services requires codingBut...— Composable to one dimension— Parameterisable recipes— Portable locations support— Pluggable with bash, Chef, Puppet, and moreGigaspaces Cloudify
  33. 33. Brooklyn & Cloudsoft AMP— Brooklyn = Apache-licensed open-source— AMP = Enterprise + Support— Deployment and Management of Anything— Hierarchical Autonomic Policies— Define blueprints procedurally
  34. 34. BrooklynContention ManagementCustom PaaSBig Data
  35. 35. public class MyWebCluster extends AbstractApplicationimplements MyWebClusterConstants {public void init() {MySqlNode mysql = addChild(EntitySpecs.spec(MySqlNode.class).configure("creationScriptUrl", DB_SETUP_SQL_URL));ControlledDynamicWebAppCluster web = addChild(EntitySpecs.spec(ControlledDynamicWebAppCluster.class).configure(WebAppService.HTTP_PORT, PortRanges.fromString("8080+")).configure(JavaWebAppService.ROOT_WAR, WAR_PATH).configure(javaSysProp("brooklyn.example.db.url"),formatString("jdbc:%s%s?user=%s&password=%s",attributeWhenReady(mysql, MySqlNode.MYSQL_URL),DB_TABLE, DB_USERNAME, DB_PASSWORD))..configure(JcloudsLocationConfig.MIN_RAM, 2048).configure(JcloudsLocationConfig.TEMPLATE_SPEC, "osFamily=centos,os64Bit=true") );}}Brooklyn: Nested Elastic Blueprints
  36. 36. public class MyWebCluster extends AbstractApplicationimplements MyWebClusterConstants {public void init() {MySqlNode mysql = ...ControlledDynamicWebAppCluster web = ...web.addEnricher(HttpLatencyDetector.builder().url(ROOT_URL).rollup(10, SECONDS).build());web.getCluster().addPolicy(AutoScalerPolicy.builder().metric(REQUESTS_PER_SECOND_IN_WINDOW_PER_NODE).metricRange(10, 100).sizeRange(2, 5).build());addEnricher(SensorPropagatingEnricher.newInstanceListeningTo(web,ROOT_URL, REQUESTS_PER_SECOND_IN_WINDOW, REQUEST_LATENCY_IN_SECONDS_IN_WINDOW));}}Brooklyn: Policies, Enrichers and KPI’s
  37. 37. Brooklyn— Defining blueprints requires codingBut...— Composable and substitutable— Parameterisable blueprints— Portable and powerful locations support— Pluggable with VM images, Heat, Chef, more
  38. 38. Applications— Components (IaaS-resources)— DeclarativeInfrastructure— Procedural— Extensible (new resources)— Portable (hybrid, metal)— Management— Composable (stacks)
  39. 39. Heat in GrizzlyManagementComposableDeclarativeComponentsExtensiblePortablethanks largely to Red Hat
  40. 40. Portland Design Summit — HeatManagementComposableDeclarativePortableComponentsExtensible
  41. 41. Portland Design Summit — HeatManagementComposableDeclarativePortableComponentsExtensiblelots of new interest and ideas
  42. 42. Portland Design Summit — HeatComposableDeclarativeComponentsManagementPortableEase-of-UseExtensibleingredients from Brooklyn, Whirr, OoO and others
  43. 43. Portland Design Summit — HeatComposableDeclarativeComponentsManagementPortableExtensibleAuto-wiringa special ingredient from RackspaceEase-of-Use
  44. 44. Portland Design Summit — HeatComposableDeclarativeComponentsManagementExtensibleAuto-wiringRelationshipsEase-of-UsePortablea special ingredient from IBM
  45. 45. ManagementComposableDeclarativeComponentsAuto-wiringRelationshipsExtensibleEase-of-UsePortablePortland Design Summit — Heat
  46. 46. Heat in HavanaManagementComposableDeclarativeComponentsAuto-wiringRelationshipsExtensibleEase-of-UsePortableConcurrentDistributedResilientFunctions / DI
  47. 47. Heat in HavanaManagementComposableDeclarativeComponentsAuto-wiringRelationshipsExtensibleEase-of-UsePortableConcurrentDistributedResilientFunctions / DIStandards ?
  48. 48. OasisTOSCAhttp://www.oasis-open.org/committees/tosca/Topology and Orchestration Specification for Cloud Applications
  49. 49. Modeling Topologies with TOSCAService  Topologies  are  described  using  the  TOSCA  “Meta-model”:Artifacts Describe Installables and Executables required toinstantiate and manage a service. Currently, theyinclude: Implementation Artifacts:– Executables or Plans that implement  a  Node’s  or  Relationship’s  Operations (e.g. a Bash script) Deployment Artifacts:– Installables of the components (e.g. a TAR file)A  service’s  Topology Model is included in a TOSCA Service Template which is packaged and shared, along withall dependent artifacts, as a TOSCA Cloud Service Archive (CSAR)Service Templates Group the nodes and relationships that make upa  service’s  topology– Allowing modeling of sub-topologies Service Templates “look  like  nodes” enabling: Composition of applications from one or moreservice templates Substitution of abstract Node types withavailable service templates of the same typeNodes Represent Components of an application or serviceand their Properties. Example nodes include:– Infrastructure: Compute, Network, Storage, etc.– Platform: OS, VM, DB, Web Server, etc.– Granular: functional Libraries, Modules, etc. Include Operations which are the managementfunctions for the node– e.g. deploy(), start(), stop(), connect(), etc. Export their dependencies on other nodesas Requirement and CapabilitiesRelationships Represent the logical Relationships betweennodes– e.g.  “hostedOn”,  “connectsTo”,  etc. Describes the valid Source and Target nodes theyare designed to couple– e.g. source  “web application”  node  is  designed  to  “connectTo”  a  target  “database”  node Have their own Properties and ConstraintsSlide origin: IBMTOSCA: Modelling Topologies
  50. 50. <Definitions id="PayrollDefinitions" targetNamespace="http://www.example.com/ste"xmlns:pay="http://www.example.com/ste/Types"><Import namespace="http://www.example.com/ste/Types" location="http://.../PayrollTypes.tosca"importType=" http://docs.oasis-open.org/tosca/ns/2011/12"/><ServiceTemplate id="Payroll" name="Payroll Service Template"><TopologyTemplate ID="PayrollTemplate"><NodeTemplate id="Payroll Application" type="pay:ApplicationNodeType"><DeploymentArtifacts><DeploymentArtifact name="PayrollEAR" type="http://..../CSARref">EARs/Payroll.ear</DeploymentArtifact></DeploymentArtifacts></NodeTemplate><NodeTemplate id="Application Server" type="pay:ApplicationServerNodeType"><DeploymentArtifacts><DeploymentArtifact name="ApplicationServerImage" type="http://.../AMIref">ami-edf2cf99</DeploymentArtifact></DeploymentArtifacts></NodeTemplate><RelationshipTemplate id="deployed_on" type="pay:deployed_on"><SourceElement ref="Payroll Application"/><TargetElement ref="Application Server"/></RelationshipTemplate></TopologyTemplate></ServiceTemplate></Definitions>TOSCA: Sample Definition
  51. 51. — Fails the“ease-of-use”test— Does not standardise an APIBut...— Powerful and broad modelling coverage— Workflow, locations, substitution— Relatively matureTOSCA
  52. 52. OasisCAMPwww.cloudspecs.orgCloud Application Management Platformhttp://www.oasis-open.org/committees/camp/
  53. 53. CAMP: Objectives and Non-Objectives— Software— an Orchestration Spec— a Modelling SystemBut CAMP is NOT— a REST API— for Deployment and Management— of ApplicationsIt provides a flexible way to use many platforms.Interoperability.
  54. 54. CAMP: Key ConceptsClientsRESTPlatformPlatformComponentTemplateRequirementTypePlatformComponentTemplatePlatformComponentTemplateRequirementTypeApp ComponentTypeApp ComponentType
  55. 55. Clients PlatformPlatformComponentTemplateRequirementTypePlatformComponentTemplatePlatformComponentTemplateRequirementTypeApp ComponentTypeApp ComponentTypePDP = PlatformDeployment Plan(blueprint)— Deployment PlanCAMP: Key ConceptsDSL & REST
  56. 56. App ComponentTypePDP = PlatformDeployment Plan(blueprint)CAMP: Key ConceptsClientsRESTPlatformPlatformComponentTemplateRequirementTypePlatformComponentTemplatePlatformComponentTemplateRequirementTypeApp ComponentType— Creating TemplatesPOST
  57. 57. App ComponentTemplateClientsRESTPlatformPlatformComponentTemplateRequirementTypePlatformComponentTemplatePlatformComponentTemplateRequirementTypeApp ComponentTypeApp ComponentType— ResolutionCAMP: Key ConceptsAssemblyTemplate(blueprint)PDP = PlatformDeployment Plan(blueprint)POST
  58. 58. CAMP: Key ConceptsClientsRESTPlatformPlatformComponentTemplateRequirementTypePlatformComponentTemplatePlatformComponentTemplateRequirementTypeApp ComponentTypeApp ComponentTypeApp ComponentTemplateRequirementTypePlatformComponentTemplateRequirementTypeApp ComponentType— ResolutionAssemblyTemplate(blueprint)POST
  59. 59. AssemblyTemplate(blueprint)CAMP: Key ConceptsClientsRESTPlatformPlatformComponentTemplateRequirementTypePlatformComponentTemplatePlatformComponentTemplateRequirementTypeApp ComponentTypeApp ComponentTypeApp ComponentTemplate— Resolution200AssemblyTemplatePOST
  60. 60. AssemblyTemplate(blueprint)CAMP: Key ConceptsClientsRESTPlatformPlatformComponentTemplateRequirementTypePlatformComponentTemplatePlatformComponentTemplateRequirementTypeApp ComponentTypeApp ComponentTypeApp ComponentTemplate— Deployment
  61. 61. Assembly(blueprint) PlatformComponentPlatformComponentApp ComponentAssemblyTemplate(blueprint)CAMP: Key ConceptsClientsRESTPlatformPlatformComponentTemplateRequirementTypeRequirementTypeApp ComponentTypeApp ComponentTypeApp ComponentTemplate— DeploymentPOSTPlatformComponentTemplatePlatformComponentTemplate
  62. 62. Assembly(blueprint) PlatformComponentPlatformComponentApp ComponentCAMP: Key ConceptsClientsRESTPlatform— DeploymentAssembly(blueprint) PlatformComponentPlatformComponentApp ComponentAssemblyTemplate(blueprint)PlatformComponentTemplateApp ComponentTemplatePlatformComponentTemplatePOST200Assembly
  63. 63. CAMP— depends on orchestrations supporting this— wants standardised requirement and component types— still needs a nice DSLBut...— all very general— designed to map on to a wide range of orchestrationsPaaS, TOSCA, Brooklyn, Heat— OSS code in progressgithub.com/brooklyncentral/camp-server
  64. 64. CAMP: PDP/Blueprint DSL (work in progress)ParametersComponentsRequirementsFunctionsparameters & sensorsOutputs**
  65. 65. CAMP: PDP/Blueprint DSL (work in progress)ParametersFunctionsparameters & sensorsOutputs**components:my_app:type: WARcontent: hello.warrequirements:hosted_on:javaVersion: [1.7,)
  66. 66. Mappings(config metadata)WaitConditions &WaitHandles(orchestration)Parameterscomponents:my_instance:type: serverram: 2048mrequires:connects_to:target: my_databaseHeat: New HOT DSL (also work in progress)FunctionsOutputsHeatOrchestrationTemplateComing inHavana
  67. 67. Heat in HavanaManagementComposableDeclarativeComponentsAuto-wiringRelationshipsExtensibleEase-of-UsePortableConcurrentDistributedResilientFunctions / DI
  68. 68. Wrap-UpIt’s about the apps.Come get involved!Declarative — OpenStack Heat (Grizzly) and AWS CloudFormationProcedural — Apache Whirr, Gigaspaces Cloudify, Cloudsoft BrooklynOpenStack Summit — Red Hat, RAX, IBM, HP, Cloudsoft,OASIS TOSCA and CAMPDSLs in Progress — CAMP PDP and OpenStack Heat (Havana)
  69. 69. Thank you.alex.heneveld@cloudsoftcorp.com@ahtweetin

×