An Application Centric Approach to Devops

1,008 views
909 views

Published on

When thinking about automation and orchestration most people have tools like CloudFormation, Chef or Puppet in mind. These tools do a great job at allocating infrastructure resources and configuring them. But in reality, when deploying and managing complete application stacks, there’s much more to it. You need things to be done in a certain order; there are dependencies to consider and information to share between your application tiers, and then there’s everything related to post deployment - recovering from failures, scaling and continuously deploying your code, to name some. In this session we’ll discuss what it takes to orchestrate and automate your applications through the entire lifecycle, and without losing control.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,008
On SlideShare
0
From Embeds
0
Number of Embeds
300
Actions
Shares
0
Downloads
27
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • We tend to think of devops in terms of infrastructure, or setting up nodes. Cloud platforms permit a level of automation that is was not possible before, and the list of services keeps growing. Today’s popular tools will automate the preparation of nodes and the deployment and configuration of software, which is great. But there is more to do before the promised land is reached. Ultimately, devops is about delivering applications to customers (internal or external), not roles of nodes. So how is the gap filled?
  • In order to implement a fully automated, continuous delivery platform, the bolting together of multiple tools is necessary. For example, a typical case I’ve encountered quite a bit is a build automation platform (Jenkins et. Al.) being used to tie together the various pieces required for automation. To deploy a technology stack to the cloud requires several build steps, and some kind of global storage. A pattern I’m familiar with is Jenkins and Chef. Jenkins runs a script or plugin that creates VMs on the cloud. Jenkins bootstraps chef client on the nodes and then runs the required cookbooks. This can be a complex multi-step process involving chef server data bags.
  • Contrast application vs infrastructure centric. Application centric creates a model of application requirements, and renders them onto an infrastructure. The infrastructure view thinks of mapping application components to hardware, which is actually more driven by traditional datacenter thinking.
  • Names familiar to everyone no doubt. Focused on installation and perhaps startup of applications targeting infrastructure, i.e. preparing nodes and installation sw.
  • What are the Specific Features in Cloudify that Makes it DevOps Oriented?I will use Matt Jaynes’ definition from his book Taste Test:Directive & Directives Script -  Cloudify currently uses Groovy-based recipes with extended DSL as its directives. The Cloudify recipe allows you to combine Chef or Puppet directives as part of the Cloudify recipie. Our future release will include support for Yaml/TOSCA directives and may also include support for Ansible.Master Node & Children Nodes - Cloudify automatically bootstraps Master and Child nodes. It has a cloud plug-in that will allow you to automate this entire process and avoid any manual host/IP configuration. The Cloudify integration with Chef takes care of provisioning of Chef Master and Child nodes as well.Remote Execution - Cloudify uses custom-commands as a way to route calls to a group of individual sets of service instances. The integration with Chef includes built-in custom commands for updating Chef Coookbooks as well as for executing Chef-Knife and other Chef commands as described here.In addition, Cloudify provides built-in services to query the state of the cluster through the ServiceContext API and share states across services in the cluster through the Attribute Store API.- See more at: http://natishalom.typepad.com/nati_shaloms_blog/2013/10/an-application-centric-approach-to-devops.html#sthash.k5cKhWhX.dpuf
  • What are the Specific Features in Cloudify that Makes it DevOps Oriented?I will use Matt Jaynes’ definition from his book Taste Test:Directive & Directives Script -  Cloudify currently uses Groovy-based recipes with extended DSL as its directives. The Cloudify recipe allows you to combine Chef or Puppet directives as part of the Cloudify recipie. Our future release will include support for Yaml/TOSCA directives and may also include support for Ansible.Master Node & Children Nodes - Cloudify automatically bootstraps Master and Child nodes. It has a cloud plug-in that will allow you to automate this entire process and avoid any manual host/IP configuration. The Cloudify integration with Chef takes care of provisioning of Chef Master and Child nodes as well.Remote Execution - Cloudify uses custom-commands as a way to route calls to a group of individual sets of service instances. The integration with Chef includes built-in custom commands for updating Chef Coookbooks as well as for executing Chef-Knife and other Chef commands as described here.In addition, Cloudify provides built-in services to query the state of the cluster through the ServiceContext API and share states across services in the cluster through the Attribute Store API.- See more at: http://natishalom.typepad.com/nati_shaloms_blog/2013/10/an-application-centric-approach-to-devops.html#sthash.k5cKhWhX.dpuf
  • AWSopsworks takes a devops approach to application management. Defines abstractions for stack, layer, network, scaling and deployment. Template driven provision with cloud formation.
  • AWSopsworks takes a devops approach to application management. Defines abstractions for stack, layer, network, scaling and deployment. Template driven provision with cloud formation.
  • AWSopsworks takes a devops approach to application management. Defines abstractions for stack, layer, network, scaling and deployment. Template driven provision with cloud formation.
  • AWSopsworks takes a devops approach to application management. Defines abstractions for stack, layer, network, scaling and deployment. Template driven provision with cloud formation.
  • “stack” level specification of roles is rendered on arbitrary IAAS platform. Extension of chef recipe concept (sort of).
  • What are the Specific Features in Cloudify that Makes it DevOps Oriented?I will use Matt Jaynes’ definition from his book Taste Test:Directive & Directives Script -  Cloudify currently uses Groovy-based recipes with extended DSL as its directives. The Cloudify recipe allows you to combine Chef or Puppet directives as part of the Cloudify recipie. Our future release will include support for Yaml/TOSCA directives and may also include support for Ansible.Master Node & Children Nodes - Cloudify automatically bootstraps Master and Child nodes. It has a cloud plug-in that will allow you to automate this entire process and avoid any manual host/IP configuration. The Cloudify integration with Chef takes care of provisioning of Chef Master and Child nodes as well.Remote Execution - Cloudify uses custom-commands as a way to route calls to a group of individual sets of service instances. The integration with Chef includes built-in custom commands for updating Chef Coookbooks as well as for executing Chef-Knife and other Chef commands as described here.In addition, Cloudify provides built-in services to query the state of the cluster through the ServiceContext API and share states across services in the cluster through the Attribute Store API.- See more at: http://natishalom.typepad.com/nati_shaloms_blog/2013/10/an-application-centric-approach-to-devops.html#sthash.k5cKhWhX.dpuf
  • What are the Specific Features in Cloudify that Makes it DevOps Oriented?I will use Matt Jaynes’ definition from his book Taste Test:Directive & Directives Script -  Cloudify currently uses Groovy-based recipes with extended DSL as its directives. The Cloudify recipe allows you to combine Chef or Puppet directives as part of the Cloudify recipie. Our future release will include support for Yaml/TOSCA directives and may also include support for Ansible.Master Node & Children Nodes - Cloudify automatically bootstraps Master and Child nodes. It has a cloud plug-in that will allow you to automate this entire process and avoid any manual host/IP configuration. The Cloudify integration with Chef takes care of provisioning of Chef Master and Child nodes as well.Remote Execution - Cloudify uses custom-commands as a way to route calls to a group of individual sets of service instances. The integration with Chef includes built-in custom commands for updating Chef Coookbooks as well as for executing Chef-Knife and other Chef commands as described here.In addition, Cloudify provides built-in services to query the state of the cluster through the ServiceContext API and share states across services in the cluster through the Attribute Store API.- See more at: http://natishalom.typepad.com/nati_shaloms_blog/2013/10/an-application-centric-approach-to-devops.html#sthash.k5cKhWhX.dpuf
  • *Tosca coming soon in 3.0 releaseWhat are the Specific Features in Cloudify that Makes it DevOps Oriented?I will use Matt Jaynes’ definition from his book Taste Test:Directive & Directives Script -  Cloudify currently uses Groovy-based recipes with extended DSL as its directives. The Cloudify recipe allows you to combine Chef or Puppet directives as part of the Cloudify recipie. Our future release will include support for Yaml/TOSCA directives and may also include support for Ansible.Master Node & Children Nodes - Cloudify automatically bootstraps Master and Child nodes. It has a cloud plug-in that will allow you to automate this entire process and avoid any manual host/IP configuration. The Cloudify integration with Chef takes care of provisioning of Chef Master and Child nodes as well.Remote Execution - Cloudify uses custom-commands as a way to route calls to a group of individual sets of service instances. The integration with Chef includes built-in custom commands for updating Chef Coookbooks as well as for executing Chef-Knife and other Chef commands as described here.In addition, Cloudify provides built-in services to query the state of the cluster through the ServiceContext API and share states across services in the cluster through the Attribute Store API.- See more at: http://natishalom.typepad.com/nati_shaloms_blog/2013/10/an-application-centric-approach-to-devops.html#sthash.k5cKhWhX.dpuf
  • Templates to describe and drive all these processes
  • An Application Centric Approach to Devops

    1. 1. Application Centric Devops Orchestration, DevOps Automation & What’s in Between DeWayne Filppi @dfilppi Dir. Solution Architecture Gigaspaces Inc.
    2. 2. What It Really Takes to Deploy and Manage Apps Provision Install Configure Deploy Monitor Scale
    3. 3. Quite often we end up with this..
    4. 4. Load Balancer Web DataBase Load Balancer Web DataBase Your Journey Start Here Your Journey Start Here Application Centric Infrastructure Centric Map Application Dependencies, SLA (performance, scaling, HA) Map Application To machine instances
    5. 5. Environment Creation Code Push Monitoring & Alarming Repairing Scaling Technical Perspective • Infrastructure Centric • Application Centric Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
    6. 6. INFRASTRUCTURE CENTRIC TOOLS
    7. 7. Configuration Management Defined
    8. 8. APPLICATION CENTRIC TOOLS
    9. 9. Configuration Management Defined • Directive & Directives Script • Master Node & Children Nodes • Remote Execution
    10. 10. App Centric Devops – the AWS way…
    11. 11. App Centric Devops – the AWS way… • Key abstractions – Stack (a complete deployment) – Layers (tiers) – Instances (per layer scaling) – Apps (from a repo)
    12. 12. App Centric Devops – the AWS way… • Layer def example
    13. 13. App Centric Devops – the AWS way… • Characteristics – Full stack definition – Lifecycle event handling – Auto healing – Network configuration – Chef integration – Elastic scaling
    14. 14. Cloudify provides the equivalent of Amazon OpsWork on OpenStack and other clouds App Centric Devops – the AWS way…
    15. 15. App Centric Devops – the Cloudify way… 1 Upload your recipe 2 Cloudify creates VMs, configures network & installs agents 3 Agents install and manage your app 4 Cloudify automates monitoring, healing and scaling
    16. 16. ® Copyright 2012 GigaSpaces Ltd. All Rights Reserved compute { template "SMALL_LINUX" } SMALL_LINUX : template imageId "us-east-1/ami-76f0061f“ remoteDirectory "/home/ec2-user/gs-files“ machineMemoryMB 1600 hardwareId "m1.small" availabilityZones([“us-east-1”,”us-east-2”]) localDirectory "upload" keyFile "myKeyFile.pem" options ([ "securityGroups" : ["default"] as String[], "keyPair" : "myKeyFile” ]) overrides (["jclouds.ec2.ami-query":"", "jclouds.ec2.cc-ami-query":""]) privileged true } SMALL_LINUX : template{ imageId "1234" machineMemoryMB 3200 hardwareId "103" remoteDirectory "/root/gs-files" localDirectory "upload" keyFile "gigaPGHP.pem“ availabilityZones([“az1”,”az2”]) options ([ "openstack.securityGroup" : "default", "openstack.keyPair" : "gigaPGHP” ]) privileged true } Declarative Compute Configuration
    17. 17. Declarative Network Configuration network { template "APPLICATION_NET" accessRules {[ incoming ([ accessRule { type "PUBLIC“ portRange "8000-8097" target "0.0.0.0/0" }, accessRule { type "APPLICATION" portRange "1-40000" target "0.0.0.0/0" } ]), outgoing ([ accessRule { type "PUBLIC" portRange "8443" target "0.0.0.0/0" }, accessRule { type "APPLICATION" portRange "1-40000" target "0.0.0.0/0" } …. Instance LevelCloud Level cloudNetwork { management { networkConfiguration { name "Cloudify-Management-Network" subnets ([ subnet { name "Cloudify-Management-Subnet" range "177.86.0.0/24" options ([ "gateway" : "177.86.0.111" ]) } ]) custom ([ "associateFloatingIpOnBootstrap" : "true" ]) } } templates ([ "APPLICATION_NET" : networkConfiguration { name "Cloudify-Application-Network" subnets ([ subnet { name "Cloudify-Application-Subnet" range "160.0.0.0/24" options { gateway "null" } } ]) custom ([ "associateFloatingIpOnBootstrap" : "true" ]) } ]) }
    18. 18. DevOps Automation: Example 1 • Very large U.S. hardware/software company. • Use Cloudify for Continuous Delivery • Stack: Cloudify, Chef, Openstack • Reason for Choosing Cloudify – Dozens of roles across multiple orgs. – Chef shop, needed backwards compatibility. – Cloud portability – SDN support – Devops integration with existing Jenkins/Chef infrastructure to build internal PAAS on internal cloud.
    19. 19. DevOps Automation: Bleeding edge • Full automation of stack deployment/scaling across multiple AZs. • Multi-tier SDN governed stack deployments (nets/subnets/firewalls/routers etc…) • Scaling to 1000s of nodes across org • Auto scaling based on load • Uses hosted chef for cookbook, attribute storage. • Replaces home grown system that uses chef server to coordinate complex/cluster deployments.
    20. 20. DevOps Automation: PaddyPower • Ireland's largest online betting service. • Use Cloudify for Continuous Delivery • Stack: Cloudify, Chef, Cloud Stack • Reason for Choosing Cloudify – PAAS strives to abstract provisioning and deployment thus reducing complexity. – PAAS constrains applications to use features provided by the platform. – Cloudify was geared to work in DevOps environment
    21. 21. PaddyPower View
    22. 22. PUTTING APPLICATION AND CONFIGURATION CENTRIC TOOLS TOGETHER
    23. 23. Cloudify Chef Integration (Deployment) • Install Chef Agent • Setup Chef Server • Support existing Chef Recipes/Cookbooks • Remote Execution • Application Dependency • Cloud portability
    24. 24. Cloudify Chef Integration (Post Deployment)
    25. 25. Cloudify Chef Integration (Advances) • Advanced Orchestration – Self Healing – Auto-Scaling – DR Automation – DevOps Policies* • Standard Recipe (Yaml/Tosca)*
    26. 26. Remote Execution
    27. 27. Update Orchestrator Jenkins API Update Load Balancer Web Database Update war() Reload() snapshot() Alter-table() addNode() removeNode() Monitoring Alarming
    28. 28. Example: Running MySQL
    29. 29. References • Cloudify Project Home Page: Cloudifysource.org • Cloudify Github site: https://github.com/CloudifySource/cloudify • Cloudify & Chef Integration • Blog Post: • Earlier Video Presentation
    30. 30. Thank You!

    ×