When thinking about automation and orchestration most people have tools like CloudFormation, Chef or Puppet in mind. These tools do a great job at allocating infrastructure resources and configuring them. But in reality, when deploying and managing complete application stacks, there’s much more to it. You need things to be done in a certain order; there are dependencies to consider and information to share between your application tiers, and then there’s everything related to post deployment - recovering from failures, scaling and continuously deploying your code, to name some. In this session we’ll discuss what it takes to orchestrate and automate your applications through the entire lifecycle, and without losing control.

1. Application Centric Devops
Orchestration, DevOps Automation
& What’s in Between
DeWayne Filppi @dfilppi
Dir. Solution Architecture
Gigaspaces Inc.

2. What It Really
Takes to
Deploy and
Manage Apps
Provision
Install
Configure
Deploy
Monitor
Scale

3. Quite often
we end up
with this..

4. Load
Balancer Web DataBase
Load
Balancer Web DataBase
Your Journey
Start Here
Your Journey
Start Here
Application Centric Infrastructure Centric
Map Application Dependencies, SLA
(performance, scaling, HA)
Map Application To machine instances

5. Environment
Creation
Code Push Monitoring
& Alarming
Repairing Scaling
Technical Perspective
• Infrastructure Centric
• Application Centric
Environment
Creation
SW Infra.
Setup &
Config
Code Push Monitoring
& Alarming
Repairing Scaling

6. INFRASTRUCTURE CENTRIC TOOLS

7. Configuration
Management
Defined

8. APPLICATION CENTRIC TOOLS

9. Configuration
Management
Defined
• Directive &
Directives Script
• Master Node &
Children Nodes
• Remote Execution

10. App Centric Devops – the AWS way…

11. App Centric Devops – the AWS way…
• Key abstractions
– Stack (a complete deployment)
– Layers (tiers)
– Instances (per layer scaling)
– Apps (from a repo)

12. App Centric Devops – the AWS way…
• Layer def example

13. App Centric Devops – the AWS way…
• Characteristics
– Full stack definition
– Lifecycle event handling
– Auto healing
– Network configuration
– Chef integration
– Elastic scaling

14. Cloudify provides the equivalent of
Amazon OpsWork on OpenStack and
other clouds
App Centric Devops – the AWS way…

15. App Centric Devops – the Cloudify way…
1 Upload your recipe
2 Cloudify creates VMs, configures
network & installs agents
3 Agents install and manage your app
4 Cloudify automates monitoring, healing
and scaling

16. ® Copyright 2012 GigaSpaces Ltd. All Rights Reserved
compute {
template "SMALL_LINUX"
}
SMALL_LINUX : template
imageId "us-east-1/ami-76f0061f“
remoteDirectory "/home/ec2-user/gs-files“
machineMemoryMB 1600
hardwareId "m1.small"
availabilityZones([“us-east-1”,”us-east-2”])
localDirectory "upload"
keyFile "myKeyFile.pem"
options ([
"securityGroups" : ["default"] as String[],
"keyPair" : "myKeyFile”
])
overrides (["jclouds.ec2.ami-query":"",
"jclouds.ec2.cc-ami-query":""])
privileged true
}
SMALL_LINUX : template{
imageId "1234"
machineMemoryMB 3200
hardwareId "103"
remoteDirectory "/root/gs-files"
localDirectory "upload"
keyFile "gigaPGHP.pem“
availabilityZones([“az1”,”az2”])
options ([
"openstack.securityGroup" : "default",
"openstack.keyPair" : "gigaPGHP”
])
privileged true
}
Declarative Compute Configuration

17. Declarative Network Configuration
network {
template "APPLICATION_NET"
accessRules {[
incoming ([
accessRule {
type "PUBLIC“
portRange "8000-8097"
target "0.0.0.0/0"
},
accessRule {
type "APPLICATION"
portRange "1-40000"
target "0.0.0.0/0"
}
]),
outgoing ([
accessRule {
type "PUBLIC"
portRange "8443"
target "0.0.0.0/0"
},
accessRule {
type "APPLICATION"
portRange "1-40000"
target "0.0.0.0/0"
}
….
Instance LevelCloud Level
cloudNetwork {
management {
networkConfiguration {
name "Cloudify-Management-Network"
subnets ([
subnet {
name "Cloudify-Management-Subnet"
range "177.86.0.0/24"
options ([ "gateway" : "177.86.0.111" ])
}
])
custom ([ "associateFloatingIpOnBootstrap" : "true" ])
}
}
templates ([
"APPLICATION_NET" : networkConfiguration {
name "Cloudify-Application-Network"
subnets ([
subnet {
name "Cloudify-Application-Subnet"
range "160.0.0.0/24"
options { gateway "null" }
}
])
custom ([ "associateFloatingIpOnBootstrap" : "true" ])
}
])
}

18. DevOps Automation: Example 1
• Very large U.S. hardware/software
company.
• Use Cloudify for Continuous
Delivery
• Stack: Cloudify, Chef, Openstack
• Reason for Choosing Cloudify
– Dozens of roles across multiple orgs.
– Chef shop, needed backwards compatibility.
– Cloud portability
– SDN support
– Devops integration with existing
Jenkins/Chef infrastructure to build internal
PAAS on internal cloud.

19. DevOps Automation: Bleeding edge
• Full automation of stack deployment/scaling
across multiple AZs.
• Multi-tier SDN governed stack deployments
(nets/subnets/firewalls/routers etc…)
• Scaling to 1000s of nodes across org
• Auto scaling based on load
• Uses hosted chef for cookbook, attribute
storage.
• Replaces home grown system that uses chef
server to coordinate complex/cluster
deployments.

20. DevOps Automation: PaddyPower
• Ireland's largest online betting
service.
• Use Cloudify for Continuous
Delivery
• Stack: Cloudify, Chef, Cloud Stack
• Reason for Choosing Cloudify
– PAAS strives to abstract provisioning and
deployment thus reducing complexity.
– PAAS constrains applications to use features
provided by the platform.
– Cloudify was geared to work in DevOps
environment

21. PaddyPower View

22. PUTTING APPLICATION AND
CONFIGURATION CENTRIC TOOLS
TOGETHER

23. Cloudify Chef
Integration
(Deployment)
• Install Chef Agent
• Setup Chef Server
• Support existing Chef
Recipes/Cookbooks
• Remote Execution
• Application
Dependency
• Cloud portability

24. Cloudify Chef
Integration
(Post Deployment)

25. Cloudify Chef
Integration
(Advances)
• Advanced
Orchestration
– Self Healing
– Auto-Scaling
– DR Automation
– DevOps Policies*
• Standard Recipe
(Yaml/Tosca)*

26. Remote Execution

27. Update
Orchestrator
Jenkins
API
Update
Load
Balancer Web Database
Update war()
Reload()
snapshot()
Alter-table()
addNode()
removeNode()
Monitoring
Alarming

28. Example: Running MySQL

29. References
• Cloudify Project Home Page:
Cloudifysource.org
• Cloudify Github site:
https://github.com/CloudifySource/cloudify
• Cloudify & Chef Integration
• Blog Post:
• Earlier Video Presentation

30. Thank You!