It’s the App, Stupid!
Orchestration, Automation, Scaling
& What’s in Between
Uri Cohen
GigaSpaces
@uri1803
#CCNA14 #ItsThe...
Meet
Petsy, Selling
Pet Art and
Supporting
Pet Artists
Since 2013
So Their Services Are Deployed
on CloudStack
Gunicorn
PostgreSQL
Hadoop
ActiveMQ
Nginx
MongoDB
Logstash
Graphite
Jenkins
N...
The Business
Is Doing
Great, Thx
(esp. Cat
paper crafts),
But…
Rolling out new
code is Painful
http://www.interestingtopic...
The Business
Is Doing
Great, Thx
(esp. Cat
paper
crafts), But…
MTTR is… mehhh
Mehhh
http://fineartamerica.com/featured/woo...
They Need to
Automate!!
Let’s have a
closer look
at DevOps
Processes
It’s about
Workflows
and
Triggers
Automated
Deployment
• Triggers: Manual,
CI Server
• Flow:
– Provision Cloud
Resources: Compute,
Storage, Network
– Config...
Infra
Upgrade
• Trigger: Critical
updates, patches, et
c.
• Flow: Take down
relevant
processes, one by
one, apply
patch, a...
Code Push
(or more
generally, CD
)
• Trigger: CI server,
Manual
• Flow: Canary,
Red/Black, A/B…
– In general – Partial
dep...
Node Failure
• Trigger: #$%@
happens…
• Flow: Detect, provision
VM, attach
storage, add to
network, reconfigure
app (load ...
Scaling
• Trigger: System
SLAs, Schedule
• Flow: Add new
nodes, push
code, reconfigure
dependent
nodes, reconfigur
e LB
Automation Encompasses All Layers
Proxy Config WAR Schema
Infrastructure
Middleware
Apache Tomcat MySQL
Application
Host N...
Automation Building Blocks
Environment
Creation
SW Infra.
Setup &
Config
Code Push Monitoring
& Alarming
Repairing Scaling
Let’s Look at
Some Tools
Orchestration Tools
Environment
Creation
SW Infra.
Setup &
Config
Code Push Monitoring
& Alarming
Repairing Scaling
CM Tools
Environment
Creation
SW Infra.
Setup &
Config
Code Push Monitoring
& Alarming
Repairing Scaling
Automation Tools
Environment
Creation
SW Infra.
Setup &
Config
Code Push Monitoring
& Alarming
Repairing Scaling
Metric & Log Collection Tools
Environment
Creation
SW Infra.
Setup &
Config
Code Push Monitoring
& Alarming
Repairing Scal...
Still, when you
need to
automate &
orchestrate, th
at’s what is
looks like…
Where You Want to Be
Environment
Creation
SW Infra.
Setup &
Config
Code Push Monitoring
& Alarming
Repairing Scaling
How It’s Done in AWS
How It’s Done in AWS
He Calls It
DevOps Automation
http://www.allthingsdistributed.com/2013/02/aws-opsworks.html
Still a Bit Too
Rigid
OpsWorks –
We Can Do
Better!
• Integrated with
CloudStack (and
other clouds)
• Custom
Workflows
• Not limited to
Chef
• Op...
Our TOSCA-Inspired Building Blocks
Application
Topologies
Workflows
Policies
Topology, Visualized
Topology, Visualized
Node
Node
Node
Connected_to
relationship
Hosted_on
relationship
Architecture
REST
GUI
Workflow
Engine
Task
Manager
Blueprint + Runtime
Data
Policy
Engine
Agent
Monitoring
Data
Agent
Moni...
Demo Time!
Where We
Are Today
Thank You!
getcloudify.org
Upcoming SlideShare
Loading in...5
×

Its the app stupid - CloudStack 2014 Collaboration Conference #CCNA14

892

Published on

Slides from my session at #CCNA14
http://sched.co/1hETwSO

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
892
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Goals:Why Workflows are critical part of automation of applications on the cloudClarify the need for something like OpsWorksWhy do we think this OpsWorks is needed in addition to other projects
  • They do it manually, pushing code to git, then compiling
  • One of their servers failed last week, and it took almost a day to configure and bring up a new server. They use a certain CM tool, Still better than the week it took the time before, but that’s doesn’t cut it
  • They also realized that in order to run well in the cloud they need DevOps and automationWhat are the options
  • The building blocks are triggers and workflowsTriggers because triggers need to be automated as well as events resulting from policies / integrationetcWorkflows – because these are complex multi-step processesLet’s have a look at few typical use cases
  • Great at creating resources, basic integration with Chef / puppet for swconfig, basic built in monitoring and alarming, moving to ceilometer in Havana
  • Env setup is not automated SW infra setup is good but has no startup orchestration
  • Env setup is not automated SW infra setup is good but has no startup orchestration
  • Healthnmon is more geared toward cloud resource monitoring and is has more opinionated domain model, ceilometer seems to be picking momentum and
  • http://www.allthingsdistributed.com/2013/02/aws-opsworks.html
  • Its the app stupid - CloudStack 2014 Collaboration Conference #CCNA14

    1. 1. It’s the App, Stupid! Orchestration, Automation, Scaling & What’s in Between Uri Cohen GigaSpaces @uri1803 #CCNA14 #ItsTheAppStupid
    2. 2. Meet Petsy, Selling Pet Art and Supporting Pet Artists Since 2013
    3. 3. So Their Services Are Deployed on CloudStack Gunicorn PostgreSQL Hadoop ActiveMQ Nginx MongoDB Logstash Graphite Jenkins Nagios
    4. 4. The Business Is Doing Great, Thx (esp. Cat paper crafts), But… Rolling out new code is Painful http://www.interestingtopics.net/storage/5762c0bec0add88fae0c3958d61effe1.jpg
    5. 5. The Business Is Doing Great, Thx (esp. Cat paper crafts), But… MTTR is… mehhh Mehhh http://fineartamerica.com/featured/wooly-sheep-ramona-johnston.html
    6. 6. They Need to Automate!!
    7. 7. Let’s have a closer look at DevOps Processes It’s about Workflows and Triggers
    8. 8. Automated Deployment • Triggers: Manual, CI Server • Flow: – Provision Cloud Resources: Compute, Storage, Network – Configure servers – Push code – Start components http://www.flickr.com/photos/smilemark/4611091236/sizes/m/in/photostream/
    9. 9. Infra Upgrade • Trigger: Critical updates, patches, et c. • Flow: Take down relevant processes, one by one, apply patch, and reconnect to cluster. In some cases restarts server
    10. 10. Code Push (or more generally, CD ) • Trigger: CI server, Manual • Flow: Canary, Red/Black, A/B… – In general – Partial deploy, verify, rollback / complete deploy
    11. 11. Node Failure • Trigger: #$%@ happens… • Flow: Detect, provision VM, attach storage, add to network, reconfigure app (load balancer in case of web serve, DB connection in case of database, etc.)
    12. 12. Scaling • Trigger: System SLAs, Schedule • Flow: Add new nodes, push code, reconfigure dependent nodes, reconfigur e LB
    13. 13. Automation Encompasses All Layers Proxy Config WAR Schema Infrastructure Middleware Apache Tomcat MySQL Application Host Networks Storage
    14. 14. Automation Building Blocks Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
    15. 15. Let’s Look at Some Tools
    16. 16. Orchestration Tools Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
    17. 17. CM Tools Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
    18. 18. Automation Tools Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
    19. 19. Metric & Log Collection Tools Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
    20. 20. Still, when you need to automate & orchestrate, th at’s what is looks like…
    21. 21. Where You Want to Be Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
    22. 22. How It’s Done in AWS
    23. 23. How It’s Done in AWS
    24. 24. He Calls It DevOps Automation http://www.allthingsdistributed.com/2013/02/aws-opsworks.html
    25. 25. Still a Bit Too Rigid
    26. 26. OpsWorks – We Can Do Better! • Integrated with CloudStack (and other clouds) • Custom Workflows • Not limited to Chef • Open Monitoring and Policies
    27. 27. Our TOSCA-Inspired Building Blocks Application Topologies Workflows Policies
    28. 28. Topology, Visualized
    29. 29. Topology, Visualized Node Node Node Connected_to relationship Hosted_on relationship
    30. 30. Architecture REST GUI Workflow Engine Task Manager Blueprint + Runtime Data Policy Engine Agent Monitoring Data Agent Monitoring Agent Application Stack Cloudify Manager App VM Invokes Reports Creates Metrics VM Logs + Events Proxy + File Server
    31. 31. Demo Time!
    32. 32. Where We Are Today
    33. 33. Thank You! getcloudify.org
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×