Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Event-driven automation, DevOps way ~IoT時代の自動化、そのリアリティとは?~

4,830 views

Published on

【StackStorm Slack日本語チャネル開設】
日本語でのStackStorm情報交換の場所として、Slackの日本語チャネルを開設しました。
SlackのStackstorm-communityチームに登録されている方はそのまま#community-japanへJoinできます。
まだstackstorm-communityチームに登録されていない方はこちらから⇒ http://brcd.co/TOM0Iu
-------------------------------------
StackStormが生まれた背景、StackStormをめぐる最新事情やリアリティ、さらに、ネットワークだけではない自動化の新たな可能性について語ります。
※2017年3月23日開催『StackStorm 勉強会 第2回』の講演資料(英語)です。

Published in: Technology
  • Be the first to comment

Event-driven automation, DevOps way ~IoT時代の自動化、そのリアリティとは?~

  1. 1. StackStorm If-This-Than-That for DevOps automation © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC.
  2. 2. @StackStorm@Stack_Storm
  3. 3. Agenda What is StackStorm Why we made it Use cases Why is it better than …
  4. 4. StackStorm: What is it? © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC.
  5. 5. StackStorm is like … 7 for DevOps
  6. 6. IFTTT, for DevOps 8 cat /opt/stackstorm/packs/st2-demos/rules/demo.yaml --- name: "sensu_crit_to_slack" pack: "st2-demos" description: "Post all critical alerts to the demo enabled: true trigger: type: "sensu.event_handler" criteria: trigger.check.status: pattern: 2 type: "equals" trigger.check.name: pattern: "demo_.*" type: "matchregex" action: ref: "slack.post_message" parameters: message: > [ALERT]{{trigger.client.name}} {{trigger.check.output}} channel: "#demos"
  7. 7. 9
  8. 8. 10 Trigger Action Rule
  9. 9. Ingredients 11 IT Domains Config mgmtStorageNetworking ContainersCloud InfraMonitoring ActionsSensors WorkflowsRules Ops Support Trigger Call
  10. 10. Automation Example 12 Automation EngineerService Monitoring Incident Management Event: “low disk on web301” Web301 is “low disk” Resolve known cases, fast. Is it /var/log? Clean up! Unknown problem, need a human Wake up, buddy. Something real is going on…
  11. 11. StackStorm: Why we made it? © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC.
  12. 12. Opalis, now Microsoft SC Orchestrator 2004-2008 © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. Event-driven automation with workflows, for Enterprise IT
  13. 13. VS VMware, 2008 - 2013
  14. 14. VS © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. 16 OpenStack VMWare
  15. 15. DevOps Tools © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. 17 VS Enterprise Suites
  16. 16. VS 18 Event-driven automation with workflows, for Cloud and DevOps DevOps Tools Enterprise Suites
  17. 17. 19 What can be it automated?
  18. 18. 20 Got StackStorm ! Every automation looks like a nail
  19. 19. What SHOULD be automated? From: Practice of Cloud System Administration, by Thomas Limoncelli
  20. 20. What HAS BEEN automated with StackStorm • Security checks – On malware detection in a VM, isolate network port on a switch • App blue-green deployment – On Jenkins tests passed, bring new vm claster, deploy and configure app, set loadbalancer to send % of traffic to new app, monitor, roll forward, or back out • Networking – On BGP peer goes down: collect troubleshooting data, post on slack & create JIRA ticket – On Link aggregation member error, check load, if capacity of rest of LAG bundle enough, disable link with error • OpenStack – orphan VM clean-up: On orphans detected, shut down, email owner, keep for few days, delete – VM evacuation on HW failures: On host RAID failure, get list of impacted VMs, email VM owners, evacuate VMs, create JIRA ticket for hardware replacement. • Service remediation: – Cassandra “node down” recovery: On ring node dying, deploy new node, configure, add to the ring. – Remediating RabbitMQ, Galera cluster, MySQL, and more… 22
  21. 21. 23 Auto-Remediation FB auto-remediates 98% alarms, can you?
  22. 22. “Auto-Remediation & Automation at Facebook” @ Auto-Remediation Meetup SF https://www.meetup.com/Auto-Remediation-and-Event-Driven-Automation/events/236704012/ Facebook FBAR: 43 % Problem Fixed 51 % False positives 94 % Automated
  23. 23. “Auto-Remediation & Automation at Facebook” @ Auto-Remediation Meetup SF https://www.meetup.com/Auto-Remediation-and-Event-Driven-Automation/events/236704012/ FBAR & StackStorm are friends StackStorm is inspired by FBAR StackStorm and FB FBAR collaborating since 2014
  24. 24. “Sleep Better at Night: OpenStack Cloud Auto-Healing” @ OpenStack Summit Barcelona Mirantis: Auto-remediating 2,000 node OpenStack cluster at Symantec with StackStorm
  25. 25. “Sleep Better at Night: OpenStack Cloud Auto-Healing” @ OpenStack Summit Barcelona Mirantis: Auto-remediating 2,000 node OpenStack cluster at Symantec with StackStorm User: Symantec (Mirantis) Use case: OpenStack cluster remediation Presented by Mirantis at OpenStack Barcelona StackStorm at Symantec
  26. 26. Engineer Wakes up Logs in and ACK Checks runbook Studies the alert Fixes the problem Runs diagnostics PagerDuty Alert 2:02 AM 2:07 AM 2:15 AM2:10 AM 2:30 AM2:20 AM2:00 AM On-call, Without Automation Source: “Winston: Helping Netflix Engineers Sleep at Night” @ Qcon ‘16 SF https://goo.gl/lHzq4r
  27. 27. False Positive Winston 2:00 AM 2:05 AM 2:05 AM 2:15 AM Assisted Diagnostics Fixed the problem On-call With Winston Source: “Winston: Helping Netflix Engineers Sleep at Night” @ Qcon ‘16 SF https://goo.gl/lHzq4r
  28. 28. Benefits • Reduce MTR (Mean Time to Resolution) • Avoid failures (fixing on computer time, not human time) • Reduce risk of human error (no fat fingers) • Positive team impact – Avoid pager fatigue and team burn-out – Turn from reactive to proactive (break reactive vicious cycle) – Capture operational knowledge – as code © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. 30
  29. 29. 31 Network Automation Now supporting Multiple Vendors
  30. 30. Device support Proven approach from cloud compute space 33 NAPALM AUTOMATION Packs PYTHON BINDINGS INTEGRATION PACKS DEVICE / OS INTERFACES Some workflows [1] leverage unique device functions, so they call actions of the device’s integration pack. Other workflows [2] need to be abstracted and treat devices alike (e.g. IXP provision on mixture of SLX and MLX). So they use “unified” Napalm pack. device drivers Integration packs expose device configurations and operations as st2 actions. VDX and SLX packs will expose most/all of device functionality. MLX is “best effort”. Napalm integration pack provide “unified” device actions, just like libcloud does in “compute” domain. While integration packs can call device interfaces directly, python bindings provide reuse, and abstract device OS versions. PyNOS binds VDX via netconf. SLX and MLX don’t exist yet. Napalm is Open source project that exposes a unified python API to interact with different network devices via device drivers. Devices expose various interfaces: RESTCONF, NETCONF, CLI/TELNET, SNMP. VDX MLX SLX [1] [2] Other vendors
  31. 31. 34 Including legacy, business apps … Integration
  32. 32. “Innovation at Dimension Data: Optimizing Operations with Event Driven Automation” https://stackstorm.com/2016/12/15/dimension-data-devops-beyond-deployment/ Dimension Data (SP, part of NTT)
  33. 33. “Innovation at Dimension Data: Optimizing Operations with Event Driven Automation” https://stackstorm.com/2016/12/15/dimension-data-devops-beyond-deployment/ • Integrate IT systems & tools • Security automation • Legacy Run-book replacement • Automation-aaS to end-users • Top st2 contributors Dimension Data (SP, part of NTT)
  34. 34. 38 IoT Fun stuff
  35. 35. 40 Serverless Grab StackStorm & DIY
  36. 36. StackStorm is like … 41 AWS Lambda AWS Step Functions OpenSource, for DIY Serverless
  37. 37. StackStorm is like … 42 ActionsSensors WorkflowsRules IT Domains Config mgmtStorageNetworking ContainersCloud InfraMonitoring Ops Support Step Functions AWS Lambda
  38. 38. Serverless with Swarm for Genomic Annotation Computing Dmitri Zimine http://github.com/dzimine/serverle @dzimine Image by Miki Yoshihito, Creative Commons license
  39. 39. Many Use Cases – One Platform 44 StackStorm automation platform Network Automation Assisted Troubleshooting Auto Remediation ITprocess integration IoT InternetofThings Serverless CI/CD ContinuousDeployment NFV Security Orchestration ChatOps
  40. 40. Why not… 45
  41. 41. Why not Scripts? 46
  42. 42. Why not Scripts? 47 • Simple to define, reason, visualize • Transparent – state is clear, execution is trackable: running, complete, failed steps • – – –
  43. 43. © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC.48
  44. 44. Workflows Better in Operations • Simple to define, reason, visualize • Transparent – state is clear, execution is trackable: running, complete, failed steps • Reliable – Workflows are long-running – Crash tolerance – “Restart from point of failure” © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. 49
  45. 45. 50 Why not legacy Runbook Automation? • Microsoft System Center Orchestrator • HP Operation Orchestrator • Cisco Process Orchestrator (CPO) • VMWare vCO / vRealize They do not DevOps
  46. 46. Infrastructure As Code Leverage social coding and collaboration OpenSource Designed for Devops
  47. 47. Infrastructure As Code Leverage social coding and collaboration OpenSource Designed for Devops
  48. 48. 53 Infrastructure as code Case Study • Service Catalog backed up by workflow • Automate provisioning on VMW/OpenStack, 4 Data centers • Before: CPO, operator updates via GUI, click and pray, x4 • After: StackStorm, dev -> code review -> staging -> QA-> prod
  49. 49. Infrastructure As Code Leverage social coding and collaboration OpenSource Designed for Devops
  50. 50. 265 Contributors Source: https://openhub.net/p/st2
  51. 51. 256 = 100,000,0002 Contributors
  52. 52. StackStorm & BWC Usage © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. 57 0 500 1000 1500 2000 2500 3000 Sep Oct Nov Dec Jan Feb Installations/month - StackStorm - BWC
  53. 53. StackStorm Exchange © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. 58
  54. 54. StackStorm Exchange © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. 59
  55. 55. © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. 60
  56. 56. © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. 61
  57. 57. Take away: * Try it * Use it * Contribute to it 62
  58. 58. 63
  59. 59. • Use StackStorm. Try it, find automation, nail POC. Let us know, good & bad. curl -sSL https://stackstorm.com/packages/install.sh | bash -s docs.stackstorm.com/install • Commit code. Become a “community maintainer” It is not hard (2 days?). We help & support. • Spread the word Blog. Tweet. Talk. Mention. Bug. Github Star! 64 Contribute! Everything counts
  60. 60. Thank You! @Stack_Storm http://github.com/StackStorm/st2 Star 1,869 Dmitri Zimine @dzimine
  61. 61. OpenSource Apache 2.0 • Github: github.com/StackStorm/st2 • Twitter: Stack_Storm • IRC: #stackstorm on FreeNode • stackstorm.slack.com on Slack • www.stackstorm.com © 2016 BROCADE COMMUNICATIONS SYSTEMS, INC. 66 StackStorm Brocade Workflow Composer Commercial Edition • Enterprise features • Priority support • brocade.com/bwc • docs: bwc-docs.brocade.com • Network lifecycle automation suite

×