Cloudy Operations - OSCON 2010

  • 14,195 views
Uploaded on

OSCON 2010 Cloud Summit presentation. How to to operate in a cloudy world.

OSCON 2010 Cloud Summit presentation. How to to operate in a cloudy world.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
14,195
On Slideshare
0
From Embeds
0
Number of Embeds
6

Actions

Shares
Downloads
133
Comments
0
Likes
11

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. John Willis VP of Services john@opscode.com twitter.com/botchagalupe Copyright © 2010 Opscode, Inc - All Rights Reserved 1 Thursday, July 29, 2010
  • 2. IT Management Podcast DevopsCafe CloudCafe Copyright © 2010 Opscode, Inc - All Rights Reserved 2 Thursday, July 29, 2010
  • 3. IT Management Podcast DevopsCafe CloudCafe Copyright © 2010 Opscode, Inc - All Rights Reserved 2 Thursday, July 29, 2010
  • 4. Private Public SaaS PaaS PaaS IaaS IaaS Managed Virtualization hosting Slide courtesy Alistair Croll - alistair@rednod.com Thursday, July 29, 2010
  • 5. Private Public nt t o SaaS a w d s, o u lo u y c If l k PaaS rs t.PaaS t a ne f i i ck o IaaS p IaaS Managed Virtualization hosting Slide courtesy Alistair Croll - alistair@rednod.com Thursday, July 29, 2010
  • 6. Infrastructure as a Service (IaaS) Amazon EC2, Rackspace Cloud, Terremark, Gogrid, Joyent (and nearly every private cloud built on Zenserver or VMWare.) Slide courtesy Alistair Croll - alistair@rednod.com Thursday, July 29, 2010
  • 7. Cloudy Operations Thursday, July 29, 2010
  • 8. Pixie Dust! Copyright © 2010 Opscode, Inc - All Rights Reserved 6 Thursday, July 29, 2010
  • 9. Did They Lie? Copyright © 2010 Opscode, Inc - All Rights Reserved 7 Thursday, July 29, 2010
  • 10. Did They Lie? I did not have “cloudy” relations with that provider Copyright © 2010 Opscode, Inc - All Rights Reserved 7 Thursday, July 29, 2010
  • 11. Infrastructure is Hard! Copyright © 2010 Opscode, Inc - All Rights Reserved 8 Thursday, July 29, 2010
  • 12. Fully Automated Infrastructure Control Provisioning Release Orchestration Deploy CI Server Issue tracker Artifact Config repository Dispatcher SCM management Build Repository OS boot/ Scheduler install Code Workflows Model Resources Configuration Events Trending Asset Identity inventory Trending Reporting Topology Host naming CMDB Monitoring Sources Copyright © 2010 Opscode, Inc - All Rights Reserved 9 Thursday, July 29, 2010
  • 13. Fully Automated Infrastructure Control Provisioning Release Orchestration Deploy CI Server Issue tracker Artifact Config repository Dispatcher SCM management Build Repository OS boot/ Scheduler install Code Workflows Model Resources Configuration Events Trending Asset Identity inventory Trending Reporting Topology Host naming CMDB Monitoring Sources Copyright © 2010 Opscode, Inc - All Rights Reserved 9 Thursday, July 29, 2010
  • 14. Network Operations Systems Administrators Software Developers Database Administrators Storage Management Project Management Change Management Continuity Planning Risk Management Web Design Performance Compliance ( ) Architecture * in no Tooling particular Testing order Security Reporting Facilities Thursday, July 29, 2010
  • 15. Thursday, July 29, 2010 SNAFU Copyright © 2010 Opscode, Inc - All Rights Reserved 11
  • 16. Cloudy Monitoring Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 17. Cloudy Monitoring Performance Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 18. Cloudy Monitoring Performance Log Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 19. Cloudy Monitoring Performance Log Alerts Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 20. Cloudy Monitoring Performance Log Alerts Event Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 21. Cloudy Monitoring Performance Log Alerts Event Correlation Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 22. Cloudy Monitoring Performance Log Alerts Event Correlation Capacity Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 23. Cloudy Monitoring Performance Log Alerts Event Correlation Capacity Analytics Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 24. Cloudy Monitoring Nagios Performance Log Alerts Event Correlation Capacity Analytics Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 25. Cloudy Monitoring Nagios Performance Collectd Log Alerts Event Correlation Capacity Analytics Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 26. Cloudy Monitoring Nagios Performance Collectd Log jcollectd Alerts Event Correlation Capacity Analytics Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 27. Cloudy Monitoring Nagios Performance Collectd Log jcollectd Alerts Ganglia Event Correlation Capacity Analytics Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 28. Cloudy Monitoring Nagios Performance Collectd Log jcollectd Alerts Ganglia Event Zenoss Correlation Capacity Analytics Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 29. Cloudy Monitoring Nagios Performance Collectd Log jcollectd Alerts Ganglia Event Zenoss Correlation JMX Capacity Analytics Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 30. Cloudy Monitoring Nagios Performance Collectd Log jcollectd Alerts Ganglia Event Zenoss Correlation JMX Capacity OpenNMS Analytics Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 31. Cloudy Monitoring Nagios Performance Collectd Log jcollectd Alerts Ganglia Event Zenoss Correlation JMX Capacity OpenNMS Analytics Munin Copyright © 2010 Opscode, Inc - All Rights Reserved 12 Thursday, July 29, 2010
  • 32. Cloudy Provisioning Copyright © 2010 Opscode, Inc - All Rights Reserved 13 Thursday, July 29, 2010
  • 33. Cloudy Provisioning Provisioning Copyright © 2010 Opscode, Inc - All Rights Reserved 13 Thursday, July 29, 2010
  • 34. Cloudy Provisioning Provisioning Configuration Copyright © 2010 Opscode, Inc - All Rights Reserved 13 Thursday, July 29, 2010
  • 35. Cloudy Provisioning Provisioning Configuration Systems Integration Copyright © 2010 Opscode, Inc - All Rights Reserved 13 Thursday, July 29, 2010
  • 36. Provisioning Nodes opslb01 opsws01 opsws02 opsdm01 opsds01 opsds02 Copyright © 2010 Opscode, Inc - All Rights Reserved 14 Thursday, July 29, 2010
  • 37. Configuration Management Roles loadbalancer webserver dbmaster dbslave Copyright © 2010 Opscode, Inc - All Rights Reserved 15 Thursday, July 29, 2010
  • 38. Systems Integration Load Balancer Recipes haproxy Web Server Web Server apache2 myssql DB Master DB Slave DB Slave Disk Disk Disk Copyright © 2010 Opscode, Inc - All Rights Reserved 16 Thursday, July 29, 2010
  • 39. Role Based Configuration name "webserver" description "Systems that serve HTTP traffic" run_list( "role[base]", "recipe[apache2]", "recipe[apache2::mod_ssl]" ) default_attributes( "apache" => { "listen_ports" => [ "80", "443" ] } ) override_attributes( "apache" => { "max_children" => "50" } ) Copyright © 2010 Opscode, Inc - All Rights Reserved 17 Thursday, July 29, 2010
  • 40. Load Balancer Example Copyright © 2010 Opscode, Inc - All Rights Reserved 18 Thursday, July 29, 2010
  • 41. Systems Integration Load Balancer Web Server Web Server DB Master DB Slave DB Slave Disk Disk Disk Copyright © 2010 Opscode, Inc - All Rights Reserved 19 Thursday, July 29, 2010
  • 42. Devops Copyright © 2010 Opscode, Inc - All Rights Reserved 20 Thursday, July 29, 2010
  • 43. Devops • Culture Copyright © 2010 Opscode, Inc - All Rights Reserved 20 Thursday, July 29, 2010
  • 44. Devops •Culture • Automation Copyright © 2010 Opscode, Inc - All Rights Reserved 20 Thursday, July 29, 2010
  • 45. Devops •Culture • Automation • Measurement Copyright © 2010 Opscode, Inc - All Rights Reserved 20 Thursday, July 29, 2010
  • 46. Devops •Culture • Automation • Measurement • Sharing Copyright © 2010 Opscode, Inc - All Rights Reserved 20 Thursday, July 29, 2010
  • 47. What Do Developers Need? Copyright © 2010 Opscode, Inc - All Rights Reserved 21 Thursday, July 29, 2010
  • 48. For Developers... Thursday, July 29, 2010
  • 49. For Developers... • Self Service Operations Thursday, July 29, 2010
  • 50. For Developers... • Self Service Operations • The infrastructure is the application (and vice versa) Thursday, July 29, 2010
  • 51. For Developers... • Self Service Operations • The infrastructure is the application (and vice versa) • Minimize Bottlenecks Thursday, July 29, 2010
  • 52. For Developers... • Self Service Operations • The infrastructure is the application (and vice versa) • Minimize Bottlenecks • The “Right” Tools Thursday, July 29, 2010
  • 53. What Does Operations Need? Copyright © 2010 Opscode, Inc - All Rights Reserved 23 Thursday, July 29, 2010
  • 54. Copyright © 2010 Opscode, Inc - All Rights Reserved 24 Thursday, July 29, 2010
  • 55. Operations http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft Thursday, July 29, 2010
  • 56. Operations • Say “Yes”. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft Thursday, July 29, 2010
  • 57. Operations • Say “Yes”. • You never liked rack and stack that much anyway. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft Thursday, July 29, 2010
  • 58. Operations • Say “Yes”. • You never liked rack and stack that much anyway. • You have never been more critical. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft Thursday, July 29, 2010
  • 59. Operations • Say “Yes”. • You never liked rack and stack that much anyway. • You have never been more critical. • Just get out of the way. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft Thursday, July 29, 2010
  • 60. Agile Infrastructure Development Team focus IDE/Workbench Agile methodology Source Control Copyright © 2010 Opscode, Inc - All Rights Reserved 26 Thursday, July 29, 2010
  • 61. Agile Infrastructure Operations Individual focus Script VI based Source control? Waterfall Copyright © 2010 Opscode, Inc - All Rights Reserved 27 Thursday, July 29, 2010
  • 62. Infrastructure as Code http://www.flickr.com/photos/wonderlane/2306082998/ Copyright © 2010 Opscode, Inc - All Rights Reserved 28 Thursday, July 29, 2010
  • 63. Infrastructure as Code is... http://www.flickr.com/photos/kwerfeldein/2634561264/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 29 Thursday, July 29, 2010
  • 64. Infrastructure as Code is... A technical domain revolving around building and managing infrastructure programmatically http://www.flickr.com/photos/kwerfeldein/2634561264/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 29 Thursday, July 29, 2010
  • 65. Enable the reconstruction of the business from nothing but a source code repository, an application data backup, and bare metal resources. Copyright © 2010 Opscode, Inc - All Rights Reserved 30 Thursday, July 29, 2010
  • 66. A Tornado Hits Your Data http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 67. A Tornado Hits Your Data • Pause your movie http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 68. A Tornado Hits Your Data • Pause your movie • Sign into your cloud provider http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 69. A Tornado Hits Your Data • Pause your movie • Sign into your cloud provider • Upload your offsite backups http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 70. A Tornado Hits Your Data • Pause your movie • Sign into your cloud provider • Upload your offsite backups • Provision, config and integrate the new servers http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 71. A Tornado Hits Your Data • Pause your movie • Sign into your cloud provider • Upload your offsite backups • Provision, config and integrate the new servers • Change DNS to point to “Hit by Tornado” page http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 72. A Tornado Hits Your Data • Pause your movie • Sign into your cloud provider • Upload your offsite backups • Provision, config and integrate the new servers • Change DNS to point to “Hit by Tornado” page • Restore the customer and application data http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 73. A Tornado Hits Your Data • Pause your movie • Sign into your cloud provider • Upload your offsite backups • Provision, config and integrate the new servers • Change DNS to point to “Hit by Tornado” page • Restore the customer and application data • Remove the “Hit by Tornado” page http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 74. A Tornado Hits Your Data • Pause your movie • Sign into your cloud provider • Upload your offsite backups • Provision, config and integrate the new servers • Change DNS to point to “Hit by Tornado” page • Restore the customer and application data • Remove the “Hit by Tornado” page • Unpause movie http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 75. A Tornado Hits Your Data • Pause your movie • Sign into your cloud provider • Upload your offsite backups • Provision, config and integrate the new servers • Change DNS to point to “Hit by Tornado” page • Restore the customer and application data • Remove the “Hit by Tornado” page • Unpause movie http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 76. A Tornado Hits Your Data • Pause your movie • Chapter 5 Sign into your cloud provider Infrastructure • Upload your offsite backups as Code • Provision, config and Jacob the new servers Adam integrate • Change DNS to point to “Hit by Tornado” page • Restore the customer and application data • Remove the “Hit by Tornado” page • Unpause movie http://www.flickr.com/photos/gi/518613153/sizes/o/ Thursday, July 29, 2010
  • 77. In a Cloudy World Your Prime Constraint Should Be http://www.flickr.com/photos/visualage/2126833132/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 32 Thursday, July 29, 2010
  • 78. In a Cloudy World Your Prime Constraint Should Be The time it takes to restore your application data http://www.flickr.com/photos/visualage/2126833132/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 32 Thursday, July 29, 2010
  • 79. Infrastructure as Code Copyright © 2010 Opscode, Inc - All Rights Reserved 33 Thursday, July 29, 2010
  • 80. Recipies Applies resources in the order they are specified http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/ Thursday, July 29, 2010
  • 81. Recipies Applies resources in the order they are specified • Can include other recipes. http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/ Thursday, July 29, 2010
  • 82. Recipies Applies resources in the order they are specified • Can include other recipes. • A DSL like Ruby. http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/ Thursday, July 29, 2010
  • 83. Tale of Two Startups “Traditional” Operations Operations - The “Secret Sauce” 50 50 40 40 # of Hours 30 30 20 20 Hardware OS Install 10 10 Config Upkeep 20 20 15 15 Servers 10 10 New 5 5 Existing 0 0 1 2 3 4 5 6 7 9 10 11 12 1 2 3 4 5 6 7 9 10 11 12 Week # Week # (http://radar.oreilly.com/archives/2007/10/operations-advantage.html) Copyright © 2010 Opscode, Inc - All Rights Reserved 35 Thursday, July 29, 2010
  • 84. Tale of Two Startups “Traditional” Operations Operations - The “Secret Sauce” 50 50 This is the secret of 40 40 Cloud Computing. Every other virtue stems from # of Hours 30 30 here. 20 20 Hardware OS Install 10 10 Config Upkeep 20 20 15 15 Servers 10 10 New 5 5 Existing 0 0 1 2 3 4 5 6 7 9 10 11 12 1 2 3 4 5 6 7 9 10 11 12 Week # Week # (http://radar.oreilly.com/archives/2007/10/operations-advantage.html) Copyright © 2010 Opscode, Inc - All Rights Reserved 35 Thursday, July 29, 2010
  • 85. Thursday, July 29, 2010
  • 86. A Period of Combinatorial Innovation Thursday, July 29, 2010
  • 87. A Period of Combinatorial Innovation • Abstract and fault tolerant components Thursday, July 29, 2010
  • 88. A Period of Combinatorial Innovation • Abstract and fault tolerant components • Integrated network accessible services Thursday, July 29, 2010
  • 89. A Period of Combinatorial Innovation • Abstract and fault tolerant components • Integrated network accessible services • Unlimited infrastructure Thursday, July 29, 2010
  • 90. Industry Shifts Be bold-and mighty forces will come to your aid Basil King Copyright © 2010 Opscode, Inc - All Rights Reserved 37 Thursday, July 29, 2010