Using Chef for Automated Infrastructure in the Cloud

  • 34,879 views
Uploaded on

Infrastructure is code – the separation between how you manage your infrastructure and how you build your applications is disappearing. Adam Jacob, CTO of Opscode and primary author of Chef, will …

Infrastructure is code – the separation between how you manage your infrastructure and how you build your applications is disappearing. Adam Jacob, CTO of Opscode and primary author of Chef, will teach you what this means in practice – through showing how to deploy real-world applications with Chef on EC2.

We’ll talk about:

Deploying Web Applications… * Ruby on Rails * Catalyst * Django * Tomcat * PHP
Setting up Database Servers… * MySQL * PostgreSQL
Automating Monitoring and Trending… * Nagios * Munin * Ganglia
Centrally managing your application configuration
Along the way we’ll be talking about best practices in systems automation, quirks about EC2, and talking about how tools like Chef, Nanite, RabbitMQ and CouchDB can make your life easier.

More in: Technology , Business , Travel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • I just download the presentation, file name is automatingthecloudwithchef-090729014217-phpapp01.key. how to use the presentation?
    Are you sure you want to
    Your message goes here
  • its really helpful...
    Are you sure you want to
    Your message goes here
  • I am pleased to see my photo used in this slideshow. The fun of releasing photos with Creative Commons licenses is that they end up in fun places like this. The attribution at the bottom is a bit small, but it seems to have worked, as someone did click through to my flickr page.
    best., robert [gurdonark]
    Are you sure you want to
    Your message goes here
  • awsome chef presentation
    Are you sure you want to
    Your message goes here
  • Awesome!
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
34,879
On Slideshare
0
From Embeds
0
Number of Embeds
8

Actions

Shares
Downloads
1,067
Comments
5
Likes
83

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Automating the Cloud with Chef Adam Jacob Co-Founder & CTO @ Opscode
  • 2. • Who am I, who are you, and why are we here? • The Method • EC2 • Chef • Nanite • The Real World • Q &A http://www.flickr.com/photos/niecieden/367343737/sizes/o/
  • 3. • 13 years as a Systems Administrator • Lots of Mergers and Acquisitions • Consultant • Wrote much of Chef • CTO at Opscode http://www.flickr.com/photos/anotherphotograph/2100904507/sizes/o/
  • 4. http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 5. • Developers? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 6. • Developers? • Systems Administrators? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 7. Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
  • 8. Total Bootstrapping Time in Weeks 8 6 Corp Approvals Agile Approvals 4 Cloud 2 0 Best Time 0 Worst Time Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
  • 9. Total Bootstrapping Time in Weeks of ret g. 8 sec utin the mp tue s is Co r vir . hi ud 6 T o he here ot m Cl Corp Approvals ery s fro Ev mApprovals Agile 4 ste Cloud 2 0 Best Time 0 Worst Time Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
  • 10. For Developers...
  • 11. For Developers... • Do it yourself.
  • 12. For Developers... • Do it yourself. • The infrastructure is the application (and vice versa).
  • 13. For Developers... • Do it yourself. • The infrastructure is the application (and vice versa). • You are not a Systems Administrator.
  • 14. For Developers... • Do it yourself. • The infrastructure is the application (and vice versa). • You are not a Systems Administrator. • You need tools.
  • 15. Sysadmins... http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 16. Sysadmins... • Say “Yes”. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 17. Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 18. Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. • You have never been more critical. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 19. Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. • You have never been more critical. • Lean into it. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 20. The Method http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 21. The Method Bootstrapping http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 22. The Method Bootstrapping http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 23. The Method Bootstrapping Configuration http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 24. The Method Bootstrapping Configuration http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 25. The Method Bootstrapping Configuration Command & Control http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 26. The Method Bootstrapping Configuration Command & Control Nanite! http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 27. Lightning Strikes! DOOM Webservers Database Servers Webservers
  • 28. Lightning Strikes! DOOM X Webservers XX Database Servers Webservers
  • 29. Lightning Strikes! DOOM X 1 1 2 1 Signals Moar! Monitoring System Webservers 5 Updates 2 1 Command & Control XX 3 1 Bootstrapping Provisions 1 3 1 Database Servers 4 1 4 1 Configuration Webservers
  • 30. EC2 Best Practices http://www.flickr.com/photos/46183897@N00/3442880227/sizes/l/
  • 31. AMIs http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 32. AMIs • Amazon Machine Images http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 33. AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 34. AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size • Use pre-existing images to bootstrap http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 35. AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size • Use pre-existing images to bootstrap • Include the Configuration and C&C Tools http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 36. AMIs http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 37. AMIs • Re-bundle for specific roles http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 38. AMIs • Re-bundle for specific roles • Use SSH Keys for access http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 39. AMIs • Re-bundle for specific roles • Use SSH Keys for access • Shut off fsck! http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 40. Instances http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 41. Instances • Actual virtual machines http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 42. Instances • Actual virtual machines • c1.medium is the best bang for your $$ http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 43. Instances • Actual virtual machines • c1.medium is the best bang for your $$ • Use User Data to inform Configuration http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 44. Instances • Actual virtual machines • c1.medium is the best bang for your $$ • Use User Data to inform Configuration • Support indexed User Data - launch more than one at a time. (launch-index) http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 45. Instances http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 46. Instances • Update the default security group to allow SSH http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 47. Instances • Update the default security group to allow SSH • Use SSH Keys for authentication http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 48. Instances • Update the default security group to allow SSH • Use SSH Keys for authentication • Ephemeral Storage is your buddy http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 49. Instances http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 50. Instances • Internal and External hostnames are not unique http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 51. Instances • Internal and External hostnames are not unique • Only trust the instance-id http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 52. EBS http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 53. EBS • Elastic Block Store http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 54. EBS • Elastic Block Store • Truly persistent storage http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 55. EBS • Elastic Block Store • Truly persistent storage • Trivial to snapshot http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 56. EBS • Elastic Block Store • Truly persistent storage • Trivial to snapshot • Snapshots can bootstrap more EBS devices http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 57. Elastic IP Addresses http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 58. Elastic IP Addresses • Static, publicly routable addresses http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 59. Elastic IP Addresses • Static, publicly routable addresses • Fast and easy to re-assign http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 60. http://www.brooklynstreetart.com/theBlog/wp-content/uploads/2008/12/swedish_chef_bork-sleeper-cell.jpg
  • 61. At a High Level... http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 62. At a High Level... • A library for configuration management http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 63. At a High Level... • A library for configuration management • A configuration management system http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 64. At a High Level... • A library for configuration management • A configuration management system • A systems integration platform http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 65. At a High Level... • A library for configuration management • A configuration management system • A systems integration platform • An API for your entire Infrastructure http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 66. http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 67. Principles • Idempotent http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 68. Principles • Idempotent • Reasonability http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 69. Principles • Idempotent • Reasonability • Sane defaults, easily changed http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 70. Principles • Idempotent • Reasonability • Sane defaults, easily changed • Hackability http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 71. Principles • Idempotent • Reasonability • Sane defaults, easily changed • Hackability • TMTOWTDI http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 72. Infrastructure as Code Manage configuration as idempotent Resources. Put them together in Recipes. Track it like source code. Configure your servers. You can learn more about Chef at http://wiki.opscode.com/display/chef/Home
  • 73. Automating Sudo Permissions Write the recipe. http://xkcd.com/149/
  • 74. Automating Sudo Permissions Write the recipe. • Install the package. http://xkcd.com/149/
  • 75. Automating Sudo Permissions Write the recipe. • Install the package. • Write out the sudoers file. http://xkcd.com/149/
  • 76. Automating Sudo Permissions Write the recipe. • Install the package. • Write out the sudoers file. • Use custom attributes for users and groups. http://xkcd.com/149/
  • 77. Automating Sudo Permissions Write the sudoers template http://xkcd.com/149/
  • 78. Automating Sudo Permissions Write the sudoers template • Add a warning banner. http://xkcd.com/149/
  • 79. Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. http://xkcd.com/149/
  • 80. Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. • Add the node-specific users. http://xkcd.com/149/
  • 81. Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. • Add the node-specific users. http://xkcd.com/149/ • Add the node-specific groups.
  • 82. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. http://xkcd.com/149/
  • 83. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. http://xkcd.com/149/
  • 84. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. http://xkcd.com/149/
  • 85. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. http://xkcd.com/149/
  • 86. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. • Add default users. http://xkcd.com/149/
  • 87. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. • Add default users. http://xkcd.com/149/ • Add default groups.
  • 88. Automating Sudo Permissions Put it all together!
  • 89. Automating Sudo Permissions Put it all together! • Assign your role to nodes.
  • 90. Automating Sudo Permissions Put it all together! • Assign your role to nodes. • When Chef runs, the sudoers file will be populated.
  • 91. Nanite http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 92. Nanite • “Self Assembling Cluster of Ruby Daemons” http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 93. Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 94. Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ • Actors register Services http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 95. Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ • Actors register Services • Distributed Map/Reduce for your Infrastructure http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 96. Nanite Architecture Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard - these slides pinched from http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 97. A Nanite Actor... Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 98. A Nanite Actor... • Advertises Services Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 99. A Nanite Actor... • Advertises Services • Advertises Tags Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 100. A Nanite Actor... • Advertises Services • Advertises Tags • Requests can route to Services and Tags Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 101. Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 102. Lightning Strikes, Monitoring /node/down Service Revisited! Signals Nanite 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 103. Lightning Strikes, Nanite boots new EC2 Nanite removes Revisited! Instances, with Chef Role + Attribute Data nodes in Chef 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 104. Lightning Strikes, Revisited! Provisions Instances, EBS, Elastic IPs 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 105. Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration Chef configures nodes according to assigned Roles
  • 106. Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 Chef updates the 1 3 1 5 1 3 1 monitoring system 4 1 4 1 Configuration
  • 107. In the Real World http://www.flickr.com/photos/kenlund/3376784956/sizes/l/
  • 108. A Simple Architecture Load Balancing Web Servers Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 109. A Simple Architecture AWS LB HAProxy S->M Load Balancing Web Servers Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 110. AWS LB A Simple Architecture HAProxy S->M Load Balancing Your App Stack M Web Servers EBS? Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 111. AWS LB A Simple Architecture HAProxy S->M Load Balancing Your App Stack M Web Servers EBS? Master/ Slave M->L->XL Databases EBS http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 112. Load Balancing http://www.flickr.com/photos/jannem/497840412/
  • 113. Load Balancing • Amazon offers Load Balancing http://www.flickr.com/photos/jannem/497840412/
  • 114. Load Balancing • Amazon offers Load Balancing • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
  • 115. Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
  • 116. Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing Configure Pools, Associate Backends • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
  • 117. Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing Configure Pools, Associate Backends • Alternative is HAProxy + Elastic IP Reconfigure based on Load, STONITH http://www.flickr.com/photos/jannem/497840412/
  • 118. Web Frameworks In General.. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 119. Web Frameworks In General.. • Try and keep things ephemeral http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 120. Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 121. Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 122. Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 123. Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 124. Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront Configuration, Deployment, • Use a shared session Discovery storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 125. Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront Configuration, Deployment, • Use a shared session Discovery storage - preferably cookie or memcached based. Trigger Deployment, Maintenance, Migrations • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 126. Ruby on Rails http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 127. Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 128. Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 129. Ruby on Rails Deploys Radiant CMS • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 130. Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin Configures Apache http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 131. Catalyst
  • 132. • Use Chef for CPAN Dependencies Catalyst
  • 133. • Use Chef for CPAN Dependencies • Follow the Chef Deploy layout Catalyst
  • 134. • Use Chef for CPAN Dependencies • Follow the Chef Deploy layout • Use Catalyst::Plugin::Session::Store::Memcached Catalyst
  • 135. http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 136. • Use Chef for Egg Dependencies http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 137. • Use Chef for Egg Dependencies • Use Chef Deploy layout http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 138. • Use Chef for Egg Dependencies • Use Chef Deploy layout • Memcached Cache Backend http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 139. • Use Chef for Egg Dependencies • Use Chef Deploy layout • Memcached Cache Backend • Use the Cache Session Backend http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 140. Databases In General.. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 141. Databases In General.. • Keep everything on EBS http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 142. Databases In General.. • Keep everything on EBS • Snapshot frequently http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 143. Databases In General.. • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 144. Databases In General.. • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 145. Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 146. Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS Configuration, Tuning, • Snapshot frequently User Credentials, • Keep a rolling backlog of Discovery snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 147. Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS Configuration, Tuning, • Snapshot frequently User Credentials, • Keep a rolling backlog of Discovery snapshots Maintenance, Migrations, • Register databases with Analytics Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 148. MySQL http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 149. MySQL • Always have a Slave http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 150. MySQL • Always have a Slave • EBS is neat, but a crash is a crash - and recovery is time consuming http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 151. MySQL • Always have a Slave • EBS is neat, but a crash is a crash - and recovery is time consuming • Building slaves is trivial http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 152. PostgreSQL http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 153. PostgreSQL • Replication choices make things harder - PGPool-II appears to be most common on EC2 http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 154. PostgreSQL • Replication choices make things harder - PGPool-II appears to be most common on EC2 • Hot standby with WAL Shipping http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 155. PostgreSQL Are you great at this on EC2? • Replication choices make things harder - PGPool-II appears to be most common on EC2 • Hot standby with WAL Shipping http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 156. Monitoring & Trending In General.. http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 157. Monitoring & Trending In General.. • Must be integrated with Command & Control http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 158. Monitoring & Trending In General.. • Must be integrated with Command & Control • Use search features in Chef to dynamically configure http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 159. Monitoring & Trending In General.. • Must be integrated with Command & Control • Use search features in Chef to dynamically configure • Make sure you monitor internally and externally http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 160. Nagios http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 161. Nagios • Do all that crazy config you have been avoiding http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 162. Nagios • Do all that crazy config you have been avoiding • Remarkably easy to automate http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 163. A word about Load...
  • 164. Typical Peak Load 1. Bring on capacity as traffic ramps up 2. Take down capacity as it ramps down 3. 10-15 Minutes on either side, fully unattended Graphs in this portion of the presentation taken from Theo Schlossnagle http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
  • 165. Atypical Load No way However, around you are Capacity still better Planning off! 1. Hope you know it is coming. 2. Increase capacity in advance. 3. Take down capacity as it ramps down. Graphs in this portion of the presentation taken from Theo Schlossnagle http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
  • 166. Round Up! http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 167. Round Up! Bring new resources online http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 168. Round Up! Bring new resources online Configure and manage them http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 169. Round Up! Bring new resources online Configure and manage them Make the whole thing sing and dance http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 170. http://www.flickr.com/photos/jackol/1766679527/sizes/l/ Q &A Please rate this talk! There is lots more to learn Email: adam@opscode.com about Chef at Twitter: adamhjk http://wiki.opscode.com IRC: irc.freenode.net #chef