SlideShare a Scribd company logo
Automating the Cloud
     with Chef
           Adam Jacob
   Co-Founder & CTO @ Opscode
•   Who am I, who are you,
    and why are we here?

•   The Method

•   EC2

•   Chef

•   Nanite

•   The Real World

•   Q &A

                       http://www.flickr.com/photos/niecieden/367343737/sizes/o/
• 13 years as a Systems
  Administrator

• Lots of Mergers and
  Acquisitions

• Consultant
• Wrote much of Chef
• CTO at Opscode
         http://www.flickr.com/photos/anotherphotograph/2100904507/sizes/o/
http://www.flickr.com/photos/timyates/2854357446/sizes/l/
• Developers?




                http://www.flickr.com/photos/timyates/2854357446/sizes/l/
• Developers?
• Systems Administrators?


                            http://www.flickr.com/photos/timyates/2854357446/sizes/l/
Why are we here?
              http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
Total Bootstrapping Time in Weeks


                     8


 6
                                       Corp Approvals
                                       Agile Approvals
                          4            Cloud

        2


                 0
     Best Time                     0
                      Worst Time




     Why are we here?
                                              http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
Total Bootstrapping Time in Weeks

                                                          of
                                                      ret g.
                     8                             sec utin
                                               the mp tue
                                          s is Co r vir .
                                        hi ud
 6                                     T o          he here
                                                  ot m
                                         Cl Corp Approvals
                                              ery s fro
                                           Ev mApprovals
                                             Agile
                          4
                                              ste
                                             Cloud

        2


                 0
     Best Time                     0
                      Worst Time




     Why are we here?
                                                  http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
For Developers...
For Developers...

• Do it yourself.
For Developers...

• Do it yourself.
• The infrastructure is the application (and
  vice versa).
For Developers...

• Do it yourself.
• The infrastructure is the application (and
  vice versa).
• You are not a Systems Administrator.
For Developers...

• Do it yourself.
• The infrastructure is the application (and
  vice versa).
• You are not a Systems Administrator.
• You need tools.
Sysadmins...




                                                                                                 http://covers.oreilly.com/images/9780596007836/lrg.jpg
Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
Sysadmins...
 • Say “Yes”.




                                                                                                 http://covers.oreilly.com/images/9780596007836/lrg.jpg
Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
Sysadmins...
 • Say “Yes”.
 • You never liked rack
          and stack that much
          anyway.




                                                                                                 http://covers.oreilly.com/images/9780596007836/lrg.jpg
Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
Sysadmins...
 • Say “Yes”.
 • You never liked rack
          and stack that much
          anyway.
 • You have never been
          more critical.


                                                                                                 http://covers.oreilly.com/images/9780596007836/lrg.jpg
Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
Sysadmins...
 • Say “Yes”.
 • You never liked rack
          and stack that much
          anyway.
 • You have never been
          more critical.
 • Lean into it.
                                                                                                 http://covers.oreilly.com/images/9780596007836/lrg.jpg
Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
The Method




  http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method
Bootstrapping




  http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method
Bootstrapping




  http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method
Bootstrapping

Configuration



  http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method
Bootstrapping

Configuration



  http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method
  Bootstrapping

   Configuration

Command & Control
     http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method
  Bootstrapping

   Configuration

Command & Control
                   Nanite!
     http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
Lightning Strikes!
            DOOM


    Webservers




                 Database Servers




    Webservers
Lightning Strikes!
            DOOM

      X
    Webservers    XX
                 Database Servers




    Webservers
Lightning Strikes!
                         DOOM

               X     1
                     1            2
                                  1
               Signals          Moar!

 Monitoring
  System Webservers

    5
            Updates
                 2
                 1
                    Command &
                      Control
                                 XX     3
                                        1
                                            Bootstrapping
                                               Provisions

    1                                                3
                                                     1


                                Database Servers

                     4
                     1            4
                                  1
Configuration


          Webservers
EC2 Best Practices


    http://www.flickr.com/photos/46183897@N00/3442880227/sizes/l/
AMIs




http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs

• Amazon Machine Images




            http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs

• Amazon Machine Images
• Have one AMI with JEOS for each instance
  size




              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs

• Amazon Machine Images
• Have one AMI with JEOS for each instance
  size
 • Use pre-existing images to bootstrap

              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs

• Amazon Machine Images
• Have one AMI with JEOS for each instance
  size
 • Use pre-existing images to bootstrap
• Include the Configuration and C&C Tools
              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs




http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs

• Re-bundle for specific roles



              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs

• Re-bundle for specific roles
• Use SSH Keys for access


              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs

• Re-bundle for specific roles
• Use SSH Keys for access
• Shut off fsck!

              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances




http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances

• Actual virtual machines




              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances

• Actual virtual machines
• c1.medium is the best bang for your $$



              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances

• Actual virtual machines
• c1.medium is the best bang for your $$
• Use User Data to inform Configuration


              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances

• Actual virtual machines
• c1.medium is the best bang for your $$
• Use User Data to inform Configuration
• Support indexed User Data - launch more
  than one at a time. (launch-index)


              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances




http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances

• Update the default security group to allow
  SSH




              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances

• Update the default security group to allow
  SSH
• Use SSH Keys for authentication


              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances

• Update the default security group to allow
  SSH
• Use SSH Keys for authentication
• Ephemeral Storage is your buddy

              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances




http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances


• Internal and External hostnames are not
  unique




              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances


• Internal and External hostnames are not
  unique
• Only trust the instance-id

               http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
EBS




http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
EBS

• Elastic Block Store




               http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
EBS

• Elastic Block Store
• Truly persistent storage



               http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
EBS

• Elastic Block Store
• Truly persistent storage
• Trivial to snapshot


               http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
EBS

• Elastic Block Store
• Truly persistent storage
• Trivial to snapshot
• Snapshots can bootstrap more EBS devices

              http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Elastic IP Addresses




      http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Elastic IP Addresses


• Static, publicly routable addresses



               http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Elastic IP Addresses


• Static, publicly routable addresses
• Fast and easy to re-assign


               http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
http://www.brooklynstreetart.com/theBlog/wp-content/uploads/2008/12/swedish_chef_bork-sleeper-cell.jpg
At a High Level...




                 http://www.flickr.com/photos/asten/2159525309/sizes/l/
At a High Level...

• A library for configuration management




                                  http://www.flickr.com/photos/asten/2159525309/sizes/l/
At a High Level...

• A library for configuration management
• A configuration management system



                                  http://www.flickr.com/photos/asten/2159525309/sizes/l/
At a High Level...

• A library for configuration management
• A configuration management system
• A systems integration platform


                                  http://www.flickr.com/photos/asten/2159525309/sizes/l/
At a High Level...

• A library for configuration management
• A configuration management system
• A systems integration platform
• An API for your entire Infrastructure
                                  http://www.flickr.com/photos/asten/2159525309/sizes/l/
http://www.flickr.com/photos/gi/518613153/sizes/o/
Principles

• Idempotent




               http://www.flickr.com/photos/gi/518613153/sizes/o/
Principles

• Idempotent
• Reasonability



                  http://www.flickr.com/photos/gi/518613153/sizes/o/
Principles

• Idempotent
• Reasonability
• Sane defaults, easily changed


                http://www.flickr.com/photos/gi/518613153/sizes/o/
Principles

• Idempotent
• Reasonability
• Sane defaults, easily changed
• Hackability

                http://www.flickr.com/photos/gi/518613153/sizes/o/
Principles

• Idempotent
• Reasonability
• Sane defaults, easily changed
• Hackability
• TMTOWTDI
                http://www.flickr.com/photos/gi/518613153/sizes/o/
Infrastructure as Code



Manage configuration as idempotent Resources.
        Put them together in Recipes.
          Track it like source code.
           Configure your servers.
  You can learn more about Chef at http://wiki.opscode.com/display/chef/Home
Automating Sudo
  Permissions
     Write the recipe.




        http://xkcd.com/149/
Automating Sudo
  Permissions
     Write the recipe.


                               •   Install the package.




        http://xkcd.com/149/
Automating Sudo
  Permissions
     Write the recipe.


                               •   Install the package.

                               •   Write out the sudoers file.



        http://xkcd.com/149/
Automating Sudo
  Permissions
     Write the recipe.


                               •   Install the package.

                               •   Write out the sudoers file.

                               •   Use custom attributes for
                                   users and groups.
        http://xkcd.com/149/
Automating Sudo
  Permissions
  Write the sudoers template




            http://xkcd.com/149/
Automating Sudo
                Permissions
                      Write the sudoers template

•   Add a warning banner.




                                http://xkcd.com/149/
Automating Sudo
                Permissions
                      Write the sudoers template

•   Add a warning banner.

•   Make sure root always
    has access.



                                http://xkcd.com/149/
Automating Sudo
                Permissions
                      Write the sudoers template

•   Add a warning banner.

•   Make sure root always
    has access.

•   Add the node-specific
    users.
                                http://xkcd.com/149/
Automating Sudo
                Permissions
                      Write the sudoers template

•   Add a warning banner.

•   Make sure root always
    has access.

•   Add the node-specific
    users.
                                http://xkcd.com/149/


•   Add the node-specific
    groups.
Automating Sudo
  Permissions
   Create a role that installs
    sudo, and sets defaults.




             http://xkcd.com/149/
Automating Sudo
                 Permissions
                      Create a role that installs
                       sudo, and sets defaults.

•   Give it a name.




                                http://xkcd.com/149/
Automating Sudo
                 Permissions
                           Create a role that installs
                            sudo, and sets defaults.

•   Give it a name.

•   A short description.



                                     http://xkcd.com/149/
Automating Sudo
                 Permissions
                           Create a role that installs
                            sudo, and sets defaults.

•   Give it a name.

•   A short description.

•   Add the sudo recipe.

                                     http://xkcd.com/149/
Automating Sudo
                 Permissions
                           Create a role that installs
                            sudo, and sets defaults.

•   Give it a name.

•   A short description.

•   Add the sudo recipe.

•   Add default users.               http://xkcd.com/149/
Automating Sudo
                 Permissions
                           Create a role that installs
                            sudo, and sets defaults.

•   Give it a name.

•   A short description.

•   Add the sudo recipe.

•   Add default users.               http://xkcd.com/149/




•   Add default groups.
Automating Sudo
  Permissions
    Put it all together!
Automating Sudo
  Permissions
    Put it all together!

                           •   Assign your role to
                               nodes.
Automating Sudo
  Permissions
    Put it all together!

                           •   Assign your role to
                               nodes.

                           •   When Chef runs, the
                               sudoers file will be
                               populated.
Nanite




http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Nanite

• “Self Assembling Cluster of Ruby Daemons”




                http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Nanite

• “Self Assembling Cluster of Ruby Daemons”
• AMQP - RabbitMQ




                http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Nanite

• “Self Assembling Cluster of Ruby Daemons”
• AMQP - RabbitMQ
• Actors register Services



                http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Nanite

• “Self Assembling Cluster of Ruby Daemons”
• AMQP - RabbitMQ
• Actors register Services
• Distributed Map/Reduce for your
  Infrastructure



                http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Nanite Architecture




Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard - these slides pinched from
                   http://www.slideshare.net/ezmobius/erlangfactory
                               http://www.flickr.com/photos/etherhill/182345209/sizes/l/
A Nanite Actor...




Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard
     http://www.slideshare.net/ezmobius/erlangfactory
                  http://www.flickr.com/photos/etherhill/182345209/sizes/l/
A Nanite Actor...

                                                                  •          Advertises Services




Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard
     http://www.slideshare.net/ezmobius/erlangfactory
                  http://www.flickr.com/photos/etherhill/182345209/sizes/l/
A Nanite Actor...

                                                                  •          Advertises Services

                                                                  •          Advertises Tags




Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard
     http://www.slideshare.net/ezmobius/erlangfactory
                  http://www.flickr.com/photos/etherhill/182345209/sizes/l/
A Nanite Actor...

                                                                  •          Advertises Services

                                                                  •          Advertises Tags

                                                                  •          Requests can route to
                                                                             Services and Tags




Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard
     http://www.slideshare.net/ezmobius/erlangfactory
                  http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Lightning Strikes,
    Revisited!
                     1
                     1                 2
                                       1
               Signals               Moar!

 Monitoring              Command &               Bootstrapping
  System      Updates      Control
                                                    Provisions
                 2
                 1
                                             3
                                             1
    5
    1                                                     3
                                                          1




                     4
                     1                 4
                                       1
Configuration
Lightning Strikes,
  Monitoring

 /node/down
    Service
             Revisited!
Signals Nanite




                               1
                               1                 2
                                                 1
                         Signals               Moar!

           Monitoring              Command &               Bootstrapping
            System      Updates      Control
                                                              Provisions
                           2
                           1
                                                       3
                                                       1
                 5
                 1                                                  3
                                                                    1




                               4
                               1                 4
                                                 1
          Configuration
Lightning Strikes,                   Nanite
                                            boots new EC2


Nanite removes
                Revisited!                Instances, with Chef
                                            Role + Attribute
                                                  Data
 nodes in Chef

                                     1
                                     1                 2
                                                       1
                               Signals               Moar!

                 Monitoring              Command &               Bootstrapping
                  System      Updates      Control
                                                                    Provisions
                                 2
                                 1
                                                             3
                                                             1
                    5
                    1                                                     3
                                                                          1




                                     4
                                     1                 4
                                                       1
             Configuration
Lightning Strikes,
    Revisited!                                                  Provisions
                                                              Instances, EBS,
                                                                Elastic IPs
                     1
                     1                 2
                                       1
               Signals               Moar!

 Monitoring              Command &               Bootstrapping
  System      Updates      Control
                                                    Provisions
                 2
                 1
                                             3
                                             1
    5
    1                                                     3
                                                          1




                     4
                     1                 4
                                       1
Configuration
Lightning Strikes,
                 Revisited!
                                      1
                                      1                 2
                                                        1
                                Signals               Moar!

                  Monitoring              Command &               Bootstrapping
                   System      Updates      Control
                                                                     Provisions
                                  2
                                  1
                                                              3
                                                              1
                     5
                     1                                                     3
                                                                           1




                                      4
                                      1                 4
                                                        1
              Configuration
      Chef
configures nodes
  according to
 assigned Roles
Lightning Strikes,
                   Revisited!
                                        1
                                        1                 2
                                                          1
                                  Signals               Moar!

                    Monitoring              Command &               Bootstrapping
                     System      Updates      Control
                                                                       Provisions
                                    2
Chef updates the
                                    1
                                                                3
                                                                1
                       5
                       1                                                     3
                                                                             1
monitoring system

                                        4
                                        1                 4
                                                          1
                Configuration
In the Real World




          http://www.flickr.com/photos/kenlund/3376784956/sizes/l/
A Simple Architecture
                                                  Load Balancing


                                                      Web Servers




                                                           Databases


       http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
A Simple Architecture
AWS LB
HAProxy
 S->M

                                                       Load Balancing


                                                           Web Servers




                                                                Databases


            http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
AWS LB
      A Simple Architecture
HAProxy
 S->M

                                                        Load Balancing
  Your
App Stack
   M
                                                            Web Servers
  EBS?




                                                                 Databases


             http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
AWS LB
      A Simple Architecture
HAProxy
 S->M

                                                        Load Balancing
  Your
App Stack
   M
                                                            Web Servers
  EBS?


 Master/
  Slave
M->L->XL                                                         Databases
  EBS


             http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
Load Balancing




    http://www.flickr.com/photos/jannem/497840412/
Load Balancing


• Amazon offers Load Balancing



               http://www.flickr.com/photos/jannem/497840412/
Load Balancing


• Amazon offers Load Balancing
• Alternative is HAProxy + Elastic IP


                 http://www.flickr.com/photos/jannem/497840412/
Load Balancing

             (De)Provision Load Balancers


• Amazon offers Load Balancing
• Alternative is HAProxy + Elastic IP


                 http://www.flickr.com/photos/jannem/497840412/
Load Balancing

              (De)Provision Load Balancers


• Amazon offers Load Balancing
               Configure Pools, Associate
                      Backends
• Alternative is HAProxy + Elastic IP


                  http://www.flickr.com/photos/jannem/497840412/
Load Balancing

              (De)Provision Load Balancers


• Amazon offers Load Balancing
               Configure Pools, Associate
                      Backends
• Alternative is HAProxy + Elastic IP
               Reconfigure based on Load,
                      STONITH


                  http://www.flickr.com/photos/jannem/497840412/
Web Frameworks
  In General..




             http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks
              In General..
•   Try and keep things
    ephemeral




                          http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks
              In General..
•   Try and keep things
    ephemeral

•   File uploads belong in S3
    or Cloudfront




                                http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks
              In General..
•   Try and keep things
    ephemeral

•   File uploads belong in S3
    or Cloudfront

•   Use a shared session
    storage - preferably
    cookie or memcached
    based.



                                http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks
              In General..
•   Try and keep things
    ephemeral

•   File uploads belong in S3
    or Cloudfront

•   Use a shared session
    storage - preferably
    cookie or memcached
    based.

•   Chef Deploy

                                http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks
              In General..
•   Try and keep things
    ephemeral                   (De)Provision Systems
•   File uploads belong in S3
    or Cloudfront

•   Use a shared session
    storage - preferably
    cookie or memcached
    based.

•   Chef Deploy

                                                        http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks
              In General..
•   Try and keep things
    ephemeral                   (De)Provision Systems
•   File uploads belong in S3
    or Cloudfront
                         Configuration, Deployment,
•   Use a shared session        Discovery
    storage - preferably
    cookie or memcached
    based.

•   Chef Deploy

                                                        http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks
              In General..
•   Try and keep things
    ephemeral                   (De)Provision Systems
•   File uploads belong in S3
    or Cloudfront
                         Configuration, Deployment,
•   Use a shared session        Discovery
    storage - preferably
    cookie or memcached
    based.                   Trigger Deployment,
                            Maintenance, Migrations
•   Chef Deploy

                                                        http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Ruby on Rails




   http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
Ruby on Rails


• Install Gem Dependencies with Chef
• Passenger, Mongrel, or Thin


              http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
Ruby on Rails


• Install Gem Dependencies with Chef
• Passenger, Mongrel, or Thin


              http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
Ruby on Rails                                                      Deploys
                                                                         Radiant CMS




• Install Gem Dependencies with Chef
• Passenger, Mongrel, or Thin


              http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
Ruby on Rails


• Install Gem Dependencies with Chef
• Passenger, Mongrel, or Thin        Configures
                                      Apache




               http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
Catalyst
• Use Chef for CPAN Dependencies




             Catalyst
• Use Chef for CPAN Dependencies
• Follow the Chef Deploy layout



             Catalyst
• Use Chef for CPAN Dependencies
• Follow the Chef Deploy layout
• Use
  Catalyst::Plugin::Session::Store::Memcached




              Catalyst
http://www.flickr.com/photos/hoerner_brett/2901426375/
• Use Chef for Egg Dependencies




             http://www.flickr.com/photos/hoerner_brett/2901426375/
• Use Chef for Egg Dependencies
• Use Chef Deploy layout




             http://www.flickr.com/photos/hoerner_brett/2901426375/
• Use Chef for Egg Dependencies
 • Use Chef Deploy layout




• Memcached Cache Backend
              http://www.flickr.com/photos/hoerner_brett/2901426375/
• Use Chef for Egg Dependencies
 • Use Chef Deploy layout




• Memcached Cache Backend
• Use the Cache Session Backend
              http://www.flickr.com/photos/hoerner_brett/2901426375/
Databases
In General..




               http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases
                    In General..
•   Keep everything on EBS




                                   http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases
                     In General..
•   Keep everything on EBS

•   Snapshot frequently




                                    http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases
                      In General..
•   Keep everything on EBS

•   Snapshot frequently

•   Keep a rolling backlog of
    snapshots




                                     http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases
                      In General..
•   Keep everything on EBS

•   Snapshot frequently

•   Keep a rolling backlog of
    snapshots

•   Register databases with
    Chef in ways that are
    easily discoverable


                                     http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases
                      In General..
                         (De)Provision Systems,
                                Build and Assign EBS
•   Keep everything on EBS

•   Snapshot frequently

•   Keep a rolling backlog of
    snapshots

•   Register databases with
    Chef in ways that are
    easily discoverable


                                                       http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases
                     In General..
                        (De)Provision Systems,
                              Build and Assign EBS
•   Keep everything on EBS
                              Configuration, Tuning,
•   Snapshot frequently
                               User Credentials,
•   Keep a rolling backlog of     Discovery
    snapshots

•   Register databases with
    Chef in ways that are
    easily discoverable


                                                      http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases
                      In General..
                         (De)Provision Systems,
                             Build and Assign EBS
•   Keep everything on EBS
                              Configuration, Tuning,
•   Snapshot frequently
                               User Credentials,
•   Keep a rolling backlog of     Discovery
    snapshots
                            Maintenance, Migrations,
•   Register databases with        Analytics
    Chef in ways that are
    easily discoverable


                                                       http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
MySQL




        http://www.flickr.com/photos/bike/2380021517/sizes/l/
MySQL

• Always have a Slave




                        http://www.flickr.com/photos/bike/2380021517/sizes/l/
MySQL

• Always have a Slave
• EBS is neat, but a crash is a crash - and
  recovery is time consuming




                                       http://www.flickr.com/photos/bike/2380021517/sizes/l/
MySQL

• Always have a Slave
• EBS is neat, but a crash is a crash - and
  recovery is time consuming
• Building slaves is trivial

                                       http://www.flickr.com/photos/bike/2380021517/sizes/l/
PostgreSQL




             http://www.flickr.com/photos/jimgris/531515485/sizes/l/
PostgreSQL

• Replication choices make things harder -
  PGPool-II appears to be most common on
  EC2




                                      http://www.flickr.com/photos/jimgris/531515485/sizes/l/
PostgreSQL

• Replication choices make things harder -
  PGPool-II appears to be most common on
  EC2
• Hot standby with WAL Shipping

                                      http://www.flickr.com/photos/jimgris/531515485/sizes/l/
PostgreSQL
                             Are you great at
                              this on EC2?


• Replication choices make things harder -
  PGPool-II appears to be most common on
  EC2
• Hot standby with WAL Shipping

                                         http://www.flickr.com/photos/jimgris/531515485/sizes/l/
Monitoring & Trending
    In General..




                 http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
Monitoring & Trending
    In General..

•   Must be integrated with Command & Control




                                                http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
Monitoring & Trending
    In General..

•   Must be integrated with Command & Control

•   Use search features in Chef to dynamically configure




                                                 http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
Monitoring & Trending
    In General..

•   Must be integrated with Command & Control

•   Use search features in Chef to dynamically configure

•   Make sure you monitor internally and externally




                                                 http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
Nagios




         http://www.flickr.com/photos/bike/2380021517/sizes/l/
Nagios


• Do all that crazy config you have been
  avoiding




                                    http://www.flickr.com/photos/bike/2380021517/sizes/l/
Nagios


• Do all that crazy config you have been
  avoiding
• Remarkably easy to automate

                                    http://www.flickr.com/photos/bike/2380021517/sizes/l/
A word about Load...
Typical Peak Load


1.       Bring on capacity as traffic ramps up
2.       Take down capacity as it ramps down
3.         10-15 Minutes on either side, fully
                    unattended


     Graphs in this portion of the presentation taken from Theo Schlossnagle
         http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
Atypical Load
No way                                                                  However,
around                                                                    you are
Capacity                                                                still better
Planning                                                                     off!

   1.           Hope you know it is coming.
   2.           Increase capacity in advance.
   3.       Take down capacity as it ramps down.
        Graphs in this portion of the presentation taken from Theo Schlossnagle
            http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
Round Up!




  http://www.flickr.com/photos/benimoto/860423498/sizes/l/
Round Up!
 Bring new resources online




   http://www.flickr.com/photos/benimoto/860423498/sizes/l/
Round Up!
 Bring new resources online


 Configure and manage them




   http://www.flickr.com/photos/benimoto/860423498/sizes/l/
Round Up!
 Bring new resources online


 Configure and manage them


 Make the whole thing sing
        and dance


   http://www.flickr.com/photos/benimoto/860423498/sizes/l/
http://www.flickr.com/photos/jackol/1766679527/sizes/l/




     Q &A                        Please rate this talk!

There is lots more to learn   Email: adam@opscode.com
       about Chef at                Twitter: adamhjk
 http://wiki.opscode.com      IRC: irc.freenode.net #chef

More Related Content

Viewers also liked

Basics of reflection in java
Basics of reflection in javaBasics of reflection in java
Basics of reflection in java
kim.mens
 
IBM SmartCloud Orchestration
IBM SmartCloud OrchestrationIBM SmartCloud Orchestration
IBM SmartCloud Orchestration
IBM Danmark
 
Reflection in Java
Reflection in JavaReflection in Java
Reflection in Java
Nikhil Bhardwaj
 
Reflection in java
Reflection in javaReflection in java
Reflection in java
upen.rockin
 
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Miguel Zuniga
 
Financial Management with ServiceNow at Franke
Financial Management with ServiceNow at FrankeFinancial Management with ServiceNow at Franke
Financial Management with ServiceNow at Franke
Aspediens
 
System Center + Cireson vs. ServiceNow
System Center + Cireson vs. ServiceNowSystem Center + Cireson vs. ServiceNow
System Center + Cireson vs. ServiceNow
Cireson
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
Robert Reiz
 
HGConcept-ITOM-Service Centric Operations
HGConcept-ITOM-Service Centric OperationsHGConcept-ITOM-Service Centric Operations
HGConcept-ITOM-Service Centric Operations
HGConcept Inc.
 
Openstack - Enterprise cloud management platform
Openstack - Enterprise cloud management platformOpenstack - Enterprise cloud management platform
Openstack - Enterprise cloud management platform
Nagaraj Shenoy
 
Ansible - Introduction
Ansible - IntroductionAnsible - Introduction
Ansible - Introduction
Stephane Manciot
 
The Service-Oriented Enterprise with ServiceNow
The Service-Oriented Enterprise with ServiceNowThe Service-Oriented Enterprise with ServiceNow
The Service-Oriented Enterprise with ServiceNow
Aspediens
 
Zenoss & ServiceNow Integration - Incident Management & CMDB
Zenoss & ServiceNow Integration - Incident Management & CMDBZenoss & ServiceNow Integration - Incident Management & CMDB
Zenoss & ServiceNow Integration - Incident Management & CMDB
Zenoss
 
Presentation cloud management platform
Presentation   cloud management platformPresentation   cloud management platform
Presentation cloud management platform
xKinAnx
 
Chef - Configuration Management for the Cloud
Chef - Configuration Management for the CloudChef - Configuration Management for the Cloud
Chef - Configuration Management for the Cloud
James Casey
 
RHTE2015_CloudForms_OpenStack
RHTE2015_CloudForms_OpenStackRHTE2015_CloudForms_OpenStack
RHTE2015_CloudForms_OpenStack
Jerome Marc
 
Cloudforms Workshop
Cloudforms WorkshopCloudforms Workshop
Cloudforms Workshop
Scalar Decisions
 
Jenkins and Chef: Infrastructure CI and Automated Deployment
Jenkins and Chef: Infrastructure CI and Automated DeploymentJenkins and Chef: Infrastructure CI and Automated Deployment
Jenkins and Chef: Infrastructure CI and Automated Deployment
Dan Stine
 
Chef for DevOps - an Introduction
Chef for DevOps - an IntroductionChef for DevOps - an Introduction
Chef for DevOps - an Introduction
Sanjeev Sharma
 
Ansible presentation
Ansible presentationAnsible presentation
Ansible presentation
John Lynch
 

Viewers also liked (20)

Basics of reflection in java
Basics of reflection in javaBasics of reflection in java
Basics of reflection in java
 
IBM SmartCloud Orchestration
IBM SmartCloud OrchestrationIBM SmartCloud Orchestration
IBM SmartCloud Orchestration
 
Reflection in Java
Reflection in JavaReflection in Java
Reflection in Java
 
Reflection in java
Reflection in javaReflection in java
Reflection in java
 
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
 
Financial Management with ServiceNow at Franke
Financial Management with ServiceNow at FrankeFinancial Management with ServiceNow at Franke
Financial Management with ServiceNow at Franke
 
System Center + Cireson vs. ServiceNow
System Center + Cireson vs. ServiceNowSystem Center + Cireson vs. ServiceNow
System Center + Cireson vs. ServiceNow
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
 
HGConcept-ITOM-Service Centric Operations
HGConcept-ITOM-Service Centric OperationsHGConcept-ITOM-Service Centric Operations
HGConcept-ITOM-Service Centric Operations
 
Openstack - Enterprise cloud management platform
Openstack - Enterprise cloud management platformOpenstack - Enterprise cloud management platform
Openstack - Enterprise cloud management platform
 
Ansible - Introduction
Ansible - IntroductionAnsible - Introduction
Ansible - Introduction
 
The Service-Oriented Enterprise with ServiceNow
The Service-Oriented Enterprise with ServiceNowThe Service-Oriented Enterprise with ServiceNow
The Service-Oriented Enterprise with ServiceNow
 
Zenoss & ServiceNow Integration - Incident Management & CMDB
Zenoss & ServiceNow Integration - Incident Management & CMDBZenoss & ServiceNow Integration - Incident Management & CMDB
Zenoss & ServiceNow Integration - Incident Management & CMDB
 
Presentation cloud management platform
Presentation   cloud management platformPresentation   cloud management platform
Presentation cloud management platform
 
Chef - Configuration Management for the Cloud
Chef - Configuration Management for the CloudChef - Configuration Management for the Cloud
Chef - Configuration Management for the Cloud
 
RHTE2015_CloudForms_OpenStack
RHTE2015_CloudForms_OpenStackRHTE2015_CloudForms_OpenStack
RHTE2015_CloudForms_OpenStack
 
Cloudforms Workshop
Cloudforms WorkshopCloudforms Workshop
Cloudforms Workshop
 
Jenkins and Chef: Infrastructure CI and Automated Deployment
Jenkins and Chef: Infrastructure CI and Automated DeploymentJenkins and Chef: Infrastructure CI and Automated Deployment
Jenkins and Chef: Infrastructure CI and Automated Deployment
 
Chef for DevOps - an Introduction
Chef for DevOps - an IntroductionChef for DevOps - an Introduction
Chef for DevOps - an Introduction
 
Ansible presentation
Ansible presentationAnsible presentation
Ansible presentation
 

Similar to Using Chef for Automated Infrastructure in the Cloud

Infrastructure Automation with Chef
Infrastructure Automation with ChefInfrastructure Automation with Chef
Infrastructure Automation with Chef
Adam Jacob
 
Continuous Integration, the minimum viable product
Continuous Integration, the minimum viable productContinuous Integration, the minimum viable product
Continuous Integration, the minimum viable product
Julian Simpson
 
Userstories a practical intro
Userstories a practical introUserstories a practical intro
Userstories a practical intro
Marcus Hammarberg
 
10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
10+ Deploys Per Day: Dev and Ops Cooperation at Flickr10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
John Allspaw
 
The Seven Wastes of Software Development
The Seven Wastes of Software DevelopmentThe Seven Wastes of Software Development
The Seven Wastes of Software Development
Matt Stine
 
TDD Boot Camp Sapporo 1.5
TDD Boot Camp Sapporo 1.5 TDD Boot Camp Sapporo 1.5
TDD Boot Camp Sapporo 1.5
Shuji Watanabe
 
前端测试之淘宝实践
前端测试之淘宝实践前端测试之淘宝实践
前端测试之淘宝实践
chencheng 云谦
 
【前端测试】淘宝前端测试实践
【前端测试】淘宝前端测试实践【前端测试】淘宝前端测试实践
【前端测试】淘宝前端测试实践
taobao.com
 
Design for Scale / Surge 2010
Design for Scale / Surge 2010Design for Scale / Surge 2010
Design for Scale / Surge 2010
Christopher Brown
 
CloudStack usage service
CloudStack usage serviceCloudStack usage service
CloudStack usage service
ShapeBlue
 
Sitecore on Azure
Sitecore on AzureSitecore on Azure
Sitecore on Azure
ClearPeople
 
[131] packetbeat과 elasticsearch
[131] packetbeat과 elasticsearch[131] packetbeat과 elasticsearch
[131] packetbeat과 elasticsearch
NAVER D2
 
Boris Stoyanov - some new features in Apache cloudStack
Boris Stoyanov - some new features in Apache cloudStackBoris Stoyanov - some new features in Apache cloudStack
Boris Stoyanov - some new features in Apache cloudStack
ShapeBlue
 
Stop Worrying about Prodweb001 and Start Loving i-98fb9856 (ARC201) | AWS re:...
Stop Worrying about Prodweb001 and Start Loving i-98fb9856 (ARC201) | AWS re:...Stop Worrying about Prodweb001 and Start Loving i-98fb9856 (ARC201) | AWS re:...
Stop Worrying about Prodweb001 and Start Loving i-98fb9856 (ARC201) | AWS re:...
Amazon Web Services
 
Deploying distributed software services to the cloud without breaking a sweat
Deploying distributed software services to the cloud without breaking a sweatDeploying distributed software services to the cloud without breaking a sweat
Deploying distributed software services to the cloud without breaking a sweat
Susan Potter
 
Distributed software services to the cloud without breaking a sweat
Distributed software services to the cloud without breaking a sweatDistributed software services to the cloud without breaking a sweat
Distributed software services to the cloud without breaking a sweat
José Ferreiro
 
Adversary Emulation and Cracking The Bridge – Overview EMERSON EDUARDO RODRIGUES
Adversary Emulation and Cracking The Bridge – Overview EMERSON EDUARDO RODRIGUESAdversary Emulation and Cracking The Bridge – Overview EMERSON EDUARDO RODRIGUES
Adversary Emulation and Cracking The Bridge – Overview EMERSON EDUARDO RODRIGUES
EMERSON EDUARDO RODRIGUES
 
Building Rackspace Cloud Monitoring
Building Rackspace Cloud MonitoringBuilding Rackspace Cloud Monitoring
Building Rackspace Cloud Monitoring
gdusbabek
 
Bitrzr - Ignite Portugal Tecnológico
Bitrzr  - Ignite Portugal TecnológicoBitrzr  - Ignite Portugal Tecnológico
Bitrzr - Ignite Portugal Tecnológico
bitrzr
 
Jabber Bot
Jabber BotJabber Bot
Jabber Bot
Aizat Faiz
 

Similar to Using Chef for Automated Infrastructure in the Cloud (20)

Infrastructure Automation with Chef
Infrastructure Automation with ChefInfrastructure Automation with Chef
Infrastructure Automation with Chef
 
Continuous Integration, the minimum viable product
Continuous Integration, the minimum viable productContinuous Integration, the minimum viable product
Continuous Integration, the minimum viable product
 
Userstories a practical intro
Userstories a practical introUserstories a practical intro
Userstories a practical intro
 
10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
10+ Deploys Per Day: Dev and Ops Cooperation at Flickr10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
 
The Seven Wastes of Software Development
The Seven Wastes of Software DevelopmentThe Seven Wastes of Software Development
The Seven Wastes of Software Development
 
TDD Boot Camp Sapporo 1.5
TDD Boot Camp Sapporo 1.5 TDD Boot Camp Sapporo 1.5
TDD Boot Camp Sapporo 1.5
 
前端测试之淘宝实践
前端测试之淘宝实践前端测试之淘宝实践
前端测试之淘宝实践
 
【前端测试】淘宝前端测试实践
【前端测试】淘宝前端测试实践【前端测试】淘宝前端测试实践
【前端测试】淘宝前端测试实践
 
Design for Scale / Surge 2010
Design for Scale / Surge 2010Design for Scale / Surge 2010
Design for Scale / Surge 2010
 
CloudStack usage service
CloudStack usage serviceCloudStack usage service
CloudStack usage service
 
Sitecore on Azure
Sitecore on AzureSitecore on Azure
Sitecore on Azure
 
[131] packetbeat과 elasticsearch
[131] packetbeat과 elasticsearch[131] packetbeat과 elasticsearch
[131] packetbeat과 elasticsearch
 
Boris Stoyanov - some new features in Apache cloudStack
Boris Stoyanov - some new features in Apache cloudStackBoris Stoyanov - some new features in Apache cloudStack
Boris Stoyanov - some new features in Apache cloudStack
 
Stop Worrying about Prodweb001 and Start Loving i-98fb9856 (ARC201) | AWS re:...
Stop Worrying about Prodweb001 and Start Loving i-98fb9856 (ARC201) | AWS re:...Stop Worrying about Prodweb001 and Start Loving i-98fb9856 (ARC201) | AWS re:...
Stop Worrying about Prodweb001 and Start Loving i-98fb9856 (ARC201) | AWS re:...
 
Deploying distributed software services to the cloud without breaking a sweat
Deploying distributed software services to the cloud without breaking a sweatDeploying distributed software services to the cloud without breaking a sweat
Deploying distributed software services to the cloud without breaking a sweat
 
Distributed software services to the cloud without breaking a sweat
Distributed software services to the cloud without breaking a sweatDistributed software services to the cloud without breaking a sweat
Distributed software services to the cloud without breaking a sweat
 
Adversary Emulation and Cracking The Bridge – Overview EMERSON EDUARDO RODRIGUES
Adversary Emulation and Cracking The Bridge – Overview EMERSON EDUARDO RODRIGUESAdversary Emulation and Cracking The Bridge – Overview EMERSON EDUARDO RODRIGUES
Adversary Emulation and Cracking The Bridge – Overview EMERSON EDUARDO RODRIGUES
 
Building Rackspace Cloud Monitoring
Building Rackspace Cloud MonitoringBuilding Rackspace Cloud Monitoring
Building Rackspace Cloud Monitoring
 
Bitrzr - Ignite Portugal Tecnológico
Bitrzr  - Ignite Portugal TecnológicoBitrzr  - Ignite Portugal Tecnológico
Bitrzr - Ignite Portugal Tecnológico
 
Jabber Bot
Jabber BotJabber Bot
Jabber Bot
 

More from Jesse Robbins

Jesse Robbins @ MWC 2015 - Building Orion Onyx - Real-time wearable push to t...
Jesse Robbins @ MWC 2015 - Building Orion Onyx - Real-time wearable push to t...Jesse Robbins @ MWC 2015 - Building Orion Onyx - Real-time wearable push to t...
Jesse Robbins @ MWC 2015 - Building Orion Onyx - Real-time wearable push to t...
Jesse Robbins
 
Orion Labs - From Bits to Atoms
Orion Labs - From Bits to AtomsOrion Labs - From Bits to Atoms
Orion Labs - From Bits to Atoms
Jesse Robbins
 
Jesse Robbins Keynote - Hacking Culture @ Cloud Expo Europe 2013
Jesse Robbins Keynote - Hacking Culture @ Cloud Expo Europe 2013Jesse Robbins Keynote - Hacking Culture @ Cloud Expo Europe 2013
Jesse Robbins Keynote - Hacking Culture @ Cloud Expo Europe 2013
Jesse Robbins
 
Continuous Deployment & Delivery + Culture Hacks @ QCON 2012
Continuous Deployment & Delivery + Culture Hacks @ QCON 2012Continuous Deployment & Delivery + Culture Hacks @ QCON 2012
Continuous Deployment & Delivery + Culture Hacks @ QCON 2012
Jesse Robbins
 
Hacking Culture at VelocityConf
Hacking Culture at VelocityConfHacking Culture at VelocityConf
Hacking Culture at VelocityConf
Jesse Robbins
 
Rebooting a Cloud
Rebooting a CloudRebooting a Cloud
Rebooting a Cloud
Jesse Robbins
 
GameDay: Creating Resiliency Through Destruction - LISA11
GameDay: Creating Resiliency Through Destruction - LISA11GameDay: Creating Resiliency Through Destruction - LISA11
GameDay: Creating Resiliency Through Destruction - LISA11
Jesse Robbins
 
DevOps @ InterOP Las Vegas - Jesse Robbins - Opscode
DevOps @ InterOP Las Vegas - Jesse Robbins - OpscodeDevOps @ InterOP Las Vegas - Jesse Robbins - Opscode
DevOps @ InterOP Las Vegas - Jesse Robbins - Opscode
Jesse Robbins
 
Gov 2.0: Scaling, Automation, & Management in the Cloud
Gov 2.0: Scaling, Automation, & Management in the CloudGov 2.0: Scaling, Automation, & Management in the Cloud
Gov 2.0: Scaling, Automation, & Management in the Cloud
Jesse Robbins
 
Cloud Operations Bootcamp: Culture - Jesse Robbins
Cloud Operations Bootcamp: Culture - Jesse Robbins Cloud Operations Bootcamp: Culture - Jesse Robbins
Cloud Operations Bootcamp: Culture - Jesse Robbins
Jesse Robbins
 
Failure Happens Interop Nyc
Failure Happens Interop NycFailure Happens Interop Nyc
Failure Happens Interop Nyc
Jesse Robbins
 
Serving Those That Serve Others Web2 Summit Jesse Robbins Final
Serving Those That Serve Others Web2 Summit Jesse Robbins FinalServing Those That Serve Others Web2 Summit Jesse Robbins Final
Serving Those That Serve Others Web2 Summit Jesse Robbins Final
Jesse Robbins
 
Failure Happens: CloudCamp Interop
Failure Happens: CloudCamp InteropFailure Happens: CloudCamp Interop
Failure Happens: CloudCamp Interop
Jesse Robbins
 
DisasterTech Presentation @ NEMA
DisasterTech Presentation @ NEMADisasterTech Presentation @ NEMA
DisasterTech Presentation @ NEMA
Jesse Robbins
 
ETech2008 DisasterTech Robbins Maron 20080305a
ETech2008 DisasterTech Robbins Maron 20080305aETech2008 DisasterTech Robbins Maron 20080305a
ETech2008 DisasterTech Robbins Maron 20080305a
Jesse Robbins
 

More from Jesse Robbins (15)

Jesse Robbins @ MWC 2015 - Building Orion Onyx - Real-time wearable push to t...
Jesse Robbins @ MWC 2015 - Building Orion Onyx - Real-time wearable push to t...Jesse Robbins @ MWC 2015 - Building Orion Onyx - Real-time wearable push to t...
Jesse Robbins @ MWC 2015 - Building Orion Onyx - Real-time wearable push to t...
 
Orion Labs - From Bits to Atoms
Orion Labs - From Bits to AtomsOrion Labs - From Bits to Atoms
Orion Labs - From Bits to Atoms
 
Jesse Robbins Keynote - Hacking Culture @ Cloud Expo Europe 2013
Jesse Robbins Keynote - Hacking Culture @ Cloud Expo Europe 2013Jesse Robbins Keynote - Hacking Culture @ Cloud Expo Europe 2013
Jesse Robbins Keynote - Hacking Culture @ Cloud Expo Europe 2013
 
Continuous Deployment & Delivery + Culture Hacks @ QCON 2012
Continuous Deployment & Delivery + Culture Hacks @ QCON 2012Continuous Deployment & Delivery + Culture Hacks @ QCON 2012
Continuous Deployment & Delivery + Culture Hacks @ QCON 2012
 
Hacking Culture at VelocityConf
Hacking Culture at VelocityConfHacking Culture at VelocityConf
Hacking Culture at VelocityConf
 
Rebooting a Cloud
Rebooting a CloudRebooting a Cloud
Rebooting a Cloud
 
GameDay: Creating Resiliency Through Destruction - LISA11
GameDay: Creating Resiliency Through Destruction - LISA11GameDay: Creating Resiliency Through Destruction - LISA11
GameDay: Creating Resiliency Through Destruction - LISA11
 
DevOps @ InterOP Las Vegas - Jesse Robbins - Opscode
DevOps @ InterOP Las Vegas - Jesse Robbins - OpscodeDevOps @ InterOP Las Vegas - Jesse Robbins - Opscode
DevOps @ InterOP Las Vegas - Jesse Robbins - Opscode
 
Gov 2.0: Scaling, Automation, & Management in the Cloud
Gov 2.0: Scaling, Automation, & Management in the CloudGov 2.0: Scaling, Automation, & Management in the Cloud
Gov 2.0: Scaling, Automation, & Management in the Cloud
 
Cloud Operations Bootcamp: Culture - Jesse Robbins
Cloud Operations Bootcamp: Culture - Jesse Robbins Cloud Operations Bootcamp: Culture - Jesse Robbins
Cloud Operations Bootcamp: Culture - Jesse Robbins
 
Failure Happens Interop Nyc
Failure Happens Interop NycFailure Happens Interop Nyc
Failure Happens Interop Nyc
 
Serving Those That Serve Others Web2 Summit Jesse Robbins Final
Serving Those That Serve Others Web2 Summit Jesse Robbins FinalServing Those That Serve Others Web2 Summit Jesse Robbins Final
Serving Those That Serve Others Web2 Summit Jesse Robbins Final
 
Failure Happens: CloudCamp Interop
Failure Happens: CloudCamp InteropFailure Happens: CloudCamp Interop
Failure Happens: CloudCamp Interop
 
DisasterTech Presentation @ NEMA
DisasterTech Presentation @ NEMADisasterTech Presentation @ NEMA
DisasterTech Presentation @ NEMA
 
ETech2008 DisasterTech Robbins Maron 20080305a
ETech2008 DisasterTech Robbins Maron 20080305aETech2008 DisasterTech Robbins Maron 20080305a
ETech2008 DisasterTech Robbins Maron 20080305a
 

Recently uploaded

IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Networks
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
Edge AI and Vision Alliance
 
Best Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdfBest Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdf
Tatiana Al-Chueyr
 
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Torry Harris
 
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
Priyanka Aash
 
The Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF GuideThe Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF Guide
Shiv Technolabs
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
SynapseIndia
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
Anant Gupta
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
Pigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending PlantPigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending Plant
LINUS PROJECTS (INDIA)
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
Figma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdfFigma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdf
Management Institute of Skills Development
 
Salesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot WorkshopSalesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot Workshop
CEPTES Software Inc
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
Neo4j
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Zilliz
 

Recently uploaded (20)

IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
 
Best Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdfBest Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdf
 
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
 
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
 
The Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF GuideThe Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF Guide
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
Pigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending PlantPigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending Plant
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
Figma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdfFigma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdf
 
Salesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot WorkshopSalesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot Workshop
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
 

Using Chef for Automated Infrastructure in the Cloud

  • 1. Automating the Cloud with Chef Adam Jacob Co-Founder & CTO @ Opscode
  • 2. Who am I, who are you, and why are we here? • The Method • EC2 • Chef • Nanite • The Real World • Q &A http://www.flickr.com/photos/niecieden/367343737/sizes/o/
  • 3. • 13 years as a Systems Administrator • Lots of Mergers and Acquisitions • Consultant • Wrote much of Chef • CTO at Opscode http://www.flickr.com/photos/anotherphotograph/2100904507/sizes/o/
  • 5. • Developers? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 6. • Developers? • Systems Administrators? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 7. Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
  • 8. Total Bootstrapping Time in Weeks 8 6 Corp Approvals Agile Approvals 4 Cloud 2 0 Best Time 0 Worst Time Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
  • 9. Total Bootstrapping Time in Weeks of ret g. 8 sec utin the mp tue s is Co r vir . hi ud 6 T o he here ot m Cl Corp Approvals ery s fro Ev mApprovals Agile 4 ste Cloud 2 0 Best Time 0 Worst Time Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
  • 11. For Developers... • Do it yourself.
  • 12. For Developers... • Do it yourself. • The infrastructure is the application (and vice versa).
  • 13. For Developers... • Do it yourself. • The infrastructure is the application (and vice versa). • You are not a Systems Administrator.
  • 14. For Developers... • Do it yourself. • The infrastructure is the application (and vice versa). • You are not a Systems Administrator. • You need tools.
  • 15. Sysadmins... http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 16. Sysadmins... • Say “Yes”. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 17. Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 18. Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. • You have never been more critical. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 19. Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. • You have never been more critical. • Lean into it. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 20. The Method http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 21. The Method Bootstrapping http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 22. The Method Bootstrapping http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 23. The Method Bootstrapping Configuration http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 24. The Method Bootstrapping Configuration http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 25. The Method Bootstrapping Configuration Command & Control http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 26. The Method Bootstrapping Configuration Command & Control Nanite! http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 27. Lightning Strikes! DOOM Webservers Database Servers Webservers
  • 28. Lightning Strikes! DOOM X Webservers XX Database Servers Webservers
  • 29. Lightning Strikes! DOOM X 1 1 2 1 Signals Moar! Monitoring System Webservers 5 Updates 2 1 Command & Control XX 3 1 Bootstrapping Provisions 1 3 1 Database Servers 4 1 4 1 Configuration Webservers
  • 30. EC2 Best Practices http://www.flickr.com/photos/46183897@N00/3442880227/sizes/l/
  • 32. AMIs • Amazon Machine Images http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 33. AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 34. AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size • Use pre-existing images to bootstrap http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 35. AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size • Use pre-existing images to bootstrap • Include the Configuration and C&C Tools http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 37. AMIs • Re-bundle for specific roles http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 38. AMIs • Re-bundle for specific roles • Use SSH Keys for access http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 39. AMIs • Re-bundle for specific roles • Use SSH Keys for access • Shut off fsck! http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 41. Instances • Actual virtual machines http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 42. Instances • Actual virtual machines • c1.medium is the best bang for your $$ http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 43. Instances • Actual virtual machines • c1.medium is the best bang for your $$ • Use User Data to inform Configuration http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 44. Instances • Actual virtual machines • c1.medium is the best bang for your $$ • Use User Data to inform Configuration • Support indexed User Data - launch more than one at a time. (launch-index) http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 46. Instances • Update the default security group to allow SSH http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 47. Instances • Update the default security group to allow SSH • Use SSH Keys for authentication http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 48. Instances • Update the default security group to allow SSH • Use SSH Keys for authentication • Ephemeral Storage is your buddy http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 50. Instances • Internal and External hostnames are not unique http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 51. Instances • Internal and External hostnames are not unique • Only trust the instance-id http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 53. EBS • Elastic Block Store http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 54. EBS • Elastic Block Store • Truly persistent storage http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 55. EBS • Elastic Block Store • Truly persistent storage • Trivial to snapshot http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 56. EBS • Elastic Block Store • Truly persistent storage • Trivial to snapshot • Snapshots can bootstrap more EBS devices http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 57. Elastic IP Addresses http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 58. Elastic IP Addresses • Static, publicly routable addresses http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 59. Elastic IP Addresses • Static, publicly routable addresses • Fast and easy to re-assign http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 61. At a High Level... http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 62. At a High Level... • A library for configuration management http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 63. At a High Level... • A library for configuration management • A configuration management system http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 64. At a High Level... • A library for configuration management • A configuration management system • A systems integration platform http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 65. At a High Level... • A library for configuration management • A configuration management system • A systems integration platform • An API for your entire Infrastructure http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 67. Principles • Idempotent http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 68. Principles • Idempotent • Reasonability http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 69. Principles • Idempotent • Reasonability • Sane defaults, easily changed http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 70. Principles • Idempotent • Reasonability • Sane defaults, easily changed • Hackability http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 71. Principles • Idempotent • Reasonability • Sane defaults, easily changed • Hackability • TMTOWTDI http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 72. Infrastructure as Code Manage configuration as idempotent Resources. Put them together in Recipes. Track it like source code. Configure your servers. You can learn more about Chef at http://wiki.opscode.com/display/chef/Home
  • 73. Automating Sudo Permissions Write the recipe. http://xkcd.com/149/
  • 74. Automating Sudo Permissions Write the recipe. • Install the package. http://xkcd.com/149/
  • 75. Automating Sudo Permissions Write the recipe. • Install the package. • Write out the sudoers file. http://xkcd.com/149/
  • 76. Automating Sudo Permissions Write the recipe. • Install the package. • Write out the sudoers file. • Use custom attributes for users and groups. http://xkcd.com/149/
  • 77. Automating Sudo Permissions Write the sudoers template http://xkcd.com/149/
  • 78. Automating Sudo Permissions Write the sudoers template • Add a warning banner. http://xkcd.com/149/
  • 79. Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. http://xkcd.com/149/
  • 80. Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. • Add the node-specific users. http://xkcd.com/149/
  • 81. Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. • Add the node-specific users. http://xkcd.com/149/ • Add the node-specific groups.
  • 82. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. http://xkcd.com/149/
  • 83. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. http://xkcd.com/149/
  • 84. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. http://xkcd.com/149/
  • 85. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. http://xkcd.com/149/
  • 86. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. • Add default users. http://xkcd.com/149/
  • 87. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. • Add default users. http://xkcd.com/149/ • Add default groups.
  • 88. Automating Sudo Permissions Put it all together!
  • 89. Automating Sudo Permissions Put it all together! • Assign your role to nodes.
  • 90. Automating Sudo Permissions Put it all together! • Assign your role to nodes. • When Chef runs, the sudoers file will be populated.
  • 92. Nanite • “Self Assembling Cluster of Ruby Daemons” http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 93. Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 94. Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ • Actors register Services http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 95. Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ • Actors register Services • Distributed Map/Reduce for your Infrastructure http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 96. Nanite Architecture Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard - these slides pinched from http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 97. A Nanite Actor... Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 98. A Nanite Actor... • Advertises Services Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 99. A Nanite Actor... • Advertises Services • Advertises Tags Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 100. A Nanite Actor... • Advertises Services • Advertises Tags • Requests can route to Services and Tags Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 101. Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 102. Lightning Strikes, Monitoring /node/down Service Revisited! Signals Nanite 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 103. Lightning Strikes, Nanite boots new EC2 Nanite removes Revisited! Instances, with Chef Role + Attribute Data nodes in Chef 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 104. Lightning Strikes, Revisited! Provisions Instances, EBS, Elastic IPs 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 105. Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration Chef configures nodes according to assigned Roles
  • 106. Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 Chef updates the 1 3 1 5 1 3 1 monitoring system 4 1 4 1 Configuration
  • 107. In the Real World http://www.flickr.com/photos/kenlund/3376784956/sizes/l/
  • 108. A Simple Architecture Load Balancing Web Servers Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 109. A Simple Architecture AWS LB HAProxy S->M Load Balancing Web Servers Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 110. AWS LB A Simple Architecture HAProxy S->M Load Balancing Your App Stack M Web Servers EBS? Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 111. AWS LB A Simple Architecture HAProxy S->M Load Balancing Your App Stack M Web Servers EBS? Master/ Slave M->L->XL Databases EBS http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 112. Load Balancing http://www.flickr.com/photos/jannem/497840412/
  • 113. Load Balancing • Amazon offers Load Balancing http://www.flickr.com/photos/jannem/497840412/
  • 114. Load Balancing • Amazon offers Load Balancing • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
  • 115. Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
  • 116. Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing Configure Pools, Associate Backends • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
  • 117. Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing Configure Pools, Associate Backends • Alternative is HAProxy + Elastic IP Reconfigure based on Load, STONITH http://www.flickr.com/photos/jannem/497840412/
  • 118. Web Frameworks In General.. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 119. Web Frameworks In General.. • Try and keep things ephemeral http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 120. Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 121. Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 122. Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 123. Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 124. Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront Configuration, Deployment, • Use a shared session Discovery storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 125. Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront Configuration, Deployment, • Use a shared session Discovery storage - preferably cookie or memcached based. Trigger Deployment, Maintenance, Migrations • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 126. Ruby on Rails http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 127. Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 128. Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 129. Ruby on Rails Deploys Radiant CMS • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 130. Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin Configures Apache http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 132. • Use Chef for CPAN Dependencies Catalyst
  • 133. • Use Chef for CPAN Dependencies • Follow the Chef Deploy layout Catalyst
  • 134. • Use Chef for CPAN Dependencies • Follow the Chef Deploy layout • Use Catalyst::Plugin::Session::Store::Memcached Catalyst
  • 136. • Use Chef for Egg Dependencies http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 137. • Use Chef for Egg Dependencies • Use Chef Deploy layout http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 138. • Use Chef for Egg Dependencies • Use Chef Deploy layout • Memcached Cache Backend http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 139. • Use Chef for Egg Dependencies • Use Chef Deploy layout • Memcached Cache Backend • Use the Cache Session Backend http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 140. Databases In General.. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 141. Databases In General.. • Keep everything on EBS http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 142. Databases In General.. • Keep everything on EBS • Snapshot frequently http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 143. Databases In General.. • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 144. Databases In General.. • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 145. Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 146. Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS Configuration, Tuning, • Snapshot frequently User Credentials, • Keep a rolling backlog of Discovery snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 147. Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS Configuration, Tuning, • Snapshot frequently User Credentials, • Keep a rolling backlog of Discovery snapshots Maintenance, Migrations, • Register databases with Analytics Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 148. MySQL http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 149. MySQL • Always have a Slave http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 150. MySQL • Always have a Slave • EBS is neat, but a crash is a crash - and recovery is time consuming http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 151. MySQL • Always have a Slave • EBS is neat, but a crash is a crash - and recovery is time consuming • Building slaves is trivial http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 152. PostgreSQL http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 153. PostgreSQL • Replication choices make things harder - PGPool-II appears to be most common on EC2 http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 154. PostgreSQL • Replication choices make things harder - PGPool-II appears to be most common on EC2 • Hot standby with WAL Shipping http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 155. PostgreSQL Are you great at this on EC2? • Replication choices make things harder - PGPool-II appears to be most common on EC2 • Hot standby with WAL Shipping http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 156. Monitoring & Trending In General.. http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 157. Monitoring & Trending In General.. • Must be integrated with Command & Control http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 158. Monitoring & Trending In General.. • Must be integrated with Command & Control • Use search features in Chef to dynamically configure http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 159. Monitoring & Trending In General.. • Must be integrated with Command & Control • Use search features in Chef to dynamically configure • Make sure you monitor internally and externally http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 160. Nagios http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 161. Nagios • Do all that crazy config you have been avoiding http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 162. Nagios • Do all that crazy config you have been avoiding • Remarkably easy to automate http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 163. A word about Load...
  • 164. Typical Peak Load 1. Bring on capacity as traffic ramps up 2. Take down capacity as it ramps down 3. 10-15 Minutes on either side, fully unattended Graphs in this portion of the presentation taken from Theo Schlossnagle http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
  • 165. Atypical Load No way However, around you are Capacity still better Planning off! 1. Hope you know it is coming. 2. Increase capacity in advance. 3. Take down capacity as it ramps down. Graphs in this portion of the presentation taken from Theo Schlossnagle http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
  • 166. Round Up! http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 167. Round Up! Bring new resources online http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 168. Round Up! Bring new resources online Configure and manage them http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 169. Round Up! Bring new resources online Configure and manage them Make the whole thing sing and dance http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 170. http://www.flickr.com/photos/jackol/1766679527/sizes/l/ Q &A Please rate this talk! There is lots more to learn Email: adam@opscode.com about Chef at Twitter: adamhjk http://wiki.opscode.com IRC: irc.freenode.net #chef