# finger $(whoami)
Login: susan               Name: Susan Potter
Directory: /home/susan     Shell: /bin/zsh
On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0
No mail on me@susanpotter.net
Plan:
  twitter: @SusanPotter
  github: mbbx6spp
# finger $(whoami)
Login: susan               Name: Susan Potter
Directory: /home/susan     Shell: /bin/zsh
On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0
No mail on me@susanpotter.net
Plan:
  twitter: @SusanPotter
  github: mbbx6spp
# finger $(whoami)
Login: susan               Name: Susan Potter
Directory: /home/susan     Shell: /bin/zsh
On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0
No mail on me@susanpotter.net
Plan:
  twitter: @SusanPotter
  github: mbbx6spp
# finger $(whoami)
Login: susan               Name: Susan Potter
Directory: /home/susan     Shell: /bin/zsh
On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0
No mail on me@susanpotter.net
Plan:
  twitter: @SusanPotter
  github: mbbx6spp
# finger $(whoami)
Login: susan               Name: Susan Potter
Directory: /home/susan     Shell: /bin/zsh
On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0
No mail on me@susanpotter.net
Plan:
  twitter: @SusanPotter
  github: mbbx6spp
Scope of Talk


    • Approaches
Scope of Talk


    • Approaches
    • Best Practices
Scope of Talk


    • Approaches
    • Best Practices
    • Pitfalls
Scope of Talk


    •   Approaches
    •   Best Practices
    •   Pitfalls
    •   Possibilities
Scope of Talk


    •   Approaches       • (not) Chef vs Puppet
    •   Best Practices
    •   Pitfalls
    •   Possibilities
Scope of Talk


    •   Approaches       • (not) Chef vs Puppet
    •   Best Practices   • (not) Why Cloud?
    •   Pitfalls
    •   Possibilities
Scope of Talk


    •   Approaches       • (not) Chef vs Puppet
    •   Best Practices   • (not) Why Cloud?
    •   Pitfalls         • (not) Why DevOps?
    •   Possibilities
Scope of Talk


    •   Approaches       •   (not) Chef vs Puppet
    •   Best Practices   •   (not) Why Cloud?
    •   Pitfalls         •   (not) Why DevOps?
    •   Possibilities    •   (not) Which Delivery Model?
Scope of Talk


    •   Approaches       •   (not) Chef vs Puppet
    •   Best Practices   •   (not) Why Cloud?
    •   Pitfalls         •   (not) Why DevOps?
    •   Possibilities    •   (not) Which Delivery Model?
                         •   (not) Release Management
Scope of Talk


    •   Approaches       •   (not) Chef vs Puppet
    •   Best Practices   •   (not) Why Cloud?
    •   Pitfalls         •   (not) Why DevOps?
    •   Possibilities    •   (not) Which Delivery Model?
                         •   (not) Release Management
                         •   (not) Monitoring
Cloud




        buzzzzz
Cloud: Delivery Models [1/2]

        Software
        (as a Service)




        Platform
        (as a Service)




      Infrastructure
        (as a Service)
Cloud: Delivery Models [1/2]

        Software
        (as a Service)




        Platform
        (as a Service)




      Infrastructure
        (as a Service)
Cloud: Delivery Models [1/2]

        Software
        (as a Service)




        Platform
        (as a Service)




      Infrastructure
        (as a Service)
Cloud: Delivery Models [1/2]

        Software
        (as a Service)




        Platform
        (as a Service)




      Infrastructure
        (as a Service)
Cloud: Delivery Models [2/2]
Cloud: Characteristics


    • Instant
      on-demand
Cloud: Characteristics


    • Instant        • Virtualized
      on-demand          performance, reliability
Cloud: Characteristics


    • Instant        • Virtualized
      on-demand          performance, reliability

    • Managed
      by others
Cloud: Characteristics


    • Instant        • Virtualized
      on-demand          performance, reliability

    • Managed        • Lack control
      by others          predictability, reliability, quality
Cloud: Characteristics


    • Instant        • Virtualized
      on-demand          performance, reliability

    • Managed        • Lack control
      by others          predictability, reliability, quality

    • Pay
      as you go
Cloud: Characteristics


    • Instant        • Virtualized
      on-demand          performance, reliability

    • Managed        • Lack control
      by others          predictability, reliability, quality

    • Pay            • Pay
      as you go          as you go!
DevOps




     Is it a command?
DevOps: Definition [1/2]


       • Share responsibility
         across organizational boundaries
DevOps: Definition [1/2]


       • Share responsibility
         across organizational boundaries

       • Invest in people
         by reducing finger pointing [togetherness] and human error [automation]
DevOps: Definition [1/2]


       • Share responsibility
         across organizational boundaries

       • Invest in people
         by reducing finger pointing [togetherness] and human error [automation]

       • Manage infrastructure
         not priority queues of production issues
DevOps: Definition [1/2]


       • Share responsibility
         across organizational boundaries

       • Invest in people
         by reducing finger pointing [togetherness] and human error [automation]

       • Manage infrastructure
         not priority queues of production issues

       • Make infrastructure predictable
         repeatable, testable, deterministic
DevOps: Definition [2/2]
Deployment Pipeline




    commit -> production
Deployment Pipeline: Prerequisites

    • Design for cloud
      e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O
Deployment Pipeline: Prerequisites

    • Design for cloud
      e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O

    • Always-ready codebase
      buildable, testable, deployable
Deployment Pipeline: Prerequisites

    • Design for cloud
      e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O

    • Always-ready codebase
      buildable, testable, deployable

    • Managed infrastructure
      read: SCM and consistent distribution to target nodes
Deployment Pipeline: Prerequisites

    • Design for cloud
      e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O

    • Always-ready codebase
      buildable, testable, deployable

    • Managed infrastructure
      read: SCM and consistent distribution to target nodes

    • Expect [system] failure
      handle failures sensibly, policies for timeouts, etc
Deployment Pipeline: Prerequisites

    • Design for cloud
      e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O

    • Always-ready codebase
      buildable, testable, deployable

    • Managed infrastructure
      read: SCM and consistent distribution to target nodes

    • Expect [system] failure
      handle failures sensibly, policies for timeouts, etc

    • Test early and often!
      outside-in development helps
Deployment Pipeline: Prerequisites

    • Design for cloud
      e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O

    • Always-ready codebase
      buildable, testable, deployable

    • Managed infrastructure
      read: SCM and consistent distribution to target nodes

    • Expect [system] failure
      handle failures sensibly, policies for timeouts, etc

    • Test early and often!
      outside-in development helps

    • Build from the ground up
      layer infrastructure, inject configuration at boot/load time
Deployment: Common Bottlenecks

   •   Automation
       build, provision, configure, integrate




                                               Figure:   http://www.flickr.com/people/laenulfean/
Deployment: Common Bottlenecks

   •   Automation
       build, provision, configure, integrate

   •   Distribution
       binaries, assets, configuration




                                               Figure:   http://www.flickr.com/people/laenulfean/
Deployment: Common Bottlenecks

   •   Automation
       build, provision, configure, integrate

   •   Distribution
       binaries, assets, configuration

   •   Timeframe
       restricted window of time




                                               Figure:   http://www.flickr.com/people/laenulfean/
Deployment: Common Bottlenecks

   •   Automation
       build, provision, configure, integrate

   •   Distribution
       binaries, assets, configuration

   •   Timeframe
       restricted window of time

   •   Data
       schema updates, data migrations



                                               Figure:   http://www.flickr.com/people/laenulfean/
Solution Approaches



      Solve problems,
      don’t use tools!
Automation Approaches

                                                   • Full stack server-driven
                                                     e.g. Chef/Knife, Puppet Master




 Figure:   http://www.flickr.com/people/krazydad/
Automation Approaches

                                                   • Full stack server-driven
                                                     e.g. Chef/Knife, Puppet Master

                                                   • Full stack client
                                                     e.g. Chef Solo




 Figure:   http://www.flickr.com/people/krazydad/
Automation Approaches

                                                   • Full stack server-driven
                                                     e.g. Chef/Knife, Puppet Master

                                                   • Full stack client
                                                     e.g. Chef Solo

                                                   • Application-tier client
                                                     e.g. Capistrano, Vlad the Deployer




 Figure:   http://www.flickr.com/people/krazydad/
Automation Approaches

                                                   • Full stack server-driven
                                                     e.g. Chef/Knife, Puppet Master

                                                   • Full stack client
                                                     e.g. Chef Solo

                                                   • Application-tier client
                                                     e.g. Capistrano, Vlad the Deployer

                                                   • Command & control
                                                     e.g. Vertibrae (inactive), Nanite




 Figure:   http://www.flickr.com/people/krazydad/
Distribution Approaches

                                                • Shared filesystem
                                                 less security and reliability in community/public or across

                                                 zones/regions




  Figure:   http://www.flickr.com/people/nsalt
Distribution Approaches

                                                • Shared filesystem
                                                  less security and reliability in community/public or across

                                                  zones/regions

                                                • Pull from source control
                                                  higher time variance as target nodes increase




  Figure:   http://www.flickr.com/people/nsalt
Distribution Approaches

                                                • Shared filesystem
                                                  less security and reliability in community/public or across

                                                  zones/regions

                                                • Pull from source control
                                                  higher time variance as target nodes increase

                                                • Bittorrent or similar
                                                  e.g. Twitter’s Murder




  Figure:   http://www.flickr.com/people/nsalt
Timeframe Approaches
                                                     • Hot upgrades
                                                      e.g. Erlang/OTP appup/code_change/3




Figure:   http://www.flickr.com/people/athenicsword
Timeframe Approaches
                                                     • Hot upgrades
                                                       e.g. Erlang/OTP appup/code_change/3

                                                     • Rolling upgrades
                                                       Software design considerations




Figure:   http://www.flickr.com/people/athenicsword
Timeframe Approaches
                                                     • Hot upgrades
                                                       e.g. Erlang/OTP appup/code_change/3

                                                     • Rolling upgrades
                                                       Software design considerations

                                                     • Environment replacement
                                                       Flip a switch, acceptance <-> production




Figure:   http://www.flickr.com/people/athenicsword
Data Approaches
                                                      • Scriptable schema
                                                        Alternative: NoSQL, but. . .




Figure:   http://www.flickr.com/people/shanghaidaddy
Data Approaches
                                                      • Scriptable schema
                                                        Alternative: NoSQL, but. . .

                                                      • Automated migrations
                                                        e.g. Rails’ Migrations




Figure:   http://www.flickr.com/people/shanghaidaddy
Data Approaches
                                                      • Scriptable schema
                                                        Alternative: NoSQL, but. . .

                                                      • Automated migrations
                                                        e.g. Rails’ Migrations

                                                      • Sanity checks
                                                        e.g. cherry picked, consistency checks




Figure:   http://www.flickr.com/people/shanghaidaddy
Common Pitfalls & Workarounds
Organizational Culture




          Figure:   http://www.flickr.com/people/lucgaloppin/
Organizational Culture [Workaround]




         Figure:   http://www.flickr.com/photos/42682395@N04/
Feature Branching




         Figure:   http://www.flickr.com/photos/33939293@N02/
Feature Branching [Workaround]
Feature Branching [Workaround]




             DON’T!
Live by the meter.
Die by the meter.
              Figure: WARNING
Live by the meter.
Die by the meter. [Workarounds]




          Figure:   http://www.flickr.com/people/dirkjankraan/
Great Expectations




          Figure:   http://www.flickr.com/people/atoxinsocks/
Great Expectations [Workaround]




          Figure:   http://www.flickr.com/people/dirkjankraan/
Note: Design & Idempotency




         Figure:   A Twitter dialog between me and @jlouis666
Possibilities




            Where.next?
Possibilities
Possibilities


    •   Dynamic resource allocation
        allocate based on load, time of day, day of week/month
Possibilities


    •   Dynamic resource allocation
        allocate based on load, time of day, day of week/month

    •   Canary deployments
        (e.g. A/B testing)
Possibilities


    •   Dynamic resource allocation
        allocate based on load, time of day, day of week/month

    •   Canary deployments
        (e.g. A/B testing)

    •   Multi-region or multi-provider
        relocate based on time of day, failover
Questions?




         Figure:   http://www.flickr.com/photos/42682395@N04/
Questions?




         Figure:   http://www.flickr.com/photos/42682395@N04/




         @SusanPotter

Deploying distributed software services to the cloud without breaking a sweat

  • 4.
    # finger $(whoami) Login:susan Name: Susan Potter Directory: /home/susan Shell: /bin/zsh On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0 No mail on me@susanpotter.net Plan: twitter: @SusanPotter github: mbbx6spp
  • 5.
    # finger $(whoami) Login:susan Name: Susan Potter Directory: /home/susan Shell: /bin/zsh On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0 No mail on me@susanpotter.net Plan: twitter: @SusanPotter github: mbbx6spp
  • 6.
    # finger $(whoami) Login:susan Name: Susan Potter Directory: /home/susan Shell: /bin/zsh On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0 No mail on me@susanpotter.net Plan: twitter: @SusanPotter github: mbbx6spp
  • 7.
    # finger $(whoami) Login:susan Name: Susan Potter Directory: /home/susan Shell: /bin/zsh On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0 No mail on me@susanpotter.net Plan: twitter: @SusanPotter github: mbbx6spp
  • 8.
    # finger $(whoami) Login:susan Name: Susan Potter Directory: /home/susan Shell: /bin/zsh On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0 No mail on me@susanpotter.net Plan: twitter: @SusanPotter github: mbbx6spp
  • 9.
    Scope of Talk • Approaches
  • 10.
    Scope of Talk • Approaches • Best Practices
  • 11.
    Scope of Talk • Approaches • Best Practices • Pitfalls
  • 12.
    Scope of Talk • Approaches • Best Practices • Pitfalls • Possibilities
  • 13.
    Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • Pitfalls • Possibilities
  • 14.
    Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • Possibilities
  • 15.
    Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities
  • 16.
    Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities • (not) Which Delivery Model?
  • 17.
    Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities • (not) Which Delivery Model? • (not) Release Management
  • 18.
    Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities • (not) Which Delivery Model? • (not) Release Management • (not) Monitoring
  • 19.
    Cloud buzzzzz
  • 20.
    Cloud: Delivery Models[1/2] Software (as a Service) Platform (as a Service) Infrastructure (as a Service)
  • 21.
    Cloud: Delivery Models[1/2] Software (as a Service) Platform (as a Service) Infrastructure (as a Service)
  • 22.
    Cloud: Delivery Models[1/2] Software (as a Service) Platform (as a Service) Infrastructure (as a Service)
  • 23.
    Cloud: Delivery Models[1/2] Software (as a Service) Platform (as a Service) Infrastructure (as a Service)
  • 24.
  • 25.
    Cloud: Characteristics • Instant on-demand
  • 26.
    Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability
  • 27.
    Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed by others
  • 28.
    Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed • Lack control by others predictability, reliability, quality
  • 29.
    Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed • Lack control by others predictability, reliability, quality • Pay as you go
  • 30.
    Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed • Lack control by others predictability, reliability, quality • Pay • Pay as you go as you go!
  • 31.
    DevOps Is it a command?
  • 32.
    DevOps: Definition [1/2] • Share responsibility across organizational boundaries
  • 33.
    DevOps: Definition [1/2] • Share responsibility across organizational boundaries • Invest in people by reducing finger pointing [togetherness] and human error [automation]
  • 34.
    DevOps: Definition [1/2] • Share responsibility across organizational boundaries • Invest in people by reducing finger pointing [togetherness] and human error [automation] • Manage infrastructure not priority queues of production issues
  • 35.
    DevOps: Definition [1/2] • Share responsibility across organizational boundaries • Invest in people by reducing finger pointing [togetherness] and human error [automation] • Manage infrastructure not priority queues of production issues • Make infrastructure predictable repeatable, testable, deterministic
  • 36.
  • 37.
    Deployment Pipeline commit -> production
  • 38.
    Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O
  • 39.
    Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable
  • 40.
    Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes
  • 41.
    Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes • Expect [system] failure handle failures sensibly, policies for timeouts, etc
  • 42.
    Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes • Expect [system] failure handle failures sensibly, policies for timeouts, etc • Test early and often! outside-in development helps
  • 43.
    Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes • Expect [system] failure handle failures sensibly, policies for timeouts, etc • Test early and often! outside-in development helps • Build from the ground up layer infrastructure, inject configuration at boot/load time
  • 44.
    Deployment: Common Bottlenecks • Automation build, provision, configure, integrate Figure: http://www.flickr.com/people/laenulfean/
  • 45.
    Deployment: Common Bottlenecks • Automation build, provision, configure, integrate • Distribution binaries, assets, configuration Figure: http://www.flickr.com/people/laenulfean/
  • 46.
    Deployment: Common Bottlenecks • Automation build, provision, configure, integrate • Distribution binaries, assets, configuration • Timeframe restricted window of time Figure: http://www.flickr.com/people/laenulfean/
  • 47.
    Deployment: Common Bottlenecks • Automation build, provision, configure, integrate • Distribution binaries, assets, configuration • Timeframe restricted window of time • Data schema updates, data migrations Figure: http://www.flickr.com/people/laenulfean/
  • 48.
    Solution Approaches Solve problems, don’t use tools!
  • 49.
    Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master Figure: http://www.flickr.com/people/krazydad/
  • 50.
    Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master • Full stack client e.g. Chef Solo Figure: http://www.flickr.com/people/krazydad/
  • 51.
    Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master • Full stack client e.g. Chef Solo • Application-tier client e.g. Capistrano, Vlad the Deployer Figure: http://www.flickr.com/people/krazydad/
  • 52.
    Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master • Full stack client e.g. Chef Solo • Application-tier client e.g. Capistrano, Vlad the Deployer • Command & control e.g. Vertibrae (inactive), Nanite Figure: http://www.flickr.com/people/krazydad/
  • 53.
    Distribution Approaches • Shared filesystem less security and reliability in community/public or across zones/regions Figure: http://www.flickr.com/people/nsalt
  • 54.
    Distribution Approaches • Shared filesystem less security and reliability in community/public or across zones/regions • Pull from source control higher time variance as target nodes increase Figure: http://www.flickr.com/people/nsalt
  • 55.
    Distribution Approaches • Shared filesystem less security and reliability in community/public or across zones/regions • Pull from source control higher time variance as target nodes increase • Bittorrent or similar e.g. Twitter’s Murder Figure: http://www.flickr.com/people/nsalt
  • 56.
    Timeframe Approaches • Hot upgrades e.g. Erlang/OTP appup/code_change/3 Figure: http://www.flickr.com/people/athenicsword
  • 57.
    Timeframe Approaches • Hot upgrades e.g. Erlang/OTP appup/code_change/3 • Rolling upgrades Software design considerations Figure: http://www.flickr.com/people/athenicsword
  • 58.
    Timeframe Approaches • Hot upgrades e.g. Erlang/OTP appup/code_change/3 • Rolling upgrades Software design considerations • Environment replacement Flip a switch, acceptance <-> production Figure: http://www.flickr.com/people/athenicsword
  • 59.
    Data Approaches • Scriptable schema Alternative: NoSQL, but. . . Figure: http://www.flickr.com/people/shanghaidaddy
  • 60.
    Data Approaches • Scriptable schema Alternative: NoSQL, but. . . • Automated migrations e.g. Rails’ Migrations Figure: http://www.flickr.com/people/shanghaidaddy
  • 61.
    Data Approaches • Scriptable schema Alternative: NoSQL, but. . . • Automated migrations e.g. Rails’ Migrations • Sanity checks e.g. cherry picked, consistency checks Figure: http://www.flickr.com/people/shanghaidaddy
  • 62.
    Common Pitfalls &Workarounds
  • 63.
    Organizational Culture Figure: http://www.flickr.com/people/lucgaloppin/
  • 64.
    Organizational Culture [Workaround] Figure: http://www.flickr.com/photos/42682395@N04/
  • 65.
    Feature Branching Figure: http://www.flickr.com/photos/33939293@N02/
  • 66.
  • 67.
  • 68.
    Live by themeter. Die by the meter. Figure: WARNING
  • 69.
    Live by themeter. Die by the meter. [Workarounds] Figure: http://www.flickr.com/people/dirkjankraan/
  • 70.
    Great Expectations Figure: http://www.flickr.com/people/atoxinsocks/
  • 71.
    Great Expectations [Workaround] Figure: http://www.flickr.com/people/dirkjankraan/
  • 72.
    Note: Design &Idempotency Figure: A Twitter dialog between me and @jlouis666
  • 73.
    Possibilities Where.next?
  • 74.
  • 75.
    Possibilities • Dynamic resource allocation allocate based on load, time of day, day of week/month
  • 76.
    Possibilities • Dynamic resource allocation allocate based on load, time of day, day of week/month • Canary deployments (e.g. A/B testing)
  • 77.
    Possibilities • Dynamic resource allocation allocate based on load, time of day, day of week/month • Canary deployments (e.g. A/B testing) • Multi-region or multi-provider relocate based on time of day, failover
  • 78.
    Questions? Figure: http://www.flickr.com/photos/42682395@N04/
  • 79.
    Questions? Figure: http://www.flickr.com/photos/42682395@N04/ @SusanPotter