continuous delivery
                           @jezhumble
                           Berlin, 4 october 2012




                         http://thoughtworks-studios.com/
Tuesday, October 9, 12
agile 101



                                      "Agile" team

                                   Analysis + Design         Centralized QA                           IT Operations

                                      Development            Integration + QA                     Release and operation
   Customer                        Testing + Showcase
                   Iteration   0      1    2     3      4
                                                                                The "last mile"




Tuesday, October 9, 12
social



                 disrupting traditional businesses




                                                 http://code.flickr.com/




Tuesday, October 9, 12
releasing frequently



                 1. build the right thing
                                                         Customer
                                                        development


                                                         Agile product
                                                         development




                                     Eric Ries, “The Lean Startup” http://bit.ly/8ZoX5F




Tuesday, October 9, 12
innovate
                           You can't just ask
                             customers what
                         they want and then
                           try to give that to
                                        them.

                         By the time you get
                         it built, they'll want
                             something new.
    Steve Jobs

Tuesday, October 9, 12
scientific method

                                                             Ideas


                 create hypothesis          Learn                             Build


                 deliver minimum
                 viable product
                                            Data                                 Code

                 get feedback                               Measure


                 (repeat)
                                     Eric Ries, “The Lean Startup” http://bit.ly/8ZoX5F

Tuesday, October 9, 12
ask this question


        “How long would it take your
        organization to deploy a change that
        involved just one single line of code? Do
        you do this on a repeatable, reliable
        basis?”



                          Mary and Tom Poppendieck, Implementing Lean Software Development, p59.

Tuesday, October 9, 12
releasing frequently



                 1. build the right thing
                 2. reduce risk of release




                         John Allspaw: “Ops Metametrics” http://slidesha.re/dsSZIr

Tuesday, October 9, 12
optimize for mtrs




Tuesday, October 9, 12
optimize for mtrs




                            MTBF                                                  MTRS

                                John Allspaw: “Building Resilience in Web Development and Operations”
                         http://www.usievents.com/en/conferences/8-paris-usi-2011/sessions/968-john-allspaw
Tuesday, October 9, 12
releasing frequently



                 1. build the right thing
                 2. reduce risk of release
                 3. real project progress




Tuesday, October 9, 12
agile manifesto



                 Our highest priority is to satisfy
                 the customer through early and
                 continuous delivery of
                 valuable software




Tuesday, October 9, 12
production-ready software



                 Fast, automated feedback on
                 the production readiness of
                 your applications every time
                 there is a change - to code,
                 infrastructure, or configuration



Tuesday, October 9, 12
continuous delivery




                 Software always production ready
                 Releases tied to business needs, not
                 operational constraints
Tuesday, October 9, 12
continuous delivery

                 automation



                 patterns and practices



                 collaboration


Tuesday, October 9, 12
ingredients

                 configuration management



                 continuous integration



                 automated testing


Tuesday, October 9, 12
Local
               Develop
                           Workstation

                                            Mainline Server

                   Build

                                     pull


                   Build                        Build
                                                              ✔
                                                          Done!
                                  push

Tuesday, October 9, 12
Local
               Develop
                           Workstation

                Everyone Commits
                          Mainline Server

                   Build
                         To the pull
                                 Mainline
                   Build
                            Every Day
                                    Build
                                            ✔
                                            Done!
                                  push

Tuesday, October 9, 12
build quality in

                              “Cease dependence on
                              mass inspection to
                              achieve quality.
                              Improve the process
                              and build quality into
                              the product in the first
                              place”

                                 W. Edwards Deming
Tuesday, October 9, 12
different kinds of testing


                                                                  Business facing
                                                     AUTOMATED                       MANUAL

                                                                                Showcases
                          Support programming




                                                Functional acceptance
                                                                              Usability testing
                                                        tests




                                                                                                           Critique project
                                                                             Exploratory testing



                                                       Unit tests               Non-functional
                                                   Integration tests          acceptance tests
                                                     System tests         (performance, scaling, ...)
                                                     AUTOMATED               MANUAL / AUTOMATED

                                                                 Technology facing



                                                                                                        Diagram invented by Brian Marick

Tuesday, October 9, 12
deployment pipeline


                 an automated implementation of your
                 system’s build, deploy, test, release process

                 visibility

                 feedback

                 control

Tuesday, October 9, 12
deployment pipeline

                         Delivery team         Version control              Build & unit           Automated       User acceptance         Release
                                                                               tests            acceptance tests         tests

                                         Check in
                                                                  Trigger

                                                    Feedback


                                         Check in
                                                                  Trigger

                                                Feedback                                   Trigger




                                                               Feedback


                                     Check in
                                                                  Trigger

                                                    Feedback                               Trigger




                                                               Feedback                                      Approval
                                                                          Feedback                                              Approval




Tuesday, October 9, 12
deployment pipeline




Tuesday, October 9, 12
deployment pipeline




Tuesday, October 9, 12
reducing release risk



                 automate provisioning and deployment




                 ensure devs, testers and ops collaborate
                 throughout


Tuesday, October 9, 12
reducing release risk


                 devops


                 incrementalism


                 decoupling deployment and release


Tuesday, October 9, 12
devops
                            culture
                          automation
                         measurement
                            sharing




Tuesday, October 9, 12
feature toggles
                blue-green deployments

                                                 canary releases


               low risk releases
                are incremental
                         dark launching

                                          production immune system

Tuesday, October 9, 12
DEPENDENT SERVICE



                                                   1.0         1.1                      Database




                                                  Abstraction layer                 Abstraction layer



                         STATIC CONTENT


                                /static/1.1
                                                                      APPLICATION

                                /static/1.0




                                                 Router /
                                              Load balancer




                                                Interwebs




Tuesday, October 9, 12
blue-green
                         deployments


Tuesday, October 9, 12
router



                                   web      app       DB
                                  server   server   server
Tuesday, October 9, 12
v1.1     v1.1     v1.1

                         router



                                   web      app       DB
                                  server   server   server
Tuesday, October 9, 12
v1.1     v1.1     v1.1

                         router
                                   v1.2     v1.2     v1.2



                                   web      app       DB
                                  server   server   server
Tuesday, October 9, 12
v1.1     v1.1     v1.1

                         router
                                   v1.2     v1.2     v1.2



                                   web      app       DB
                                  server   server   server
Tuesday, October 9, 12
Canary Releasing


Tuesday, October 9, 12
Tuesday, October 9, 12
Tuesday, October 9, 12
canary releasing


                 Reduce risk of release


                 Multi-variant testing


                 Performance testing


Tuesday, October 9, 12
immune system


                         what if someone replaced your
                         “buy” button with spacer.gif?




                                 T cells http://www.flickr.com/photos/gehealthcare/3326186490/
Tuesday, October 9, 12
release !=
                         deployment


Tuesday, October 9, 12
feature toggles

                   Config File
             [featureToggles]
             wobblyFoobars: true
                                                                               some.php
             flightyForkHandles: false
                                                                   <?if ($wobblyFoobars) {?>
                                                                     ... various UI elements
                                                                   <?}?>




                         other.php
                         $fork_handle = ($featureConfig->isOn(‘flightlyForkHandles)) ?
                                         new flightyForkHander(aCandle) :
                                         new forkHandler(aCandle);




                                         Stolen from Martin Fowler http://martinfowler.com/bliki/FeatureToggle.html

Tuesday, October 9, 12
Dark Launching


Tuesday, October 9, 12
Tuesday, October 9, 12
Tuesday, October 9, 12
enterprise governance

   risk management

   SOX, ITIL, COBIT

   segregation of duties

   change management

   auditing and compliance

Tuesday, October 9, 12
people are the key

                 Get everyone together at the beginning

                 Keep meeting

                 Make it easy for everyone to see what’s
                 happening

                 Continuous improvement (kaizen)


Tuesday, October 9, 12
questions
        @jezhumble | jez@thoughtworks.com
        http://continuousdelivery.com/



        ThoughtWorks is hiring!
        http://jobs.thoughtworks.com/

        Australia | Brazil | Canada | China
        Germany | India | Singapore | South Africa
        Uganda | UK | USA


                                                            © 2012 ThoughtWorks, Inc.


                         http://thoughtworks-studios.com/
Tuesday, October 9, 12

Continuous Delivery

  • 1.
    continuous delivery @jezhumble Berlin, 4 october 2012 http://thoughtworks-studios.com/ Tuesday, October 9, 12
  • 2.
    agile 101 "Agile" team Analysis + Design Centralized QA IT Operations Development Integration + QA Release and operation Customer Testing + Showcase Iteration 0 1 2 3 4 The "last mile" Tuesday, October 9, 12
  • 3.
    social disrupting traditional businesses http://code.flickr.com/ Tuesday, October 9, 12
  • 4.
    releasing frequently 1. build the right thing Customer development Agile product development Eric Ries, “The Lean Startup” http://bit.ly/8ZoX5F Tuesday, October 9, 12
  • 5.
    innovate You can't just ask customers what they want and then try to give that to them. By the time you get it built, they'll want something new. Steve Jobs Tuesday, October 9, 12
  • 6.
    scientific method Ideas create hypothesis Learn Build deliver minimum viable product Data Code get feedback Measure (repeat) Eric Ries, “The Lean Startup” http://bit.ly/8ZoX5F Tuesday, October 9, 12
  • 7.
    ask this question “How long would it take your organization to deploy a change that involved just one single line of code? Do you do this on a repeatable, reliable basis?” Mary and Tom Poppendieck, Implementing Lean Software Development, p59. Tuesday, October 9, 12
  • 8.
    releasing frequently 1. build the right thing 2. reduce risk of release John Allspaw: “Ops Metametrics” http://slidesha.re/dsSZIr Tuesday, October 9, 12
  • 9.
  • 10.
    optimize for mtrs MTBF MTRS John Allspaw: “Building Resilience in Web Development and Operations” http://www.usievents.com/en/conferences/8-paris-usi-2011/sessions/968-john-allspaw Tuesday, October 9, 12
  • 11.
    releasing frequently 1. build the right thing 2. reduce risk of release 3. real project progress Tuesday, October 9, 12
  • 12.
    agile manifesto Our highest priority is to satisfy the customer through early and continuous delivery of valuable software Tuesday, October 9, 12
  • 13.
    production-ready software Fast, automated feedback on the production readiness of your applications every time there is a change - to code, infrastructure, or configuration Tuesday, October 9, 12
  • 14.
    continuous delivery Software always production ready Releases tied to business needs, not operational constraints Tuesday, October 9, 12
  • 15.
    continuous delivery automation patterns and practices collaboration Tuesday, October 9, 12
  • 16.
    ingredients configuration management continuous integration automated testing Tuesday, October 9, 12
  • 17.
    Local Develop Workstation Mainline Server Build pull Build Build ✔ Done! push Tuesday, October 9, 12
  • 18.
    Local Develop Workstation Everyone Commits Mainline Server Build To the pull Mainline Build Every Day Build ✔ Done! push Tuesday, October 9, 12
  • 19.
    build quality in “Cease dependence on mass inspection to achieve quality. Improve the process and build quality into the product in the first place” W. Edwards Deming Tuesday, October 9, 12
  • 20.
    different kinds oftesting Business facing AUTOMATED MANUAL Showcases Support programming Functional acceptance Usability testing tests Critique project Exploratory testing Unit tests Non-functional Integration tests acceptance tests System tests (performance, scaling, ...) AUTOMATED MANUAL / AUTOMATED Technology facing Diagram invented by Brian Marick Tuesday, October 9, 12
  • 21.
    deployment pipeline an automated implementation of your system’s build, deploy, test, release process visibility feedback control Tuesday, October 9, 12
  • 22.
    deployment pipeline Delivery team Version control Build & unit Automated User acceptance Release tests acceptance tests tests Check in Trigger Feedback Check in Trigger Feedback Trigger Feedback Check in Trigger Feedback Trigger Feedback Approval Feedback Approval Tuesday, October 9, 12
  • 23.
  • 24.
  • 25.
    reducing release risk automate provisioning and deployment ensure devs, testers and ops collaborate throughout Tuesday, October 9, 12
  • 26.
    reducing release risk devops incrementalism decoupling deployment and release Tuesday, October 9, 12
  • 27.
    devops culture automation measurement sharing Tuesday, October 9, 12
  • 28.
    feature toggles blue-green deployments canary releases low risk releases are incremental dark launching production immune system Tuesday, October 9, 12
  • 29.
    DEPENDENT SERVICE 1.0 1.1 Database Abstraction layer Abstraction layer STATIC CONTENT /static/1.1 APPLICATION /static/1.0 Router / Load balancer Interwebs Tuesday, October 9, 12
  • 30.
    blue-green deployments Tuesday, October 9, 12
  • 31.
    router web app DB server server server Tuesday, October 9, 12
  • 32.
    v1.1 v1.1 v1.1 router web app DB server server server Tuesday, October 9, 12
  • 33.
    v1.1 v1.1 v1.1 router v1.2 v1.2 v1.2 web app DB server server server Tuesday, October 9, 12
  • 34.
    v1.1 v1.1 v1.1 router v1.2 v1.2 v1.2 web app DB server server server Tuesday, October 9, 12
  • 35.
  • 36.
  • 37.
  • 38.
    canary releasing Reduce risk of release Multi-variant testing Performance testing Tuesday, October 9, 12
  • 39.
    immune system what if someone replaced your “buy” button with spacer.gif? T cells http://www.flickr.com/photos/gehealthcare/3326186490/ Tuesday, October 9, 12
  • 40.
    release != deployment Tuesday, October 9, 12
  • 41.
    feature toggles Config File [featureToggles] wobblyFoobars: true some.php flightyForkHandles: false <?if ($wobblyFoobars) {?> ... various UI elements <?}?> other.php $fork_handle = ($featureConfig->isOn(‘flightlyForkHandles)) ? new flightyForkHander(aCandle) : new forkHandler(aCandle); Stolen from Martin Fowler http://martinfowler.com/bliki/FeatureToggle.html Tuesday, October 9, 12
  • 42.
  • 43.
  • 44.
  • 45.
    enterprise governance risk management SOX, ITIL, COBIT segregation of duties change management auditing and compliance Tuesday, October 9, 12
  • 46.
    people are thekey Get everyone together at the beginning Keep meeting Make it easy for everyone to see what’s happening Continuous improvement (kaizen) Tuesday, October 9, 12
  • 47.
    questions @jezhumble | jez@thoughtworks.com http://continuousdelivery.com/ ThoughtWorks is hiring! http://jobs.thoughtworks.com/ Australia | Brazil | Canada | China Germany | India | Singapore | South Africa Uganda | UK | USA © 2012 ThoughtWorks, Inc. http://thoughtworks-studios.com/ Tuesday, October 9, 12