Continuous Delivery
                           Applied

Mike McGarr
jmcgarr@gmail.com
@jmichaelmcgarr
http://earlyandoften.wordpress.com
http://www.meetup.com/DC-continuous-integration/
About Me
• J. Michael (Mike) McGarr
• 12 years as a software engineer
• Director of DevOps for Learn, Blackboard
• Founder of the DC Continuous Integration,
  Delivery, and Deployment Meetup




                                              2
Continuous Delivery
   “Continuous delivery is a set
   of practices and principles
   aimed at, building, testing
   and releasing software
   faster and more frequently”

        cycle time




                                   3
4
Quality   or Speed



                     5
Cycle Time

“How long would it take your organization to
deploy a change [to production] that involves
just one single line of code? Do you do this on a
repeatable, reliable basis?”
                          - Mary and Tom Poppendieck,
                 Implementing Lean Software Development




                                                      6
The Last Mile

        Manual Deployments
Insufficient Configuration Management
Infrequent, Error Prone Deployments



                                        7
Stressful
Releases




        8
Frequent Automated
Deployments




                     http://flic.kr/p/29Ree
                                        9
Always Production Ready




                          10
Applying Continuous Delivery

Principles:                            Practices:

• Improve continuously                       Continuous Deployment (DevOps)
• Releasing software MUST be
  repeatable and reliable                              Build Pipelines
• Automate everything!
• If it hurts, do it more often                   Deployment Automation
• Keep everything in version control
• Done means “released”
• Build quality in!                    Configuration    Continuous
                                                                          Testing
• Everybody has responsibility for     Management       Integration
  the release process

                                                           Agile



                                                                                    11
Deployment Pipelines




http://www.fotopedia.com/users/chmehl   12
Deployment Pipelines

A Deployment Pipeline is an automated
manifestation of your process for getting
software from version control into the
hands of your users.




                     13
Deployment Pipelines
  (aka Build Pipelines)




                          14
Continuous what?


Continuous    Continuous   Continuous
Integration    Delivery    Deployment

   commit =    commit =      commit =
    a build    shippable    production
                product       release




                                         15
Getting Started




http://commons.wikimedia.org/wiki/User:VaGla/The_Human_Race_10k_2008_Warszawa
Understand your Organization




                               17
Understand your Value Stream




                               18
Visualize Flow




                 19
Small Batches




                20
DevOps




http://www.jedi.be/blog/2012/05/12/codifying-devops-area-practices/   21
It’s all about culture!




http://dev2ops.org/2012/10/devops-transformation-workshop-for-technical-managers/

                                                                                    22
Configuration Management




                           23
Version Control

• Version Everything

• Tool should stay out of the
  way

• Good Branching Strategy

• Don’t check-in binary files




                                   24
Automate the Build

• No IDE Builds!
• KISS
• Fail Fast (but smartly)
• No Wiki’s to Build
• Pick an appropriate
  Technology
• Keep it organized
• Dependency Management


                                 25
Build Once, Deploy Many




                          26
Binary Repositories




                      27
Testing




          28
Testing is not a Phase




                         http://flic.kr/p/6bcg   29
Types of Tests




                 30
Testing Pyramid




                  31
Unit Testing

Fast
Independent
Repeatable
Self-Verifying
Timely
                     32
Test Driven Development




http://reddevnews.com/articles/2007/11/01/testdriven-development-tdd.aspx   33
Specification by Example




                           34
Specification by Example




                           35
Automated Performance Testing




                                36
Static Code Analysis




       CheckStyle


                       37
Continuous
http://flic.kr/p/6hDAEK
                          Integration
Continuous Integration
Check-in Daily
Commit to Trunk
Automate the Build
Keep the Build Fast
Every Commit results in Build
Test in Clone of Production
Automate Deployment

                                  39
Build Pipeline in Jenkins




                            40
Deploying
        41
Automated Deployments




                        42
Code Deployments
       1.   Push New Binary to Server
       2.   Stop Server
       3.   Archive Old Binary
       4.   Deploy New Binary
       5.   Start Server




                                   43
Version your Database




                        44
Database Deployments

                      HTML    .sql
                                     Production
                DBA



  DB
Change                                  Test

  DB
           Version        Build
Change
           Control       Server

 DB                                     Dev
Change




                                                  45
46
Infrastructure as Code




                         47
Puppet




         48
Vagrant




          49
Architecture
and Design




           http://www.hdwallpapers.in/eiffel_tower_at_night_paris_france-wallpapers.html
Testable Design

                Coherence




Coupling




                            51
Modularization




    http://www.voltron.com/#/history/show/lion-force-voltron   52
Evolutionary Design




                      53
Continuous Deployment




                        54
Monitoring (sucks)




  https://github.com/monitoringsucks

                                       55
Key Takeaways

• Continuous Delivery is hard
• Fast Feedback is the key
• Small, frequent releases
• Version everything
• Automate everything
• Think culture
                                56
Contact Me


Mike McGarr
jmcgarr@gmail.com
@jmichaelmcgarr
http://earlyandoften.wordpress.com




                                     57
Further Reading
• Continuous Delivery: Reliable Software Releases through Build, Test
  and Deployment Automation, by Jez Humble and David Farley -
  http://www.amazon.com/Continuous-Delivery-Deployment-Automation-
  Addison-Wesley/dp/0321601912
• Continuous Integration, By Paul M Duvall -
  http://www.amazon.com/Continuous-Integration-Improving-Software-
  Reducing/dp/0321336380
• Refactoring Databases, By Scott Ambler and Pramod Sadalage -
  http://www.amazon.com/Refactoring-Databases-Evolutionary-
  paperback-Addison-Wesley/dp/0321774515
• Refactoring, By Martin Fowler - http://www.amazon.com/Refactoring-
  Improving-Design-Existing-Code/dp/0201485672
• Growing Object-Oriented Software, Guided by Tests, by Steven
  Freeman and Nat Pryce - http://www.amazon.com/Growing-Object-
  Oriented-Software-Guided-Tests/dp/0321503627


                                                                        58
Further Reading
• Extreme Programming Explained, by Kent Beck -
  http://www.amazon.com/Extreme-Programming-Explained-Embrace-
  Change/dp/0201616416
• Release It!, by Michael Nygard -
  http://pragprog.com/book/mnee/release-it
• xUnit Test Patterns, by Gerard Meszaros -
  http://www.amazon.com/xUnit-Test-Patterns-Refactoring-
  Code/dp/0131495054
• Working Effectively with Legacy Code, by Michael Feathers -
  http://www.amazon.com/Working-Effectively-Legacy-Michael-
  Feathers/dp/0131177052
• Agile Testing, by Lisa Crispin and Janet Gregory -
  http://www.amazon.com/Agile-Testing-Practical-Guide-
  Testers/dp/0321534468



                                                                 59
Further Reading
• Test Driven Development (TDD) – http://en.wikipedia.org/wiki/Test-
  driven_development
• Introducing BDD, by Dan North – http://dannorth.net/introducing-bdd/
• Agile Manifesto – http://agilemanifesto.org/
• Scrum – http://www.scrumalliance.org/learn_about_scrum
• Continuous Integration, by Martin Fowler –
  http://martinfowler.com/articles/continuousIntegration.html
• Specification by Example, by Gojko Adzic -
  http://specificationbyexample.com/
• Build Pipelines - http://www.magpiebrain.com/2009/12/13/a-brief-and-
  incomplete-history-of-build-pipelines/
• Implementing Lean Software Development, by Mary and Tom
  Poppendieck - http://www.amazon.com/Implementing-Lean-Software-
  Development-Concept/dp/0321437381



                                                                         60
Further Reading
• Maven Releases on Steriods, by Axel Fontaine –
  http://www.axelfontaine.com/2011/01/maven-releases-on-steroids-
  adios.html
• What is in a Name? Usually a version number, actually., by James
  Betteley - http://jamesbetteley.wordpress.com/2011/07/07/what-is-in-a-
  name-usually-a-version-number-actually/
• Build Once, Deploy Many -
  http://earlyandoften.wordpress.com/2010/09/09/build-once-deploy-
  many/
• Evolutionary Design - http://martinfowler.com/articles/designDead.html
• Continuous Deployment -
  http://timothyfitz.wordpress.com/2009/02/08/continuous-deployment/
• Sonar’s Technical Debt Calculation -
  http://www.sonarsource.org/evaluate-your-technical-debt-with-sonar/
• Gherkin - https://github.com/cucumber/cucumber/wiki/Gherkin


                                                                           61
Further Reading
• Branch by Abstraction, by Paul Hammant
  http://paulhammant.com/blog/branch_by_abstraction.html
• Making Large Scale Changes Incrementally with Branch by
  Abstraction, by Jez Humble -
  http://continuousdelivery.com/2011/05/make-large-scale-changes-
  incrementally-with-branch-by-abstraction/
• What is DevOps?, by Damon Edwards -
  http://dev2ops.org/2010/02/what-is-devops/
• Monitoring Sucks - https://github.com/monitoringsucks




                                                                    62
Tools
•   Git - http://git-scm.com/
•   Subversion - http://subversion.tigris.org/
•   Mercurial - http://mercurial.selenic.com/
•   Rational ClearCase - http://www-
    01.ibm.com/software/awdtools/clearcase/
•   Serena Dimensions CM - http://www.serena.com/products/dimensions-
    cm/index.html
•   Ant - http://ant.apache.org/
•   Ivy - http://ant.apache.org/ivy/
•   Maven - http://maven.apache.org/
•   Gradle - http://gradle.org/
•   JUnit – http://www.junit.org/
•   Mockito – http://code.google.com/p/mockito/
•   Hamcrest – http://code.google.com/p/hamcrest/
•   Spock – http://code.google.com/p/spock/
•   dbUnit – http://www.dbunit.org/
•   Unitils – http://unitils.org/summary.html
                                                                        63
Tools
•   Findbugs – http://findbugs.sourceforge.net/
•   PMD – http://pmd.sourceforge.net/
•   Checkstyle – http://checkstyle.sourceforge.net/
•   JIRA – http://www.atlassian.com/software/jira/overview
•   GitHub – https://github.com/
•   Jenkins - http://jenkins-ci.org/
•   TeamCity – http://www.jetbrains.com/teamcity/
•   Nexus – http://www.sonatype.org/nexus/
•   Artifactory – http://www.jfrog.com/products.php
•   Sonar – http://www.sonarsource.org/
•   FitNesse – http://fitnesse.org/
•   Concordion – http://www.concordion.org/
•   Cucumber – http://cukes.info/
•   easyb – http://www.easyb.org/
•   jBehave - http://jbehave.org/
•   geb - http://www.gebish.org/

                                                             64
Tools
•   Liquibase – www.liquibase.org/
•   Flyway – http://code.google.com/p/flyway/
•   Escape – http://code.google.com/p/escservesconfig/
•   Capistrano - https://github.com/capistrano/capistrano
•   Deployinator - https://github.com/etsy/deployinator
•   Puppet – http://puppetlabs.com/
•   Chef – http://www.opscode.com/chef/
•   Ansible - http://ansible.cc/
•   Vagrant – http://vagrantup.com/
•   JMeter – http://jmeter.apache.org/
•   Nagios - http://www.nagios.org/
•   Logstash - http://logstash.net/
•   StatsD – https://github.com/etsy/statsd
•   Metrics – http://metrics.codahale.com/
•   Ganglia – http://ganglia.sourceforge.net/
•   Graphite - http://graphite.wikidot.com/

                                                            65
Testing Pyramids
• http://fabiopereira.me/blog/2012/03/05/testing-pyramid-a-case-study/
• http://www.takadevelop.com/2013/02/09/testing-pyramid-for-agile-
  development/
• http://blogs.agilefaqs.com/2011/02/01/inverting-the-testing-pyramid/
• http://martinfowler.com/bliki/TestPyramid.html
• http://mifos.org/contributors/qa-testing/future-quality-improvements-
  mifos
• http://www.ibm.com/developerworks/java/library/j-aopwork11/




                                                                          66

Continuous delivery applied (RJUG)

  • 1.
    Continuous Delivery Applied Mike McGarr jmcgarr@gmail.com @jmichaelmcgarr http://earlyandoften.wordpress.com http://www.meetup.com/DC-continuous-integration/
  • 2.
    About Me • J.Michael (Mike) McGarr • 12 years as a software engineer • Director of DevOps for Learn, Blackboard • Founder of the DC Continuous Integration, Delivery, and Deployment Meetup 2
  • 3.
    Continuous Delivery “Continuous delivery is a set of practices and principles aimed at, building, testing and releasing software faster and more frequently” cycle time 3
  • 4.
  • 5.
    Quality or Speed 5
  • 6.
    Cycle Time “How longwould it take your organization to deploy a change [to production] that involves just one single line of code? Do you do this on a repeatable, reliable basis?” - Mary and Tom Poppendieck, Implementing Lean Software Development 6
  • 7.
    The Last Mile Manual Deployments Insufficient Configuration Management Infrequent, Error Prone Deployments 7
  • 8.
  • 9.
    Frequent Automated Deployments http://flic.kr/p/29Ree 9
  • 10.
  • 11.
    Applying Continuous Delivery Principles: Practices: • Improve continuously Continuous Deployment (DevOps) • Releasing software MUST be repeatable and reliable Build Pipelines • Automate everything! • If it hurts, do it more often Deployment Automation • Keep everything in version control • Done means “released” • Build quality in! Configuration Continuous Testing • Everybody has responsibility for Management Integration the release process Agile 11
  • 12.
  • 13.
    Deployment Pipelines A DeploymentPipeline is an automated manifestation of your process for getting software from version control into the hands of your users. 13
  • 14.
    Deployment Pipelines (aka Build Pipelines) 14
  • 15.
    Continuous what? Continuous Continuous Continuous Integration Delivery Deployment commit = commit = commit = a build shippable production product release 15
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
    It’s all aboutculture! http://dev2ops.org/2012/10/devops-transformation-workshop-for-technical-managers/ 22
  • 23.
  • 24.
    Version Control • VersionEverything • Tool should stay out of the way • Good Branching Strategy • Don’t check-in binary files 24
  • 25.
    Automate the Build •No IDE Builds! • KISS • Fail Fast (but smartly) • No Wiki’s to Build • Pick an appropriate Technology • Keep it organized • Dependency Management 25
  • 26.
  • 27.
  • 28.
  • 29.
    Testing is nota Phase http://flic.kr/p/6bcg 29
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
    Static Code Analysis CheckStyle 37
  • 38.
  • 39.
    Continuous Integration Check-in Daily Committo Trunk Automate the Build Keep the Build Fast Every Commit results in Build Test in Clone of Production Automate Deployment 39
  • 40.
  • 41.
  • 42.
  • 43.
    Code Deployments 1. Push New Binary to Server 2. Stop Server 3. Archive Old Binary 4. Deploy New Binary 5. Start Server 43
  • 44.
  • 45.
    Database Deployments HTML .sql Production DBA DB Change Test DB Version Build Change Control Server DB Dev Change 45
  • 46.
  • 47.
  • 48.
  • 49.
  • 50.
    Architecture and Design http://www.hdwallpapers.in/eiffel_tower_at_night_paris_france-wallpapers.html
  • 51.
    Testable Design Coherence Coupling 51
  • 52.
    Modularization http://www.voltron.com/#/history/show/lion-force-voltron 52
  • 53.
  • 54.
  • 55.
    Monitoring (sucks) https://github.com/monitoringsucks 55
  • 56.
    Key Takeaways • ContinuousDelivery is hard • Fast Feedback is the key • Small, frequent releases • Version everything • Automate everything • Think culture 56
  • 57.
  • 58.
    Further Reading • ContinuousDelivery: Reliable Software Releases through Build, Test and Deployment Automation, by Jez Humble and David Farley - http://www.amazon.com/Continuous-Delivery-Deployment-Automation- Addison-Wesley/dp/0321601912 • Continuous Integration, By Paul M Duvall - http://www.amazon.com/Continuous-Integration-Improving-Software- Reducing/dp/0321336380 • Refactoring Databases, By Scott Ambler and Pramod Sadalage - http://www.amazon.com/Refactoring-Databases-Evolutionary- paperback-Addison-Wesley/dp/0321774515 • Refactoring, By Martin Fowler - http://www.amazon.com/Refactoring- Improving-Design-Existing-Code/dp/0201485672 • Growing Object-Oriented Software, Guided by Tests, by Steven Freeman and Nat Pryce - http://www.amazon.com/Growing-Object- Oriented-Software-Guided-Tests/dp/0321503627 58
  • 59.
    Further Reading • ExtremeProgramming Explained, by Kent Beck - http://www.amazon.com/Extreme-Programming-Explained-Embrace- Change/dp/0201616416 • Release It!, by Michael Nygard - http://pragprog.com/book/mnee/release-it • xUnit Test Patterns, by Gerard Meszaros - http://www.amazon.com/xUnit-Test-Patterns-Refactoring- Code/dp/0131495054 • Working Effectively with Legacy Code, by Michael Feathers - http://www.amazon.com/Working-Effectively-Legacy-Michael- Feathers/dp/0131177052 • Agile Testing, by Lisa Crispin and Janet Gregory - http://www.amazon.com/Agile-Testing-Practical-Guide- Testers/dp/0321534468 59
  • 60.
    Further Reading • TestDriven Development (TDD) – http://en.wikipedia.org/wiki/Test- driven_development • Introducing BDD, by Dan North – http://dannorth.net/introducing-bdd/ • Agile Manifesto – http://agilemanifesto.org/ • Scrum – http://www.scrumalliance.org/learn_about_scrum • Continuous Integration, by Martin Fowler – http://martinfowler.com/articles/continuousIntegration.html • Specification by Example, by Gojko Adzic - http://specificationbyexample.com/ • Build Pipelines - http://www.magpiebrain.com/2009/12/13/a-brief-and- incomplete-history-of-build-pipelines/ • Implementing Lean Software Development, by Mary and Tom Poppendieck - http://www.amazon.com/Implementing-Lean-Software- Development-Concept/dp/0321437381 60
  • 61.
    Further Reading • MavenReleases on Steriods, by Axel Fontaine – http://www.axelfontaine.com/2011/01/maven-releases-on-steroids- adios.html • What is in a Name? Usually a version number, actually., by James Betteley - http://jamesbetteley.wordpress.com/2011/07/07/what-is-in-a- name-usually-a-version-number-actually/ • Build Once, Deploy Many - http://earlyandoften.wordpress.com/2010/09/09/build-once-deploy- many/ • Evolutionary Design - http://martinfowler.com/articles/designDead.html • Continuous Deployment - http://timothyfitz.wordpress.com/2009/02/08/continuous-deployment/ • Sonar’s Technical Debt Calculation - http://www.sonarsource.org/evaluate-your-technical-debt-with-sonar/ • Gherkin - https://github.com/cucumber/cucumber/wiki/Gherkin 61
  • 62.
    Further Reading • Branchby Abstraction, by Paul Hammant http://paulhammant.com/blog/branch_by_abstraction.html • Making Large Scale Changes Incrementally with Branch by Abstraction, by Jez Humble - http://continuousdelivery.com/2011/05/make-large-scale-changes- incrementally-with-branch-by-abstraction/ • What is DevOps?, by Damon Edwards - http://dev2ops.org/2010/02/what-is-devops/ • Monitoring Sucks - https://github.com/monitoringsucks 62
  • 63.
    Tools • Git - http://git-scm.com/ • Subversion - http://subversion.tigris.org/ • Mercurial - http://mercurial.selenic.com/ • Rational ClearCase - http://www- 01.ibm.com/software/awdtools/clearcase/ • Serena Dimensions CM - http://www.serena.com/products/dimensions- cm/index.html • Ant - http://ant.apache.org/ • Ivy - http://ant.apache.org/ivy/ • Maven - http://maven.apache.org/ • Gradle - http://gradle.org/ • JUnit – http://www.junit.org/ • Mockito – http://code.google.com/p/mockito/ • Hamcrest – http://code.google.com/p/hamcrest/ • Spock – http://code.google.com/p/spock/ • dbUnit – http://www.dbunit.org/ • Unitils – http://unitils.org/summary.html 63
  • 64.
    Tools • Findbugs – http://findbugs.sourceforge.net/ • PMD – http://pmd.sourceforge.net/ • Checkstyle – http://checkstyle.sourceforge.net/ • JIRA – http://www.atlassian.com/software/jira/overview • GitHub – https://github.com/ • Jenkins - http://jenkins-ci.org/ • TeamCity – http://www.jetbrains.com/teamcity/ • Nexus – http://www.sonatype.org/nexus/ • Artifactory – http://www.jfrog.com/products.php • Sonar – http://www.sonarsource.org/ • FitNesse – http://fitnesse.org/ • Concordion – http://www.concordion.org/ • Cucumber – http://cukes.info/ • easyb – http://www.easyb.org/ • jBehave - http://jbehave.org/ • geb - http://www.gebish.org/ 64
  • 65.
    Tools • Liquibase – www.liquibase.org/ • Flyway – http://code.google.com/p/flyway/ • Escape – http://code.google.com/p/escservesconfig/ • Capistrano - https://github.com/capistrano/capistrano • Deployinator - https://github.com/etsy/deployinator • Puppet – http://puppetlabs.com/ • Chef – http://www.opscode.com/chef/ • Ansible - http://ansible.cc/ • Vagrant – http://vagrantup.com/ • JMeter – http://jmeter.apache.org/ • Nagios - http://www.nagios.org/ • Logstash - http://logstash.net/ • StatsD – https://github.com/etsy/statsd • Metrics – http://metrics.codahale.com/ • Ganglia – http://ganglia.sourceforge.net/ • Graphite - http://graphite.wikidot.com/ 65
  • 66.
    Testing Pyramids • http://fabiopereira.me/blog/2012/03/05/testing-pyramid-a-case-study/ •http://www.takadevelop.com/2013/02/09/testing-pyramid-for-agile- development/ • http://blogs.agilefaqs.com/2011/02/01/inverting-the-testing-pyramid/ • http://martinfowler.com/bliki/TestPyramid.html • http://mifos.org/contributors/qa-testing/future-quality-improvements- mifos • http://www.ibm.com/developerworks/java/library/j-aopwork11/ 66

Editor's Notes

  • #6 Most orgs struggle to balance…Viewed as mutually exclusive
  • #7 Cycle Time is a measure of Continuous Delivery maturity
  • #8 Collaboration (DEV + Customer, DEV + Operations)Talking early and often…break down walls. [DevOps]Automation. Automate Everything! Build, Test, Deploy!Frequency. Frequent means small. Easier to troubleshoot, rollback. Reduces Risk.Feedback. Change triggers feedback. Feedback is fast. Team acts on it.
  • #9 Manual DeploymentsInsufficient Configuration ManagementInfrequent, Error Prone Deployments
  • #12 Continuous improvementAutomation everythingVersion everythingQuality quality qualityShared responsibility