RECIPES FOR
CONTINUOUS DELIVERY
Gurpreet Luthra - Lead Consultant, ThoughtWorks
1
_zenx_
EXPERIMENTATION
2
TOPICS
3
• Continuous Integration (CI)
• Continuous Deployment (CD?) / Continuous Delivery (CD)
• Phoenix Servers
• Branching for Release
• Feature Branches
• Trunk Based Development
• Pull Request Model
• Toggles
• Branch by Abstraction & Strangulation
4
CONTINUOUS INTEGRATION
https://www.thoughtworks.com/continuous-integration
5
https://www.martinfowler.com/articles/continuousIntegration.html
May 2006!
6
From: https://www.youtube.com/watch?v=HnWuIjUw_Q8
THE PRACTICES
7
• Maintain a single source repository
• Automate the build
• Make your build self-testing
• Every commit should build on an integration machine
• Keep the build fast
• Test in a clone of the production environment
• Make it easy for anyone to get the latest executable
• Everyone can see what’s happening
• Automate deployment
HOW TO DO IT
8
• Developers check out code into their private workspaces.
• When done, commit the changes to the repository.
• The CI server monitors the repository and checks out
changes when they occur.
• The CI server builds the system and runs unit and
integration tests.
• The CI server releases deployable artefacts for testing.
• The CI server assigns a build label to the version of the
code it just built.
• The CI server informs the team of the successful build.
TEAM RESPONSIBILITIES
9
• Check in frequently
• Don’t check in broken code
• Don’t check in untested code
• Don’t check in when the build is broken
• Don’t go home after checking in until the system builds
Coffee Cup Story!
Unmesh Joshi
(Head of Tech - TWI)
THE UNDERLYING ASSUMPTION
10
• The quality and impact of your CI processes are solely
dependant on the quality of your TESTS.
• If you don’t write good quality tests, with reasonable
coverage, all the CI infrastructure in the world can’t
protect you.
WRITE GOOD TESTS!!!
11
DELIVERY OR DEPLOYMENT?
12
Image from: http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment
CONTINUOUS DEPLOYMENT
13
https://www.quora.com/What-are-the-best-examples-of-companies-using-continuous-
deployment
PHOENIX SERVERS
14
• Configuration Drift
• Snowflake servers
• No SSH servers
• Immutable Servers
• Phoenix servers
PLEASE READ: http://martinfowler.com/bliki/PhoenixServer.html
and http://martinfowler.com/bliki/ImmutableServer.html
VM (OR CONTAINERS) AS BUILD AGENTS
15
SLIDE COPIED FROM: http://decks.eric.pe/pantheon-ci
BUILD AGENTS AS PHONEIX SERVERS
16
BUILD AGENTS AS PHONEIX SERVERS
17
BUILD AGENTS AS PHONEIX SERVERS
18
PHOENIX SERVERS
19
Diagram and full article at: https://boxfuse.com/blog/no-ssh
PROMOTE IMAGES
20
BOOKS
21
BRANCHING
22
(Painting by Chitra S)
BRANCHING FOR RELEASE
23
Image from: http://paulhammant.com/blog/branch_by_abstraction.html
BRANCHING FOR RELEASE
24
FOR EACH RELEASE HOW MUCH EFFORT DO YOU PUT
IN TO CREATE CI PIPELINES ?
BRANCHING FOR RELEASE
25From: http://www.bogotobogo.com/DevOps/DevOps_CI_CD_Pipeline_Sample.php
FEATURES BRANCHES
26
Image from: http://blogs.riskotech.com/riskfactor/post/An-Alternate-Branching-Strategy.aspx
THE REALITY OF FEATURE BRANCHES
27
Image from: http://paulhammant.com/blog/branch_by_abstraction.html
Martin Fowler- Promiscuous Integration: https://dzone.com/articles/promiscuous-integration-vs
BRANCH TRACKING
28
A GIT BRANCHING MODEL
29
Image from: http://nvie.com/posts/a-successful-git-branching-model/
TRUNK BASED DEVELOPMENT
30
• Trunk Based Development (TBD) is where all developers
(for a particular deployable unit) commit to one shared
branch under source-control called “trunk” or “mainline”.
• Branches are made only for a release in the shared
repository.
• Trunk always stays GREEN.
• Developers never break the build with any commit.
Alternatively, broken commits are immediately rolled
back. Or such commits may be blocked before being
merged with the trunk.
• Developers don’t work in ISOLATION, in their “happy
branches” for days. Update your workstation daily!
TRUNK BASED DEVELOPMENT
31
Image from: http://paulhammant.com/2014/01/08/googles-vs-facebooks-trunk-based-development/
GOOGLE MONOREPO
32
Rachel Potvin: https://trunkbaseddevelopment.com/game-changers/index.html#google-
revealing-their-monorepo-trunk-2016
Piper + CitC
GOOGLE MONOREPO
33
Rachel Potvin: https://trunkbaseddevelopment.com/game-changers/index.html#google-
revealing-their-monorepo-trunk-2016
Piper + CitC
PULL REQUEST MODEL
34
master
fork’s master
Fork
branch
feature-1 branch
branch
Raise pull request
Update to latest
Raise pull request
feature-2 branch
review + no mergereview + merge
review + merge
PULL REQUEST MODEL
35
Image from: code.tutsplus.com/tutorials/travis-ci-what-why-how--net-34771
TRUNK BASED DEVELOPMENT
36
Feature branching is a poor man’s modular architecture,
instead of building systems with the ability to easily swap in
and out features at runtime/deploy time, you’re coupling
yourself to the source control providing this mechanism
through manual merging.
- Dan Bodart
Mentioned By
http://martinfowler.com/bliki/FeatureBranch.html
AND
http://sethgoings.com/feature-branching
TRUNK BASED DEVELOPMENT
37
OK. I WILL TRY & AVOID FEATURE BRANCHES. HOW DO
I ENSURE I CAN DEVELOP BIG FEATURES, YET, KEEP
TRUNK GREEN & STABLE?
FEATURE TOGGLES
38
FEATURE TOGGLES
39
• Release Toggles
• Business Toggles
• Build Time Toggles
• Run Time Toggles
• A/B Testing
• Canary Releases
• Rollback in production
PLEASE READ
http://martinfowler.com/articles/feature-toggles.html
Pete Hodgson
EXAMPLES
40
1. Introduce a simple search box on your home page
2. Add a new feature to favorite products + My Favorite page (client side
only, nothing on server side)
3. Upgrade from Java7 and Java 8 (slowly across servers)
4. Improve your search algo (in case it returns no results, then apply,
distance algo)
5. Change upload logic for images (earlier each image was assigned a
thread). New logic: Create a queue, perform a de-dup, assign to a thread,
batch size of 5, upload.
6. Add FB Integration (FB Like + Count -- pulled from FB)
7. Change JS code to use Require.JS for dependencies (shim/polyfill/etc)
8. Introduce dependency injection via Spring/Guice
9. Change Repository code from Hibernate to Toplink
IMPLEMENTATION
41
• Simple IF statement (top most layer)
• DB / File / Code based configuration
• Polymorphic Substitution / Injection
• Plugin Based / Dynamic JAR Lookup / Discovery
• Hook based extensions
• Toggle Based Frameworks (Use Google!)
• Many more..
BRANCH BY ABSTRACTION
42
IMAGES AND ARTICLE:
http://martinfowler.com/bliki/BranchByAbstraction.html
BRANCH BY ABSTRACTION
43
• Technique for making large scale changes in a gradual way
• Use an abstraction layer to allow multiple implementations to
co-exist
• Use the notion of one abstraction and multiple
implementations to perform the migration from one
implementation to the other
• Ensure that the system builds and runs correctly at all times
YES. IT’S JUST GOOD OBJECT ORIENTED DESIGN.
STRANGULATION PATTERN
44
• Slowly strangulate one system for another.
• Slow strangulate one model / one component for another.
PLEASE READ
http://agilefromthegroundup.blogspot.in/2011/03/strangulation-pattern-of-choice-for.html
IMAGE FROM:
https://dzone.com/articles/legacy-java-applications
WHERE TO NEXT?
45
• Write good tests. Make them blazing fast. Depend on them!
• Commit & push frequently to Trunk. Build should be ALWAYS
Green.
• Every morning, pull latest code to your machine.
• Consider using Immutable Servers / Phoenix Servers.
• Figure out creative ways to keep software ready-for-release.
• Plan for rollbacks, or for functionality switches.
• Consider strategies like Toggles or Branch-by-Abstraction.
• Don’t work in ISOLATED branches.
• Be thoughtful in your code… and in your design.
AIM FOR Continuous Delivery!
THANK YOU
46
Gurpreet Luthra - Lead Consultant, ThoughtWorks
_zenx_
Thank you for all your thoughts on CI & CD by
Martin Fowler, Paul Hammant, Jez Humble, Pete Hodgson, ThoughtWorkers
and
of course Calvin & Hobbes

Recipes for Continuous Delivery (ThoughtWorks Geeknight)

  • 1.
    RECIPES FOR CONTINUOUS DELIVERY GurpreetLuthra - Lead Consultant, ThoughtWorks 1 _zenx_
  • 2.
  • 3.
    TOPICS 3 • Continuous Integration(CI) • Continuous Deployment (CD?) / Continuous Delivery (CD) • Phoenix Servers • Branching for Release • Feature Branches • Trunk Based Development • Pull Request Model • Toggles • Branch by Abstraction & Strangulation
  • 4.
  • 5.
  • 6.
  • 7.
    THE PRACTICES 7 • Maintaina single source repository • Automate the build • Make your build self-testing • Every commit should build on an integration machine • Keep the build fast • Test in a clone of the production environment • Make it easy for anyone to get the latest executable • Everyone can see what’s happening • Automate deployment
  • 8.
    HOW TO DOIT 8 • Developers check out code into their private workspaces. • When done, commit the changes to the repository. • The CI server monitors the repository and checks out changes when they occur. • The CI server builds the system and runs unit and integration tests. • The CI server releases deployable artefacts for testing. • The CI server assigns a build label to the version of the code it just built. • The CI server informs the team of the successful build.
  • 9.
    TEAM RESPONSIBILITIES 9 • Checkin frequently • Don’t check in broken code • Don’t check in untested code • Don’t check in when the build is broken • Don’t go home after checking in until the system builds Coffee Cup Story! Unmesh Joshi (Head of Tech - TWI)
  • 10.
    THE UNDERLYING ASSUMPTION 10 •The quality and impact of your CI processes are solely dependant on the quality of your TESTS. • If you don’t write good quality tests, with reasonable coverage, all the CI infrastructure in the world can’t protect you. WRITE GOOD TESTS!!!
  • 11.
  • 12.
    DELIVERY OR DEPLOYMENT? 12 Imagefrom: http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment
  • 13.
  • 14.
    PHOENIX SERVERS 14 • ConfigurationDrift • Snowflake servers • No SSH servers • Immutable Servers • Phoenix servers PLEASE READ: http://martinfowler.com/bliki/PhoenixServer.html and http://martinfowler.com/bliki/ImmutableServer.html
  • 15.
    VM (OR CONTAINERS)AS BUILD AGENTS 15 SLIDE COPIED FROM: http://decks.eric.pe/pantheon-ci
  • 16.
    BUILD AGENTS ASPHONEIX SERVERS 16
  • 17.
    BUILD AGENTS ASPHONEIX SERVERS 17
  • 18.
    BUILD AGENTS ASPHONEIX SERVERS 18
  • 19.
    PHOENIX SERVERS 19 Diagram andfull article at: https://boxfuse.com/blog/no-ssh
  • 20.
  • 21.
  • 22.
  • 23.
    BRANCHING FOR RELEASE 23 Imagefrom: http://paulhammant.com/blog/branch_by_abstraction.html
  • 24.
    BRANCHING FOR RELEASE 24 FOREACH RELEASE HOW MUCH EFFORT DO YOU PUT IN TO CREATE CI PIPELINES ?
  • 25.
    BRANCHING FOR RELEASE 25From:http://www.bogotobogo.com/DevOps/DevOps_CI_CD_Pipeline_Sample.php
  • 26.
    FEATURES BRANCHES 26 Image from:http://blogs.riskotech.com/riskfactor/post/An-Alternate-Branching-Strategy.aspx
  • 27.
    THE REALITY OFFEATURE BRANCHES 27 Image from: http://paulhammant.com/blog/branch_by_abstraction.html Martin Fowler- Promiscuous Integration: https://dzone.com/articles/promiscuous-integration-vs
  • 28.
  • 29.
    A GIT BRANCHINGMODEL 29 Image from: http://nvie.com/posts/a-successful-git-branching-model/
  • 30.
    TRUNK BASED DEVELOPMENT 30 •Trunk Based Development (TBD) is where all developers (for a particular deployable unit) commit to one shared branch under source-control called “trunk” or “mainline”. • Branches are made only for a release in the shared repository. • Trunk always stays GREEN. • Developers never break the build with any commit. Alternatively, broken commits are immediately rolled back. Or such commits may be blocked before being merged with the trunk. • Developers don’t work in ISOLATION, in their “happy branches” for days. Update your workstation daily!
  • 31.
    TRUNK BASED DEVELOPMENT 31 Imagefrom: http://paulhammant.com/2014/01/08/googles-vs-facebooks-trunk-based-development/
  • 32.
    GOOGLE MONOREPO 32 Rachel Potvin:https://trunkbaseddevelopment.com/game-changers/index.html#google- revealing-their-monorepo-trunk-2016 Piper + CitC
  • 33.
    GOOGLE MONOREPO 33 Rachel Potvin:https://trunkbaseddevelopment.com/game-changers/index.html#google- revealing-their-monorepo-trunk-2016 Piper + CitC
  • 34.
    PULL REQUEST MODEL 34 master fork’smaster Fork branch feature-1 branch branch Raise pull request Update to latest Raise pull request feature-2 branch review + no mergereview + merge review + merge
  • 35.
    PULL REQUEST MODEL 35 Imagefrom: code.tutsplus.com/tutorials/travis-ci-what-why-how--net-34771
  • 36.
    TRUNK BASED DEVELOPMENT 36 Featurebranching is a poor man’s modular architecture, instead of building systems with the ability to easily swap in and out features at runtime/deploy time, you’re coupling yourself to the source control providing this mechanism through manual merging. - Dan Bodart Mentioned By http://martinfowler.com/bliki/FeatureBranch.html AND http://sethgoings.com/feature-branching
  • 37.
    TRUNK BASED DEVELOPMENT 37 OK.I WILL TRY & AVOID FEATURE BRANCHES. HOW DO I ENSURE I CAN DEVELOP BIG FEATURES, YET, KEEP TRUNK GREEN & STABLE?
  • 38.
  • 39.
    FEATURE TOGGLES 39 • ReleaseToggles • Business Toggles • Build Time Toggles • Run Time Toggles • A/B Testing • Canary Releases • Rollback in production PLEASE READ http://martinfowler.com/articles/feature-toggles.html Pete Hodgson
  • 40.
    EXAMPLES 40 1. Introduce asimple search box on your home page 2. Add a new feature to favorite products + My Favorite page (client side only, nothing on server side) 3. Upgrade from Java7 and Java 8 (slowly across servers) 4. Improve your search algo (in case it returns no results, then apply, distance algo) 5. Change upload logic for images (earlier each image was assigned a thread). New logic: Create a queue, perform a de-dup, assign to a thread, batch size of 5, upload. 6. Add FB Integration (FB Like + Count -- pulled from FB) 7. Change JS code to use Require.JS for dependencies (shim/polyfill/etc) 8. Introduce dependency injection via Spring/Guice 9. Change Repository code from Hibernate to Toplink
  • 41.
    IMPLEMENTATION 41 • Simple IFstatement (top most layer) • DB / File / Code based configuration • Polymorphic Substitution / Injection • Plugin Based / Dynamic JAR Lookup / Discovery • Hook based extensions • Toggle Based Frameworks (Use Google!) • Many more..
  • 42.
    BRANCH BY ABSTRACTION 42 IMAGESAND ARTICLE: http://martinfowler.com/bliki/BranchByAbstraction.html
  • 43.
    BRANCH BY ABSTRACTION 43 •Technique for making large scale changes in a gradual way • Use an abstraction layer to allow multiple implementations to co-exist • Use the notion of one abstraction and multiple implementations to perform the migration from one implementation to the other • Ensure that the system builds and runs correctly at all times YES. IT’S JUST GOOD OBJECT ORIENTED DESIGN.
  • 44.
    STRANGULATION PATTERN 44 • Slowlystrangulate one system for another. • Slow strangulate one model / one component for another. PLEASE READ http://agilefromthegroundup.blogspot.in/2011/03/strangulation-pattern-of-choice-for.html IMAGE FROM: https://dzone.com/articles/legacy-java-applications
  • 45.
    WHERE TO NEXT? 45 •Write good tests. Make them blazing fast. Depend on them! • Commit & push frequently to Trunk. Build should be ALWAYS Green. • Every morning, pull latest code to your machine. • Consider using Immutable Servers / Phoenix Servers. • Figure out creative ways to keep software ready-for-release. • Plan for rollbacks, or for functionality switches. • Consider strategies like Toggles or Branch-by-Abstraction. • Don’t work in ISOLATED branches. • Be thoughtful in your code… and in your design. AIM FOR Continuous Delivery!
  • 46.
    THANK YOU 46 Gurpreet Luthra- Lead Consultant, ThoughtWorks _zenx_ Thank you for all your thoughts on CI & CD by Martin Fowler, Paul Hammant, Jez Humble, Pete Hodgson, ThoughtWorkers and of course Calvin & Hobbes