百倍速交付
~ 談Trunk-Based Development
Bryan Liu 劉栢峰 2017
Some Definitions
Long-Lived (Feature) Branch
• Creates lots of overhead
Git Flow
GitHub Flow
Short-Lived Feature Branch
• Branch’s length of life: less than 2 days
• Shared for code-review and CI only
• Easily becomes long-lived branch
https://trunkbaseddevelopment.com/short-lived-feature-branches/
Trunk-Based Development (TBD)
• Commit in trunk at least once a day
• Trunk is always in a releasable state
• Hide unfinished code with feature toggle
• Refactor with branch by abstraction
http://www.alwaysagileconsulting.com/articles/organisation-pattern-trunk-based-development/
Facts
Always the bridesmaid* from Paul Hammant
TBD
Only search few of these kings…
Also other TBD giants, ex: Google, Netflix
were not included in this startups research.
Never mind become A Category King
2000 ~ 2015 yr
600k companies
34 of them
76 % market cap
Winner-takes-all
Category King
‘‘Somewhere in the middle of this two-month process you’re going to run
across a bridge and burn it behind you.’’
‘‘LinkedIn’s big switch to continuous deployment has been linked to very
concrete and visible financial success…’’
~ Kevin Scott, the senior VP of engineering
http://www.wired.com/2013/04/linkedin-software-revolution/
https://paulhammant.com/2013/03/13/facebook-tbd-take-2/
Facebook’s
Trunk-Based Development
branching models
v.s.
release frequency
TBD & Feature Toggle
if ( ToggleManager.isEnabled(“FastCheckout”, user) ) {
useFastCheckout()
} else {
useOriginCheckout()
}
https://martinfowler.com/articles/feature-toggles.html
What’s Toggle
Business Toggle:
• A/B Testing
Low Risk Release and Zero-Downtime
• Canary Release, Dark Launch, Blue-Green deployment
• DB/Schema Migration
Decouple Feature Rollout from Code Deployment
• Re-planning
• Microservices
Some Toggle Use Cases
Canary Release / Blue-Green Deployment & A / B Testing
http://featureflags.io/
• Phased rollout
• Monitor performance
• Check user feedback
• Features in live but NOT visible to user
• Warm up cache, load DB tables with real
production traffic
• Soak test, verify performance
Dark Launch
DB / Schema Change
• Forwards & backwards compatible
• Stage roll-out with verification online
• No downtime needed
Q:
What are the requirements
to achieve previous scenarios ?
A:
• Feature toggles &
• Old and new code co-exists in the same delivery
“Get good with build-flags and
toggles” ~ Paul Hammant
“ This is also the major difference
between short-lived branch and TBD!”
Inefficiencies ...
Decouple Release from Code Deployment
• TBD reduce merge pain
• Merge hell is error prone and risky
• Inefficiency on spending time in bug fix - merge - test cycle.
As in Microservices:
Any attempt trying to set release pace is not realistic!!
Feature flags
+ Microservices
= True decoupling
Re-planning
• It happens
• Leads to weeks of un-merge for teams
Image from: https://devops.paulhammant.com/
Inefficiencies in SW development:
• It slows down throughput
• Can’t keep talent
Spotify Release Train
https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/
Frequently commits to mainline:
• More communication with your team
• More frequent builds and tests (CI)
• More frequent deployment (CD)
• More experiments and faster feedback
Implementation
Pre-Commit Testing & Code Review
• Pre-commit (CI) test (option but best)
• Continuous (code) review
• Code ownership
Figure from https://trunkbaseddevelopment.com/continuous-integration/
Why Continuous Review (Large Diffs Hurts)
• Large diffs won’t get reviewed
• Reverting large diffs is hard
• Rebasing/merging large diffs is error-prone
Code review statistics: http://blogs.atlassian.com/2011/07/creating_optimal_reviews/
Pull Request Code Review
git commit -m "fixed issue with fan"
https://zachholman.com/posts/deploying-software
Code Review Page
• Code statistics
• CI test results
• Review discussion &
comments
Hello, about me …
DevOps Evangelist
QA Architect
Bryan Liu 劉栢峰
• trunkbaseddevelopment.com

• trunk-based development, Paul Hammant

• Facebook's Trunk-Based Development, Paul Hammant

• Branch by Abstraction, Martin Fowler

• Feature Toggles, martinfowler.com

• Category King, J. Van Grove and Play Bigger

• The Death of Continuous Integration, Steve Smith

• And many links inside each slide deck ~
Reference

Trunk-Based Development