Dave has three decades of experience in developer tools, developer communities, and evangelizing sustainable software delivery practices. He has held programming, product management, and product marketing roles at Sun Microsystems, Gupta Technologies, Remedy Software, Marimba, Keynote Systems (Dynatrace), SOASTA, and BlazeMeter. Dave’s current passion is demystifying progressive delivery, especially the ways it enables better outcomes by removing constraints and building-in feedback loops. Speech Overview: We build pipelines to automate processes and minimize human toil. Have you applied this same approach to how you expose new features to users and measure the impact? Progressive delivery may be relatively new as a term, but the underlying practices of progressive experimentation (where features are gradually rolled out to users and statistical engines are used to detect impact during the rollout instead of after) are not. We’ll discuss the layers, and benefits, from the foundation up.

1. Powering Progressive
Delivery With Data
@SplitSoftware
A Layered Approach

2. Do you do this when
your team releases
to production?
Evan-Amos, CC BY-SA 3.0 <https://creativecommons.org/licenses/by-sa/3.0>, via Wikimedia Commons
https://upload.wikimedia.org/wikipedia/commons/a/ac/Hands-Fingers-Crossed.jpg

3. Can you remember
a release night
that went like this?
https://imgflip.com/user/Knightrogens
https://imgflip.com/i/1mp8tq

4. How do you respond
when someone asks,
“How successful
was that release?”
https://www.flickr.com/photos/tomhilton/16686579989

5. linkedin.com/in/davekarow
What I’m up to: demystifying progressive delivery, especially
the role of automated data attribution, early in partial releases.
How I got here: Three decades of experience in developer tools,
developer communities, and evangelizing sustainable software
delivery practices that deliver impact, without burning out
humans.
Where I’ve been: DHL Worldwide Express, Sun Microsystems,
Gupta Technologies, Remedy Software, Marimba (BMC), Keynote
Systems (Dynatrace), SOASTA (Akamai), BlazeMeter
(CA/Broadcom) and now Split Software.
It doesn’t have to be that way!
@SplitSoftware

6. Powering Progressive Delivery With Data
A Layered Approach
Progressive Delivery: What Is It, Really?
01
Role Models: Progressive Delivery In The Wild
The Foundation: Decouple Deploy from Release
The Upper Layers: Data-Informed Practices, Automated
02
03
04
@SplitSoftware

7. Progressive Delivery
What Is It, Really?
01
@SplitSoftware

8. The Roots of “Progressive Delivery”
“Well, when we’re rolling out services. What we do is
progressive experimentation because what really
matters is the blast radius. How many people will be
affected when we roll that service out and what can
we learn from them?”
Sam Guckenheimer, quoted in
https://www.infoq.com/presentations/progressive-delivery/
@SamGuckenheimer @monkchips
(James Governor)
(James Governor)
@SplitSoftware

9. The Roots of “Progressive Delivery”
@monkchips
(James Governor)
...a new basket of skills and technologies concerned
with modern software development, testing and
deployment.
@SplitSoftware

10. Carlos Sanchez
(Sr. Cloud Software Engineer @ Adobe)
https://blog.csanchez.org/2019/01/22/progressive-delivery-in-kubernetes-blue-green-and-canary-deployments/
Progressive Delivery is the next step after Continuous Delivery,
where new versions are deployed to a subset of users
and are evaluated in terms of correctness and performance
before rolling them to the totality of the users
and rolled back if not matching some key metrics.
@SplitSoftware

11. Role Models:
Progressive Delivery In the Wild
02
@SplitSoftware

12. Example:
Walmart EXPO

13. Example:
LinkedIn LiX

14. The Foundation:
Decouple Deploy from Release
03
@SplitSoftware

15. You Are Here

16. 4 Ways to Decouple Deploy From Release (How You Roll Matters)
Approach
Benefits
Blue/Green
Deployment
Canary Release
(container based)
Feature Flags Feature Flags +
Data, Integrated
Avoid
Downtime
Limit The
Blast Radius
Limit WIP /
Achieve Flow
Learn During
The Process
https://www.split.io/blog/learn-the-four-shades-of-progressive-delivery/
Harvey Balls by Sschulte at English Wikipedia [CC BY-SA (https://creativecommons.org/licenses/by-sa/3.0)] @SplitSoftware

17. Approach
Benefits
Blue/Green
Deployment
Canary Release
(container based)
Feature Flags Feature Flags +
Data, Integrated
Avoid
Downtime
Limit The
Blast Radius
Limit WIP /
Achieve Flow
Learn During
The Process
https://www.split.io/blog/learn-the-four-shades-of-progressive-delivery/
Harvey Balls by Sschulte at English Wikipedia [CC BY-SA (https://creativecommons.org/licenses/by-sa/3.0)] @SplitSoftware
4 Ways to Decouple Deploy From Release (How You Roll Matters)

18. Approach
Benefits
Blue/Green
Deployment
Canary Release
(container based)
Feature Flags Feature Flags +
Data, Integrated
Avoid
Downtime
Limit The
Blast Radius
Limit WIP /
Achieve Flow
Learn During
The Process
https://www.split.io/blog/learn-the-four-shades-of-progressive-delivery/
Harvey Balls by Sschulte at English Wikipedia [CC BY-SA (https://creativecommons.org/licenses/by-sa/3.0)] @SplitSoftware
4 Ways to Decouple Deploy From Release (How You Roll Matters)

19. Approach
Benefits
Blue/Green
Deployment
Canary Release
(container based)
Feature Flags Feature Flags +
Data, Integrated
Avoid
Downtime
Limit The
Blast Radius
Limit WIP /
Achieve Flow
Learn During
The Process
https://www.split.io/blog/learn-the-four-shades-of-progressive-delivery/
Harvey Balls by Sschulte at English Wikipedia [CC BY-SA (https://creativecommons.org/licenses/by-sa/3.0)] @SplitSoftware
4 Ways to Decouple Deploy From Release (How You Roll Matters)

20. Approach
Benefits
Blue/Green
Deployment
Canary Release
(container based)
Feature Flags Feature Flags +
Data, Integrated
Avoid
Downtime
Limit The
Blast Radius
Limit WIP /
Achieve Flow
Learn During
The Process
https://www.split.io/blog/learn-the-four-shades-of-progressive-delivery/
Harvey Balls by Sschulte at English Wikipedia [CC BY-SA (https://creativecommons.org/licenses/by-sa/3.0)] @SplitSoftware
4 Ways to Decouple Deploy From Release (How You Roll Matters)

21. Feature Flag
Like a dimmer switch for changes
0%
10%
20%
50%
100%
@SplitSoftware

22. What a Feature Flag Looks Like In Code
treatment = flags.getTreatment(“related-posts”);
if (treatment == “on”) {
// show related posts
} else {
// skip it
}
Simple “on/off” example
@SplitSoftware

23. What a Feature Flag Looks Like In Code
Multiple variant example
treatment = flags.getTreatment(“search-algorithm”);
if (treatment == “v1”) {
// use v1 of new search algorithm
} else if (treatment == “v2”) {
// use v2 of new search algorithm
} else {
// use existing search algorithm
}
@SplitSoftware

24. Foundational Capabilities Unlocked By Feature Flags
Decouple Deploy From Release
With Feature Flags
● Incremental Feature Development for Flow
● Testing In Production
● Kill Switch (big red button)
@SplitSoftware

25. The Upper Layers:
Data-Informed Practices,
Automated
04
@SplitSoftware

26. DEPLOY
Code deployed
No exposure
Why Automate Data-Informed Practices?
A Different Way to “Ship” Becomes Possible
@SplitSoftware

27. DEPLOY
Code deployed
No exposure
ERROR MITIGATION
0-50% Ramp
Identify bugs/crashes
@SplitSoftware
Why Automate Data-Informed Practices?
A Different Way to “Ship” Becomes Possible

28. DEPLOY
Code deployed
No exposure
ERROR MITIGATION
0-50% Ramp
Identify bugs/crashes
MEASURE
Maximum Power Ramp
Understand impact
@SplitSoftware
Why Automate Data-Informed Practices?
A Different Way to “Ship” Becomes Possible

29. DEPLOY
Code deployed
No exposure
ERROR MITIGATION
0-50% Ramp
Identify bugs/crashes
MEASURE
Maximum Power Ramp
Understand impact
SCALE MITIGATION
50-100% Ramp
Identify scaling issues
@SplitSoftware
Why Automate Data-Informed Practices?
A Different Way to “Ship” Becomes Possible

30. DEPLOY
Code deployed
No exposure
ERROR MITIGATION
0-50% Ramp
Identify bugs/crashes
MEASURE
Maximum Power Ramp
Understand impact
SCALE MITIGATION
50-100% Ramp
Identify scaling issues
RELEASE
Complete rollout
@SplitSoftware
Why Automate Data-Informed Practices?
A Different Way to “Ship” Becomes Possible

31. New Release Metrics Change
“Can’t we just
change things
and monitor
what happens?”
Heads Up! Not Everything Is As It Seems
@SplitSoftware

32. New Release Metrics Change
Everything else in
the world
● Product changes
● Marketing
campaigns
● Global Pandemics
● Nice Weather
Problem To Solve: Separating Signal From Noise
@SplitSoftware

33. How do you feel when this happens?
What’s the effect on your culture if this happens often?
@SplitSoftware
100%
Rolled Out

34. How would you feel if you could find problems
here, instead?
@SplitSoftware
5%
Rolled Out

35. Existing New
5%
95%
@SplitSoftware
Solution: Random Assignment + Statistical Comparison

36. Decouple Deploy From Release
With Feature Flags
Automate Guardrails
Safety: Automate Guardrails/Do-No-Harm Testing
● Alert on Exception / Performance Early In Rollout
● “Limit The Blast Radius” w/o Manual Heroics
@SplitSoftware

37. Guardrail Metrics
@SplitSoftware

38. Measure
Release Impact
Decouple Deploy From Release
With Feature Flags
Automate Guardrails
Meaning: Measure Release Impact
● Iteration w/o Measurement = Feature Factory 😡
● Direct Evidence of Our Efforts → Pride 😎
@SplitSoftware

39. Measure
Release Impact
Decouple Deploy From Release
With Feature Flags
Automate Guardrails
Test
to Learn
(A/B Test)
Humility: Test to Learn (A/B Test)
● Take Bigger Risks, Safely
● Learn Faster With Less Investment
○ Dynamic Config
○ Painted Door
@SplitSoftware

40. This is What Sustainable Software Delivery Looks Like:
DEPLOY
Code deployed
No exposure
ERROR MITIGATION
0-50% Ramp
Identify bugs/crashes
MEASURE
Maximum Power Ramp
Understand impact
SCALE MITIGATION
50-100% Ramp
Identify scaling issues
RELEASE
Complete rollout
@SplitSoftware

41. Progressive Delivery → bit.ly/pd-thoughts
Coding Practice + Free Accts → bit.ly/split4devs
Split Explainer Video (2 min) → bit.ly/what-is-split
Connect → linkedin.com/in/davekarow or @davekarow
@SplitSoftware
Let’s Chat! Q&A / Resources

42. Feature flags
Sensors
Correlation
Stats Engine
Causation
“Holy Grail”
Mgmt console
System of record
Alerting
| > $50M annual cost
| > $30M annual cost
| > $25M annual cost
Increasing functionality & company adoption
Cost
to
build
and
maintain
How In-House Progressive Delivery Platforms Paved The Way For Split
(They Proved the value of Layering Up, But Required Huge Investments)
@SplitSoftware