Splashy, big-bang launches used to be the gold standard for software releases. But for every successful big bang launch, there were many failed launches that either never gained traction or that were so harmful, they actually caused successful companies to stumble or crash. Join Jon Noronha, Senior Director of Product at Optimizely, to learn why you should stop launching and start rolling out products in order to ship faster and reduce risk. Attend this webinar to learn: - Lessons from failed product launches - How to use feature flags to release new features safer - How to start small and validate your product with different audiences before releasing to everyone

1. 1
This Launch will
Self-Destruct in
3
Jon Noronha
Sr. Director, Product Management
Optimizely
Twitter: @thatsjonsense
...2...1...

2. The Old Way
DESIGN BUILD LAUNCH * PRAY
* Infrequently, only when done, with no real-time customer feedback

3. The New Way
EXPERIMENT ITERATEDESIGN BUILD

11. BUILD
LEARN
MEASURE
Product Experimentation
Think hypothetically
● Painted doors
● Validating MVPs
● Qualitative research
● Exploratory A/B testing
Quantify product impact
● Feature validation
● Multivariate tests
● Iterative adjustments
● Measuring business impact
Release confidently
● Frequent, small releases
● Feature flagging
● Staged rollouts
● Safe rollbacks

12. Painted Door Tests
• Instead of building out an entire
feature, create the suggestion of a
feature, and see how many users
want to utilize it

13. Would people use “save for later”
functionality on the web?

15. Option 1
Build the full “Save for later” functionality on web, meaning
parity between platforms.

16. Option 1
Build the full “Save for later” functionality on web, meaning
parity between platforms.
Option 2 (painted door experiment)
Measure demand with just a
icon, without any functionality behind it.

18. “Save for later” isn’t available yet on our desktop
website but we’re thinking about adding it.
Is this a feature you would use?
Yes No
Experiment-driven product development lifecycle

19. BUILD
LEARN
MEASURE
Product Experimentation
Think hypothetically
● Painted doors
● Validating MVPs
● Qualitative research
● Exploratory A/B testing
Quantify product impact
● Feature validation
● Multivariate tests
● Iterative adjustments
● Measuring business impact
Release confidently
● Frequent, small releases
● Feature flagging
● Staged rollouts
● Safe rollbacks

21. 21

22. 22

23. 23

27. BUILD
LEARN
MEASURE
Product Experimentation
Think hypothetically
● Painted doors
● Validating MVPs
● Qualitative research
● Exploratory A/B testing
Quantify product impact
● Feature validation
● Multivariate tests
● Iterative adjustments
● Measuring business impact
Release confidently
● Frequent, small releases
● Feature flagging
● Staged rollouts
● Safe rollbacks

28. 28
What could possibly go wrong?
Big bang launch horror stories

33. 33
There’s a better way.
Confident launches with rollouts.

35. if feature_flag is ‘on’:
do_something()
else:
do_something_else()
Feature Flags

36. Feature Flags

37. mobile_app_header
Feature Flags

38. mobile_app_header
search_ranking_algorithm
Feature Flags

39. mobile_app_header
checkout_flow
search_ranking_algorithm
Feature Flags

40. enabled: True
header_color: #F829EA
header_sticky: True
header_height: 20
mobile_app_header
enabled: True
checkout_version: 3
quick_checkout: True
header_height: 20
checkout_flow
enabled: True
fresh_boost: 1.25
profile_boost: 2.10
local_boost: 0.75
search_ranking_algorithm
Feature Flags

41. enabled: True
header_color: #F829EA
header_sticky: True
header_height: 20
mobile_app_header

42. enabled: True
header_color: #F829EA
header_sticky: True
header_height: 20
mobile_app_header
Feature is on
Toggling a feature

43. enabled: False
header_color: #F829EA
header_sticky: True
header_height: 20
mobile_app_header
Feature is off
Toggling a feature

45. enabled: True
header_color: #F829EA
header_sticky: True
header_height: 20
mobile_app_header
enabled: False
header_color: #F829EA
header_sticky: True
header_height: 15
mobile_app_header
Feature is on Feature is off
Rolling out a feature

46. Feature Rollouts

47. Feature Rollouts

48. Feature Rollouts

49. Feature Rollouts

50. Feature Rollouts

51. Feature Rollouts

53. 53
Types of rollout
100% ROLLOUTGRADUAL
ROLLOUTBETA USERSSTAGING
ENVIRONMENT
LOCAL
DEVELOPMENT

54. Trunk-based development
Develop features concurrently by
implementing flags set to “off”. Deploy to
master at any time while avoiding merge
conflicts.
Canary Releases
Use a flag to send a small percentage of
traffic to a new build of your application to
discovery any unforeseen issues prior to
rolling out to all users.
Beta or Exclusive Access
Give business users the power to grant
exclusive access to new features or beta
programs for your best customers or
prospects in a sales cycle.
Kill Switch
With feature flags, you can quickly turn off
problem features without rolling back an
entire release.
Dark Launches
Deploy features behind “off” flags, then give
marketers or PMs the ability to turn on the
feature independent of a new deploy.
Remote Configuration
Add parameterized variables to feature flags
to update features in real time, without a
code push or app update. Variables can also
be used to A/B test variations of features.

56. BUILD
LEARN
MEASURE
Product Experimentation
Think hypothetically
● Painted doors
● Validating MVPs
● Qualitative research
● Exploratory A/B testing
Quantify product impact
● Feature validation
● Multivariate tests
● Iterative adjustments
● Measuring business impact
Release confidently
● Frequent, small releases
● Feature flagging
● Staged rollouts
● Safe rollbacks
Rollouts
Web
Full Stack

57. optimizely.com/rollouts
Free, open source feature flagging & rollouts

58. Begin your journey to product experimentation
Start with Optimizely Rollouts for free to ship faster with
less risk
When you’re ready to drive measurable product impact then
upgrade to Optimizely Full Stack experimentation

59. 59
Thank you!
Questions?
Jon Noronha
optimizely.com/rollouts
Twitter: @thatsjonsense

60. Your applicationYour control panel
mobile_app_header
search_ranking_algorithm
checkout_flow