SlideShare a Scribd company logo
1 of 12
Download to read offline
Reduce Release Risk
with
Feature Monitoring
SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO
The challenge of gradual rollouts in the world of APM
When you release all or nothing, detecting
change is about time correlation
p95
release t
metric
With feature flags and gradual rollouts,
detecting signal takes an integrated solution
p95
10%
rollout
t
metric
20%
rollout
new
old
SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO
The newest addition to Split’s Feature Delivery Platform
Feature Delivery Platform
Manage
Deploy continuously
with feature flags, gradual
rollouts, kill switch, and
dynamic configurations.
Monitor
Release safely
with automated detection
and alerting of user
performance and errors.
Experiment
Create impact
with statistically significant
A/B tests for every feature
you launch.
Engineering KPIs:
User errors &
performance
Product KPIs:
Customer
analytics
Client & Server SDKs
Your web &
mobile apps
SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO
Extends existing Feature Management capabilities
Control the release of new software
with feature flags
Manage frontend, backend, and
mobile apps with SDKs for 11
popular languages
Target new features to specific
segments (e.g. QA, beta, geo)
Rollout gradually to a % of users
Kill features with the click of a
button
SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO
Safely ship software with Feature Monitoring
Real-time Sentry error data
surfaces specific user issues.
Real user performance data
from a lightweight JS agent.
Automated detection of
significant errors.
Automated attribution back
to each feature in the release.
Alerting to notify the specific
team responsible.
Instant feature kill switch so
users no longer have a poor
experience.
SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO
With integrations, see how each feature impacts errors
Lightweight JS agent included alongside our
javascript SDK
Capture performance data per user:
● Page load time
● Time to first byte
● Time to DOM interactive
● Time to first interactive
● Count of errors
● Count of JS exceptions
Pre-built integration available as the first feature
flag partner in the Sentry Integration Platform
Capture error data for each feature:
● Count of exceptions per user
● Count of exceptions by Issue
SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO
Find failed features before your customers
Detect elevated error
rates or performance
degradation within 30
minutes of a feature
release and reduce
MTTR to minutes.
Identify the exact
feature in your
release that caused
the error rate, so you
can immediately
triage it.
Turn off features
within 5 seconds,
resolving customer
instantly, without the
need for rollbacks or
hotfixes.
Detect issues before
you expose features
to more than 5-10%
of users, reducing the
blast radius.
feature delivery with engineering KPIs
“With Feature Monitoring, our engineering team can deploy rapidly while ensuring
that new functionality won’t negatively impact our core KPIs such as page load time
and errors per user.”
Seth Moore,
Vistaprint Lead Software Engineer
Benefits:
● Instant root cause analysis
● Avoid outages and service disruptions
● No manual code rollbacks or hotfixes
● Faster time to restore service
8
deployment and feature rollout
“Our team uses Split feature flags to deploy quickly and experiment with new ideas.
Feature Monitoring gives us a safe way to release features to our global user base
with less risk.”
Bjorn Levidow,
GoDaddy Sr. Director PM
Benefits:
● Increase the pace of innovation
● Move quickly to rollout features with less risk
9
SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO
Demo script
Show JS include
Show site and load it
Show metric and event data for load time
Show Split alert policy
Show alert tab
SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO
The statistics applied to alerting

More Related Content

What's hot

Continuous delivery mobile application development
Continuous delivery mobile application developmentContinuous delivery mobile application development
Continuous delivery mobile application development
Thoughtworks
 
Seven Steps to Remove Barriers and Accelerate Mobile Testing
Seven Steps to Remove Barriers and Accelerate Mobile TestingSeven Steps to Remove Barriers and Accelerate Mobile Testing
Seven Steps to Remove Barriers and Accelerate Mobile Testing
Keynote Mobile Testing
 

What's hot (20)

SpiraTest Overview Presentation (2021)
SpiraTest Overview Presentation (2021)SpiraTest Overview Presentation (2021)
SpiraTest Overview Presentation (2021)
 
Automated Visual Testing at Scale : Real-life Example from Dow Jones
Automated Visual Testing at Scale : Real-life Example from Dow JonesAutomated Visual Testing at Scale : Real-life Example from Dow Jones
Automated Visual Testing at Scale : Real-life Example from Dow Jones
 
Take your code and quality to the next level by Serena Software
Take your code and quality to the next level by Serena SoftwareTake your code and quality to the next level by Serena Software
Take your code and quality to the next level by Serena Software
 
DevOps Monitoring and Alerting
DevOps Monitoring and AlertingDevOps Monitoring and Alerting
DevOps Monitoring and Alerting
 
Automated Testing Services
Automated Testing ServicesAutomated Testing Services
Automated Testing Services
 
Continuous delivery mobile application development
Continuous delivery mobile application developmentContinuous delivery mobile application development
Continuous delivery mobile application development
 
Deploy Fast Without Breaking Things Webinar Presentation June 25
Deploy Fast Without Breaking Things Webinar Presentation June 25Deploy Fast Without Breaking Things Webinar Presentation June 25
Deploy Fast Without Breaking Things Webinar Presentation June 25
 
Digital Assurance - Today & Tomorrow
Digital Assurance - Today & TomorrowDigital Assurance - Today & Tomorrow
Digital Assurance - Today & Tomorrow
 
Kloia Quality Assurance
Kloia Quality AssuranceKloia Quality Assurance
Kloia Quality Assurance
 
Seven Steps to Remove Barriers and Accelerate Mobile Testing
Seven Steps to Remove Barriers and Accelerate Mobile TestingSeven Steps to Remove Barriers and Accelerate Mobile Testing
Seven Steps to Remove Barriers and Accelerate Mobile Testing
 
EXTENT-2016: The Future of Software Testing
EXTENT-2016:	 The Future of Software TestingEXTENT-2016:	 The Future of Software Testing
EXTENT-2016: The Future of Software Testing
 
Scriptless Automation Testing: A Unique Framework To Accelerate Delivery
Scriptless Automation Testing: A Unique Framework To Accelerate DeliveryScriptless Automation Testing: A Unique Framework To Accelerate Delivery
Scriptless Automation Testing: A Unique Framework To Accelerate Delivery
 
Functional to Visual: AI-powered UI Testing from Testim and Applitools
Functional to Visual: AI-powered UI Testing from Testim and ApplitoolsFunctional to Visual: AI-powered UI Testing from Testim and Applitools
Functional to Visual: AI-powered UI Testing from Testim and Applitools
 
Zephyr 2.6: Comprehensive Test Management
Zephyr 2.6: Comprehensive Test ManagementZephyr 2.6: Comprehensive Test Management
Zephyr 2.6: Comprehensive Test Management
 
Testwarez 2009 Use Proper Tool
Testwarez 2009 Use Proper ToolTestwarez 2009 Use Proper Tool
Testwarez 2009 Use Proper Tool
 
Tales from the Dark Side: Ori Bendet Selenium Conference India 2016
Tales from the Dark Side: Ori Bendet Selenium Conference India 2016Tales from the Dark Side: Ori Bendet Selenium Conference India 2016
Tales from the Dark Side: Ori Bendet Selenium Conference India 2016
 
Soasta New Features in CloudTest & TouchTest Webinar
Soasta New Features in CloudTest & TouchTest WebinarSoasta New Features in CloudTest & TouchTest Webinar
Soasta New Features in CloudTest & TouchTest Webinar
 
Cd
CdCd
Cd
 
TechTalk: Report Bugs Like a Boss
TechTalk: Report Bugs Like a BossTechTalk: Report Bugs Like a Boss
TechTalk: Report Bugs Like a Boss
 
Manual Testing Services
Manual Testing ServicesManual Testing Services
Manual Testing Services
 

Similar to Reduce Release Risk with Feature Monitoring

Application Quality Best Practices with Visual Studio 2010 - Adrian Dunne
Application Quality Best Practices with Visual Studio 2010 - Adrian DunneApplication Quality Best Practices with Visual Studio 2010 - Adrian Dunne
Application Quality Best Practices with Visual Studio 2010 - Adrian Dunne
Spiffy
 

Similar to Reduce Release Risk with Feature Monitoring (20)

MuleSoft Surat Virtual Meetup#4 - Anypoint Monitoring and MuleSoft dataloader.io
MuleSoft Surat Virtual Meetup#4 - Anypoint Monitoring and MuleSoft dataloader.ioMuleSoft Surat Virtual Meetup#4 - Anypoint Monitoring and MuleSoft dataloader.io
MuleSoft Surat Virtual Meetup#4 - Anypoint Monitoring and MuleSoft dataloader.io
 
Application Quality Best Practices with VS2010
Application Quality Best Practices with VS2010Application Quality Best Practices with VS2010
Application Quality Best Practices with VS2010
 
Application Quality Best Practices with Visual Studio 2010 - Adrian Dunne
Application Quality Best Practices with Visual Studio 2010 - Adrian DunneApplication Quality Best Practices with Visual Studio 2010 - Adrian Dunne
Application Quality Best Practices with Visual Studio 2010 - Adrian Dunne
 
implementing_ai_for_improved_performance_testing_the_key_to_success.pdf
implementing_ai_for_improved_performance_testing_the_key_to_success.pdfimplementing_ai_for_improved_performance_testing_the_key_to_success.pdf
implementing_ai_for_improved_performance_testing_the_key_to_success.pdf
 
How to Deliver Winning Mobile Apps
How to Deliver Winning Mobile AppsHow to Deliver Winning Mobile Apps
How to Deliver Winning Mobile Apps
 
06 operations and feedback
06   operations and feedback06   operations and feedback
06 operations and feedback
 
prod-dev-management.pptx
prod-dev-management.pptxprod-dev-management.pptx
prod-dev-management.pptx
 
Agile Development in Aerospace and Defense
Agile Development in Aerospace and DefenseAgile Development in Aerospace and Defense
Agile Development in Aerospace and Defense
 
What_is_SAP_Change_Impact_Analysis.pdf
What_is_SAP_Change_Impact_Analysis.pdfWhat_is_SAP_Change_Impact_Analysis.pdf
What_is_SAP_Change_Impact_Analysis.pdf
 
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
 
SplunkLive! Munich 2018: Monitoring the End-User Experience with Splunk
SplunkLive! Munich 2018: Monitoring the End-User Experience with SplunkSplunkLive! Munich 2018: Monitoring the End-User Experience with Splunk
SplunkLive! Munich 2018: Monitoring the End-User Experience with Splunk
 
Continuous Integration Practices
Continuous Integration Practices Continuous Integration Practices
Continuous Integration Practices
 
Continuous integration practices to improve the software quality
Continuous integration practices to improve the software qualityContinuous integration practices to improve the software quality
Continuous integration practices to improve the software quality
 
Agile process with a fixed cost
Agile process with a fixed costAgile process with a fixed cost
Agile process with a fixed cost
 
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
 
SplunkLive! Frankfurt 2018 - Monitoring the End User Experience with Splunk
SplunkLive! Frankfurt 2018 - Monitoring the End User Experience with SplunkSplunkLive! Frankfurt 2018 - Monitoring the End User Experience with Splunk
SplunkLive! Frankfurt 2018 - Monitoring the End User Experience with Splunk
 
Performance Metrics Driven CI/CD - Introduction to Continuous Innovation and ...
Performance Metrics Driven CI/CD - Introduction to Continuous Innovation and ...Performance Metrics Driven CI/CD - Introduction to Continuous Innovation and ...
Performance Metrics Driven CI/CD - Introduction to Continuous Innovation and ...
 
SplunkLive! Zurich 2018: Monitoring the End User Experience with Splunk
SplunkLive! Zurich 2018: Monitoring the End User Experience with SplunkSplunkLive! Zurich 2018: Monitoring the End User Experience with Splunk
SplunkLive! Zurich 2018: Monitoring the End User Experience with Splunk
 
Take Control of Application Performance
Take Control of Application PerformanceTake Control of Application Performance
Take Control of Application Performance
 
How to accelerate UAT & Regression Testing
How to accelerate UAT & Regression TestingHow to accelerate UAT & Regression Testing
How to accelerate UAT & Regression Testing
 

Recently uploaded

Digital Communication Essentials: DPCM, DM, and ADM .pptx
Digital Communication Essentials: DPCM, DM, and ADM .pptxDigital Communication Essentials: DPCM, DM, and ADM .pptx
Digital Communication Essentials: DPCM, DM, and ADM .pptx
pritamlangde
 
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
AldoGarca30
 
DeepFakes presentation : brief idea of DeepFakes
DeepFakes presentation : brief idea of DeepFakesDeepFakes presentation : brief idea of DeepFakes
DeepFakes presentation : brief idea of DeepFakes
MayuraD1
 
Standard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power PlayStandard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power Play
Epec Engineered Technologies
 

Recently uploaded (20)

UNIT 4 PTRP final Convergence in probability.pptx
UNIT 4 PTRP final Convergence in probability.pptxUNIT 4 PTRP final Convergence in probability.pptx
UNIT 4 PTRP final Convergence in probability.pptx
 
DC MACHINE-Motoring and generation, Armature circuit equation
DC MACHINE-Motoring and generation, Armature circuit equationDC MACHINE-Motoring and generation, Armature circuit equation
DC MACHINE-Motoring and generation, Armature circuit equation
 
Tamil Call Girls Bhayandar WhatsApp +91-9930687706, Best Service
Tamil Call Girls Bhayandar WhatsApp +91-9930687706, Best ServiceTamil Call Girls Bhayandar WhatsApp +91-9930687706, Best Service
Tamil Call Girls Bhayandar WhatsApp +91-9930687706, Best Service
 
Online food ordering system project report.pdf
Online food ordering system project report.pdfOnline food ordering system project report.pdf
Online food ordering system project report.pdf
 
Basic Electronics for diploma students as per technical education Kerala Syll...
Basic Electronics for diploma students as per technical education Kerala Syll...Basic Electronics for diploma students as per technical education Kerala Syll...
Basic Electronics for diploma students as per technical education Kerala Syll...
 
Digital Communication Essentials: DPCM, DM, and ADM .pptx
Digital Communication Essentials: DPCM, DM, and ADM .pptxDigital Communication Essentials: DPCM, DM, and ADM .pptx
Digital Communication Essentials: DPCM, DM, and ADM .pptx
 
Ground Improvement Technique: Earth Reinforcement
Ground Improvement Technique: Earth ReinforcementGround Improvement Technique: Earth Reinforcement
Ground Improvement Technique: Earth Reinforcement
 
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
 
Thermal Engineering -unit - III & IV.ppt
Thermal Engineering -unit - III & IV.pptThermal Engineering -unit - III & IV.ppt
Thermal Engineering -unit - III & IV.ppt
 
Ghuma $ Russian Call Girls Ahmedabad ₹7.5k Pick Up & Drop With Cash Payment 8...
Ghuma $ Russian Call Girls Ahmedabad ₹7.5k Pick Up & Drop With Cash Payment 8...Ghuma $ Russian Call Girls Ahmedabad ₹7.5k Pick Up & Drop With Cash Payment 8...
Ghuma $ Russian Call Girls Ahmedabad ₹7.5k Pick Up & Drop With Cash Payment 8...
 
HOA1&2 - Module 3 - PREHISTORCI ARCHITECTURE OF KERALA.pptx
HOA1&2 - Module 3 - PREHISTORCI ARCHITECTURE OF KERALA.pptxHOA1&2 - Module 3 - PREHISTORCI ARCHITECTURE OF KERALA.pptx
HOA1&2 - Module 3 - PREHISTORCI ARCHITECTURE OF KERALA.pptx
 
Thermal Engineering Unit - I & II . ppt
Thermal Engineering  Unit - I & II . pptThermal Engineering  Unit - I & II . ppt
Thermal Engineering Unit - I & II . ppt
 
DeepFakes presentation : brief idea of DeepFakes
DeepFakes presentation : brief idea of DeepFakesDeepFakes presentation : brief idea of DeepFakes
DeepFakes presentation : brief idea of DeepFakes
 
Orlando’s Arnold Palmer Hospital Layout Strategy-1.pptx
Orlando’s Arnold Palmer Hospital Layout Strategy-1.pptxOrlando’s Arnold Palmer Hospital Layout Strategy-1.pptx
Orlando’s Arnold Palmer Hospital Layout Strategy-1.pptx
 
Jaipur ❤CALL GIRL 0000000000❤CALL GIRLS IN Jaipur ESCORT SERVICE❤CALL GIRL IN...
Jaipur ❤CALL GIRL 0000000000❤CALL GIRLS IN Jaipur ESCORT SERVICE❤CALL GIRL IN...Jaipur ❤CALL GIRL 0000000000❤CALL GIRLS IN Jaipur ESCORT SERVICE❤CALL GIRL IN...
Jaipur ❤CALL GIRL 0000000000❤CALL GIRLS IN Jaipur ESCORT SERVICE❤CALL GIRL IN...
 
Max. shear stress theory-Maximum Shear Stress Theory ​ Maximum Distortional ...
Max. shear stress theory-Maximum Shear Stress Theory ​  Maximum Distortional ...Max. shear stress theory-Maximum Shear Stress Theory ​  Maximum Distortional ...
Max. shear stress theory-Maximum Shear Stress Theory ​ Maximum Distortional ...
 
Computer Networks Basics of Network Devices
Computer Networks  Basics of Network DevicesComputer Networks  Basics of Network Devices
Computer Networks Basics of Network Devices
 
Standard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power PlayStandard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power Play
 
💚Trustworthy Call Girls Pune Call Girls Service Just Call 🍑👄6378878445 🍑👄 Top...
💚Trustworthy Call Girls Pune Call Girls Service Just Call 🍑👄6378878445 🍑👄 Top...💚Trustworthy Call Girls Pune Call Girls Service Just Call 🍑👄6378878445 🍑👄 Top...
💚Trustworthy Call Girls Pune Call Girls Service Just Call 🍑👄6378878445 🍑👄 Top...
 
Signal Processing and Linear System Analysis
Signal Processing and Linear System AnalysisSignal Processing and Linear System Analysis
Signal Processing and Linear System Analysis
 

Reduce Release Risk with Feature Monitoring

  • 2. SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO The challenge of gradual rollouts in the world of APM When you release all or nothing, detecting change is about time correlation p95 release t metric With feature flags and gradual rollouts, detecting signal takes an integrated solution p95 10% rollout t metric 20% rollout new old
  • 3. SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO The newest addition to Split’s Feature Delivery Platform Feature Delivery Platform Manage Deploy continuously with feature flags, gradual rollouts, kill switch, and dynamic configurations. Monitor Release safely with automated detection and alerting of user performance and errors. Experiment Create impact with statistically significant A/B tests for every feature you launch. Engineering KPIs: User errors & performance Product KPIs: Customer analytics Client & Server SDKs Your web & mobile apps
  • 4. SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO Extends existing Feature Management capabilities Control the release of new software with feature flags Manage frontend, backend, and mobile apps with SDKs for 11 popular languages Target new features to specific segments (e.g. QA, beta, geo) Rollout gradually to a % of users Kill features with the click of a button
  • 5. SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO Safely ship software with Feature Monitoring Real-time Sentry error data surfaces specific user issues. Real user performance data from a lightweight JS agent. Automated detection of significant errors. Automated attribution back to each feature in the release. Alerting to notify the specific team responsible. Instant feature kill switch so users no longer have a poor experience.
  • 6. SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO With integrations, see how each feature impacts errors Lightweight JS agent included alongside our javascript SDK Capture performance data per user: ● Page load time ● Time to first byte ● Time to DOM interactive ● Time to first interactive ● Count of errors ● Count of JS exceptions Pre-built integration available as the first feature flag partner in the Sentry Integration Platform Capture error data for each feature: ● Count of exceptions per user ● Count of exceptions by Issue
  • 7. SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO Find failed features before your customers Detect elevated error rates or performance degradation within 30 minutes of a feature release and reduce MTTR to minutes. Identify the exact feature in your release that caused the error rate, so you can immediately triage it. Turn off features within 5 seconds, resolving customer instantly, without the need for rollbacks or hotfixes. Detect issues before you expose features to more than 5-10% of users, reducing the blast radius.
  • 8. feature delivery with engineering KPIs “With Feature Monitoring, our engineering team can deploy rapidly while ensuring that new functionality won’t negatively impact our core KPIs such as page load time and errors per user.” Seth Moore, Vistaprint Lead Software Engineer Benefits: ● Instant root cause analysis ● Avoid outages and service disruptions ● No manual code rollbacks or hotfixes ● Faster time to restore service 8
  • 9. deployment and feature rollout “Our team uses Split feature flags to deploy quickly and experiment with new ideas. Feature Monitoring gives us a safe way to release features to our global user base with less risk.” Bjorn Levidow, GoDaddy Sr. Director PM Benefits: ● Increase the pace of innovation ● Move quickly to rollout features with less risk 9
  • 10.
  • 11. SPLIT.IOPROPRIETARY&CONFIDENTIALSPLIT.IO Demo script Show JS include Show site and load it Show metric and event data for load time Show Split alert policy Show alert tab