Testing in production is essential if you want to test software as rigorously as possible.
Why? While testing early in the pipeline (i.e., shift-left testing) is necessary and highly encouraged, it’s simply not enough on its own. Companies practicing agile testing methodologies and building a disposable infrastructure are ready to perform testing in production, which is sometimes called shift-right testing.
By testing in production, you build another level of confidence in releases after performing various checks in a live production environment. Testing in production allows the company to see how an application reacts to newly pushed code changes in the wild. It should become a significant component of your future application quality strategy going forward.
Below, I explain why it’s important to test in production, then offer tips for developing a shift-right testing strategy.
Session Takeaways:
- What Are the Advantages of Testing in Production?
- What Are the Risks?
- Potential Testing in Production Tactics
3. What Are We Talking About Today?
http://unsplash.com
Testing In Production, Why?
Before Shifting Right, You Must Go Left
What Are The Advantages of Testing in Production?
What Are the Risks?
Potential Testing in Production Tactics
Traditional Strategies Won’t Cut It
5. Allows you to see how an application
reacts to newly pushed code change in the
wild 👍 🔥 ☠
Testing In Production, Why?
6. Significant component of your future
application quality strategy going forward
Testing In Production, Why?
7. Testing in production, builds another level of
confidence in releases after performing
various quality checks in live production
environment 💯 💰
Testing In Production, Why?
9. We should continue to shift left with testing
at every stage of the pipeline that enables
faster feedback from testing and integrating
code
Shift-Left
Shift-RIGHT
Before Shifting Right, You Must Go Left
10. Ask Yourself: What Are the Advantages of Testing in Production?
http://unsplash.com
11. The big difference here is changing the norm
by extending the continuous testing
feedback loop from live production data and
real user traffic while performing tests in
production
Ask Yourself: What Are the Advantages of Testing in Production?
12. You will find a collection of defect bugs not
found from other testing environment
activities (dev, staging, or pre-prod)
Ask Yourself: What Are the Advantages of Testing in Production?
13. The goal is to elevate quality by building
various quality guards around the
application
Ask Yourself: What Are the Advantages of Testing in Production?
14. Advantages of testing in a production
environment
Risk
elim
inate
Data
Ask Yourself: What Are the Advantages of Testing in Production?
24. Potential Testing in Production Tactics
A / B Testing
50% vistors see variation A
50% vistors see variation B
25. Rollback
Potential Testing in Production Tactics
Auto Rollback Strategy
fail
rollback
Release
Strategy
Deployment
Validation
Canary
Blue Green
Rolling
Availability
Performance
Logs
Quality
Security
More…
Continuous Integration
Continuous Delivery
26. Testing in Production Methods
New Relic Synthetic User Testing
Visual Testing with Applitools
Disaster Recovery Testing
Lightweight User Acceptance Testing
Infrastructure Testing
Potential Testing in Production Tactics