https://devoxx.com.ua/talk-details/?id=24405

1. Testing on
Production
deep back-end edition
Mykola V. Gurov

2. bol.com
webshop & platform
ecommerce

3. Deep Back-End
Logistics
Purchasing

4. Testing Environments
• The place where the value is generated. The
rest is an overhead.

5. Testing Environments
• Isolation (unit, integration test). Exclusive.
Controlled. Fast. Dev workstations / CI
servers. Best for functional testing.

6. Testing Environments
• E2E. Staging. Acceptance. Shared.

7. 💧
@Ngurov

9. Shadow Transactions

10. Shadow - new algo

11. Functionally tested
• Unit and Integration testing (BDD-style)
• Knowns

12. Why not to test
on Production?
It just sits there waiting…

15. –Kaner, Cem (November 17, 2006). Exploratory Testing (PDF).
Quality Assurance Institute Worldwide Annual Software Testing
Conference. Orlando, FL. Retrieved November 22, 2014.
via https://en.wikipedia.org/wiki/Software_testing
“Software testing is an investigation
conducted to provide stakeholders with
information about the quality of the
software product or service under test."
What is testing?

16. 💧
@Ngurov

17. Database Change

18. Avoid branches

19. Separate release from
deployment

20. 💧@Ngurov

21. Shadow across systems
🚚

22. Shadow across systems
🚚

23. Gradually out of shadow
🚚

24. 💧@Ngurov

25. Out of the
shadow

26. Golden Order
a.k.a. “Lucky” customer

27. White-listing

28. 💧@Ngurov

29. Performance testing
• Collect, visualize and alert on metrics
• Shut some nodes down
• Pause a queue consumer

30. Chaos
engineering
Disruption on purpose

31. 💧@Ngurov

32. Best used together
• Feature Toggles
• (Trunk Based Development + Continuous
Delivery)
• Microservices

33. Testing Environments
• Data & Behavior
• Always second class -> Delays
• Test tooling

34. Testing Environments
Investment into the most valuable
environment:
Observability
Monitoring
Resilience

36. 💧
Conclusions

37. Testing on Production
isn't a sin but rather an
opportunity
done responsibly

38. Youtube materials
• What do you know about testing in
production? https://youtu.be/z-ATZTUgaAo
Michael Bryzek QCon San Francisco 2016
• GTAC 2007: Ed Keyes - Sufficiently Advanced
Monitoring is Indistinguishable from
Testing https://www.youtube.com/watch?
v=uSo8i1N18oc

39. @Ngurov