Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Testing inproduction svcc18

83 views

Published on

While testing in demo and stage is good (indeed, essential), testing in production is all too often overlooked. Deploying to production and hoping for the best is a gamble, not a strategy.

In this talk, we discuss
1) Better production deployment and testing strategies including dark pool testing, canary releases and feature switching.
2) After deployment, your work is still not done. We'll talk about Observability, including monitoring, tracing and metrics.
3) Finally, even with the best deployment strategies and monitoring techniques, your software WILL fail in production. It's a question of when, not if. So why not simulate those failures first? We'll finish with game days and chaos engineering.

This talk should be of interest to all developers, QA and Ops folks who are responsible for getting working software in front of users.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Testing inproduction svcc18

  1. 1. Confidential 1 Testing in Production Saturday 13th October, 2018 shaun@abram.com shaunabram.com @shaunabram linkedin.com/in/sabram/ Evaluation: https://goo.gl/VTAwgw
  2. 2. Confidential 2
  3. 3. Confidential Testing in Production 3 How is Production different?
  4. 4. Confidential 4 Testing in Production IS NOT a replacement for non-prod testing Treat production validation with the respect it deserves.
  5. 5. Confidential 5 Testing in Production Observability Testing at Release Chaos Engineering
  6. 6. Confidential 6 Observability The ability to ask new questions of your system without deploying new code
  7. 7. Confidential We need Observability in our systems 7  Everything is sometimes broken  Something is always broken  If nothing seems broken... …your monitoring is broken It’s impossible to predict the myriad states of partial failures we’ll see
  8. 8. Confidential Privileged and Confidential 8 Observability How do we observe our apps? Logs Metrics Monitoring & Alerting Traces Tools
  9. 9. Confidential 9 Testing in Production Observability
  10. 10. Confidential 10 Testing in Production Observability Testing at Release Chaos Engineering
  11. 11. Confidential 11 Testing in Production Testing at Release
  12. 12. Confidential 12 Testing at Release Deploy Config Tests Smoke Tests Shadowing Load Tests Release Canary release Internal release Post-Release Feature Flags A/B Testing Chaos Engineering…
  13. 13. Confidential 13 Testing in Production Testing at Release
  14. 14. Confidential 14 Testing in Production Observability Testing at Release Chaos Engineering
  15. 15. Confidential 15 Testing in Production Chaos Engineering Carefully planned experiments designed to reveal weaknesses in our systems aka Resilience Engineering
  16. 16. Confidential Game Days 16 An exercise where we place systems under stress to learn and improve resilience (And even just getting the team together to discuss resilience can be worthwhile)
  17. 17. Confidential Chaos Engineering – a step by step guide 17 Hypothesis (Steady state) Minimize Blast Radius Run Analyze Increase Repeat, Automate
  18. 18. Confidential 18 Testing in Production Chaos Engineering
  19. 19. Confidential 19 Testing in Production Observability Testing at Release Chaos Engineering
  20. 20. Confidential Reading material 20 Chaos Engineering (free eBook) https://www.oreilly.com/webops-perf/free/chaos-engineering.csp Distributed Systems Observability (free eBook) https://distributed-systems-observability-ebook.humio.com/
  21. 21. Confidential Reading material 21 shaunabram.com Principles of Chaos Engineering principlesofchaos.org How to run a Game Day https://www.gremlin.com/community/tutorials/how-to-run-a-gameday/ Testing in production: https://medium.com/@copyconstruct/testing-in-production-the-safe-way-18ca102d0ef1 Monitoring in the time of Cloud Native https://medium.com/@copyconstruct/monitoring-in-the-time-of-cloud-native-c87c7a5bfa3e Deploy != Release https://blog.turbinelabs.io/deploy-not-equal-release-part-one-4724bc1e726b
  22. 22. Confidential Testing in production – The Industry Experts 22 Nora Jones Charity Majors Cindy Sridharan Tammy Butow
  23. 23. Confidential Questions? And please evaluate! https://goo.gl/VTAwgw 23

×