Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Moving From Staged To Pervasive Testing


Published on

A Keynote I have delivered to EuroSTAR 2014 and Test+Verification Conference.

How did we move from yearly releases to weekly releases and how did our testing change.

Published in: Software
  • Be the first to comment

Moving From Staged To Pervasive Testing

  1. 1. Continuous Delivery and DevOps Moving from Staged to Pervasive Testing Intelligent Testing Conference 2015 Rob Lambert @rob_lambert
  2. 2. Slow Boring Ineffective testing I’ve spent a lot of time doing
  3. 3. It’s no surprise then that we didn’t ship very often
  4. 4. Test case management system of DOOM
  5. 5. Mind the gap
  6. 6. A simple calculation
  7. 7. Anyone who can click a mouse became a tester….doesn’t that say something about the testing?
  8. 8. I can’t believe it took so long to test Slow Boring Ineffective I don’t mind helping out but my word, testing is so boring We ran 10 million tests and we still had issues!
  9. 9. I joined NewVoiceMedia just before the last giant release. We vowed never to do a year long release again. Let’s do something different
  10. 10. Why stop doing big releases?
  11. 11. This is what we shipped (One tower) This is what our customers now needed (a whole bridge) Photo by Vaidotas Mišeikis - Clifton Suspension Bridge at Dusk - February 19, 2011
  12. 12. When your customer’s needs move quickly, so too must your release process.
  13. 13. The ultimate feedback loop
  14. 14. How did we make the change? We created a vision
  15. 15. 1. Adopt Agile 2. Prioritise work 3. Bring Dev and Ops together - DevOps 4. Everyone tests. Test all the time. 5. Become “one” with data The Vision Weekly Releases
  16. 16. 1. Removed testers from the centre of the universe Testers are important, but no more than any other member of the team Instead we put “Testing” at the centre of our universe
  17. 17. 2. Focus on automating checks BDD TDD CI UI Testing
  18. 18. 3. Ate our own product Dog-fooding (Pre- Production)
  19. 19. Event logs, internal customer feedback and regression tests
  20. 20. 4. Tested and monitored after release TRUST Own usage Monitoring
  21. 21. Every team supports their work in to production
  22. 22. Data from production tells us how our product is behaving This is immensely powerful in guiding your testing
  23. 23. 5. Emphasised discovery over assumptions Exploratory Testing What is the system telling us?
  24. 24. Testing that is data informed
  25. 25. 6. Learned. • 5 Whys on everything • Continuous Improvement • Training and personal development “It’s like studying for a degree but getting paid for it”
  26. 26. 7. Dealt with fires • “If it hurts. Keep doing it” • Political • Squash “Naysaying” and “Silent Plotting”
  27. 27. Everyone knows how to test. But it’s not that sort of testing you need. You need testing that is relevant.
  28. 28. • At least one release each week, sometimes more • We have more to improve though • Feedback from live is guiding our testing • Feedback from our testing is immediate Did it work? And is that it now?
  29. 29. Thank you. Any Questions. NVM How To Remain Relevant – The Book Cultivated Management Seminar Day Blog Twitter @rob_lambert