Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Keeping your tests lean

340 views

Published on

We've come to realize that automation provides an immense amount of value in preventing regressions and helping to deliver quality software. As your automation grows and grows, it requires continuous maintenance so that tests remain fast, reliable, and valuable. If you're not scaling efficiently, your automation suite will turn into a messy, uncontrollable beast. Having a lean test suite will help to combat this.

This presentation shares methods that you can use to keep your automated test suites lean and mean, so they always provide quick and accurate feedback to your software delivery team.

Published in: Software
  • Be the first to comment

Keeping your tests lean

  1. 1. KEEPING YOUR TESTS LEAN
  2. 2. @iammeaghanlewis meaghan@lever.co meaghan-lewis
  3. 3. WHAT DOES LEAN MEAN ANYWAY?
  4. 4. VALUABLE RELIABLE FAST
  5. 5. 1000+ TESTS
  6. 6. 12 HR BUILD
  7. 7. 100% FAILURE RATE
  8. 8. HOW DID WE GET A LEAN TEST SUITE?
  9. 9. NOT QUITE...
  10. 10. VALUABLE
  11. 11. Tests provide useful information to your team
  12. 12. TESTS CAN GET OUTDATED
  13. 13. Ideas for VALUABLE tests
  14. 14. Re-evaluate & Analyze ● What is this test doing? ● What value is this test providing? ● Is this testing the right thing?
  15. 15. Quality vs Quantity
  16. 16. Collaborate on testing
  17. 17. Treat test code like production code
  18. 18. RELIABLE
  19. 19. Tests should always have the same result
  20. 20. Antipattern: Unreliable tests EVEN ONE FAILURE MAKES A DIFFERENCE...
  21. 21. Ideas for RELIABLE tests
  22. 22. Isolate flaky tests Isolate flaky test Analyze Fix Move back to build
  23. 23. Monitor flakiness ● For each test run, record the failures ○ Test ○ Failure ○ # of times failed
  24. 24. Automatically rerun failures
  25. 25. Use specific test selectors
  26. 26. Independent tests ● Tests should all be hermetic ● Execution of one test does not affect others ● Run in dedicated testing environment
  27. 27. FAST
  28. 28. TESTS HAVE A SHORT FEEDBACK LOOP
  29. 29. TESTS SHOULDN’T FEEL LIKE A MARATHON
  30. 30. UNIT & INTEGRATION TESTS DEPLOY TO STAGING UI TESTS DEPLOY TO PRODUCTION POST DEPLOY TESTS CONTINUOUS DEPLOYMENT PIPELINE
  31. 31. Ideas for fasts tests
  32. 32. Parallelize
  33. 33. Avoid testing in the UI
  34. 34. Headless tests ● XVFB ● PhantomJS
  35. 35. REMEMBER...
  36. 36. There are tradeoffs
  37. 37. It’s okay to start over
  38. 38. Testing is a team effort
  39. 39. THANKS! @iammeaghanlewis linkedin.com/in/meaghan-lewis/ meaghan@lever.co

×