Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
KEEPING
YOUR TESTS
LEAN
@iammeaghanlewis
meaghan@lever.co
meaghan-lewis
WHAT DOES
LEAN
MEAN ANYWAY?
VALUABLE
RELIABLE
FAST
1000+ TESTS
12 HR BUILD
100% FAILURE RATE
HOW DID WE GET
A LEAN
TEST SUITE?
NOT QUITE...
VALUABLE
Tests provide useful
information to your team
TESTS CAN GET OUTDATED
Ideas for VALUABLE tests
Re-evaluate & Analyze
● What is this test doing?
● What value is this test providing?
● Is this testing the right thing?
Quality vs Quantity
Collaborate on testing
Treat test code like
production code
RELIABLE
Tests should always have
the same result
Antipattern: Unreliable tests
EVEN ONE FAILURE MAKES A DIFFERENCE...
Ideas for RELIABLE tests
Isolate flaky tests
Isolate
flaky test
Analyze Fix
Move
back to
build
Monitor flakiness
● For each test run, record the failures
○ Test
○ Failure
○ # of times failed
Automatically rerun failures
Use specific test selectors
Independent tests
● Tests should all be hermetic
● Execution of one test does not affect others
● Run in dedicated testing...
FAST
TESTS HAVE A SHORT FEEDBACK
LOOP
TESTS SHOULDN’T FEEL LIKE A MARATHON
UNIT &
INTEGRATION
TESTS
DEPLOY TO
STAGING UI TESTS DEPLOY TO
PRODUCTION
POST DEPLOY
TESTS
CONTINUOUS DEPLOYMENT PIPELINE
Ideas for fasts tests
Parallelize
Avoid testing in the UI
Headless tests
● XVFB ● PhantomJS
REMEMBER...
There are tradeoffs
It’s okay to start over
Testing is a team effort
THANKS!
@iammeaghanlewis
linkedin.com/in/meaghan-lewis/
meaghan@lever.co
Keeping your tests lean
Keeping your tests lean
Keeping your tests lean
Keeping your tests lean
Upcoming SlideShare
Loading in …5
×

Keeping your tests lean

557 views

Published on

We've come to realize that automation provides an immense amount of value in preventing regressions and helping to deliver quality software. As your automation grows and grows, it requires continuous maintenance so that tests remain fast, reliable, and valuable. If you're not scaling efficiently, your automation suite will turn into a messy, uncontrollable beast. Having a lean test suite will help to combat this.

This presentation shares methods that you can use to keep your automated test suites lean and mean, so they always provide quick and accurate feedback to your software delivery team.

Published in: Software
  • Be the first to comment

Keeping your tests lean

  1. 1. KEEPING YOUR TESTS LEAN
  2. 2. @iammeaghanlewis meaghan@lever.co meaghan-lewis
  3. 3. WHAT DOES LEAN MEAN ANYWAY?
  4. 4. VALUABLE RELIABLE FAST
  5. 5. 1000+ TESTS
  6. 6. 12 HR BUILD
  7. 7. 100% FAILURE RATE
  8. 8. HOW DID WE GET A LEAN TEST SUITE?
  9. 9. NOT QUITE...
  10. 10. VALUABLE
  11. 11. Tests provide useful information to your team
  12. 12. TESTS CAN GET OUTDATED
  13. 13. Ideas for VALUABLE tests
  14. 14. Re-evaluate & Analyze ● What is this test doing? ● What value is this test providing? ● Is this testing the right thing?
  15. 15. Quality vs Quantity
  16. 16. Collaborate on testing
  17. 17. Treat test code like production code
  18. 18. RELIABLE
  19. 19. Tests should always have the same result
  20. 20. Antipattern: Unreliable tests EVEN ONE FAILURE MAKES A DIFFERENCE...
  21. 21. Ideas for RELIABLE tests
  22. 22. Isolate flaky tests Isolate flaky test Analyze Fix Move back to build
  23. 23. Monitor flakiness ● For each test run, record the failures ○ Test ○ Failure ○ # of times failed
  24. 24. Automatically rerun failures
  25. 25. Use specific test selectors
  26. 26. Independent tests ● Tests should all be hermetic ● Execution of one test does not affect others ● Run in dedicated testing environment
  27. 27. FAST
  28. 28. TESTS HAVE A SHORT FEEDBACK LOOP
  29. 29. TESTS SHOULDN’T FEEL LIKE A MARATHON
  30. 30. UNIT & INTEGRATION TESTS DEPLOY TO STAGING UI TESTS DEPLOY TO PRODUCTION POST DEPLOY TESTS CONTINUOUS DEPLOYMENT PIPELINE
  31. 31. Ideas for fasts tests
  32. 32. Parallelize
  33. 33. Avoid testing in the UI
  34. 34. Headless tests ● XVFB ● PhantomJS
  35. 35. REMEMBER...
  36. 36. There are tradeoffs
  37. 37. It’s okay to start over
  38. 38. Testing is a team effort
  39. 39. THANKS! @iammeaghanlewis linkedin.com/in/meaghan-lewis/ meaghan@lever.co

×