Nobody likes it. Everyone has to do it: Maintenance for code and tests. In this presentation we go over a strategy for taking care of our tests, and how to fix the bad ones.

1. @gil_zilberfeld@gil_zilberfeld
Zen and the Art of
Test Maintenance

2. @gil_zilberfeld
Hello!
I AM GIL ZILBERFELD
TestinGil
www.gilzilberfeld.com
www.everydayunittesting.com
@gil_zilberfeld

3. @gil_zilberfeld@gil_zilberfeld
Want the book at 50% discount?
http://www.everydayunittesting.com/testil-book-coupon

4. @gil_zilberfeld@gil_zilberfeld
Test Maintenance
What is it good for?

5. @gil_zilberfeld

6. @gil_zilberfeld
The value of our tests
◉ Alignment with product focus
◉ Cover the right areas
◉ Stability
◉ Find bugs
◉ Pinpoint the problems

7. @gil_zilberfeld@gil_zilberfeld
Strategy

8. @gil_zilberfeld

9. @gil_zilberfeld@gil_zilberfeld
Once we got a map
We can create a plan of where we want to go

10. @gil_zilberfeld
High level view
◉ What is important for the product?
◉ What are the risks?
◉ What should we focus on?
◉ At this stage and the short-mid term future

11. @gil_zilberfeld

12. @gil_zilberfeld@gil_zilberfeld
What about Facebook?
Importance, risk, focus

13. @gil_zilberfeld
◉ Important:
◉ Risks
◉ Focus
Ad sales, trust
Losing money, finding out what we
really do, showing relevant
information
Algorithms
Facebook

14. @gil_zilberfeld@gil_zilberfeld
What about Kayak?
Importance, risks, focus

15. @gil_zilberfeld
◉ Important:
◉ Risks:
◉ Focus:
Commerce, Immediacy
Dependency on other systems
Information, reliability, accuracy
Integration
Kayak

16. @gil_zilberfeld

17. @gil_zilberfeld
◉ The tests we have
◉ The workflows we
cover (value)
◉ The risks we answer
◉ The costs incurred by
the tests
◉ The dependencies we
rely on
◉ The architecture
◉ The resources
◉ The skills
Testing cartography

18. @gil_zilberfeld

19. @gil_zilberfeld
◉ “Main” Workflow
coverage
◉ Architectural stability
◉ Workflow stability
◉ Manual regression
testing time
◉ Time to feedback
◉ Escaped bugs that we
could have found(by
customer or internally)
What to track

20. @gil_zilberfeld

21. @gil_zilberfeld
Test suite clean up
◉ Organize sanity, regression and acceptance
◉ Delete “Ignored” tests
◉ Fix grouping for test cohesion

22. @gil_zilberfeld@gil_zilberfeld
Where should we add the next test?

23. @gil_zilberfeld
When adding tests
◉ Add tests at the right level
◉ Move “older stable” area tests to later pipeline
cycles
◉ Move “newer unstable” area tests to earlier
cycles

24. @gil_zilberfeld

25. @gil_zilberfeld
Calculator Requirements

26. @gil_zilberfeld

27. @gil_zilberfeld
Uninformative tests
◉ Tests that don’t point to the problem
◉ Not enough information on failure
◉ Checking too many operations
◉ No overlapping between test types
(triangulation)

28. @gil_zilberfeld

29. @gil_zilberfeld
Unreliable tests
◉ Inability to consistently run anywhere, anytime
◉ Flaky tests
◉ Dependency on unstable resources, platforms
◉ Test isolation (resources, initialization, cleanup)

30. @gil_zilberfeld

31. @gil_zilberfeld
◉ Data transformation
◉ Tests that always pass
◉ Unclear test names
◉ Logic in tests
◉ Readability
◉ Code matching
◉ No assertions
Misleading tests

32. @gil_zilberfeld

33. @gil_zilberfeld
Tests that hurt you
◉ Hard to setup
◉ Test run length
◉ Build cycle run length

34. @gil_zilberfeld

35. @gil_zilberfeld
Maintenance issues
◉ Copy-paste, duplication
◉ Wrong location for tests
◉ Tests that do same thing “just to be on the safe
side”
◉ Verbose and big setup, no framework (Page
object)

36. @gil_zilberfeld

37. @gil_zilberfeld
Lazy creates the norm
◉ Copy-and-paste duplication
◉ Bigger tests over smaller ones
◉ It may not be that good for you

38. @gil_zilberfeld

39. @gil_zilberfeld
Testability
◉ Accessibility
◉ Seams
◉ Big setup and mocks
◉ Shared state
◉ Unknown side effects

40. @gil_zilberfeld

41. @gil_zilberfeld

42. @gil_zilberfeld

43. @gil_zilberfeld
Thanks!
ANY QUESTIONS?
You can find me at:
gil.zilberfeld@gmail.com
@gil_zilberfeld
Book coupon