Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Zen And the Art of Test Maintenance Presentation

136 views

Published on

A Presentation on automated test maintenance and test issues that we can improve. Presented at the Test.IL Northern meetup.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Zen And the Art of Test Maintenance Presentation

  1. 1. @gil_zilberfeld@gil_zilberfeld Zen and the Art of Test Maintenance
  2. 2. @gil_zilberfeld Hello! I AM GIL ZILBERFELD TestinGil www.gilzilberfeld.com www.everydayunittesting.com @gil_zilberfeld
  3. 3. @gil_zilberfeld@gil_zilberfeld Test Maintenance What is it good for?
  4. 4. @gil_zilberfeld
  5. 5. @gil_zilberfeld The value of our tests ◉ Alignment with product focus ◉ Cover the right areas ◉ Stability ◉ Find bugs ◉ Pinpoint the problems
  6. 6. @gil_zilberfeld@gil_zilberfeld Strategy
  7. 7. @gil_zilberfeld
  8. 8. @gil_zilberfeld@gil_zilberfeld Once we got a map We can create a plan of where we want to go
  9. 9. @gil_zilberfeld High level view ◉ What is important for the product? ◉ What are the risks? ◉ What should we focus on? ◉ At this stage and the short-mid term future
  10. 10. @gil_zilberfeld
  11. 11. @gil_zilberfeld@gil_zilberfeld What about Facebook? Importance, risk, focus
  12. 12. @gil_zilberfeld ◉ Important: ◉ Risks ◉ Focus Ad sales, trust Losing money, finding out what we really do, showing relevant information Algorithms Facebook
  13. 13. @gil_zilberfeld@gil_zilberfeld What about Kayak? Importance, risks, focus
  14. 14. @gil_zilberfeld ◉ Important: ◉ Risks: ◉ Focus: Commerce, Immediacy Dependency on other systems Information, reliability, accuracy Integration Kayak
  15. 15. @gil_zilberfeld
  16. 16. @gil_zilberfeld ◉ The tests we have ◉ The workflows we cover (value) ◉ The risks we answer ◉ The costs incurred by the tests ◉ The dependencies we rely on ◉ The architecture ◉ The resources ◉ The skills Testing cartography
  17. 17. @gil_zilberfeld
  18. 18. @gil_zilberfeld ◉ “Main” Workflow coverage ◉ Architectural stability ◉ Workflow stability ◉ Manual regression testing time ◉ Time to feedback ◉ Escaped bugs that we could have found(by customer or internally) What to track
  19. 19. @gil_zilberfeld
  20. 20. @gil_zilberfeld Test suite clean up ◉ Organize sanity, regression and acceptance ◉ Delete “Ignored” tests ◉ Fix grouping for test cohesion
  21. 21. @gil_zilberfeld When adding tests ◉ Add tests at the right level ◉ Move “older stable” area tests to later build cycles ◉ Move “newer unstable” area tests to earlier cycles
  22. 22. @gil_zilberfeld
  23. 23. @gil_zilberfeld
  24. 24. @gil_zilberfeld Uninformative tests ◉ Tests that don’t point to the problem ◉ Not enough information on failure ◉ Checking too many operations ◉ No overlapping between test types (triangulation)
  25. 25. @gil_zilberfeld
  26. 26. @gil_zilberfeld Unreliable tests ◉ Inability to consistently run anywhere, anytime ◉ Flaky tests ◉ Dependency on unstable resources, platforms ◉ Test isolation (resources, initialization, cleanup)
  27. 27. @gil_zilberfeld
  28. 28. @gil_zilberfeld ◉ Data transformation ◉ Tests that always pass ◉ Unclear test names ◉ Logic in tests ◉ Readability ◉ Code matching ◉ No assertions Misleading tests
  29. 29. @gil_zilberfeld
  30. 30. @gil_zilberfeld Tests that hurt you ◉ Hard to setup ◉ Test run length ◉ Build cycle run length
  31. 31. @gil_zilberfeld
  32. 32. @gil_zilberfeld Maintenance issues ◉ Copy-paste, duplication ◉ Where do I add the next test? ◉ Tests that do same thing “just to be on the safe side” ◉ Verbose and big setup, no framework (Page object)
  33. 33. @gil_zilberfeld
  34. 34. @gil_zilberfeld Lazy creates the norm ◉ Copy-and-paste duplication ◉ Bigger tests over smaller ones ◉ It may not be that good for you
  35. 35. @gil_zilberfeld
  36. 36. @gil_zilberfeld Testability ◉ Accessibility ◉ Seams ◉ Big setup and mocks ◉ Shared state ◉ Unknown side effects
  37. 37. @gil_zilberfeld
  38. 38. @gil_zilberfeld
  39. 39. @gil_zilberfeld
  40. 40. @gil_zilberfeld Thanks! ANY QUESTIONS? You can find me at: gil.zilberfeld@gmail.com @gil_zilberfeld http://www.GilZilberfeld.com http://www.EverydayUnitTesting.com

×