Download to read offline
Uploaded byGil Zilberfeld
Zen and the Art of Test Maintenance
This document discusses strategies for test maintenance. It emphasizes the importance of aligning tests with product focus and risks. It provides examples of how to evaluate tests and identifies common issues like uninformative, unreliable, and misleading tests. The document also covers testability, refactoring, behavior-driven development, and techniques like test reviews and pair programming. Exercises are included to help learn test maintenance practices.
More Related Content
Similar to Zen and the Art of Test Maintenance
Zen and the Art of Test Maintenance
- 1. @gil_zilberfeld@gil_zilberfeld Zen and theArt of Test Maintenance
- 2. @gil_zilberfeld Hello! I AM GILZILBERFELD www.gilzilberfeld.com www.everydayunittesting.com @gil_zilberfeld
- 3. @gil_zilberfeld For the exercises: ◉https://github.com/gilzilberfeld/test-maintenance-exercises ◉https://bit.ly/2JaM3Pb ◉Also a Diff tool (KDiff3)
- 4.
- 5.
- 6. @gil_zilberfeld The value ofour tests ◉ Alignment with product focus ◉ Cover the right areas ◉ Stability ◉ Find bugs ◉ Pinpoint the problems
- 7.
- 8.
- 9. @gil_zilberfeld@gil_zilberfeld Once we gota map We can create a plan of where we want to go
- 10. @gil_zilberfeld High level view ◉What is important for the product? ◉ What are the risks? ◉ What should we focus on? ◉ At this stage and the short-mid term future
- 11.
- 12.
- 13. @gil_zilberfeld ◉ Important: ◉ Risks ◉Focus Ad sales, trust Losing money, finding out what we really do, showing relevant information Algorithms Facebook
- 14.
- 15. @gil_zilberfeld ◉ Important: ◉ Risks: ◉Focus: Commerce, Immediacy Dependency on other systems Information, reliability, accuracy Integration Kayak
- 16.
- 17. @gil_zilberfeld ◉ The testswe have ◉ The workflows we cover (value) ◉ The risks we answer ◉ The costs incurred by the tests ◉ The dependencies we rely on ◉ The architecture ◉ The resources ◉ The skills Testing cartography
- 18.
- 19. @gil_zilberfeld ◉ “Main” Workflow coverage ◉Architectural stability ◉ Workflow stability ◉ Manual regression testing time ◉ Time to feedback ◉ Escaped bugs that we could have found(by customer or internally) What to track
- 20.
- 21.
- 22. @gil_zilberfeld Exercise ◉ Build atest plan ◉ Examine the code and tests ◉ Recommend what to do next ◉ And why
- 23.
- 24. @gil_zilberfeld Test suite cleanup ◉ Organize sanity, regression and acceptance ◉ Delete “Ignored” tests ◉ Fix grouping for test cohesion
- 25. @gil_zilberfeld When adding tests ◉Add tests at the right level ◉ Move “older stable” area tests to later build cycles ◉ Move “newer unstable” area tests to earlier cycles
- 26.
- 27. @gil_zilberfeld Exercise: Unit Tests ◉Find issues and ways to improve ◉ Refactor the tests ◉ Save the older version
- 28.
- 29. @gil_zilberfeld Uninformative tests ◉ Teststhat don’t point to the problem ◉ Not enough information on failure ◉ Checking too many operations ◉ No overlapping between test types (triangulation)
- 30.
- 31. @gil_zilberfeld Unreliable tests ◉ Inabilityto consistently run anywhere, anytime ◉ Flaky tests ◉ Dependency on unstable resources, platforms ◉ Test isolation (resources, initialization, cleanup)
- 32.
- 33. @gil_zilberfeld ◉ Data transformation ◉Tests that always pass ◉ Unclear test names ◉ Logic in tests ◉ Readability ◉ Code matching ◉ No assertions Misleading tests
- 34.
- 35. @gil_zilberfeld Tests that hurtyou ◉ Hard to setup ◉ Test run length ◉ Build cycle run length
- 36.
- 37. @gil_zilberfeld Maintenance issues ◉ Copy-paste,duplication ◉ Where do I add the next test? ◉ Tests that do same thing “just to be on the safe side” ◉ Verbose and big setup, no framework (Page object)
- 38. @gil_zilberfeld Exercise: Integration tests ◉Check out the integration tests ◉ Also consider the unit tests! ◉ Refactor ◉ Save the original version
- 39.
- 40. @gil_zilberfeld Lazy creates thenorm ◉ Copy-and-paste duplication ◉ Bigger tests over smaller ones ◉ It may not be that good for you
- 41. @gil_zilberfeld Exercise: Adding testsfor existing functionality ◉ From the list of your test plan ◉ Reorganize the suite ◉ Decide where to put it ◉ Find where it hurts and why
- 42.
- 43. @gil_zilberfeld Testability ◉ Accessibility ◉ Seams ◉Big setup and mocks ◉ Shared state ◉ Unknown side effects
- 44.
- 45. @gil_zilberfeld@gil_zilberfeld Refactoring for testability Anda very short introduction to ApprovalTests
- 46. @gil_zilberfeld Characterization tests ◉ Describethe existing state of the implementation ◉ Run the test without an assert ◉ Capture the result in an assert ◉ Then we can refactor
- 47. @gil_zilberfeld ApprovalTests ◉ Open-source toolfor approval testing ◉ Used to lock a state by capturing output ◉ Can be used as a substitute for “other” tests ◉ Creates a Golden Standard ◉ Start refactoring with small steps
- 48.
- 49. @gil_zilberfeld Exercise ◉ Write upa few tests cases in an Approval test ◉ Refactor the code after adding the tests ◉ Replace the approval tests with “regular” tests
- 50.
- 51. @gil_zilberfeld Behavior Driven Development ◉Created to allow “humans” to read the tests ◉ Gherkin statements are translated into “glue” code that operate the underlying code ◉ You don’t really need it for that ◉ But it may be beneficial in some cases
- 52.
- 53. @gil_zilberfeld Exercise ◉ Write themulti-parameter test in Gherkin and add the implementation ◉ How would you rearrange the tests given this capability
- 54.
- 55.
- 56. @gil_zilberfeld Why review? ◉ Findproblems early ◉ Fight complexity ◉ Knowledge sharing ◉ Collective code ownership ◉ Maintain conventions
- 57. @gil_zilberfeld@gil_zilberfeld Ask: Can Imaintain this code?
- 58. @gil_zilberfeld Who reviews? ◉ Teamlead ◉ Architect ◉ Developer ◉ Peers ◉ The person who can bring the most value
- 59.
- 60. @gil_zilberfeld Short reviews ◉ Shortreviews means higher availability ◉ No more than 60 minutes ◉ No more than 200 lines of code ◉ Less is better
- 61. @gil_zilberfeld Async review ◉ Smellof attitude ◉ Better together ◉ Better for distributed teams ◉ Consider the commit status
- 62.
- 63.
- 64.
- 65. @gil_zilberfeld When to review? ◉Before committing ◉ After merging ◉ When you’re ready
- 66. @gil_zilberfeld What to review? ◉Functionality ◉ Are all necessary cases covered? ◉ Are they written correctly? ◉ Is the test run time quick? ◉ Remove ignored tests
- 67. @gil_zilberfeld What to review? ◉“What if” scenarios ◉ English ◉ Performance ◉ Impact on other systems ◉ Dependency on other systems
- 68. @gil_zilberfeld What to review? ◉Compliance Conventions Tools Legal
- 69.
- 70. @gil_zilberfeld Pair programming andtesting ◉ Better together ◉ No need to review! ◉ Be nice
- 71. @gil_zilberfeld Exercise: Review thechanges ◉ Pair with someone ◉ Express views ◉ Try to stick to the patterns
- 72.
- 73. @gil_zilberfeld Thanks! ANY QUESTIONS? You canfind me at: @gil_zilberfeld http://www.GilZilberfeld.com http://www.EverydayUnitTesting.com