Testing Antipatterns <ul><li>Matt Robinson </li></ul><ul><li>Puppet Labs </li></ul>
Anti - What?
Antipattern ( Dark Pattern ): <ul><li>A pattern that is commonly used but is ineffective and/or counterproductive </li></ul>
 
Who has an extensive test suite?
Who feels like they should test more?
Who dislikes, even dreads, writing tests?
Testing Can Be Fun Text Text Text Text http://xkcd.com/583/
For example:
Red   Green  TDD Addiction
Testing Framework Games (although these can be a distraction) (although these can be a distraction)
Not looking like an idiot - that’s fun right?
Why Automated Testing? <ul><li>Prevents regressions </li></ul><ul><li>Improves confidence </li></ul><ul><li>Documents requ...
Overheard Reasons Not to Test <ul><li>Don’t trust the tests </li></ul><ul><li>Who tests the tests? Quis custodiet ipsos cu...
Antipatterns that have bitten me the most
Not Running Tests Often Enough
If it’s Hard and Important Do it More Often
testing, deploying, releasing, exercising, meditating, writing, public speaking
How Often? <ul><li>Before deployment or release </li></ul><ul><li>After every commit: CI (Continuous Integration) - Jenkin...
Not Fixing Broken Tests Before Committing New Code
The Slowpoke
How Slow Is Too Slow?
Unit Test Suite Took 8 Hours to Run - On One Project
Single Test Took 2 Minutes to Start
So slow that rewriting it in PHP would make it faster
How to Get Faster <ul><li>Make your implementation code faster </li></ul><ul><li>Run your tests in parallel </li></ul><ul>...
The Mockery
Why mock when the real things will do?
Testing at the Wrong Level
unit, smoke, functional, integration, acceptance
Success Against All Odds (test never fails) (test never fails)
 
Testing Implementation Instead of Behavior
Testing Implementation
Testing Behavior
Order Dependent Failures
You’ve got too much global state
Excessive Setup
What are we testing again?
<ul><li>The Dodger - Never tests desired behavior </li></ul><ul><li>The Stranger - Misplaced tests </li></ul><ul><li>Local...
Summary
Avoid Antipatterns and Testing Can Be Fun
Links <ul><li>http://blog.james-carr.org/2006/11/03/tdd-anti-patterns/ </li></ul><ul><li>http://jasonrudolph.com/blog/test...
Upcoming SlideShare
Loading in...5
×

Open source bridge testing antipatterns presentation

773

Published on

These are the slides for the presentation I gave on antipatterns used in testing. This was at Open Source Bridge in June of 2011.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
773
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Ask questions whenever you want
  • Talk about Design Patterns general reusable solution to a commonly occurring problem
  • Realized while writing this talk that I was focusing on what pissed me off and made me not want to talk about it. Should be humor in testing.
  • Red failing Green passing
  • Not really an antipattern since it’s really only solving being lazy
  • CI essentially glorified cron job
  • The Joel Test #5: Fix bugs before writing new code Mention Lava Lamps
  • Martin Fowler Mocks aren&apos;t Stubs. Never met a mockist whose tests I liked. Not mocking means mini integration tests. Possibly controversial. When to mock: code you don&apos;t control - file system, web server, external libraries like ssl
  • Too much mocking might indicate this
  • Everyone has their own terms
  • Test Driven Development prevents this
  • Behaviour Driven Development is a larger term with more meaning
  • These are annoying but can’t be ignored. How to find these failures - binary search.
  • Testing antipatterns can indicate antipatterns in your implementation code
  • Data driven web apps tend to be even worse
  • Open source bridge testing antipatterns presentation

    1. 1. Testing Antipatterns <ul><li>Matt Robinson </li></ul><ul><li>Puppet Labs </li></ul>
    2. 2. Anti - What?
    3. 3. Antipattern ( Dark Pattern ): <ul><li>A pattern that is commonly used but is ineffective and/or counterproductive </li></ul>
    4. 5. Who has an extensive test suite?
    5. 6. Who feels like they should test more?
    6. 7. Who dislikes, even dreads, writing tests?
    7. 8. Testing Can Be Fun Text Text Text Text http://xkcd.com/583/
    8. 9. For example:
    9. 10. Red Green TDD Addiction
    10. 11. Testing Framework Games (although these can be a distraction) (although these can be a distraction)
    11. 12. Not looking like an idiot - that’s fun right?
    12. 13. Why Automated Testing? <ul><li>Prevents regressions </li></ul><ul><li>Improves confidence </li></ul><ul><li>Documents requirements </li></ul><ul><li>Catches errors before codes ships </li></ul><ul><li>You fill in the blank _________ </li></ul>
    13. 14. Overheard Reasons Not to Test <ul><li>Don’t trust the tests </li></ul><ul><li>Who tests the tests? Quis custodiet ipsos custodes? </li></ul><ul><li>Makes refactoring harder </li></ul><ul><li>Not enough return on investment (ROI) </li></ul><ul><li>Too hard and/or time consuming </li></ul>
    14. 15. Antipatterns that have bitten me the most
    15. 16. Not Running Tests Often Enough
    16. 17. If it’s Hard and Important Do it More Often
    17. 18. testing, deploying, releasing, exercising, meditating, writing, public speaking
    18. 19. How Often? <ul><li>Before deployment or release </li></ul><ul><li>After every commit: CI (Continuous Integration) - Jenkins, Cruise Control, BuildBot, Cerebrus, CI Joe </li></ul><ul><li>Before every commit (or push): discipline, git hooks </li></ul><ul><li>Every save: autotest, watchr </li></ul>
    19. 20. Not Fixing Broken Tests Before Committing New Code
    20. 21. The Slowpoke
    21. 22. How Slow Is Too Slow?
    22. 23. Unit Test Suite Took 8 Hours to Run - On One Project
    23. 24. Single Test Took 2 Minutes to Start
    24. 25. So slow that rewriting it in PHP would make it faster
    25. 26. How to Get Faster <ul><li>Make your implementation code faster </li></ul><ul><li>Run your tests in parallel </li></ul><ul><li>Optimize your tests (new framework, trick garbage collector, rewrite culprits) </li></ul><ul><li>Mark some tests as slow and only run them occasionally </li></ul>
    26. 27. The Mockery
    27. 28. Why mock when the real things will do?
    28. 29. Testing at the Wrong Level
    29. 30. unit, smoke, functional, integration, acceptance
    30. 31. Success Against All Odds (test never fails) (test never fails)
    31. 33. Testing Implementation Instead of Behavior
    32. 34. Testing Implementation
    33. 35. Testing Behavior
    34. 36. Order Dependent Failures
    35. 37. You’ve got too much global state
    36. 38. Excessive Setup
    37. 39. What are we testing again?
    38. 40. <ul><li>The Dodger - Never tests desired behavior </li></ul><ul><li>The Stranger - Misplaced tests </li></ul><ul><li>Local Hero - Worked on my dev box </li></ul><ul><li>Generous Leftovers - Fills your hard drive </li></ul><ul><li>The Giant - You may have a God object </li></ul><ul><li>The Loudmouth - Spams your test logs </li></ul>Other Antipatterns
    39. 41. Summary
    40. 42. Avoid Antipatterns and Testing Can Be Fun
    41. 43. Links <ul><li>http://blog.james-carr.org/2006/11/03/tdd-anti-patterns/ </li></ul><ul><li>http://jasonrudolph.com/blog/testing-anti-patterns-how-to-fail-with-100-test-coverage/ </li></ul><ul><li>http://blog.rubybestpractices.com/posts/gregory/034-issue-5-testing-antipatterns.html </li></ul><ul><li>http://c2.com/cgi/wiki?AntiPatternsCatalog </li></ul><ul><li>http://martinfowler.com/articles/mocksArentStubs.html </li></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×