Your SlideShare is downloading. ×
Are Your Tests REALLY     Helping?Sunday, October 9, 2011
Motivation           Why do we test, anyway?Sunday, October 9, 2011
Bad MotivationSunday, October 9, 2011
Bad Motivation                    • “My boss told me to”Sunday, October 9, 2011
Bad Motivation                    • “My boss told me to”                    • “I’m a completist”Sunday, October 9, 2011
Bad Motivation                    • “My boss told me to”                    • “I’m a completist”                    • “I  ...
Correct MotivationSunday, October 9, 2011
Correct Motivation                    • Verify CorrectnessSunday, October 9, 2011
Correct Motivation                    • Verify Correctness                    • Gain ConfidenceSunday, October 9, 2011
Correct Motivation                    • Verify Correctness                    • Gain Confidence                    • Commun...
Test Types                          Vocabulary Break!Sunday, October 9, 2011
Functional Tests                          Does the overall                          product satisfy the                   ...
Integration Tests                          Do the pieces fit                          together?Sunday, October 9, 2011
Unit Tests                          Is the logic correct in                          that function?Sunday, October 9, 2011
Functional             Understand the product.Sunday, October 9, 2011
The ToolsSunday, October 9, 2011
The Tools                    • BeHat (Cucumber)Sunday, October 9, 2011
The Tools                    • BeHat (Cucumber)                    • PHPSpecSunday, October 9, 2011
The Tools                    • BeHat (Cucumber)                    • PHPSpec                    • Keyboard, Mouse, and You...
When Do They Work?                    • During prototyping                    • Focused on the product requirements       ...
...Stop Helping?                    • Focused on the implementation                    • Rapidly changing functionality   ...
Unit Tests              Simple and to the point.Sunday, October 9, 2011
Verify Correctness                    • Line coverage                    • Branch coverage                    • Icky Bits-...
Gain Confidence                    • Individual functions                    • Variety of parameters                    • W...
Communication                    • Show how to use the function                    • Show expected interactions with other...
PHPUnit StandardSunday, October 9, 2011
Naming                    • Pick a directory just for tests                    • Pick a test file name suffix, ie. Test.php ...
Classes                    • One class per test file                    • Name should match the file name                   ...
Functions                    • No private functions                    • Protected methods are reserved for               ...
Control Structures                    • There should be NO control structuresSunday, October 9, 2011
PHPUnit Code Standard                •         AllowedFunctionOverride                •         ClassName                •...
PHP_CodeSniffer                • CyclomaticComplexity (maximum of 1)                • NestingLevel (maximum of 0)         ...
PHP_CodeSniffer                          Standard Available at:                          http://github.com/elblinkin/     ...
Save Time                    • Test code MTTI of 0                    • Reduce the connascence                    • Scales...
More...                    • Ensure only one class is under test                    • Avoid accessing globals             ...
Integration                          For everything in-between.Sunday, October 9, 2011
You’re Paranoid                    • Experimenting with third-party code or                      service                  ...
Save Time                    • Save Time                          • Discover integration issues early                    •...
Warning                    • Use them for experimentation                    • Consider an alert instead                  ...
Good or Bad?                    • Mock         • Code Coverage                    • Manual       • Automated              ...
Review                    • Functional Tests                          • prototyping, early stage, small team              ...
Other Test Types                    • Performance Testing                    • Load Testing                    • Security ...
• Who Am I?                          @elblinkin : Etsy, Inc.                • Want More?                          http://c...
Upcoming SlideShare
Loading in...5
×

Are Your Tests Really Helping You?

4,515

Published on

Warning: This is advanced content. Prior knowledge of PHPUnit and functional testing are required.

Published in: Technology, Business
3 Comments
8 Likes
Statistics
Notes
  • Look at my blog for more details: http://elblinkin.info
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Private methods can grow logic. Also, there are two better alternatives. First you can copy/paste and that will make your tests more readable up front, instead of scrolling back and forth through code to read the test; however, if you find you are copy/pasting a lot of lines, then that is a smell indicating that you should extract to the method to a helper class in your test framework/extension code so that it can be tested. The second could also be a smell indicating that your api is not appropriate in that it requires a lot of context to work and thus is not encapsulated properly.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • i don’t understand why it’s good to eliminate private methods from test classes. it’ll either lead to duplication or very specific test utility classes. I’ve found them really useful, and several patterns use them in Xunit Test Patterns, for example: http://xunitpatterns.com/Test%20Utility%20Method.html
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
4,515
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
18
Comments
3
Likes
8
Embeds 0
No embeds

No notes for slide

Transcript of "Are Your Tests Really Helping You?"

  1. 1. Are Your Tests REALLY Helping?Sunday, October 9, 2011
  2. 2. Motivation Why do we test, anyway?Sunday, October 9, 2011
  3. 3. Bad MotivationSunday, October 9, 2011
  4. 4. Bad Motivation • “My boss told me to”Sunday, October 9, 2011
  5. 5. Bad Motivation • “My boss told me to” • “I’m a completist”Sunday, October 9, 2011
  6. 6. Bad Motivation • “My boss told me to” • “I’m a completist” • “I Test Frameworks”Sunday, October 9, 2011
  7. 7. Correct MotivationSunday, October 9, 2011
  8. 8. Correct Motivation • Verify CorrectnessSunday, October 9, 2011
  9. 9. Correct Motivation • Verify Correctness • Gain ConfidenceSunday, October 9, 2011
  10. 10. Correct Motivation • Verify Correctness • Gain Confidence • CommunicationSunday, October 9, 2011
  11. 11. Test Types Vocabulary Break!Sunday, October 9, 2011
  12. 12. Functional Tests Does the overall product satisfy the the requirements?Sunday, October 9, 2011
  13. 13. Integration Tests Do the pieces fit together?Sunday, October 9, 2011
  14. 14. Unit Tests Is the logic correct in that function?Sunday, October 9, 2011
  15. 15. Functional Understand the product.Sunday, October 9, 2011
  16. 16. The ToolsSunday, October 9, 2011
  17. 17. The Tools • BeHat (Cucumber)Sunday, October 9, 2011
  18. 18. The Tools • BeHat (Cucumber) • PHPSpecSunday, October 9, 2011
  19. 19. The Tools • BeHat (Cucumber) • PHPSpec • Keyboard, Mouse, and YouSunday, October 9, 2011
  20. 20. When Do They Work? • During prototyping • Focused on the product requirements • Refactoring • Regression of key features • Better for smaller teamsSunday, October 9, 2011
  21. 21. ...Stop Helping? • Focused on the implementation • Rapidly changing functionality • Large organizationsSunday, October 9, 2011
  22. 22. Unit Tests Simple and to the point.Sunday, October 9, 2011
  23. 23. Verify Correctness • Line coverage • Branch coverage • Icky Bits-o-LogicSunday, October 9, 2011
  24. 24. Gain Confidence • Individual functions • Variety of parameters • Works for expected interactions with collaboratorsSunday, October 9, 2011
  25. 25. Communication • Show how to use the function • Show expected interactions with other collaborators • Increase discoverability of possible reuseSunday, October 9, 2011
  26. 26. PHPUnit StandardSunday, October 9, 2011
  27. 27. Naming • Pick a directory just for tests • Pick a test file name suffix, ie. Test.php • Match the name of the test file to the name of the file being testedSunday, October 9, 2011
  28. 28. Classes • One class per test file • Name should match the file name • Extend only tested TestCase classes • No interfaces or abstract classesSunday, October 9, 2011
  29. 29. Functions • No private functions • Protected methods are reserved for overriding parent methods • Only provider and test methods are allowed • Make sure your provider and test methods are publicSunday, October 9, 2011
  30. 30. Control Structures • There should be NO control structuresSunday, October 9, 2011
  31. 31. PHPUnit Code Standard • AllowedFunctionOverride • ClassName • ExtraneousClass • FilenameSuffix • NoInterfaces • NoPrivateMethods • ProvenTestCase • TestOrProviderFunctionsOnly • TestOrProviderIsPublic • UnusedProviderSunday, October 9, 2011
  32. 32. PHP_CodeSniffer • CyclomaticComplexity (maximum of 1) • NestingLevel (maximum of 0) • UnnecessaryOverrideSunday, October 9, 2011
  33. 33. PHP_CodeSniffer Standard Available at: http://github.com/elblinkin/ PHPUnit-CodeSnifferSunday, October 9, 2011
  34. 34. Save Time • Test code MTTI of 0 • Reduce the connascence • Scales with growing development teamSunday, October 9, 2011
  35. 35. More... • Ensure only one class is under test • Avoid accessing globals • Environment variables • Configuration from disk • Prevent interactions with external resources • Code Standard for TestabilitySunday, October 9, 2011
  36. 36. Integration For everything in-between.Sunday, October 9, 2011
  37. 37. You’re Paranoid • Experimenting with third-party code or service • You do not trust that your collaborators work as specifiedSunday, October 9, 2011
  38. 38. Save Time • Save Time • Discover integration issues early • Waste of Time • Time spent testing third-party codeSunday, October 9, 2011
  39. 39. Warning • Use them for experimentation • Consider an alert instead • Do not inflict them on othersSunday, October 9, 2011
  40. 40. Good or Bad? • Mock • Code Coverage • Manual • Automated • Copy/Paste • ReuseSunday, October 9, 2011
  41. 41. Review • Functional Tests • prototyping, early stage, small team • Unit Tests • can be too much for early stage • scale with a growing team size and code base • Integration Tests • only because you are paranoidSunday, October 9, 2011
  42. 42. Other Test Types • Performance Testing • Load Testing • Security Testing • Monitoring • LoggingSunday, October 9, 2011
  43. 43. • Who Am I? @elblinkin : Etsy, Inc. • Want More? http://codeascraft.etsy.com http://github.com/etsy http://elblinkin.infoSunday, October 9, 2011

×