Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An Introduction to Developer Testing


Published on

A whirlwind introduction to developer testing, with a .NET flavor

Published in: Technology
  • Be the first to comment

  • Be the first to like this

An Introduction to Developer Testing

  1. 1. Testing: 1… 2… 3…<br />An Introduction to DeveloperTesting<br />Will Green<br /><br /><br />@hotgazpacho<br />
  2. 2. Caveat<br />I am Not an Expert<br /><br />
  3. 3. I am not Kent Beck<br /><ul><li>Father of Extreme Programming
  4. 4. Pioneer of Software Design Patterns
  5. 5. Originator of CRC Cards
  6. 6. One of original signatories of the Agile Manifesto
  7. 7. Author, JUnit (unit testing framework for Java)
  8. 8. Author of several books, including Test Driven Development: By Example
  9. 9. Co-Author: Refactoring: Improving the Design of Existing Code</li></ul><br />
  10. 10. I am not Ward Cunningham<br /><ul><li>Creator of the Wiki
  11. 11. CTO of
  12. 12. Pioneer of Extreme Programming
  13. 13. Inventor of FIT (Framework for Integrated Tests)</li></ul><br />
  14. 14. I am not Martin Fowler<br /><ul><li>Chief Scientist at Thoughtworks
  15. 15. Co-Author: Refactoring: Improving the Design of Existing Code
  16. 16. Co-Author: Patterns of Enterprise Application Architecture
  17. 17. Author of several other books
  18. 18. One of original signatories of the Agile Manifesto</li></ul><br />
  19. 19. I am not Robert C. Martin<br /><ul><li>Author, Clean Code: A Handbook of Agile Software Craftsmanship
  20. 20. Author, Agile Principles, Patterns, and Practices in C#
  21. 21. Author, Agile Software Development: Principles, Patterns and Practices
  22. 22. Creator of FITNesse(combining FIT with a Wiki)
  23. 23. One of original signatories of the Agile Manifesto
  24. 24. Founder and President of Object Mentor, Inc.</li></ul><br />
  25. 25. I amWill Green<br /><ul><li>I am an Aspiring Software Craftsman
  26. 26. Co-host of The Iron Languages Podcast
  27. 27. I Read A LOT
  28. 28. I Practice A LOT
  29. 29. I Learn A LOT</li></ul><br />
  30. 30. Ready to drink from the fire hose?<br /><br />
  31. 31. Don’t Think “Test”<br />To-may-to, To-mah-to…<br /><br />
  32. 32.<br />Testing as Design<br />
  33. 33. Testing as Design<br /><br />
  34. 34. Testing as Design<br /><br />
  35. 35. Tests aren’t Tests<br />“Test” doesn’t describe what is going on<br />Tests are specification<br />Tests are design<br />Test suites are an artifact of the design process<br />
  36. 36. TDD: Test Driven Design<br />Write the code you wish you had<br />Hard to test == Hard to use<br />Better to find this out now than 2 months from now when you have to change it<br />Done right, helps you adhere to the Principles of Object Oriented Design<br />
  37. 37. SOLID<br />Like a Rock<br /><br />
  38. 38. Dependency Management<br />Dependency Management is <br />Hard<br />The single greatest influencer of software quality<br />The foundation for creating software that exhibits the qualities we desire<br />
  39. 39. Poorly Managed Dependencies<br />Lead to code that is<br />Hard to change<br />Fragile<br />Not reusable<br />
  40. 40. Well Managed Dependencies<br />Lead to code that is<br />Flexible<br />Robust<br />Reusable<br />
  41. 41. Qualities of Software<br />Undesirable<br />Hard to change<br />Fragile<br />Not reusable<br />Desirable<br />Easy to change<br />Robust<br />Reusable<br />
  42. 42. Dependency Management and SOLID<br />SOLID is<br />The foundational set of principles for managing dependencies<br />Therefor the foundation for creating software that exhibits the qualities we desire<br />
  43. 43.<br />
  44. 44. What is SOLID?<br />Single Responsibility Principle<br />Open Closed Principle<br />Liskov Substitution Principle<br />Interface Segregation Principle<br />Dependency Inversion Principle<br />
  45. 45.<br />
  46. 46. Single Responsibility Principle<br />A class should have one, and only one, reason to change<br /><br />
  47. 47.<br />
  48. 48. Open Closed Principle<br />You should be able to extend the behavior of a class, without modifying it<br />Classes should be open for extension but closed for modification<br /><br />
  49. 49.<br />
  50. 50. Liskov Substitution Principle<br />Derived classes must be substitutable for their base classes<br /><br />
  51. 51.<br />
  52. 52. Interface Segregation Principle<br />Make fine-grained interfaces that are client-specific<br /><br />
  53. 53.<br />
  54. 54. Dependency Inversion Principle<br />Depend on abstractions, not on specific implementations<br /><br />
  55. 55. Test Driven Design<br />The Path to Success: 3 Simple Rules<br /><br />
  56. 56. Rule #1<br />You are not allowed to write any production code unless it is to make a failing unit test pass.<br />
  57. 57. Rule #2<br />You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.<br />
  58. 58. Rule #3<br />You are not allowed to write any more production code than is sufficient to pass the one failing unit test.<br />
  59. 59. The Flow of Test Driven Design<br />Write a Test<br />Watch Test Fail<br />Write Code<br />Watch Test Pass<br />Refactor<br />See step 1<br /><br />
  60. 60. TDD Demo!<br />Fibonacci Number Generator<br />Sequence of numbers<br />First two numbers are 0 and 1<br />Each subsequent number is the sum of the two previous numbers<br />0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …<br />
  61. 61. How To Test<br /><br />
  62. 62. An Example<br />
  63. 63. Test Naming<br />Method being tested<br />Conditions<br />Expected Outcome<br />
  64. 64. Testing Pattern<br />Arrange, Act, Assert<br />Separates what is being tested from the setup and verification steps<br />Makes some Test Smells more obvious<br />Assertions mixed in with “Act” code.<br />Test methods that try to test too much at once<br />
  65. 65. Arrange<br />Arrange all necessary preconditions and inputs<br />
  66. 66. Act<br />Act on the object or method under test<br />
  67. 67. Assert<br />Assert that the expected results have occurred<br />
  68. 68. Things to Remember<br />Triple A<br />Take Small Steps<br />Run tests after every change<br />Tests are Code; maintain them!<br />
  69. 69. Test Doubles<br /><br />
  70. 70. Mock Repository?<br />Mock? Stub? What is this?<br />
  71. 71. Test Doubles<br />Think “stunt double”<br />Replacement for a Production object for testing purposes<br />There are many types of Test Doubles, each with its own purpose<br />
  72. 72. Dummy Objects<br />Passed around but never used<br />Simply to fill parameter lists<br />
  73. 73. Fake Objects<br />Have working implementations<br />Usually just enough for them to work in the context of the specific test<br />However, not suitable for Production<br />E.g. In-Memory Database<br />
  74. 74. Stubs<br />Provide canned answers to SUT<br />Usually don’t respond to anything more than what was programmed for the context of the test<br />
  75. 75. Spies<br />Stubs that record information on how they were called<br />Allows inspection for Assertion<br />Example<br />Email Service that records the messages it was sent (without actually sending them)<br />
  76. 76. Mocks<br />Pre-programmed with expectations<br />Specify how they should be called by the SUT<br />Throws an exception if they receive a call they were not expecting<br />Checked during the Assert to ensure they got all the calls they were expecting<br />
  77. 77. Refactoring<br />You could do that better…<br /><br />
  78. 78. Why Refactoring?<br />Improve human readability of code<br />Reduce the complexity in order to improve maintainability<br />Create a more expressive internal structure<br />It’s about DESIGN!!<br />
  79. 79. What is Refactoring?<br />Adisciplinedtechnique for restructuring an existing body of code<br />Altering code’s internal structure without changing its external behavior<br />A series of small steps that do very little on their own<br />Risk of change is mitigated by running automated tests after each step<br />
  80. 80. Refactoring is NOT<br />An undisciplined technique<br />Making changes to code that change its external behavior<br />One giant step<br />Done without a safety net of automated tests<br />
  81. 81. You Are Not Refactoring<br />If you are doing any of those things, your are not refactoring.<br />You’re just changing things willy-nilly.<br /><br />
  82. 82. How NOT to Test<br />Yeah, don’t do that<br /><br />
  83. 83. Test Smells<br />Things that make you go “that just ain’t right”<br />Code Smells<br />Test Smells<br />Project Smells<br />Most of what follows is summarized from<br /> <br />the book XUnit Test Patterns: Refactoring Test Code<br />
  84. 84. Test Smells – Code Smells<br />Smells that are found while looking at test code<br />Obscure Test<br />Conditional Test Logic<br />Hard-to-Test Code<br />Test Code Duplication<br />Test Logic in Production<br />
  85. 85. Obscure Test<br />There is trouble understanding what behavior a test is verifying<br />too much or too little information in the test method<br />Possible Solutions<br />Keep tests small and focused<br />Do not depend on external resources (files, databases, web services, etc.) <br />
  86. 86. Conditional Test Logic<br />Hard to know exactly what a test is going to do when it really matters<br />conditional logic to handle when SUT fails to return valid data<br />Loops to verify contents of collections<br />Conditionals to verify complex objects<br />Possible Solutions<br />Custom Assertion Methods<br />
  87. 87. Hard to Test Code<br />Code is difficult to test<br />Highly Coupled Code<br />Asynchronous Code<br />Untestable Test Code<br />Possible Solutions<br />Reduce coupling by using TDD and adhering to SOLID principles<br />Use Test Doubles to isolate the SUT<br />
  88. 88. Test Code Duplication<br />The same code is repeated many times<br />Cut-and-Paste Code Reuse<br />Reinventing the Wheel<br />Possible Solutions<br />Refactor your test code to create Utility Methods<br />
  89. 89. Test Logic in Production<br />Production code contains logic that should only be exercised during tests<br />Conditional logic “For Tests Only”<br />Test Dependency in Production<br />Test-Specific Equality<br />Possible Solutions<br />Test-specific sub-classes of SUT<br />Manage Test Dependencies<br />Custom Assertions<br />
  90. 90. Test Smells – Behavior Smells<br />Smells that jump out at you while running tests<br />Assertion Roulette<br />Erratic Test<br />Fragile Test<br />Frequent Debugging<br />Manual Intervention<br />Slow Tests<br />
  91. 91. Assertion Roulette<br />Hard to tell which of several assertions in the same test caused the test to fail<br />Test tries to verify too much<br />Assertions with unclear failure messages<br />Possible Solutions<br />One logical assertion per test<br />Include a failure message for the assertion<br />Use a GUI test runner<br />
  92. 92. Erratic Test<br />One or more test pass or fail intermittently<br />Tests depend on each other<br />Resource Leakage / Scarcity<br />Shared state<br />Using different/random values for each test run<br />Possible Solutions<br />Use a fresh fixture for each test<br />Clean up after yourself<br />Use Database sandboxes or a Fake Database<br />
  93. 93. Fragile Test<br />A test fails to compile or run when the SUT is changed in ways that do not affect the part the test is exercising<br />Data Sensitivity<br />Over-specifying behavior<br />Context Sensitivity<br />Possible Solutions<br />Do not depend on a database<br />Encapsulate setup behind Creation Methods<br />Abstract away dependencies (including Time)<br />
  94. 94. Frequent Debugging<br />Manual debugging is required to determine the cause of most test failures<br />Caused by<br />Lack of Defect Localization (missing tests)<br />Infrequent Test Runs<br />Possible Solutions<br />Only create behavior after a failing test case<br />Runs tests as part of Continuous Integration builds<br />
  95. 95. Manual Intervention<br />A test requires a person to do some manual action each time it is run, or verify results manually<br />Lack of attention to automated test fixture setup<br />Test is not self-checking<br />Test conditions that are hard to generate programmatically<br />Possible Solutions<br />Invest in automating test setup<br />Use Assertion Methods<br />Use Test stubs to simulate test conditions<br />
  96. 96. Slow Tests<br />Tests take a long time to run<br />Interaction with external resources (Databases, File Systems, Web Services)<br />Heavy Test fixture rebuilds for every test<br />Asynchronous Test / Explicit delays in the test<br />Too many tests<br />Possible Solutions<br />Abstract away database, file system, web services<br />Use an immutable shared fixture<br />Separate event from the behavior it executes<br />Run a fast subset more, and the entire suite less<br />
  97. 97. Test Smells – Project Smells<br />Smells that a project manager can watch out for<br />Buggy Tests<br />Developers Not Writing Tests<br />High Test Maintenance Cost<br />Production Bugs<br />
  98. 98. Buggy Tests<br />Bugs are regularly found in automated tests<br />Fragile Tests<br />Obscure Tests<br />Hard to Test Code<br />Possible Solutions<br />Learn to write tests properly<br />Refactor legacy code to make testing easier & more robust<br />Use TDD<br />
  99. 99. Developers Not Writing Tests<br />The Usual Excuses<br />Not Enough Time<br />Hard to Test Code<br />Wrong Test Automation Strategy<br />Possible Solutions<br />Learn & Practice <br />writing tests takes less time as you become more familiar with doing it<br />
  100. 100. High Test Maintenance Cost<br />Too much time is spent maintaining tests<br />Fragile Tests<br />Obscure Tests<br />Hard to Test code<br />Possible Solutions<br />Learning good Test Automation<br />Practicing TDD<br />
  101. 101. Production Bugs<br />Too many bugs found during formal test or in production<br />Infrequently Run Tests<br />Untested Code<br />Missing Unit Tests<br />Lost Tests<br />Possible Solutions<br />Speed up test suite<br />Use TDD<br />Write tests to cover untested code<br />Ensure all tests are part of some suite that is run regularly<br />
  102. 102. .NET Testing Tools<br /><br />
  103. 103. Testing Frameworks<br />NUnit, MBUnit,<br />Testing frameworks in the xUnit tradition<br />Assertion Engine with syntactical sugar on top<br />
  104. 104. Support Libraries<br />Object Factories<br />NBuilder<br />Test Doubles / Isolation Frameworks<br />Hand-rolled mock objects<br />Rhino Mocks<br />Moq<br />NMock<br />EasyMock.NET<br />Typemock Isolator<br />Telerik Just Mock<br />
  105. 105. Test Runners<br />NUnitcommand line<br />NUnit GUI<br />TestDriven.NET & Visual Nunit<br />Addins for Visual Studio<br />JetBrainsReSharper or DevExpressCodeRush<br />Integrates with Visual Studio<br />test status next to the test method in the editor<br />JetBrainsTeamCity<br />Continuous Integration server<br />
  106. 106. Test Guidance<br />Pair Programming (Dev and QA)<br />Code Reviews<br />Automated Tools<br />Typemock Test Lint<br />
  107. 107. Resources<br /><br />
  108. 108. Agile Principles, Patterns, and Practices in C# <br />Robert C. Martin<br />Micah Martin<br />
  109. 109. Working Effectively with Legacy Code<br />Michael Feathers<br />
  110. 110. Refactoring: Improving the Design of Existing Code<br />Martin Fowler<br />Kent Beck<br />John Brant<br />William Opdyke<br />Don Roberts<br />
  111. 111. The Art of Unit Testing with Examples in C#<br />Roy Osherove<br />
  112. 112. Growing Object-Oriented Software, Guided by Tests<br />Steve Freeman<br />Nat Pryce<br />
  113. 113. Testing ASP.NET Web Applications<br />Jeff McWherter<br />Ben Hall<br />
  114. 114. MSDN Magazine Articles<br />June 2008 – Patterns in Practice: The Open Closed Principle<br />October 2008 – Patterns in Practice: Cohesion and Coupling<br />December 2008 – Patterns in Practice: Design for Testability<br />September 2007 – Unit Testing: Exploring the Continuum of Test Doubles<br />
  115. 115. Videos<br />TDD with Kent Beck<br />Test First and Test Driven conversations with Corey Haines and JB Rainsberger<br />Moving Specificity Towards the Tests with Corey Haines and JB Rainsberger<br />Codemanship’s Videos with Jason Gorman<br />TekPub Concepts series<br />
  116. 116. Podcasts<br />Hanselminutes Podcast 145 - SOLID Principles with Uncle Bob - Robert C. Martin<br />Hanselminutes Podcast 150 – Uncle Bob Martin: SOLID: This Time With Feeling<br />Hanselminutes Podcast 146 – Test Driven Development is Design – The Last Word on TDD with Scott Bellware<br />Hanselminutes Podcast 31 – Test Driven<br />Coding QA Podcast 41 – iPhone TDD<br />Software Engineering Radio Podcast 155 - Johannes Link & LasseKoskelaon TDD<br />
  117. 117. Web Sites<br />The Principles of Object Oriented Design<br />xUnitTest Patterns<br />Object Mentor Blog<br />Martin Fowler’s Blog<br />Roy Osherove’s Blog<br />