Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Demo NUnit, NUnitAddin, Resharper Templates, etc. Talk about VSTS
  • C16_Effective_Unit_Testing.ppt

    1. 1. Effective Unit Testing with NUnit Win-Dev 2004 Friday, October 29, 2004 Peter Provost Software Design Engineer patterns & practices Microsoft Corporation
    2. 2. Agenda <ul><li>What is Unit Testing? </li></ul><ul><li>Typical Unit Testing Problems </li></ul><ul><li>Best Practices for Effective Unit Testing </li></ul><ul><li>Tools Demo </li></ul><ul><li>Q & A </li></ul>
    3. 3. What is Unit Testing?
    4. 4. A Definition <ul><li>Unit Testing is code that… </li></ul><ul><ul><li>Is written by developers, for developers. </li></ul></ul><ul><ul><li>Exercises a small, specific area of functionality. </li></ul></ul><ul><ul><li>Helps “prove” that a piece of code does what the developer expects it to do. </li></ul></ul>
    5. 5. What Unit Testing Is Not <ul><li>Acceptance Testing (aka Functional Testing) </li></ul><ul><li>Performance Testing </li></ul><ul><li>Scalability Testing </li></ul>
    6. 6. Marick’s Four Quadrants of Testing Source: Brian Marick – http://testing.com/
    7. 7. Why Unit Test <ul><li>It will make your life easier </li></ul><ul><li>Better code </li></ul><ul><li>Better designs </li></ul><ul><li>Code is easier to maintain later </li></ul><ul><li>Confidence when you code </li></ul>
    8. 8. Common Excuses <ul><li>I’m not a tester! </li></ul><ul><li>It takes too much time. </li></ul><ul><li>It takes too long to run the tests. </li></ul><ul><li>I don’t know how to test it. </li></ul><ul><li>I don’t really know what it is supposed to do, so I can’t test it. </li></ul><ul><li>But it compiles! It doesn’t need tests. </li></ul>
    9. 9. Typical Unit Testing Problems
    10. 10. Testing Is Monotonous and Boring <ul><li>Often indicates bad tooling or bad techniques </li></ul><ul><li>Unit testing should… </li></ul><ul><ul><li>Be easy to do </li></ul></ul><ul><ul><li>Provide rapid feedback </li></ul></ul>
    11. 11. Poor Test Coverage <ul><li>Often happens when tests not written first </li></ul><ul><li>Hard to retrofit later </li></ul><ul><li>Test coverage is orthogonal to test quality </li></ul>
    12. 12. Purpose of Tests is Misunderstood <ul><li>Remember why we do unit testing </li></ul><ul><li>Easy to get distracted by irrelevant things </li></ul><ul><li>What is the scope of the test? </li></ul>
    13. 13. Informal Testing Process <ul><li>How does your team do unit testing? </li></ul><ul><ul><li>Test-first </li></ul></ul><ul><ul><li>Intuitive “poking and prodding” style testing </li></ul></ul><ul><ul><li>Smoke testing </li></ul></ul><ul><ul><li>Whatever the developer wants </li></ul></ul>
    14. 14. Inconsistent Testing <ul><li>Are tests required before check in? </li></ul><ul><li>How is it enforced? </li></ul>
    15. 15. Low Test Quality <ul><li>Single biggest problem in unit testing today </li></ul><ul><li>Testing requires a special mindset </li></ul><ul><li>Tools only solve half the problem </li></ul>
    16. 16. Code Not Designed for Test <ul><li>Often the cause of low test quality </li></ul><ul><li>Hard to design testable code </li></ul><ul><li>However, testable code often is better designed </li></ul>
    17. 17. Tests Not Maintained <ul><li>So you’ve shipped your code and someone finds a bug… </li></ul><ul><li>Do you fix the tests? Or just fix the bug? </li></ul><ul><li>What about the rest of the team? </li></ul>
    18. 18. Best Practices for Effective Unit Testing
    19. 19. Automate Your Tests <ul><li>If it is hard to run tests, you won’t do it </li></ul><ul><li>Manual (aka scripted) tests make regression testing very hard </li></ul>
    20. 20. Use Good Tools <ul><li>The tools should make it… </li></ul><ul><ul><li>Easy to write tests </li></ul></ul><ul><ul><li>Easy to organize your tests </li></ul></ul><ul><ul><li>Easy to run tests (all, some, one) </li></ul></ul><ul><li>They should provide… </li></ul><ul><ul><li>Quick feedback of pass/fail </li></ul></ul><ul><ul><li>Details on the fail conditions </li></ul></ul>
    21. 21. Use Good Tools <ul><li>Testing Focused Tools </li></ul><ul><li>NUnit </li></ul><ul><li>csUnit </li></ul><ul><li>MbUnit </li></ul><ul><li>Test Driven .NET (aka NUnitAddin) </li></ul><ul><li>NMock </li></ul><ul><li>DotNetMock </li></ul><ul><li>Supporting Tools </li></ul><ul><li>NAnt </li></ul><ul><li>CodeRush </li></ul><ul><li>Resharper </li></ul><ul><li>CodeSmith </li></ul><ul><li>And of course… </li></ul><ul><li>Visual Studio .NET </li></ul>
    22. 22. Tools Demo
    23. 23. Get a Mentor <ul><li>Testing is as much an art as a science </li></ul><ul><li>Learning to write good tests is hard </li></ul><ul><li>Some people are naturally good at it and some aren’t </li></ul>
    24. 24. Use a Test List <ul><li>Start your development activities by writing down a list of things you want to test </li></ul><ul><li>You will often think of a test while writing another one. When you do, add it to the list. </li></ul><ul><li>Review your list frequently </li></ul>
    25. 25. Write Tests First <ul><li>Test-driven development (TDD) is a proven way of improving quality* </li></ul><ul><li>TDD’s main objective is not testing software! (this is a side effect) </li></ul><ul><li>TDD’s main objective is to aid programmers and customers during the development process with unambiguous requirements. </li></ul><ul><li>Code is written with testing as a primary motivation. In short, the code is testable. </li></ul><ul><li>* - http://collaboration.csc.ncsu.edu/laurie/Papers/TDDpaperv8.pdf </li></ul>
    26. 26. What to Test <ul><li>The “did we get what we expected” test isn’t always good enough </li></ul><ul><li>Ask yourself the question… </li></ul><ul><ul><li>“If the code ran correctly, how would I know?” </li></ul></ul>
    27. 27. What to Test - Testing Heuristics <ul><li>Test at the boundaries </li></ul><ul><li>Test every error message </li></ul><ul><li>Test different configurations </li></ul><ul><li>Run tests that are annoying to setup </li></ul><ul><li>Avoid redundant tests </li></ul><ul><ul><li>Source: Lessons Learned in Software Testing, by Cem Kaner, James Bach, Brett Pettichord </li></ul></ul>
    28. 28. What to Test – BICEP <ul><ul><li>B Are the boundary conditions correct? </li></ul></ul><ul><ul><li>I Can you check inverse relationships? </li></ul></ul><ul><ul><li>C Can you cross check using other means? </li></ul></ul><ul><ul><li>E Can you force an error condition to happen? </li></ul></ul><ul><ul><li>P Are the performance characteristics acceptable? </li></ul></ul><ul><li>Source: Pragmatic Unit Testing in C# with NUnit by Andrew Hunt and Dave Thomas </li></ul>
    29. 29. Test Structure <ul><li>William Wake’s 3-A Pattern </li></ul><ul><ul><li>Arrange </li></ul></ul><ul><ul><li>Act </li></ul></ul><ul><ul><li>Assert </li></ul></ul>
    30. 30. Designing for Test <ul><li>One simple question to help you write good code… </li></ul><ul><li>“ How am I going to test this?” </li></ul><ul><li>If you don’t know the answer, then you probably need to reconsider your design. </li></ul>
    31. 31. Stubs and Mocks <ul><li>Stub Defined </li></ul><ul><li>“ A portion is a portion of code that is inserted at runtime in place of real code, in order to isolate calling code from the real implementation. The intent is to replace a complex behavior with a simpler one that allows independent testing of some portion of real code.” </li></ul><ul><li>Mock Objects Defined </li></ul><ul><li>“ A mock object is an object created to stand in for an object that your code will be collaborating with. Your code can call methods on the mock object, will deliver results as set up by your tests.” </li></ul><ul><li>Source: JUnit in Action, by Vincent Massol </li></ul>
    32. 32. When are stubs appropriate? <ul><li>Objects that are expensive to create to manipulate </li></ul><ul><li>At the edges of a system, or around complex clusters of objects </li></ul><ul><li>Course-grained testing, integration testing </li></ul>
    33. 33. Stub Downsides <ul><li>A Complex system might require a complex stub </li></ul><ul><li>Hard to write in a verifiable way </li></ul><ul><li>Difficult to maintain </li></ul>
    34. 34. When are mocks appropriate? <ul><li>Real object has non-deterministic behavior </li></ul><ul><li>Real object is difficult to set up </li></ul><ul><li>Real object has behavior that is difficult to cause </li></ul><ul><li>Real object is slow </li></ul><ul><li>Source:http://c2.com/cgi/wiki?MockObjects </li></ul>
    35. 35. Mock Downsides <ul><li>Code complexity of being able to switch between real and mock implementations </li></ul><ul><li>Maintaining the mock </li></ul>
    36. 36. Properties of Good Tests <ul><li>Automatic Tests need to be run checked automatically </li></ul><ul><li>Thorough Test everything that could possibly break </li></ul><ul><li>Repeatable Tests should produce the same results each time they are run </li></ul><ul><li>Independent A test should exercise only one thing at a time </li></ul><ul><li>Professional Tests are code too! Write them professionally. </li></ul><ul><li>Source: Pragmatic Unit Testing in C# with NUnit by Andrew Hunt and Dave Thomas </li></ul>
    37. 37. Test Organization Options <ul><li>Tests in the VS Project being tested </li></ul><ul><ul><li>Best for experiments, spikes, etc. </li></ul></ul><ul><li>Tests in a separate VS Project </li></ul><ul><ul><li>Most commonly used arrangement </li></ul></ul><ul><ul><li>Tests are not part of distribution </li></ul></ul><ul><li>Tests in a separate VS Solution </li></ul><ul><ul><li>The usual reference issues, but possible </li></ul></ul>
    38. 38. Recommended Test Organization <ul><li>Tests in a separate project/assembly </li></ul><ul><li>Naming conventions </li></ul><ul><ul><ul><li>If you are testing the class: </li></ul></ul></ul><ul><ul><ul><li>Foo.Bar.Baz </li></ul></ul></ul><ul><ul><ul><li>Then you should name your tests </li></ul></ul></ul><ul><ul><ul><li>Foo.Bar.Tests.BazTests </li></ul></ul></ul><ul><li>Note the namespace and test class name! </li></ul>
    39. 39. When to Run Tests <ul><li>When you write a new method… </li></ul><ul><ul><li>…compile and run local unit tests </li></ul></ul><ul><li>When you fix a bug… </li></ul><ul><ul><li>…run the test that illustrates that bug. </li></ul></ul><ul><li>Any successful compile… </li></ul><ul><ul><li>…run local unit tests. </li></ul></ul><ul><li>Before you check in… </li></ul><ul><ul><li>…run all tests. </li></ul></ul><ul><li>Continuously… </li></ul><ul><ul><li>...check out and build the project from scratch including all unit tests. </li></ul></ul>
    40. 40. Do not check in code that… <ul><li>Is incomplete (e.g. missing dependencies) </li></ul><ul><li>Doesn’t compile </li></ul><ul><li>Compiles but breaks other code </li></ul><ul><li>Doesn’t have unit tests </li></ul><ul><li>Has failing unit tests </li></ul><ul><li>Passes its tests but causes other tests to fail. </li></ul>
    41. 41. Additional Resources
    42. 42. Web Sites <ul><li>http://www.testdriven.com </li></ul><ul><li>http://www.xprogramming.com </li></ul><ul><li>http://workspaces.gotdotnet.com/tdd </li></ul>
    43. 43. Weblogs <ul><li>Peter Provost </li></ul><ul><ul><li>http://www.peterprovost.org </li></ul></ul><ul><li>Jim Newkirk </li></ul><ul><ul><li>http://weblogs.asp.net/jamesnewkirk </li></ul></ul><ul><li>Brian Button </li></ul><ul><ul><li>http://dotnetjunkies.com/WebLog/oneagilecoder/ </li></ul></ul><ul><li>Martin Fowler </li></ul><ul><ul><li>http://www.martinfowler.com/bliki/ </li></ul></ul><ul><li>Darrell Norton </li></ul><ul><ul><li>http://dotnetjunkies.com/WebLog/darrell.norton/ </li></ul></ul>
    44. 44. Mailing Lists <ul><li>Yahoo Group: testdrivendevelopment </li></ul><ul><li>Yahoo Group: agiledotnet </li></ul><ul><li>Yahoo Group: extremeprogramming </li></ul>
    45. 45. Books <ul><li>Test-Driven Development in Microsoft .NET – James Newkirk and Alexei Vorontsov </li></ul><ul><li>Pragmatic Unit Testing in C# with NUnit – Andrew Hunt and David Thomas </li></ul><ul><li>Test-Driven Development, by Example – Kent Beck </li></ul><ul><li>Refactoring to Patterns – Joshua Kerievsky </li></ul><ul><li>Lessons Learned in Software Testing – Cem Kaner, James Bach, and Brett Pettichord </li></ul>
    46. 46. Questions? Comments <ul><li>Peter Provost </li></ul><ul><li>Software Design Engineer </li></ul><ul><li>patterns & practices </li></ul><ul><li>Microsoft Corporation </li></ul><ul><li>E-mail: peterpr@microsoft.com </li></ul><ul><li>Weblog: http://www.peterprovost.org/ </li></ul><ul><li>NOTE : Updated slides available at http://www.peterprovost.org/Files/C16_Effective_Unit_Testing.ppt </li></ul>