Quality Software With Unit Test


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Quality Software With Unit Test

  1. 1. Quality Software with Unit Test Alice YANG
  2. 2. Some definitions <ul><li>Unit testing – testing a single class to ensure that it performs according to its API specs (i.e., its Javadoc) </li></ul><ul><li>Integration testing – testing that the units interact appropriately </li></ul><ul><li>Functional testing – testing that the integrated units meet the system requirements </li></ul><ul><li>Regression testing – testing that changes to code have not (re-)introduced unexpected changes in performance, inputs, or outputs </li></ul>
  3. 3. <ul><li>The Brass Ring: We generally want our code to be as free of bugs as is economically feasible </li></ul><ul><li>Testing is the only way to know how bug-free your code is </li></ul><ul><li>All four kinds of testing mentioned can be automated with repeatable suites of tests </li></ul>
  4. 4. <ul><li>The better your test suite, the more confidence you can have in your code’s correctness </li></ul><ul><li>Junit is the most commonly used way to automate unit tests for Java code </li></ul><ul><ul><li>Suites of repeatable tests are commonly built up over time </li></ul></ul><ul><ul><li>QA involves running these suites of tests on a regular basis </li></ul></ul>
  5. 5. <ul><li>Junit can also be used to do integration, functional, and regression testing </li></ul><ul><ul><li>Integration tests theoretically should create two objects, and test their interactions </li></ul></ul><ul><ul><li>Functional tests can simulate the user interacting with the system and verifying its outcomes </li></ul></ul><ul><ul><li>Regression testing is typically making sure that changes do not introduce test failures in the growing suite of automated tests </li></ul></ul>
  6. 6. Mock Objects <ul><li>Are you isolating your objects under test? </li></ul><ul><ul><li>If a test uses two objects and the objects interact, a test failure can be attributed to either of the two objects, or because they were not meant to interact </li></ul></ul><ul><ul><li>Mock objects are a common solution </li></ul></ul><ul><ul><ul><li>One and only one real code object is tested – the other objects are “mock objects” which simulate the real objects for test purposes </li></ul></ul></ul><ul><ul><ul><li>Allows test writer to simulate conditions that might be otherwise difficult to create </li></ul></ul></ul><ul><ul><li>This problem is well-known and amply addressed by several products (e.g., EasyMock) </li></ul></ul>
  7. 7. Code Coverage <ul><li>Do you have enough tests? What’s tested and what isn’t? </li></ul><ul><ul><li>Well-known problem with numerous tools to help, such as Emma, Jcoverage, Cobertura, and Clover. These tools monitor which pieces of code under test get executed during the test suite. </li></ul></ul><ul><ul><li>All the code that executed during the test is considered covered, and the other code is considered uncovered. </li></ul></ul><ul><ul><li>This provides a numeric measurement of test coverage (e.g., “Package x has 49% class coverage”) </li></ul></ul>
  8. 8. JUnit Fallacy # 1 <ul><li>“ The code is just fine – all our tests pass” </li></ul><ul><li>Test success does not mean the code is fine </li></ul><ul><li>Consider the following test: </li></ul><ul><li>public void testMethod() { </li></ul><ul><li>;// Do absolutely nothing </li></ul><ul><li>} </li></ul><ul><li>This test will pass every time. </li></ul>
  9. 9. What the world really needs <ul><li>Some way of measuring how rigorous each test is </li></ul><ul><ul><li>A test that makes more assertions about the behaviour of the class under test is presumably more rigorous than one that makes fewer assertions </li></ul></ul><ul><ul><li>If only we had some sort of measure of how many assertions are made per something-or-other </li></ul></ul>
  10. 10. “ Assertion Density” <ul><li>Assertion Density for a test is defined by the equation shown, where </li></ul><ul><ul><li>A is the assertion density </li></ul></ul><ul><ul><li>a is the number of assertions made during the execution of the test </li></ul></ul><ul><ul><li>m is the number of method calls made during the execution of the test </li></ul></ul><ul><li>Yep, I just made this up </li></ul>
  11. 11. Junit Fallacy #2 <ul><li>“ Our code is thoroughly tested – Cobertura says we have 95% code coverage” </li></ul><ul><li>Covered is not the same as tested </li></ul><ul><li>Many modules call other modules which call other modules. </li></ul>
  12. 12. Indirect Testing <ul><li>Class A, Class B, and Class C all execute as Test A runs </li></ul><ul><li>Code coverage tools will register Class A, Class B, and class C as all covered, even though there was no test specifically written for Class B or Class C </li></ul>Test A Class A Class B Class C Calls &quot;Covered&quot; &quot;Covered&quot; Calls Tests &quot;Covered&quot;
  13. 13. What the world really needs <ul><li>Some way of measuring how directly a class is tested </li></ul><ul><ul><li>A class that is tested directly and explicitly by a test designed for that class is better-tested than one that only gets run when some other class is tested </li></ul></ul><ul><ul><li>If only we had some sort of “test directness” measure… </li></ul></ul><ul><ul><li>Perhaps a reduced quality rating the more indirectly a class is tested? </li></ul></ul>
  14. 14. Testedness <ul><li>Testedness is defined by the formula shown, where </li></ul><ul><ul><li>t is the testedness </li></ul></ul><ul><ul><li>d is the test distance </li></ul></ul><ul><ul><li>n d is the number of calls at test distance d </li></ul></ul><ul><li>Yep, I made this one up too </li></ul>