Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Recap <ul><li>Design by Contract </li></ul>
  2. 2. Today <ul><li>Software Testing </li></ul>
  3. 3. Software Testing <ul><li>Testing is an important part of the software engineering process, because there are almost always differences between what is specified and what is initially implemented. </li></ul><ul><ul><li>Misunderstanding of specifications. </li></ul></ul><ul><ul><li>Incorrect choices of algorithms or data structures. </li></ul></ul><ul><ul><li>Pathological cases that aren’t considered. </li></ul></ul><ul><ul><li>Typographical errors that happen to compile. </li></ul></ul>
  4. 4. Types of Software Testing <ul><li>There are various types of software testing: </li></ul><ul><ul><li>Unit testing attempts to locate errors in isolated pieces of a software system. </li></ul></ul><ul><ul><li>Integration testing attempts to locate errors that result from interactions between pieces of a software system. </li></ul></ul><ul><ul><li>Interactive testing attempts to locate problems that arise when humans must interact with a software system - such as flaws in its human interface, or in the assumptions made about users’ knowledge or experience. </li></ul></ul><ul><li>We’ll mainly concern ourselves with the first two, because our user interfaces consist primarily of command line parameters. </li></ul>
  5. 5. Unit Testing <ul><li>A unit test is a test that attempts to validate one particular piece of a software system. </li></ul><ul><li>The size of the piece being tested can vary: </li></ul><ul><ul><li>Method </li></ul></ul><ul><ul><li>Class </li></ul></ul><ul><ul><li>Package </li></ul></ul><ul><li>A set of unit tests is called a test suite . </li></ul><ul><li>There are two main forms of unit test, whitebox and blackbox . </li></ul>
  6. 6. Unit Testing Whitebox <ul><li>Whitebox tests access the internal structure of the component being tested </li></ul><ul><ul><li>Assertion checking at runtime is a common form of whitebox testing. </li></ul></ul><ul><ul><li>A test method on a class that accesses the class’s private fields or methods is also a whitebox test. </li></ul></ul><ul><li>Whitebox tests are almost always incorporated into the component being tested. </li></ul><ul><ul><li>This is necessary in most object-oriented systems because of information hiding. </li></ul></ul>
  7. 7. Unit Testing Blackbox <ul><li>Blackbox tests use only the public interfaces of the components being tested. </li></ul><ul><ul><li>A separate class that exercises one or more other classes would be an example. </li></ul></ul><ul><li>Blackbox tests are typically outside the component (class, package) being tested. </li></ul><ul><ul><li>Ensures that they don’t “accidentally” use inside information that they shouldn’t use. </li></ul></ul><ul><ul><li>Allows the component to be built without the test being included, if necessary. </li></ul></ul>
  8. 8. Unit Testing <ul><li>The world of blackbox and whitebox testing isn’t entirely black and white - some situations require hybrid, or greybox , tests that observe the internal workings of a component. </li></ul><ul><li>Most systems require both blackbox and whitebox testing - neither one alone is enough. </li></ul><ul><li>Testing isn’t a substitute for good design and programming methods. </li></ul>
  9. 9. Unit Testing Responsibility <ul><li>The people who design and build software systems are not necessarily the people who write the test suites for those systems. </li></ul><ul><li>Whitebox tests are almost always written by the same people as the component being tested, because they’re internal to the component or rely on internal information. </li></ul><ul><li>Blackbox tests are sometimes written by the same people, but are often written by special “test designers” whose job it is to try to find things that the software designers and builders didn’t account for. </li></ul>
  10. 10. Unit Test Coverage <ul><li>What should be covered in unit tests? </li></ul><ul><li>Whitebox tests should primarily cover consistency conditions (preconditions, postconditions, invariants). </li></ul><ul><li>Blackbox tests should exercise every feature of every class in a system. </li></ul><ul><ul><li>This may involve invoking multiple features as part of a single test, to set up an environment or establish some particular condition. </li></ul></ul>
  11. 11. Unit Testing Example <ul><li>Assume we have a Point class, with the following methods: </li></ul><ul><ul><li>x , y , rho , theta - accessors for the coordinates of the point. </li></ul></ul><ul><ul><li>distance - takes a point as a parameter and calculates the distance between the two points. </li></ul></ul><ul><ul><li>translate , scale , rotate - change the position of the point. </li></ul></ul><ul><ul><li>Constructors for polar and Cartesian coordinates. </li></ul></ul><ul><ul><li>Other standard methods ( equals , hashCode, clone ). </li></ul></ul><ul><li>Assume further that a reasonable set of whitebox tests are included in the class as assertions. </li></ul><ul><li>What blackbox tests can we do to satisfy ourselves that all these methods work properly? </li></ul>
  12. 12. Unit Testing Example <ul><li>Constructors, equals , hashCode , x , y , rho , theta : </li></ul><ul><li>Create a few points and verify that they get created correctly, by examining x , y , rho , theta . </li></ul><ul><li>Ensure that points created with equivalent polar and Cartesian coordinates end up equal to each other, and have the same hashCode . </li></ul><ul><li>Degenerate conditions: create points with theta >= 2π , or theta < 0 , and ensure that they’re created correctly (this only applies if we don’t have a precondition restricting the values of theta ). </li></ul>
  13. 13. Unit Testing Example <ul><li>distance </li></ul><ul><li>Check that the distance between a point and itself is 0. </li></ul><ul><li>Check that the distance between a point and a clone of that point is 0. </li></ul><ul><li>Create two points a known distance away from each other and check to see that distance gives the known distance. </li></ul><ul><ul><li>Call distance on both objects and ensure the result is identical. </li></ul></ul><ul><ul><li>Do this with enough pairs of points to convince yourself that distances work between points in different quadrants, and when the origin is one of the points. </li></ul></ul>
  14. 14. Unit Testing Example <ul><li>translate </li></ul><ul><li>Boundary condition: attempt to translate a point by (0, 0). </li></ul><ul><li>Attempt to translate a point by (+, +), (+, -), (-, +), (-, -) amounts. </li></ul><ul><li>Attempt to translate a point between quadrants, through the origin, etc. </li></ul><ul><li>Perhaps attempt to induce a numeric overflow and see how it’s handled… </li></ul>
  15. 15. Unit Testing Example <ul><li>scale </li></ul><ul><li>Boundary conditions: scale by a factor of 0 (should make the point (0, 0), regardless of the original point) and a factor of 1 (should leave the point unchanged). </li></ul><ul><li>Scale by positive and negative factors. </li></ul><ul><li>Perhaps attempt to induce a numeric overflow and see how it’s handled… </li></ul>
  16. 16. Unit Testing Example <ul><li>rotate </li></ul><ul><li>Boundary condition: attempt to rotate the point (0, 0) by various amounts. </li></ul><ul><li>Boundary condition: attempt to rotate a point by an angle of 0. </li></ul><ul><li>Attempt to rotate a point by various amounts and check the results (probably cover all 4 quadrants). </li></ul><ul><li>Degenerate conditions: attempt to rotate points with angle >= 2π , or angle < 0 , and ensure that they’re rotated correctly (this only applies if we don’t have a precondition restricting the values of angle ). </li></ul>
  17. 17. Unit Testing <ul><li>That’s a whole lot of tests for one little class, and in many cases you wouldn’t need to be quite so exhaustive. </li></ul><ul><li>Every class for which it makes sense to test in isolation should have its own individual suite of unit tests (in addition to its assertions). </li></ul><ul><ul><li>Sometimes this requires a test harness , a special environment in which to conduct a test of a particular component. </li></ul></ul><ul><ul><li>For instance, if you’re testing a class that implements a dealer in a card game, you need to “fool” it into thinking that it has players to deal to… </li></ul></ul>
  18. 18. Unit Testing Implementation <ul><li>Tests can get pretty complex. </li></ul><ul><li>Our tests for Point were all very simple, most of them involving only one or two method calls, but sometimes it takes a lot more effort to set up a particular situation to be tested. </li></ul><ul><li>For example, to test the behavior of a Stack class when it reaches its maximum capacity, you’d have to put in a whole lot of distinct objects, then remove them again and ensure that the order you got them back in is exactly what it should be. </li></ul>
  19. 19. Unit Testing Automation <ul><li>Automation can help a lot, both with creating and with running unit tests. </li></ul><ul><li>One thing you absolutely never want to do is to sit at a keyboard and interact with a unit test. </li></ul><ul><ul><li>Waste of time - you should be working on your design and code, not interacting with some test program. </li></ul></ul><ul><ul><li>Error-prone - you’re not necessarily guaranteed to provide the same input over and over again, which makes tests less useful. </li></ul></ul><ul><li>Unit tests should be automated to the maximum extent possible. </li></ul>
  20. 20. Unit Testing Automation <ul><li>Jtest and JUnit are unit testing automation tools for Java, and we’ll be using them this term. </li></ul><ul><li>JUnit is free, and lets you create unit tests just by implementing some classes with specially formatted methods. </li></ul><ul><li>Jtest is commercial, and can automatically generate a unit test suite based on your Jcontract assertions. It has the added benefit of automatically generating JUnit skeletons for you, which you can fill in with your own more complex blackbox tests. </li></ul>
  21. 21. Integration Testing <ul><li>Integration testing can also be automated, in much the same ways as unit testing. </li></ul><ul><ul><li>For example, if you write a class that sends and receives a bunch of messages through your router, that is effectively an integration test. </li></ul></ul><ul><li>Integration testing should only be done once you’re convinced that every component works on its own. </li></ul><ul><ul><li>Integration can cause interesting things to happen, especially in concurrent systems, and if the individual components don’t work correctly in isolation it’s a lot harder to figure out why the interesting things happen… </li></ul></ul>
  22. 22. Interactive Testing <ul><li>Interactive testing should usually be the last phase of testing. </li></ul><ul><ul><li>Once the system is working according to its specification, then you can see how it works when subjected to the various abuses that users can put on it. </li></ul></ul><ul><li>Interactive testing can often reveal the need for more unit or integration tests, when a user does something unpredictable. </li></ul><ul><li>User interfaces are typically the components most affected by interactive testing. </li></ul>
  23. 23. Test Failures <ul><li>So we have all these different kinds of tests - what happens when one fails? </li></ul><ul><li>Blackbox tests and integration tests are only going to fail during explicit testing runs, so those failures are handled by tracking down the problem and re-running the tests. </li></ul><ul><li>Whitebox tests can fail at runtime even after the software has been delivered to clients (assertion checks). </li></ul>
  24. 24. Test Failures <ul><li>When a whitebox test fails, it’s sometimes appropriate to stop the entire system, but there are often better ways to handle it. </li></ul><ul><ul><li>Attempt to fix the problem automatically, through exception handling, retries, etc. </li></ul></ul><ul><ul><li>Keep the system running but limit its functionality (“the requested function is currently unavailable, please try again later”). </li></ul></ul>
  25. 25. Homework 3 Overview <ul><li>Homework 3 is the first design assignment. </li></ul><ul><li>Goal: To carry out a design using Design by Contract, and work with more threads. </li></ul><ul><li>Design and implement a thread-safe active router. </li></ul><ul><ul><li>Functionality just like the passive router, except that it has its own thread(s) that handle message delivery, and messages may be sent to multiple destinations rather than a single destination. </li></ul></ul><ul><ul><li>Must explicitly copy message objects, so they can’t be modified after being sent. This will likely affect your design. </li></ul></ul>
  26. 26. Homework 3 Overview <ul><li>What you have to turn in: </li></ul><ul><ul><li>A set of class skeletons, containing full Javadoc comments and full contracts for the entire public API of your router. </li></ul></ul><ul><ul><li>Textual documentation (text, HTML, it’s up to you as long as it’s not a Word file) describing aspects of your design that don’t necessarily belong in Javadoc comments. </li></ul></ul><ul><li>Homework 4 will be the implementation of this, along with a test suite. </li></ul><ul><li>It’s extremely unlikely that you will get a grade for Homework 3 before having to work on Homework 4, but we’ll take that into account when grading both of them. </li></ul>
  27. 27. Next Class <ul><li>Formal Reasoning about Concurrent Systems </li></ul>