CPP09 - Testing


Published on

This is an introductory lecture on C++, suitable for first year computing students or those doing a conversion masters degree at postgraduate level.

Published in: Software, Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

CPP09 - Testing

  1. 1. Testing Michael Heron
  2. 2. Introduction • Getting a program running is only the first step. • Getting it working is the next… and this is often more costly. • Having a structured approach to systems testing is one of the key requirements for building robust and correct software.
  3. 3. Testing • Testing helps to ensure that a program is correct. • But what do we mean by correct? • There are a range of criteria that we can apply to any given program to judge its correctness.
  4. 4. Testing • Correctness: • It must handle all the requested functionality for all valid data. • It must not generate unexpected behaviour. • It must respond appropriately to all anticipated data. • It must have sufficient error checking to ensure the program will not produce unexpected results in the case of being given unexpected input.
  5. 5. Testing • This is a very tall order for most software projects. • Sometimes it’s even a tall order for very simple projects. • It is impossible to exhaustively test a program of any significant complexity. • There are just far too many variables. • Not all of the variables are found in the program itself.
  6. 6. Testing • The goal of testing is to reveal inadequacies in a piece of software. • This is an important point. • Programmers are often not the best people to write the test cases for their own programs. • Many large companies have dedicated testing departments to ensure a minimalisation of bias.
  7. 7. Testing • As a developer, it is tempting to write biased test cases that code is very likely to pass. • A good test however is one that reveals a problem. • A structured approach to testing involves creating a number of strong test cases for a program and comparing the actual results against the expected results.
  8. 8. Test Sets • For each test case in a test set, documentation should include: • The exact input data and the conditions under which the data were inputted. • The purpose of the test. • The exact output expected from the program with the given test data. • Once the test is completed, the actual output is compared to the expected output.
  9. 9. Test Strategies • The kind of test cases used in a given test set depends on the kind of testing strategy adopted. • There are two main families of testing: • Black box testing • White box testing • We shall look at these in turn.
  10. 10. Black Box Testing • Black box (or functional) testing is centred around creating a set of test cases based on sample data that is representative of all data. • Black box testing requires no knowledge of the program architecture… the program is a ‘black box’ that we cannot see into. • Think of Searle’s Chinese room.
  11. 11. Black Box Testing • In Black Box testing, all we consider are the inputs and the outputs. • Because it is impossible to exhaustively test a program, we must choose data that is representative of other data. • We must also consider special cases where errors are particularly likely.
  12. 12. Black Box Testing • Consider a program designed to multiply two numbers together. • If it works for 1 * 1, and 2 * 2, you may think it is safe to think it will work for any positive integer. • But this is not always true. • Consider the following table
  13. 13. Black Box Testing Number 1 Number 2 Result 1 1 1 2 2 4 3 3 27
  14. 14. Black Box Testing • It is important that you choose a large enough sample from any representative set. • It is also important to check for specific kinds of input: • Boundaries • Maximum and minimums • Invalid input
  15. 15. Black Box Testing • Boundaries are those areas where errors are very likely to creep in because of the transition between sets. • Between positive and negative • Between valid and invalid • These areas are often fruitful tests because programmers often make errors in boundary checking:
  16. 16. Examples • If (a = a) { do_something(); } vs if (a == a) { do_something(); }
  17. 17. Examples • If (a > b) { do_something(); } vs if (a >= b) { do_something(); }
  18. 18. Examples • For (int I = 0; I < 100; i++) { do_something_with (i); } vs for (int I = 0; I <= 100; i++) { do_something_with(i); }
  19. 19. Black Box Testing • It is very easy for these kind of errors to slip into a program. • The ramifications can often be very subtle. • A good test case will take numerous samples from a representative set and also check at the boundaries of each set. • Remember – good test cases are those that reveal errors.
  20. 20. White Box Testing • A counterpoint to black box testing is White Box (or structural) testing. • In White Box testing we do not care about the matching of expected output to actual output. • We do care that every statement in a program gets executed at some point. • White Box testing requires significant knowledge of a program’s architecture.
  21. 21. White Box Testing • In White Box testing, test cases are developed to ensure every path of execution through a program is touched. • Where there are loops and conditionals, test cases should be devised to ensure that these are executed at some point with valid test data.
  22. 22. White Box Testing • Consider the following piece of code: public static void main (String args[]) { int a, b; a = Integer.parseInt (args[0]); b = Integer.parseInt (args[1]); if (a < b); { System.out.println (“A is less than B”); } if (a > b) { System.out.println (“A is greater than B”); } else { System.out.println (“A is equal to B”); } }
  23. 23. White Box Testing Value of A Value of B Expected Actual 1 2 A is less than B A is less than B A is equal to B 2 1 A is greater than B A is less than B A is greater than B
  24. 24. White Box Testing • White box testing allows us to test every conditional to ensure that it gets called with the right sets of data. • White box testing is not a replacement for black box testing. • The two approaches work best as complements to each other.
  25. 25. Complementary Approaches • Why use White Box Testing in addition to Black Box Testing? • Logic errors are usually made when coding for ‘special cases’. The correct execution of these logic paths should be tested. • Assumptions about execution paths can be incorrect, and white box testing shows this. • Errors are random, and just as likely to be on an obscure path as a mainstream path.
  26. 26. Complementary Approaches • Why use Black Box Testing in addition to White Box Testing? • We can check to see how tightly a method conforms to the software specification • We can ensure reliability of user input. • We cannot assume that just because an execution path is followed that the logic will be correct. Black Box Testing highlights this.
  27. 27. Unit Testing • Most programs are more complex than the ones shown in this lecture. • How is it possible to test these using proper test cases? • It is not possible to test most complex programs as a whole. • It’s just too complicated.
  28. 28. Unit Testing • Instead, we use a technique called unit testing. • We take each of the methods or classes in turn and test them with all valid sets of data in a harness. • A harness is just a small program that calls the method with various appropriate sets of data and outputs the returned values.
  29. 29. Unit Testing • This takes advantage of the modularity of modern software development. • Once a unit has been tested with white and black box testing and has deemed to be ‘correct’, then it is usually safe to assume the unit will work correctly when called by the program itself. • However, this is not always true!
  30. 30. Integration Testing • However, we can apply a more structured approach to this also with Integration Testing. • We start off with a unit that has no dependencies on other units and test that to ensure that it works. • Then we test any units that make use of that unit.
  31. 31. Integration Testing • Once we have tested these, then we use Integration Testing to test them together to ensure that the result is correct. • This is a time consuming, but very valuable process. • In this way, a whole program can be broken up into chunks and then reintegrated with a good degree of confidence.
  32. 32. Summary • Testing is vital in ensuring correctness in programming. • It is however very time consuming and often frustrating. • It is possible to develop a structured approach to testing that makes debugging an easier process.