• Like
  • Save
COM3082C2 - Systems Development slide 6- 1
Upcoming SlideShare
Loading in...5
×
 

COM3082C2 - Systems Development slide 6- 1

on

  • 1,651 views

 

Statistics

Views

Total Views
1,651
Views on SlideShare
1,651
Embed Views
0

Actions

Likes
3
Downloads
69
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    COM3082C2 - Systems Development slide 6- 1 COM3082C2 - Systems Development slide 6- 1 Presentation Transcript

    • Software Testing Dr Z He University of Ulster
    • Lecture 6: Software Testing
      • Objectives
        • Issues of software testing
          • Software testing, Software testing principles, Need for testing
        • Verification and validation
        • Testing stages
          • unit, integration, system, acceptance
        • Test Planning
          • Test plan
        • Techniques for testing
          • static: reading, walkthrough and inspections, correctness proofs, stepwise abstraction
          • dynamic: functional, structural
        • Testing Coverage
        • Further Testing Stages
    • Software Testing
        • once source code has been generated, software must be tested to uncover(and correct) as many errors as possible before delivery to the customer
        • the goal is to design a series of test cases that have a high likelyhood of finding errors
        • software testing techniques provide systematic guidance for designing tests that
          • exercise the internal logic of software components, and
          • exercise the input and output domains of the program to uncover errors in program function, behaviour and performance
        • is a process of executing a program with intent of finding an error
        • a good test case is one that has a high probability of finding an uncovered error
        • a successful test is one that uncovers an undiscovered error
        • cannot show the absence of errors and defects, it can only show that software error and defect are present
    • Software Testing Principles
      • All tests should be traceable to customer requirements
      • Tests should be planned before testing begins. Test planning can begin as soon as the requirements model is complete. Detailed definition of test cases can begin as soon as the design model has been solidified
      • Pareto principle applies to software testing (80% of all errors uncovered during testing will likely to be traceable to 20% of all program components)
      • Testing should begin in small (components) and progress toward testing in large(ultimately the entire system)
      • Exhaustive testing is not possible
      • To be most effective, testing should be conducted by an independent third party
    • Need for Testing
      • 30-85 errors per 1000 lines of code (Boehm 1981)
      • After testing, 0.5-3 errors per 1000 lines of code (Myers 1986)
      • Errors are expensive
      • The later errors are found the more expensive they are to fix
      Cost to fix Requirements Operation
        • Phase
    • Need for Testing
      • Testing shows presence of errors but cannot prove their absence
      • In most cases it is impossible to test exhaustively
      • E.g.
        • for i from 1 to 100 do
        • print (if a[i] = true then 1 else 0 endif);
        • has 2 100 different outcomes - approx. 3x10 14 years to exhaustively test. Given a machine can execute 10 million print instructions per second.
    • Verification and Validation
      • Verification - ‘Are we building the product right’?
      • Validation - ‘Are we building the right product’?
      • 1. At the requirements analysis stage
      • cheapest to correct at this stage
      • Test criteria:
        • completeness - are all the requirements gathered
        • consistency - no contradictions within the system or with external components
        • feasibility - benefits should outweigh costs
        • testability - requirements should be unambiguous, otherwise they cannot be tested
    • Verification and Validation
      • 2. At the Design stage
      • criteria as for requirements stage.
      • Elements from the requirements should be traced to the design
      • simulation, walkthroughs, inspections to test quality
    • Verification and Validation
      • 3. Implementation stage
      • Programs can be checked by reading through them (not by the author)
      • Stepwise abstraction - determine the function of code from a number of its steps
      • Tools are available to support the testing of code
        • Static analysis - test the code without running it - E.g. have all the variables been declared?
        • Dynamic analysis - test by executing the code
      • Unit testing
        • testing of individual components
      • Integration testing
        • testing collections of modules integrated into sub-systems
      • System testing
        • testing the complete system prior to delivery
      • Acceptance testing
        • testing by users to check that the system satisfies requirements. Sometimes called alpha(  ) testing
      Testing Stages
    • Testing Stages S u b - s y s t e m t e s t i n g U n i t t e s t i n g S y s t e m t e s t i n g A c c e p t a n c e t e s t i n g
      • Test Plan
        • The testing process
        • Requirements traceability
        • Tested items
        • Testing schedule
        • Test recording procedures
        • Hardware and software requirements
        • Constraints
      Test Planning
    • Techniques for Testing - Static
      • 1. Reading, Walkthroughs and inspection s
      • All involve someone else looking at the code
      • Look for
        • inappropriate usage of data. E.g. uninitialised variables, arrays out of bounds, dangling pointers
        • errors in declarations. E.g. undeclared variables, repeated names
        • Faults in computations. E.g. division by zero, overflow, type mismatches, erroneous operator order (*,/,+,- etc.)
        • faults in logical operators. E.g. < instead of >
        • faults in control. E.g. infinite loops, loop executing the wrong number of times
        • faults in interfaces. E.g. Wrong number of parameters
      • Many of these are detected by the compiler
      • Can be applied to documents
      • Can be applied to all stages of life cycle
    • Techniques for Testing - Static
      • 2. Correctness Proofs
      • Does a program meet its specification.
      • If specification is expressed formally and proved the program can be proved
      • Difficult to use.
      • Cannot prove every aspect.
      • Validation can only be done by testing.
      • 3. Stepwise Abstraction
      • bottom-up process
      • start with code and derive functions
      • check these with the requirements
    • Techniques for Testing - Dynamic
      • Dynamic Testing Techniques
        • Black Box Testing
          • also called functional testing , or behavioural testing
          • focuses on the functional requirements of the software
          • enables the software engineer to derive sets of input conditions that will fully exercise all functional requirements for a program (test cases derived from specifications)
          • tends to be applied during later stages of testing
          • purposely disregards control structure, attention is focused on the information domain
          • exhaustive functional testing is normally not feasible
          • use characteristics of input domain to find the minimum set of test cases to reveal most defects
          • we will study:
            • Equivalence Partitioning
            • Boundary value analysis
    • Techniques for Testing - Dynamic
      • Dynamic Testing Techniques
        • White Box Testing
          • also called glass box testing , or structural testing
          • uses the control structure of the procedural design to derive test cases
          • exhaustive structural testing is normally not feasible
          • to find the minimum set of test cases to exercise most important logical paths
          • We will study:
            • Basis Path Testing
            • Loop Testing
            • Testing Coverage
    • Techniques for Testing - Dynamic
      • Equivalence Partitioning
        • a black box /functional testing technique
        • in determining a set of test cases, this technique is to partition the program domain into a (small) number of equivalent classes
        • the assumption is that if a representative element from a class is tested and passed, all other elements in the class need not to be tested assuming they would pass the test
        • How to define equivalent classes:
          • if a input condition specifies a range , one valid and two invalid equivalent classes are defined
          • if an input condition requires a specific value , one valid and two invalid equivalent classes are defined
          • if an input condition specifies a member of a set , one valid and one invalid equivalent classes are defined
          • if an input condition is Boolean , one valid and one invalid class are defined
    • Techniques for Testing - Dynamic
      • Equivalence Partitioning
        • Example 1: Online banking
          • The user can access the bank using his personal computer, providing a 6-digit password, and follow with a series of typed commands that trigger various banking functions. The software supplied for baking application accepts data in the form:
            • area code: blank or three digit number not beginning with 0 or 1
            • password: six digit alphanumeric string
            • commands: “check”, “deposit”, or “bill pay”
          • Equivalent classes can be identified for each field
            • area code(boolean, range, specific value):
              • ec1: blank; ec2: 3-digit numbers (<200); ec3: 3-digit numbers (200-999); ec4: less than 3-digit numbers; ec5: more than 3-digit numbers (at least one value is needed for test case design)
    • Techniques for Testing - Dynamic
      • Equivalence Partitioning
            • password (boolean, specific value, set):
              • ec6: blank; ec7: 6-digit alphanumeric strings(valid passwords); ec8: 6-digit alphanumeric strings(invalid passwords); ec9: less than 6-digit alphanumeric strings; ec10: more than 6-digit alphanumeric strings
            • commands(boolean, set):
              • ec10: blank; ec11: valid (“check”, “deposit”, “bill pay”); ec12: invalid
        • Example 2: an input condition “each book has a 6-digit number” (specific value) can be classified as either valid or invalid equivalent classes
          • one valid class: 6-digit numbers
          • two invalid classes: more or less than 6 digits
          • test cases can be constructed to cover the three equivalent classes
        • Weak point: partitioning may be inadequate so that the assumption may not hold
    • Techniques for Testing - Dynamic
      • Boundary Value Analysis (BVA)
        • complements the equivalent class partitioning technique
        • Assumes software usually fails at boundaries.
        • concentrate on the extreme (boundary) values from each class (rather than random values from each class)
        • BVA leads to a selection of test cases that exercise bounding values
        • output can also be partitioned into equivalent classes, and extreme values from each class are tested as well
        • Guidelines for BVA:
          • if an input condition specifies a range bounded by value a and b, test cases should be designed with values a and b, just above and just below a and b, respectively (a, b, --a, a++, --b, b++)
          • if an input condition specifies a number of values , test cases should be developed that exercise the minimum and maximum numbers. Values just above and below minimum and maximum are also tested
    • Techniques for Testing - Dynamic
      • Boundary Value Analysis (BVA)
          • Apply guidelines 1 and 2 to output conditions . For example, assume that a temperature vs. pressure table is required as output from an engineering analysis program. Test cases should be designed to create an output report that produces the maximum (and minimum) allowable number of table entries.
          • If internal program data structures have prescribed boundaries (e.g., an array has a defined limit of 100 entries), be certain to design a test case to exercise that data structure at its boundary.
    • Techniques for Testing - Dynamic
      • Basis Path Testing
        • a white-box testing technique, proposed by Tom McCabe, 1976
        • to derive a logical complexity measure of a procedural design, and use this measure as a guide for defining a basis set of execution paths
        • test cases derived to exercise every statement and branch in the program at least once during testing (statement/branch coverage)
        • if every condition in a compound condition is considered, condition coverage can be achieved
        • Steps:
          • Draw a (control) flow graph , using the flowchart or the code
          • Calculate the cyclomatic complexity , using the flow graph
          • Determine the basis set of linearly independent paths
          • Design test cases to exercise each path in the basis set
    • Basis Path Testing
      • Flow Graph
        • used to depict program control structure
        • can be drawn from a flowchart (a procedural design representation)
        • can be drawn from a piece of source code
        • Flow Graph Notation
          • a flow graph composed of edges and nodes
          • an edge starts from a node and ends to another node
        • Sequence if-then-else While Repeat-until Case
    • Basis Path Testing
      • Flow Graph
        • Draw a flow graph from source code
        • 1 procedure insert(a, b, n, x)
        • 2 begin bool found:=false;
        • 3 for I:=1 to n do
        • 4 if a[I]=x
        • 5 then found:=true; goto leave endif
        • 6 enddo;
        • 7 leave:
        • 8 if found
        • 9 then b[I]:=b[I]+1
        • 10 else n:=n+1; a[n]:=x; b[n]:=1 endif
        • 11 end insert
    • Basis Path Testing
      • Flow Graph
        • Draw a flow graph from a flowchart
    • Basis Path Testing
      • Cyclomatic Complexity
        • a software metric that provides a quantitative measure of the logical complexity of a program
        • Basis set: is a maximal linearly independent set of paths through a graph
        • An independent path: is any path through a program that introduces at least one new set of processing statements or a new condition (I.e. at least one new edge in a flow graph)
        • Cyclomatic complexity defines the number of independent path in the basis set of a program
        • gives an upper bound for the number of tests that must be conducted to achieve statement/branch/condition coverage
        • How to calculate cyclomatic complexity:
              • cc = e - n + 2p
        • e - number of edges; n - number of nodes; p - number of components;
        • if all nodes in a graph are connected, then p = 1, thus
              • cc = e - n + 2
    • Basis Path Testing: Example 1
      • 1. Draw a flow graph
        • 0 /*Finding the maximum of three integers*/
        • 1 #include <stdio.h>
        • 2 intmaximum(int, int, int);
        • 3 main(){
        • 4 int a,b,c;
        • 5 printf((“Enter three integers: “);
        • 6 scanf(“%d%d%d”, &a,&b,&c);
        • 7 printf(“Maximum is: %d ”, maximum(a,b,c));
        • 8 }
        • 9 int maximum(int x, int y, int z){
        • 10 int max=x;
        • 11 if(y>max)
        • 12 max=y;
        • 13 if(z>max)
        • 14 max=z;
        • 15 return max
        • 16 }
      ==>
    • Basis Path Testing: Example 1
      • 2. Calculate cyclomatic complexity
        • e=7, n=6, p=1; so that cc=7-6+2=3
      • 3. Identify a basis set of independent paths
        • p1: a-b-c-d-e-f (y>x, z>y)
        • p2: a-b-d-e-f (y<=x, z>x)
        • p3: a-b-c-d-f (y>x, z<=y)
      • 4. Design test cases
    • Basis Path Testing: Example 2
      • 1. Draw a flow graph
        • see slide 6-24: source code, flow graph
      • 2. Calculate cyclomatic complexity
        • e = 12; n = 10; p = 1
        • cc = 12 - 10 + 2 x 1 =4
      • 3. Determine a basis set of independent paths
        • expect to specify 4 independent paths
        • p1: 1-2-3-7-8-9-11
        • p2: 1-2-3- 4-5 -7-8-9-11
        • p3: 1-2-3-4-5-7-8- 10 -11
        • p4: 1-2- 3-4- 6 -3 -7-8-10-11 ( 1 or more times )
        • HOWEVER: by reading source code, we found
          • 3-7 => 10; 5 => 9
          • p1 and p3 must be modified
    • Basis Path Testing: Example 2
      • 3. Determine a basis set of independent paths
          • if p3 modified, it would be the same as p2. Thus p3 should be deleted.
          • But the new paths introduced by p3 (8-10-11) must be covered by other paths! We found p4 covers them.
          • Modify p1, delete p3, we can have three independent paths
            • p1: 1-2-3-7-8-10-11
            • p2: 1-2-3-4-5-7-8-9-11
            • p3: 1-2-3-4-6-3-7-8-10-11
          • if you study the program carefully, you will find the following is better
            • p1: 1-2-3-7-8-10-11 (insert x when a[] is empty)
            • p2: 1-2-3-4-5-7-8-9-11(insert x when a[1]=x)
            • p3: 1-2- 3-4-6-3 -4-5-7-8-9-11 (insert x when a[i]=x,i>1, n>=i)
            • p4: 1-2- 3-4-6-3 -7-8-10-11 (insert x when a[] is not empty and x is not in a[]; p4 does not introduce any new edge but it exercises a new combination of the program logic!)
    • Basis Path Testing: Example 2
      • 4. Design test cases
        • Path 1 test case: 1-2-3-7-8-10-11 (insert x when a[] is empty)
          • input data: n=0; x=8; a[1]=0; b[1]=0;
          • expected results: a[1]=8; b[1]=1; n=1;
        • Path 2 test case: 1-2-3-4-5-7-8-9-11(insert x when a[1]=x)
          • input data: n=3; x=9; a[1]=9; a[2]=2; a[3]=3;b[1]=2;b[2]=5;b[3]=8;
          • expected results: b[1]=3
        • Path 3 test case: 1-2- 3-4-6-3 -4-5-7-8-9-11 (insert x when a[i]=x,i>1, n>=i)
          • input data: n=3;x=3;a[1]=9;a[2]=2;a[3]=3;b[1]=3;b[2]=2;b[3]=8;
          • expected results: b[3]=9
        • Path 4 test case: 1-2- 3-4-6-3 -7-8-10-11 (insert x when a[] is not empty and x is not in a[])
          • input data: n=3;x=6;a[1]=9;a[2]=2;a[3]=3;b[1]=3;b[2]=2;b[3]=8;
          • expected results: a[4]=6; b[4]=1; n=4;
    • Techniques for Testing - Dynamic
      • Loop Testing
        • a white box/structural testing technique
        • focuses exclusively on the validity of loop constructs
        • four different classes of loops can be defined:
        • simple loops nested loops concatenated loops unconstructed loops
    • Techniques for Testing - Dynamic
      • Loop Testing
        • simple loops
          • if n is the maximum number of allowable passes through the loop
          • the following set of test cases (7 cases) can be applied
            • skip the loop entirely (0)
            • only one pass through the loop (1)
            • two passes through the loop (2)
            • m passes through the loop where m<n (m)
            • n-1, n, n+1 passes through the loop (n-1, n, n+1)
          • Class Exercise
            • use the example for the basis path testing to derive 7 test cases
            • for real testing, you need to write a program to test the function
    • Techniques for Testing - Dynamic
      • Loop Testing
        • nested loops
          • start at the innermost loop. Set all other loops to minimum values
          • conduct simple loop tests for the innermost loop while holding the outer loops at their minimum iteration parameter values.
          • Work outward, conducting tests for the next loop, but keeping all other outer loops at minimum values and other nested loops to “typical” values
          • continue until all loops have been tested
        • concatenated loops
          • if concatenated loops are independent, use the approach defined for simple loops
          • if not independent(the loop counter for loop 1 is used as the initial value for loop 2), then the approach applied to nested loops is recommended
    • Techniques for Testing - Dynamic
      • Testing Coverage
        • Statement coverage - Run a series of tests which ensure that every statement is tested at least once. A tool is used to keep track of how many times a statement has been executed. BUT - not every situation is dealt with!
        • Branch coverage - Run a series of tests that ensure that every possible branch is tested at least once. Again a tool is required.
        • Condition coverage - Run a series of tests that ensure that every possible outcome of each condition is tested at least once.
        • Path Coverage - Run a series of tests that cover all possible combination of paths. This may be impossible so paths have to be selected
        • All-definition-use-path coverage - Create a test case for all paths that lead from a definition e.g. x=1 to a variable x = x +1.
    • Further Test Stages
      • So far we have dealt with module testing or unit testing
      • Integration Testing
        • Modules need to be tested when combined
        • Start with low level modules and move up
          • difficult to get an impression of the overall system
        • Start at the top and work down to subsystems
          • may prove repetitive and laborious
        • A combination of both approaches can be used
      • System Test : Testing the whole system
      • Acceptance Test : Testing the usability
      • Installation Test: If the operating environment is different from the development environment