Software Testing Techniques Bill Bond 4/17/03

Overview This week - examine detailed testing techniques (technology) Next week - discuss strategies that can be used to apply techniques (framework)

This week - examine detailed testing techniques (technology)

Next week - discuss strategies that can be used to apply techniques (framework)

Objectives Testing - executing a program with the intent of finding an error Good test case - high probability of finding an as-yet undiscovered error Successful test - uncovers an as-yet undiscovered error A destructive process

Testing - executing a program with the intent of finding an error

Good test case - high probability of finding an as-yet undiscovered error

Successful test - uncovers an as-yet undiscovered error

A destructive process

Two ways White box - Knowing internal working, verify operation of all the pieces, all the paths, early stages of testing Black box - Knowing function desired, verify that function is fully operational, late stages of testing

White box - Knowing internal working, verify operation of all the pieces, all the paths, early stages of testing

Black box - Knowing function desired, verify that function is fully operational, late stages of testing

Limitations Testing can not prove the absence of defects only that software defects are present It is impossible to exercise “all” combinations Even small programs would take years

Testing can not prove the absence of defects only that software defects are present

It is impossible to exercise “all” combinations

Even small programs would take years

White Box Testing Derive test cases to Exercise all independent paths in module all least once Exercise all logical decisions on true and false sides Execute loops on boundaries and within operational bounds Exercise internal data structures

Derive test cases to

Exercise all independent paths in module all least once

Exercise all logical decisions on true and false sides

Execute loops on boundaries and within operational bounds

Exercise internal data structures

Black Box Testing Focus on functionality and interface Incorrect, missing functions Interface errors Errors in data structures / external DB access Performance errors Initialization / termination errors

Focus on functionality and interface

Incorrect, missing functions

Interface errors

Errors in data structures / external DB access

Performance errors

Initialization / termination errors

Basis Path Testing Determine complexity Test each execution path (execute every statement once) Flow graph Nodes - one or more procedural statements Arcs - (edges) - flow of control (flow chart arrows) Edges must terminate at node (even if null)

Determine complexity

Test each execution path (execute every statement once)

Flow graph

Nodes - one or more procedural statements

Arcs - (edges) - flow of control (flow chart arrows)

Edges must terminate at node (even if null)

Graphs Sequence If While Until Case

More Definitions Region - Area bounded by edges and nodes (area outside “all” regions is also counted as a region) Predicate node - node that contains a condition

Region - Area bounded by edges and nodes (area outside “all” regions is also counted as a region)

Predicate node - node that contains a condition

Special Case Separate node created for each condition in “ if a or b then x else y” a b x x y

Cyclomatic Complexity Provides a quantitative measure of logical complexity Defines number of independent paths in basis set ( upper bound on number of tests ) Independent path - introduces a new set of processing statements or new condition (moves along a new edge) Basis set - set of independent paths: all statements executed, all conditions tested

Provides a quantitative measure of logical complexity

Defines number of independent paths in basis set ( upper bound on number of tests )

Independent path - introduces a new set of processing statements or new condition (moves along a new edge)

Basis set - set of independent paths: all statements executed, all conditions tested

Cyclomatic Complexity V(G) = number of regions V(G) = edges - nodes + 2 V(G) = predicate nodes + 1

V(G) = number of regions

V(G) = edges - nodes + 2

V(G) = predicate nodes + 1

Deriving Test Cases Use design or code - draw flow graph Determine cyclometric complexity Determine basis set of paths Prepare test cases to force execution of each path

Use design or code - draw flow graph

Determine cyclometric complexity

Determine basis set of paths

Prepare test cases to force execution of each path

How Many Test Cases?

How Many Test Cases?

Equivalence Partitioning Divide input domain into classes of data Select test that represents class of data Uncover a class of errors Valid / Invalid states Range Specific value Set Boolean

Divide input domain into classes of data

Select test that represents class of data

Uncover a class of errors

Valid / Invalid states

Range

Specific value

Set

Boolean

Boundary Value Analysis Errors occur at class boundaries Concentrate on boundaries Range (a, b) - a, b, above, below Set - min, max, above, below Exercise data structures at boundary - example array of 100 elements

Errors occur at class boundaries

Concentrate on boundaries

Range (a, b) - a, b, above, below

Set - min, max, above, below

Exercise data structures at boundary - example array of 100 elements