SOFTWARE ENGINEERING UNIT 6 Ch14

3/18/20093/18/2009 S.Sreenivasa RaoS.Sreenivasa Rao 1
Chapter 14Chapter 14
Software Testing TechniquesSoftware Testing Techniques
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Testability:Testability: how easily a software can behow easily a software can be
testedtested
•• OperabilityOperability—— the better it works, the more efficiently it can be tested.the better it works, the more efficiently it can be tested. -- ItIt
operates cleanlyoperates cleanly
•• ObservabilityObservability——What you see is what you test. The results of each testWhat you see is what you test. The results of each test
case are readily observed (clear and visible variables and statecase are readily observed (clear and visible variables and states, incorrects, incorrect
output,output, ……))
•• ControllabilityControllability——the degree to which testing can be automated andthe degree to which testing can be automated and
optimizedoptimized
•• DecomposabilityDecomposability——testing can be targeted to independent modulestesting can be targeted to independent modules
•• SimplicitySimplicity——reduce complex architecture and logic to simplify testsreduce complex architecture and logic to simplify tests
•• StabilityStability——few changes are requested during testingfew changes are requested during testing
•• UnderstandabilityUnderstandability——of the designof the design
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

What is aWhat is a ““GoodGood”” Test?Test?
•• A good test has a high probability of finding anA good test has a high probability of finding an
errorerror
•• A good test is not redundant.A good test is not redundant.
•• A good test should beA good test should be ““best of breedbest of breed”” (best(best
between those with similar intent)between those with similar intent)
•• A good test should be neither too simple norA good test should be neither too simple nor
too complextoo complex
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Test Case DesignTest Case Design
"Bugs lurk in corners"Bugs lurk in corners
and congregate atand congregate at
boundaries ..."boundaries ..."
BorisBoris BeizerBeizer
OBJECTIVEOBJECTIVE
CRITERIACRITERIA
CONSTRAINTCONSTRAINT
to uncover errorsto uncover errors
in a complete mannerin a complete manner
with a minimum of effort and timewith a minimum of effort and time
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Exhaustive TestingExhaustive Testing
loop < 20 Xloop < 20 X
There are 10 possible paths! If we execute oneThere are 10 possible paths! If we execute one
test per millisecond, it would take 3,170 years totest per millisecond, it would take 3,170 years to
test this program!!test this program!!
1414
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Selective TestingSelective Testing
loop < 20 Xloop < 20 X
Selected pathSelected path
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Software TestingSoftware Testing
Methods
Strategies
white-box
methods
black-box
methods
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

WhiteWhite--Box TestingBox Testing
... our goal is to ensure that all... our goal is to ensure that all
statements and conditions havestatements and conditions have
been executed at least once ...been executed at least once ...
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Why Cover?Why Cover?
logic errors and incorrect assumptionslogic errors and incorrect assumptions
are inversely proportional to a path'sare inversely proportional to a path's
execution probabilityexecution probability
we oftenwe often believebelieve that a path is notthat a path is not
likely to be executed; in fact, reality islikely to be executed; in fact, reality is
often counter intuitiveoften counter intuitive
typographical errors are random; it'stypographical errors are random; it's
likely that untested paths will containlikely that untested paths will contain
somesome
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

3/18/20093/18/2009 S.Sreenivasa RaoS.Sreenivasa Rao
Basic Path TestingBasic Path Testing
•• Define a basic set of execution pathsDefine a basic set of execution paths
•• Guarantee that every statement isGuarantee that every statement is
executed at least onceexecuted at least once
•• Using a logical complexity measure of aUsing a logical complexity measure of a
procedural design and use it as a guide forprocedural design and use it as a guide for
defining a basic set of execution pathdefining a basic set of execution path
10
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Flow Graph NotationFlow Graph Notation
•• A representation for control flow ofA representation for control flow of
programprogram
•• Each structured constructEach structured construct --> a flow graph> a flow graph
symbolsymbol
11
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Independent program pathIndependent program path
•• Any path through the program that introduces atAny path through the program that introduces at
least one new set of processing statements or aleast one new set of processing statements or a
new condition.new condition.
•• An independent path must move along at leastAn independent path must move along at least
one edge that has not been traversed before theone edge that has not been traversed before the
path is defined.path is defined.
•• The basic set is not unique.The basic set is not unique.
•• How do we know how many paths to look for?How do we know how many paths to look for?
–– CyclomaticCyclomatic complexitycomplexity
12
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Basis Path Testing:Basis Path Testing: CyclomaticCyclomatic
ComplexityComplexity
First, we compute the cyclomatic
complexity:
number of simple decisions + 1
(simple decisions: predicate node)
or
number of enclosed areas + 1
or
E-N+2 (E number of edges, N number of nodes)
In this case, V(G) = 4
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

CyclomaticCyclomatic ComplexityComplexity
A number of industry studies have indicatedA number of industry studies have indicated
that the higher V(G), the higher the probabilitythat the higher V(G), the higher the probability
or errors.or errors.
V(G)V(G)
modulesmodules
modules in this range aremodules in this range are
more error pronemore error prone
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Basis Path TestingBasis Path Testing
Next, we derive theNext, we derive the
independent paths:independent paths:
Since V(G) = 4,Since V(G) = 4,
there are four pathsthere are four paths
Path 1: 1,2,3,6,7,8Path 1: 1,2,3,6,7,8
Path 2: 1,2,3,5,7,8Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8Path 3: 1,2,4,7,8
Path 4: 1,2,4,7,2,4,...7,8Path 4: 1,2,4,7,2,4,...7,8
Finally, we derive testFinally, we derive test
cases to exercise thesecases to exercise these
paths.paths.
11
22
33
44
55 66
77
88
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Basis Path Testing NotesBasis Path Testing Notes
you don't need a flow chart,you don't need a flow chart,
but the picture will help whenbut the picture will help when
you trace program pathsyou trace program paths
count each simple logical test,count each simple logical test,
compound tests count as 2 orcompound tests count as 2 or
moremore
basis path testing should bebasis path testing should be
applied to critical modulesapplied to critical modules
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Deriving test casesDeriving test cases
•• Using the design or code, draw a flowUsing the design or code, draw a flow
diagramdiagram
•• Determine theDetermine the cyclomaticcyclomatic complexity ofcomplexity of
the flow graphthe flow graph
•• Determine a basis set of independentDetermine a basis set of independent
pathspaths
•• Prepare test cases to force the executionPrepare test cases to force the execution
of each pathof each path 17
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Graph MatricesGraph Matrices
•• A tabular representation of a flow graphA tabular representation of a flow graph
•• A graph matrix is a square matrix whose sizeA graph matrix is a square matrix whose size
(i.e., number of rows and columns) is equal to(i.e., number of rows and columns) is equal to
the number of nodes on a flow graphthe number of nodes on a flow graph
–– Each row and column corresponds to an identifiedEach row and column corresponds to an identified
node, andnode, and
–– matrix entries correspond to connections (an edge)matrix entries correspond to connections (an edge)
between nodes.between nodes.
•• By adding aBy adding a link weightlink weight to each matrix entry,to each matrix entry,
the graph matrix can become a powerful toolthe graph matrix can become a powerful tool
for evaluating program control structure duringfor evaluating program control structure during
testingtesting
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

•• Link weights can beLink weights can be
–– The probability of execution of that edgeThe probability of execution of that edge
–– The processing time for execution that edgeThe processing time for execution that edge
–– The memory requiredThe memory required
–– The resource requiredThe resource required
•• Help to design test casesHelp to design test cases
19
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Other Control Structure TestingOther Control Structure Testing
•• Condition testingCondition testing —— a test case design methoda test case design method
that exercises the logical conditions containedthat exercises the logical conditions contained
in a program modulein a program module
•• Data flow testingData flow testing —— selects test paths of aselects test paths of a
program according to the locations ofprogram according to the locations of
definitions (D) and uses (U) of variables in thedefinitions (D) and uses (U) of variables in the
program (every DU chain be covered at leastprogram (every DU chain be covered at least
once)once)
•• Loop testingLoop testing –– focus on validity of loopfocus on validity of loop
constructsconstructs
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Loop TestingLoop Testing
four different classes of loopfour different classes of loop
NestedNested
LoopsLoops
ConcatenatedConcatenated
LoopsLoops UnstructuredUnstructured
LoopsLoops
SimpleSimple
looploop
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Loop Testing: SimpleLoop Testing: Simple
LoopsLoops
Minimum conditionsMinimum conditions——Simple LoopsSimple Loops
1. skip the loop entirely1. skip the loop entirely
2. only one pass through the loop2. only one pass through the loop
3. two passes through the loop3. two passes through the loop
4. m passes through the loop m < n4. m passes through the loop m < n
5. (n5. (n--1), n, and (n+1) passes through1), n, and (n+1) passes through
the loopthe loop
where n is the maximum numberwhere n is the maximum number
of allowable passesof allowable passes
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Loop Testing: Nested LoopsLoop Testing: Nested Loops
Start at the innermost loop. Set all outer loops to theirStart at the innermost loop. Set all outer loops to their
minimum iteration parameter values.minimum iteration parameter values.
Test the min+1, typical, maxTest the min+1, typical, max--1 and max for the1 and max for the
innermost loop, while holding the outer loops at theirinnermost loop, while holding the outer loops at their
minimum values.minimum values.
Move out one loop and set it up as in step 2, holding allMove out one loop and set it up as in step 2, holding all
other loops at typical values. Continue this step untilother loops at typical values. Continue this step until
the outermost loop has been tested.the outermost loop has been tested.
NestedNested LoopsLoops
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Loop Testing: ConcatenatedLoop Testing: Concatenated
LoopsLoops
If the loops are independent of one anotherIf the loops are independent of one another
then treat each as a simple loopthen treat each as a simple loop
else* treat as nested loopselse* treat as nested loops
endifendif**
for example, the final loop counter value of loop 1 isfor example, the final loop counter value of loop 1 is
used to initialize loop 2.used to initialize loop 2.
Concatenated LoopsConcatenated Loops
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

BlackBlack--Box TestingBox Testing
requirementsrequirements
eventseventsinputinput
outputoutput
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

•• Incorrect or missing functionsIncorrect or missing functions
•• Interface errorsInterface errors
•• Errors in Data structures or external dataErrors in Data structures or external data
base accessbase access
•• Behavior or performance errorsBehavior or performance errors
•• initialization and termination errorsinitialization and termination errors
26
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

BlackBlack--Box TestingBox Testing
•• How is functional validity tested?How is functional validity tested?
•• How is system behavior and performance tested?How is system behavior and performance tested?
•• What classes of input will make good test cases?What classes of input will make good test cases?
•• Is the system particularly sensitive to certain inputIs the system particularly sensitive to certain input
values?values?
•• How are the boundaries of a data class isolated?How are the boundaries of a data class isolated?
•• What data rates and data volume can the systemWhat data rates and data volume can the system
tolerate?tolerate?
•• What effect will specific combinations of data have onWhat effect will specific combinations of data have on
system operation?system operation?
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

GraphGraph--Based MethodsBased Methods
new
file
menu select generates
(generation time < 1.0 sec)
document
window
document
tex
t
is represented as
contains
Attributes:
background color: white
text color: default color
or preferences
(b)
object
#1
Directed link
(link weight)
object
#2
object
#
3
Undirected link
Parallel links
Node weight
(value
)
(a)
allows editing
of
To understand theTo understand the
objects that areobjects that are
modeled inmodeled in
software and thesoftware and the
relationships thatrelationships that
connect theseconnect these
objectsobjects
In this context, weIn this context, we
consider the termconsider the term
““objectsobjects”” in the broadestin the broadest
possible context. Itpossible context. It
encompasses dataencompasses data
objects, traditionalobjects, traditional
components (modules),components (modules),
and objectand object--orientedoriented
elements of computerelements of computer
software.software.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Graph based testingGraph based testing
•• Transaction flow modelingTransaction flow modeling
•• Finite state modelingFinite state modeling
•• Data flow modelingData flow modeling
•• Timing modeling (sequence of executionTiming modeling (sequence of execution
and its time)and its time)
29
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Equivalence PartitioningEquivalence Partitioning
•• Divides the input domain of a programDivides the input domain of a program
into classes of data from which test casesinto classes of data from which test cases
can be derived.can be derived.
•• Define test cases that uncovers classes ofDefine test cases that uncovers classes of
errorserrors
30
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

•• If an Input condition isIf an Input condition is
–– A rangeA range
•• One valid and two invalid equivalence classesOne valid and two invalid equivalence classes
–– A specific valueA specific value
•• One valid and two invalid equivalence classesOne valid and two invalid equivalence classes
–– A member of a setA member of a set
•• One valid and one invalid equivalence classesOne valid and one invalid equivalence classes
–– BooleanBoolean
•• One valid and one invalid equivalence classesOne valid and one invalid equivalence classes 31
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Boundary value checkingBoundary value checking
•• A greater number of errors occurs at theA greater number of errors occurs at the
boundaries of the input domainboundaries of the input domain
32
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Comparison Testing (NComparison Testing (N--versioning)versioning)
•• Used only in situations in which the reliability ofUsed only in situations in which the reliability of
software is absolutely critical (e.g., humansoftware is absolutely critical (e.g., human--
rated systems)rated systems)
–– Separate software engineering teams developSeparate software engineering teams develop
independent versions of an application using theindependent versions of an application using the
same specificationsame specification
–– Each version can be tested with the same test dataEach version can be tested with the same test data
to ensure that all provide identical outputto ensure that all provide identical output
–– Then all versions are executed in parallel with realThen all versions are executed in parallel with real--
time comparison of results to ensure consistencytime comparison of results to ensure consistency
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Orthogonal Array TestingOrthogonal Array Testing
•• Used when the number of input parameters isUsed when the number of input parameters is
small and the values that each of thesmall and the values that each of the
parameters may take are clearly boundedparameters may take are clearly bounded
•• Dispersed uniformly throughout the test domainDispersed uniformly throughout the test domain
(balancing property)(balancing property)
One input item at a time L9 orthogonal array
XY
Z
X
Y
Z
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

OOTOOT——Test Case DesignTest Case Design
BerardBerard [BER93] proposes the following approach:[BER93] proposes the following approach:
11.. Each test case should be uniquely identified and should be expliEach test case should be uniquely identified and should be explicitlycitly
associated with the class to be tested,associated with the class to be tested,
2.2. The purpose of the test should be stated,The purpose of the test should be stated,
3.3. A list of testing steps should be developed for each test and shA list of testing steps should be developed for each test and shouldould
contain [BER94]:contain [BER94]:
a.a. a list of specified states for the object that is to be testeda list of specified states for the object that is to be tested
b.b. a list of messages and operations that will be exercised asa list of messages and operations that will be exercised as
a consequence of the testa consequence of the test
c.c. a list of exceptions that may occur as the object is testeda list of exceptions that may occur as the object is tested
d.d. a list of external conditions (i.e., changes in the environmenta list of external conditions (i.e., changes in the environment
external to the software that must exist in order to properly coexternal to the software that must exist in order to properly conductnduct
the test)the test)
e.e. supplementary information that will aid in understanding orsupplementary information that will aid in understanding or
implementing the test.implementing the test.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

Testing MethodsTesting Methods
•• FaultFault--based testing (same for conventional)based testing (same for conventional)
–– The tester looks for plausible faultsThe tester looks for plausible faults
•• aspects of the implementation of the system that may resultaspects of the implementation of the system that may result
in defects.in defects.
–– To determine whether these faults exist, test cases are designedTo determine whether these faults exist, test cases are designed
to exercise the design or code.to exercise the design or code.
•• Class Testing and the Class HierarchyClass Testing and the Class Hierarchy
–– Inheritance does not obviate the need for thorough testing of alInheritance does not obviate the need for thorough testing of alll
derived classes.derived classes.
–– In fact, it can actually complicate the testing process.In fact, it can actually complicate the testing process.
•• ScenarioScenario--Based Test DesignBased Test Design
–– ScenarioScenario--based testing concentrates on what the user does, notbased testing concentrates on what the user does, not
what the product does.what the product does.
–– This means capturing the tasks (via useThis means capturing the tasks (via use--cases) that the user hascases) that the user has
to perform, then applying them and their variants as tests.to perform, then applying them and their variants as tests.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

OOT Methods: Random Testing class basedOOT Methods: Random Testing class based
•• Random testing:Random testing: check class instance life historiescheck class instance life histories
–– identify operations applicable to a classidentify operations applicable to a class
–– define constraints on their usedefine constraints on their use
–– identify a minimum test sequenceidentify a minimum test sequence
•• an operation sequence that defines the minimum life history of tan operation sequence that defines the minimum life history of the class (object)he class (object)
–– generate a variety of random (but valid) testgenerate a variety of random (but valid) test
sequencessequences
•• exercise other (more complex) class instance life historiesexercise other (more complex) class instance life histories
Like for an instance of Account class: open, setup, depositLike for an instance of Account class: open, setup, deposit,,
withdraw, close And open, setup, deposit, [deposit, withdraw,withdraw, close And open, setup, deposit, [deposit, withdraw,
balance,balance, creditlimitcreditlimit], withdraw, close], withdraw, close
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

OOT Methods: PartitionOOT Methods: Partition TestingclassTestingclass basedbased
•• Partition TestingPartition Testing
–– reduces the number of test cases required to test a class inreduces the number of test cases required to test a class in
much the same way as equivalence partitioning formuch the same way as equivalence partitioning for
conventional softwareconventional software
–– statestate--based partitioningbased partitioning
•• categorize and test operations based on their ability to changecategorize and test operations based on their ability to change the state of a classthe state of a class
–– attributeattribute--based partitioningbased partitioning
•• categorize and test operations based on the attributes that theycategorize and test operations based on the attributes that they useuse
•• For example: partition the operations according to their use/modFor example: partition the operations according to their use/modify/donify/don’’t use ort use or
modify an attributemodify an attribute
–– categorycategory--based partitioningbased partitioning
•• categorize and test operations based on the generic function eaccategorize and test operations based on the generic function each performsh performs
•• For example: operations can be categorized asFor example: operations can be categorized as
initialization/computation/queries/terminationinitialization/computation/queries/termination
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

OOT Methods: InterOOT Methods: Inter--Class TestingClass Testing
(During integration)(During integration)
•• RandomRandom
–– Like in class testingLike in class testing
•• PartitioningPartitioning
–– In (intraIn (intra--)class testing: state)class testing: state--based, attributebased, attribute--based,based,
categorycategory--basedbased
Here:Here:
–– Partition according to messages which serve differentPartition according to messages which serve different
classesclasses
–– StateState--basedbased
•• BehavioralBehavioral
–– StateState--basedbased
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

OOT Methods: InterOOT Methods: Inter--Class TestingClass Testing
(During integration)(During integration)
•• RandomRandom
–– For each client class, use the list of class operators toFor each client class, use the list of class operators to
generate a series of random test sequences. The operatorsgenerate a series of random test sequences. The operators
will send messages to other server classes.will send messages to other server classes.
–– For each message that is generated, determine theFor each message that is generated, determine the
collaborator class and the corresponding operator in thecollaborator class and the corresponding operator in the
server object.server object.
–– For each operator in the server object (that has beenFor each operator in the server object (that has been
invoked by messages sent from the client object), determineinvoked by messages sent from the client object), determine
the messages that it transmits.the messages that it transmits.
–– For each of the messages, determine the next level ofFor each of the messages, determine the next level of
operators that are invoked and incorporate these into theoperators that are invoked and incorporate these into the
test sequencetest sequence
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

OOT Methods: Behavior TestingOOT Methods: Behavior Testing
empty
acctopen setupAccnt
set up
acct
deposit
(initial)
working
acct
withdrawal
(final)
dead
acct close
nonworking
acct
deposit
withdraw
balance
credit
accntInfo
Figure 14.3 St at e diagram f or Account class (adapt ed f rom [ KIR94] )
The tests to beThe tests to be
designed shoulddesigned should
achieve all stateachieve all state
coverage [KIR94].coverage [KIR94].
That is, theThat is, the
operationoperation
sequences shouldsequences should
cause thecause the
Account class toAccount class to
make transitionmake transition
through allthrough all
allowable statesallowable states
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

SOFTWARE ENGINEERING UNIT 6 Ch14

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (14)

Similar to SOFTWARE ENGINEERING UNIT 6 Ch14

Similar to SOFTWARE ENGINEERING UNIT 6 Ch14 (20)

More from Azhar Shaik

More from Azhar Shaik (13)

Recently uploaded

Recently uploaded (20)

SOFTWARE ENGINEERING UNIT 6 Ch14