Fundamental Test Design Techniques

TC
AM Tutorial
10/14/2014 8:30:00 AM
"Fundamental Test Design Techniques"
Presented by:
Lee Copeland
Software Quality Engineering
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Lee Copeland
Software Quality Engineering
With more than thirty years of experience as an information systems professional at commercial
and nonprofit organizations, Lee Copeland has held technical and managerial positions in
applications development, software testing, and software process improvement. At Software
Quality Engineering, Lee has developed and taught numerous training courses on software
development and testing issues, and is a sought-after speaker at software conferences in the
United States and abroad. Lee presents at software conferences in the United States and
abroad. He is the author of the popular reference book, A Practitioner’s Guide to Software Test
Design.

Mastering Test Design V 3.2 © SQE 2001-2006 1

This is the definition for “test” from IEEE Std 610, which is a dictionary of
technical terms related to software. It is both a verb and a noun.
technical terms related to software. It is both a verb and a noun.
This and all other IEEE Software Engineering Standards are available from
computer.org (ieee.org is for all IEEE standards, including many related to
hardware. It has a link to the Computer Society’s site, which is where the
software related standards are found.

The definition of what constitutes a “test case” varies. This is the definition for
purposes of this class. It includes everything needed to execute the test in a
purposes of this class. It includes everything needed to execute the test in a
reproducible fashion, which includes:
• One or more data values which are to be input
• The trigger event that starts the test
• One or more data values which will be generated as a result of the input(s)
being processed (the output)
• The tools and data required to allow the test to execute (the environment)
• The order in which the tests are to be run

Often, the only inputs considered are those entered through the user interface.
In reality, there are often many other inputs.
In reality, there are often many other inputs.

Outputs are just as varied as inputs.

Testing every possible data value, every possible navigation path through the
code, or every possible combination of input values is almost always an infinite
code, or every possible combination of input values is almost always an infinite
task which is impossible to complete. Even if it were possible, it is not a good
idea because many of the test cases would be redundant, consume resources
to create, delay time to market, and not add anything of value.

Since it is impossible to test everything, the coverage is based on trade-offs,
considering factors such as the following:
considering factors such as the following:
• How much time is available to prepare and/or execute tests
• The most cost-effective approach in terms of the return on the investment
• Priorities based on risk (test more of higher risk)
• Use tools when appropriate
• Find as many defects as possible in the time available

This series of sections is the heart of the tutorial.
This chapter includes the “scientific” methods—the ones with definitions and
This chapter includes the “scientific” methods—the ones with definitions and
specific ways for selecting test data. It is the largest and most comprehensive
chapter in the tutorial. It is followed by white box or structure based tests. The
last chapter in this section will address the “creative” (not structured) ways of
testing.

Each of these techniques has certain capabilities and limitations.There are
some types of defects/problems that each technique can detect and there are
some types of defects/problems that each technique can detect and there are
some things that the individual techniques are not designed to detect. By
combining several techniques the individual strengths can be used to offset the
weaknesses.

The techniques presented in this chapter apply to all levels of test—from a
single unit to many interfacing systems.The common factor is the emphasis on
single unit to many interfacing systems.The common factor is the emphasis on
input data ranges.

There are four basic rules to defining equivalence classes. These are noted
here and on the next slide.
here and on the next slide.
Depending on the complexity of the input it may also be necessary to subdivide
a partition into smaller partitions. Although the need to subdivide may be
predicated on what the set of values for a partition is.
For example:
The invalid partitions above may also contain alphabetic or special
characters and possibly negative numbers. These particular values can
be used in either invalid partition, essentially subdividing that partition into
four groups: Numeric, negative numeric, alphabetic and special
character.
Whether you do this subdivision or not is up to the tester. It may not be
necessary as these are actually “field type” tests and could be done as a
separate test depending on how the fields are created for the user interface.
If a developers kit is used to define the fields a single test of the kit for
these events may be sufficient.

You can also define classes for outputs as well as inputs.

Although the technique focuses on one requirement at a time it is permissible to
create test cases looking at all the requirements as a set. This will reduce the
create test cases looking at all the requirements as a set. This will reduce the
number of test required to validate the requirements under analysis.
If we were to write test to validate each individual requirement the number of
tests could increase substantially, especially if many requirements are involved.

Most will note that these are not the “best” test cases.The key is to define the
“minimum” number of tests required to test the requirements “partitions.”
“minimum” number of tests required to test the requirements “partitions.”
By combining requirements together it takes one test to check all the valid
partitions and six to test all the invalid partitions. We have to test invalids
individually because we do not know how the developer coded the error
handling (this is black box) and it is possible that the first error could stop the
test. Later, if we discover the developer uses a table (or similar construct) to
show all errors, we can combine the error tests as we did the valid tests.

One of the key reasons to do Equivalence Partitioning is to locate the
boundaries. If developers are going to make a mistake they will probably do it at
boundaries. If developers are going to make a mistake they will probably do it at
a boundary.

As noted sets are different. To ensure the set is complete and defined correctly
each member of the set is tested to ensure the set is coded correctly as they
each member of the set is tested to ensure the set is coded correctly as they
don’t actually have a boundary. So the third valid test is for the type of property
variable. You can choose any valid value for the other partitions.
These values do a better job of testing the partitions and the boundaries of the
partitions. You will notice that by focusing on multiple inputs together (four
corners) we don’t actually increase the number of tests but make them more
effective.
Invalids must still be tested individually for the same reasons previously noted.

The ground rule would be to have every test include (“cross”) at least three
functional areas. For example, Test 1 includes Remote Application 1, Mail
functional areas. For example, Test 1 includes Remote Application 1, Mail
server 1, and the User interface. Test 3 crosses five functional areas.

Conditions “1” through “X” show the allowed input conditions
Actions “1” through “Z” are the actions that will be taken based on combinations
of conditions.
Each Rule defines a unique set of conditions that result in the indicated set of
actions taking effect. Actions do not care about the order of the conditions or the
order in which they are checked, the key is their combined values.

Conditions that are simply yes or no events, binary decisions, are very easy to
map to a decision table.
map to a decision table.
You’ll notice that rules one and two contain a “immaterial” condition. The
“Unearned income >$1560” has no affect on the resulting action. Technically we
could ignore this value in testing, however, if we did how could we be sure that it
did not alter the outcome/action?

If claims and age are valid, then the three actions will be applied accordingly.
If claims or age is invalid, there is no action to take. In fact it is “impossible” to
have an invalid claim. If you enter a negative number this would be a “field type”
test and in reality you would not file a “negative” claim. All claims above 5 are
treated the same, no matter how many.
As noted earlier, you may have a license and be over 85 or under 16 but this
application does not accept those values and would generate an error. If we
wanted to we could add a fourth action, generate error message.

Here is where we can combine techniques.
Using EP/BV and four corners, we can reduce the number of tests required for
Using EP/BV and four corners, we can reduce the number of tests required for
each rule.
• The lower boundary value of all but Rule 1 puts you into another rule.
• The upper boundary value of all but Rule 8 puts you into another rule.

This is a simplified diagram for making an airline reservation.
After each user activity “event,” the system/application is in some “state.” It is
this state that will affect what the requestor can or cannot do next. To test this
process, you need to know how to perform each activity, what the acceptable
pathways are supposed to be, and the business rules that apply, before, during
and after each state and event.
This type of diagram helps you plan your functional testing to ensure you not
only exercise the functions of the systems but also check the system/design
itself.

Trace Matrices can support multiple levels. This is an example of a four level
matrix.
matrix.

This is an example of a Traceability Matrix linking requirements to design. It can
be used for any predecessor/successor pair of documents. It can also be used
be used for any predecessor/successor pair of documents. It can also be used
to document any requirements which are NOT going to be addressed by the
tests. Some useful optional additional columns:
• Comments: any questions, assumptions, etc.
• Priority: the importance of the requirement (with the implication that the
lower rated ones have less tests which may not even get run when time
is short).

This is a summary of the techniques in this section vs. the possible applicable
levels of Black Box testing. It is not intended in any sense to be restrictive - use
levels of Black Box testing. It is not intended in any sense to be restrictive - use
whatever works in your environment.

This is the last of the Test Design Techniques sections. It includes some very
interesting tests - those which are NOT based on a logically structured
interesting tests - those which are NOT based on a logically structured
technique!

All of the scientific techniques in the last two chapters are of great value, but
there is still more needed. Since the programming field tends to attract people
who are very logical, it is helpful to have someone—who can do the illogical—
test the system, too. After the system is in production, real circumstances will
not be constrained to only logical processes, so it is good to test them before
the system goes into use.

All of the scientific techniques in the last two chapters are of great value, but
who are very logical, it is helpful to have someone—who can do the illogical—
test the system, too. After the system is in production, real circumstances will
not be constrained to only logical processes, so it is good to test them before
the system goes into use.

This is a talent that some people have, and some don’t (the author of this
course is a “don’t”). Some people can just walk up to a system, take one look at
course is a “don’t”). Some people can just walk up to a system, take one look at
it, and say, “I bet if I do this it will break,” and be right! It is great for a testing
team to have people with this kind of talent participating.

One of many excellent James Bach quotes!

This is a formalization of what many testers have been doing for a long time.
Even with specifications, testers rarely have all of the information they really
Even with specifications, testers rarely have all of the information they really
need. And not everyone gets a complete set—or sometimes any—
specifications!
James Bach, the leader in this field, is researching and working on quantifying
what a good tester does when confronted with unproven software and not
enough information.

These are the basic steps for exploratory testing, which consists of starting with
whatever we know and going forward in a constructive manner. This is based on
whatever we know and going forward in a constructive manner. This is based on
James Bach’s description of the process.

The last category has no bounds at all. Just do the tests totally illogically!

These slides on Taxonomies are based on information, provided by James Bach
and Cem Kaner in their course on “black box testing” and from Lee Copeland’s
and Cem Kaner in their course on “black box testing” and from Lee Copeland’s
book, A Practitioner’s Guide to Software Test Design.

These are sources for some example Taxonomies

This is Boris Beizer’s high level taxonomy for where defects can occur.
You may want to build a similar taxonomy based on your applications and
systems. You can use one of the samples noted to get started.

Even considering the top two levels, it is quite extensive. All four levels of the
taxonomy constitute a fine grained framework on which to categorize defects.
taxonomy constitute a fine grained framework on which to categorize defects.

Your instructor will review which parts of the class helped with the challenges
identified in the beginning of the course.
identified in the beginning of the course.

Some suggestions for your evaluations!

Hope this class really helped!

Fundamental Test Design Techniques

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (13)

Similar to Fundamental Test Design Techniques

Similar to Fundamental Test Design Techniques (20)

More from TechWell

More from TechWell (20)

Recently uploaded

Recently uploaded (20)

Fundamental Test Design Techniques