The document outlines the principles and techniques for test design within software testing, highlighting the importance of test conditions, cases, and procedures. It emphasizes the need for traceability between requirements and tests, as well as the differing levels of formality in test documentation depending on the context. The document also discusses the process of determining expected results and grouping test cases effectively for implementation.

1. Taufik Hidayat
Information System Department
Faculty of Science and Technology
State Islamic University of Sultan Syarif Kasim Riau
(UIN SUSKA RIAU)
http://sif.uin-suska.ac.id/
http://fst.uin-suska.ac.id/
http://www.uin-suska.ac.id/
Test design techniques

2. Before we can actually execute a test, we need to know what we are trying to test, the inp
uts, the results that should be produced by those inputs, and how we actually get ready for and ru
n the tests.
In this section we are looking at three things: test conditions, test cases and test procedur
es (or scripts) – they are described in the sections below. Each is specified in its own document, a
ccording to the Test Documentation Standard [IEEE829].
Test conditions are documented in a Test Design Specification. We will look at how to cho
ose test conditions and prioritize them.
Test cases are documented in a Test Case Specification. We will look at how to write a go
od test case, showing clear traceability to the test basis (e.g. the requirement specification) as well
as to test conditions.
In this section, look for the definitions of the glossary terms: test case, test case specific
ation, test condition, test data, test procedure specification, test script and traceability.
Introduction

3. Testing may be performed with varying degrees of formality. Very formal testing would hav
e extensive documentation which is well controlled, and would expect the documented detail of th
e tests to include the exact and specific input and expected outcome of the test. Very informal testi
ng may have no documentation at all, or only notes kept by individual testers, but we'd still expect
the testers to have in their minds and notes some idea of what they intended to test and what they
expected the outcome to be. Most people are probably somewhere in between! The right level of f
ormality for you depends on your context: a commercial safety-critical application has very differen
t needs than a one-off application to be used by only a few people for a short time.
Formality of test documentation

4. Test analysis is the process of looking at something that can be used to derive test informa
tion. This basis for the tests is called the 'test basis'. It could be a system requirement, a technical s
pecification, the code itself (for structural testing), or a business process. Sometimes tests can be b
ased on an experienced user's knowledge of the system, which may not be documented. The test
basis includes whatever the tests are based on. This was also discussed in Chapter 1. From a testi
ng perspective, we look at the test basis in order to see what could be tested – these are the test c
onditions. A test condition is simply something that we could test. If we are looking to measure co
verage of code decisions (branches), then the test basis would be the code itself, and the list of tes
t conditions would be the decision outcomes (True and False). If we have a requirements specificat
ion, the table of contents can be our initial list of test conditions.
Test analysis: identifying test
conditions

5. Why is traceability important? Consider these examples:
• The requirements for a given function or feature have changed. Some of the fields now have
different ranges that can be entered. Which tests were looking at those boundaries? They now
need to be changed. How many tests will actually be affected by this change in the requireme
nts? These questions can be answered easily if the requirements can easily be traced to the
tests.
• A set of tests that has run OK in the past has started to have serious prob lems. What function
ality do these tests actually exercise? Traceability between the tests and the requirement bein
g tested enables the functions or features affected to be identified more easily.
• Before delivering a new release, we want to know whether or not we have tested all of the sp
ecified requirements in the requirements specification. We have the list of the tests that have
passed – was every requirement tested?
Test analysis: identifying test
conditions

6. Test conditions can be rather vague, covering quite a large range of possibilities as we sa
w with our mobile phone company example (e.g. a teenager in the mid-west), or a test condition m
ay be more specific (e.g. a particular male customer on pay-as-you-go with less than $10 credit). H
owever when we come to make a test case, we are required to be very specific; in fact we now nee
d exact and detailed specific inputs, not general descriptions (e.g. Jim Green, age 17, living in Gran
d Rapids, Michigan, with credit of $8.64, expected result: add to Q4 marketing campaign). Note tha
t one test case covers a number of conditions (teenager, male, mid-west area, pay-as-you-go, and
credit of less than $10).
Test design: specifying test cases

7. In order to know what the system should do, we need to have a source of information abo
ut the correct behavior of the system – this is called an 'oracle' or a test oracle. This has nothing to
do with databases or companies that make them. It comes from the ancient Greek Oracle at Delphi
, who supposedly could predict the future with unerring accuracy. Actually her answers were so vag
ue that people interpreted them in whatever way they wanted – perhaps a bit like requirements spe
cifications!
Once a given input value has been chosen, the tester needs to determine what the expect
ed result of entering that input would be and document it as part of the test case.
Expected results include information displayed on a screen in response to an input, but they also in
clude changes to data and/or states, and any other consequences of the test (e.g. a letter to be pri
nted overnight).
Test design: specifying test cases

8. The next step is to group the test cases in a sensible way for executing them and to specif
y the sequential steps that need to be done to run the test. For example, a set of simple tests that c
over the breadth of the system may form a regression suite, or all of the tests that explore the worki
ng of a given functionality or feature in depth may be grouped to be run together.
Some test cases may need to be run in a particular sequence. For example, a test may cr
eate a new customer record, amend that newly created record and then delete it. These tests need
to be run in the correct order, or they won't test what they are meant to test.
Test implementation: specifying test
procedures or scripts

9. Test procedure DB15: Set up customers for marketing campaign Y. Step 1: Open database with wr
ite privilege Step 2: Set up customer Bob Flounders
male, 62, Hudsonville, contract Step 3: Set up customer Jim Green
male, 17, Grand Rapids, pay-as-you-go, $8.64 Step 4: ...
We may then have another test procedure to do with the marketing campaign:
Test procedure MC03: Special offers for low-credit teenagers Step 1: Get details for Jim Green fro
m database Step 2: Send text message offering double credit Step 3: Jim Green requests $20 cre
dit, $40 credited
Writing the test procedure is another opportunity to prioritize the tests, to ensure that the b
est testing is done in the time available. A good rule of thumb is 'Find the scary stuff first'. However
the definition of what is 'scary' depends on the business, system or project. For example, is it wors
e to raise Bob Founders' credit limit when he is not a good credit risk (he may not pay for the credit
he asked for) or to refuse to raise his credit limit when he is a good credit risk (he may go elsewhe
re for his phone service and we lose the opportunity of lots of income from him).
Test implementation: specifying test
procedures or scripts

10. Reference
Graham, Dorothy, et al. “Foundation of Software Testing ISTQB Certification”.
Chapter 4 Test Design Techniques. 2011