Form and structure of test case MATTERS!

Form and Structure
of Test Case
MATTERS.

T Ashok
ash@stagsoftware.com

in.linkedin.com/in/AshokSTAG
ash_thiru

Webinar: Dec 7, 2011, 1430-1530 IST

Form is about the
external shape,
the way it is presented.

Courtesy http://carwallpapers.in

Structure is about the
composition of the
elements of a system.

Courtesy http://www.pcars.us

© 2011. STAG Software Private Limited. All rights reserved. 2

The Roller Coaster

Form & Structure is the key to:

Creating the twists & turns
From htt
p://gadget
s.gunaxin
.com/ﬁfty
-roller-co
asters/65403
to
Experience the thrill of high G forces
yet
Ensuring safety
and enable
it to be built from desired materials

Test cases should excite the system in interesting
ways to shake out the bugs that matter.

i.e. Effective


Test cases or Stimuli is the “combination of inputs”.
We know that these are combinatorial in nature.

The key is to generate the smallest yet sufﬁcient set
and be optimal.

i.e Efficient


The underlying architecture i.e.
Form & Structure plays a vital
role in ensuring a “scaffolding”
that is light yet strong.

Effective & Efficient


Form & Structure of Test Cases must ENABLE:

Clarity of the entity that I am testing?

The types of defects am I looking for - Clear objective

The stage when should I execute these Rapid generation of test cases

The importance of these- Clear prioritisation A logical & methodical review

Frequency of execution - How frequent? Foster just-enough documentation

Objective of assessment - Conformance or Robustness?

The ease of conversion into automated scripts?


A good “Form & Structure” fosters
simplicity, robustness and most importantly
enables clear thinking.

Clarity of thinking is key to producing good
artifacts that can be objectively assessed to
ensure conﬁdence.
... or shall we say Guarantee?


Properties to be satisfied by
a Good Form & Structure

Enable setting a clear baseline for test Design
Enable good design

Enable objective evaluation of adequacy Review
Aid in checking if it is designed well

Enable quick yet ﬂexible and robust automation Automation

Foster rapid design & test Execution
Allow for choosing test cases to execute logically

Enable clear & objective assessment of “cleanliness” Assessment

Allow for continual improvement and “de-weeding” Improvement


Hypothesis Based Testing - HBT 2.0
A Quick Introduction Personal, scientiﬁc test methodology.
SIX stage methodology powered by
EIGHT disciplines of thinking (STEMTM).

Setup Hypothesize
Cleanliness Criteria Potential Defect Types

SUT
Nine Stage
Cleanliness Assessment
Defect Removal Filter

Click here to know more about HBT.
http://slidesha.re/qBMNiy


Cleanliness Levels in HBT 2.0
Nine-Stage Filter to Detect Defects
Objective Issues

That user expectations are met
L9 End user value User flows, experience

That it deploys well in the real environment
L8 Clean Deployment Compatibility, migration
That the stated attributes are met
L7 Attributes met Performance, security, volume, load...
That it does not mess up the environment
L6 Environment cleanliness Resource leaks, Compatibility...

That end-to-end flows work correctly
L5 Flow correctness Business flow conditions, Linkages
That the functional behaviour is correct
L4 Behaviour correctness Functionality

L3 Structural integrity That the internal structure is robust
Internal structural issues
That the user interface is clean
L2 Input interface cleanliness UI issues

That inputs are handled well
L1 Input cleanliness
Input data handling


HBT Test Case Architecture
Nine-dimensional FORM
Organised by Cleanliness levels
sub-ordered by items (features/modules..),
Level segregated by type,
ranked by importance/priority,
Item sub-divided into conformance(+) and robustness(-),
classiﬁed by early (smoke)/late-stage evaluation,
Type tagged by evaluation frequency,
linked by optimal execution order,
classiﬁed by execution mode (manual/automated)
Priority

Focus

Stage
A well architected set of test cases is like a
effective bait that can ‘attract‘ defects in the
Frequency
system.
Order

It is equally important to ensure that they
Mode
are well organised to enable execution
optimisation and have the right set of
information to ensure easy automation.


Focused Scenarios enable
Form
Rapid and Easy automation

L9 End user value Level based test scenarios foster
L8 Clean Deployment
“single-path” script.

L7 Attributes met

L6 Environment cleanliness

L5 Robustness

L4 Behaviour correctness

L3 Structural integrity

L2 Input interface cleanliness

L1 Input cleanliness

... yields shorter scripts
... enables rapid scripting
... makes it more ﬂexible
... eases maintenance

Clear Assessment of Cleanliness Form

L4

PDT10 TT8

PDT9 TT7 Quality report
CC1 CC2 CC3 CC4
L3 PDT9 TT6 R1 Met
R2
PDT8 TT5 Not met
R3
Cleanliness

Partially met
PDT7 R4
TT4
L2 PDT6 R5

PDT5
TT3
PDT4
L1

PDT3
TT2
PDT2

PDT1 TT1

Stage


Better Assessment of Adequacy Structure

should satisfy impeded by
Fault Traceability
Cleanliness Criteria

Potential Defect
Entity under test
Types

Fault
Requirements Test Cases traceability
traceability “test for what”
“what to test”

It is necessary for test cases to be purposeful/goal-focused.
Tracing test cases to requirement is a necessary condition for this, but not sufﬁcient!
Two way tracing i.e. Requirements + FAULT makes this sufﬁcient.


Better Assessment of Adequacy Structure

Fault Traceability
Fault Fault
traceability traceability
Test for what?
PD1 R1 TC1 PD1
PD2 R2 TC2 PD2
PD3 R3 TC3 PD3
... ... ... ...
PDn Rm TCi PDn
Requirements
traceability What to test?

That test cases for a given requirement shall have
the ability to detect speciﬁc types of defects
FAULT COVERAGE


Entity Behavioural
Stimuli Structure
under test scenarios

Inputs Conditions Outputs

}
1
C4
i1 2
C2
3
i2 C1
4
C3
i3 5

• What an input(stimuli) is an therefore its values depends on the cleanliness level.
• Conditions govern behaviour, what a condition is also depends on the cleanliness level


Structure of a Test Scenario in HBT

[EUT-ID + TS-ID] [Behaviour evaluation goal]

R1.TS1 Ensure that the system does/does-not
or Check that.../Validate that...

A clear and concise one-liner.
A collection of these shall cover all the behaviours (intended
or un-intended behaviours)

The structure of test scenario “forces” clarity of thinking and is
sharply goal-focused i.e. what do we want to check in the behaviour.

Create a behavioural model and use this to generate test scenarios.

Techniques
landscape


Structure of a Test Scenario (Detailed)

[EUT-ID + TS-ID] [Behaviour evaluation goal]

{
[Evaluation pre-conditions]
[Evaluation steps]
[Evaluation post-conditions/oracle]

1. Ensuring consistency in execution
Useful for
2. Improving execution efﬁciency automation

Clarity of purpose of a scenario is critical.
Having a set of steps without clarity of purpose is like listing the
directions without being clear about the destination.


Structure of Test Cases

[Combination of values of
[EUT-ID + TS-ID + TC-ID] [Expected value]
pertinent inputs]
{I11 , I21, , I31}
Techniques
{I12 , I22, , I32} landscape

A set of test cases for a scenario is the
optimal cartesian product.
Document in a tabular form.
Input granularity
principle

What an “input” is is key here.
The values for each input depend on “what an input is”.
Remember Basic Types & Abstract Data Types in programming languages?


Results of using this Form & Structure by
Applying HBT

Ability to question requirements better
1. Find defects in requirements
2. Ensuring adequacy in scenarios/cases.

Improved existing scenarios/cases by 2x-3x increasing defect yield

Reduced in-process defect escapes by 30% due to better dev test criteria

Lower maintenance on automation and better ROI on functional automation

Lessen the dependency on the experience as key factor to good test design

Able to convince customer logically about completeness


Summarising...

FORM = External Shape

STRUCTURE = The way it is composed
Level

Clarity of test Beneﬁts Form and Structure of Item
Test Case MATTERS. NINE Dimensional Form

Test adequacy
Type

Logical review-ability
Priority

Shorter scripts
Stage
Structure EUT-->TS-->TC

Rapid design
Frequency
Fault traceability

Better criteria
Order
Cleanliness index

Prioritisation
Mode


Thank you!

Connect with us...
@stagsoft
blog.stagsoftware.com

HBT is the intellectual property of STAG Software Private Limited.
STEMTM is the trademark of STAG Software Private Limited.

© 2011. STAG Software Private Limited. All rights reserved. www.stagsoftware.com

Form and structure of test case MATTERS!

More Related Content

Similar to Form and structure of test case MATTERS!

More from STAG Software Private Limited

Recently uploaded

Form and structure of test case MATTERS!