The document describes STAG Software's HBT Quality Visualization Tool. The tool allows users to assess quality across three areas in 3 sentences: 1) Are test assets good? It evaluates the quality of test cases by analyzing factors like applicable test types, test case counts by importance and quality level, and positive/negative ratios. 2) Have we assessed completely? It measures the quality of execution by analyzing execution metrics like percentages completed by test, entity, quality level, and progress over cycles. 3) How good are the outcomes? It determines the quality of the product/application by calculating a cleanliness index based on passed and total test cases, and analyzing performance by entity, cleanliness criteria, and

1. Deliver superior outcomes using
HBT Quality Visualisation Tool
Powered by HBT
T Ashok
Founder & CEO
STAG Software Private Limited
Architect - HBT
in.linkedin.com/in/AshokSTAG
Ash_Thiru

2. Delivering superior outcomes requires:
An excellent orchestration of
Good work good understanding, purposeful strategy, robust
test design and rapid execution
Good “mirror” for a clear & objective feedback rapidly to aid
continuous adjustment & adaptation.
© 2014 STAG Software Private Limited. All rights reserved.
2

3. The typical “mirror”
Normally this is done by collecting & analysing typical
“extrinsic measures” like:
Requirements traceability, Coverage- Code/Test
#Cycles, TC planned vs executed, Blockers
TC - Pass/fail rate, First pass yield
Defect info- Distribution, Rate, Density, Leakage
These are good, but not good enough
to be sharply goal focused.
© 2014 STAG Software Private Limited. All rights reserved.
3

4. Good “mirror” needs to tell ...
1
Are test assets good?
quality of test cases
2
Have we assessed completely?
quality of execution
3
How good are the outcomes?
quality of product/application
© 2014 STAG Software Private Limited. All rights reserved.
4

5. Hypothesis Based Testing - HBT
should satisfy
Cleanliness
Criteria
System Under
Test
Requirements
traceability
“what to test”
© 2014 STAG Software Private Limited. All rights reserved.
impeded by
Potential Defect
Types
Test Cases
Fault
traceability
“test for what”
5

6. Quality Levels & PDTs - Powered by HBT
L9
L8
L7
L6
L5
L4
L3
L2
L1
End user value
That user expectations are met
PDTs related to User flows, User experience
Clean Deployment
That it deploys well in the real environment
PDTs related to Compatibility, Migration
Attributes met
That the stated attributes are met
PDTs related to Performance, Load, Volume......
Environment cleanliness
That it does not mess up the environment
PDTs related to Resource leaks, Compatibility...
Flow correctness
That end-to-end flows work correctly
PDTs related to Flow behaviour, Interactions
Behaviour correctness
That the functional behaviour is correct
PDTs related to Functionality
Structural integrity
That the internal structure is robust
PDTs related to Structural aspects
Input interface cleanliness
That the user interface is clean
PDTs related to UI
Input cleanliness
That inputs are handled well
PDTs related to Input data correctness
© 2014 STAG Software Private Limited. All rights reserved.
6

7. The W3T
End user value test
PDT 57-59
QL8 Clean deployment
Installation test
PDT 54-55
QL7 Attributes met
LSPS test
PDT 44-51
QL6 Environment cleanliness
Good citizen test
PDT 39-41
QL5 Flow correctness
Interaction test
PDT 35-38
QL4 Behaviour correctness
Functionality test
PDT 24-31
QL3 Structural integrity
Structural test
PDT 14-23
QL2 Input interface cleanliness API validation test
PDT 5-7
QL1 Input cleanliness
QL - Quality Level
PDT - Potential Defect Type
Migration test
PDT 56
Reliability test
PDT 52
Security test
PDT 53
Compatibility test
PDT 42-43
Powered by HBT
QL9 End user value
Access control test
PDT 32-34
GUI validation test
PDT 8-13
Input validation test
PDT 1-4
© 2014 STAG Software Private Limited. All rights reserved.
7

8. HBT Baseline - The KEY to Visualisation!
L1
T1
L2
Test-for-What(Expectation)
T2
T3
L9
Tn
What-to-Test (W2T)
PDT1-2
CC
PDT3-6
PDT7-10
PDT11-14
CC1
CC2
CC3
CCn
E1
✓
✓
E2
✓
E3
✓
E4
✓
Cleanliness Criteria
...
✓
List the entities under test (E)
(Features, Requirements...) in the
order of priority/importance.
Map them to the expectations of
the customers.
✓
✓
✓
TfW-E
Test-for-What(Expectation)
PDT Potential Defect Type
TfW-G
Test-for-What(Goal)
T*
Test Type
WhT
What-Test
L*
Quality Level
WhnT
When-Test
© 2014 STAG Software Private Limited. All rights reserved.
8

9. HBT Baseline - The KEY to Visualisation!
L1
L2
Test-for-What(Goal)
L9
CC
T2
T3
Tn
PDT1-2
What-to-Test (W2T)
T1
PDT3-6
PDT7-10
PDT11-n
CC1
CC2
CC3
CCn
E1
✓
✓
E2
✓
E3
✓
E4
✓
Cleanliness Criteria
...
✓
TfW-E
✓
✓
✓
TfW-E
Test-for-What(Expectation)
PDT Potential Defect Type
TfW-G
Test-for-What(Goal)
T*
Test Type
WhT
What-Test
L*
Quality Level
WhnT
When-Test
© 2014 STAG Software Private Limited. All rights reserved.
9

10. HBT Baseline - The KEY to Visualisation!
What-Test
L2
L1
L9
CC
T2
T3
PDT1-2
What-to-Test (W2T)
T1
PDT3-6
PDT7-10
CC1
CC2
CC3
E1
✓
✓
E2
✓
E3
✓
E4
✓
Cleanliness Criteria
Tn
PDT11-n TfW-G
...
✓
CCn
TfW-E
✓
✓
✓
TfW-E
Test-for-What(Expectation)
PDT Potential Defect Type
TfW-G
Test-for-What(Goal)
T*
Test Type
WhT
What-Test
L*
Quality Level
WhnT
When-Test
© 2014 STAG Software Private Limited. All rights reserved.
10

11. HBT Baseline - The KEY to Visualisation!
When-Test
L1
L2
L9
CC
T2
T3
PDT1-2
What-to-Test (W2T)
T1
PDT3-6
PDT7-10
CC1
CC2
CC3
E1
✓
✓
E2
✓
E3
✓
E4
✓
Cleanliness Criteria
Tn
WhT
PDT11-n TfW-G
...
✓
CCn
TfW-E
✓
✓
✓
TfW-E
Test-for-What(Expectation)
PDT Potential Defect Type
TfW-G
Test-for-What(Goal)
T*
Test Type
WhT
What-Test
L*
Quality Level
WhnT
When-Test
© 2014 STAG Software Private Limited. All rights reserved.
11

12. HBT Baseline - The KEY to Visualisation!
When-What & Test-for-What (W3T)
L1
L2
L9
Tn
CC
T2
T3
PDT1-2
What-to-Test (W2T)
T1
PDT3-6
PDT7-10
CC1
CC2
CC3
E1
✓
✓
E2
✓
E3
✓
E4
✓
Cleanliness Criteria
WhnT
WhT
PDT11-n TfW-G
...
✓
CCn
TfW-E
✓
✓
✓
TfW-E
Test-for-What(Expectation)
PDT Potential Defect Type
TfW-G
Test-for-What(Goal)
T*
Test Type
WhT
What-Test
L*
Quality Level
WhnT
When-Test
© 2014 STAG Software Private Limited. All rights reserved.
12

13. 1
Are test assets good?
1
Applicable test
types considered?
2
Any anomalies in
#TC wrt importance
3
#TC decreasing
wrt levels?
4
Countable?
+ve:-ve ratio ok?
quality of test cases
1
W3T
L1
L2
L9
T1
T3
Tn
PDT1-2
PDT3-6
PDT7-10
PDT11-14
CC1
4
T2
CC2
CC3
CC4
-ve
W2T
E1
#TC
+ve
E2
100
...
40
500
60
100
400
2
E3
E4
100
60
TOT
#TC
40
-VE
+VE
3
© 2014 STAG Software Private Limited. All rights reserved.
13

14. 2
Have we assessed completely?
quality of execution
1
2
L1
L2
L9
T1
T2
T3
Tn
PDT1-2
PDT3-6
PDT7-10
CC2
CC3
CC4
-ve%
W2T
E1 Tot%
E2
How are we doing
wrt a test & level?
4
Are we making
progress every cycle?
PDT11-14
CC1
How are we doing
wrt entity as a whole?
3
W3T
How are we doing
wrt test for a entity?
+ve%
1
90%
...
50%
2
85%
100%
75%
85%
E3
E4
3
97%
4
87%
100%
100%
100%
100%
© 2014 STAG Software Private Limited. All rights reserved.
TOT
%
-VE
+VE
14

15. 3
How good are the outcomes?
quality of product/application
If TC are proven complete, then
CI (Cleanliness Index) =
#TC-Passed/#TC-Total
W3T
L1
L2
Note that blocked TC would be
considered as potentially
failed.
L9
T1
T2
T3
Tn
PDT1-2
PDT3-6
PDT7-10
PDT11-14
CC1
CC2
CC3
CC4
Note we are measuring the
“intrinsic cleanliness” here.
-CI%
E1 CI(%)
+CI%
E3
2
How clean is the
full entity?
E4
W2T
1
How clean is the
entity wrt CC?
How are the entities
3 f faring wrt a CC?
E2
1
90%
...
50%
85%
100%
3
97%
2
75%
85%
4
87%
100%
100%
100%
100%
© 2014 STAG Software Private Limited. All rights reserved.
TOT
%
-VE
+VE
4
How clean is the
system as of now?
15

16. Summarising : Good “mirror” needs to tell ...
1
Are test assets good?
quality of test cases
2
Have we assessed completely?
quality of execution
3
How good are the outcomes?
quality of product/application
© 2014 STAG Software Private Limited. All rights reserved.
16

17. HBT Overview
SIX staged purposeful activities, powered by EIGHT disciplines of thinking
SIX Stages of DOING
S1: Understand expectations
S2: Understand context
S3: Formulate hypothesis
S4: Devise proof
S5: Tooling support
S6: Assess & Analyse
S1
S6
D1
D2
D8
S5
Core Concepts
D7
D6
D3
D5
D4
S4
S3
S2
EIGHT Disciplines of Thinking
D1: Business value understanding
D2: Defect hypothesis
D3: Strategy & Planning
D4: Test design
D5: Tooling
D6:Visibility
D7: Execution & Reporting
D8: Analysis & Management
Uses 32 Core Concepts
For Problem solving
- Techniques, Principles, Guideline
Click here to know more about HBT.
http://stagsoftware.com/blog?p=570
© 2014 STAG Software Private Limited. All rights reserved.
17

18. 1
2
Have we assessed completely?
“Quality of execution”
3
Deliver superior outcomes using
HBT Quality Visualisation Tool
Are test assets good?
“Quality of assets”
How good are the outcomes?
“Quality of the product/application”
Powered by HBT
Thank you.
Connect with us...
@stagsoft
blog.stagsoftware.com
HBT is the intellectual property of STAG Software Private Limited.
STEMTM is the trademark of STAG Software Private Limited.
© 2014 STAG Software Private Limited. All rights reserved.
www.stagsoftware.com