Regression is not easily understood, as it seemingly manifests from nowhere. But if you can identify methods to help spot quality failure ‘trends’, you stand a better chance of understanding the root causes. This presentation serves to highlight a number of risk identification and planning techniques that you could add to your arsenal! Presented at TestExpo in London, UK, on 31 Oct 2017.

1. An analytical approach to effective
risk based test planning
Francis Miles

2. A ‘quick’ Introduction to Infuse

3. WhoWe Are – Modern SoftwareTesting Experts
Established in 2002
• Infuse Consulting Ltd was founded in London, UK
• Based in Westminster, London
• Focused on modern software testing
• 80 consultants spread over UK, Portugal and India
Strong alliance and partnerships
• Hewlett Packard Enterprise Gold Partner in Application Delivery Management (ADM)
• Oracle Gold Partner in Application Quality Management (AQM)
• SAP Partner
• Modern SoftwareTesting partner for Capgemini Nordics, DXCTechnology (globally) and
Deloitte (UK)
Innovation is in our DNA
• Pioneering automated testing for Mobile Products since 2003
• First automated Business ProcessTesting Specialists in the UK 2004
• First Oracle partner to build its own IP around Oracle’s test tools
• Test Automation solutions for Oracle, SAP,Web, etc.
Objective and Independent
• Our IP: useMango – ScriptlessTest Automation for SAP, Oracle, Web
• Open Source
• Our enterprise partners’ products:
• Oracle ApplicationTesting Suite and HP Enterprise ADM Solutions

4. ScriptlessTest Automation for SAP, Oracle, Web

5. If software deliverables from external (or internal)
sources were managed well, WHY is it we see too
many regression manifested quality risks towards the
end of the project?
(Spec issues not the focus of the talk today )

6. No matter what methodology you adopt, you will find
yourself accruing MANY test cases over the lifetime of
a project. Is it really feasible to run all of these on a
regular basis?
To understand regression within your
project, there is a strong desire to rerun most,
if not all test cases

7. The regression test conundrum
Integration
Supplier A
Supplier B
Supplier C
Supplier D
or
Reliable background,
proven track record
Little is known about
these people
These guys are
CHEAP!
These guys are
EXPENSIVE!!

8. Regression
IS
A
DISEASE
But not completely incurable

9. Everyone, regardless of whether you are running
Waterfall
V Model
Agile
Etc
Should have a regression strategy
(Especially if your tests are predominantly manual)

10. ATypical test project… some may disagree

11. Total number of features
(Feature Test Case Totals not shown)
Derivation of a project test Model
Feature Complete
An example: 34 Features defined,
Feature complete after 23 weeks

12. Derivation of a project test Model
Test resource needed
For Full retest per release
Test team capacity = 5;
Clearly not enough
people!
Resource Model – Not Realistic!
Absolutely worst case
[Full Retest per release strategy]

13. The Choice for today

14. Example Project test plan –VERY simplified

15. Calculating risk - Automatically
We will build a
TREE as an
example!

17. Analysis #1 : Regression (A pure qualitive measurement)
If we measure the PASS to FAIL transitions, what do we get?
Highest
Risk?
Lowest
Risk?

18. Analysis #2:Test Latency
(number of releases not tested) Time Since Last Run
Highest
Risk?
Lowest
Risk?
Medium
Risk?

19. Analysis #4 -(TSLP)
‘Time Since Last Passed’ - not including NR
The number of releases a test case has not passed
Analysis #5 -TSLF
‘Time Since Last Fail’ – not including NR
Perhaps not as high a priority as the TSLR or Regression metrics
Analysis #6 - CTPC
‘Consecutive Test Pass Consistency’ – not including NR
The number of consecutive passes (analysed from NOW backwards in time)
Other Measurement Ideas…

20. Bringing it all together
But How do
we use this
data?

21. This data is not normalised. We need to
 Introduce Weighting factors to ensure
data fields have the correct priority
This can differ from company to company.
The priorities I chose are as follows:
TSLR
2
TSLP
1.5
TSLF
1
REG
CTPC

23. Lets Sort the
data to make it
easier to read

24. Perhaps choose the
top 25% of high risk
tests for regression?

25. Summary – Identifying Risk through analysis
Provides a clearer picture of the true project risk towards the end
As tests stagnate, their risk score increases, increasing the likelihood of
retest
During regression testing, it helps Identify severe risk factors faster
You gain a new insight into how ‘features’ are maturing
Some feature may have little risk scores, some may have high risk scores
This could indicate deliverables from a specific supplier are poorly
managed (Poor practices, no process, waste of your money!)
Help to identify, and mitigation this supplier risk faster – Your supplier
project management models get better in the long term
It’s a GREAT way of selecting candidates for automation!

26. You start to question the norms, and evolve a clearer
picture of how maturity evolves within your
organisation
Don’t Forget:
– More data samples, lead to better models -

27. Want to learn more?
Contact Francis Miles - Francis.Miles@infuse.it
Or visit our website: infuse.it