EuroSTAR Software Testing Conference 2009 presentation on by Michael Monaghan. See more at conferences.eurostarsoftwaretesting.com/past-presentations/

1. Paula O’Grady

2. Prioritising Tests - Why?
You never have enough time to do all the testing
you want/need to do.
Allocated testing time may ‘shrink’.
How to decide which tests take precedence?
Attach a severity or priority, or a calculation of
the two.
Other labour-intensive methods.
Limitations with this approach in my experience!

3. Prioritising Tests – Issues?
Ironically, the traditional prioritising techniques you
use may deplete available time even more.
Like buying an expensive wallet and not having
enough money left to put into it!
Perhaps we need to examine the methods we use to
prioritise tests to maximise efficiency.
In addition, alternative techniques may provide
insight, and a new viewpoint.
Historically, the best ideas have come from the most
unlikely sources.

4. Possible Solutions
Lateral-thinking approach.
As with most ‘radical’ methods, they work best when
used in tandem with the tried-and-tested traditional
methods.
Betting & weighting approaches involving teams
guessing where most bugs will be found.
Dynamic Betting Exchanges.
Split-Second Decision Making and Rapid-Cognition.
Gut-Instinct.

5. Betting Approach
Divide the product into functional/test areas – this is
the difficult part in my experience.
Give each person in the group a virtual €100 to ‘bet’,
using as many denominations as they wish.
The non-technical team members were especially
encouraged to provide gut-instinct feedback,
independent of ‘developmenttest’ thinking.
In theory, participants can change their bets on the fly
and as regularly as they wish – exchange aspect.
No need for sophisticated tools.

6. Betting Approach – Case Study
Collate data:
JW FM EMG DMW PD SOS POG CK CON DMC AK RR MM AC Total %
1Create Webs 10 2 5 10 2 2.6
2Alerts and Notifications 10
2 1.1
3Documentation 5 10 15 10 10 5 15 5 6.8
4Export Report and Generate Report 5
3 2 10 5 4 2.6
5Globalisation & Localisation
5 5 5 1.4
6Kit – Install and Uninstall
5 10 10 5 5 3.2
7Kit – Upgrade 5 15 10 10 5 5 4.5
8Project Schedule List 10 33 40 35 10 15 20 15 10 17.0
9All Other Lists and Libraries
0.0
10Usability 5 0.5
11Security 10 2 5 1.5
12Load and Performance 2 10 10 5 2.1 2.6
13BrightWork Reporter & Resource Reporting* 50
5 30 10 30 30 15 20 10 27.9 20.7
14Gantt Web Part 1 3 2 25 2.8
15*Chart Web Part 5 10 10 30 15 20 15 8 20 12.1
16Agile Template 1 5 3 2 1.0
17Sure Step Templates 2 5 5 3 2 10 2.4
18All Other Templates 10 20 10 10 5 5.0
19All Other Web Parts 4 0.4
20Choice Indicator Icon Column*
2 5 10 5 1 5 2.5
21Project Calendar* 5 20 30 10 10 2 25 9.2
22Other – Please Specify 0.0

7. Betting Approach – Case Study
The bets were charted. Scale runs from 0% to 25 %.

8. Betting Approach – Case Study
Advantages Disadvantages
Quick.
Fun.
Give people a chance to get
involved (who otherwise
wouldn’t).
Less weighted by the
thinking of the core technical
group.
Allows for ‘unexpected’
anomalies to be found.
Difficult to define the ‘areas’
to bet on.
May not be taken seriously.
Cannot be used on its own.
People may try to ‘follow the
crowd’ and bet safely – this
defeats the purpose.
Needs management buy-in.

9. Rapid Cognition
Looking away from the arena of Software Testing.
Books that provide insight into how to make
effective decisions quickly:
 Blink – Malcolm Gladwell
 The Gift of Fear – Gavin deBecker
 Other Books
Not written specifically for a Software Test audience,
but the principles and how they apply to decision-
making are universal.

10. Rapid Cognition
What can these books teach us about prioritising
tests?
The lessons here are universal.
In times of stress, decisions need to be made.
How firemen, policemen & doctors make life-
changing and life-saving decisions.
Is there such a thing as ‘Too Much Information’ when
making decisions?

11. Gut Instinct
Does “Gut feeling = Irrational Decision Making”?
In testing terms, using gut instinct could be viewed as
analogous to Exploratory Test Techniques for software.
Not having to explain each small decision and just
going with your instinct.
Not necessarily being psychic – you may feel a piece of
code has a bug in it because of something that
happened the day it was being developed.
The feeling may not necessarily be fully articulated.

12. Conclusions
The techniques described in this presentation should
be used in addition to traditional methods
As Agile methods become more commonplace, we
need to look at modernising all activities to do with
Test Planning.
The bugs that make it into production are generally
the ones ‘that we didn’t expect’.
Less traditional methods of prioritising may help
mitigate this risk.

13. www.brightwork.com
Copyright of BrightWork 2009