Test Management for Busy People

TD
Half-day Tutorials
5/6/2014 8:30:00 AM
Test Management for Busy
People
Presented by:
Lloyd Roden
Lloyd Roden Consultancy
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Lloyd Roden
Lloyd Roden Consultancy
With more than twenty-eight years in the software industry, Lloyd Roden has worked as a
developer, test analyst, and test manager for many different organizations. Lloyd was a
consultant/partner with Grove Consultants for twelve years. In 2011 he created Lloyd Roden
Consultancy, an independent UK-based training and consultancy company specializing in
software testing. Lloyd’s passion is to enthuse, excite, and inspire people in the area of
software testing. He has spoken at conferences worldwide including STAREAST,
STARWEST, Better Software, EuroSTAR, AsiaSTAR, and Special Interest Groups in
software testing in several countries. In 2004, he won the European Testing Excellence
award.

Test Management
for Busy People
A half-day workshop
Written by
Lloyd Roden
www.lloydrodenconsultancy.com
Version 1_0
© Lloyd Roden

LRC110815
Copyright Notice
It is strictly prohibited to reproduce any of these materials in
any form without prior written permission from Lloyd Roden
Consultancy. This includes photocopying, scanning and the
use of any recording devices.

© Lloyd RodenTMBP140308
Test Management for Busy People Workshop-1
Test Management for
Busy People
1
Contents
Introduction
Doing more with less
Building an elite test team
Does it matter – does it matter much
2

Busyness is both good and bad
!   busy good
!   work is exciting, challenging and enjoyable
!   the day goes quickly
!   job satisfaction
!   busy bad
!   too much work to cope with
!   finding stress levels are constantly high
!   family suffer
are you currently busy
good or busy bad ?
3
Modern life wants more time…but we have less
time
4

Contents
Introduction
5
#1: Challenge redundant documentation
!   are these test documents really needed?
!   test plans
!   written test reports
!   test logs
!   detailed test scripts
!   test strategy
!   test retrospective
!   are these documents used or read?
!   how do you know?
!   what feedback do you receive?
!   are they kept up to date?
if you produce documents that are not used or
considered worthwhile then don’t produce them
you have
86,000
seconds in a day
How will you
spend them?
Don’t waste them?
6

#2: Eliminate waste
!   concept of eliminating waste is good
!   examples
!   meetings
!   emails
!   duplication of effort
!   documents not being used
!   maintenance of the tests taking longer
than the test execution
!   however…
!   sometimes cutting back too much causes
more problems
7
Wastage example 1: meetings
10 people @ 1 hour meeting = 10 hours
•  1 person arrives 6 minutes late………………………………
•  4 people receive texts and respond taking 2 minutes……..
•  1 person gets a call and leaves room for 15 minutes……...
•  6 people on laptops responding to emails…………………..
….1 hour
….80 minutes
…. 25 minutes
…. 3 hours
total: 6 hours wasted
= 60% inefficiencysuggestion: conduct meetings with etiquette
!   no laptops
!   no phones
!   no interruptions
!   start on time
8

Wastage example 2: bugs raised in error
(or re-raised)
Percentage
defects
re-raised
Version A
TOTAL 31/96 = 33%
Percentage
defects raised
in error
Version A
TOTAL 23/96 = 24%
N.B: figures are based on 1 day to raise and investigate.
And a cost to the company of 250€ per day
31 days
23 days
total wastage
13500€
54 days
(6% of the
project)
9
#3: Don’t count test cases…
…without knowing the quality of them !
!   quantity without quality is meaningless
!   example, in the last hour of testing:
!   tester 1 has run 30 test cases
!   know the quality of the tests
!   test the tests regularly
!   review the tests
!   check on coverage
the only number
with meaning is
zero
10

#4: Provide some slack
!   maximum velocity does not equal maximum productivity
August 2010: productivity in the United States
unexpectedly decreased in the second quarter
after employers expanded the workweek by
the most in four years. Employees output
decreased by 0.9% per hour of output
7
4
52
61
8 3
9
efficiency is improved
by 11.1% !
BUT we cannot play the game…
productivity is reduced…to zero "
source: Tom DeMarco, “Slack”
7
4
52
61
8
3
11.1% spare capacity
Car engines work at
their optimum efficiency
(mpg) at 56mph
Management’s decision:
to fill spare capacity to
increase productivity
Productivity declines
unexpectedly as US
workweek lengthens
11
People are not a fungible resource
money is fungible:
net result: still have £100
but people are not fungible:
net result: at least 30% loss of productivity
“on average there will be a net loss of 15%
productivity due to task switching”
bank 1
£100
bank 1
£30
bank 2
£50
bank 3
£20
100% on
project A
30% on
project A
50% on
project B
20% on
project C
therefore be careful of the saying
“women can multi-task and men can’t”
decide to
divide money
into 3 banks
management
decide to divide
your time into 3
projects
12

Case study: Personal Assistants – what a waste!
!   remember PAs: the manager’s ultimate resource
from this
assigned to one manager
available all the time
able to do anything anytime
cheaper resource
BUT only 60% productive
net result: was not able to
photocopy document for
manager so they had to
photocopy 30 copies…and this is
efficient and cost effective?
to this
reduce PAs by 40%
shared resource
100% productive
13
Companies who have introduced “slack”
Gore 10% dabble time
3M 15% think time
Google 20% anytime
Atlessian 20% Fedex days
14

when would an electric paintbrush be beneficial?
when would a manual paintbrush be beneficial?
manual versus automated
but using
both
requires
skill and
manual
intervention
#5: Tools are to “support” testing
15
Use tools wisely
!   tools can help increase productivity and staff morale
!   using tools requires new skills
!   tools can automate tedious/boring tasks
!   tools are less likely to make mistakes
!   some tools can find bugs quicker
!   tools can help with testing that cannot be achieved
manually
!   tools can decrease productivity and staff morale
!   tools do not make good testers
!   too much automation can lead to poor testers
!   a fool with a tool is still a fool
!   automating rubbish produces faster rubbish
!   tools are software and have bugs in them
16

#6: Take control of your day
!   learn to say “yes” to yourself occasionally
!   saying “yes” to others ultimately means saying “no” to you
!   don’t allow others people to control your day all the
time
!   this leads to frustration
!   learn to control unplanned activities
!   think before reacting
!   set yourself daily goals and achieve them
!   John, Susan, Nora and George
17
#7: Less scripted more exploratory testing
a problem with scripted testing
A
B
18

Dangers of detailed scripts
!   reasons to write detailed scripts
!   describes detailed tests for inexperienced testers
!   automation
!   provide an audit trail
!   risks
!   boring to write…boring to run
!   more time spent writing than running tests
!   maintenance nightmare
!   false confidence in the quality of the
software
!   your best testers will probably leave
suggestion: introduce exploratory testing sessions
and limit the number of detailed scripts produced
19
!   tests are prioritised against the
product risks
!   highest priority risks are mitigated first
!   testers will use appropriate tools and
techniques to mitigate the risks
efficiently
!   re-prioritisation of the tests will be
initiated by the re-prioritisation of the
risks
!   product risks will be monitored and
reported to stakeholders on a regular
basis
#8: Risk based testing is a must
20

Determining the “cut off”
Product Risk Probability Impact Exposure Risk Value
Poor response times High High 9 90
Security violation High High 9 90
Tax calculation incorrect Medium High 8 80
Holiday pay not included High Medium 7 70
Bank Transfer interface problems High Medium 7 70
User interface may not be liked Low High 6 30
New employees not paid Medium Medium 5 25
Unable to delete accounts Low Medium 3 3
Reports are unaligned Medium Low 2 2
provides a priority list and a focus for testing
21
#9: Personal daily retrospectives
!   take time to reflect
!   15 minutes at the end of the day
!   what did you do well
!   what could you improve on
!   what did people appreciate
!   take time to change
!   to be the best you can be
!   improvement starts with “I”
!   take time to learn
!   you are never to old !
true nobility is not being better than anyone else
but being better than you used to be
22

#10: When “speed up” really means “slow down”
!   this is only efficient
if the workload
remains constant
!   if priorities change
or workloads
increase this might cause
unpredicted consequences:
me
John
Sue
Amy
flow of work
e.g. John needs information from me urgently, therefore my
workload/priorities change to fulfill the expectations.
The net effect is that Sue becomes less busy, but she knows the
“hurry up” mantra which to her means “stay busy”, so is Sue is
becoming less busy, how does she show others that she is busy?
Sue slows down!! 23
Contents
Introduction
24

#11: Create an “elite” team of testers
!   a team that knows, understands and can apply…
!   systematic test design techniques
!   exploratory testing
!   bug hunting
!   flexibility
!   creativity
!   a team that will want to
!   collaborate
!   increase their knowledge and expertise
!   adapt to the circumstance
lead by example, coach, mentor and work with your testers
a team whose
“whole” is greater
than the sum of
its parts
25
#12: Use the Tester’s style analysis questionnaire
things to note
!   there is no right or wrong answer
!   try not to think too much
!   it helps us to assess our strengths
!   can be used with other psychometric questionnaires (e.g.
Belbin, Myers/Briggs, 16PF)
!   two parts
!   answer questionnaire and
!   plot the values on the graph
26

The questionnaire - how to complete (x axis)
Friendly
Approachable
Casual
Open
Unstructured
Social
Intuitive
Random
Warm
Perceptive
Formal
Retiring
Business Like
Guarded
Organised
Introvert
Logical
Focused
Cool
Insensitive
X - Axis
x
x
x
x
x
x
x
x
x
x
6 27
The questionnaire - how to complete (y axis)
To the point
Challenging
Quick
Insistent
Lively
Impatient
Adventurous
Confronting
Competitive
Strong Minded
Indirect
Accepting
Leisurely
Thoughtful
Relaxed
Patient
Cautious
Receptive
Co-operative
Analytical
Y - Axis
x
x
x
x
x
x
x
x
x
x
2 28

Plot your score on the grid
X
(6,2)
29
The Model
The
Pioneer
The
Facilitator
The
Pragmatist
The
Analyst
30

#13: Test the testers/tests
testing is quality control on development
BUT…how good are your tester/tests? Who is performing
quality control on the testers?
developers bug seed, plant 100 bugs and see how many are
found
provide an application for new testers with known bugs and see
how good they are at finding them
we deserve the best…mediocracy is not an option
31
Areas where this idea can be used
!   testing your regression tests
!   manual or automated
!   discover how effective they are
!   testing the outsourced teams
!   let’s wake up!
!   discover whether they are cost effective
!   testing new testers wanting to join the team
!   aim to recruit the best
!   quality gate into testing
32

#14: learn to be a weather person
weather people…
!   predict what is likely to happen using
trends and forecasting
!   understand statistics and probability
!   use historical data to substantiate
their synopsis
!   understand correlation between
systems
!   are becoming more accurate
!   don’t always get it right
!   not always liked…but it is not their
fault!
!   but we still use them to plan and
implement our activities
however…
33
Learn to predict the “projects” weather
!   to predict what is likely to happen
at an early stage
!   to use basic statistics and
probability
!   to gather and analyse historical
data
!   understand the correlation
between graphs
!   to become more accurate
BUT: you wont always get it right, you wont always
be liked, but it’s not your fault
0
50
100
150
200
250
1 2 3 4 5 6 7 8 9 10
0
10
20
30
40
50
1 2 3 4 5 6 7 8 9 10
34

#15: Tailor your test documentation…. and use
modern technology
don’t be boring
make it exclusive
challenge quality and
quantity of what is
produced
exploit modern
technology
35
Make it relevant and make it personal
!   providing information is a major aspect of testing and
must be…
!   accurate
!   timely
!   predictive
!   factual
!   objective
!   personal
!   relevant
CEO’s Dashboard
Project Manager’s Dashboard
One size doesn’t fit all 36

#16: Understand data correlation before reporting
…plus basic data significance
example 1: defects found
example 2: estimated versus example 3 effectiveness of testing
actual quality
good or
bad?
what about test effectiveness after 1
month of live
Project 1: DDP = 85%
Project 2 : DDP = 60%
conclusion: reward team 3, punish team 4?
why don’t we have a look at the numbers
good or
bad?
37
#17: Learn the tools of the trade
…and become skilled at what you do
!   test techniques
!   state transition, pairwise, domain analysis etc.
!   the system
!   application, architecture etc.
!   test tools
!   know what is available
!   practice makes perfect
!   test on a regular basis
38

#18: Stand out and be different
what have these stores
got in common?
what have these stores
got in common?
39
Create a unique selling point…
!   what makes you/your team
different from others…
!   that are cheaper?
!   that are faster?
!   your USP is so that you thrive…not just survive
!   example USPs
!   recognising and eliminating waste
!   being able to find those “nasty” bugs quickly
!   measuring and reporting the value of testing
!   having a positive attitude at work
!   collaborating always
!   constantly improving efficiency
!   estimating quality… 40

#19: Apprentice train
!   benefits for the apprentice
!   learn from others
!   hear (forget)
!   see (remember)
!   do (understand)
!   fast track
!   learning curve is shorter
!   benefits for the mentor
!   rewarding to see progress
!   improve ourselves
!   learn from the apprentice
!   invest time in people
!   greatest contribution we can make
41
Let’s reminisce…
!   think back to the first time you
were apprentice trained
!   cooking with your mum?
!   home improvement with your dad?
!   washing up (or nowadays - loading
the dishwasher!)
!   what type of experience was it?
!   good/bad/exciting/non-event
42

what they are
!   opportunity to learn and
increase skills
!   a safe environment to
make mistakes
!   a framework to prove
yourself worthy of the
current/next job role
!   an environment to
provide constructive
feedback
!   to be taken seriously
what they are not
!   errand running
!   a career trap "
!   a guarantee
!   an excuse to make lots
of mistakes
!   a training course/
certification scheme, all
theory and no practice
Apprenticeships: what they are/are not
43
Chief Tester one person is responsible for
mentoring all new staff
Hierarchical you mentor subordinates and
you are mentored by your
manager
Buddy pairing testers to mentor each
other, accountability is
encouraged
Coterie everyone mentors everyone!
Various models to consider
44

“An excellent tester this one is”
!   a “jedi” tester will…
!   let themselves be distracted by anomalies
and new ideas
!   encourage variability, avoid repeating the
same test twice
!   use their confusion as a resource
!   know their limits and get to know/use
subject matter experts
!   adapt testing to fit the context
!   develop resources and tools
!   train their minds to be curious, cautious
and critical
!   take notes and report results in a useful
and concise way
45
#20: Take time to sharpen your axe
!   don’t procrastinate
!   create tools and utilities to make testing more efficient
!   share your ideas with others
!   understand new methodologies and technologies and how
they will affect you (keeping one step ahead)
!   measure your own effectiveness and improve
!   create ‘me’ time (kick the leaves)
!   try something new
!   become a champion…
complacency is only
one step ahead of
apathy
make it your mission to become the
best at what you do
46

Contents
Introduction
47
#21: Choose your battles
!   you can’t fight every battle… so what is important?
!   career progression for testers
!   remuneration
!   providing slack
!   a good working environment
!   fairness
!   more resources
!   advice given is used
!   purchasing test tools
!   efficiency
!   a good test environment
!   test estimates being agreed
48

#22: Don’t worry about test estimation…too much
!   understanding the problem with test estimation
! recognising that we can test in less
!   provide options using a variety of techniques
Testing
4 weeks
Testing Testing Testing
2 weeks1 week
49
#23: Estimating quality is a must
fact: estimation is based on 2 aspects
effort + people = schedule
half truth for testing:
effort + people + quality = schedule
we must also estimate the number and severity
of the bugs we are going to find to make our
estimates meaningful (a direct correlation)
WARNING: be careful of Brook s law – 9 women cannot have a baby in 1
month! Some activities cannot be accelerated by adding more resource
50

Estimate, monitor and report quality
how to estimate quality
!   guess
!   use historical data (previous projects, defect density)
!   use models
what to monitor and report
!   actual versus expected
!   trend analysis and forecasting
!   correlation between graphs
0
10
20
30
40
50
1 2 3 4 5 6 7 8 9 10
0
50
100
150
200
250
1 2 3 4 5 6 7 8 9 10
Cycle
#
Actual
bugs
Masked
rate
Observed
bugs
Fix
Rate
Fixed
bugs
Insert
rate
Inserted
bugs
Remaining
bugs
1 10 30% 7 80% 6 20% 1 5
2 5 30% 4 80% 3 20% 1 3
3 3 0% 3 100% 3 0% 0 0
4 0 full regression test
51
#24: Remuneration and career path
!   remuneration must represent their skill
! recognising that testing is a skill and profession
!   fairness throughout the organisation
!   define a career structure for testers
!   where management is not the only option
!   encourages testers to aspire in their career
!   essential to recruit and retain good quality testers
functional career model
management technical application
functional non-functionaltest consultant
test manager
team leaders
senior technician
DBA/env technician
test automator
designer designer
executorexecutor
52

#25: Demonstrate the ROI for testing
requirements team…
has an idea and forms
requirements
design team…
takes (vague) requirements and
designs the product
development team…
uses the design to develop the
product
test team…?
if testing doesn’t provide a good
ROI…
then it is not a good investment
YES NO
ROI
!   all teams must provide the company with a good ROI,
the problem is perception…
ROI must be
measured in
quantifiable
terms
(CSPB,
DDP, DFP,
PRDFP etc.)
53
#26: Provide management with feedback on
decisions that they make
!   example: 4 weeks of testing:
!   new data obtained:
0
20
40
60
80
100
120
0 1 2 3 4
Number of bugs
Actual
Management want to implement
the software, but you suggest a
further week of exploratory testing
and a final week of regression
testing due to the high number of
bugs that have been found
0
20
40
60
80
100
120
140
0 1 2 3 4 5 6
Number of bugs
Actual
25 new bugs found and fixed…full
regression pack run
54

Using data to report back
#  cost of bugs to find and fix in test = 1000€
#  cost of bugs to find and fix in live = 3000€
#  1st month of live operation customers find 10 bugs
!   if the old date (4 weeks) was used
!   assume 35 bugs found by customer
!   DDP = 100/135 = 74%
!   cost of fixing = 35 * 3000€ = 105,000€
!   if the new date (6 weeks) was used
!   DDP = 125/135 = 93%
!   cost of fixing = 10 * 3000€ = 30,000€
+ 25 * 1000€ = 25,000€
= 55,000€
cost saving
50,000€
DDP increase 19%
Therefore a good
decision by
management !
55
#27: Make your test environment the strongest
link
!   often the test environment becomes the
weakest link
!   concentrate too much on test cases and
test scripts
!   reliant on other people
!   often difficult to set up well
!   we believe the environment will “just” happen
!   make it the strongest link
!   constantly think about your environment
!   make it a priority prior to test execution
!   learn to recognise environmental issues
and raise these with your managers
56

Bad environments
if your environment is wrong, out-of-date, volatile, not
representative, or messy then testing cannot succeed
ignore your test environment and the bugs will breed
57
#28: Is there any value in raising bug reports
!   fact: there is very little value in raising bug reports:
!   during development (component/integration tests)
!   if you are only concerned in fixing bugs
in an automated environment
!   if you are sitting with the developer and they
can fix the bug immediately
!   if you are not concerned with learning lessons
or improving the process
!   if there is a negative ROI for the time it
takes to raise a bug log
!   BUT…it depends on what we mean by “value”
…relative worth, merit
or importance
58

your company
The value of our bug logs
negative ROI
•  information not used
•  no improvements
•  takes more time to raise than fix
positive ROI
•  help with progress
•  study bugs for defect prevention
•  testers improve testing skills
•  reduce wastage
•  data is used wisely 59
#29: Can we “go-live”? Answer is always “yes”
…however
!   Test Manager/Tester’s responsibility
!   to provide information to stakeholders to
make informed decisions
!   to offer opinions about the quality of the
software that may/may not be agreed with
!   to be the quality conscience of the
project
!   to highlight warning signs clearly
yes no
60

#30: Believe in yourself…and your abilities
!   low self-esteem is one of the main
contributors to lacking drive and enthusiasm:
!   reignite your passion
!   fan the flame before it goes out
!   see change as an opportunity rather than a threat
!   create good habits
!   66 days to create a good habit
!   but only 25 days to create a bad habit
practice looking in the mirror
each morning and saying…
“I am great!”
no one on their
deathbed said
“I wish I had
spent more
hours at the
office
61
Summary
!   busyness can be both good
and bad
!   being busy bad requires
action
!   introduce various ways of
doing more with less
!   build an elite test team that
you can trust and rely on
!   there is a need to decide
whether certain aspects are
worth being concerned about
!   learn to say “no” to others
more so that you can say
“yes” to yourself
62

© Lloyd Roden Testers Style Analysis Page 1 of 1
X-Axis Y-Axis
Friendly __ Formal __ To the Point __ Indirect __
Approachable __ Retiring __ Challenging __ Accepting __
Casual __ Business Like __ Quick __ Leisurely __
Open __ Guarded __ Insistent __ Thoughtful __
Unstructured __ Organised __ Lively __ Relaxed __
Sociable __ Introvert __ Impatient __ Patient __
Intuitive __ Logical __ Adventurous __ Cautious __
Random __ Focused __ Confronting __ Receptive __
Warm __ Cool __ Competitive __ Co-operative __
Perceptive __ Insensitive __ Strong Minded __ Analytical __
__ __
Grid 10
9
8
7
6
0 1 2 3 4 5 6 7 8 9 10
4
3
2
1
0
Based upon “The Communication Styles Analysis”
Tester’s Style Analysis Questionnaire
Name

Test Management for Busy People

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (14)

Similar to Test Management for Busy People

Similar to Test Management for Busy People (20)

More from TechWell

More from TechWell (20)

Recently uploaded

Recently uploaded (20)

Test Management for Busy People