As the testing discipline continues to evolve—and the demands on testers increase—we need to look for new paradigms to guide our work. Thomas McCoy believes the profession of journalism has much to offer in helping us ask the right kinds of questions, be heard, and deliver bad news effectively. In many ways, our profession has ideals similar to those of journalism: our first obligation should be to the truth, we must maintain independence (even when embedded in agile teams), and our mission includes the protection of society. While not all techniques (such as hacking into the voicemail accounts of developers) would be appropriate, established values like impact, proximity, and timeliness can help guide our communication and make it more succinct and effective. Join Thomas to learn about employing investigative techniques, developing sources, capturing an audience, and writing compelling stories, using proven methods from the world’s second-oldest profession.

1. KT1
Keynote
5/2/2013 8:30:00 AM
Asking the Right Questions? What
Journalism Can Teach Testers
Presented by:
Thomas McCoy
Australian Department of Families, Housing, Community Services and
Indigenous Affairs
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Thomas McCoy
Australian software testing professional, teacher, and journalist Thomas McCoy has worked in the IT
industry for more than two decades. Much of this time was spent as a software developer and IT manager
with Australian government agencies. Wanting to make a greater contribution to software quality, Thomas
re-oriented his career into the emerging discipline of software testing and has been an enthusiastic
promoter of the profession ever since. At conferences in Australia, New Zealand, Canada, and the US,
Thomas is a popular keynote speaker, who has received several best presentation awards. You can reach
Thomas at tom@thomasmccoy.com.au.

3. STAR East 2013
2 May 2013
DISCLAIMER
ASKING THE RIGHT QUESTIONS
The views expressed in this
presentation are not necessarily
those of the Department of
Families, Housing, Community
Services and Indigenous Affairs,
the Australian Government, or
even the presenter. ☺
What Journalism Can Teach Testers
Thomas McCoy
BJourn, BSc, DipEd, MInfTech
Australian Department of Families, Housing,
Community Services and Indigenous Affairs
2
STAR EAST 2013
Australia
I’m from Australia, which is around 9,500 miles
away and now has a population of 23 million (just
a few million more than the population of Florida).
STAR EAST 2013
Thomas McCoy
Canberra
I’m based in Canberra, Australia’s capital city, which is in the southeast
of Australia between Sydney and Melbourne. It was built in 1913 and
designed by Chicago architect Walter Burley Griffin (who won an
international design competition). Its population is around 370,000.
3
STAR EAST 2013
4
1

4. STAR East 2013
2 May 2013
FaHCSIA
Background
Department of Families,
Housing, Community Services
and Indigenous Affairs
FaHCSIA, the government department I work for, is responsible for
social security in Australia, with payments and support for groups
including pensioners, the unemployed, people with disabilities, and our
Indigenous Australians. It manages about one fifth of the federal budget.
We have testing staff working within both the agile (embedded in
development teams for the smaller systems) and waterfall (working as a
large testing group for our major application) contexts.
STAR EAST 2013
5
Thomas McCoy
6
STAR EAST 2013
JOURNALISM SKILLS
WHAT DOES SOFTWARE TESTING
HAVE TO DO WITH JOURNALISM?
STAR EAST 2013
Started in journalism and media, then moved into IT, first into
programming and finally into software testing.
Have become increasingly aware of the importance of communication,
both in terms of project team communication and computer / end user
communication.
This session has been inspired by the parallels I’ve seen between
journalism and software testing.
• A questioning, curious,
and inquisitive mind
• Communication skills
• Writing ability
• News sense
• Listening skills
• General knowledge
• Empathy, patience and
understanding
• Sense of public
responsibility
7
STAR EAST 2013
•
•
•
•
•
•
•
•
•
•
Language skills
Interviewing skills
Ability to meet deadlines
Knowledge of computers
and technology
Analysis skills
Resourcefulness
Persistence
Truthfulness
Accuracy
Integrity
Notice big overlap with testing skills.
8
2

5. STAR East 2013
2 May 2013
JOURNALISM PRINCIPLES
THE FOURTH ESTATE
• Its first obligation is to the truth
Journalism has been called “the Fourth Estate”. Comes from British
Parliament which has the monarch (1st estate), House of Lords,
containing nobility and clergy (2nd estate), and House of Commons
containing the other elected members (3rd estate).
“In the reporters gallery yonder there sits a fourth estate more important
by far than they all,” said 18th century British politician Edmund Burke.
Fourth Estate rests on idea that it’s the media's function to act as a
guardian of the public interest and as a watchdog on those in power.
Are there parallels between this power structure and IT projects?
• Its practitioners must maintain independence from
those they cover
• It must serve as an independent monitor of power
• Its practitioners must be allowed to exercise their
personal conscience
These are principles in an “ideal world”, but again there is strong
overlap with our testing career.
9
STAR EAST 2013
STAR EAST 2013
10
THE FOURTH ESTATE
Estate Journalism
1st King
2nd House of
Lords
(nobility)
3rd House of
Commons
4th Journalists
STAR EAST 2013
Thomas McCoy
IT
Business areas?
Project Managers?
TESTING: JOB OR VOCATION?
The enormous impact technology has on society, and our
position in reviewing quality (too often at the end of the
process), has led some to ask whether software testing is
more of a vocation than a job. Vocations are sometimes
differentiated from jobs because of their strong sense of
social responsibility (e.g. teachers, nurses, clergy,
journalists, etc.).
Developers?
Testers?
11
STAR EAST 2013
12
3

6. STAR East 2013
2 May 2013
TESTING: JOB OR VOCATION?
• Sacrifice
• Burnout
• Long hours
• Commitment
• Thanklessness?
• Social conscience
• Self-development
• Fighting “the good fight”
Impact of a recent technology issue.
Apart from that, we’ve had people killed, fried by radiation machines,
not be able to use the brakes on their car, etc.
Do you identify with any of
these aspects in your role?
13
STAR EAST 2013
14
STAR EAST 2013
SHARING NEWS OF OUR SUCCESS
Because of the nature of our role it’s sometimes
challenging to publicise our success without
upsetting people.
We might want to do something like this …
Then there are the less publicised daily
annoyances caused by deficient technology
(which can’t be good for people’s health).
STAR EAST 2013
Thomas McCoy
15
STAR EAST 2013
16
4

7. STAR East 2013
2 May 2013
More palatable news
Testing News
… but this could cause offence, so we publicise in more benign
ways, as in my newsletter story about our new test lab …
STAR EAST 2013
17
STAR EAST 2013
NEWS VALUES
DELIVERING BAD NEWS 1
Three useful news values are:
• For us, this is a common
and difficult
communication task
• Nobody likes to hear bad
news
• Nobody ever says “thank
you” for receiving bad
news
• Impact: angle your message for maximum
impact in relation to your target audience
• Proximity: if something is close to your
target audience or their aspirations it will
have greater effect
• Timeliness: don’t delay messages
unnecessarily
STAR EAST 2013
Thomas McCoy
18
19
STAR EAST 2013
20
5

8. STAR East 2013
2 May 2013
DELIVERING BAD NEWS 2
The original Larson dog cartoon …
• Avoid it being a surprise (perhaps it
can first be passed through informal
communication channels so people
can adjust to the idea)
Sometimes “selective
listening” is a
problem when
receiving bad news,
as illustrated by this
re-tooling of a classic
Larson cartoon.
• Pick a time when the recipient
is relaxed (sometimes Friday
afternoons work well)
• If possible, propose some
alternatives when delivering
the news
STAR EAST 2013
21
STAR EAST 2013
22
REHEARSING BAD NEWS DELIVERY
WITH AUSTRALIAN NATIVE
ANIMAL PERSONAS
… can be adapted to our situation
Sometimes, when having to deliver bad news, it can help to think about
the “persona” you’ll adopt when delivering the message.
In Australia, we have native animals with unique characteristics that can
be used to rehearse the bad news delivery (we even have hand
puppets). ☺
Sometimes we might have to go in aggressively, while at other times it
might be with an attitude of resignation (because we choose not to fight
this particular battle), or even with a humorous approach.
STAR EAST 2013
Thomas McCoy
23
STAR EAST 2013
24
6

9. STAR East 2013
2 May 2013
TASMANIAN DEVIL
KANGAROO
Aggressive and energetic
Cheerful and easy going
“Well what do you expect when we weren’t given the
resources we needed!”
“OK, we’ll see what can be done”
25
STAR EAST 2013
KOALA BEAR
KOOKABURRA
Always ready for a laugh
Relaxed and nonchalant
“Ha, ha, you’re joking, you know that was always going
to be impossible.”
“Yeah, right, whatever.”
STAR EAST 2013
Thomas McCoy
26
STAR EAST 2013
27
STAR EAST 2013
28
7

10. STAR East 2013
2 May 2013
“I keep six honest serving-men
(They taught me all I knew);
Their names are What and Why
and When and How and Where
and Who”
A KEY PART OF JOURNALISM:
THE 5WS & 1H
Think about the 5 Ws & 1H in
anything you write.
For example, in a defect report, as
shown on the next slide …
29
STAR EAST 2013
5WS & 1H
• Who:
Rudyard Kipling from “The Elephant’s Child”
Fred Nurk
30
STAR EAST 2013
Defect report example. Note that you
won’t necessarily need all of the 5Ws &
1H in everything you write so consider
which ones to leave out.
• What:
Discovered system crashes if
negative quantities entered
• When: Last night
WRITING EFFECTIVELY
• Where: Test lab
• Why:
• How:
STAR EAST 2013
Thomas McCoy
Inadequate input field
checking?
Running automated test scripts
31
STAR EAST 2013
32
8

11. STAR East 2013
2 May 2013
GEORGE ORWELL’S ADVICE
“Always grab the reader by the
throat in the first paragraph, sink
your thumbs into his wind pipe
in the second, and hold him
against the wall until the last
word.”
1. Never use a long word when
a short one will do.
2. If it is possible to cut out a
word, always cut it out.
3. Never use the passive voice
when you can use the active.
This might be a bit ambitious for the
average test plan but the reason so few
documents are read is because they’re
so totally unengaging.
4. Break any of these rules
sooner than saying anything
outright barbarous.
If you remember nothing
else from this session,
remember these rules.
You will sometimes need to
violate #3 and use passive
voice to avoid blaming
somebody.
Active Voice: Fred Nurk introduced a defect (blames Fred)
Passive Voice: A defect was introduced (blames nobody)
33
STAR EAST 2013
THE INVERTED PYRAMID
Top
Content
Position
STAR EAST 2013
34
AN EFFECTIVE LAYOUT
People are so
deluged with
textual information
that they may not
read more than the
first few lines.
So use the
inverted pyramid to
present your
information.
Bottom
• First paragraph: summarise
the situation, covering the
most important aspects of
who, what, when, where, why
and how (this may be the
only paragraph the person
reads)
When communicating about
issues needing attention you may
want to have a “call to action” at
the end rather (than entirely
following the inverted pyramid
• Next: provide more details
style of having less and less
• Finally: use a “call to action” important information going down
the page).
to have the issue resolved
Importance (signified by width)
STAR EAST 2013
Thomas McCoy
35
STAR EAST 2013
36
9

12. STAR East 2013
2 May 2013
WRITING 1
EXAMPLE
• Need a strong headline that will push
relevant buttons in people. The
“headline” might be, for example, the:
On the registration date screen February 29 is being
accepted for all years, not just leap years.
[Summary, covers “where” and “what”]
The problem may be due to the new external date
routine, which was recently implemented on this screen.
(The other screens still work fine.)
[More info]
This could damage customer relations and create
adverse publicity for the organisation and needs urgent
attention.
[Call to action]
STAR EAST 2013
37
WRITING 2
• The headline may be all they read
• Think about the “angle” of your
message (what are you trying to
achieve)?
38
STAR EAST 2013
• Clichés should be avoided like
the plague ☺
• Check spelling and grammar
• Secondary information later
• A reading level of Grade 8 is
recommended (can use FleshKincaid readability checker in
Word)
• Use Plain English (avoid jargon)
• Use mix of long and short sentences
• Keep paragraphs to 3 sentences on average
(large slabs of writing intimidate readers)
Thomas McCoy
subject line of your e-mail message
WRITING 3
• Start with most important point, which
should answer some of 5Ws & 1H
STAR EAST 2013
summary of your defect report; or the
The “buttons”
that will work
best depend
on our target
audience and
this is where
research and
prior planning
are
important.
Newspapers are typically
written at Grade 8 level.
39
STAR EAST 2013
40
10

13. STAR East 2013
2 May 2013
Settings …
USING THE READABILITY CHECKER
These are some of the
grammar checks the system
can do under Grammar &
Style but be aware that there
are many it can’t do well
(because writing is a human
activity) so don’t rely on it too
much and don’t follow it
blindly (sometimes it gets
confused).
Passive sentences is worth
using, though.
In Word 2010 do: File Tab, Options, Proofing…
… and make sure you tick
the Show readability
statistics checkbox. Also
think about what you want to
use for Writing Style (e.g. just
Grammar or Grammar &
Style) and check further
options with the Settings …
button.
STAR EAST 2013
41
42
STAR EAST 2013
READABILITY STATISTICS
These are only a guide and have some
quirky aspects.
Flesh-Kincaid is based on average number of
syllables per word and average number of
words per sentence.
This results in the kids’ classic “Green Eggs
and Ham” achieving a grade level of minus
1.3, which would suggest it could be
understood by kids before they start school
and have learnt to read.
We tried an interesting experiment …
STAR EAST 2013
Thomas McCoy
We ran the Encyclopaedia
Britannica article on Nuclear
Physics through FleshKincaid and it came back
with a Grade level of 15.5
(i.e. written at university
level) …
Nuclear Physics: 15.5 grade level
43
STAR EAST 2013
44
11

14. STAR East 2013
2 May 2013
… and then we pasted the
entire text of Green Eggs
and Ham at the end of the
article, and when we ran
readability again it brought
the Grade level down to 2.4!
Does this mean a kid halfway
through Grade 2 could now
understand nuclear physics?
Nuclear Physics & Green Eggs: 2.4 grade level
OTHER WAYS OF PRESENTING
INFORMATION
After this somebody helpfully suggested that we should paste Green
Eggs and Ham at the end of all our unreadable documents (using a 1point font and white text for minimum visibility) to bring our readability
scores down, but that’s another story …
STAR EAST 2013
45
Wordle Word Cloud of Seven Testing Principles
46
STAR EAST 2013
Graphics can be a useful way to present information (and they have
enormous credibility).
Here is a graph of a project where the overtime hours are doubling
every month, from 200 to 400 to 800 to 1,600.
But would reporting this trend cause unnecessary alarm?
Overtime Hours
2000
1500
1000
500
0
A word cloud is an innovative way of illustrating key
concepts in a document. The more often a term occurs the
larger it is.
This is from the 7 principles of testing in the ISTQB
Foundation Syllabus and was generated by Wordle
(http://www.wordle.net/).
STAR EAST 2013
Thomas McCoy
47
Jan
Feb
Mar
Apr
True Representation
STAR EAST 2013
48
12

15. STAR East 2013
2 May 2013
Not if we switch to a logarithmic scale, which ensures that the world is
still a happy place (and the slippery slope suddenly looks so much less
slippery.)
Or how about choosing an completely irrelevant chart type (Excel offers
so many, and I’ve often seen them misused).
Overtime Hours
Overtime Hours
10000
Jan, 200
1000
Apr, 1600
100
Feb, 400
Mar, 800
10
1
Jan
Feb
Mar
Apr
Logarithmic Scale
Irrelevant Pie Chart
49
STAR EAST 2013
Or a completely meaningless chart (but you should probably remove the
numbers from the cones).
By the way, I’m not suggesting that we should deceive people. I’m just
demonstrating how easily information can be manipulated even with socalled “scientific” approaches.
Overtime Hours
Apr,
1600
Mar, 800
100%
And many years ago (before everybody could do this on their iPhone) we
created a rap-song video to expose the organisation to automation testing.
It was done with a low-end camera and heaps of staff support.
Some of the claims were (admittedly) exaggerated but this clip is still
available on Sticky Minds (www.stickyminds.com/) just search for
“Automation Rap”.
Think about whether using video might be useful in your communication.
Feb
Jan, 200
50
Mar
Feb, 400
STAR EAST 2013
Jan
50%
Apr
0%
Meaningless 100% Stacked Horizontal Cone Chart
STAR EAST 2013
Thomas McCoy
51
STAR EAST 2013
52
13

16. STAR East 2013
2 May 2013
VOICE 1
VOICE 2
The way we use our voice is critical to getting our
message across effectively.
And vocal variation can help keep people awake.
There are 4 important aspects:
2.
1. Pitch
3.
Quick pace can also convey urgency. Or can
be used to gloss over boring details. Use
slow pace for important information
Pause
An appropriate pause (count silently to 5) can
help focus attention before a key word
We need to raise our
pitch to instil a sense
of urgency and panic
(or lower it to restore
calm)
STAR EAST 2013
Pace
4.
Projection
Need to project voice to ensure it is heard
53
54
STAR EAST 2013
VOICE 3
• Invest some time in voice work
• Use a recorder to play yourself back (and don’t be
shocked the first time you hear yourself – that
happens to all of us)
• Volunteer for presentations (even in non-work
environments such as sporting clubs, kids school,
etc.)
ACQUIRING INFORMATION
• Think about joining a group like Toastmasters
(http://www.toastmasters.org/) which is dedicated
to public speaking
STAR EAST 2013
Thomas McCoy
55
STAR EAST 2013
56
14

17. STAR East 2013
2 May 2013
INVESTIGATIVE JOURNALISM
SOURCES
• Find several sources (don’t burn them)
• Can increase our
understanding of “what’s
really going on”
• Speak to as many relevant people as
possible
• Can serve as early
warning of risks
• Verify information through reliable, factual
(preferably documented) evidence
• Respect “off the record”
comments
• Don’t take everything at face value
(because people may be running their
own agendas)
You can independently verify off the record comments but you can’t
credit them to the source, or even mention the source.
57
STAR EAST 2013
DEVELOPING SOURCES 2
DEVELOPING SOURCES 1
Finance
Management
And there are many ways of reaching out beyond your immediate work
environment. I can’t sing but still joined the office choir and was able to
hear how the IT group appeared to people on the outside.
Business
• Work social club
Tentacled
Tester
• Sporting teams
• Lunches
Technical
Administrative
• Attending presentations
and events
Marketing
To develop sources of the greatest value it can be useful to put your
tentacles out as widely as possible (rather than remaining cloistered
within IT).
STAR EAST 2013
Thomas McCoy
58
STAR EAST 2013
• Joining the office choir
59
STAR EAST 2013
Departmental Choir
60
15

18. STAR East 2013
2 May 2013
CRITERIA FOR EFFECTIVE SOURCES
INVESTIGATIVE TECHNIQUES
• Past suitability
• Authoritativeness
• Infiltration
• Reliability
• Articulateness
• Impersonation
• Trustworthiness
• Discretion
• Secret filming and/or recording
Sometimes it may be necessary to “infiltrate” meetings you haven’t been
included in.
Or to impersonate somebody (e.g. via a phone call) to obtain information.
(You should not impersonate an existing person, but you might
impersonate a generic role e.g. “I’m ringing from tech support”).
Be very careful with secret filming or recording (more on this later).
61
STAR EAST 2013
MAPPING THE ORGANISATION 1
MAPPING THE ORGANISATION 2
is cousin of
Sheila
Chaz
Bluey
Kylie
Ernie
STAR EAST 2013
Thomas McCoy
Molly
Tezz
Nicole
Sidney
Snowy
62
STAR EAST 2013
Magda
Talisha
Brian
Bazza
In understanding the operation of an
organisation it can be useful to map the
informal relationships onto the formal org
chart as shown in the next slide.
Sheila
Chaz
Bluey
Hugo
Kylie
Kezza
Ernie
Tezz
Molly
Nicole
Sidney
Snowy
disagrees with
got promoted over
teaches trombone to son of
63
STAR EAST 2013
plays rugby
with
Much more informative.
Magda
Talisha
Brian
Bazza
Hugo
Kezza
RELATIONSHIPS
64
16

19. STAR East 2013
2 May 2013
ALWAYS DO USABILITY TESTING
• It is cheap and can be run on a laptop
• Testing even one user is better than no users
• Have them perform some tasks and ask them
to “think out loud” as they complete them
OVERT RECORDING:
USABILITY TESTING
• Software like TechSmith’s Morae product will
capture the screen, the user and the audio in
real time
• The “video evidence” can be compelling
when shown to developers
STAR EAST 2013
65
66
STAR EAST 2013
Fake SAVE button
We watched a usability clip where the “save” button had been removed
from a new version of the system (which now did automatic saves).
This traumatised a user of the old system who could no longer find a
“save” button and did now know whether her work would be lost when she
exited. She became “stuck” and would have rung the help desk for advice.
STAR EAST 2013
Thomas McCoy
67
SAVE
My recommended, innovative, low-cost solution, of adding a fake “save”
button to the application so users could still click it and reduce their
anxiety, was rejected.
STAR EAST 2013
68
17

20. STAR East 2013
2 May 2013
COVERT RECORDING 1
• With the capability of today’s mobile devices
it is trivial to record audio and/or video covertly
(you no longer need to be a secret agent)
COVERT RECORDING
• Many technical meetings generate so much information so
quickly that recording is the only way of obtaining accurate
notes
• Ideally you would ask participants if you can record
(however, this makes some people so uncomfortable that
sometimes not asking may be more humane) but check with
local laws and company policies
STAR EAST 2013
69
STAR EAST 2013
70
COVERT RECORDING 2
• In the ACT (where I come from) we have a funny law where
you’re allowed to covertly record any conversation that you
were part of as long as at least one person agrees (and that
can be you) ☺
• Note that covert recordings should never be used against
somebody (once relevant notes have been made they
should be destroyed)
OUR TESTING RESPONSIBILITY
(AND BAD ERROR MESSAGES)
• Here is an example where somebody inappropriately used
covert recording to request a testing extension while the
boss was going about his business (with the aim of later
playing it back to the boss as proof of the extension):
www.thomas.mccoy.name/DontDoThis.mp3
STAR EAST 2013
Thomas McCoy
71
STAR EAST 2013
72
18

21. STAR East 2013
2 May 2013
Client Registration
Family Name:
Nurk
Given Name:
Fred
Date of Birth:
01/01/2015
Date of Death:
Gender:
M
In testing a client
registration
application I entered
a future Date of Birth
to check for the error
condition (note that
Date of Death was
left blank).
As expected, an error
message was
displayed and I could
have marked the test
as “passed”.
However, I was horrified by the meaningless, technobabble “error
message” that was clearly written by tech-heads with zero communication
skills. As testers, when we see something like this, I feel we owe it to the
end user to report it, even if it isn’t on the official test plan.
System Date/Time: 22/4/2013 2:25PM
STAR EAST 2013
73
STAR EAST 2013
74
[1] Wrong field values or value types detected in Date of Birth.
[1] Wrong field values or value types detected in Date of Birth.
Please re-enter your field values. If you need additional
assistance, please refer to the documentation.(SBL-UIF-00299)
Please re-enter your field values. If you need additional
assistance, please refer to the documentation.(SBL-UIF-00299)
[2] The value '01/01/2015' for field 'Date of Birth' is required to be
'<= Today() AND <= [Date of Death]'.
[2] The value '01/01/2015' for field 'Date of Birth' is required to be
'<= Today() AND <= [Date of Death]'.
Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth
Date] <= '.(SBL-DAT-00521)
Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth
Date] <= '.(SBL-DAT-00521)
Looking at this message reveals many common problems:
1. Should not use terms like “wrong” and “detected”, they are negative
2. Should not use “field values or value types”, this is technobabble
3. The cryptic error code is meaningless
4. Why are there three versions of the error message?
5. Why is a function call i.e. Today() in an error message?
6. Why are mathematical comparison operators and Boolean logic used?
7. Why is Date of Birth written in 3 different ways: Date of Birth, ‘Date of
Birth’ and [Birth Date]?
8. Why are square brackets used around variable names?
9. Why do dates sometimes have single quotes and sometimes not?
STAR EAST 2013
Thomas McCoy
75
STAR EAST 2013
76
19

22. STAR East 2013
2 May 2013
FIREFOX ERROR MESSAGE
[1] Wrong field values or value types detected in Date of Birth.
Please re-enter your field values. If you need additional
assistance, please refer to the documentation.(SBL-UIF-00299)
[2] The value '01/01/2015' for field 'Date of Birth' is required to be
'<= Today() AND <= [Date of Death]'.
Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth
Date] <= '.(SBL-DAT-00521)
10. Why does the last comparison end with <= '. (this is because Date of
Death was blank, so why are we comparing with blank)?
11. Why is there a second meaningless error code?
The entire error message, of more than 50 words and meaningless codes
and symbols, could be replaced with:
Date of Birth may not be in the future. Please re-enter.
77
STAR EAST 2013
Describe Problem (clearly and succinctly)
WHAT
Cause (in as much detail as necessary)
WHY
3.
Solution (steps to fix problem)
HOW
STAR EAST 2013
Thomas McCoy
78
Should be:
understandable
relevant
actionable
user-centred
brief
clear
specific
courteous
rare
• Good error messages should answer some
key 5W&1H questions:
2.
STAR EAST 2013
ERROR MESSAGES 2
ERROR MESSAGES 1
1.
Firefox has moved towards some soft and cuddly error messages where
the computer takes the blame and becomes humanised.
This does not add value and quickly becomes tedious.
Better to just write meaningful and useful messages in plain English.
79
STAR EAST 2013
80
20

23. STAR East 2013
2 May 2013
PROMPTING ACTION
THE TESTING TIMES
• If you are experiencing difficulty in conveying the
importance of having an issue addressed, mocking up
adverse media coverage can help
Cryptic error
message
causes user chaos
• On the next slide I have created a fictitious story about
the error message and can then innocently ask “but
what would happen if something like this came out in
the paper”?
• Having it looking real, rather than being hypothetical,
may be more persuasive
• “Stock prices plummet” might be another example
The cryptic error message
STAR EAST 2013
May 2013
www.testingtimes.com.au
81
Help Desk phones
were inundated as
users struggled
with a cryptic error
message that
would require a
computer science
degree to decipher.
The message, which
included comparison
operators, Boolean
logic, function calls,
field names and
variables, was
described by one user
as “complete
gobbledegook”.
The error condition
prompting the
message was triggered
when a user typed a
date of birth for a client
that was in the future.
While this is clearly
invalid, the message
was of little use in
MISTAKES
• Since we critique the work of others it is reasonable
that we are held to high standards
• In the same was as newspapers sometimes issue
corrections, we should promptly admit mistakes
CHECKING ACCURACY (OURS)
• An excellent way of checking our written material is
to hear it read out loud
• The Office 2010 products (Word, Outlook, Excel)
have an inbuilt reader that can read any text
highlighted by the user
STAR EAST 2013
Thomas McCoy
83
STAR EAST 2013
84
21

24. STAR East 2013
2 May 2013
THE OFFICE 2010 INBUILT READER
• To use the reader (called “Speak”) you need to add
the Speak icon to your Quick Access Toolbar
• You can then select any text and click the icon to
have it read out
QUESTIONING
• You can use it to read back e-mails before you
press “Send” or to read out long, boring
documents when your eyes are fading
• A video to show you how to do this is available at:
www.thomas.mccoy.name/SpeakFeature.htm
STAR EAST 2013
85
STAR EAST 2013
QUESTIONING
EFFECTIVE QUESTIONING 1
• At the heart of journalism
• Do research, plan questions and strategy and
write them down
• Requires substantial research and
preparation
• Be clear on the objective
of every question
• Journalist often knows answer in advance
• Think about likely responses
and your follow-up question
• Questions can be used to convey an
implicit message in a less threatening way
than statements
STAR EAST 2013
Thomas McCoy
86
• Make person comfortable with informal
conversation to start with
87
STAR EAST 2013
88
22

25. STAR East 2013
2 May 2013
EFFECTIVE QUESTIONING 2
EFFECTIVE QUESTIONING
• Ask mixture of open and closed
questions
• Record answers (shows interest)
• Repeat answer back for confirmation
• Take special note of anything that comes “after the
interview”
• Use leading questions, if necessary e.g. “When this
error message frustrates the user, who are they
supposed to call”
• Do not make it feel like an
interrogation
• Adopt a non-threatening and
inquisitive manner
• Smile and nod, endure silences,
don’t interrupt, be polite,
stay neutral
STAR EAST 2013
CHOOSING THE MEDIUM
• Face to face
• E-mail
• Phone call
• Post-it note
89
Your medium will
depend on the
message. For an
informal message a
Post-it note may be
fine. For something
more serious you may
want to do face to
face with e-mail
follow-up.
STAR EAST 2013
CHOOSING THE CONTEXT
• One-on-one in private
• One-on-one in open plan
• Daily stand up
Choose your context
carefully and keep
participants to an
absolute minimum.
• Team meeting
• SMS
• Project meeting
• Social media
• Executive meeting
STAR EAST 2013
Thomas McCoy
90
91
STAR EAST 2013
92
23

26. STAR East 2013
2 May 2013
WHAT KIND OF QUESTIONS?
EMBEDDED MESSAGE EXAMPLES
• This depends on who you are talking to,
what you are hoping to achieve and
what motivates them, for example:
Project Manager
Project Manager – not meeting deadlines
Business Analyst - incomplete or defective
requirements
Developer - coding errors
STAR EAST 2013
93
• “How much overtime are we
able to accommodate under
the current budget?”
• Embedded message: There is
no way we can finish testing
on time with the existing
resources.
STAR EAST 2013
EMBEDDED MESSAGE EXAMPLES
EMBEDDED MESSAGE EXAMPLES
Business Analyst
Developer
• “I’m just wondering what
happens on the registration
screen if the user has
registered more than once?”
• “What happens if the
user puts in the year
with just the last two
digits?”
• Embedded message: These
requirements have holes.
• Embedded message:
There is a bug in the
date routine.
STAR EAST 2013
Thomas McCoy
94
95
STAR EAST 2013
96
24

27. STAR East 2013
2 May 2013
EMBEDDED JOURNALISM …
DEVELOPERS AND TESTERS
HOW CLOSE IS TOO CLOSE?
… OR IN-BEDDED?
STAR EAST 2013
97
STAR EAST 2013
98
AGILE TEAMS: TOO CLOSE?
• Gordon Dillow was one of around 800 journalists
embedded with soldiers in the Iraq war
“I found myself falling in love with my subject.
I fell in love with ‘my’ Marines … the point
wasn’t that I wasn’t reporting the truth, the
point was that I was reporting the marine grunt
truth … which had also become my truth”.
• They depended entirely on the soldiers for food,
shelter and safety
• It has been suggested, not just by Gordon, that
their reporting lost some of its independence
US journalist Gordon Dillow quoted in Journalism
Studies journal.
STAR EAST 2013
Thomas McCoy
• Is there a risk that our testing may lose
independence (despite all the benefits that Agile
allegedly offers)?
99
STAR EAST 2013
100
25

28. STAR East 2013
2 May 2013
INDEPENDENT TEST TEAM
INDEPENDENT TEST TEAM
• With our independent testing
team we have found that it:
There are several ways in which a test
team could be kept “independent”:
• Physical?
• Administrative?
• Both?
STAR EAST 2013
• frees members from
relationship pressures
Give me a hug,
I found a bug!
• lessens “informal” fixes
• builds a collegiate atmosphere
• allows us to unashamedly
celebrate successes (which may
be development failures)
101
STAR EAST 2013
102
INDEPENDENT TEST TEAM
It has its own space, which
allows staff contributions and
celebrations to be put on public
display, creating positive
energy.
And it allows us to
shamelessly promote testing
principles (and call out the
numbers when the
inevitable defect misses
occur).
STAR EAST 2013
Thomas McCoy
103
STAR EAST 2013
104
26

29. STAR East 2013
2 May 2013
PROJECT MANAGER AFTER READING
YOUR E-MAIL ON HIS IPHONE
PERSONAL COURAGE
Like journalism, testing
often requires personal
courage, as shown in the
following slides…
STAR EAST 2013
105
PROJECT MANAGER DISCUSSING
IT WITH YOU
STAR EAST 2013
Thomas McCoy
STAR EAST 2013
106
HOW YOU FEEL AFTER THE
“DISCUSSION”
107
STAR EAST 2013
108
27

30. STAR East 2013
2 May 2013
PROMOTING OURSELVES
FINAL THOUGHTS
• As with journalism, we have to keep
re-inventing ourselves
• We created a 2-minute video
clip entitled Life Without Testing
• Our successes need to be communicated
appropriately
• The clip was effective in focusing attention
on the serious consequences of undetected
bugs in the Medicare payment system
• We have to expand our network of allies
• And we must keep the end users at the
centre of what we do and advocate for them
whenever possible
• You can watch it at:
https://vimeo.com/65502104
STAR EAST 2013
109
STAR EAST 2013
KEY POINTS
KEY POINTS
• Journalism and testing have similar skills and
principles, particularly as they relate to an
obligation to the truth and a sense of public
responsibility.
110
• Take the angle into account, use the 5Ws & 1H
where possible, and remember the inverted
pyramid.
• Take care with delivering bad news. Sometimes
people only hear what they want to hear.
• Testing is considered by some to be a vocation,
rather than a job.
• Publicising our successes is a challenge.
• Use personas to rehearse bad news delivery (if that
works for you).
• Think about news values such as impact, proximity
and timeliness in your communication.
• Work on writing and speaking skills so your
communication cuts through the noise.
STAR EAST 2013
Thomas McCoy
111
STAR EAST 2013
112
28

31. STAR East 2013
2 May 2013
KEY POINTS
KEY POINTS
• Pay special attention to the “headline” and first
paragraph in your writing; people may not read
beyond that.
• Consider using innovative communication
methods, such as word clouds, video clips, and
even mocking up a negative news story to illustrate
the potential public impact of a defect.
• Use Flesh-Kincaid to check readability (but avoid
using Green Eggs and Ham to fake it).
• Remember that graphs can lie.
• Practice recording yourself to improve voice
delivery.
• Use the Speak feature in Word and Outlook to help
you catch writing errors before you send them out
(it also lets you rest your eyes when reading long
documents).
• Develop sources throughout the organisation and
consider mapping its relationships onto the formal
structure chart.
• Remember George Orwell's writing advice.
STAR EAST 2013
113
KEY POINTS
KEY POINTS
• Do a usability test whenever possible.
• Are embedded testers (e.g. agile) sometimes too
close to developers?
• Review error messages for clarity and usefulness
as they appear during your testing.
• Personal courage is an essential aspect of
journalism and testing. Be brave.
• Always correct mistakes and review your written
work.
• Don’t lose your sense of humour (especially not if
you can use it in a video to get your message
through).
• Use questioning to gather information, but also to
convey implicit messages.
Thomas McCoy
114
• When planning questioning, think about the
medium, context, drivers (for the person you are
questioning) and what you hope to achieve.
• Capture information in “overload” situations by
recording audio and/or video.
STAR EAST 2013
STAR EAST 2013
115
STAR EAST 2013
116
29

32. STAR East 2013
2 May 2013
THANK YOU!
tom@thomasmccoy.com.au
STAR EAST 2013
Thomas McCoy
117
30