Download to read offline
![STAR East 2013
2 May 2013
WRITING 1
EXAMPLE
• Need a strong headline that will push
relevant buttons in people. The
“headline” might be, for example, the:
On the registration date screen February 29 is being
accepted for all years, not just leap years.
[Summary, covers “where” and “what”]
The problem may be due to the new external date
routine, which was recently implemented on this screen.
(The other screens still work fine.)
[More info]
This could damage customer relations and create
adverse publicity for the organisation and needs urgent
attention.
[Call to action]
STAR EAST 2013
37
WRITING 2
• The headline may be all they read
• Think about the “angle” of your
message (what are you trying to
achieve)?
38
STAR EAST 2013
• Clichés should be avoided like
the plague ☺
• Check spelling and grammar
• Secondary information later
• A reading level of Grade 8 is
recommended (can use FleshKincaid readability checker in
Word)
• Use Plain English (avoid jargon)
• Use mix of long and short sentences
• Keep paragraphs to 3 sentences on average
(large slabs of writing intimidate readers)
Thomas McCoy
subject line of your e-mail message
WRITING 3
• Start with most important point, which
should answer some of 5Ws & 1H
STAR EAST 2013
summary of your defect report; or the
The “buttons”
that will work
best depend
on our target
audience and
this is where
research and
prior planning
are
important.
Newspapers are typically
written at Grade 8 level.
39
STAR EAST 2013
40
10](https://image.slidesharecdn.com/tk1-140129114408-phpapp01/75/Keynote-Asking-the-Right-Questions-What-Journalism-Can-Teach-Testers-12-2048.jpg)
![STAR East 2013
2 May 2013
Client Registration
Family Name:
Nurk
Given Name:
Fred
Date of Birth:
01/01/2015
Date of Death:
Gender:
M
In testing a client
registration
application I entered
a future Date of Birth
to check for the error
condition (note that
Date of Death was
left blank).
As expected, an error
message was
displayed and I could
have marked the test
as “passed”.
However, I was horrified by the meaningless, technobabble “error
message” that was clearly written by tech-heads with zero communication
skills. As testers, when we see something like this, I feel we owe it to the
end user to report it, even if it isn’t on the official test plan.
System Date/Time: 22/4/2013 2:25PM
STAR EAST 2013
73
STAR EAST 2013
74
[1] Wrong field values or value types detected in Date of Birth.
[1] Wrong field values or value types detected in Date of Birth.
Please re-enter your field values. If you need additional
assistance, please refer to the documentation.(SBL-UIF-00299)
Please re-enter your field values. If you need additional
assistance, please refer to the documentation.(SBL-UIF-00299)
[2] The value '01/01/2015' for field 'Date of Birth' is required to be
'<= Today() AND <= [Date of Death]'.
[2] The value '01/01/2015' for field 'Date of Birth' is required to be
'<= Today() AND <= [Date of Death]'.
Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth
Date] <= '.(SBL-DAT-00521)
Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth
Date] <= '.(SBL-DAT-00521)
Looking at this message reveals many common problems:
1. Should not use terms like “wrong” and “detected”, they are negative
2. Should not use “field values or value types”, this is technobabble
3. The cryptic error code is meaningless
4. Why are there three versions of the error message?
5. Why is a function call i.e. Today() in an error message?
6. Why are mathematical comparison operators and Boolean logic used?
7. Why is Date of Birth written in 3 different ways: Date of Birth, ‘Date of
Birth’ and [Birth Date]?
8. Why are square brackets used around variable names?
9. Why do dates sometimes have single quotes and sometimes not?
STAR EAST 2013
Thomas McCoy
75
STAR EAST 2013
76
19](https://image.slidesharecdn.com/tk1-140129114408-phpapp01/75/Keynote-Asking-the-Right-Questions-What-Journalism-Can-Teach-Testers-21-2048.jpg)
![STAR East 2013
2 May 2013
FIREFOX ERROR MESSAGE
[1] Wrong field values or value types detected in Date of Birth.
Please re-enter your field values. If you need additional
assistance, please refer to the documentation.(SBL-UIF-00299)
[2] The value '01/01/2015' for field 'Date of Birth' is required to be
'<= Today() AND <= [Date of Death]'.
Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth
Date] <= '.(SBL-DAT-00521)
10. Why does the last comparison end with <= '. (this is because Date of
Death was blank, so why are we comparing with blank)?
11. Why is there a second meaningless error code?
The entire error message, of more than 50 words and meaningless codes
and symbols, could be replaced with:
Date of Birth may not be in the future. Please re-enter.
77
STAR EAST 2013
Describe Problem (clearly and succinctly)
WHAT
Cause (in as much detail as necessary)
WHY
3.
Solution (steps to fix problem)
HOW
STAR EAST 2013
Thomas McCoy
78
Should be:
understandable
relevant
actionable
user-centred
brief
clear
specific
courteous
rare
• Good error messages should answer some
key 5W&1H questions:
2.
STAR EAST 2013
ERROR MESSAGES 2
ERROR MESSAGES 1
1.
Firefox has moved towards some soft and cuddly error messages where
the computer takes the blame and becomes humanised.
This does not add value and quickly becomes tedious.
Better to just write meaningful and useful messages in plain English.
79
STAR EAST 2013
80
20](https://image.slidesharecdn.com/tk1-140129114408-phpapp01/75/Keynote-Asking-the-Right-Questions-What-Journalism-Can-Teach-Testers-22-2048.jpg)
More Related Content
Similar to Keynote: Asking the Right Questions? What Journalism Can Teach Testers
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Keynote: Asking the Right Questions? What Journalism Can Teach Testers
- 1. KT1 Keynote 5/2/2013 8:30:00 AM Askingthe Right Questions? What Journalism Can Teach Testers Presented by: Thomas McCoy Australian Department of Families, Housing, Community Services and Indigenous Affairs Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
- 2. Thomas McCoy Australian softwaretesting professional, teacher, and journalist Thomas McCoy has worked in the IT industry for more than two decades. Much of this time was spent as a software developer and IT manager with Australian government agencies. Wanting to make a greater contribution to software quality, Thomas re-oriented his career into the emerging discipline of software testing and has been an enthusiastic promoter of the profession ever since. At conferences in Australia, New Zealand, Canada, and the US, Thomas is a popular keynote speaker, who has received several best presentation awards. You can reach Thomas at tom@thomasmccoy.com.au.
- 3. STAR East 2013 2 May 2013 DISCLAIMER ASKING THE RIGHTQUESTIONS The views expressed in this presentation are not necessarily those of the Department of Families, Housing, Community Services and Indigenous Affairs, the Australian Government, or even the presenter. ☺ What Journalism Can Teach Testers Thomas McCoy BJourn, BSc, DipEd, MInfTech Australian Department of Families, Housing, Community Services and Indigenous Affairs 2 STAR EAST 2013 Australia I’m from Australia, which is around 9,500 miles away and now has a population of 23 million (just a few million more than the population of Florida). STAR EAST 2013 Thomas McCoy Canberra I’m based in Canberra, Australia’s capital city, which is in the southeast of Australia between Sydney and Melbourne. It was built in 1913 and designed by Chicago architect Walter Burley Griffin (who won an international design competition). Its population is around 370,000. 3 STAR EAST 2013 4 1
- 4. STAR East 2013 2 May 2013 FaHCSIA Background Department of Families, Housing,Community Services and Indigenous Affairs FaHCSIA, the government department I work for, is responsible for social security in Australia, with payments and support for groups including pensioners, the unemployed, people with disabilities, and our Indigenous Australians. It manages about one fifth of the federal budget. We have testing staff working within both the agile (embedded in development teams for the smaller systems) and waterfall (working as a large testing group for our major application) contexts. STAR EAST 2013 5 Thomas McCoy 6 STAR EAST 2013 JOURNALISM SKILLS WHAT DOES SOFTWARE TESTING HAVE TO DO WITH JOURNALISM? STAR EAST 2013 Started in journalism and media, then moved into IT, first into programming and finally into software testing. Have become increasingly aware of the importance of communication, both in terms of project team communication and computer / end user communication. This session has been inspired by the parallels I’ve seen between journalism and software testing. • A questioning, curious, and inquisitive mind • Communication skills • Writing ability • News sense • Listening skills • General knowledge • Empathy, patience and understanding • Sense of public responsibility 7 STAR EAST 2013 • • • • • • • • • • Language skills Interviewing skills Ability to meet deadlines Knowledge of computers and technology Analysis skills Resourcefulness Persistence Truthfulness Accuracy Integrity Notice big overlap with testing skills. 8 2
- 5. STAR East 2013 2 May 2013 JOURNALISM PRINCIPLES THE FOURTHESTATE • Its first obligation is to the truth Journalism has been called “the Fourth Estate”. Comes from British Parliament which has the monarch (1st estate), House of Lords, containing nobility and clergy (2nd estate), and House of Commons containing the other elected members (3rd estate). “In the reporters gallery yonder there sits a fourth estate more important by far than they all,” said 18th century British politician Edmund Burke. Fourth Estate rests on idea that it’s the media's function to act as a guardian of the public interest and as a watchdog on those in power. Are there parallels between this power structure and IT projects? • Its practitioners must maintain independence from those they cover • It must serve as an independent monitor of power • Its practitioners must be allowed to exercise their personal conscience These are principles in an “ideal world”, but again there is strong overlap with our testing career. 9 STAR EAST 2013 STAR EAST 2013 10 THE FOURTH ESTATE Estate Journalism 1st King 2nd House of Lords (nobility) 3rd House of Commons 4th Journalists STAR EAST 2013 Thomas McCoy IT Business areas? Project Managers? TESTING: JOB OR VOCATION? The enormous impact technology has on society, and our position in reviewing quality (too often at the end of the process), has led some to ask whether software testing is more of a vocation than a job. Vocations are sometimes differentiated from jobs because of their strong sense of social responsibility (e.g. teachers, nurses, clergy, journalists, etc.). Developers? Testers? 11 STAR EAST 2013 12 3
- 6. STAR East 2013 2 May 2013 TESTING: JOB ORVOCATION? • Sacrifice • Burnout • Long hours • Commitment • Thanklessness? • Social conscience • Self-development • Fighting “the good fight” Impact of a recent technology issue. Apart from that, we’ve had people killed, fried by radiation machines, not be able to use the brakes on their car, etc. Do you identify with any of these aspects in your role? 13 STAR EAST 2013 14 STAR EAST 2013 SHARING NEWS OF OUR SUCCESS Because of the nature of our role it’s sometimes challenging to publicise our success without upsetting people. We might want to do something like this … Then there are the less publicised daily annoyances caused by deficient technology (which can’t be good for people’s health). STAR EAST 2013 Thomas McCoy 15 STAR EAST 2013 16 4
- 7. STAR East 2013 2 May 2013 More palatable news TestingNews … but this could cause offence, so we publicise in more benign ways, as in my newsletter story about our new test lab … STAR EAST 2013 17 STAR EAST 2013 NEWS VALUES DELIVERING BAD NEWS 1 Three useful news values are: • For us, this is a common and difficult communication task • Nobody likes to hear bad news • Nobody ever says “thank you” for receiving bad news • Impact: angle your message for maximum impact in relation to your target audience • Proximity: if something is close to your target audience or their aspirations it will have greater effect • Timeliness: don’t delay messages unnecessarily STAR EAST 2013 Thomas McCoy 18 19 STAR EAST 2013 20 5
- 8. STAR East 2013 2 May 2013 DELIVERING BAD NEWS2 The original Larson dog cartoon … • Avoid it being a surprise (perhaps it can first be passed through informal communication channels so people can adjust to the idea) Sometimes “selective listening” is a problem when receiving bad news, as illustrated by this re-tooling of a classic Larson cartoon. • Pick a time when the recipient is relaxed (sometimes Friday afternoons work well) • If possible, propose some alternatives when delivering the news STAR EAST 2013 21 STAR EAST 2013 22 REHEARSING BAD NEWS DELIVERY WITH AUSTRALIAN NATIVE ANIMAL PERSONAS … can be adapted to our situation Sometimes, when having to deliver bad news, it can help to think about the “persona” you’ll adopt when delivering the message. In Australia, we have native animals with unique characteristics that can be used to rehearse the bad news delivery (we even have hand puppets). ☺ Sometimes we might have to go in aggressively, while at other times it might be with an attitude of resignation (because we choose not to fight this particular battle), or even with a humorous approach. STAR EAST 2013 Thomas McCoy 23 STAR EAST 2013 24 6
- 9. STAR East 2013 2 May 2013 TASMANIAN DEVIL KANGAROO Aggressive andenergetic Cheerful and easy going “Well what do you expect when we weren’t given the resources we needed!” “OK, we’ll see what can be done” 25 STAR EAST 2013 KOALA BEAR KOOKABURRA Always ready for a laugh Relaxed and nonchalant “Ha, ha, you’re joking, you know that was always going to be impossible.” “Yeah, right, whatever.” STAR EAST 2013 Thomas McCoy 26 STAR EAST 2013 27 STAR EAST 2013 28 7
- 10. STAR East 2013 2 May 2013 “I keep sixhonest serving-men (They taught me all I knew); Their names are What and Why and When and How and Where and Who” A KEY PART OF JOURNALISM: THE 5WS & 1H Think about the 5 Ws & 1H in anything you write. For example, in a defect report, as shown on the next slide … 29 STAR EAST 2013 5WS & 1H • Who: Rudyard Kipling from “The Elephant’s Child” Fred Nurk 30 STAR EAST 2013 Defect report example. Note that you won’t necessarily need all of the 5Ws & 1H in everything you write so consider which ones to leave out. • What: Discovered system crashes if negative quantities entered • When: Last night WRITING EFFECTIVELY • Where: Test lab • Why: • How: STAR EAST 2013 Thomas McCoy Inadequate input field checking? Running automated test scripts 31 STAR EAST 2013 32 8
- 11. STAR East 2013 2 May 2013 GEORGE ORWELL’S ADVICE “Alwaysgrab the reader by the throat in the first paragraph, sink your thumbs into his wind pipe in the second, and hold him against the wall until the last word.” 1. Never use a long word when a short one will do. 2. If it is possible to cut out a word, always cut it out. 3. Never use the passive voice when you can use the active. This might be a bit ambitious for the average test plan but the reason so few documents are read is because they’re so totally unengaging. 4. Break any of these rules sooner than saying anything outright barbarous. If you remember nothing else from this session, remember these rules. You will sometimes need to violate #3 and use passive voice to avoid blaming somebody. Active Voice: Fred Nurk introduced a defect (blames Fred) Passive Voice: A defect was introduced (blames nobody) 33 STAR EAST 2013 THE INVERTED PYRAMID Top Content Position STAR EAST 2013 34 AN EFFECTIVE LAYOUT People are so deluged with textual information that they may not read more than the first few lines. So use the inverted pyramid to present your information. Bottom • First paragraph: summarise the situation, covering the most important aspects of who, what, when, where, why and how (this may be the only paragraph the person reads) When communicating about issues needing attention you may want to have a “call to action” at the end rather (than entirely following the inverted pyramid • Next: provide more details style of having less and less • Finally: use a “call to action” important information going down the page). to have the issue resolved Importance (signified by width) STAR EAST 2013 Thomas McCoy 35 STAR EAST 2013 36 9
- 12. STAR East 2013 2 May 2013 WRITING 1 EXAMPLE • Needa strong headline that will push relevant buttons in people. The “headline” might be, for example, the: On the registration date screen February 29 is being accepted for all years, not just leap years. [Summary, covers “where” and “what”] The problem may be due to the new external date routine, which was recently implemented on this screen. (The other screens still work fine.) [More info] This could damage customer relations and create adverse publicity for the organisation and needs urgent attention. [Call to action] STAR EAST 2013 37 WRITING 2 • The headline may be all they read • Think about the “angle” of your message (what are you trying to achieve)? 38 STAR EAST 2013 • Clichés should be avoided like the plague ☺ • Check spelling and grammar • Secondary information later • A reading level of Grade 8 is recommended (can use FleshKincaid readability checker in Word) • Use Plain English (avoid jargon) • Use mix of long and short sentences • Keep paragraphs to 3 sentences on average (large slabs of writing intimidate readers) Thomas McCoy subject line of your e-mail message WRITING 3 • Start with most important point, which should answer some of 5Ws & 1H STAR EAST 2013 summary of your defect report; or the The “buttons” that will work best depend on our target audience and this is where research and prior planning are important. Newspapers are typically written at Grade 8 level. 39 STAR EAST 2013 40 10
- 13. STAR East 2013 2 May 2013 Settings … USING THEREADABILITY CHECKER These are some of the grammar checks the system can do under Grammar & Style but be aware that there are many it can’t do well (because writing is a human activity) so don’t rely on it too much and don’t follow it blindly (sometimes it gets confused). Passive sentences is worth using, though. In Word 2010 do: File Tab, Options, Proofing… … and make sure you tick the Show readability statistics checkbox. Also think about what you want to use for Writing Style (e.g. just Grammar or Grammar & Style) and check further options with the Settings … button. STAR EAST 2013 41 42 STAR EAST 2013 READABILITY STATISTICS These are only a guide and have some quirky aspects. Flesh-Kincaid is based on average number of syllables per word and average number of words per sentence. This results in the kids’ classic “Green Eggs and Ham” achieving a grade level of minus 1.3, which would suggest it could be understood by kids before they start school and have learnt to read. We tried an interesting experiment … STAR EAST 2013 Thomas McCoy We ran the Encyclopaedia Britannica article on Nuclear Physics through FleshKincaid and it came back with a Grade level of 15.5 (i.e. written at university level) … Nuclear Physics: 15.5 grade level 43 STAR EAST 2013 44 11
- 14. STAR East 2013 2 May 2013 … and thenwe pasted the entire text of Green Eggs and Ham at the end of the article, and when we ran readability again it brought the Grade level down to 2.4! Does this mean a kid halfway through Grade 2 could now understand nuclear physics? Nuclear Physics & Green Eggs: 2.4 grade level OTHER WAYS OF PRESENTING INFORMATION After this somebody helpfully suggested that we should paste Green Eggs and Ham at the end of all our unreadable documents (using a 1point font and white text for minimum visibility) to bring our readability scores down, but that’s another story … STAR EAST 2013 45 Wordle Word Cloud of Seven Testing Principles 46 STAR EAST 2013 Graphics can be a useful way to present information (and they have enormous credibility). Here is a graph of a project where the overtime hours are doubling every month, from 200 to 400 to 800 to 1,600. But would reporting this trend cause unnecessary alarm? Overtime Hours 2000 1500 1000 500 0 A word cloud is an innovative way of illustrating key concepts in a document. The more often a term occurs the larger it is. This is from the 7 principles of testing in the ISTQB Foundation Syllabus and was generated by Wordle (http://www.wordle.net/). STAR EAST 2013 Thomas McCoy 47 Jan Feb Mar Apr True Representation STAR EAST 2013 48 12
- 15. STAR East 2013 2 May 2013 Not if weswitch to a logarithmic scale, which ensures that the world is still a happy place (and the slippery slope suddenly looks so much less slippery.) Or how about choosing an completely irrelevant chart type (Excel offers so many, and I’ve often seen them misused). Overtime Hours Overtime Hours 10000 Jan, 200 1000 Apr, 1600 100 Feb, 400 Mar, 800 10 1 Jan Feb Mar Apr Logarithmic Scale Irrelevant Pie Chart 49 STAR EAST 2013 Or a completely meaningless chart (but you should probably remove the numbers from the cones). By the way, I’m not suggesting that we should deceive people. I’m just demonstrating how easily information can be manipulated even with socalled “scientific” approaches. Overtime Hours Apr, 1600 Mar, 800 100% And many years ago (before everybody could do this on their iPhone) we created a rap-song video to expose the organisation to automation testing. It was done with a low-end camera and heaps of staff support. Some of the claims were (admittedly) exaggerated but this clip is still available on Sticky Minds (www.stickyminds.com/) just search for “Automation Rap”. Think about whether using video might be useful in your communication. Feb Jan, 200 50 Mar Feb, 400 STAR EAST 2013 Jan 50% Apr 0% Meaningless 100% Stacked Horizontal Cone Chart STAR EAST 2013 Thomas McCoy 51 STAR EAST 2013 52 13
- 16. STAR East 2013 2 May 2013 VOICE 1 VOICE 2 Theway we use our voice is critical to getting our message across effectively. And vocal variation can help keep people awake. There are 4 important aspects: 2. 1. Pitch 3. Quick pace can also convey urgency. Or can be used to gloss over boring details. Use slow pace for important information Pause An appropriate pause (count silently to 5) can help focus attention before a key word We need to raise our pitch to instil a sense of urgency and panic (or lower it to restore calm) STAR EAST 2013 Pace 4. Projection Need to project voice to ensure it is heard 53 54 STAR EAST 2013 VOICE 3 • Invest some time in voice work • Use a recorder to play yourself back (and don’t be shocked the first time you hear yourself – that happens to all of us) • Volunteer for presentations (even in non-work environments such as sporting clubs, kids school, etc.) ACQUIRING INFORMATION • Think about joining a group like Toastmasters (http://www.toastmasters.org/) which is dedicated to public speaking STAR EAST 2013 Thomas McCoy 55 STAR EAST 2013 56 14
- 17. STAR East 2013 2 May 2013 INVESTIGATIVE JOURNALISM SOURCES • Findseveral sources (don’t burn them) • Can increase our understanding of “what’s really going on” • Speak to as many relevant people as possible • Can serve as early warning of risks • Verify information through reliable, factual (preferably documented) evidence • Respect “off the record” comments • Don’t take everything at face value (because people may be running their own agendas) You can independently verify off the record comments but you can’t credit them to the source, or even mention the source. 57 STAR EAST 2013 DEVELOPING SOURCES 2 DEVELOPING SOURCES 1 Finance Management And there are many ways of reaching out beyond your immediate work environment. I can’t sing but still joined the office choir and was able to hear how the IT group appeared to people on the outside. Business • Work social club Tentacled Tester • Sporting teams • Lunches Technical Administrative • Attending presentations and events Marketing To develop sources of the greatest value it can be useful to put your tentacles out as widely as possible (rather than remaining cloistered within IT). STAR EAST 2013 Thomas McCoy 58 STAR EAST 2013 • Joining the office choir 59 STAR EAST 2013 Departmental Choir 60 15
- 18. STAR East 2013 2 May 2013 CRITERIA FOR EFFECTIVESOURCES INVESTIGATIVE TECHNIQUES • Past suitability • Authoritativeness • Infiltration • Reliability • Articulateness • Impersonation • Trustworthiness • Discretion • Secret filming and/or recording Sometimes it may be necessary to “infiltrate” meetings you haven’t been included in. Or to impersonate somebody (e.g. via a phone call) to obtain information. (You should not impersonate an existing person, but you might impersonate a generic role e.g. “I’m ringing from tech support”). Be very careful with secret filming or recording (more on this later). 61 STAR EAST 2013 MAPPING THE ORGANISATION 1 MAPPING THE ORGANISATION 2 is cousin of Sheila Chaz Bluey Kylie Ernie STAR EAST 2013 Thomas McCoy Molly Tezz Nicole Sidney Snowy 62 STAR EAST 2013 Magda Talisha Brian Bazza In understanding the operation of an organisation it can be useful to map the informal relationships onto the formal org chart as shown in the next slide. Sheila Chaz Bluey Hugo Kylie Kezza Ernie Tezz Molly Nicole Sidney Snowy disagrees with got promoted over teaches trombone to son of 63 STAR EAST 2013 plays rugby with Much more informative. Magda Talisha Brian Bazza Hugo Kezza RELATIONSHIPS 64 16
- 19. STAR East 2013 2 May 2013 ALWAYS DO USABILITYTESTING • It is cheap and can be run on a laptop • Testing even one user is better than no users • Have them perform some tasks and ask them to “think out loud” as they complete them OVERT RECORDING: USABILITY TESTING • Software like TechSmith’s Morae product will capture the screen, the user and the audio in real time • The “video evidence” can be compelling when shown to developers STAR EAST 2013 65 66 STAR EAST 2013 Fake SAVE button We watched a usability clip where the “save” button had been removed from a new version of the system (which now did automatic saves). This traumatised a user of the old system who could no longer find a “save” button and did now know whether her work would be lost when she exited. She became “stuck” and would have rung the help desk for advice. STAR EAST 2013 Thomas McCoy 67 SAVE My recommended, innovative, low-cost solution, of adding a fake “save” button to the application so users could still click it and reduce their anxiety, was rejected. STAR EAST 2013 68 17
- 20. STAR East 2013 2 May 2013 COVERT RECORDING 1 •With the capability of today’s mobile devices it is trivial to record audio and/or video covertly (you no longer need to be a secret agent) COVERT RECORDING • Many technical meetings generate so much information so quickly that recording is the only way of obtaining accurate notes • Ideally you would ask participants if you can record (however, this makes some people so uncomfortable that sometimes not asking may be more humane) but check with local laws and company policies STAR EAST 2013 69 STAR EAST 2013 70 COVERT RECORDING 2 • In the ACT (where I come from) we have a funny law where you’re allowed to covertly record any conversation that you were part of as long as at least one person agrees (and that can be you) ☺ • Note that covert recordings should never be used against somebody (once relevant notes have been made they should be destroyed) OUR TESTING RESPONSIBILITY (AND BAD ERROR MESSAGES) • Here is an example where somebody inappropriately used covert recording to request a testing extension while the boss was going about his business (with the aim of later playing it back to the boss as proof of the extension): www.thomas.mccoy.name/DontDoThis.mp3 STAR EAST 2013 Thomas McCoy 71 STAR EAST 2013 72 18
- 21. STAR East 2013 2 May 2013 Client Registration Family Name: Nurk GivenName: Fred Date of Birth: 01/01/2015 Date of Death: Gender: M In testing a client registration application I entered a future Date of Birth to check for the error condition (note that Date of Death was left blank). As expected, an error message was displayed and I could have marked the test as “passed”. However, I was horrified by the meaningless, technobabble “error message” that was clearly written by tech-heads with zero communication skills. As testers, when we see something like this, I feel we owe it to the end user to report it, even if it isn’t on the official test plan. System Date/Time: 22/4/2013 2:25PM STAR EAST 2013 73 STAR EAST 2013 74 [1] Wrong field values or value types detected in Date of Birth. [1] Wrong field values or value types detected in Date of Birth. Please re-enter your field values. If you need additional assistance, please refer to the documentation.(SBL-UIF-00299) Please re-enter your field values. If you need additional assistance, please refer to the documentation.(SBL-UIF-00299) [2] The value '01/01/2015' for field 'Date of Birth' is required to be '<= Today() AND <= [Date of Death]'. [2] The value '01/01/2015' for field 'Date of Birth' is required to be '<= Today() AND <= [Date of Death]'. Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth Date] <= '.(SBL-DAT-00521) Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth Date] <= '.(SBL-DAT-00521) Looking at this message reveals many common problems: 1. Should not use terms like “wrong” and “detected”, they are negative 2. Should not use “field values or value types”, this is technobabble 3. The cryptic error code is meaningless 4. Why are there three versions of the error message? 5. Why is a function call i.e. Today() in an error message? 6. Why are mathematical comparison operators and Boolean logic used? 7. Why is Date of Birth written in 3 different ways: Date of Birth, ‘Date of Birth’ and [Birth Date]? 8. Why are square brackets used around variable names? 9. Why do dates sometimes have single quotes and sometimes not? STAR EAST 2013 Thomas McCoy 75 STAR EAST 2013 76 19
- 22. STAR East 2013 2 May 2013 FIREFOX ERROR MESSAGE [1]Wrong field values or value types detected in Date of Birth. Please re-enter your field values. If you need additional assistance, please refer to the documentation.(SBL-UIF-00299) [2] The value '01/01/2015' for field 'Date of Birth' is required to be '<= Today() AND <= [Date of Death]'. Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth Date] <= '.(SBL-DAT-00521) 10. Why does the last comparison end with <= '. (this is because Date of Death was blank, so why are we comparing with blank)? 11. Why is there a second meaningless error code? The entire error message, of more than 50 words and meaningless codes and symbols, could be replaced with: Date of Birth may not be in the future. Please re-enter. 77 STAR EAST 2013 Describe Problem (clearly and succinctly) WHAT Cause (in as much detail as necessary) WHY 3. Solution (steps to fix problem) HOW STAR EAST 2013 Thomas McCoy 78 Should be: understandable relevant actionable user-centred brief clear specific courteous rare • Good error messages should answer some key 5W&1H questions: 2. STAR EAST 2013 ERROR MESSAGES 2 ERROR MESSAGES 1 1. Firefox has moved towards some soft and cuddly error messages where the computer takes the blame and becomes humanised. This does not add value and quickly becomes tedious. Better to just write meaningful and useful messages in plain English. 79 STAR EAST 2013 80 20
- 23. STAR East 2013 2 May 2013 PROMPTING ACTION THE TESTINGTIMES • If you are experiencing difficulty in conveying the importance of having an issue addressed, mocking up adverse media coverage can help Cryptic error message causes user chaos • On the next slide I have created a fictitious story about the error message and can then innocently ask “but what would happen if something like this came out in the paper”? • Having it looking real, rather than being hypothetical, may be more persuasive • “Stock prices plummet” might be another example The cryptic error message STAR EAST 2013 May 2013 www.testingtimes.com.au 81 Help Desk phones were inundated as users struggled with a cryptic error message that would require a computer science degree to decipher. The message, which included comparison operators, Boolean logic, function calls, field names and variables, was described by one user as “complete gobbledegook”. The error condition prompting the message was triggered when a user typed a date of birth for a client that was in the future. While this is clearly invalid, the message was of little use in MISTAKES • Since we critique the work of others it is reasonable that we are held to high standards • In the same was as newspapers sometimes issue corrections, we should promptly admit mistakes CHECKING ACCURACY (OURS) • An excellent way of checking our written material is to hear it read out loud • The Office 2010 products (Word, Outlook, Excel) have an inbuilt reader that can read any text highlighted by the user STAR EAST 2013 Thomas McCoy 83 STAR EAST 2013 84 21
- 24. STAR East 2013 2 May 2013 THE OFFICE 2010INBUILT READER • To use the reader (called “Speak”) you need to add the Speak icon to your Quick Access Toolbar • You can then select any text and click the icon to have it read out QUESTIONING • You can use it to read back e-mails before you press “Send” or to read out long, boring documents when your eyes are fading • A video to show you how to do this is available at: www.thomas.mccoy.name/SpeakFeature.htm STAR EAST 2013 85 STAR EAST 2013 QUESTIONING EFFECTIVE QUESTIONING 1 • At the heart of journalism • Do research, plan questions and strategy and write them down • Requires substantial research and preparation • Be clear on the objective of every question • Journalist often knows answer in advance • Think about likely responses and your follow-up question • Questions can be used to convey an implicit message in a less threatening way than statements STAR EAST 2013 Thomas McCoy 86 • Make person comfortable with informal conversation to start with 87 STAR EAST 2013 88 22
- 25. STAR East 2013 2 May 2013 EFFECTIVE QUESTIONING 2 EFFECTIVEQUESTIONING • Ask mixture of open and closed questions • Record answers (shows interest) • Repeat answer back for confirmation • Take special note of anything that comes “after the interview” • Use leading questions, if necessary e.g. “When this error message frustrates the user, who are they supposed to call” • Do not make it feel like an interrogation • Adopt a non-threatening and inquisitive manner • Smile and nod, endure silences, don’t interrupt, be polite, stay neutral STAR EAST 2013 CHOOSING THE MEDIUM • Face to face • E-mail • Phone call • Post-it note 89 Your medium will depend on the message. For an informal message a Post-it note may be fine. For something more serious you may want to do face to face with e-mail follow-up. STAR EAST 2013 CHOOSING THE CONTEXT • One-on-one in private • One-on-one in open plan • Daily stand up Choose your context carefully and keep participants to an absolute minimum. • Team meeting • SMS • Project meeting • Social media • Executive meeting STAR EAST 2013 Thomas McCoy 90 91 STAR EAST 2013 92 23
- 26. STAR East 2013 2 May 2013 WHAT KIND OFQUESTIONS? EMBEDDED MESSAGE EXAMPLES • This depends on who you are talking to, what you are hoping to achieve and what motivates them, for example: Project Manager Project Manager – not meeting deadlines Business Analyst - incomplete or defective requirements Developer - coding errors STAR EAST 2013 93 • “How much overtime are we able to accommodate under the current budget?” • Embedded message: There is no way we can finish testing on time with the existing resources. STAR EAST 2013 EMBEDDED MESSAGE EXAMPLES EMBEDDED MESSAGE EXAMPLES Business Analyst Developer • “I’m just wondering what happens on the registration screen if the user has registered more than once?” • “What happens if the user puts in the year with just the last two digits?” • Embedded message: These requirements have holes. • Embedded message: There is a bug in the date routine. STAR EAST 2013 Thomas McCoy 94 95 STAR EAST 2013 96 24
- 27. STAR East 2013 2 May 2013 EMBEDDED JOURNALISM … DEVELOPERSAND TESTERS HOW CLOSE IS TOO CLOSE? … OR IN-BEDDED? STAR EAST 2013 97 STAR EAST 2013 98 AGILE TEAMS: TOO CLOSE? • Gordon Dillow was one of around 800 journalists embedded with soldiers in the Iraq war “I found myself falling in love with my subject. I fell in love with ‘my’ Marines … the point wasn’t that I wasn’t reporting the truth, the point was that I was reporting the marine grunt truth … which had also become my truth”. • They depended entirely on the soldiers for food, shelter and safety • It has been suggested, not just by Gordon, that their reporting lost some of its independence US journalist Gordon Dillow quoted in Journalism Studies journal. STAR EAST 2013 Thomas McCoy • Is there a risk that our testing may lose independence (despite all the benefits that Agile allegedly offers)? 99 STAR EAST 2013 100 25
- 28. STAR East 2013 2 May 2013 INDEPENDENT TEST TEAM INDEPENDENTTEST TEAM • With our independent testing team we have found that it: There are several ways in which a test team could be kept “independent”: • Physical? • Administrative? • Both? STAR EAST 2013 • frees members from relationship pressures Give me a hug, I found a bug! • lessens “informal” fixes • builds a collegiate atmosphere • allows us to unashamedly celebrate successes (which may be development failures) 101 STAR EAST 2013 102 INDEPENDENT TEST TEAM It has its own space, which allows staff contributions and celebrations to be put on public display, creating positive energy. And it allows us to shamelessly promote testing principles (and call out the numbers when the inevitable defect misses occur). STAR EAST 2013 Thomas McCoy 103 STAR EAST 2013 104 26
- 29. STAR East 2013 2 May 2013 PROJECT MANAGER AFTERREADING YOUR E-MAIL ON HIS IPHONE PERSONAL COURAGE Like journalism, testing often requires personal courage, as shown in the following slides… STAR EAST 2013 105 PROJECT MANAGER DISCUSSING IT WITH YOU STAR EAST 2013 Thomas McCoy STAR EAST 2013 106 HOW YOU FEEL AFTER THE “DISCUSSION” 107 STAR EAST 2013 108 27
- 30. STAR East 2013 2 May 2013 PROMOTING OURSELVES FINAL THOUGHTS •As with journalism, we have to keep re-inventing ourselves • We created a 2-minute video clip entitled Life Without Testing • Our successes need to be communicated appropriately • The clip was effective in focusing attention on the serious consequences of undetected bugs in the Medicare payment system • We have to expand our network of allies • And we must keep the end users at the centre of what we do and advocate for them whenever possible • You can watch it at: https://vimeo.com/65502104 STAR EAST 2013 109 STAR EAST 2013 KEY POINTS KEY POINTS • Journalism and testing have similar skills and principles, particularly as they relate to an obligation to the truth and a sense of public responsibility. 110 • Take the angle into account, use the 5Ws & 1H where possible, and remember the inverted pyramid. • Take care with delivering bad news. Sometimes people only hear what they want to hear. • Testing is considered by some to be a vocation, rather than a job. • Publicising our successes is a challenge. • Use personas to rehearse bad news delivery (if that works for you). • Think about news values such as impact, proximity and timeliness in your communication. • Work on writing and speaking skills so your communication cuts through the noise. STAR EAST 2013 Thomas McCoy 111 STAR EAST 2013 112 28
- 31. STAR East 2013 2 May 2013 KEY POINTS KEY POINTS •Pay special attention to the “headline” and first paragraph in your writing; people may not read beyond that. • Consider using innovative communication methods, such as word clouds, video clips, and even mocking up a negative news story to illustrate the potential public impact of a defect. • Use Flesh-Kincaid to check readability (but avoid using Green Eggs and Ham to fake it). • Remember that graphs can lie. • Practice recording yourself to improve voice delivery. • Use the Speak feature in Word and Outlook to help you catch writing errors before you send them out (it also lets you rest your eyes when reading long documents). • Develop sources throughout the organisation and consider mapping its relationships onto the formal structure chart. • Remember George Orwell's writing advice. STAR EAST 2013 113 KEY POINTS KEY POINTS • Do a usability test whenever possible. • Are embedded testers (e.g. agile) sometimes too close to developers? • Review error messages for clarity and usefulness as they appear during your testing. • Personal courage is an essential aspect of journalism and testing. Be brave. • Always correct mistakes and review your written work. • Don’t lose your sense of humour (especially not if you can use it in a video to get your message through). • Use questioning to gather information, but also to convey implicit messages. Thomas McCoy 114 • When planning questioning, think about the medium, context, drivers (for the person you are questioning) and what you hope to achieve. • Capture information in “overload” situations by recording audio and/or video. STAR EAST 2013 STAR EAST 2013 115 STAR EAST 2013 116 29
- 32. STAR East 2013 2 May 2013 THANK YOU! tom@thomasmccoy.com.au STAR EAST2013 Thomas McCoy 117 30