TestingStage18

1. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 1
If the users can’t use it,
it doesn’t work
Testing Stage 2018 Rolf Molich
DialogDesign
Denmark
Vejle Station, DK, track 6, Saturday morning 23-02-2013

2. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 2

3. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 3
The passenger
explained that
because of the bright,
red ring he was
totally convinced that
this was the
emergency brake.
He explained that in
a situation like this
you don’t read text,
you act.
He did not notice the
real emergency
brake handle. After
the incident, the
emergency brake
handle was pointed
out to him.

4. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 4
Are the train’s systems
working according to
specifications ?
Yes!
Is there a problem?
Yes!
This is a usability problem
Rolf Molich, DialogDesign
► Independent usability consultant, DialogDesign,
www.DialogDesign.dk, Denmark. Since 1993
► Education: MSc in Software Engineering, 1974
► Invented the heuristic evaluation method in 1990 with
Jakob Nielsen
► Comparative Usability Evaluation (CUE) studies
1998 - ...
► UXPA-I Lifetime Achievement Award 2014 for the
CUE-studies
► Passionate about UX certification: www.uxqb.org

5. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 5
cabbage
> Sorry, the password must be more than 8 characters.
boiledcabbage
> Sorry, the password must contain 1 numerical character.
1 boiled cabbage
> Sorry, the password cannot have blank spaces.
50frackingboiledcabbages
> Sorry, the password must contain at least one upper case character.
50FRACKINGboiledcabbages
> Sorry, the password cannot use more than one upper case character consecutively.
50FrackingBoiledCabbagesShovedUpYourArse.IfYouDon’tGiveMeAccessImmediately
> Sorry, the password cannot contain punctuation.
NowIAmGettingReallyPissedOff50FrackingBoiledCabbagesShovedUpYourArse
IfYouDontGiveMeAccessImmediately
>Sorry, that password is already in use.

6. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 6
If the users
can’t use it,
it doesn’t work
-- Susan Dray
Credo 1
12
I want a carousel
on the home page

7. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 7
HiPPO = Highest Paid Person’s Opinion
Data
is important
Opinions
are unimportant
Credo 2

8. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 8
Users:
Pests with requests.
-- Anonymous
Users:
Those things that
make it possible for us
to pay you.
-- Matthew Oliphant

9. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 9
Usability
The extent to which an interactive system can be used by specified users
to achieve specified goals with effectiveness, efficiency and satisfaction
in a specified context of use.
Or, perhaps slightly simpler:
► Can the product do what *I* want?
► Can it do what I want, fast?
► Is the product pleasant to use?
An alternative definition of usability

10. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 10
Quiz
Judging from the home page of Arngren.net, is this website usable?
A. Yes, it is usable
B. It depends ...
C. There is insufficient information to answer the question
D. No, it is not usable

11. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 11
Usability test
A usability evaluation that involves representative users performing
specific tasks with the interactive system to enable identification and analysis of
usability problems, or the measurement of effectiveness, efficiency, and user
satisfaction.
A usability test consists of a number of usability test sessions.
In each session, a usability test participant attempts to carry out
representative usability test tasks using the interactive system or a prototype of
the interactive system.

12. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 12
Usability test: The ”think-aloud” method
Usability test: The ”think-aloud” method

13. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 13
The core of a usability test
► In a usability test, test participants solve realistic tasks
with as little help from the moderator as possible.
► We are not showing test participants how to solve tasks.
► We are not interested in test participants’ opinions.
We want to know whether they can solve the tasks we give them.

14. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 14
Sample usability test tasks for Ryanair’s website
► Book a one-way flight with Ryanair from London to Copenhagen on May 1.
► Find out how much cabin baggage you can bring on the flight from London to Copenhagen
► You can’t fly on May 1 after all. Change the date of the booking to May 3.
Quiz
Which one of the following is a valid usability test task for Ryanair’s
website for travellers?
A. Look around on Ryanair’s website. Is it easy to use?
B. Is Ryanair’s price for a return flight from London to Madrid on July 9
reasonable?
C. Find out how long before departure you have to be at the gate.
D. Find Ryanair’s most recent press release.

15. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 15
Usability test: The ”think-aloud” method
REPORT THE RESULTSPREPARE FOR THE
USABILITY TEST
Write a usability test plan
Write the usability test script
> Include usability test tasks
Recruit test participants
CONDUCT THE USABILITY
TEST SESSIONS
Briefing: “We are not testing you!”
Pre-session interview
Solve usability test tasks
Post-session interview
Write the usability test report
Communicate the usability
findings
Usability test of a prototype

16. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 16
“Paper Prototyping Training Video” by Nielsen Norman Group, www.nngroup.com
The human-centred software development cycle
ANALYSIS:
UNDERSTAND
AND SPECIFY
THE CONTEXT OF USE
SPECIFY THE USER
REQUIREMENTS
PRODUCE DESIGN
SOLUTIONS TO MEET
USER REQUIREMENTS
USABILITY TEST THE
DESIGNS AGAINST
USER REQUIREMENTS
PLAN THE HUMAN-
CENTERED DESIGN
PROCESS
DESIGN SOLUTION
MEETS USER
REQUIREMENTS

17. Testing State 2018, Kiev, Ukraine: If the users can’t use it, it doesn’t work
Slides Copyright 2018 Rolf Molich, page 17
If you want to know more ...
► CPUX-F curriculum: Usability explained in 60 pages
Certification and free quiz questions that allow you to check your understanding are available
Available at no cost from UXQB: uxqb.org/en/documents/
► CPUX-UT curriculum: Usability testing explained in 60 pages
Certification and free quiz questions that allow you to check your understanding are available
Available at no cost from UXQB: uxqb.org/en/documents/
► CTFL Usability Tester curriculum: Usability testing explained in 60 pages
Certification and free quiz questions that allow you to check your understanding are available
Available at no cost from GTB/ISTQB: www.german-testing-board.info/lehrplaene/istqbr-certified-
tester-schema/lehrplaene/
► Steve Krug: Rocket surgery made easy (book)
► Carol Barnum: Usability testing essentials (book)
Summary
► Usability:
 Can the product do what *I* want?
 Can it do what I want fast?
 Is the product pleasant to use?
► Discussions about usability should be based on real user data, not opinions
► You can get real user data from usability tests
► In a usability test, representative users solve representative tasks
while they are being observed by a neutral moderator
► Usability test
 Prepare the usability test
 Conduct usability test sessions
 Report the results
Voting system: Shakespeak.com