Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
UXPA Conference Panel, London 2014:
"Remote, unmoderated
testing is as reliable as
lab-based testing" — and
other controve...
Presentation of panelists

Jakob Biesterfeldt
• Managing Director of
UserZoom Germany,
provider of UX research tools
•...
Presentation of panelists

David Travis
• Managing Director of Userfocus, a
London-based UX consultancy
• Author of ‘E...
Presentation of panelists

Steve Krug
• Author of Don’t Make Me Think
and Rocket Surgery Made Easy
• Usability consult...
Presentation of panelists

Rolf Molich
• Owner, manager of DialogDesign,
a small Danish usability
consultancy
• Manage...
Please use a few minutes to mark your position on each of the 10 statements
7
Procedure
To keep the discussion lively and cover as many
statements as possible, the procedure will be:
► A statement i...
1
Truth or myth?
Remote, unmoderated testing is
as reliable as lab-based testing
Agree
Both agree & disagree
Disagree
2
Truth or myth?
Five users are enough to catch
85% of the usability problems in
practically any product
Agree
Both agree ...
2
Truth or myth?
Five users are enough to catch
85% of the usability problems in
practically any product
Agree
Both agree ...
”The parabola of optimism” (Jared Spool)
12
Real data: CUE studies
CUE = Comparative Usability Evaluation
► 10-17 professional teams test the same website.
► In al...
Five users are
enough to drive a
useful iterative
cycle
3
Truth or myth?
The moderator should say as little
as possible while the test
participant is working with the test
tasks
...
3
Truth or myth?
The moderator should say as little
as possible while the test
participant is working with the test
tasks
...
4
Truth or myth?
Report only the 3-5 most serious
problems to the client. Tell them
that you would be happy to give
them m...
5
Truth or myth?
It's simple to identify disastrous
usability problems
Agree
Both agree & disagree
Disagree
18
Watch two video clips
From Comparative Usability Evaluation 9 (CUE-9)
► Usability test of U-Haul.com
► Participants hav...
6
Truth or myth?
Observers should be told to keep
quiet if loud sounds from the
observation room can be heard in
the test ...
7
Truth or myth?
Usability testing is an iterative
process - one-off tests are useless
Agree
Both agree & disagree
Disagree
8
Truth or myth?
Users have valuable insight into
why they struggle
Agree
Both agree & disagree
Disagree
9
Truth or myth?
Showing picture-in-picture of the
participant’s face is an
unnecessary distraction for
observers
Agree
Bo...
10
Truth or myth?
If you've found 90 serious or
critical problems in a usability test,
it's OK to report them all in the
u...
Evaluation
Did this presentation change some
of your views of usability testing?
Yes
Don’t know
No
11
25
Main Takeaways
► As usability professionals we must set a good example for
the world. Our products, presentations, doin...
Upcoming SlideShare
Loading in …5
×
Upcoming SlideShare
Adapting to Responsive Design - Matt Gibson
Next
Download to read offline and view in fullscreen.

0

Share

Download to read offline

Remote, unmoderated testing is as reliable as lab-based testing - and other controversial beliefs about usability testing (Rolf Molich, David Travis, Jakob Biesterfeldt, Steve Krug)

Download to read offline

Usability testing is by far the most widely used usability method. Nonetheless, it's often conducted with poor or unsystematic methodology and thus doesn't always live up to its full potential. This panel will present a lively discussion about a number of controversial beliefs about usability testing and discuss if they are truths or myths.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

Remote, unmoderated testing is as reliable as lab-based testing - and other controversial beliefs about usability testing (Rolf Molich, David Travis, Jakob Biesterfeldt, Steve Krug)

  1. 1. UXPA Conference Panel, London 2014: "Remote, unmoderated testing is as reliable as lab-based testing" — and other controversial beliefs about usability testing Rolf Molich David Travis Jakob Biesterfeldt Steve Krug
  2. 2. Presentation of panelists  Jakob Biesterfeldt • Managing Director of UserZoom Germany, provider of UX research tools • 12+ years of UX research experience • Focus on professionalization of UX research
  3. 3. Presentation of panelists  David Travis • Managing Director of Userfocus, a London-based UX consultancy • Author of ‘E-Commerce Usability’ • PhD in Psychology • Worked as an internal and external usability consultant since 1989 • Logged over 1500 hours in usability tests and trained over 500 students in usability testing
  4. 4. Presentation of panelists  Steve Krug • Author of Don’t Make Me Think and Rocket Surgery Made Easy • Usability consultant for 25 years • Has taught thousands of people to do their own usability testing • Thinks people rarely change their minds about anything
  5. 5. Presentation of panelists  Rolf Molich • Owner, manager of DialogDesign, a small Danish usability consultancy • Managed the Comparative Usability Evaluations (CUE-1 to 9) • 30+ years of usability experience • Focus on Quality and Professionalism in usability work
  6. 6. Please use a few minutes to mark your position on each of the 10 statements
  7. 7. 7 Procedure To keep the discussion lively and cover as many statements as possible, the procedure will be: ► A statement is presented ► Attendees vote for or against ► The supporting panelist speaks for at most 2 minutes ► The opposing panelist speaks for at most 2 minutes ► 1-2 comments from the audience Please queue at the microphone. Limit yourself to 30 secs ► Attendees vote again
  8. 8. 1 Truth or myth? Remote, unmoderated testing is as reliable as lab-based testing Agree Both agree & disagree Disagree
  9. 9. 2 Truth or myth? Five users are enough to catch 85% of the usability problems in practically any product Agree Both agree & disagree Disagree
  10. 10. 2 Truth or myth? Five users are enough to catch 85% of the usability problems in practically any product Agree Both agree & disagree Disagree
  11. 11. ”The parabola of optimism” (Jared Spool)
  12. 12. 12 Real data: CUE studies CUE = Comparative Usability Evaluation ► 10-17 professional teams test the same website. ► In all CUE-studies, more than 60% of problems were uniquely reported. ► In CUE-4, more than 300 issues were reported.
  13. 13. Five users are enough to drive a useful iterative cycle
  14. 14. 3 Truth or myth? The moderator should say as little as possible while the test participant is working with the test tasks Agree Both agree & disagree Disagree
  15. 15. 3 Truth or myth? The moderator should say as little as possible while the test participant is working with the test tasks Agree Both agree & disagree Disagree
  16. 16. 4 Truth or myth? Report only the 3-5 most serious problems to the client. Tell them that you would be happy to give them more problems to work on, but ONLY after they had fixed the most serious ones. Agree Both agree & disagree Disagree
  17. 17. 5 Truth or myth? It's simple to identify disastrous usability problems Agree Both agree & disagree Disagree
  18. 18. 18 Watch two video clips From Comparative Usability Evaluation 9 (CUE-9) ► Usability test of U-Haul.com ► Participants have been asked to rent a moving truck ► Participants select rental location, then damage coverage ► Look out for potentially disastrous usability problem
  19. 19. 6 Truth or myth? Observers should be told to keep quiet if loud sounds from the observation room can be heard in the test room Agree Both agree & disagree Disagree
  20. 20. 7 Truth or myth? Usability testing is an iterative process - one-off tests are useless Agree Both agree & disagree Disagree
  21. 21. 8 Truth or myth? Users have valuable insight into why they struggle Agree Both agree & disagree Disagree
  22. 22. 9 Truth or myth? Showing picture-in-picture of the participant’s face is an unnecessary distraction for observers Agree Both agree & disagree Disagree
  23. 23. 10 Truth or myth? If you've found 90 serious or critical problems in a usability test, it's OK to report them all in the usability test report Agree Both agree & disagree Disagree
  24. 24. Evaluation Did this presentation change some of your views of usability testing? Yes Don’t know No 11
  25. 25. 25 Main Takeaways ► As usability professionals we must set a good example for the world. Our products, presentations, doings must be usable and useful: Correct, relevant, brief, constructive ► Five users are enough to drive a useful iterative cycle. Never claim that you can find all usability problems ► Moderators should say as little as possible while the test participant is working with the test tasks in order not to provide inadvertent clues ► The usability of a test report is influenced by the number of problems reported

Usability testing is by far the most widely used usability method. Nonetheless, it's often conducted with poor or unsystematic methodology and thus doesn't always live up to its full potential. This panel will present a lively discussion about a number of controversial beliefs about usability testing and discuss if they are truths or myths.

Views

Total views

806

On Slideshare

0

From embeds

0

Number of embeds

3

Actions

Downloads

8

Shares

0

Comments

0

Likes

0

×