These slides provide an introduction to usability testing for digital marketeers. This well-known method in user-centred design is used to improve products, by having participants interact with these products and by measuring their performances and responses. I presented this topic as a guest lecturer to students attending the Minor Digital Marketing at the Fontys ICT Eindhoven at April 5th, 2017. Providing examples and best practices from Dutch digital design agency Mirabeau, I explained to them the required steps for the preparation, the moderation, and the analysis of usability tests.

1. An Introduction to
Usability Testing.
Lennart Overkamp
April 5th, 2017
Fontys Eindhoven.

2. 2010-2013: Bachelor Psychology @UT
2013-2014: Master Human Factors & Engineering Psychology @UT
2014-2016: Post-Master User System Interaction @TU/e
2016-now: Interaction Designer @Mirabeau

5. Building a platform
from the ground up
Fully responsive platform
Mobile, iPad & desktop
Booking flow redesign
Overview and options for customers

6. Fully responsive platform
Mobile, iPad & desktop

7. Booking flow redesign
Overview and options for customers

8. Redesign Nuon.nl
Relevance & customisation
Personal product selector
High-quality service
Quick, accessible & contextual
Inspirational
Telling a story through design

9. Relevance & customisation
Personal product selector

10. High-quality service
Quick, accessible & contextual

11. Inspirational
Telling a story through design

12. Redesign Vrouw.nl
Human2Human strategy
Content adapts to customer behaviour
Omni-channel approach
Seamlessly from magazine to online
Online look&feel
Connection to product portfolio

13. Human2Human strategy
Content adapts to customer behaviour

14. Omni-channel approach
Seamlessly from magazine to online

15. Online look&feel
Connection to product portfolio

17. Creative
Consultants
Interaction
Designers
Visual
Designers
Front-end
Developers
UX

19. The Red Thread?

20. An Introduction to
Usability Testing.
Lennart Overkamp
April 5th, 2017
Fontys Eindhoven.

21. Three Phases:
1. Preparation
2. Moderation
3. Analysis

22. Phase #1: Preparation.

23. “
”
The primary goal of the usability test
is to improve the usability
of the product that is being used.
Source: A Practical Guide to Usability Testing (Dumash & Redish, 1999)
Step #1.1: Define the goal.

25. Step #1.2: Define the users.
Users = ?
All (prospective) customers

26. Step #1.2: Define the users.
All (prospective) customers
Users = short-stay international
students

27. Step #1.2: Define the users.
Users = short-stay international
students
(the population)
All (prospective) customers

28. Step #1.3: Recruit users.
Users = short-stay international
students
(the population)
Participants (the sample)
All (prospective) customers

29. Step #1.4: Define the measures.
Performance measures
(e.g. time-on-task,
errors)
Preference measures
(e.g. desirability,
perceived usability)

30. Step #1.5: Prepare the procedure.
Which materials?
How?
When?
Where?
How long?
With whom?

32. Step #1.6: Prepare the script.
Scenario (example):
For a three-months exchange program in the Netherlands, you are looking for a
place to live from February 2017 until the end of April 2017. You have a maximum
budget of 500 euros, and would like to have a space of at least 30 square meters.
Tasks (examples):
1. On the website, find a place that fits your needs.
2. Calculate the time it takes from Central Station to this place by public transport.

33. Step #1.7: Prepare the prototype.
(If you have one)

34. Step #1.8: Prepare the consent form.

35. Step #1.8: Prepare the consent form.
Contents:
• Goal
• Procedure
• Duration
• Participant’s rights
EVALUATION VESTIDE.NL
Goal
The goal of this evaluation is to test the usability of vestide.nl, to improve
this website in terms of user-friendliness.
Procedure
You will carry out tasks with the website, which will be provided by the
facilitator. While carrying out the tasks, you are asked to think aloud.
After completing the tasks, you will fill in a short questionnaire and the
facilitator will ask you some questions. The evaluation will take around 45
minutes.
You may stop, take a break, or ask questions at any moment.
Recording
To facilitate the analysis of the test results, we ask for your permission to
record the evaluation. The recording will only be used by the design team
to improve the prototype, and will not be shared with anyone outside the
team.

36. Recording
To facilitate the analysis of the test results, we ask for your permission to
record the evaluation. The recording will only be used by the design team
to improve the prototype, and will not be shared with anyone outside the
team.
Participation does not have any risks or consequences for you.
• Your answers and details will be processed anonymously.
• We judge the quality of the prototype, not your performances.
Informed consent
By signing this form, you acknowledge to have read and understood the
information in this document, and to agree with this information.
Name:
Date:
Signature:
Do you agree with recording?
☐ Yes, video- and sound recording.
☐ Yes, only sound recording.
☐ No, rather not.
Step #1.8: Prepare the consent form.
Contents:
• Recording explanation
• Consequences
• Signature
• Recording consent

37. Step #1.9: Test your test.

38. Phase #2: Moderation.

39. Step #2.1: Welcome your participant.
Coffee?

40. Step #2.2: Introduce the test.
Introduce yourself.
Explain your role.
Explain what is expected of the participant.
Mention the duration of the test.
Provide reassurance: you are not testing the participant.
Explain the purpose of the test.

41. Step #2.3: Obtain informed consent.
Sign two copies!

42. Step #2.4: Explain the think-aloud protocol.
“
”
In a thinking aloud test, you ask test participants
to use the system while continuously thinking out
loud — that is, simply verbalizing their thoughts
as they move through the user interface.
Source: Jakob Nielsen (www.nngroup.com)

43. Step #2.4: Explain the think-aloud protocol.
Example: https://www.youtube.com/watch?v=g34tOmyKaMM

44. Step #2.5: Start recording.

48. Step #2.6: Participant performs tasks.
Explain the scenarios & tasks.
Let the participant do the talking!
Remind participant to think aloud, when necessary.
Keep questions and distractions to a minimum.
When a scenario/task is finished, explain the next one.

49. Step #2.7: Post-questionnaire.
(Optional, depending on goal/measures)
Example #2:
Microsoft Reaction
Card Method
Measures
product desirability.
Example #1:
System Usability
Scale (SUS)
Measures
perceived usability.

50. Step #2.8: Post-interview.
Ask for a general summary.
Elaborate on interesting observations/comments.
Ask any remaining questions you prepared beforehand.

51. Step #2.9: Debriefing.
Explain (again) the purpose of the test.
Answer any remaining questions from the participant.
Explain (again) what you will do with the results.
Give the participant a reward!

52. Phase #3: Analysis.

53. Quantitative data
(SUS scores, errors,
Reaction Cards,
time-on-task)
Qualitative data
(comments,
explanations,
responses)
Step #3.1: Process your data.

54. Step #3.2: Prioritise the usability problems.
Problem #1
Problem #2
Problem #3
Problem #4
Problem #5

55. Step #3.3: Communicate your results.
Type #2:
Presentation slides
Summarised, easily
shared or presented
Type #1:
Full report
Detailed, useful
as a reference.
Type #3:
Infographic
Highly visual, useful
as a persuasion tool

57. #Conclusion.
Advantages:
Solid, well-researched
Tests many aspects of product
Both qualitative & quantitative
Disadvantages:
Costly in terms of time, money
and (human) resources

58. Recommended resources - Books

59. Recommended resources - Online
smashingmagazine.com
nngroup.com

60. Recommended resources - Prototyping & Testing
InVisionSketch

61. Recommended resources - Analysing
ExcelRStudio

62. Recommended resources - Communicating results
SketchKeynoteWord

63. Neuro Usability Test.
Bonus:

65. Aspect #1:
Eye Tracking
Reference of users’ focus.
Input for heat maps.
Aspect #2:
EEG
Captures brain activity
to pinpoint affects.
Aspect #3:
Neuro Feedback
Users comment on
significant EEG findings.
Aspect #4:
Questionnaire
Users are interviewed and
fill in a questionnaire.

66. Thank you.