These slides were used during a workshop on Usability Testing - an intro. We covered the following topics: 1) What is user experience? 2) Why is usability important 3) How do we evaluate usability?

1. Usability Testing:
Gaining people’s insight
DX Toronto Workshop 2014

3. Adie
Linn
Dale
Nick

4. Topics
• What is user experience?
• What is usability?
• Why is usability important?
• How do we evaluate usability?
• Things to keep in mind
• The workshop

5. User Experience (UX)
User experience is how
useful and usable a product,
system or service is and how
people feel about it.

6. What is usability?
• Learnability
• Efficiency
• Memorability
• Recoverability
• Satisfaction
Nielsen Norman Group

7. Why is usability important
People will leave…
…If it’s difficult
…If they get lost
…If they find it hard to read

8. How do we evaluate
Usability?
• Recruit some representative
users
• Ask users to perform tasks
• Observe users successes and
failures

9. Types of
Usability Testing
• Low fidelity
• Formal
• Remote
• Intercept
• Accessibility

10. Accessibility vs Usability
• An accessible website will
benefit all users
• Websites aren’t usable until
they’re accessible
• Usability implies accessibility,
not always in reverse

11. Accessibility Testing vs
Usability Testing
• Methodology is the same
• Recruiting is more specific
• Tech set up is often very
personal to the user

16. Overview
• Recruiting
• Preparing
• Logistics (what and where)
• Facilitating
• Capturing
• Analyzing
• Reporting

17. Be sure to…
• Test users individually
• Run several small tests
• User testing is NOT
a focus group

18. Remember to…
• Recruit correctly
• Think about privacy
• Incentive
• Thank you!

19. Devices & Recording
• Mobile
• Desktop
• Paper prototype
• Smart watch

20. Time for some drama!

22. Running a Usability Test
Usability Tests help you to validate your design decisions with real users. They are a key
ingredient to creating a successful experience.
The Tip Sheet
• Create research objectives
• Identify recruitment specs
• Write recruitment screener
• Test plan and script
Prepare Test Plan
1
.!0!ƫ.!/!.$ƫ+!0%2!/
Identify recruitment specs
.%0!ƫ.!.1%0)!*0ƫ/.!!*!.
10$+.ƫ0!/0ƫ,(*ƫ* ƫ/.%,0
2 Logistics
++'ƫƫ.++)ĥ%(%05
Recruit participants
..*#!ƫ0!$*%(ƫ/!0ƫ1,
Test plan run through
3 Execute Test
Invite stakeholders to observe
Facilitate usability sessions
Observe and record results
$*'ƫ,.0%%,*0/Čƫ,.+2% !ƫ%*!*0%2!
Analyze Results
Review transcripts analyst notes
Synthesize data - identify themes
,01.!ƫ.!/1(0/ƫġƫ.!,+.0ƫ3.%0%*#
.!/!*0ƫü* %*#/Čƫ %/1//ƫ*!40ƫ/0!,/
4

23. The workshop!

24. Usability Test Roles
• Facilitator
• Note taker
• Analyst
• Participant

25. Workshop Steps
1) Define research objectives
2) Define the tasks
3) Create the test script
4) Conduct the usability test
5) Analysis and report
6) Discuss findings and
recommendations

26. About objectives
• The client
• Define research objectives
• Examples

27. Step 1:
Define 3 research objectives:
“What do we want to learn about this site
and how participants are using it?”
Example:
“How easily can the participant locate the
search field in its current position?”

28. About tasks
• Based on objectives
• Specific
• Examples

29. Step 2:
Define 3 Tasks:
“What are the primary tasks our users will
do on this site?”
Example:
“Using the site, show me how you would
find upcoming events on August 14, 2014.”

30. About the script
• Intro
• Tasks
• Wrap up

31. Step 3:
Create the script:
Based on the tasks, create the test script
that the facilitator will read from during the
test
Example:
“Using the device, show me how you would
search for an event.”

32. Facilitating
• Observation not conversation
• Don’t stay completely silent
• Don’t give away the answer
• Think aloud protocol

33. Probing Techniques
• Echo
• Boomerang
• Columbo

34. Capturing
• Recordings – screen, room
• Notes/transcripts
• Type of material needed –
Quotes? Video clips?

35. Step 4:
Conduct the test:
The facilitator will read the script and walk
the participant through the each task
Example:
“Using the device, show me how you would
search for an event.”

36. Analysis
• Carefully review data
• Qualitative data
• Recommendations

37. Reporting
• Detailed report
• Direct implementation
• Video reel

38. Step 5:
Analysis and Report:
The group will discuss findings and how to
present.
Example:
During our usability test we discovered this
and this. Here is our recommendation *think
back to the various reporting methods*

39. Thank you!