Design usability tests to make data-driven design decisions by John McGloon

Design usability tests to make
data-driven design decisions
John McGloon
Teresa Washburn

© 2013 Teresa John McGloon
© 2013 Teresa Washburn andWashburn and John McGloon

Things we’ll cover
•
•
•
•
•
•
•

Write better screeners by linking personas
Prioritize with task sampling and selection
Write a test script
Use various prototypes
Facilitate a test
Plan for observers
Incorporate additional methods (card sorts,
system usability scales)
• Summarize a usability test

Step 1: Have a plan
• What are your goals?
• What does success look like?
– Meet with your stakeholders and define success

• Why should you define success with
stakeholders?
– Gets everyone on the same page
– When observing, everyone should be coming from
the same position
– After the test, there will be less chance of
miscommunication

What are personas?
• Personas are a guide to help the team keep
the user top of mind
• Audience and purpose
• Make generalizations!


Persona map


Link personas to your screener!
• Write screener questions that only Taylor
Biggs could answer
• Watch out for Taylor imposters
• Constructing the right screener will get you
the right participant
• Make better design decisions based on the
right set of participants


Prioritize with task sampling and
selection
• How do you decide what to have the
participant do?
– What are your goals?

• Complete a task analysis
Goal

Task

Task

Subtask

Task

Subtask
© 2013 Teresa Washburn and John
© 2013 Teresa Washburn and John McGloon McGloon

Task analysis: Checkout
2.0
Scan
products
(repeat for
ea. Item)

2.1
Locate
barcode on
product

2.1.1
No
barcode
located

2.2
Place
Barcode in
infrared
beam

2.3
Rotate until
price
registers
(“beep”)

2.4
Verify
scanned
price
=expected
price)

2.3.1
Price does
not register

2.4.1
Price is not
correct
Go to 2.6
(FAIL)


Write the Test Script
• Script everything – even your greeting.
Consistency is key!
• Start with your goals
• Consider how learnable your task is
– Switch the order of tasks

• What are you trying to observe?
– Be careful not to help them if they get stuck


Use various prototypes
• Prototyping: keep it simple
• Don’t have enough budget? Consider paper
prototyping.


Facilitate a test
• Facilitating is an art
• Professionals learn as an apprentices with little
formal practice
• The importance of “Think aloud”
• Dumas’ and Loring’s (10) Golden Rules:
– Participants are the experts; you are in charge.
– Let the participants speak!
– Be unbiased.

• Moderator role: all logistics, pacing, interaction

Plan for observers
Pros

Cons

See the usability items under
consideration
Always surprised by what they see

Can be distracting

May build confidence in UX

May latch on to single observation

Can make participants feel
uncomfortable


Incorporate additional methods (card
sorts, system usability scales)
• Card sorts
• System Usability scales
• Product reaction cards


© 2013 Teresa Washburn and John McGloon

Card sorts
Card sorts


Card sorts: results


System Usability Scale


System Usability Scale results
Aggregate usability score:

81

19


Product reaction cards
Accessible

Desirable

Gets in the way

Patronizing

Stressful

Appealing

Easy to use

Hard to use

Personal

Time-consuming

Attractive

Efficient

High quality

Predictable

Time-saving

Busy

Empowering

Inconsistent

Relevant

Too technical

Collaborative

Exciting

Intimidating

Reliable

Trustworthy

Complex

Familiar

Inviting

Rigid

Uncontrollable

Comprehensive

Fast

Motivating

Simplistic

Unconventional

Confusing

Flexible

Not valuable

Slow

Unpredictable

Connected

Fresh

Organized

Sophisticated

Usable

Consistent

Frustrating

Overbearing

Stimulating

Useful

Customizable

Fun

Overwhelming

Straight Forward

Valuable


Product reaction cards: results


Summarize a usability test:
sample outline
1. Overview or executive summary
2. Goals
3. Methodology
4. Participant Profile
5. Tasks
6. Findings / Recommendations
7. Appendix
(And the “question” of highlights.)

Summarize a usability test
• Audience and purpose
• Budget and timing
• Organizing
– By severity
– By category
– Content
– By action required


Overview Screen: Usability Issues
Update Numbers
Some participants were
unsure what the update
numbers referred to.
One thought they
represented summary
info of the maintenance
alerts below. Others
thought they referred to
the number of devices
in each of those states.
Severity
3 - Moderate

25


Questions about anything we’ve
covered:
•
•
•
•

•
•
•
•

Write better screeners by linking personas
Prioritize with task sampling and selection
Write a test script
Use various prototypes (no prototyping will be
done, but examples will be provided)
Facilitate a test
Plan for observers
Incorporate additional methods (card sorts,
system usability scales)
Summarize a usability test

Further reading
• Moderating Usability Tests: Principles and
Practices for Interacting (Interactive
Technologies) by Joseph Dumas and Beth
Loring.
• User and Task Analysis for Interface Design by
JoAnn Hackos, and Janice Redish.
• A Practical Guide to Usability Testing by
Joseph Dumas and Janice Redish.

Interested to learn more?
If you would like more information about
usability testing, the user experience field, or
how technical communication fits with those,
feel free to provide your contact information
and we’ll be in touch.
Thank you!


Design usability tests to make data-driven design decisions by John McGloon

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (16)

Similar to Design usability tests to make data-driven design decisions by John McGloon

Similar to Design usability tests to make data-driven design decisions by John McGloon (20)

Recently uploaded

Recently uploaded (20)

Design usability tests to make data-driven design decisions by John McGloon

Editor's Notes