Need a little usability?
What you can learn from usability testing
Whitney Quesenbery
Center for Civic Design
2
What questions do you
want to answer with a
usability test?
3
Usability testing answers questions, so we have to
understand the question we are asking.
If we want to know:
Which desi...
4
Helpful models for thinking about usability
A’s Ability, aptitude, attitude
(what users bring to the interaction)
E’s Ef...
5
The As help you think about diverse participants
Attitude
motivation, emotion, risk tolerance, persistence, optimism, to...
6
Different users – and different contexts of use – can
change user needs
7
5Es help you plan usability tests
Usability Goal Types of usability testing needed
Evaluate tasks for how accurately the...
8
The recipe for planning a usability test
Location and context: formal informal
Recruiting: defined opportunistic
Activit...
9
A few examples of questions
and how we answered them
These examples all come from work on
elections and civic design..bu...
10
Comparing two designs
Can a design improve user performance?
Plain vs. traditional language
About this project
 45 par...
11
Comparing two designs
Can a minor difference have a major impact?
Left aligned or right aligned
names
About this projec...
12
Testing a new design
What problems might voters have with a new ballot design?
About this project
 1 of 5 similar test...
13
Exploring new ideas
Can we extend a design to new audiences?
http://anywhereballot.org
http://civicdesigning.org/featur...
14
Answers quickly and inexpensively
Can we find participants "where they are?"
http://anywhereballot.org
http://civicdesi...
15
Learning about comprehension
Can voters explain and act on that understanding?
About this project
 20 participants
 4...
16
Gauging opinion
Which accessible voting system do voters prefer?
About this project
 496 surveys; multi-disabilit
 11...
17
Planning a
usability program
18
Look for a balance of methods by approach and
context of use
19
Early in
Project
Many Users
Few Users
Comparative
benchmarking
Summative testing
Card sorting
Satisfaction surveys
Mark...
20
Look for the 'cadence' of what kind of usability work
you do at different times
Seeing the Elephant: Defragmenting User...
21
Things to watch out for when
planning a usability test
22
Recruiting for diversity
Gender
Race/ethnicity
Income/education
Disability
23
Treating participants with respect
Language and cultural issues
Managing context
Participant protections and informed c...
24
Avoiding bias
Leading questions
Closed vs. open questions
Sampling bias
Moderator influence
Observer (or moderator reac...
25
Whitney Quesenbery
whitneyq@wqusability.com
www.wqusability.com
@whitneyq
Center for Civic Design
whitneyq@centerforciv...
26
Storytelling for User Experience
with Kevin Brooks
Global UX
with Daniel Szuc
A Web for Everyone
with Sarah Horton
http...
Upcoming SlideShare
Loading in …5
×

What you can learn from usability testing

2,694 views
2,373 views

Published on

Every study starts with a question. This session at CSUN 2014 started by examining the questions that usability testing can answer. Short case studies illustrate how the right technique will help us know not only what is happening but also why it’s happening. It's an overview of usability testing as a research method, and what you can (and can’t) learn from working with real people as they try to use a web site or other product.

Published in: Design
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,694
On SlideShare
0
From Embeds
0
Number of Embeds
340
Actions
Shares
0
Downloads
34
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • A tool to help you focus on the attributes that matterAccount for what the user brings to the tableIdentify both center and outliers
  • The 'Anywhere Ballot'Designed through iterative testing with low literacy participants
  • On April 4th, 2005, citizens with disabilities from all over the state tested voting systems and completed surveys documenting their reactions to each system. A total of 496 surveys were completed.The survey results on the following page will be one of several factors used to help the Oregon Secretary of State decide which voting system or systems to purchase to give citizens with disabilities the opportunity to vote privately and independently. Survey was a modified SUS – 12 questionsI would like to use this voting system in an election. There were too many steps in using this voting system.I thought this voting system was easy to use. The instructions for this voting system were difficult to understand. The buttons or touch screen were easy to use. It was hard to move around the ballot with this system. I think that most people could learn to use this voting system very quickly.I found this voting system awkward or difficult to use. I felt very confident that my vote was cast correctly with this voting system. I would need help each time I used this voting system. The ballot text was easy to read or hear. Casting my vote was easy.
  • What you can learn from usability testing

    1. 1. Need a little usability? What you can learn from usability testing Whitney Quesenbery Center for Civic Design
    2. 2. 2 What questions do you want to answer with a usability test?
    3. 3. 3 Usability testing answers questions, so we have to understand the question we are asking. If we want to know: Which design helps users complete tasks more effectively? We are asking a question that is Comparative, quantitative (errors, time) (and why they made those mistakes) If we want to know: Do voters understand how to mark their ballot? We are asking a question that is abut Mental models, variability, qualitative insights (and how prevalent each model is)
    4. 4. 4 Helpful models for thinking about usability A’s Ability, aptitude, attitude (what users bring to the interaction) E’s Efficient, effective, engaging, error-tolerant, easy to learn (dimensions of usability) Use Useful, usable (operable), desirable (context of use)
    5. 5. 5 The As help you think about diverse participants Attitude motivation, emotion, risk tolerance, persistence, optimism, tolerance for frustration Aptitude current knowledge, ability to make inferences or innovate solutions, expertise, habits Ability physical and cognitive attributes Source: Dana Chisnell "Character Creator" http://www.slideshare.net/danachisnell/character-creator
    6. 6. 6 Different users – and different contexts of use – can change user needs
    7. 7. 7 5Es help you plan usability tests Usability Goal Types of usability testing needed Evaluate tasks for how accurately they were completed and how often they produce errors Effective Error-tolerant Easy to learn Control how much instruction is given to test participants, or recruit participants with different levels of knowledge Construct task scenarios to create situations with potential problems Engaging User satisfaction surveys to gauge acceptance review logs for „time on site‟ Efficient Time tasks with realistic tasks and working versions of the software
    8. 8. 8 The recipe for planning a usability test Location and context: formal informal Recruiting: defined opportunistic Activities: instructed tasks free tasks Questions: structured unstructured Data collection: observation task/data only Results quantitative qualitative
    9. 9. 9 A few examples of questions and how we answered them These examples all come from work on elections and civic design..but the principles apply to any project.
    10. 10. 10 Comparing two designs Can a design improve user performance? Plain vs. traditional language About this project  45 participants  3 lab setting, 8 days  2 moderators  Instructions for voting  Careful observation  Satisfaction survey We learned about  Voter accuracy  Order effects  Voter preference Results  Statistical analysis  Participant preference
    11. 11. 11 Comparing two designs Can a minor difference have a major impact? Left aligned or right aligned names About this project  100 participants  2 locations, 1 day  4 moderators + 6 others  Instructions for voting  Minimal observation We learned about  Voter preference  Time to vote  Number of errors Results  Statistical analysis of marked ballots
    12. 12. 12 Testing a new design What problems might voters have with a new ballot design? About this project  1 of 5 similar tests  10 participants  1 locations, 1 day  2 moderators + 1 others  Self-defined tasks  Observation  Post-task questionnaire We learned about  Navigation on the ballot  Instructions Results  Found pattern of errors for one interaction  Time on task
    13. 13. 13 Exploring new ideas Can we extend a design to new audiences? http://anywhereballot.org http://civicdesigning.org/featured-story/rapid-responsive-radical-the-anywhere-ballot-is-born/ About this project  Research project  Paper prototyping  Low literacy  We learned about  What they understsand  Navigation  Instructions  Interactions  Results  Simplified everything for a clear, universal design
    14. 14. 14 Answers quickly and inexpensively Can we find participants "where they are?" http://anywhereballot.org http://civicdesigning.org/featured-story/rapid-responsive-radical-the-anywhere-ballot-is-born/ About this project  13 participants  Observed and interviewed as they filled in a new voter registration form  We learned about  How people filled in the form  Results  Improved voter registration forms This project and poster were done by Jenny Greeve, the Design Fellow at the Washington Department of Elections
    15. 15. 15 Learning about comprehension Can voters explain and act on that understanding? About this project  20 participants  4 locations, 1 day  8 moderators  Self-defined tasks  Minimal observation We learned about  Navigation on the ballot  Instructions  Mental models of counting  Whether there‟s a problem Results  Qualitative insights
    16. 16. 16 Gauging opinion Which accessible voting system do voters prefer? About this project  496 surveys; multi-disabilit  11 systems  Multiday “voting fair” They learned about  Voter assessment of usability  Voter comments about accessibility  Differences by disability Results  Statistical analysis survey results  Qualitative analysis of comments This was a project of the Oregon Department of Elections
    17. 17. 17 Planning a usability program
    18. 18. 18 Look for a balance of methods by approach and context of use
    19. 19. 19 Early in Project Many Users Few Users Comparative benchmarking Summative testing Card sorting Satisfaction surveys Market research Iterative evaluation of prototypes Evaluation of Designs Ethnography Use the right tool at the right time Stakeholder interviews User observation Walkthroughs Expert Reviews A-B Testing
    20. 20. 20 Look for the 'cadence' of what kind of usability work you do at different times Seeing the Elephant: Defragmenting User Research by Lou Rosenfeld http://alistapart.com/article/seeing-the-elephant-defragmenting-user-research
    21. 21. 21 Things to watch out for when planning a usability test
    22. 22. 22 Recruiting for diversity Gender Race/ethnicity Income/education Disability
    23. 23. 23 Treating participants with respect Language and cultural issues Managing context Participant protections and informed consent
    24. 24. 24 Avoiding bias Leading questions Closed vs. open questions Sampling bias Moderator influence Observer (or moderator reactions)
    25. 25. 25 Whitney Quesenbery whitneyq@wqusability.com www.wqusability.com @whitneyq Center for Civic Design whitneyq@centerforcivicdesign centerforcivicdesign.org @ChadButterfly
    26. 26. 26 Storytelling for User Experience with Kevin Brooks Global UX with Daniel Szuc A Web for Everyone with Sarah Horton http://rosenfeldmedia.com/books/a-web-for-everyone/

    ×