How to Conduct Usability Studies
A Librarian Primer
Tao Zhang, PhD
Digital User Experience Specialist
Purdue University Li...
Learning Objectives
2
•  Participants will be able to describe the usability
concept and it's applications to libraries.
•...
Agenda
3
•  What is a usability study?
•  Usability study process
•  Conducting the study
•  Analyzing Results
•  Communic...
About Us
Ilana Barnes Tao Zhang
4
Tell us about you!
What type of library do you work in?
A)  Public
B)  Academic
C)  Other
How familiar are you with Usabil...
What is User Experience?
6
Definition
Usability study refers to“evaluating a
product or service by testing it with
representative users.“
- http://ww...
How UX fits into Reference Services
•  Decisions that user experience research can help
you with:
•  What should our libra...
How to plan your usability study topic
•  What are my objectives?
•  How much time do I have?
•  How many teammates do I h...
When to Test
•  Early or middle stages of development
•  Types:
Exploratory | Assessment | Comparison | Validation
•  Star...
Usability Study Process
11
Recrui'ng	
   Preparing	
   Conduc'ng	
   Analyzing	
   Presen'ng	
  
Choosing Features
Creatin...
Recruiting
•  Determining the target audience (behavior)
•  Finding representative members
–  Behavioral criteria, demogra...
Recruiting
Best people to invite:
•  People who are going to need the service in the
near future
•  People who have used a...
Choosing Features
•  Input from: Development team | Stakeholders | Users
•  Content inventory & task analysis
14
Creating Tasks
Tasks should be:
•  Reasonable and doable
•  Described in terms of end goals
•  Specific
Run through tasks ...
Discussion
You need to do a usability study of your library’s
website, because users find it difficult to find
information t...
Writing A Study Script
•  Introduction (5 min)
•  Warm-up interview, demographics survey (5 min)
•  Scenarios and tasks (2...
Conducting the Study
The physical layout with observer
18
Conducting the Study
The physical layout without observer
19
Conducting the Study
TechSmith Morae www.techsmith.com/morae.html
20
Recorder	
  
Observer	
  #1	
  
Participant Test Mode...
Conducting the Study
21
http://silverbackapp.com/
Conducting the Study
Testing on mobile devices
22
IPEVO Ziggi-HD High-Definition
USB Document Camera
DIY Mobile Usability ...
Moderation
•  Follow script but allow some flexibility
•  Encourage participants to“think aloud”
•  Ask about expectations...
Two Basic Techniques
Observation
Interview
24
Observation
•  Watch, Understand
•  Limit the time users have to perform a task
•  Encourage users to think aloud with neu...
Interview
•  Introduction
•  Warm-up
•  General issues
•  Deep focus
•  Wrap-up
26
Nondirected Interview
Prepare nondirected questions
•  Focused on the person answering it
•  Focused on experience, not ex...
Nondirected Interview
Examples
“Is this a useful feature?”
“Is this feature valuable to the work you do right now?”
“Do yo...
Nondirected Interview
Running the interview
•  Define (technical) terms
•  Don’t force opinions
•  Restate answers
•  Neve...
Analyzing Results
•  Collecting observations
–  Notes from moderator and observer
–  Quantitative information: task succes...
Analyzing Results
•  A list of usability issues
–  Quantitative and qualitative data as evidence
–  Possible causes
–  Effe...
Communicating Findings
Usability test report
•  Project overview
•  Participants
•  Testing environment
•  Tasks
•  Proced...
Communicating Findings
Presentation
•  Prepare the audience
•  Use supporting evidence (e.g., highlight videos)
•  Emphasi...
Communicating Findings
Common problems:
•  “This is not statistically significant.”
•  “This user is our typical user.”“Th...
What’s Essential?
•  Develop a study plan
•  Find participants
•  Conduct the sessions
•  Debrief with observers and decid...
Discussion
Think of an existing design that you want to evaluate
for effectiveness
•  What are my objectives?
•  What types...
Other Types of Usability Studies
•  Remote usability testing http://www.userzoom.com/
•  Eye tracking
37
http://www.normal...
Where to Find Out More
•  6 Must-See Usability Testing Videos
http://blog.usabilla.com/7-must-see-usability-testing-videos...
Learning Objectives
39
•  Participants will be able to describe the
usability concept and it's applications to
libraries.
...
Agenda
40
•  What is a usability study?
•  Usability study process
•  Conducting the study
•  Analyzing Results
•  Communi...
Works Cited
•  6 Must-See Usability Testing Videos - The Usabilla Blog. (n.d.). Retrieved from http://
blog.usabilla.com/7...
Final Notes
Evaluation of this webinar
Questions?
Tao Zhang zhan1022@purdue.edu
Ilana Barnes ibarnes@purdue.edu
42
Upcoming SlideShare
Loading in...5
×

How to Conduct Usability Studies: A Librarian Primer

423

Published on

Slides used for RUSA webinar on September 4, 2013

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
423
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

How to Conduct Usability Studies: A Librarian Primer

  1. 1. How to Conduct Usability Studies A Librarian Primer Tao Zhang, PhD Digital User Experience Specialist Purdue University Libraries zhan1022@purdue.edu Sponsored by ALA RUSA #RUSAUX Ilana Barnes Business Information Specialist Purdue University Libraries ibarnes@purdue.edu
  2. 2. Learning Objectives 2 •  Participants will be able to describe the usability concept and it's applications to libraries. •  Participants will be aware of the basic workflows and techniques needed when doing usability tests. •  Participants will be aware of resources for usability studies and where to find out more.
  3. 3. Agenda 3 •  What is a usability study? •  Usability study process •  Conducting the study •  Analyzing Results •  Communicating •  Where to find out more
  4. 4. About Us Ilana Barnes Tao Zhang 4
  5. 5. Tell us about you! What type of library do you work in? A)  Public B)  Academic C)  Other How familiar are you with Usability Studies/Tests? A)  Very familiar B)  Somewhat familiar C)  Not familiar but excited to learn! Chat box: Areas you especially want us to talk about 5
  6. 6. What is User Experience? 6
  7. 7. Definition Usability study refers to“evaluating a product or service by testing it with representative users.“ - http://www.usability.gov/how-to-and-tools/methods/usability-testing.html 7
  8. 8. How UX fits into Reference Services •  Decisions that user experience research can help you with: •  What should our libraries front page look like? •  What do our undergraduates expect from the library? •  What would we like our undergraduates to expect from the library that they do not know or understand or need? •  Understanding your users aims and needs help you help them and show your impact at the same time. 8
  9. 9. How to plan your usability study topic •  What are my objectives? •  How much time do I have? •  How many teammates do I have? •  Who am I presenting this to? •  Is this for an existing product? 9
  10. 10. When to Test •  Early or middle stages of development •  Types: Exploratory | Assessment | Comparison | Validation •  Start preparing at least three weeks before actual test •  DO pilot tests 10
  11. 11. Usability Study Process 11 Recrui'ng   Preparing   Conduc'ng   Analyzing   Presen'ng   Choosing Features Creating Tasks Preparing Test Script
  12. 12. Recruiting •  Determining the target audience (behavior) •  Finding representative members –  Behavioral criteria, demographics –  Avoid sampling bias •  Convincing participants to come –  Incentives –  Informed content (duration of study, data to collect) •  Scheduling –  Reminding –  Deal with no shows 12
  13. 13. Recruiting Best people to invite: •  People who are going to need the service in the near future •  People who have used a competing/previous version service in the recent past Number of participants: •  5-8 •  Also consider representativeness and tasks 13
  14. 14. Choosing Features •  Input from: Development team | Stakeholders | Users •  Content inventory & task analysis 14
  15. 15. Creating Tasks Tasks should be: •  Reasonable and doable •  Described in terms of end goals •  Specific Run through tasks to see how long each will take Determine what quantitative data to collect 15
  16. 16. Discussion You need to do a usability study of your library’s website, because users find it difficult to find information they want on the website. What are some possible tasks you can use in the usability study? 16
  17. 17. Writing A Study Script •  Introduction (5 min) •  Warm-up interview, demographics survey (5 min) •  Scenarios and tasks (20-30 min) •  Interview (10-15 min) •  Wrap-up (5 min) 17
  18. 18. Conducting the Study The physical layout with observer 18
  19. 19. Conducting the Study The physical layout without observer 19
  20. 20. Conducting the Study TechSmith Morae www.techsmith.com/morae.html 20 Recorder   Observer  #1   Participant Test Moderator Observer Manager   Observer  #2  …  
  21. 21. Conducting the Study 21 http://silverbackapp.com/
  22. 22. Conducting the Study Testing on mobile devices 22 IPEVO Ziggi-HD High-Definition USB Document Camera DIY Mobile Usability Testing - SXSW Interactive 2012 http://www.slideshare.net/beleniq/diy- mobile-usability-testing-sxsw- interactive-2012
  23. 23. Moderation •  Follow script but allow some flexibility •  Encourage participants to“think aloud” •  Ask about expectations •  Ask“why”a lot •  Suggest solutions when participants are stuck •  Discuss with observer right after test 23
  24. 24. Two Basic Techniques Observation Interview 24
  25. 25. Observation •  Watch, Understand •  Limit the time users have to perform a task •  Encourage users to think aloud with neutral prompts •  Elicit detailed information 25
  26. 26. Interview •  Introduction •  Warm-up •  General issues •  Deep focus •  Wrap-up 26
  27. 27. Nondirected Interview Prepare nondirected questions •  Focused on the person answering it •  Focused on experience, not extrapolation •  Concentrate on immediate experience •  Avoid judgmental language •  Focus questions on a single topic •  Keep questions open-ended •  Avoid binary questions 27
  28. 28. Nondirected Interview Examples “Is this a useful feature?” “Is this feature valuable to the work you do right now?” “Do you think this feature would be better if it was also available on smart phones?” 28
  29. 29. Nondirected Interview Running the interview •  Define (technical) terms •  Don’t force opinions •  Restate answers •  Never say the participant is wrong •  Work with a partner 29
  30. 30. Analyzing Results •  Collecting observations –  Notes from moderator and observer –  Quantitative information: task successfulness, navigational path, number of steps, errors, time to complete, subjective ratings (System Usability Scale) –  Qualitative feedback from participants •  Extracting trends –  Repeated concerns –  Multiple issues with possible common origin –  Identify effects on user experience 30
  31. 31. Analyzing Results •  A list of usability issues –  Quantitative and qualitative data as evidence –  Possible causes –  Effects on user experience •  Assign importance/severity •  Suggest solutions 31
  32. 32. Communicating Findings Usability test report •  Project overview •  Participants •  Testing environment •  Tasks •  Procedure •  Key findings –  Issues –  Recommendations 32
  33. 33. Communicating Findings Presentation •  Prepare the audience •  Use supporting evidence (e.g., highlight videos) •  Emphasize user perspectives •  Use numbers carefully (because of small sample) •  Leave enough time for questions 33
  34. 34. Communicating Findings Common problems: •  “This is not statistically significant.” •  “This user is our typical user.”“This user does not have relevant experience.” •  “User X did Y; therefore, everyone must do Y.” •  “More than half of users did not like this design; therefore, we should change it.” 34
  35. 35. What’s Essential? •  Develop a study plan •  Find participants •  Conduct the sessions •  Debrief with observers and decide together •  Present results 35
  36. 36. Discussion Think of an existing design that you want to evaluate for effectiveness •  What are my objectives? •  What types of resources do I have? •  Who am I presenting this to? 36
  37. 37. Other Types of Usability Studies •  Remote usability testing http://www.userzoom.com/ •  Eye tracking 37 http://www.normalmodes.com/blog/2009/09/28/eye-tracking- heatmap-gallery-a-preview-discussion-of-ui-considerations/
  38. 38. Where to Find Out More •  6 Must-See Usability Testing Videos http://blog.usabilla.com/7-must-see-usability-testing-videos/ •  Moderating Usability Tests Joe Dumas & Beth Loring (2008) •  A Field Guide to Usability Testing Smashing eBook https://shop.smashingmagazine.com/guide-to-usability-testing-1.html •  Handbook of Usability Testing Rubin & Chisnell (2008) •  Don’t Make me Think by Steve Krug Chapter:“Usability Testing on 10 cents a Day”’ 38
  39. 39. Learning Objectives 39 •  Participants will be able to describe the usability concept and it's applications to libraries. •  Participants will be aware of the basic workflows and techniques needed when doing usability tests. •  Participants will be aware of resources for usability studies and where to find out more.
  40. 40. Agenda 40 •  What is a usability study? •  Usability study process •  Conducting the study •  Analyzing Results •  Communicating •  Where to find out more
  41. 41. Works Cited •  6 Must-See Usability Testing Videos - The Usabilla Blog. (n.d.). Retrieved from http:// blog.usabilla.com/7-must-see-usability-testing-videos/ •  Affairs, A. S. for P. (2013). Usability Testing. Retrieved from http://www.usability.gov/how-to-and-tools/methods/usability-testing.html •  Clear, Consistent, Capable, NICE» Blog Archive » Eye Tracking Heatmap Gallery: A Preview & Discussion of UI Considerations. (n.d.). Retrieved from http://www.normalmodes.com/blog/ 2009/09/28/eye-tracking-heatmap-gallery-a-preview-discussion-of-ui-considerations/ •  Dumas. Joe, & Loring, B. (2008). Moderating Usability Tests. •  DIY Mobile Usability Testing - SXSW Interactive 2012. (n.d.). Retrieved from http://www.slideshare.net/beleniq/diy-mobile-usability-testing-sxsw-interactive-2012 •  Morae usability testing software from TechSmith. (n.d.). Retrieved from http://www.techsmith.com/morae.html •  Krug, S. (2005). Don’t Make Me Think. •  Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing. •  UserZoom | zooming in on the user experience. (n.d.). Retrieved from http://www.userzoom.com/ •  Silverback — guerrilla usability testing. (n.d.). Retrieved from http://silverbackapp.com/ 41
  42. 42. Final Notes Evaluation of this webinar Questions? Tao Zhang zhan1022@purdue.edu Ilana Barnes ibarnes@purdue.edu 42
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×