Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Fast and Unbiased Usability Research

243 views

Published on

Tips and tricks for optimizing your research process.

Published in: Design
  • Be the first to comment

Fast and Unbiased Usability Research

  1. 1. @kaniasty | #uxpabos17 You can have it both ways Fast and Unbiased Usability Research Eva Kaniasty, Red Pill UX
  2. 2. @kaniasty | #uxpabos17 2 What do we want? Everything. When do we want it? Now
  3. 3. @kaniasty | #uxpabos17 3 What is good research? “You can’t always get what you want.”
  4. 4. @kaniasty | #uxpabos17 4 Bias is everywhere! • Inter-evaluator bias • Loaded or leading questions • Cognitive biases (e.g. primacy, recency) • Group dynamics (i.e. groupthink) • Prejudgements • Selection bias • Respondent bias (e.g. positive, social) “Soylent Green… Bias… 
 it’s made of people!”
  5. 5. @kaniasty | #uxpabos17 5 • Plan • Structure • Focus • Automate • Don’t rely on memory Reduce Your Cognitive Load “An ounce of prevention is worth a pound of cure.” - Ben Franklin
  6. 6. @kaniasty | #uxpabos17 6 “Efficiency is intelligent laziness.” Objectivity through Optimization
  7. 7. @kaniasty | #uxpabos17 Good + Efficient Participant Recruiting
  8. 8. @kaniasty | #uxpabos17 8 Recruiting Recruiting Brief Screening Survey Scheduling - Research Recruiter - UserInterviews.com - Stakeholders Participant Data - Contact Info - Time Zones - Screening Data “Keep it simple…”
  9. 9. @kaniasty | #uxpabos17 9 Customer Super Screener (i.e. survey) Better screening - Larger pool of potential participants - Foot in the door - Comparison sample - Bias fighter Measure twice, cut once Collect data as you screen - Demographics - Feature usage - Card sorts - Desirability (product reaction) - Technical - Quotes (collected judiciously) - Opt-in for future research 
 (online & moderated)*Don’t forget legal review
  10. 10. @kaniasty | #uxpabos17 10 Recruiting Logistics Written Consent •Context •Logistics •Recording •Compensation •(Optional) Nondisclosure *Don’t forget legal review “80 percent of success is just showing up.” Reducing No-Shows •Written consent •Email confirmation •Remote tool testing/setup •Phone/Texting Session Status Tracking Compensation
  11. 11. @kaniasty | #uxpabos17 Good + Efficient Study Design
  12. 12. @kaniasty | #uxpabos17 12 “Get your priorities straight.”
  13. 13. @kaniasty | #uxpabos17 13 Moderator Guide Pre-Test Interview - Background / Profile Fit - Past experience/usage - Top 3 tasks/needs First Impressions - (if website) HP overview - (if navigation) Navigation expectations - (if new user) Free exploration - (if current user) Recent task Tasks or Process - Probing questions - Ease of use ratings - Task Success Wrap-Up - Salient Likes/Dislikes - “Change one thing” - System Usability Scale/Global Measures “How the sausage gets made.”
  14. 14. @kaniasty | #uxpabos17 Good & Efficient Data Collection
  15. 15. @kaniasty | #uxpabos17 15 Session Housekeeping Participant Briefing - Context-setting - Recording consent - Think-aloud instruction - Honest Feedback - Neutral Observer Logistics - Apps / Links - Notes templates - Equipment - Logins - Recording - Reset “Don’t forget to eat.” - Mom
  16. 16. @kaniasty | #uxpabos17 16 Recording & Photography • Personal room • Test meeting OR Tech call • Mac & Windows compatible Mobile Screen Mobile Device • Sleds
  17. 17. @kaniasty | #uxpabos17 17 Super Notes! Mod Guide + Notes + Audio PearNote Mac OneNote Windows Livescribe
  18. 18. @kaniasty | #uxpabos17 18 The Session Summary This participant had major issues with X and Y features because she expected [X] and [Y] to happen. She followed an interesting workflow that was different from what we expected. She also did [this interesting or surprising thing].
 
 She was hesitant to explore the UI and fit the ‘Ms. X’ persona pretty well… “Take 5.”
  19. 19. @kaniasty | #uxpabos17 19 Avoiding the Slow TV Syndrome • Re-watching videos • Relying on notes alone • Human transcription • Automatic transcription VLC player
  20. 20. @kaniasty | #uxpabos17 20 Task Data
  21. 21. @kaniasty | #uxpabos17 21 Usability Measures Custom Tools Question Banks
  22. 22. @kaniasty | #uxpabos17 22 Question Banks Product Reaction CardsSemantic Differentials Ease of Use
  23. 23. @kaniasty | #uxpabos17 23 Spreadsheets
  24. 24. @kaniasty | #uxpabos17 Good & Efficient Reporting
  25. 25. @kaniasty | #uxpabos17 25 Analysis & Reporting Strategy Findings Report - Narrative - Includes detailed study methodology & background - Includes raw data - Includes participant detail - Includes session summaries - Focused on detailed findings Strategic Brief Presentation - Focused on the big picture - Focused on priorities - Focused on critical findings - Includes recommendations - Visually impactful - Includes quotes - Synthesizes data from multiple sources - Summary deliverables: 
 (e.g. profiles, personas, user journeys) “Use the right tool 
 for the job.”
  26. 26. @kaniasty | #uxpabos17 26 NOT Survey Reports No Custom Color Coding Always reports %
  27. 27. @kaniasty | #uxpabos17 Participant Data 27 ♀ 5 Female ♂ 5 Male 4 on Laptops 4 on Smartphones 
 (4 iPhone, 3 Android, 1 Windows Phone) Customers Company 1, Company 2, Company 3 Customers RepresentedProfile 1 | Profile 2 Session Summaries as part of detailed findings / or with recordings
  28. 28. @kaniasty | #uxpabos17 28 Task Data
  29. 29. @kaniasty | #uxpabos17 29 Structured Data
  30. 30. @kaniasty | #uxpabos17 30 Usability Measures Tullis & Albert
  31. 31. @kaniasty | #uxpabos17 31 Data Analysis
  32. 32. @kaniasty | #uxpabos17 32 Infogr.am PROS: Range of Graphs Fun viz options Full control over colors Option to show values (not %) CONS: Limited font control No option to show confidence intervals
  33. 33. @kaniasty | #uxpabos17 33 plot.ly
  34. 34. @kaniasty | #uxpabos17 34 “Last but not least… qualitative data analysis.”
  35. 35. @kaniasty | #uxpabos17 35 Final Presentation Strategic Brief - Focused on the big picture - Focused on priorities - Focused on critical findings - Includes recommendations - Visually impactful - Includes quotes - Synthesizes data from multiple sources - Summary deliverables: 
 (e.g. profiles, personas, user journeys) “Looks matter”
  36. 36. @kaniasty | #uxpabos17 Questions?

×