Downloaded 20 times
![@kaniasty | #uxpabos17
18
The Session Summary
This participant had major issues with X and Y features
because she expected [X] and [Y] to happen. She followed an
interesting workflow that was different from what we
expected. She also did [this interesting or surprising thing].
She was hesitant to explore the UI and fit the ‘Ms. X’ persona
pretty well…
“Take 5.”](https://image.slidesharecdn.com/fastefficientresearch-170522173109/85/Fast-and-Unbiased-Usability-Research-18-320.jpg)
Uploaded byEva Kaniasty
Fast and Unbiased Usability Research
The document discusses methods for conducting fast and unbiased usability research. It addresses reducing bias through careful planning, structuring studies, focusing on priorities, automating tasks, and reducing cognitive load. It provides recommendations for efficient participant recruiting, study design, data collection, analysis, and reporting. The goal is to produce high-quality insights and actionable recommendations in an optimized manner.
More Related Content
Similar to Fast and Unbiased Usability Research
Fast and Unbiased Usability Research
- 1. @kaniasty | #uxpabos17 Youcan have it both ways Fast and Unbiased Usability Research Eva Kaniasty, Red Pill UX
- 2. @kaniasty | #uxpabos17 2 Whatdo we want? Everything. When do we want it? Now
- 3. @kaniasty | #uxpabos17 3 Whatis good research? “You can’t always get what you want.”
- 4. @kaniasty | #uxpabos17 4 Biasis everywhere! • Inter-evaluator bias • Loaded or leading questions • Cognitive biases (e.g. primacy, recency) • Group dynamics (i.e. groupthink) • Prejudgements • Selection bias • Respondent bias (e.g. positive, social) “Soylent Green… Bias… it’s made of people!”
- 5. @kaniasty | #uxpabos17 5 •Plan • Structure • Focus • Automate • Don’t rely on memory Reduce Your Cognitive Load “An ounce of prevention is worth a pound of cure.” - Ben Franklin
- 6. @kaniasty | #uxpabos17 6 “Efficiencyis intelligent laziness.” Objectivity through Optimization
- 7. @kaniasty | #uxpabos17 Good+ Efficient Participant Recruiting
- 8. @kaniasty | #uxpabos17 8 Recruiting RecruitingBrief Screening Survey Scheduling - Research Recruiter - UserInterviews.com - Stakeholders Participant Data - Contact Info - Time Zones - Screening Data “Keep it simple…”
- 9. @kaniasty | #uxpabos17 9 CustomerSuper Screener (i.e. survey) Better screening - Larger pool of potential participants - Foot in the door - Comparison sample - Bias fighter Measure twice, cut once Collect data as you screen - Demographics - Feature usage - Card sorts - Desirability (product reaction) - Technical - Quotes (collected judiciously) - Opt-in for future research (online & moderated)*Don’t forget legal review
- 10. @kaniasty | #uxpabos17 10 RecruitingLogistics Written Consent •Context •Logistics •Recording •Compensation •(Optional) Nondisclosure *Don’t forget legal review “80 percent of success is just showing up.” Reducing No-Shows •Written consent •Email confirmation •Remote tool testing/setup •Phone/Texting Session Status Tracking Compensation
- 11. @kaniasty | #uxpabos17 Good+ Efficient Study Design
- 12. @kaniasty | #uxpabos17 12 “Getyour priorities straight.”
- 13. @kaniasty | #uxpabos17 13 ModeratorGuide Pre-Test Interview - Background / Profile Fit - Past experience/usage - Top 3 tasks/needs First Impressions - (if website) HP overview - (if navigation) Navigation expectations - (if new user) Free exploration - (if current user) Recent task Tasks or Process - Probing questions - Ease of use ratings - Task Success Wrap-Up - Salient Likes/Dislikes - “Change one thing” - System Usability Scale/Global Measures “How the sausage gets made.”
- 14. @kaniasty | #uxpabos17 Good& Efficient Data Collection
- 15. @kaniasty | #uxpabos17 15 SessionHousekeeping Participant Briefing - Context-setting - Recording consent - Think-aloud instruction - Honest Feedback - Neutral Observer Logistics - Apps / Links - Notes templates - Equipment - Logins - Recording - Reset “Don’t forget to eat.” - Mom
- 16. @kaniasty | #uxpabos17 16 Recording& Photography • Personal room • Test meeting OR Tech call • Mac & Windows compatible Mobile Screen Mobile Device • Sleds
- 17. @kaniasty | #uxpabos17 17 SuperNotes! Mod Guide + Notes + Audio PearNote Mac OneNote Windows Livescribe
- 18. @kaniasty | #uxpabos17 18 TheSession Summary This participant had major issues with X and Y features because she expected [X] and [Y] to happen. She followed an interesting workflow that was different from what we expected. She also did [this interesting or surprising thing]. She was hesitant to explore the UI and fit the ‘Ms. X’ persona pretty well… “Take 5.”
- 19. @kaniasty | #uxpabos17 19 Avoidingthe Slow TV Syndrome • Re-watching videos • Relying on notes alone • Human transcription • Automatic transcription VLC player
- 20.
- 21. @kaniasty | #uxpabos17 21 UsabilityMeasures Custom Tools Question Banks
- 22. @kaniasty | #uxpabos17 22 QuestionBanks Product Reaction CardsSemantic Differentials Ease of Use
- 23.
- 24. @kaniasty | #uxpabos17 Good& Efficient Reporting
- 25. @kaniasty | #uxpabos17 25 Analysis& Reporting Strategy Findings Report - Narrative - Includes detailed study methodology & background - Includes raw data - Includes participant detail - Includes session summaries - Focused on detailed findings Strategic Brief Presentation - Focused on the big picture - Focused on priorities - Focused on critical findings - Includes recommendations - Visually impactful - Includes quotes - Synthesizes data from multiple sources - Summary deliverables: (e.g. profiles, personas, user journeys) “Use the right tool for the job.”
- 26. @kaniasty | #uxpabos17 26 NOTSurvey Reports No Custom Color Coding Always reports %
- 27. @kaniasty | #uxpabos17 ParticipantData 27 ♀ 5 Female ♂ 5 Male 4 on Laptops 4 on Smartphones (4 iPhone, 3 Android, 1 Windows Phone) Customers Company 1, Company 2, Company 3 Customers RepresentedProfile 1 | Profile 2 Session Summaries as part of detailed findings / or with recordings
- 28.
- 29.
- 30. @kaniasty | #uxpabos17 30 UsabilityMeasures Tullis & Albert
- 31. @kaniasty | #uxpabos17 31 DataAnalysis
- 32. @kaniasty | #uxpabos17 32 Infogr.am PROS: Rangeof Graphs Fun viz options Full control over colors Option to show values (not %) CONS: Limited font control No option to show confidence intervals
- 33.
- 34. @kaniasty | #uxpabos17 34 “Lastbut not least… qualitative data analysis.”
- 35. @kaniasty | #uxpabos17 35 FinalPresentation Strategic Brief - Focused on the big picture - Focused on priorities - Focused on critical findings - Includes recommendations - Visually impactful - Includes quotes - Synthesizes data from multiple sources - Summary deliverables: (e.g. profiles, personas, user journeys) “Looks matter”
- 36.