Successfully reported this slideshow.
Your SlideShare is downloading. ×

When Qual Met UX

Loading in …3

Check these out next

1 of 30 Ad

More Related Content

Similar to When Qual Met UX (20)

Recently uploaded (20)


When Qual Met UX

  1. 1. When Qual Met UX: How to Combine Traditional Qualitative Methods and User Research Approaches to Satisfy Your Clients and Add Value to Your Findings Kinsey Gimbel & Jon Strohl 2018 QRCA Annual Conference
  2. 2. 2 • Focus Groups • Driven by Marketing Department • Conducted by Market Researchers • Terminology often centers around “participants,” “themes,” and “discussions.” • Usability Testing • Driven by Engineering or Product Development • Conducted by UX Researchers • Terminology often centers around “users,” “tasks,” “success/failure metrics,” and “severity ratings.” How It Can Appear Qualitative Research User Experience Research
  3. 3. 3 How We See It
  4. 4. 4 Kinsey Jon Image from: Image from: Introductions
  5. 5. 5 Bringing Different Kinds of Research Together!
  6. 6. 6 1. What is user experience research? 2. How do you implement a cross-cutting Qual/UX research program? 3. What can you take away to start weaving UX into your work? 4. In summary... Our Road Map for Today
  7. 7. What is User Experience Research?
  8. 8. 8 References: The History of UX
  9. 9. 9 Image from: Usability and Much More...
  10. 10. 10 Qualitative Quantitative BehavioralAttitudinal UX Research Spectrum
  11. 11. 11 User Interviews Focus Groups Observation Diary Writing Surveys Personas Website Analytics Competitive Analysis Stakeholder Interviews Journey Mapping Heuristic Evaluation UX Testing Benchmarking Card Sorting Wire-Framing Prototyping Eye Tracking Tree Testing Task Analysis Surveys Experiments A/B Testing Website Analytics Benchmarking Discover Understand Users and Environment Design and Develop Early and Late Stage Testing Evaluate Monitor Performance PHASE 1 PHASE 2 PHASE 3 UX Research Phases
  12. 12. 12 Image from: Product Development Process
  13. 13. UX Methods
  14. 14. 14 A method for detecting usability issues in an interface in which a small set of evaluators examine the interface and judge its compliance with usability best practices and principles (heuristics). Visibility of system status Match between system and the real world User control and freedom Consistency and standards Error prevention Affordance of the system Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recognize, diagnose, and recover from errors Help and documentation Minimize the cognitive load/burden Heuristic Evaluations *Learn more about heuristic evaluations at
  15. 15. 15 A Typical UX Test Session
  16. 16. 16 UX Testing Logistics
  17. 17. 17 Screen-based eye tracker Eye-tracking glasses Eye-tracking VR integration Mobile device testing UX Testing Tools – Eye Tracking
  18. 18. 18 Metrics Tells us… Performance + First click accuracy + Task accuracy + Time to complete tasks + Click patterns + Conversion rate + How easily users can get started + How easily users complete tasks + The actions users take during use + How efficiently the product can be navigated + Likelihood of converting to desired outcome Self-Report + Difficulty ratings + Satisfaction ratings + Think-aloud protocol + Debriefing interview + Perceived usability of the product + How satisfying users find the experience + What users are thinking during use + Users’ feedback on completing tasks Implicit + Eye tracking + Behavioral analysis + Verbalization analysis + Pupil dilation + Visibility of design elements + How attention is allocated during use + Behavioral and verbalization patterns + Mental workload while completing tasks UX Metrics
  19. 19. 19 Card Sorting
  20. 20. 20 Image from: Tree Testing
  21. 21. 21 Focus Groups + Past participant experiences, beliefs, opinions + In-group activities UX Test + Difficulty and satisfaction ratings +Task accuracy and clicks + Think-aloud comments + Debriefing interview + SUS, NPS Heuristic Evaluation + Expert reviews + Severity ratings UX as a Mixed-Methods Approach
  22. 22. Integrating Qual and UX
  23. 23. 23 • Challenge: Improve a federal agency’s website designed for job seekers and provide strategic direction for future development. • Method: Conducted focus groups with multiple audience segments across the United States. • Result: Insights about what job seekers think about applying for federal employment and their past experiences with the process. Focus Groups for a Career Building and Development Website
  24. 24. 24 • Difficult to receive useful feedback on how participants would behave • Lacked depth on feedback of interface Focus Groups for a Career Building and Development Website: Method Limitations
  25. 25. 25 Privacy Security • Challenge: Understand the perception of a social media organization’s security, and identify ways the organization could improve it’s privacy functions. • Method: Conducted focus groups with multiple audience segments across the United States to learn how people comprehend privacy and security in the online space. • Result: Insights about how consumers conceptualize security and privacy. Focus Groups on Social Media Privacy and Security
  26. 26. 26 • Difficult to get interactive and truly responsive feedback. • Types of tasks that can be provided to a group are more limited. Focus Groups on Social Media Privacy and Security: Method Limitations
  27. 27. 27 • Challenge: Provide guidance on the optimal design of a student loan form. • Phase 1: Focus groups on consumers’ experiences with student loans. IDIs on students’ own paperwork. • Phase 2: Multiple rounds of testing with draft forms. • Result: An optimized form that was ready for dissemination. Mixed-Methods Multiphase Form Design
  28. 28. 28 Mixed-Methods Multiphase Interactive Voice Response (IVR) System Design • Challenge: Redesign an agency’s IVR system’s structure and scripts. • Phase 1: Focus groups to assess how people feel about calling the line, their past experiences, and initial impressions. • Phase 2: Iterative rounds of UX testing with revised selection options. • Result: Recommendations and scripts for development and implementation. Image from:
  29. 29. What Can You Take Away to Start Integrating UX Into Your Work?
  30. 30. 30 Adding a UX Twist to What You Already Do Well 1. Consider using different methods across disciplines to answer your client’s research questions—both the ones they ask and the ones they don’t! 2. Consider including specialized tasks and exercises in your data collection. 3. Have participants “think aloud” when completing a task, and follow up later with discussion questions and standardized scales/survey questions. 4. Report observations of participants’ behavior/metrics in addition to what they say. 5. Ensure recommendations will create a positive user experience!