This presentation covers how to combine traditional qualitative methods and user research approaches to satisfy your clients and add value to findings.
DBA Basics: Getting Started with Performance Tuning.pdf
When Qual Met UX
1. When Qual Met UX:
How to Combine Traditional Qualitative
Methods and User Research Approaches
to Satisfy Your Clients and Add Value to
Your Findings
Kinsey Gimbel & Jon Strohl
2018 QRCA Annual Conference
2. 2
• Focus Groups
• Driven by Marketing Department
• Conducted by Market Researchers
• Terminology often centers around
“participants,” “themes,” and “discussions.”
• Usability Testing
• Driven by Engineering or Product Development
• Conducted by UX Researchers
• Terminology often centers around “users,”
“tasks,” “success/failure metrics,” and
“severity ratings.”
How It Can Appear
Qualitative
Research
User Experience
Research
6. 6
1. What is user experience research?
2. How do you implement a cross-cutting Qual/UX research program?
3. What can you take away to start weaving UX into your work?
4. In summary...
Our Road Map for Today
14. 14
A method for detecting usability issues in an interface in which a small set of
evaluators examine the interface and judge its compliance with usability best
practices and principles (heuristics).
Visibility of system status
Match between system and
the real world
User control and freedom
Consistency and standards
Error prevention
Affordance of the system
Recognition rather than recall
Flexibility and efficiency of use
Aesthetic and minimalist design
Help users recognize, diagnose, and
recover from errors
Help and documentation
Minimize the cognitive load/burden
Heuristic Evaluations
*Learn more about heuristic evaluations at https://www.nngroup.com/articles/ten-usability-heuristics/
18. 18
Metrics Tells us…
Performance
+ First click accuracy
+ Task accuracy
+ Time to complete tasks
+ Click patterns
+ Conversion rate
+ How easily users can get started
+ How easily users complete tasks
+ The actions users take during use
+ How efficiently the product can be navigated
+ Likelihood of converting to desired outcome
Self-Report
+ Difficulty ratings
+ Satisfaction ratings
+ Think-aloud protocol
+ Debriefing interview
+ Perceived usability of the product
+ How satisfying users find the experience
+ What users are thinking during use
+ Users’ feedback on completing tasks
Implicit
+ Eye tracking
+ Behavioral analysis
+ Verbalization analysis
+ Pupil dilation
+ Visibility of design elements
+ How attention is allocated during use
+ Behavioral and verbalization patterns
+ Mental workload while completing tasks
UX Metrics
23. 23
• Challenge: Improve a federal
agency’s website designed for job
seekers and provide strategic
direction for future development.
• Method: Conducted focus groups
with multiple audience segments
across the United States.
• Result: Insights about what job
seekers think about applying for
federal employment and their
past experiences with the
process.
Focus Groups for a Career Building and
Development Website
24. 24
• Difficult to receive useful feedback on
how participants would behave
• Lacked depth on feedback of interface
Focus Groups for a Career Building and
Development Website: Method Limitations
25. 25
Privacy
Security
• Challenge: Understand the perception of
a social media organization’s security,
and identify ways the organization could
improve it’s privacy functions.
• Method: Conducted focus groups with
multiple audience segments across the
United States to learn how people
comprehend privacy and security in the
online space.
• Result: Insights about how consumers
conceptualize security and privacy.
Focus Groups on Social Media
Privacy and Security
26. 26
• Difficult to get interactive and truly
responsive feedback.
• Types of tasks that can be provided
to a group are more limited.
Focus Groups on Social Media
Privacy and Security: Method Limitations
27. 27
• Challenge: Provide guidance on the optimal design of a student loan
form.
• Phase 1: Focus groups on consumers’ experiences with student
loans. IDIs on students’ own paperwork.
• Phase 2: Multiple
rounds of testing with
draft forms.
• Result: An optimized
form that was ready for
dissemination.
Mixed-Methods Multiphase Form Design
28. 28
Mixed-Methods Multiphase Interactive Voice
Response (IVR) System Design
• Challenge: Redesign an agency’s IVR
system’s structure and scripts.
• Phase 1: Focus groups to assess how
people feel about calling the line, their
past experiences, and initial
impressions.
• Phase 2: Iterative rounds of UX testing
with revised selection options.
• Result: Recommendations and scripts
for development and implementation.
Image from: http://money.cnn.com/gallery/smallbusiness/2013/12/19/worst-customer-service/index.html
29. What Can You Take Away
to Start Integrating UX Into
Your Work?
30. 30
Adding a UX Twist to What You Already Do Well
1. Consider using different methods across disciplines to
answer your client’s research questions—both the ones they
ask and the ones they don’t!
2. Consider including specialized tasks and exercises in your
data collection.
3. Have participants “think aloud” when completing a task,
and follow up later with discussion questions and
standardized scales/survey questions.
4. Report observations of participants’ behavior/metrics in
addition to what they say.
5. Ensure recommendations will create a positive user
experience!