The document outlines the agenda and content for a UX research methods meetup. The schedule includes socializing, introductions from the presenter about his background and jobs, a brief presentation on why research is important and common methods used, and a workshop. The presentation covers qualitative methods like focus groups, interviews, design ethnography and usability studies. It also discusses quantitative methods such as surveys, metrics, and usability studies with numbers. Examples of each method are provided. The meetup will conclude with a workshop and optional afterparty.

1. Research Methods in UX
UXMad
9/19/13

2. Schedule
● Socializing/Mingling
● Intros/Announcements
○ Jobs/Opportunities
● Brief Presentation
○ Why research?
○ What research is
○ A few methods
● Workshop!
● Afterparty?

3. About Me

4. Brad Orego
me@bradorego.com
@bradorego
B.S. in Computer Science, B.A. in Psychology
UX Lead at Nextt
UXMad Meetup Co-Organizer
Entrepreneur
Dancer, Curler, Ultimate-er, Cyclist, Beer-batter pancake blogger

5. Why research?

6. Why?
● UX is important
● UX sells
● UX is the key differentiator
● People matter
● We aren’t our users
● Build a solution that people want
● Solve a problem people have

7. How?
● Qualitative
○ Focus Groups/Interviews
○ Design Ethnography
○ Usability Studies
○ Diary Studies
● Quantitative
○ Surveys
○ Metrics
○ Usability studies

8. Details!
Qualitative

9. Qualitative
● Goals
○ Meaning - how people see the world
○ Context - the world in which people act
○ Process - what actions and activities people do
○ Reasoning - why people behave the way they do
● Attributes
○ Creativity - what might be interesting to see?
○ Subjective - influence by personal opinions
○ Inductive reasoning - few examples, then extrapolate
● Data
○ field notes, recordings, diary entries, …

10. Focus Groups & Interviews
● Benefits
○ may be structured, directed, detailed
○ put real people/faces/stories behind data
● Problems
○ highly subjective; open to bias by interviewer
○ not in-situ
● Tools
○ Flow - keep them talking when it’s interesting
○ Non-direction - let them meander
○ Transition - close current; open new conversation
○ Depth - tell me more

11. Design Ethnography
● Comes from anthropology
○ Science!
● In-situ
○ observation, interviews
○ get the real story
● How
○ What people say, do, and use
○ Why they do it this way. Why not another way?
● Be…
○ Non-disruptive, non-interventionist, unbiased

12. Usability Studies
● More appropriate for actual artifacts
○ mock-ups, existing systems, prototypes, etc
● Methods
○ Heuristic Evaluation - rate it against a set of criteria
○ Conceptual Modelling - show & tell
○ Direct Observation - watch people play with it
● Interviews
○ Retrospective - record it then show them
○ Critical Incidence - the parts that stood out
● Data
○ audio/video recordings, handwritten notes

13. Diary Studies
● Participants answer over a period of time
○ Same questions? Different questions? Open/closed?
● Pros
○ Longitudinal, in-depth self-report data
○ Highly detailed; very realistic
○ Trends over time
● Cons
○ Expensive; difficult to get right
○ Self-report data
○ Availability Bias

14. Numbers!
Quantitative

15. Quantitative
● Goals
○ Numbers
○ Trends in data
○ Read between the lines
○ Answer specific questions
● Attributes
○ Large(r) sample sizes
○ Phone, online, in-person
○ Open vs. closed questions
● Data
○ Means, medians, SDs, regressions, correlations

16. Surveys
● Questionnaires with specific measures,
hypotheses, and results
○ SPECIFIC
● Types of Questions
○ Multiple Choice; Scalar; Matching/Association
○ Open-ended
● Things to consider
○ What to ask? How to ask it?
○ Ordering effects
○ Validity threats

17. Metrics
● Measuring real behavior, interactions, use
○ Doesn’t have to be on a real system
● Numbers!
○ Click-through rates
○ Time spent on an activity
○ Error rate
● Method
○ Have a specific thing to measure
○ Figure out how to measure it
○ Figure out why the results are the way they are

18. Usability Studies
● Not as hard-and-fast as other Qualitative Data
○ In-person, so you can dive deeper
● Be specific (numbers!)
○ Measure success/failure rate
○ Measure time
○ A/B test
● Continuous Evaluation
○ Measure real systems in real use
○ Metrics, analytics, log files

19. How?
● Qualitative
○ Focus Groups/Interviews
○ Design Ethnography
○ Usability Studies
○ Diary Studies
● Quantitative
○ Surveys
○ Metrics
○ Usability studies

20. Now what?

21. Tonight's Meetup
● Socializing/Mingling
● Intros/Announcements
○ Jobs
● Brief Presentation
○ Why research?
○ What research is
○ A few methods
● Workshop!
● Afterparty?

22. Workshop!

23. References/Further Reading
● http://carmster.com/hci/
● http://www.cs.rochester.edu/courses/212/fall2012/
● http://www.upassoc.
org/usability_resources/usability_in_the_real_world/roi_of_usability.html
● http://www.youtube.com/watch?v=O94kYyzqvTc
● http://uxmag.com/topics/research-methods-and-techniques
● http://www.usability.gov/
● http://www.nngroup.com/articles/which-ux-research-methods/
● https://speakerdeck.com/tsharon/validating-assumptions-with-12-ux-
research-methods
● http://www.uxbooth.com/articles/complete-beginners-guide-to-design-
research/
● http://www.slideshare.net/dgcooley/introduction-to-ux-research-methods
● http://www.nngroup.com/articles/discount-usability-20-years/