You’re Doing The Wrong
Research
Laura Klein
@lauraklein
laura@usersknow.com
http://www.usersknow.com
Wednesday, December 5, 2012

@lauraklein
GET OUT OF THE BUILDING!
Wednesday, December 5, 2012

@lauraklein
What You Expect
1. Get out of the building
2. Talk to “users”
3. Profit
Wednesday, December 5, 2012

@lauraklein
What Really Happens
1. Get out of the building
2. [A lot of really hard work]
3. Profit????
Wednesday, December 5, 2012

@lauraklein
Let’s figure out that middle bit...
Wednesday, December 5, 2012

@lauraklein
How Many Ways Are You
Listening To Users?
Did you know that there are dozens of ways to do this?
Wednesday, December 5, 2012

@lauraklein
Here Are A Few Of Them
Landing Pages Product Stubs (Fake Doors)
Guerilla User Tests Task Based Usability
Wizard of Oz Brain Imaging (yes, really!)
Analytics
New User Interviews A/B Testing
Customer Development Interviews
Prototype Usability Observational Usability
Sales Click Tests
NPS Surveys
Unmoderated Testing Focus Groups Surveys
Wednesday, December 5, 2012

@lauraklein
I’m not going to teach you how to do
all of those today.
(You’re welcome.)
Wednesday, December 5, 2012

@lauraklein
What you want to learn determines
the type of research you need to do!
Wednesday, December 5, 2012

@lauraklein
You Only Need To Know About
Two Types Of Research*
Generative & Evaluative
*for now
Wednesday, December 5, 2012

@lauraklein
Generative Research
You want to know what to do next.
Wednesday, December 5, 2012

@lauraklein
You Don’t Have An Idea
Find a great idea by learning about problems within
specific markets.
Best Methods:
•Contextual Inquiry
•Customer Development Interviews
Wednesday, December 5, 2012

@lauraklein
Like This
Wednesday, December 5, 2012

@lauraklein
You Have A Great Idea
(maybe)
Learn more about your potential customers and figure
out what’s necessary for an MVP.
Best Methods:
•Contextual Inquiry
•Customer Development Interviews
•Observational Studies of Competitive Products
Wednesday, December 5, 2012

@lauraklein
Like This
Wednesday, December 5, 2012

@lauraklein
You Have An MVP & Want To
Know How People Use It
Understand more about your users and what they’re
really doing with your product.
Best Methods:
•Observational Studies
•Diary Studies
•Analytics
Wednesday, December 5, 2012

@lauraklein
Like This
Wednesday, December 5, 2012

@lauraklein
Or This
Wednesday, December 5, 2012

@lauraklein
See A Pattern?
Generative research is about Generating Ideas.
Wednesday, December 5, 2012

@lauraklein
Evaluative Research
You want to know if what you did was right.
Wednesday, December 5, 2012

@lauraklein
You’re Doing A Redesign & Want
To Make Sure It’s Better
See whether users can perform key tasks in your
product easily and without confusion.
Best Methods:
•Task Based Usability Tests
•Observation of Interactive Prototypes
Wednesday, December 5, 2012

@lauraklein
Like This
Wednesday, December 5, 2012

@lauraklein
You Created A New Landing
Page & Want To Test Messaging
Understand whether users really understand your new
tag line, and whether your messaging conveys what you
think it should.
Best Methods:
•Five Second Tests
•Guerilla Usability
•A/B Tests
Wednesday, December 5, 2012

@lauraklein
Like This
Wednesday, December 5, 2012

@lauraklein
You’re Thinking Of Adding A
Feature
Validate whether or not it’s a good idea before
spending time designing and building it.
Best Methods:
•Feature Stubs (Fake Doors)
•Wizard of Oz Features
Wednesday, December 5, 2012

@lauraklein
Like This
“This feature is coming soon!”
Wednesday, December 5, 2012

@lauraklein
You Added A Feature & Want To
Know The Impact
Learn whether your changes improve both the user
experience and your key business metrics.
Best Methods:
•Customer Interviews
•Observation Studies
•A/B Test Against Control
•Analytics
Wednesday, December 5, 2012

@lauraklein
Like This
Wednesday, December 5, 2012

@lauraklein
See A Pattern?
Evaluative research is about Validating Hypotheses.
Wednesday, December 5, 2012

@lauraklein
Did you notice that some methods are
quantitative?
That’s because Qualitative Research
sucks for some things.
Wednesday, December 5, 2012

@lauraklein
Quantitative Research tells you what.
Qualitative Research tells you why.
Wednesday, December 5, 2012

@lauraklein
Some Common Questions
Wednesday, December 5, 2012

@lauraklein
What Are Some Good Research
Methods That Don’t Involve
People Outside The Company?
(This page intentionally left blank)
Wednesday, December 5, 2012

@lauraklein
But What If I’m Solving My Own
Problem?
That depends. How many of you are there?
Wednesday, December 5, 2012

@lauraklein
What If I Don’t Have Any Users?
Get some.
Wednesday, December 5, 2012

@lauraklein
But What If I Can’t Get Enough
Users For Quantitative Research?
Rely more heavily on Qualitative Research.
Wednesday, December 5, 2012

Ok, Your Turn
Wednesday, December 5, 2012

@lauraklein
Exercise!
Fill this in with real life examples of some research.
Type Metric Plan
Wednesday, December 5, 2012

@lauraklein
Example!
Type Metric Plan
Generative Observe 4 people who have never used the product
Observational Revenue before go through the payment flow to find pain
Testing points that are causing drop off in the funnel.
Generate ideas for fixing those usability bugs.
Test whether potential users understand the new
messaging on the home page by running 20 people
Evaluative
Sign Up who have never seen the site before through a 5
5 Second Test
second test and asking the question, “What does this
product do?”
Wednesday, December 5, 2012

More Questions?
You know where to find me...
@lauraklein
laura@usersknow.com
http://www.usersknow.com
pssst...this will all be in the book!
To be published by O’Reilly in early 2013.
Wednesday, December 5, 2012