Which research when

You’re DoingThe Wrong
Research
Laura Klein
@lauraklein
laura@usersknow.com
http://www.usersknow.com
Wednesday, December 5, 2012

GET OUT OFTHE BUILDING!
@lauraklein

WhatYou Expect
@lauraklein
1. Get out of the building
2.Talk to “users”
3. Proﬁt

What Really Happens
@lauraklein
1. Get out of the building
2. [A lot of really hard work]
3. Proﬁt????

@lauraklein
Let’s ﬁgure out that middle bit...

How Many Ways AreYou
ListeningTo Users?
@lauraklein
Did you know that there are dozens of ways to do this?

Here Are A Few OfThem
@lauraklein
Task Based Usability
Analytics
Observational Usability
NPS Surveys
New User Interviews A/BTesting
Landing Pages
Wizard of Oz
Product Stubs (Fake Doors)
Customer Development Interviews
Prototype Usability
Sales
SurveysFocus Groups
Brain Imaging (yes, really!)
Guerilla UserTests
UnmoderatedTesting
ClickTests

@lauraklein
I’m not going to teach you how to do
all of those today.
(You’re welcome.)

@lauraklein
What you want to learn determines
the type of research you need to do!

You Only NeedTo Know About
TwoTypes Of Research*
@lauraklein
Generative & Evaluative
*for now

Generative Research
@lauraklein
You want to know what to do next.

You Don’t Have An Idea
@lauraklein
Find a great idea by learning about problems within
speciﬁc markets.
Best Methods:
•Contextual Inquiry
•Customer Development Interviews

LikeThis
@lauraklein

You Have A Great Idea
(maybe)
@lauraklein
Learn more about your potential customers and ﬁgure
out what’s necessary for an MVP.
Best Methods:
•Contextual Inquiry
•Customer Development Interviews
•Observational Studies of Competitive Products

You Have An MVP & WantTo
Know How People Use It
@lauraklein
Understand more about your users and what they’re
really doing with your product.
Best Methods:
•Observational Studies
•Diary Studies
•Analytics

OrThis
@lauraklein

See A Pattern?
@lauraklein
Generative research is about Generating Ideas.

Evaluative Research
@lauraklein
You want to know if what you did was right.

You’re Doing A Redesign & Want
To Make Sure It’s Better
@lauraklein
See whether users can perform key tasks in your
product easily and without confusion.
Best Methods:
•Task Based UsabilityTests
•Observation of Interactive Prototypes

You Created A New Landing
Page & WantToTest Messaging
@lauraklein
Understand whether users really understand your new
tag line, and whether your messaging conveys what you
think it should.
Best Methods:
•Five SecondTests
•Guerilla Usability
•A/BTests

You’reThinking Of Adding A
Feature
@lauraklein
Validate whether or not it’s a good idea before
spending time designing and building it.
Best Methods:
•Feature Stubs (Fake Doors)
•Wizard of Oz Features

LikeThis
@lauraklein
“This feature is coming soon!”

You Added A Feature & WantTo
KnowThe Impact
@lauraklein
Learn whether your changes improve both the user
experience and your key business metrics.
Best Methods:
•Customer Interviews
•Observation Studies
•A/BTest Against Control
•Analytics

See A Pattern?
@lauraklein
Evaluative research is about Validating Hypotheses.

@lauraklein
Did you notice that some methods are
quantitative?
That’s because Qualitative Research
sucks for some things.

@lauraklein
Quantitative Research tells you what.
Qualitative Research tells you why.

@lauraklein
Some Common Questions

What Are Some Good Research
MethodsThat Don’t Involve
People OutsideThe Company?
@lauraklein
(This page intentionally left blank)

But What If I’m Solving My Own
Problem?
@lauraklein
That depends. How many of you are there?

What If I Don’t Have Any Users?
@lauraklein
Get some.

But What If I Can’t Get Enough
Users For Quantitative Research?
@lauraklein
Rely more heavily on Qualitative Research.

Ok,YourTurn

Exercise!
@lauraklein
Type Metric Plan
Fill this in with real life examples of some research.

Example!
@lauraklein
Type Metric Plan
Generative
Observational
Testing
Revenue
Observe 4 people who have never used the product
before go through the payment flow to find pain
points that are causing drop off in the funnel.
Generate ideas for fixing those usability bugs.
Evaluative
5 SecondTest
Sign Up
Test whether potential users understand the new
messaging on the home page by running 20 people
who have never seen the site before through a 5
second test and asking the question,“What does this
product do?”

More Questions?
You know where to ﬁnd me...
@lauraklein
laura@usersknow.com
http://www.usersknow.com
pssst...this will all be in the book!
To be published by O’Reilly in early 2013.

Which research when

Recommended

Recommended

More Related Content

What's hot

What's hot (7)

Viewers also liked

Viewers also liked (7)

Similar to Which research when

Similar to Which research when (20)

More from Robert-Emmanuel Mayssat

More from Robert-Emmanuel Mayssat (20)

Recently uploaded

Recently uploaded (20)

Which research when