Which research when

You’re Doing The Wrong
Research

Laura Klein
@lauraklein
laura@usersknow.com
http://www.usersknow.com
Wednesday, December 5, 2012

@lauraklein

GET OUT OF THE BUILDING!


@lauraklein

What You Expect

1. Get out of the building
2. Talk to “users”
3. Proﬁt


@lauraklein

What Really Happens

1. Get out of the building
2. [A lot of really hard work]
3. Proﬁt????


@lauraklein

Let’s ﬁgure out that middle bit...


@lauraklein

How Many Ways Are You
Listening To Users?
Did you know that there are dozens of ways to do this?


@lauraklein

Here Are A Few Of Them
Landing Pages Product Stubs (Fake Doors)
Guerilla User Tests Task Based Usability
Wizard of Oz Brain Imaging (yes, really!)
Analytics
New User Interviews A/B Testing
Customer Development Interviews
Prototype Usability Observational Usability
Sales Click Tests
NPS Surveys
Unmoderated Testing Focus Groups Surveys

@lauraklein

I’m not going to teach you how to do
all of those today.
(You’re welcome.)


@lauraklein

What you want to learn determines
the type of research you need to do!


@lauraklein

You Only Need To Know About
Two Types Of Research*

Generative & Evaluative

*for now


@lauraklein

Generative Research

You want to know what to do next.


@lauraklein

You Don’t Have An Idea
Find a great idea by learning about problems within
speciﬁc markets.

Best Methods:
•Contextual Inquiry
•Customer Development Interviews


@lauraklein

Like This


@lauraklein

You Have A Great Idea
(maybe)
Learn more about your potential customers and ﬁgure
out what’s necessary for an MVP.

Best Methods:
•Contextual Inquiry
•Customer Development Interviews
•Observational Studies of Competitive Products


@lauraklein

You Have An MVP & Want To
Know How People Use It
Understand more about your users and what they’re
really doing with your product.

Best Methods:
•Observational Studies
•Diary Studies
•Analytics


@lauraklein

Or This


@lauraklein

See A Pattern?

Generative research is about Generating Ideas.


@lauraklein

Evaluative Research

You want to know if what you did was right.


@lauraklein

You’re Doing A Redesign & Want
To Make Sure It’s Better
See whether users can perform key tasks in your
product easily and without confusion.

Best Methods:
•Task Based Usability Tests
•Observation of Interactive Prototypes


@lauraklein

You Created A New Landing
Page & Want To Test Messaging
Understand whether users really understand your new
tag line, and whether your messaging conveys what you
think it should.

Best Methods:
•Five Second Tests
•Guerilla Usability
•A/B Tests


@lauraklein

You’re Thinking Of Adding A
Feature
Validate whether or not it’s a good idea before
spending time designing and building it.

Best Methods:
•Feature Stubs (Fake Doors)
•Wizard of Oz Features


@lauraklein

Like This

“This feature is coming soon!”


@lauraklein

You Added A Feature & Want To
Know The Impact
Learn whether your changes improve both the user
experience and your key business metrics.

Best Methods:
•Customer Interviews
•Observation Studies
•A/B Test Against Control
•Analytics


@lauraklein

See A Pattern?

Evaluative research is about Validating Hypotheses.


@lauraklein

Did you notice that some methods are
quantitative?

That’s because Qualitative Research
sucks for some things.


@lauraklein

Quantitative Research tells you what.

Qualitative Research tells you why.


@lauraklein

Some Common Questions


@lauraklein

What Are Some Good Research
Methods That Don’t Involve
People Outside The Company?

(This page intentionally left blank)


@lauraklein

But What If I’m Solving My Own
Problem?

That depends. How many of you are there?


@lauraklein

What If I Don’t Have Any Users?

Get some.


@lauraklein

But What If I Can’t Get Enough
Users For Quantitative Research?

Rely more heavily on Qualitative Research.


Ok, Your Turn


@lauraklein

Exercise!
Fill this in with real life examples of some research.

Type Metric Plan


@lauraklein

Example!
Type Metric Plan

Generative Observe 4 people who have never used the product
Observational Revenue before go through the payment flow to find pain
Testing points that are causing drop off in the funnel.
Generate ideas for fixing those usability bugs.

Test whether potential users understand the new
messaging on the home page by running 20 people
Evaluative
Sign Up who have never seen the site before through a 5
5 Second Test
second test and asking the question, “What does this
product do?”


More Questions?
You know where to ﬁnd me...
@lauraklein
laura@usersknow.com
http://www.usersknow.com

pssst...this will all be in the book!
To be published by O’Reilly in early 2013.


Which research when

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (14)

Similar to Which research when

Similar to Which research when (20)

Recently uploaded

Recently uploaded (20)

Which research when