The Right Research
Method for Any Problem
USER RESEARCH FOR EVERYONE, ROSENFELD MEDIA
OCTOBER 2016
When it comes to research, you have questions.
How do I convince
people research is
important?
How do I turn this
research into better
products?
How do I make sure
everybody understands
the research?
How can we do this
faster and cheaper?
How do I pick which
research to do?
How do I get more
participants?
When it comes to research, you have questions.
How do I convince
people research is
important?
How do I turn this
research into better
products?
How do I make sure
everybody understands
the research?
How can we do this
faster and cheaper?
How do I pick which
research to do?
How do I get more
participants?
Evidence to support
decisions to make changes.
WHAT DO WE MEAN BY RESEARCH?
CORPORATE
STRATEGY
MARKET
RESEARCH
CUSTOMER
EXPERIENCE
RESEARCH
We often treat research as a departmental concern.
USER
RESEARCH
CORPORATE
STRATEGY
MARKET
RESEARCH
CUSTOMER
EXPERIENCE
RESEARCH
We often treat research as a departmental concern.
USER
RESEARCH
Instead, we can think of research as a series of
questions that build on each other.
Instead, we can think of research as a series of
questions that build on each other.
1.
What is
happening
around us?
Instead, we can think of research as a series of
questions that build on each other.
1.
What is
happening
around us?
2.
What do
people need?
Instead, we can think of research as a series of
questions that build on each other.
1.
What is
happening
around us?
2.
What do
people need?
3.
What can we
make that will
help?
Instead, we can think of research as a series of
questions that build on each other.
1.
What is
happening
around us?
2.
What do
people need?
3.
What can we
make that will
help?
4.
Does our
solution really
work?
Instead, we can think of research as a series of
questions that build on each other.
1.
What is
happening
around us?
5.
What is
the impact?
2.
What do
people need?
3.
What can we
make that will
help?
4.
Does our
solution really
work?
1.
What is happening around us?
Intuit conducted a series of studies to understand how
its customers’ world will change in the next 10 years.
Source: about.intuit.com/futureofsmallbusiness/
They identified trends that will shape customers’ needs and
experiences…
Source: about.intuit.com/futureofsmallbusiness/
… And highlighted implications for how products and
services will need to evolve to support people’s needs.
The goal here is to understand significant shifts that
will soon impact your organization—or may already.
Futures Research
Macrotrend Analysis
Demographic Research
Mining Customer Data
Comparative Benchmarking
Signals and Trend Spotting
RESEARCH ACTIVITIES TOOLS & PROVIDERS
$$$$
$
APPROACH
Analytical
Empirical
Periodic Continuous
2.
What do people need?
Philips shadows patients and healthcare professionals
to understand how they do their jobs today.
Source: alicehyde.com
Using observations from field research, the Philips team
identifies key issues, which get captured as ‘issue cards.’
Source: Philips Design
Then the team maps out a framework to understand problems,
paint points, emotional needs, and mindset of users as they
move through their journey.
Source: Philips Design
From there, the team begins to explore potential concepts and
solutions.
Source: Philips Design
Potential solutions are reviewed, refined, and vetted with
stakeholders and team members in an opportunity workshop.
Source: Philips Design
The goal here is to understand people’s needs,
behaviors, mindsets, and motivations.
Ethnographic Research
Market Research
Research Communities
Diary Studies
Extreme User Research
Participatory Design
Customer Interviews
Customer Journey Mapping
Bodystorming
RESEARCH ACTIVITIES TOOLS & PROVIDERS
$$$$
$
APPROACH
Analytical
Empirical
Periodic Continuous
3.
What can we make that will help?
Source: Medium, ‘Designing Facebook Collage’
For its Collage feature, Facebook first observed user
behaviors that suggested a need for a new feature.
Source: TheNextWeb.com
Next, Facebook prototyped successive explorations,
moving eventually into an MVP.
Source: Medium, ‘Designing Facebook Collage’
As a small initial beta, Facebook ‘dogfoods’ apps
with its own employees.
Source: Medium, ‘Designing Facebook Collage’
Finally, they tested in a regional beta and then
released it to all of their customers.
The goal here is to know the range of possible
solutions and which direction is most successful.
Concept Testing
A/B testing
Card Sorting & Tree Testing
Lean Experiments
Landing Page Tests
5 Second Tests
RITE Testing
RESEARCH ACTIVITIES TOOLS & PROVIDERS
$$$$
$
APPROACH
Analytical
Empirical
Periodic Continuous
<html>
4.
Does our solution really work?
Source: “Data-Driven Design In the Real World,” SmashingMagazine.com
An internal team at Volusion was tasked with improving
its registration conversion rate.
Source: “Data-Driven Design In the Real World,” SmashingMagazine.com
After finding that an alternate design made no difference
in conversion, they dug into their analytics data.
Source: “Data-Driven Design In the Real World,” SmashingMagazine.com
Based on this research, they developed a new trial page
and saw a 48% increase in trial registrations and a 57%
increase in trial-to-store conversions.
The goal here is to understand if people can
successfully use your products as you intend.
Beta Testing
Mining Analytics
Usability testing
Heuristic reviews
Dogfooding
RITE Testing
RESEARCH ACTIVITIES TOOLS & PROVIDERS
$$$$
$
APPROACH
Analytical
Empirical
Periodic Continuous
5.
What is the impact?
Google’s HEART framework:
Happiness
Source: Google Ventures
Google’s HEART framework:
Happiness
Engagement
Source: Google Ventures
Google’s HEART framework:
Happiness
Engagement
Adoption
Source: Google Ventures
Google’s HEART framework:
Happiness
Engagement
Adoption
Retention
Source: Google Ventures
Google’s HEART framework:
Happiness
Engagement
Adoption
Retention
Task Success
Source: Google Ventures
Google’s HEART framework:
Happiness
Engagement
Adoption
Retention
Task Success
Goals Signals Metrics
Source: Google Ventures
Google’s HEART framework:
Happiness
Engagement
Adoption
Retention
Task Success
Goals Signals Metrics
Source: Google Ventures
Usabilla applied this framework to measure the success of
new two-step verification feature.
Source: Usabilla
Source: Usabilla
The goal here is to know how your product impacts
your customers (and by extension your organization).
VoC Programs
Behavioral Analytics
Text Analytics
Sentiment & Loyalty Analysis
Social Listening
Channel Surveys
RESEARCH ACTIVITIES TOOLS & PROVIDERS
$$$$
$
APPROACH
Analytical
Empirical
Periodic Continuous
Where do you go from here?
LANGUAGE YOU MIGHT HEAR
What should we do with [mobile/social, etc.]?
What should we prioritize first?
We have some ideas about what we need to do. How do we know we’re right?
You think customers will do X. I think they’ll do Y. Who’s right?
What’s the right [language, imagery, interaction method, etc.] to use here?
Did our changes make a difference?
Why didn’t our changes make a difference?
Why isn’t this [feature, product, etc.] getting as much traction as we thought?
What is the impact of [design, UX, research, etc.] on our business?
LANGUAGE YOU MIGHT HEARFOCUS HERE
What should we do with [mobile/social, etc.]?
What should we prioritize first?
We have some ideas about what we need to do. How do we know we’re right?
You think customers will do X. I think they’ll do Y. Who’s right?
What’s the right [language, imagery, interaction method, etc.] to use here?
Did our changes make a difference?
Why didn’t our changes make a difference?
Why isn’t this [feature, product, etc.] getting as much traction as we thought?
What is the impact of [design, UX, research, etc.] on our business?
LANGUAGE YOU MIGHT HEARFOCUS HERE
What should we do with [mobile/social, etc.]?
What should we prioritize first?
We have some ideas about what we need to do. How do we know we’re right?
You think customers will do X. I think they’ll do Y. Who’s right?
What’s the right [language, imagery, interaction method, etc.] to use here?
Did our changes make a difference?
Why didn’t our changes make a difference?
Why isn’t this [feature, product, etc.] getting as much traction as we thought?
What is the impact of [design, UX, research, etc.] on our business?
LANGUAGE YOU MIGHT HEARFOCUS HERE
What should we do with [mobile/social, etc.]?
What should we prioritize first?
We have some ideas about what we need to do. How do we know we’re right?
You think customers will do X. I think they’ll do Y. Who’s right?
What’s the right [language, imagery, interaction method, etc.] to use here?
Did our changes make a difference?
Why didn’t our changes make a difference?
Why isn’t this [feature, product, etc.] getting as much traction as we thought?
What is the impact of [design, UX, research, etc.] on our business?
LANGUAGE YOU MIGHT HEARFOCUS HERE
What should we do with [mobile/social, etc.]?
What should we prioritize first?
We have some ideas about what we need to do. How do we know we’re right?
You think customers will do X. I think they’ll do Y. Who’s right?
What’s the right [language, imagery, interaction method, etc.] to use here?
Did our changes make a difference?
Why didn’t our changes make a difference?
Why isn’t this [feature, product, etc.] getting as much traction as we thought?
What is the impact of [design, UX, research, etc.] on our business?
LANGUAGE YOU MIGHT HEARFOCUS HERE
What should we do with [mobile/social, etc.]?
What should we prioritize first?
We have some ideas about what we need to do. How do we know we’re right?
You think customers will do X. I think they’ll do Y. Who’s right?
What’s the right [language, imagery, interaction method, etc.] to use here?
Did our changes make a difference?
Why didn’t our changes make a difference?
Why isn’t this [feature, product, etc.] getting as much traction as we thought?
What is the impact of [design, UX, research, etc.] on our business?
LANGUAGE YOU MIGHT HEARFOCUS HERE
What should we do with [mobile/social, etc.]?
What should we prioritize first?
We have some ideas about what we need to do. How do we know we’re right?
You think customers will do X. I think they’ll do Y. Who’s right?
What’s the right [language, imagery, interaction method, etc.] to use here?
Did our changes make a difference?
Why didn’t our changes make a difference?
Why isn’t this [feature, product, etc.] getting as much traction as we thought?
What is the impact of [design, UX, research, etc.] on our business?
Web: leahbuley.com
Email: leah@leahbuley.com
Twitter: @leahbuley
DOWNLOAD AT
BULEY.CO/RESEARCH-METHODS-FOR-ANY-PROBLEM

The Right Research Method For Any Problem (And Budget)

  • 1.
    The Right Research Methodfor Any Problem USER RESEARCH FOR EVERYONE, ROSENFELD MEDIA OCTOBER 2016
  • 2.
    When it comesto research, you have questions. How do I convince people research is important? How do I turn this research into better products? How do I make sure everybody understands the research? How can we do this faster and cheaper? How do I pick which research to do? How do I get more participants?
  • 3.
    When it comesto research, you have questions. How do I convince people research is important? How do I turn this research into better products? How do I make sure everybody understands the research? How can we do this faster and cheaper? How do I pick which research to do? How do I get more participants?
  • 4.
    Evidence to support decisionsto make changes. WHAT DO WE MEAN BY RESEARCH?
  • 5.
  • 6.
  • 7.
    Instead, we canthink of research as a series of questions that build on each other.
  • 8.
    Instead, we canthink of research as a series of questions that build on each other. 1. What is happening around us?
  • 9.
    Instead, we canthink of research as a series of questions that build on each other. 1. What is happening around us? 2. What do people need?
  • 10.
    Instead, we canthink of research as a series of questions that build on each other. 1. What is happening around us? 2. What do people need? 3. What can we make that will help?
  • 11.
    Instead, we canthink of research as a series of questions that build on each other. 1. What is happening around us? 2. What do people need? 3. What can we make that will help? 4. Does our solution really work?
  • 12.
    Instead, we canthink of research as a series of questions that build on each other. 1. What is happening around us? 5. What is the impact? 2. What do people need? 3. What can we make that will help? 4. Does our solution really work?
  • 13.
  • 14.
    Intuit conducted aseries of studies to understand how its customers’ world will change in the next 10 years.
  • 15.
    Source: about.intuit.com/futureofsmallbusiness/ They identifiedtrends that will shape customers’ needs and experiences…
  • 16.
    Source: about.intuit.com/futureofsmallbusiness/ … Andhighlighted implications for how products and services will need to evolve to support people’s needs.
  • 17.
    The goal hereis to understand significant shifts that will soon impact your organization—or may already. Futures Research Macrotrend Analysis Demographic Research Mining Customer Data Comparative Benchmarking Signals and Trend Spotting RESEARCH ACTIVITIES TOOLS & PROVIDERS $$$$ $ APPROACH Analytical Empirical Periodic Continuous
  • 18.
  • 19.
    Philips shadows patientsand healthcare professionals to understand how they do their jobs today. Source: alicehyde.com
  • 20.
    Using observations fromfield research, the Philips team identifies key issues, which get captured as ‘issue cards.’ Source: Philips Design
  • 21.
    Then the teammaps out a framework to understand problems, paint points, emotional needs, and mindset of users as they move through their journey. Source: Philips Design
  • 22.
    From there, theteam begins to explore potential concepts and solutions. Source: Philips Design
  • 23.
    Potential solutions arereviewed, refined, and vetted with stakeholders and team members in an opportunity workshop. Source: Philips Design
  • 24.
    The goal hereis to understand people’s needs, behaviors, mindsets, and motivations. Ethnographic Research Market Research Research Communities Diary Studies Extreme User Research Participatory Design Customer Interviews Customer Journey Mapping Bodystorming RESEARCH ACTIVITIES TOOLS & PROVIDERS $$$$ $ APPROACH Analytical Empirical Periodic Continuous
  • 25.
    3. What can wemake that will help?
  • 26.
    Source: Medium, ‘DesigningFacebook Collage’ For its Collage feature, Facebook first observed user behaviors that suggested a need for a new feature.
  • 27.
    Source: TheNextWeb.com Next, Facebookprototyped successive explorations, moving eventually into an MVP.
  • 28.
    Source: Medium, ‘DesigningFacebook Collage’ As a small initial beta, Facebook ‘dogfoods’ apps with its own employees.
  • 29.
    Source: Medium, ‘DesigningFacebook Collage’ Finally, they tested in a regional beta and then released it to all of their customers.
  • 30.
    The goal hereis to know the range of possible solutions and which direction is most successful. Concept Testing A/B testing Card Sorting & Tree Testing Lean Experiments Landing Page Tests 5 Second Tests RITE Testing RESEARCH ACTIVITIES TOOLS & PROVIDERS $$$$ $ APPROACH Analytical Empirical Periodic Continuous <html>
  • 31.
  • 32.
    Source: “Data-Driven DesignIn the Real World,” SmashingMagazine.com An internal team at Volusion was tasked with improving its registration conversion rate.
  • 33.
    Source: “Data-Driven DesignIn the Real World,” SmashingMagazine.com After finding that an alternate design made no difference in conversion, they dug into their analytics data.
  • 34.
    Source: “Data-Driven DesignIn the Real World,” SmashingMagazine.com Based on this research, they developed a new trial page and saw a 48% increase in trial registrations and a 57% increase in trial-to-store conversions.
  • 35.
    The goal hereis to understand if people can successfully use your products as you intend. Beta Testing Mining Analytics Usability testing Heuristic reviews Dogfooding RITE Testing RESEARCH ACTIVITIES TOOLS & PROVIDERS $$$$ $ APPROACH Analytical Empirical Periodic Continuous
  • 36.
  • 37.
  • 38.
  • 39.
  • 40.
  • 41.
  • 42.
    Google’s HEART framework: Happiness Engagement Adoption Retention TaskSuccess Goals Signals Metrics Source: Google Ventures
  • 43.
    Google’s HEART framework: Happiness Engagement Adoption Retention TaskSuccess Goals Signals Metrics Source: Google Ventures
  • 44.
    Usabilla applied thisframework to measure the success of new two-step verification feature. Source: Usabilla
  • 45.
  • 46.
    The goal hereis to know how your product impacts your customers (and by extension your organization). VoC Programs Behavioral Analytics Text Analytics Sentiment & Loyalty Analysis Social Listening Channel Surveys RESEARCH ACTIVITIES TOOLS & PROVIDERS $$$$ $ APPROACH Analytical Empirical Periodic Continuous
  • 47.
    Where do yougo from here?
  • 48.
    LANGUAGE YOU MIGHTHEAR What should we do with [mobile/social, etc.]? What should we prioritize first? We have some ideas about what we need to do. How do we know we’re right? You think customers will do X. I think they’ll do Y. Who’s right? What’s the right [language, imagery, interaction method, etc.] to use here? Did our changes make a difference? Why didn’t our changes make a difference? Why isn’t this [feature, product, etc.] getting as much traction as we thought? What is the impact of [design, UX, research, etc.] on our business?
  • 49.
    LANGUAGE YOU MIGHTHEARFOCUS HERE What should we do with [mobile/social, etc.]? What should we prioritize first? We have some ideas about what we need to do. How do we know we’re right? You think customers will do X. I think they’ll do Y. Who’s right? What’s the right [language, imagery, interaction method, etc.] to use here? Did our changes make a difference? Why didn’t our changes make a difference? Why isn’t this [feature, product, etc.] getting as much traction as we thought? What is the impact of [design, UX, research, etc.] on our business?
  • 50.
    LANGUAGE YOU MIGHTHEARFOCUS HERE What should we do with [mobile/social, etc.]? What should we prioritize first? We have some ideas about what we need to do. How do we know we’re right? You think customers will do X. I think they’ll do Y. Who’s right? What’s the right [language, imagery, interaction method, etc.] to use here? Did our changes make a difference? Why didn’t our changes make a difference? Why isn’t this [feature, product, etc.] getting as much traction as we thought? What is the impact of [design, UX, research, etc.] on our business?
  • 51.
    LANGUAGE YOU MIGHTHEARFOCUS HERE What should we do with [mobile/social, etc.]? What should we prioritize first? We have some ideas about what we need to do. How do we know we’re right? You think customers will do X. I think they’ll do Y. Who’s right? What’s the right [language, imagery, interaction method, etc.] to use here? Did our changes make a difference? Why didn’t our changes make a difference? Why isn’t this [feature, product, etc.] getting as much traction as we thought? What is the impact of [design, UX, research, etc.] on our business?
  • 52.
    LANGUAGE YOU MIGHTHEARFOCUS HERE What should we do with [mobile/social, etc.]? What should we prioritize first? We have some ideas about what we need to do. How do we know we’re right? You think customers will do X. I think they’ll do Y. Who’s right? What’s the right [language, imagery, interaction method, etc.] to use here? Did our changes make a difference? Why didn’t our changes make a difference? Why isn’t this [feature, product, etc.] getting as much traction as we thought? What is the impact of [design, UX, research, etc.] on our business?
  • 53.
    LANGUAGE YOU MIGHTHEARFOCUS HERE What should we do with [mobile/social, etc.]? What should we prioritize first? We have some ideas about what we need to do. How do we know we’re right? You think customers will do X. I think they’ll do Y. Who’s right? What’s the right [language, imagery, interaction method, etc.] to use here? Did our changes make a difference? Why didn’t our changes make a difference? Why isn’t this [feature, product, etc.] getting as much traction as we thought? What is the impact of [design, UX, research, etc.] on our business?
  • 54.
    LANGUAGE YOU MIGHTHEARFOCUS HERE What should we do with [mobile/social, etc.]? What should we prioritize first? We have some ideas about what we need to do. How do we know we’re right? You think customers will do X. I think they’ll do Y. Who’s right? What’s the right [language, imagery, interaction method, etc.] to use here? Did our changes make a difference? Why didn’t our changes make a difference? Why isn’t this [feature, product, etc.] getting as much traction as we thought? What is the impact of [design, UX, research, etc.] on our business?
  • 55.
    Web: leahbuley.com Email: leah@leahbuley.com Twitter:@leahbuley DOWNLOAD AT BULEY.CO/RESEARCH-METHODS-FOR-ANY-PROBLEM